AI-based vision-based automatic PCB defect filtering and inspection system

Abstract

This invention relates to an AI vision-based automatic PCB defect filtering and inspection system, belonging to the field of image processing technology. It includes: an image acquisition module for acquiring an initial image of the PCB board in an initial pose, and performing displacement compensation on the initial pose based on multiple key alignment points and the initial image of the PCB board; after displacement compensation, acquiring the image of the PCB board; a region segmentation module for dividing the image of the PCB board into multiple sub-images corresponding to different regions according to an optimal partitioning strategy; a model building module for constructing a defect recognition model; and a defect detection module for extracting image features of the sub-images corresponding to each region, determining whether a defect exists based on the image features of the sub-images corresponding to the region, and if so, identifying the defect information of the sub-images corresponding to the region through the defect recognition model. This system has the advantage of improving the efficiency and accuracy of PCB defect inspection.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an AI vision-based automatic PCB defect filtering and inspection system. Background Technology

[0002] In the field of electronics manufacturing, PCBs (printed circuit boards) are the core basic components of electronic products, and their quality directly determines the overall performance and reliability of the device. As electronic products rapidly develop towards high density, miniaturization, and multi-layer composites, the spacing between PCB lines and the size of solder pads continue to shrink, placing extremely high demands on the accuracy and efficiency of quality inspection.

[0003] However, traditional inspection methods have significant limitations: manual visual inspection relies on the experience and fatigue of quality inspectors, is easily affected by subjective factors, and struggles to consistently identify micron-level defects (such as fine line gaps and tiny foreign objects). Furthermore, the inspection speed cannot meet the demands of large-scale production, resulting in high labor costs and frequent missed or false detections. While rule-based machine vision inspection can achieve partial automation, it requires designing complex rules for different defect types. Its ability to identify defects with complex backgrounds and irregular shapes (such as rodent bites and burrs) is limited, and rule configuration is cumbersome, making it difficult to quickly adapt to the inspection needs of new products and processes. In addition, traditional methods lack the ability to deeply mine and analyze inspection data, failing to provide data-driven decision-making support for process optimization. This leads to the recurrence of similar defects, hindering improvements in production efficiency and yield.

[0004] Therefore, there is a need to provide an AI vision-based automatic PCB defect filtering and inspection system to improve the efficiency and accuracy of PCB defect inspection. Summary of the Invention

[0005] This invention provides an AI vision-based automatic PCB defect filtering and inspection system, comprising: an image acquisition module for acquiring an initial image of the PCB board in an initial pose, and performing displacement compensation on the initial pose based on multiple key alignment points and the initial image of the PCB board, and acquiring an image of the PCB board after displacement compensation; a region segmentation module for dividing the image of the PCB board into multiple sub-images corresponding to different regions according to an optimal partitioning strategy of the PCB board; a model building module for constructing a defect recognition model; and a defect detection module for extracting image features of the sub-images corresponding to each region, determining whether a defect exists based on the image features of the sub-images corresponding to the region, and if so, identifying the defect information of the sub-images corresponding to the region through the defect recognition model.

[0006] Furthermore, multiple key alignment points are identified, including: acquiring initial images of multiple test PCBs in the initial pose; identifying multiple alignment points; for each alignment point, determining the interference index of the alignment point, and based on the initial images of multiple test PCBs, determining the offset index of the alignment point; filtering multiple alignment points according to the interference index and offset index of each alignment point to identify multiple candidate alignment points; and identifying multiple key alignment points based on the initial images of multiple test PCBs and the multiple candidate alignment points.

[0007] Furthermore, based on initial images of multiple test PCBs and multiple candidate alignment points, multiple key alignment points are determined, including: determining multiple alignment point combinations as an initial population based on the interference and offset indices of each candidate alignment point and the coordinate distance between any two candidate alignment points, wherein each alignment point combination includes at least three candidate alignment points; for each alignment point combination and each test PCB, determining the translational deviation and rotation angle corresponding to the alignment point combination based on the initial image of the test PCB, and constructing a fitness function based on the translational deviation and rotation angle of the test PCB corresponding to the alignment point combination; wherein the fitness function is related to the compensation effectiveness index corresponding to the alignment point combination, and the compensation effectiveness index corresponding to the alignment point combination is determined based on the translational deviation and rotation angle of each test PCB corresponding to the alignment point combination; and iteratively optimizing the population using a genetic algorithm to determine multiple key alignment points.

[0008] Furthermore, based on the interference and offset indices of each candidate pair site and the coordinate distance between any two candidate pair sites, multiple pair site combinations are determined as the initial population, including: S11, constructing a range of potential pair sites; S12, for any two candidate pair sites, calculating the combination probability of the two candidate pair sites based on the interference and offset indices of each candidate pair site and the coordinate distance between the two candidate pair sites; S13, for each pair site combination, randomly generating the number of pair sites in the pair site combination from the range of potential pair sites, and generating the pair site combination based on the number of pair sites in the pair site combination and the combination probability of any two candidate pair sites; S14, deduplicating the generated pair site combinations; S15, determining whether the number of deduplicated pair site combinations meets the preset requirements. If not, output the initial population; otherwise, execute S13.

[0009] Furthermore, based on the number of pairs in a pair combination and the combination probability of any two candidate pairs, a pair combination is generated, including: determining the adjacent candidate pairs of each candidate pair based on the combination probability of any two candidate pairs; generating a weighted undirected graph based on the adjacent candidate pairs of each candidate pair, wherein the edges in the weighted undirected graph are related to the combination probability of the two candidate pairs connected; for each candidate pair, generating a global combination probability of the candidate pair based on the combination probability of any two candidate pairs; sorting all candidate pairs based on the global combination probability of each candidate pair, generating a sorting result; extracting candidate pairs based on the sorting result; and generating a pair combination based on the weighted undirected graph, the extracted candidate pairs, and the number of pairs in a pair combination.

[0010] Furthermore, based on the translational deviation and rotation angle of each test PCB board corresponding to the point combination, the compensation effectiveness index corresponding to the point combination is determined, including: for each test PCB board, displacement compensation is performed based on the translational deviation and rotation angle of each test PCB board corresponding to the point combination; after displacement compensation, the compensated image of the test PCB board is acquired; based on the initial image, the compensated image, and multiple candidate alignment points of the test PCB board, the effective compensation value of the point combination corresponding to the test PCB board is calculated; based on the effective compensation value of the point combination corresponding to each test PCB board, the compensation effectiveness index corresponding to the point combination is determined.

[0011] Furthermore, determining the optimal partitioning strategy for the PCB board includes: for each pixel and each test PCB board, calculating the RGB difference between the pixel and the test PCB board based on the compensated image of the test PCB board and the standard image; for each pixel, calculating the validity index corresponding to the pixel based on the RGB difference between the pixel and each test PCB board; and determining the optimal partitioning strategy for the PCB board based on the validity index corresponding to each pixel.

[0012] Furthermore, based on the validity index corresponding to each pixel, the optimal partitioning strategy for the PCB board is determined, including: filtering multiple valid pixels based on the validity index corresponding to each pixel; for any two valid pixels, calculating the correlation of the RGB difference between the two valid pixels based on the RGB difference of each test PCB board corresponding to the two valid pixels; and determining the optimal partitioning strategy for the PCB board based on the correlation of the RGB difference between the two valid pixels and the coordinate distance.

[0013] Furthermore, the image features of the sub-images corresponding to the region are extracted, and the presence of defects is determined based on the image features of the sub-images corresponding to the region. This includes: extracting the image features of the sub-images corresponding to the region, wherein the image features of the sub-images include at least the RGB values ​​of each valid pixel; calculating the global RGB difference based on the RGB values ​​of each valid pixel in the sub-image and the RGB values ​​of each valid pixel in the standard image; and determining the presence of defects based on the global RGB difference.

[0014] Furthermore, the defect recognition model includes convolutional layers, pooling layers, a feature pyramid network, a global average pooling layer, and a Softmax classifier. The convolutional layers use learnable convolutional kernels to slide and extract local features of sub-images. The pooling layers are used to downsample the feature maps generated by the convolutional layers. The feature pyramid network is used to generate multi-scale feature maps. The Softmax classifier is used to output the defect category and confidence level corresponding to the region.

[0015] Compared to existing technologies, the AI ​​vision-based automatic PCB defect filtering and inspection system provided in this specification has at least the following advantages: 1. By screening multiple key alignment points and optimizing with a genetic algorithm, combined with dynamic compensation for translational deviation and rotation angle to correct the initial pose, interference such as deformation and jitter during PCB board transport is effectively eliminated. Based on a combined probability model of interference index, offset index, and coordinate distance, the key alignment points are ensured to have both noise resistance and reasonable spatial distribution, significantly reducing the misjudgment rate caused by displacement deviation. 2. By analyzing the correlation between pixel validity index and RGB difference, the PCB board can be adaptively partitioned to achieve color change synchronicity and spatial proximity of pixels in each sub-region. 3. By constructing a range of candidate pairings and dynamically calculating their combination probabilities based on interference, offset, and coordinate distance, we ensure that the generated initial population possesses both diversity and rationality. On the one hand, randomly generating the number of pairings and sampling based on combination probabilities avoids the limitations of fixed combination patterns and enhances the population's ability to cover different spatial distributions and feature combinations. On the other hand, through deduplication and iterative supplementation mechanisms, we ensure that the population size meets the optimization requirements of the genetic algorithm while eliminating redundant combinations to improve computational efficiency. Furthermore, the combination probability incorporates the triple constraints of interference, offset, and coordinate distance, making the initial population more likely to include pairings with high stability and good compensation effects, laying the foundation for the subsequent genetic algorithm to quickly converge to the global optimum. Attached Figure Description

[0016] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein: Figure 1 This is a block diagram of an AI vision-based automatic PCB defect filtering and inspection system shown in one embodiment of this application; Figure 2 This is a flowchart illustrating the determination of multiple key alignment sites in one embodiment of this application; Figure 3 This is a flowchart illustrating the determination of multiple pairs of sites in one embodiment of this application; Figure 4 This is a flowchart illustrating the generation of a pair of sites in one embodiment of this application. Detailed Implementation

[0017] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0018] Figure 1 This is a block diagram of an AI vision-based automatic PCB defect filtering and inspection system shown in one embodiment of this application, as follows: Figure 1 As shown, the AI ​​vision-based PCB defect automatic filtering and inspection system can include an image acquisition module, a region segmentation module, a model building module, and a defect detection module.

[0019] The image acquisition module is used to acquire an initial image of the PCB board in an initial pose, and to perform displacement compensation on the initial pose based on multiple key alignment points and the initial image of the PCB board. After displacement compensation, the image of the PCB board is acquired.

[0020] Specifically, multiple key alignment points are pre-designed markers on the PCB board with high contrast and unique positional characteristics, such as circular, square, or cross-shaped metallized patterns (e.g., Mark points, positioning holes, or special symbols on process edges). These points have precise coordinate information in the PCB design file. The image acquisition module identifies alignment points in the initial image, calculates the deviation between their actual and designed positions, and then generates displacement compensation parameters to perform sub-pixel adjustments to the camera or stage, ensuring that subsequently acquired images are perfectly aligned with the PCB design and eliminating image offset caused by mechanical clamping errors or vibrations.

[0021] Figure 2 This is a flowchart illustrating the determination of multiple key alignment sites in one embodiment of this application, as shown below. Figure 2 As shown, several key alignment sites were identified, including: In the initial pose, acquire initial images of multiple test PCB boards; Identify multiple alignment sites; For each pair of sites, the interference index of the pair of sites is determined, and the offset index of the pair of sites is determined based on the initial images of multiple test PCBs. Based on the interference and offset indices of each pair site, multiple pair sites are screened to determine multiple candidate pair sites. Based on initial images of multiple test PCBs and multiple candidate alignment sites, several key alignment sites were identified.

[0022] Specifically, the initial pose refers to the preset position and angle state of the image acquisition device (such as an industrial camera) of the image acquisition module when it first takes a picture of the PCB board.

[0023] Multiple alignment points can be set in areas where no elements are set.

[0024] The interference index is determined based on the component and circuit distribution density in the local area surrounding the target point. Specifically, a circular analysis area with a fixed radius (e.g., 2-5 mm) is delineated centered on each target point. Image processing algorithms (e.g., edge detection, connected component analysis) are used to statistically analyze the number of components (e.g., surface mount devices such as resistors and capacitors), the density of circuits (e.g., copper foil traces), and the presence of interfering factors such as reflective metal surfaces and silkscreened text within this area. If the area has a large number of components, dense circuits, or strong reflective materials, light reflection and occlusion effects will significantly reduce the recognition accuracy of the target point in the image, resulting in a higher interference index value (e.g., close to 1 in the 0-1 range). Conversely, if the area is open and has clear contrast, a lower interference index value is assigned. This index quantitatively reflects the degree of interference from environmental factors on target point detection.

[0025] For each alignment point, the determination of the offset index relies on the coordinate statistical analysis of multiple test images. By acquiring initial images of multiple test PCB boards in their initial poses, the image coordinates (e.g., pixel positions) of the alignment points in each initial image are extracted. The deviation between these coordinates and the image coordinates of the alignment points in the standard image is calculated, and the average deviation of the alignment points across all test boards is used as the offset index value for the alignment points. The standard image refers to an image acquired from a test PCB board that has not undergone any offset.

[0026] The ranking value of a site can be calculated based on the interference and offset indices of the site using the following formula: ;

[0027] in, This represents the ordinal value of the locus. and As weight, and Greater than 0, ,For example, , , As an indicator of interference at the site, This is an index of offset at the locus.

[0028] Pairs can be sorted according to their ranking values, and a certain percentage (e.g., the top 50%) of pairs can be selected as candidate pairs.

[0029] First, alignment points were established in areas without components. Interference indices were calculated based on factors such as local component density, line density, and reflective interference, effectively mitigating identification errors caused by high-density components or highly reflective materials, ensuring the detectability of alignment points in complex environments. Second, offset indices were calculated through coordinate statistical analysis of multiple test images, quantifying the positional repeatability of alignment points and selecting points with high mechanical stability to reduce positioning deviations caused by clamping errors or vibrations. Furthermore, a weighted formula was used to calculate a ranking value combining interference and offset indices, achieving objective selection of alignment points through a quantitative scoring mechanism, avoiding the subjectivity of manual selection, and ensuring that key alignment points possess both low interference and high stability. Finally, the top-ranked alignment points were selected as candidates, optimizing the number and distribution of positioning references. In some embodiments, based on initial images of multiple test PCBs and multiple candidate alignment sites, multiple key alignment sites are determined, including: Based on the interference and offset indices of each candidate pair site and the coordinate distance between any two candidate pair sites, multiple pair site combinations are determined as the initial population, wherein each pair site combination includes at least three candidate pair sites. For each point combination and each test PCB board, based on the initial image of the test PCB board, determine the translational deviation and rotation angle corresponding to the point combination, and then determine the translational deviation and rotation angle of the test PCB board corresponding to the point combination. Construct a fitness function, which is related to the compensation effectiveness index corresponding to the point combination. The compensation effectiveness index corresponding to the point combination is determined based on the translation deviation and rotation angle of each test PCB board corresponding to the point combination. The larger the compensation effectiveness index corresponding to the point combination, the larger the fitness function. The population was iteratively optimized using a genetic algorithm to identify multiple key alignment sites.

[0030] Figure 3 This is a flowchart illustrating the determination of multiple pairing site combinations in one embodiment of this application, such as... Figure 3 As shown, based on the interference and offset indices of each candidate pair site and the coordinate distance between any two candidate pair sites, multiple pair site combinations are determined as the initial population, including: S11. Construct a range of quantities, for example, [5, 10], [3, 8], etc.; S12. For any two candidate pair sites, calculate the combination probability of the two candidate pair sites based on the interference index and offset index of each candidate pair site and the coordinate distance between the two candidate pair sites. S13. For each pair of pairs, randomly generate the number of pairs of pairs from the range of numbers, and generate pairs of pairs based on the number of pairs of pairs of pairs and the combination probability of any two candidate pairs of pairs. S14. Remove duplicates from the generated pairwise combinations; S15. Determine whether the number of duplicated pair combinations meets the preset requirement (e.g., greater than 20). If not, output the initialized population. If not, execute S13.

[0031] Specifically, the ranking value of the candidate pair sites is calculated based on the interference and offset indices of the candidate pair sites. The ranking values ​​of the two candidate pair sites are then averaged to obtain the ranking mean.

[0032] The sorted mean and coordinate distance of the two candidate sites are normalized, and the normalized sorted mean and coordinate distance are both in the range of [0, 1].

[0033] The combined probability of two candidate sites is calculated using the following formula: ; in, Let be the combination probability of the i-th candidate pair site and the j-th candidate pair site. This represents the normalized sorted mean of the i-th candidate pair locus and the j-th candidate pair locus. This represents the normalized coordinate distance between the i-th candidate site and the j-th candidate site.

[0034] The above formula takes the normalized sorting mean and coordinate distance as inputs, calculates the combination probability through the arithmetic mean, and comprehensively reflects their respective interference and offset indices. The higher the sorting value, the less the location is affected by component occlusion, line interference or mechanical offset, and the more reliable the quality. The normalized distance between the two location points in the image coordinate system measures their spatial dispersion. The greater the distance, the wider the area of ​​the board surface covered by the combination, which can improve the adaptability of the positioning system to different local defects.

[0035] Figure 4 This is a flowchart illustrating the generation of pairwise site combinations in one embodiment of this application, as shown below. Figure 4As shown, in some embodiments, a pair of sites is generated based on the number of pairs in the pair and the combination probability of any two candidate pairs, including: Based on the combination probability of any two candidate pair sites, determine the adjacent candidate pair sites of each candidate pair site. For example, if the combination probability of two candidate pair sites is greater than 50%, then the two candidate pair sites are considered to be adjacent candidate pair sites. A weighted undirected graph is generated based on the adjacent candidate pairs of each candidate pair. In this graph, each edge is related to the combination probability of the two candidate pairs they connect. Specifically, the weighted undirected graph is a graph structure with candidate pairs as vertices and adjacency relationships as edges, where each edge carries a weight reflecting the reliability of the vertex pair combination. Specifically, each vertex in the graph represents a candidate pair. If the combination probability of two vertices exceeds a preset threshold (e.g., 50%), an undirected edge is constructed between these vertices, and the combination probability is assigned to this edge as its weight. For example, if the combination probability of vertex A and vertex B is 0.7, they are adjacent vertices, and the edge weight is 0.7; if the combination probability of vertex A and vertex C is 0.3, no edge is established. In this way, the weighted undirected graph transforms discrete candidate pairs into an interconnected network. For each candidate pair, the global combination probability of the candidate pair is generated based on the combination probability of any two candidate pair pairs. For example, the average of the combination probabilities of the candidate pair with any other candidate pair is used as the global combination probability of the candidate pair. Based on the global combination probability of each candidate pair site, all candidate pair sites are sorted to generate a sorting result. For example, all candidate pair sites are sorted from largest to smallest according to the global combination probability to generate a sorting result. Based on the sorting results, candidate pair sites are extracted. For example, the candidate pair sites that are ranked first and have not been extracted are selected. Based on the weighted undirected graph, the extracted candidate pair sites, and the number of pair sites in the pair site combination, a pair site combination is generated.

[0036] Specifically, the process of generating pairing point combinations is based on the adjacency relationships of a weighted undirected graph. It achieves efficient point selection by progressively expanding the neighborhood of candidate points and combining weight optimization. Specifically, starting with the extracted candidate pairing points, the first-order adjacent nodes (i.e., directly connected vertices) are checked first, and the total number of first-order adjacent nodes of the current candidate point is counted. If this total number is greater than the number of pairing points in a pairing point combination, nodes with higher weights are directly selected from the first-order adjacent nodes based on edge weights, ensuring that the points in the combination are both reasonably distributed and of reliable quality. If the total number is insufficient, the process continues to expand to second-order adjacent nodes (i.e., vertices indirectly connected through first-order nodes), and the counting and selection process is repeated until the total number of nodes reaches the number of pairing points in a pairing point combination or all adjacency orders are traversed. During this process, the weight value serves as a key selection criterion, prioritizing the retention of nodes corresponding to edges with high connection strength (high combination probability), thereby ensuring that the pairing points in the combination spatially distribute and cover the key areas of the PCB, while possessing low interference and low offset characteristics in terms of quality. For example, if three pairing points need to be combined, and the only first-order adjacent node of the extracted candidate pairing point A is B (out of a total of 2), then the second-order adjacent node C of A is further included. If the edge weights of AB, AC, and BC are 0.8, 0.6, and 0.7 respectively, then A, B, and C with higher weights are preferentially selected as the final combination. This strategy balances the coverage and reliability of the combination through dynamic expansion of the adjacency order and gradient selection of weights, making it particularly suitable for high-density or complex PCB positioning scenarios.

[0037] When deduplicating the generated pairwise pairwise combinations, all combinations must be traversed. If two combinations contain the exact same set of pairwise pairs, only one is retained. For example, combinations ABC and BAC are considered duplicates because they contain the same pairwise pairs. This step avoids redundant results caused by adjacency expansion or multipath search, ensuring that the final output pairwise pairwise combinations are unique and meet the requirements for positioning accuracy and robustness.

[0038] The process of determining the translational deviation and rotation angle corresponding to the point combination based on the initial image of the test PCB board essentially quantifies the actual offset of the positioning point group by comparing the geometric transformation relationship between the initial image and the reference template (or standard coordinate system). Specifically, firstly, the coordinates of the generated alignment point combination are extracted from the initial image and matched with the ideal coordinates of the corresponding points in the reference template; then, the optimal transformation matrix between the two sets of points is calculated using the least squares method or a geometric fitting algorithm (such as RANSAC), which can be decomposed into translational deviation and rotation angle. The translational deviation represents the lateral and longitudinal displacement of the PCB in the image plane, while the rotation angle reflects the rotational offset of the PCB around its central axis.

[0039] In some embodiments, based on the translational deviation and rotation angle of each test PCB board corresponding to the point combination, the compensation effectiveness index corresponding to the point combination is determined, including: For each test PCB, displacement compensation is performed based on the translational deviation and rotation angle of each test PCB corresponding to the point combination. After displacement compensation, the compensated image of the test PCB is acquired. Based on the initial image of the test PCB, the compensated image, and multiple candidate alignment points, the effective compensation value of the test PCB corresponding to the point combination is calculated. Based on the effective compensation value of each test PCB board corresponding to the point combination, the compensation effectiveness index corresponding to the point combination is determined.

[0040] Specifically, for each test PCB board, the image acquisition device or robotic arm is first compensated in reverse (e.g., by adjusting the camera position or the robotic arm's gripping posture) based on the translational deviation and rotation angle corresponding to its point combination, so that the PCB board returns to its ideal position in the theoretical coordinate system. Then, the compensated PCB image is acquired and compared with the initial image. At this point, using multiple candidate alignment points included in the point combination, the actual coordinates of the corresponding alignment points in the compensated image are calculated through feature matching algorithms (such as SIFT, SURF, or deep learning matching models), thus obtaining the compensated alignment point coordinate residual (i.e., the deviation between the compensated coordinates and the ideal coordinates). The larger the compensated alignment point coordinate residual, the smaller the effective compensation value for the test PCB board corresponding to the point combination.

[0041] The average of the effective compensation values ​​for each test PCB board corresponding to the point combination is used to obtain the compensation effectiveness index corresponding to the point combination.

[0042] The process of iteratively optimizing a population using a genetic algorithm to determine key pairing sites essentially simulates the natural selection mechanism, searching for the optimal combination of solutions in the solution space. Specifically, firstly, combinations of candidate pairing sites are encoded as chromosomes, each chromosome representing a potential combination scheme, forming the initial population. Then, the fitness function is used to evaluate the quality of each chromosome; a higher compensation effectiveness index corresponds to a higher fitness value, indicating that the combination is closer to the optimal solution. During iteration, selection operations (such as roulette wheel selection or tournament selection) preserve high-fitness chromosomes to ensure the transmission of high-quality genes; crossover operations randomly exchange parts of the encoding of two chromosomes (such as single-point crossover or uniform crossover) to generate new combinations and expand the search range; mutation operations randomly flip encoding bits with low probability (such as binary mutation or Gaussian mutation) to avoid getting trapped in local optima. After each generation, the fitness of the new population is recalculated, and the selection, crossover, and mutation operations are repeated until the termination condition is met (such as reaching the maximum number of iterations or fitness convergence). Finally, the combination of sites corresponding to the chromosome with the highest fitness at algorithm convergence is the globally optimal set of key pairing sites.

[0043] In the PCB board positioning and inspection process, acquiring an initial image and performing displacement compensation based on key alignment points are crucial steps to ensure high-precision operation. Specifically, an initial image of the PCB board is first acquired using a high-resolution industrial camera or vision sensor, serving as the reference for subsequent positioning and compensation. Subsequently, the system extracts multiple pre-marked or automatically identified key alignment points (such as high-contrast pads, special markers, or feature component edges) from the initial image. The coordinates of these points have known ideal positions in the theoretical coordinate system. By comparing the actual extracted alignment point coordinates with the ideal coordinates, the current translational deviation and rotation angle of the PCB board are calculated. This is then solved using the least squares method or geometric transformation matrix to quantify the offset of the PCB board relative to the ideal pose.

[0044] Based on the calculated deviation parameters, the system performs reverse compensation on the image acquisition device (e.g., adjusting the camera position) or the robotic arm (e.g., correcting the gripping posture) to bring the PCB board back to its ideal position in the theoretical coordinate system. For example, if the PCB board is detected to be offset 2mm to the right and rotated 0.5° clockwise, the system controls the camera to move 2mm to the left and rotate 0.5° counterclockwise, or adjusts the gripping trajectory of the robotic arm to offset these deviations. After compensation, the system re-acquires an image of the PCB board; this time, the coordinates of the alignment points in the image should be closer to the ideal value, thus significantly reducing the false positive rate in subsequent detections.

[0045] The region segmentation module is used to divide the PCB board image into multiple sub-images corresponding to different regions based on the optimal partitioning strategy of the PCB board.

[0046] In some embodiments, determining the optimal partitioning strategy for the PCB board includes: For each pixel and each test PCB board, calculate the RGB difference between the pixel and the test PCB board based on the compensated image of the test PCB board and the standard image. For each pixel, the validity index corresponding to the pixel is calculated based on the RGB difference of each test PCB board. The optimal partitioning strategy for the PCB board is determined based on the validity index corresponding to each pixel.

[0047] Specifically, for each pixel, it is necessary to first traverse all test PCBs and calculate the RGB three-channel difference between the compensated image and the standard image at the corresponding position. These differences reflect the degree of deviation of the pixel from the ideal state after actual compensation.

[0048] For each pixel and each test PCB, if the RGB difference of a pixel in the test PCB is greater than the RGB difference threshold (e.g., 15), the pixel is determined to be abnormal. Based on the actual defect of the test PCB, if the result of the abnormality determination is consistent with the result of the actual defect (i.e., the point is indeed located in the defect area), then the pixel is considered to effectively reflect the defect characteristics in this test board, and the effective value of the pixel for the test PCB is 1.

[0049] The mean of the effective values ​​of each pixel on each test PCB board is used as the effectiveness index for that pixel.

[0050] In some embodiments, the optimal partitioning strategy for the PCB board is determined based on the validity index corresponding to each pixel, including: Based on the validity index corresponding to each pixel, multiple valid pixels are filtered out. For example, pixels with a validity index greater than the validity index threshold (e.g., 0.5) are considered valid pixels. For any two valid pixels, the correlation of the RGB difference between the two valid pixels is calculated based on the RGB difference of each test PCB board corresponding to the two valid pixels. For example, the RGB difference between the two valid pixels corresponding to each test PCB board can be substituted into the correlation coefficient calculation formula (e.g., Pearson correlation coefficient, Spearman rank correlation coefficient, etc.) to obtain the correlation of the RGB difference between the two valid pixels. The optimal partitioning strategy for the PCB board is determined based on the correlation between the RGB differences and the coordinate distance between any two valid pixels.

[0051] Specifically, the optimal partitioning strategy for a PCB board is determined using the K-means clustering algorithm based on the RGB difference correlation and coordinate distance between any two valid pixels. First, for each test PCB board, the RGB difference correlation coefficient between all pairs of valid pixels is calculated to quantify the similarity of their color change trends. Simultaneously, the Euclidean coordinate distance of the pixels in the image is calculated to reflect spatial relationships. Then, the correlation coefficient and coordinate distance of each pair of pixels are combined into a two-dimensional feature vector, which serves as the input data for K-means clustering. By setting a reasonable number of clusters K (e.g., based on the elbow rule or contour coefficient optimization), the algorithm divides the pixels into K clusters, ensuring that pixels within the same cluster have both high correlation (strong synchronicity of color changes) and spatial proximity (close coordinate distance), while the correlation and distance differences between pixels in different clusters are significant. Finally, each cluster corresponds to a potential partition, and its boundary is determined by the coordinate distribution of pixels within the cluster, for example, by generating the sub-region contour using the convex hull algorithm or the minimum bounding rectangle. The advantages of this strategy are: ensuring consistent pixel compensation behavior within a partition through correlation constraints (such as similar deformation patterns), reducing cross-region compensation errors; and avoiding sub-region fragmentation through distance constraints.

[0052] The model building module is used to build defect identification models.

[0053] Specifically, the defect recognition model includes convolutional layers, pooling layers, a feature pyramid network, a global average pooling layer, and a softmax classifier. The convolutional layers use learnable convolutional kernels to slide and extract local features of sub-images. The pooling layers are used to downsample the feature maps generated by the convolutional layers. The feature pyramid network is used to generate multi-scale feature maps. The softmax classifier is used to output the defect category and confidence level corresponding to the region.

[0054] Specifically, firstly, the convolutional layer, as the core module for feature extraction, slides a set of learnable convolutional kernels (such as 3×3 or 5×5) across the input sub-image. It utilizes local connectivity and weight sharing mechanisms to capture spatial relationships between pixels (such as edges and textures), generating multi-channel feature maps, with each channel corresponding to a specific feature response. Subsequently, pooling layers (such as max pooling or average pooling) downsample the feature maps, reducing the number of parameters by preserving the maximum or average values ​​of local regions. This enhances the model's robustness to minor deformations such as translation and rotation, reducing the risk of overfitting.

[0055] To address the challenge of diverse PCB defect scales (such as minute scratches and large-area stains), a Feature Pyramid Network (FPN) is introduced. This network generates multi-level feature maps by fusing top-down (upsampling of high-level semantic features) and bottom-up (lateral connection of low-level detailed features) pathways. For example, high-level feature maps have low resolution but rich semantic information, making them suitable for detecting large defects; low-level feature maps have high resolution and clear details, enabling the localization of minute defects. FPN allows the model to utilize multi-scale information simultaneously in a single forward propagation, significantly improving its ability to identify defects of different sizes.

[0056] Finally, the global average pooling layer compresses the multi-scale feature map output by the FPN into a fixed-dimensional feature vector, replacing the traditional fully connected layer to reduce the number of parameters while preserving global spatial information. This vector is then input into the Softmax classifier, which calculates the probability distribution of each defect category using a normalized exponential function, outputting the category label (such as short circuit, open circuit, spur) and the corresponding confidence score (a value between 0 and 1, representing classification certainty).

[0057] The defect detection module is used to extract image features of the sub-images corresponding to each region, determine whether there are defects based on the image features of the sub-images corresponding to the region, and if so, identify the defect information of the sub-images corresponding to the region through the defect recognition model.

[0058] In some embodiments, image features of the sub-image corresponding to the region are extracted, and based on the image features of the sub-image corresponding to the region, it is determined whether a defect exists, including: Extract the image features of the sub-image corresponding to the region, wherein the image features of the sub-image include at least the RGB value of each valid pixel; The global RGB difference is calculated based on the RGB values ​​of each valid pixel in the sub-image and the RGB values ​​of each valid pixel in the standard image. Specifically, for each valid pixel, the difference between the RGB values ​​of the valid pixel in the sub-image and the RGB values ​​of the valid pixel in the standard image is calculated to obtain the RGB difference of the valid pixel. The average of the RGB differences of all valid pixels is then calculated to obtain the global RGB difference. Based on the global RGB difference, it is determined whether a defect exists. If the global RGB difference is greater than the global RGB difference threshold (e.g., 20), a defect is determined to exist. If so, the sub-image corresponding to the region is input into the defect recognition model to identify the defect information of the sub-image corresponding to the region.

[0059] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. An AI vision-based automatic PCB defect filtering and inspection system, characterized in that, include: The image acquisition module is used to acquire an initial image of the PCB board in an initial pose, and to perform displacement compensation on the initial pose based on multiple key alignment points and the initial image of the PCB board. After displacement compensation, the image of the PCB board is acquired. The region segmentation module is used to divide the PCB board image into multiple sub-images corresponding to different regions based on the optimal partitioning strategy of the PCB board. The model building module is used to build defect identification models; The defect detection module is used to extract image features of the sub-images corresponding to each region, determine whether there are defects based on the image features of the sub-images corresponding to the region, and if so, identify the defect information of the sub-images corresponding to the region through the defect recognition model.

2. The AI ​​vision-based automatic PCB defect filtering and inspection system according to claim 1, characterized in that, Several key alignment sites were identified, including: In the initial pose, acquire initial images of multiple test PCB boards; Identify multiple alignment sites; For each pair of sites, the interference index of the pair of sites is determined, and the offset index of the pair of sites is determined based on the initial images of multiple test PCBs. Based on the interference and offset indices of each pair site, multiple pair sites are screened to determine multiple candidate pair sites. Based on initial images of multiple test PCBs and multiple candidate alignment sites, several key alignment sites were identified.

3. The AI ​​vision-based automatic PCB defect filtering and inspection system according to claim 2, characterized in that, Based on initial images of multiple test PCBs and multiple candidate alignment sites, several key alignment sites were identified, including: Based on the interference and offset indices of each candidate pair site and the coordinate distance between any two candidate pair sites, multiple pair site combinations are determined as the initial population, wherein each pair site combination includes at least three candidate pair sites. For each point combination and each test PCB board, based on the initial image of the test PCB board, determine the translational deviation and rotation angle corresponding to the point combination, and then determine the translational deviation and rotation angle of the test PCB board corresponding to the point combination. A fitness function is constructed, which is related to the compensation effectiveness index corresponding to the point combination. The compensation effectiveness index corresponding to the point combination is determined based on the translation deviation and rotation angle of each test PCB board corresponding to the point combination. The population was iteratively optimized using a genetic algorithm to identify multiple key alignment sites.

4. The AI ​​vision-based automatic PCB defect filtering and inspection system according to claim 3, characterized in that, Based on the interference and offset indices of each candidate pair site and the coordinate distance between any two candidate pair sites, multiple pair site combinations are determined as the initial population, including: S11, Quantity range; S12. For any two candidate pair sites, calculate the combination probability of the two candidate pair sites based on the interference index and offset index of each candidate pair site and the coordinate distance between the two candidate pair sites. S13. For each pair of pairs, randomly generate the number of pairs of pairs from the range of numbers, and generate pairs of pairs based on the number of pairs of pairs of pairs and the combination probability of any two candidate pairs of pairs. S14. Remove duplicates from the generated pairwise combinations; S15. Determine whether the number of duplicated pair combinations meets the preset requirements. If not, output the initialized population. If not, execute S13.

5. The AI ​​vision-based automatic PCB defect filtering and inspection system according to claim 4, characterized in that, Based on the number of pairs in the pair combination and the combination probability of any two candidate pairs, pair combinations are generated, including: Based on the combination probability of any two candidate pair sites, determine the adjacent candidate pair sites of each candidate pair site. A weighted undirected graph is generated based on the adjacent candidate pairs of each candidate pair, where the edges in the weighted undirected graph are related to the combination probability of the two candidate pairs connected. For each candidate pair site, generate the global combination probability of the candidate pair sites based on the combination probability of any two candidate pair sites. Based on the global combination probability of each candidate pair site, all candidate pair sites are sorted to generate a sorting result; Based on the sorting results, candidate alignment sites are extracted; Based on the weighted undirected graph, the extracted candidate pair sites, and the number of pair sites in the pair site combination, a pair site combination is generated.

6. The AI ​​vision-based automatic PCB defect filtering and inspection system according to claim 4, characterized in that, Based on the translational deviation and rotation angle of each test PCB board corresponding to the point combination, the compensation effectiveness index corresponding to the point combination is determined, including: For each test PCB, displacement compensation is performed based on the translational deviation and rotation angle of each test PCB corresponding to the point combination. After displacement compensation, the compensated image of the test PCB is acquired. Based on the initial image of the test PCB, the compensated image, and multiple candidate alignment points, the effective compensation value of the test PCB corresponding to the point combination is calculated. Based on the effective compensation value of each test PCB board corresponding to the point combination, the compensation effectiveness index corresponding to the point combination is determined.

7. The AI ​​vision-based automatic PCB defect filtering and inspection system according to claim 6, characterized in that, Determining the optimal partitioning strategy for the PCB board includes: For each pixel and each test PCB board, calculate the RGB difference between the pixel and the test PCB board based on the compensated image of the test PCB board and the standard image. For each pixel, the validity index corresponding to the pixel is calculated based on the RGB difference of each test PCB board. The optimal partitioning strategy for the PCB board is determined based on the validity index corresponding to each pixel.

8. The AI ​​vision-based automatic PCB defect filtering and inspection system according to claim 7, characterized in that, Based on the validity index corresponding to each pixel, the optimal partitioning strategy for the PCB board is determined, including: Based on the validity index corresponding to each pixel, filter out multiple valid pixels; For any two valid pixels, the correlation between the RGB differences of the two valid pixels is calculated based on the RGB differences of the two valid pixels corresponding to each test PCB board. The optimal partitioning strategy for the PCB board is determined based on the correlation between the RGB differences and the coordinate distance between any two valid pixels.

9. The AI ​​vision-based automatic PCB defect filtering and inspection system according to claim 8, characterized in that, Extract image features from the sub-images corresponding to the region, and determine whether defects exist based on these features, including: Extract image features of the sub-image corresponding to the region, wherein the image features of the sub-image include at least the RGB value of each valid pixel; Calculate the global RGB difference based on the RGB values ​​of each valid pixel in the sub-image and the RGB values ​​of each valid pixel in the standard image; Determine if a defect exists based on the global RGB difference.

10. The AI ​​vision-based automatic PCB defect filtering and inspection system according to any one of claims 1-9, characterized in that, The defect recognition model includes convolutional layers, pooling layers, a feature pyramid network, a global average pooling layer, and a Softmax classifier. The convolutional layers use learnable convolutional kernels to slide and extract local features of sub-images. The pooling layers are used to downsample the feature maps generated by the convolutional layers. The feature pyramid network is used to generate multi-scale feature maps. The Softmax classifier is used to output the defect category and confidence level corresponding to the region.

Images