AI vision-based pcba appearance defect intelligent detection method

Abstract

The application discloses an AI vision-based PCBA appearance defect intelligent detection method, relates to the technical field of PCBA appearance detection, and comprises the following steps: acquiring a high-resolution digital image of a PCBA to be detected and generating a high-dimensional feature descriptor; inputting the high-dimensional feature descriptor into an anomaly detection model constructed by an isolation forest algorithm, calculating an anomaly score of a potential defect area, and screening out a candidate defect area; extracting contour boundary coordinates of the candidate defect area, and cutting a corresponding area subimage; inputting the subimage into a classification model constructed by a capsule network, and completing fine-grained recognition of a defect type; generating a visual detection result image of a labeled defect position and type, and outputting a final detection report by packing related data. The method can realize accurate preliminary screening and fine-grained classification of PCBA appearance defects, improve detection precision and efficiency, and adapt to large-scale electronic manufacturing detection requirements.

Description

Technical Field

[0001] This invention belongs to the field of PCBA appearance inspection technology, specifically an intelligent detection method for PCBA appearance defects based on AI vision. Background Technology

[0002] As a core component of electronic devices, the appearance defects of PCBAs directly affect the reliability and lifespan of products. Therefore, PCBA appearance defect detection is a crucial step in the electronic manufacturing process. Currently, PCBA appearance defect detection mainly employs two methods: traditional manual inspection and conventional machine vision inspection. Manual inspection relies on the experience of the inspectors, who visually observe the PCBA surface for defects. Conventional machine vision inspection often uses algorithms such as threshold segmentation and template matching, or combines them with ordinary convolutional neural networks to build detection models to locate and classify defects.

[0003] Traditional manual inspection is inefficient and susceptible to factors such as fatigue and subjective judgment, resulting in high rates of missed and false defects, making it difficult to meet the needs of large-scale assembly line production. In conventional machine vision inspection, threshold segmentation and template matching algorithms are poorly adapted to complex backgrounds and minor defects, and cannot effectively capture subtle features on the PCBA surface. Ordinary convolutional neural networks are prone to inaccurate localization during initial defect screening and lack the ability to identify fine-grained defect types, making it difficult to distinguish subtle defects of similar types. At the same time, the overall inspection process is redundant, making it difficult to balance inspection accuracy and efficiency.

[0004] Existing technologies cannot achieve an efficient combination of precise initial screening and fine-grained classification of defects, making it difficult to quickly and accurately detect PCBA appearance defects. This fails to meet the high requirements of the electronics manufacturing industry for detection accuracy and efficiency, necessitating an intelligent PCBA appearance defect detection method that can solve the above problems. Summary of the Invention

[0005] This invention aims to solve at least one of the technical problems existing in the prior art; Therefore, this invention proposes an intelligent detection method for PCBA appearance defects based on AI vision, including: A high-resolution digital image of the PCBA to be inspected is acquired, and a high-dimensional feature descriptor is generated based on the high-resolution digital image; The high-dimensional feature descriptor is input into the anomaly detection model built based on the isolated forest algorithm to calculate the anomaly score for each potential defect region. Based on the abnormal scores, the potential defect areas are initially screened, and areas with scores higher than a first threshold are marked as candidate defect areas. For the marked candidate defect regions, extract their contour boundary coordinates, and crop out the corresponding region sub-images from the original high-resolution digital image based on the coordinates; The sub-image of the region is input into a classification model based on a capsule network to perform fine-grained identification of defect types and obtain defect type classification results. By combining the contour boundary coordinates of the candidate defect region with the defect type classification result, a visual detection result map labeled with the defect location and type is generated on the original high-resolution digital image; The visualization detection result image, the defect type classification result, and the corresponding contour boundary coordinate data are packaged and output as the final detection report.

[0006] Further, based on the high-resolution digital image, a high-dimensional feature descriptor is generated, including: A high-resolution digital image of the PCBA to be inspected is acquired, and the high-resolution digital image is converted to a color space to obtain a standardized image. Multi-scale image pyramids are constructed on the standardized images to generate an image pyramid set containing different resolution levels; Adaptive threshold segmentation is performed on each level of the image pyramid set to initially separate potential defect areas from normal background areas in the image. Extract various morphological and textural features from the potential defect region to generate a high-dimensional feature descriptor; The process of acquiring a high-resolution digital image of the PCBA to be inspected and performing color space conversion on the high-resolution digital image to obtain a standardized image includes: Under a preset uniform lighting environment, the original image of the PCBA to be inspected is acquired using an industrial color camera; Adjust the exposure parameters and white balance of the industrial color camera to ensure that the brightness and color temperature of the original image meet the preset standards; The acquired raw image is converted from the RGB color space to the Lab color space; In the Lab color space, histogram equalization is performed on the L channel of the image to enhance the overall contrast of the image. Gaussian filtering is applied to the processed image to smooth out random noise. The filtered image is converted from the Lab color space back to the RGB color space to obtain the standardized image.

[0007] Furthermore, a multi-scale image pyramid is constructed on the standardized image to generate an image pyramid set containing different resolution levels, including: The standardized image is used as the bottom layer image of the image pyramid; The image at the current level is subjected to Gaussian blur, and then downsampled at a fixed rate to generate the image at the next higher level. Repeat the Gaussian blur and downsampling operations until the size of the generated image is smaller than the preset minimum size threshold; All images from bottom to top are arranged in order from high resolution to low resolution to form the image pyramid set containing different resolution levels; Generate a corresponding image index for each layer of the image pyramid set.

[0008] Furthermore, adaptive threshold segmentation is performed on each level of the image pyramid set to initially separate potential defect regions from normal background regions in the image, including: Read an image from a specific level of the image pyramid set; On the image at a certain level, a neighborhood window of a fixed size is defined with each pixel in the image as the center; Calculate the average grayscale value of all pixels within the neighborhood window; Subtract a fixed constant from the mean gray value to obtain the adaptive segmentation threshold for the pixel. The grayscale value of the pixel is compared with the adaptive segmentation threshold. If the grayscale value is less than the threshold, it is marked as a foreground pixel; otherwise, it is marked as a background pixel. Traverse all pixels of the image at a certain level to complete the segmentation and generate a binary segmentation image, in which the foreground region is the potential defect region and the background region is the normal background region; For other levels of images in the image pyramid set, the operation from reading the image to generating the binarized segmented image is repeated.

[0009] Furthermore, multiple morphological and texture features of the potential defect region are extracted to generate a high-dimensional feature descriptor, including: For each potential defect region, calculate its area, perimeter, aspect ratio of the minimum bounding rectangle, and average gray value within the region; Extract the Hu invariant moments of the potential defect region to obtain a set of eigenvalues ​​describing the geometric invariance of the region; Calculate the gray-level co-occurrence matrix of the potential defect region, and extract four texture features—contrast, correlation, energy, and homogeneity—from the gray-level co-occurrence matrix. All calculated area, perimeter, aspect ratio, average gray value, Hu invariant moment eigenvalue, contrast, correlation, energy, and homogeneity are arranged into a feature vector in a preset order. The feature vector is normalized to scale the feature values ​​of each dimension to the range of zero to one, thus obtaining the high-dimensional feature descriptor.

[0010] Furthermore, the high-dimensional feature descriptor is input into an anomaly detection model constructed based on the isolated forest algorithm to calculate the anomaly score for each potential defect region, including: A large number of known normal PCBA image region samples are used in advance to extract their high-dimensional feature descriptors, and the anomaly detection model based on the isolated forest algorithm is trained. The anomaly detection model based on the isolated forest algorithm contains multiple independently constructed isolated trees; Input the high-dimensional feature descriptor of the potential defect region to be tested into the model; The model traverses each isolation tree and calculates the path length from the tree root to the leaf node describing the potential defect region; Calculate the average of the path lengths calculated for all isolated trees; The average path length is substituted into a preset anomaly score calculation function to obtain the anomaly score, where the shorter the path, the higher the anomaly score.

[0011] Further, based on the abnormal scores, the potential defect regions are initially screened, and regions with scores higher than a first threshold are marked as candidate defect regions, including: Set a first threshold for filtering; Read the anomaly score corresponding to each potential defect area in sequence; The anomaly score is compared with the first threshold. Potentially defective regions with anomaly scores greater than the first threshold are retained and assigned a unique region identifier. The region identifier of each retained potential defect region, its location information in the image, and its anomaly score are associated and stored, and they are marked as the candidate defect regions.

[0012] Furthermore, the sub-image of the region is input into a classification model built on a capsule network to perform fine-grained identification of defect types, obtaining defect type classification results, including: The sub-images of the region are normalized to a fixed size that meets the input requirements of the classification model built on the capsule network. The normalized region sub-images are input into the primary convolutional layer of the classification model to extract basic image features; Through the main capsule layer of the classification model, the basic image features are encapsulated into a set of activity vectors, each activity vector representing the existence and parameters of a specific entity in the image; Information is transmitted between capsule layers through the dynamic routing algorithm of the classification model, and the activation of higher-level capsules depends on the voting consensus of lower-level capsules. At the highest level capsule layer, a classification vector representing the defect type is output, and the magnitude of the classification vector represents the probability of the corresponding defect type existing. The defect type corresponding to the classification vector with the largest modulus is output as the defect type classification result.

[0013] Furthermore, by combining the contour boundary coordinates of the candidate defect region with the defect type classification result, a visual detection result map labeled with the defect location and type is generated on the original high-resolution digital image, including: Read the original high-resolution digital image; Based on the contour boundary coordinates of each candidate defect region, the corresponding region is located on the original high-resolution digital image; Draw the position of the smallest bounding rectangle in the candidate defect area using a rectangle with a preset color and width; Obtain the defect type classification result corresponding to the candidate defect region; Near the drawn rectangle, use text labels with preset fonts and colors to indicate the specific name of the defect type classification result; The image with all defect rectangles and type text labels marked is saved as a new image file, which is the visualization detection result image.

[0014] Furthermore, the visualized inspection result image, the defect type classification result, and the corresponding contour boundary coordinate data are packaged and output as a final inspection report, including: Create a structured test report document; The visualized detection result image is embedded as an image in a designated location within the detection report document; Create a table below the visualized detection results image; In the table, create a record for each candidate defect area; In each row of records, fill in the defect sequence number corresponding to the current record, the specific name of the defect type classification result, the contour boundary coordinate data of the corresponding candidate defect area, and the calculated anomaly score in sequence. Add a header to the test report document containing the test time and the PCBA product number to be tested; Save the complete test report document in a preset format and output it to the specified storage path.

[0015] Compared with the prior art, the beneficial effects of the present invention are: High-dimensional feature descriptors generated from high-resolution digital images of the PCBA to be inspected are input into an anomaly detection model built based on the Isolation Forest algorithm. The model calculates anomaly scores for each potential defect region, and anomaly scores exceeding a first threshold are used as the judgment criterion to complete the initial screening of candidate defect regions. Compared to conventional threshold segmentation, template matching, or direct deep learning detection box localization, high-dimensional feature descriptors can comprehensively capture subtle features of the PCBA surface, while the Isolation Forest algorithm can quickly identify anomaly regions, reduce interference from irrelevant regions, lower the probability of missed and false detections in the initial screening process, improve the accuracy of candidate defect region localization, shorten the initial screening time, and adapt to the rapid inspection needs of large-scale production lines.

[0016] For candidate defect regions initially screened by isolated forests, their contour boundary coordinates are first extracted. Then, based on these coordinates, corresponding sub-images are cropped from the original high-resolution image. These local sub-images are input into a classification model built on capsule networks to complete fine-grained defect type identification. Compared to conventional CNNs that directly classify the entire image or coarsely segmented regions, or classify first and then locate, local sub-images can focus on defect regions, reducing interference from background information. Capsule networks can effectively capture the spatial structural features of defects, accurately distinguish subtle defects of similar types, improve the accuracy of defect type identification, and avoid classification errors caused by similar defect features. At the same time, local detection reduces the computational load of the model, further improving detection efficiency. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the steps of the AI ​​vision-based intelligent detection method for PCBA appearance defects described in this invention. Figure 2 Flowchart for constructing an image pyramid; Figure 3 A flowchart for calculating anomaly scores for an anomaly detection model; Figure 4 This is a graph showing the relationship between the anomaly score threshold and the defect detection rate and false detection rate. Figure 5 This is a histogram showing the frequency distribution of abnormal scores in PCBA defect detection. Detailed Implementation

[0018] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] See Figure 1The process involves acquiring a high-resolution digital image of the PCBA to be inspected and generating a high-dimensional feature descriptor based on the image. This high-dimensional feature descriptor is then input into an anomaly detection model built on an isolated forest algorithm to calculate an anomaly score for each potential defect region. Based on these scores, the potential defect regions are initially screened, with regions scoring above a first threshold marked as candidate defect regions. The contour boundary coordinates of these candidate defect regions are extracted, and corresponding region sub-images are cropped from the original high-resolution digital image based on these coordinates. These region sub-images are then input into a classification model built on a capsule network to perform fine-grained defect type identification, resulting in a defect type classification result. Combining the contour boundary coordinates of the candidate defect regions with the defect type classification result, a visual detection result image labeled with the defect location and type is generated on the original high-resolution digital image. Finally, the visual detection result image, the defect type classification result, and the corresponding contour boundary coordinate data are packaged and output as the final inspection report.

[0020] In one embodiment of the present invention, the AI ​​vision-based intelligent detection method for PCBA appearance defects involves the generation process of high-dimensional feature descriptors, acquiring a high-resolution digital image of the PCBA to be inspected, and performing color space conversion on the high-resolution digital image to obtain a standardized image. Under a preset uniform lighting environment, the original image of the PCBA to be inspected is acquired using an industrial color camera. The exposure parameters and white balance of the industrial color camera are adjusted to ensure that the brightness and color temperature of the original image meet preset standards. For example, the exposure parameters can be set to an exposure time of 10 milliseconds and a gain of 1.5 times, and the white balance parameters can be set to a red channel gain of 1.2 and a blue channel gain of 0.9, compared with the image without adjusted parameters. Compared to the original image, the adjusted image has a more uniform brightness distribution and reduced color cast. The original image is converted from the RGB color space to the Lab color space. In the Lab color space, histogram equalization is performed on the L channel of the image to enhance the overall contrast. Gaussian filtering is then applied to the processed image to smooth random noise. The kernel size of the Gaussian filter can be set to 5 pixels by 5 pixels, and the standard deviation can be set to 1.5. The filtered image is then converted back from the Lab color space to the RGB color space to obtain a standardized image. The histogram distribution of the L channel of the standardized image after processing in the Lab color space is flatter, and the contrast enhancement effect is obvious compared to the original RGB image.

[0021] In some embodiments, a multi-scale image pyramid is constructed from the standardized image to generate an image pyramid set containing different resolution levels, see [reference]. Figure 2Using a standardized image as the bottom layer of the image pyramid, Gaussian blurring is applied to the image at the current level, followed by downsampling at a fixed rate to generate the image at the next level. For example, the downsampling rate is fixed at 0.5. The Gaussian blurring uses a Gaussian kernel with a kernel size of 3 pixels by 3 pixels and a standard deviation of 0.5. The Gaussian blurring and downsampling operations are repeated until the size of the generated image is smaller than a preset minimum size threshold, which can be set to a width of 32 pixels and a height of 32 pixels. All images from the bottom layer to the top layer are arranged in order from high resolution to low resolution to form an image pyramid set containing different resolution levels. A corresponding image index is generated for each layer of the image pyramid set. The image index can be an integer number, for example, the bottom layer index is 0, and the index increases by 1 for each layer above. The image size of different resolution levels is halved at each level. Compared with single-scale images, multi-scale image pyramids can cover a range of defect sizes from minute to obvious.

[0022] In practice, adaptive thresholding is performed on each level of the image pyramid set to initially separate potential defect areas from normal background areas. An image at a specific level is read from the image pyramid set. For each pixel in that level, a fixed-size neighborhood window is defined, centered on the pixel. The window size can be set to 15 pixels wide and 15 pixels high. The average grayscale value of all pixels within the neighborhood window is calculated. This average grayscale value is then subtracted from a fixed constant, which serves as the adaptive segmentation threshold for the pixel. This fixed constant can be set to 8. The pixel's grayscale value is then compared with the adaptive segmentation threshold. The values ​​are compared. If the gray value is less than the threshold, it is marked as a foreground point; otherwise, it is marked as a background point. All pixels of a certain level of the image are traversed to complete the segmentation and generate a binary segmented image. The foreground region is the potential defect region, and the background region is the normal background region. For other levels of the image pyramid set, the operation from reading the image to generating the binary segmented image is repeated. For example, at higher resolution levels, the neighborhood window size remains unchanged, but there are more pixels, and the segmentation result is more refined. Compared with fixed threshold segmentation, adaptive threshold segmentation can adapt to local gray-level changes in the image and reduce the impact of uneven lighting.

[0023] Optionally, various morphological and textural features of potential defect regions are extracted to generate high-dimensional feature descriptors. For each potential defect region, its area, perimeter, aspect ratio of the minimum bounding rectangle, and average gray value within the region are calculated. The area is represented by the number of pixels, the perimeter by the length of the pixel boundary, the aspect ratio of the minimum bounding rectangle is the ratio of its width to its height, and the average gray value is the arithmetic mean of the gray values ​​of all pixels within the region. Hu invariant moments of the potential defect regions are extracted to obtain a set of feature values ​​describing the geometric invariance of the regions. Hu invariant moments are calculated based on image moments, and the formula is expressed as: in: It is the first Hu invariant moment. and Here, p and q are the normalized central moments, where p and q are the orders of the moments. p=2, q=0 and p=0, q=2 are used to calculate the gray-level co-occurrence matrix of the potential defect region. From this matrix, four texture features are extracted: contrast, correlation, energy, and homogeneity. Contrast reflects gray-level differences, correlation measures the linear relationship between gray levels, energy represents texture uniformity, and homogeneity describes local gray-level similarity. All calculated area, perimeter, aspect ratio, average gray value, Hu invariant moment eigenvalues, contrast, correlation, energy, and homogeneity are arranged in a preset order to form a feature vector, for example, area, perimeter, aspect ratio, average gray value, seven Hu invariant moment eigenvalues, contrast, correlation, energy, and homogeneity. The feature vector is then normalized, scaling the feature values ​​of each dimension to the range of zero to one, resulting in a high-dimensional feature descriptor. The normalization uses the minimum-maximum scaling method, as shown in the formula: Where x is the original eigenvalue, These are the normalized eigenvalues. and These are the minimum and maximum values ​​of this feature in the training set.

[0024] It is understandable that the above implementation methods describe the generation of high-dimensional feature descriptors through specific parameters and steps. In color space conversion, the Lab color space processing contrasts with the direct processing of the RGB color space, highlighting the advantage of independent adjustment of the brightness channel. In the construction of multi-scale image pyramids, the specific data of image sizes at different resolution levels demonstrate scale changes. In adaptive threshold segmentation, the example values ​​of neighborhood window size and fixed constants reflect the difference between local adaptive and global methods. In feature extraction, morphological features such as area and perimeter are combined with texture features such as contrast and energy to provide a comprehensive description of the region. The use of the Hu invariant moment formula enhances the description of shape invariance, and feature vector normalization ensures data scale consistency.

[0025] In one embodiment of the present invention, an anomaly detection model based on the isolated forest algorithm is used for calculation and preliminary screening, see reference. Figure 3A large number of known normal PCBA image region samples are used in advance to extract their high-dimensional feature descriptors. An anomaly detection model based on the isolation forest algorithm is then trained. The training process involves randomly sampling from the set of high-dimensional feature descriptors of normal samples to construct multiple isolation trees. The anomaly detection model based on the isolation forest algorithm contains multiple independently constructed isolation trees, for example, 100 isolation trees can be constructed. When constructing each isolation tree, a specified number of sample subsets are randomly selected from the training samples, and a feature dimension and segmentation value are randomly selected at each node to divide the samples into left and right subtrees until the leaf node contains only one sample or the maximum depth limit of the tree is reached. Compared with the classification model that uses all abnormal samples for training, the anomaly detection model based on the isolation forest algorithm only needs normal samples to complete the training. In some embodiments, a high-dimensional feature descriptor of the potential defect region to be tested is input into the model. The model traverses each isolation tree and calculates the path length from the root to the leaf node describing the potential defect region. The path length is measured by the number of edges traversed. For example, a high-dimensional feature descriptor traverses 12 edges from the root node to the leaf node in the first isolation tree and 15 edges in the second isolation tree. The average path length calculated for all isolation trees is taken, and the average path length is substituted into a preset anomaly score calculation function to obtain the anomaly score. The shorter the path, the higher the anomaly score. The anomaly score calculation function can be defined as follows: in: This represents the high-dimensional feature descriptor of the input. Representing high-dimensional feature descriptors Average path length across all isolated trees yes Expected value Given the number of trees The average path length over time, used for standardization. The function output value represents the total number of isolation trees. This is the anomaly score, with a value range between 0 and 1. The closer the score is to 1, the higher the probability of an anomaly. Compared with directly using the original path length, this function provides a standardized measure of anomalies.

[0026] In practice, potential defect areas are initially screened based on abnormal scores. Areas with scores higher than a first threshold are marked as candidate defect areas. A first threshold for screening is set, which can be 0.65. The abnormal scores corresponding to each potential defect area are read sequentially and compared with the first threshold. Potential defect areas with abnormal scores greater than the first threshold are retained and assigned a unique region identifier. The region identifier can be a continuously increasing integer number, such as 1, 2, 3. Compared with retaining all areas without setting a threshold, this step can filter out a large number of low-scoring background or noise areas. Optionally, the region identifier, its position information in the image, and its abnormal score of each retained potential defect area are associated and stored, and they are marked as candidate defect areas. The position information can be the coordinates of the upper left and lower right corners of the smallest bounding rectangle of the potential defect area. For example, the candidate defect area with region identifier 1 has coordinates (120, 85, 155, 120) and an abnormal score of 0.72. The associated data structure can be a list or a dictionary.

[0027] It is understood that the above implementation describes the specific process of anomaly detection and screening. The training of the anomaly detection model based on the isolated forest algorithm relies only on normal samples, which contrasts with supervised models that require both positive and negative samples. When calculating the anomaly score, the function... Mapping the average path length to a standardized probability value, path length The sample size of 5 is greater than the path length. Samples with an anomaly score of 10 will be calculated with a higher anomaly score. During the screening process, the first threshold of 0.65 is used as a dividing line to retain regions with anomaly scores of 0.70 and remove regions with anomaly scores of 0.60. The associated storage of region identifiers and coordinates provides structured data for subsequent processing.

[0028] In one embodiment of the present invention, a region sub-image is input into a classification model constructed based on a capsule network to perform fine-grained identification of defect types and obtain defect type classification results. The region sub-image is then normalized to a fixed size that meets the input requirements of the classification model constructed based on the capsule network. For example, if the original size of the region sub-image is 80 pixels by 60 pixels, the normalization process will scale it to a fixed size of 32 pixels by 32 pixels. Compared with directly inputting the original size image, size normalization ensures the consistency of the model input. The normalized region sub-image is then input into the primary convolutional layer of the classification model to extract basic image features. The primary convolutional layer can contain multiple convolutional kernels, for example, using 32 convolutional kernels of size 5 pixels by 5 pixels, performing convolution operations with a stride of 1, and then processing them through the ReLU activation function to output a set of basic feature maps.

[0029] In some embodiments, the basic image features are encapsulated into a set of activity vectors through the main capsule layer of the classification model. Each activity vector represents the existence and parameters of a specific entity in the image. The main capsule layer can contain multiple capsules, each of which outputs a multi-dimensional activity vector. For example, 64 capsules can be set, with each capsule outputting an 8-dimensional activity vector. The direction of the activity vector encodes the instantiation parameters of the entity, such as position and orientation, while the magnitude of the activity vector encodes the probability of the entity's existence. Compared to a scalar neuron outputting a single numerical value, the activity vector can simultaneously represent the probability of existence and attributes. Optionally, information is passed between capsule layers through a dynamic routing algorithm of the classification model. The activation of higher-level capsules depends on the voting consensus of lower-level capsules. The dynamic routing algorithm iteratively updates the coupling coefficients, which represent the connection weights between lower-level and higher-level capsules. The output vectors of lower-level capsules are projected through a transformation matrix to form prediction vectors for higher-level capsules. The input of higher-level capsules is a weighted sum of these prediction vectors, which is then processed through a nonlinear "compression" function to obtain its output vector. The iterative process can be executed in three rounds to refine the coupling coefficients.

[0030] In practical implementation, the coupling coefficient in the dynamic routing algorithm The updates follow these rules: in: These are the original coupling coefficients initialized to 0, representing the lower-level capsules. With premium capsules The correlation between them is updated iteratively. It is a summation index that iterates through all possible advanced capsules; the formula calculates... This is the normalized coupling coefficient, used for weighted summation of the prediction vectors. At the highest-level capsule layer, a classification vector representing the defect type is output. The magnitude of the classification vector represents the probability of the corresponding defect type existing. For example, the highest-level capsule layer contains 5 capsules, corresponding to five defect types: "solder bridging," "missing component," "bent lead," "scratches," and "stains." Each capsule outputs a 16-dimensional vector. The magnitude of the output vector from the "solder bridging" capsule is 0.92, and the magnitude of the output vector from the "missing component" capsule is 0.15.

[0031] It is understandable that the classification model built on capsule networks performs fine-grained identification through vectorization and dynamic routing mechanisms. Compared with traditional neural networks that only output class probabilities, the magnitude of the activity vector in a capsule network directly represents the existence probability and the direction encoding attribute. Among the classification vectors output by the highest-level capsule layer, the classification vector with the largest magnitude corresponds to the defect type, which is output as the defect type classification result. For example, the capsule with the largest vector magnitude of "solder bridging" (0.92) is the defect type classification result "solder bridging". The dynamic routing algorithm updates the coupling coefficient iteratively. This allows information to be routed based on the consistency between capsules, enhancing the model's ability to recognize spatial hierarchical relationships between objects. The 64 8-dimensional activity vectors output by the main capsule layer are compared with the scalar feature maps output by the convolutional layer, providing a richer representation.

[0032] In one embodiment of the present invention, by combining the contour boundary coordinates of candidate defect regions with the defect type classification results, a visual detection result map labeled with the defect location and type is generated on the original high-resolution digital image. The original high-resolution digital image is read, which can be a three-dimensional array stored in memory with a size of 2448 pixels by 2048 pixels. Based on the contour boundary coordinates of each candidate defect region, the corresponding region is located on the original high-resolution digital image. The contour boundary coordinates of the candidate defect regions are stored in the form of a list. For example, the contour boundary coordinate list of the first candidate defect region is [(120,85),(155,85),(155,120),(120,120)]. These coordinates define a polygonal region. Compared with directly using rectangular coordinates, contour boundary coordinates can more accurately describe the shape of irregular defects.

[0033] In some embodiments, a rectangle with a preset color and width is drawn at the position of the minimum bounding rectangle of the candidate defect area. The preset color can be set to red (RGB value 255,0,0), and the preset width can be set to 2 pixels. Calculating the minimum bounding rectangle requires finding the smallest regular rectangle that can completely enclose the contour boundary coordinates. For example, based on the contour boundary coordinate list [(120,85),(155,85),(155,120),(120,120)], the coordinates of the top left corner of the minimum bounding rectangle are calculated to be (120,85), the width is 35 pixels, and the height is 35 pixels. A red rectangle is then drawn at this position. Optionally, the defect type classification result corresponding to the candidate defect area is obtained. The defect type classification result is stored in string form. For example, the defect type classification result of the first candidate defect area is "solder bridging", and the defect type classification result of the second candidate defect area is "missing component".

[0034] In practice, near the drawn rectangle, text labels with preset fonts and colors are used to indicate the specific name of the defect type classification result. The preset font can be set to "Arial" with a font size of 12 points, and the preset color can be set to blue (RGB value 0,0,255). The placement of the text labels can be determined by calculating the upper right corner coordinates of the rectangle and offsetting it by a certain number of pixels. For example, if the upper right corner coordinates of the rectangle are (155,85), it can be offset by (5,0) pixels to place the starting coordinates of the text label at (160,85). The label content is the defect type classification result string "Solder Bridge". Compared with a plain outline drawing without text labels, text labels provide direct defect type information. It can be understood that the above steps generate graphic and text labels for each candidate defect area, as shown in Table 1.

[0035] Table 1: Information Table of Candidate Defect Areas Among them, the starting coordinates of the labeled text Based on the coordinates of the minimum bounding rectangle The calculation formula is as follows: in: and These are the x and y coordinates of the starting coordinates of the text label. and It is the coordinate of the top-left corner of the smallest bounding rectangle. and These are the width and height of the smallest bounding rectangle. It is the preset horizontal offset.

[0036] See Figure 4 This is a graph showing the impact of the anomaly score threshold on the defect detection rate and false positive rate. The defect detection rate curve remains relatively stable at around 100%, with only a very slight decrease when the threshold approaches 0.9. This indicates that under the Isolation Forest algorithm, as long as the anomaly score threshold is set below 0.9, almost all real defects can be successfully detected. As the anomaly score threshold gradually increases from 0.1 to 0.9, the false positive rate drops sharply from approximately 130% to about 2%. The lower the threshold, the easier it is for the model to misclassify normal areas as defects; the higher the threshold, the significantly lower the false positive rate, but it also slightly affects the detection rate. When the threshold is in the range of 0.2 to 0.3, the false positive rate drops rapidly from over 80% to below 60%, representing the stage with the most significant performance improvement.

[0037] In one embodiment of the present invention, the process of generating and outputting the final inspection report includes packaging the visualized inspection result image, the defect type classification result, and the corresponding contour boundary coordinate data into a final inspection report, creating a structured inspection report document, embedding the visualized inspection result image as an image in a designated location in the inspection report document, creating a table below the visualized inspection result image, creating a record for each candidate defect area in the table, and sequentially filling in the defect sequence number corresponding to the current record, the specific name of the defect type classification result, the contour boundary coordinate data of the corresponding candidate defect area, and the calculated anomaly score in each record, adding header information including the inspection time and the PCBA product number to be inspected to the inspection report document, saving the complete inspection report document in a preset format, and outputting it to a designated storage path.

[0038] In practice, the visual inspection results, defect type classification results, and corresponding contour boundary coordinate data are packaged and output as the final inspection report. A structured inspection report document is created, which can be initialized and generated in memory using a document processing library. The document page size is set to A4, with 2.5 cm margins on all sides. The visual inspection results are embedded as an image in a specified location within the inspection report document. This specified location can be the centered area at the top of the document page. By calling the image insertion function of the document processing library, the file path of the visual inspection results is passed in, and the width of the inserted image is set to 15 cm while maintaining the original aspect ratio. Compared with a plain text document, the embedded image provides intuitive visual information on defect location. In some embodiments, a table is created below the visual inspection result image. The table creation function can specify 6 rows and 5 columns. The first row is used as the table header. The table line width is set to a single solid line of 0.5 points, and the width of each column is adjusted to fit the content. In the table, a record is created for each candidate defect area. For example, if 3 candidate defect areas are detected, 3 data records are created below the header row. In each record, the defect sequence number corresponding to the current record, the specific name of the defect type classification result, the contour boundary coordinate data of the corresponding candidate defect area, and the calculated anomaly score are filled in sequentially. The defect sequence number starts from 1 and is numbered sequentially. The specific name of the defect type classification result is such as "Solder Bridge". The contour boundary coordinate data can be the coordinate values ​​of the smallest bounding rectangle, in the format of "top left corner X, top left corner Y, bottom right corner X, bottom right corner Y". The anomaly score is retained to two decimal places. A complete record data is such as "1;Solder Bridge;210,150,250,190;0.72".

[0039] Optionally, a header containing the inspection time and the PCBA product number to be inspected can be added to the inspection report document. The inspection timestamp is obtained by calling the system time function and formatted as "year-month-day hour:minute:second", such as "2023-10-26 14:30:15". The PCBA product number to be inspected is passed from the upstream production management system, such as "PCBA-20231026-001". The header information can be placed at the top of the document and displayed in a large, bold font. In specific implementation, the complete inspection report document is saved in a preset format and output to a specified storage path. The preset format can be PDF format, which is a cross-platform and difficult-to-modify document format. The document processing library's save function is called to render and save the document object built in memory as a PDF file. The file name can include the product number and inspection time. The specified storage path can be the network shared directory "\server\inspection_reports\". The save operation will write the final PDF file to this path. Compared with unstructured scattered images and data files, this step generates an integrated and standardized result document.

[0040] As can be understood, the above implementation describes the process of generating and outputting the final inspection report. The structured inspection report document integrates image, table, and text information. The inspection report document is saved in PDF format to ensure format stability. The defect serial number, the specific name of the defect type classification result, the contour boundary coordinate data, and the anomaly score recorded in the table provide a structured data list. The header information consisting of the inspection time "2023-10-26 14:30:15" and the PCBA product number to be inspected "PCBA-20231026-001" uniquely identifies an inspection job. The file is output to the path "\server\inspection_reports\" for convenient centralized archiving and access.

[0041] See Figure 5This is a histogram showing the frequency distribution of anomaly scores in PCBA defect detection, visually illustrating the distribution of anomaly scores for potential defect regions using the Isolation Forest algorithm. A screening threshold of 0.6 was used for initial screening of candidate defect regions. The average score for all potential defect regions was 0.84, indicating a generally high degree of anomaly. Anomaly scores were mainly concentrated in the 0.9–1.0 range, occurring 18 times, indicating that most regions were classified as highly anomaly by the model. Scores in the 0.6–0.9 range also showed some distribution, indicating the presence of regions with moderate anomalies. Regions with scores below 0.6 had extremely low frequency, indicating good model discrimination against normal backgrounds. The threshold of 0.6 is significantly lower than the average score of 0.84, meaning that almost all highly anomaly regions are retained as candidate defects, ensuring a high detection rate and avoiding missed detections. It also includes some moderately anomaly regions, which may slightly increase the workload of subsequent fine-grained classification, but effectively reduces the risk of missed detections.

[0042] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. An AI vision-based PCBA appearance defect intelligent detection method, characterized in that, include: A high-resolution digital image of the PCBA to be inspected is acquired, and a high-dimensional feature descriptor is generated based on the high-resolution digital image; The high-dimensional feature descriptor is input into the anomaly detection model built based on the isolated forest algorithm to calculate the anomaly score for each potential defect region. Based on the abnormal scores, the potential defect areas are initially screened, and areas with scores higher than a first threshold are marked as candidate defect areas. For the marked candidate defect regions, extract their contour boundary coordinates, and crop out the corresponding region sub-images from the original high-resolution digital image based on the coordinates; The sub-image of the region is input into a classification model based on a capsule network to perform fine-grained identification of defect types and obtain defect type classification results. By combining the contour boundary coordinates of the candidate defect region with the defect type classification result, a visual detection result map labeled with the defect location and type is generated on the original high-resolution digital image; The visualization detection result image, the defect type classification result, and the corresponding contour boundary coordinate data are packaged and output as the final detection report.

2. The AI vision-based PCBA appearance defect intelligent detection method according to claim 1, characterized in that, Based on the high-resolution digital image, a high-dimensional feature descriptor is generated, including: A high-resolution digital image of the PCBA to be inspected is acquired, and the high-resolution digital image is converted to a color space to obtain a standardized image. Multi-scale image pyramids are constructed on the standardized images to generate an image pyramid set containing different resolution levels; Adaptive threshold segmentation is performed on each level of the image pyramid set to initially separate potential defect areas from normal background areas in the image. Extract various morphological and textural features from the potential defect region to generate a high-dimensional feature descriptor; The process of acquiring a high-resolution digital image of the PCBA to be inspected and performing color space conversion on the high-resolution digital image to obtain a standardized image includes: Under a preset uniform lighting environment, the original image of the PCBA to be inspected is acquired using an industrial color camera; Adjust the exposure parameters and white balance of the industrial color camera to ensure that the brightness and color temperature of the original image meet the preset standards; The acquired raw image is converted from the RGB color space to the Lab color space; In the Lab color space, histogram equalization is performed on the L channel of the image to enhance the overall contrast of the image. Gaussian filtering is applied to the processed image to smooth out random noise. The filtered image is converted from the Lab color space back to the RGB color space to obtain the standardized image.

3. The AI vision-based PCBA appearance defect intelligent detection method according to claim 2, characterized in that, Multi-scale image pyramids are constructed on the standardized images to generate an image pyramid set containing different resolution levels, including: The standardized image is used as the bottom layer image of the image pyramid; The image at the current level is subjected to Gaussian blur, and then downsampled at a fixed rate to generate the image at the next higher level. Repeat the Gaussian blur and downsampling operations until the size of the generated image is smaller than the preset minimum size threshold; All images from bottom to top are arranged in order from high resolution to low resolution to form the image pyramid set containing different resolution levels; Generate a corresponding image index for each layer of the image pyramid set.

4. The AI vision-based PCBA appearance defect intelligent detection method according to claim 3, characterized in that, Adaptive threshold segmentation is performed on each level of the image pyramid set to initially separate potential defect regions from normal background regions in the image, including: Read an image from a specific level of the image pyramid set; On the image at a certain level, a neighborhood window of a fixed size is defined with each pixel in the image as the center; Calculate the average grayscale value of all pixels within the neighborhood window; Subtract a fixed constant from the mean gray value to obtain the adaptive segmentation threshold for the pixel. The grayscale value of the pixel is compared with the adaptive segmentation threshold. If the grayscale value is less than the threshold, it is marked as a foreground pixel; otherwise, it is marked as a background pixel. Traverse all pixels of the image at a certain level to complete the segmentation and generate a binary segmentation image, in which the foreground region is the potential defect region and the background region is the normal background region; For other levels of images in the image pyramid set, the operation from reading the image to generating the binarized segmented image is repeated.

5. The AI vision-based PCBA appearance defect intelligent detection method according to claim 4, characterized in that, Extracting various morphological and textural features from the potential defect region to generate a high-dimensional feature descriptor, including: For each potential defect region, calculate its area, perimeter, aspect ratio of the minimum bounding rectangle, and average gray value within the region; Extract the Hu invariant moments of the potential defect region to obtain a set of eigenvalues ​​describing the geometric invariance of the region; Calculate the gray-level co-occurrence matrix of the potential defect region, and extract four texture features—contrast, correlation, energy, and homogeneity—from the gray-level co-occurrence matrix. All calculated area, perimeter, aspect ratio, average gray value, Hu invariant moment eigenvalue, contrast, correlation, energy, and homogeneity are arranged into a feature vector in a preset order. The feature vector is normalized to scale the feature values ​​of each dimension to the range of zero to one, thus obtaining the high-dimensional feature descriptor.

6. The AI vision-based PCBA appearance defect intelligent detection method according to claim 5, characterized in that, The high-dimensional feature descriptor is input into an anomaly detection model built based on the isolated forest algorithm to calculate the anomaly score for each potential defect region, including: A large number of known normal PCBA image region samples are used in advance to extract their high-dimensional feature descriptors, and the anomaly detection model based on the isolated forest algorithm is trained. The anomaly detection model based on the isolated forest algorithm contains multiple independently constructed isolated trees; Input the high-dimensional feature descriptor of the potential defect region to be tested into the model; The model traverses each isolation tree and calculates the path length from the tree root to the leaf node describing the potential defect region; Calculate the average of the path lengths calculated for all isolated trees; The average path length is substituted into a preset anomaly score calculation function to obtain the anomaly score, where the shorter the path, the higher the anomaly score.

7. The AI vision-based PCBA appearance defect intelligent detection method according to claim 6, characterized in that, The potential defect regions are initially screened based on the abnormal scores, and regions with scores higher than a first threshold are marked as candidate defect regions, including: Set a first threshold for filtering; Read the anomaly score corresponding to each potential defect area in sequence; The anomaly score is compared with the first threshold. Potentially defective regions with anomaly scores greater than the first threshold are retained and assigned a unique region identifier. The region identifier of each retained potential defect region, its location information in the image, and its anomaly score are associated and stored, and they are marked as the candidate defect regions.

8. The AI vision-based PCBA appearance defect intelligent detection method according to claim 7, characterized in that, The sub-image of the region is input into a classification model built on a capsule network to perform fine-grained identification of defect types, resulting in defect type classification results, including: The sub-images of the region are normalized to a fixed size that meets the input requirements of the classification model built on the capsule network. The normalized region sub-images are input into the primary convolutional layer of the classification model to extract basic image features; Through the main capsule layer of the classification model, the basic image features are encapsulated into a set of activity vectors, each activity vector representing the existence and parameters of a specific entity in the image; Information is transmitted between capsule layers through the dynamic routing algorithm of the classification model, and the activation of higher-level capsules depends on the voting consensus of lower-level capsules. At the highest level capsule layer, a classification vector representing the defect type is output, and the magnitude of the classification vector represents the probability of the corresponding defect type existing. The defect type corresponding to the classification vector with the largest modulus is output as the defect type classification result.

9. The AI vision-based PCBA appearance defect intelligent detection method according to claim 8, characterized in that, By combining the contour boundary coordinates of the candidate defect region with the defect type classification result, a visual detection result map labeled with the defect location and type is generated on the original high-resolution digital image, including: Read the original high-resolution digital image; Based on the contour boundary coordinates of each candidate defect region, the corresponding region is located on the original high-resolution digital image; Draw the position of the smallest bounding rectangle in the candidate defect area using a rectangle with a preset color and width; Obtain the defect type classification result corresponding to the candidate defect region; Near the drawn rectangle, use text labels with preset fonts and colors to indicate the specific name of the defect type classification result; The image with all defect rectangles and type text labels marked is saved as a new image file, which is the visualization detection result image.

10. The AI vision-based PCBA appearance defect intelligent detection method according to claim 9, characterized in that, The visualized inspection result image, the defect type classification result, and the corresponding contour boundary coordinate data are packaged and output as the final inspection report, including: Create a structured test report document; The visualized detection result image is embedded as an image in a designated location within the detection report document; Create a table below the visualized detection results image; In the table, create a record for each candidate defect area; In each row of records, fill in the defect sequence number corresponding to the current record, the specific name of the defect type classification result, the contour boundary coordinate data of the corresponding candidate defect area, and the calculated anomaly score in sequence. Add a header to the test report document containing the test time and the PCBA product number to be tested; Save the complete test report document in a preset format and output it to the specified storage path.

Images