Intelligent monitoring methods for PCB circuit board production based on machine vision

Abstract

This invention belongs to the field of image enhancement technology, specifically relating to an intelligent monitoring method for PCB circuit board production based on machine vision. The method includes: obtaining the structural homogeneity of pixels based on the difference in gradient direction between a pixel and its surrounding pixels; obtaining the filter window size for each pixel based on the structural homogeneity, filtering the brightness map using the pixel's filter size to obtain an initial illumination component map; obtaining the saliency map of the circuit board image, and obtaining a detail preservation coefficient based on the saliency map; obtaining the reflection component of the circuit board image using the Retinex algorithm, and correcting the reflection component using the detail preservation coefficient to obtain a corrected reflection component. A reconstructed image is obtained using the tone map, saturation map, and corrected reflection component, and then used for PCB circuit board production monitoring. This invention avoids false defect detections by using reconstructed images for PCB circuit board production monitoring.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology. More specifically, this invention relates to a machine vision-based intelligent monitoring method for PCB circuit board production. Background Technology

[0002] In the PCB manufacturing process, PCB production monitoring is usually carried out by acquiring PCB images. However, the PCB surface has a complex material composition, with both highly diffuse reflective solder mask and highly specular reflective metal solder joints and pins. Therefore, the acquired PCB images may be locally overexposed due to uneven lighting, which may cause the PCB's defect features to be obscured.

[0003] The Retinex algorithm is commonly used for circuit board image enhancement to reduce the impact of uneven illumination on defect detection and prevent lighting interference during the circuit board manufacturing process. However, the Retinex algorithm uses Gaussian blur with a fixed window to estimate the illumination component. Gaussian blur indiscriminately processes high-frequency edges on the PCB board, such as component edges and pad boundaries. This phenomenon can lead to artifacts in areas with drastic changes in brightness. These artifacts may resemble circuit board defects such as solder bridging or excess adhesive, making these areas easily identified as defects during subsequent circuit board manufacturing monitoring, resulting in false alarms. Furthermore, the Retinex algorithm enhances the image by removing the illumination component from each of the RGB channels individually. Therefore, image enhancement may alter the proportions of pixel values ​​in the RGB channels, causing color shifts. Consequently, using the existing Retinex algorithm for circuit board image enhancement can, to some extent, interfere with circuit board manufacturing monitoring. Summary of the Invention

[0004] To address the technical problems of artifacts and color shifts easily generated during image enhancement of circuit board images, which interfere with circuit board production monitoring, this invention provides an intelligent monitoring method for PCB circuit board production based on machine vision. The method includes: converting the circuit board image from RGB color space to HSV color space to obtain a hue map, saturation map, and brightness map; obtaining the structural homogeneity of pixels based on the differences in gradient directions between pixels and their surrounding pixels; obtaining the filter window size for each pixel based on the structural homogeneity; and filtering the brightness map using a window the size of the pixel's filter size as the filter window to obtain an initial... The initial illumination component map is generated; the reflection component of the circuit board image is obtained by logarithmic subtraction between the circuit board image and the initial illumination component map using the Retinex algorithm; the saliency map of the circuit board image is obtained using a saliency detection algorithm, and the detail preservation coefficient is obtained based on the saliency map; the reflection component is corrected using the detail preservation coefficient to obtain the corrected reflection component; the hue map, saturation map, and second brightness map are combined into an HSV color space image, and the HSV color space image is converted into an RGB color space to obtain a reconstructed image, where the second brightness map is an image composed of the normalized values ​​of the corrected reflection component of all pixels; the reconstructed image is used for circuit board production monitoring.

[0005] This invention uses the image composed of the corrected reflection components after mapping all pixels as a second brightness map, removing the original brightness information contaminated by uneven illumination and shadows, and replacing the original brightness information with a second brightness map that reflects the essential attributes of the object. Since the second brightness map itself has undergone detail enhancement and correction, it not only eliminates illumination interference and illuminance estimation errors, but also injects enhancement effects on key features such as solder joints and components into the brightness channel of the final image. This generates an ideal image free of illumination artifacts and color distortion, providing reliable visual input for subsequent automated monitoring and making circuit board production monitoring more reliable.

[0006] Preferably, the structural homogeneity satisfies the following relation: In the formula, For the circuit board image, the first Structural homogeneity of individual pixels For the circuit board image, the first The first pixel Gradient direction features of neighboring pixels For the circuit board image, the first The neighborhood gradient direction features of each pixel The side length of the neighborhood of a pixel. This is the cosine similarity function.

[0007] This invention addresses the issue of artifacts appearing at the edges of circuit boards when using a fixed filtering window size in the traditional Retinex algorithm. It guides adaptive filtering by calculating structural homogeneity. A small window is used for structurally complex areas to preserve edge details, while a large window is used for flat areas to smooth out the effects of illumination, resulting in more accurate illuminance components, effectively suppressing artifacts, improving the accuracy of illumination estimation, and reducing interference with PCB production monitoring.

[0008] Preferably, the size of the filtering window satisfies the following relationship: In the formula, For the circuit board image, the first The size of the filtering window used per pixel Minimum filtering window For the maximum filtering window, For the circuit board image, the first Structural homogeneity of individual pixels For odd number transformation functions, where , This is an example input for the odd number conversion function.

[0009] Preferably, the filtering of the luminance map includes: performing Gaussian filtering on the luminance map.

[0010] Preferably, the saliency detection algorithm is a frequency tuning saliency region detection algorithm.

[0011] Preferably, the significance detection algorithm is a spectral residual significance detection algorithm.

[0012] Preferably, the detail preservation coefficients satisfy the following relationship: In the formula, For the circuit board image, the first Detail preservation coefficient per pixel For the circuit board image, the first The significance value of each pixel. The adjustment parameter controls the degree to which saliency affects the preservation of detail.

[0013] This invention constructs a detail retention coefficient based on saliency. This coefficient identifies regions crucial for defect detection and enhances and corrects the reflection components of these regions. This method addresses the problem of lost critical details such as solder joint surface texture. By enhancing and preserving these critical details, circuit board defects can be identified more accurately, thereby reducing the false negative rate.

[0014] Preferably, the corrected reflection component satisfies the following relationship: In the formula, For the circuit board image, the first Corrected reflection component values ​​for each pixel For the circuit board image, the first The reflection component value of each pixel For the circuit board image, the first The detail retention factor for each pixel.

[0015] This invention addresses the problem that traditional Retinex algorithms may cause blurring of details in critical areas such as solder joints and components. By using a detail preservation coefficient to correct the reflection component, it selectively enhances the details and contrast of important areas, making minor defects more prominent in the enhanced image and improving detection sensitivity.

[0016] Preferably, the step of monitoring circuit board production by reconstructing images includes: acquiring a standard image of a PCB circuit board, acquiring an image of a PCB circuit board to be inspected, acquiring a reconstructed image of the image of the PCB circuit board to be inspected, using a template matching algorithm to obtain the similarity between the standard image and the reconstructed image, and when the similarity is greater than or equal to a threshold, the PCB circuit board production process is normal, and when the similarity is less than the threshold, the PCB circuit board production process is abnormal.

[0017] This invention addresses the problem of color shift caused by directly processing the RGB three channels in the traditional Retinex algorithm. It processes the image in the HSV color space and reconstructs the image by modifying only the luminance component while keeping the hue and saturation unchanged. This ensures that the enhanced image has natural and unshifted colors, which is more in line with human visual perception and provides a reliable basis for PCB circuit board production monitoring.

[0018] Preferably, the method for obtaining the threshold includes: obtaining a reconstructed image of a standard image, obtaining the similarity between the standard image and its reconstructed image through a template matching algorithm, and using the similarity as the threshold.

[0019] The beneficial effects of this invention are as follows: This invention suppresses artifacts by constructing a structural homogeneity index, and simultaneously enhances the details of key areas such as solder joints using visual saliency, further reducing artifact interference and effectively preserving the morphological characteristics of minute defects, thus avoiding missed detections. Furthermore, this invention operates independently on the luminance channel within the HSV space, ensuring high color fidelity in the final reconstructed image and avoiding the impact of color shifts on circuit board production monitoring. This invention provides high-quality images with uniform illumination, clear details, and no color distortion for subsequent production monitoring, significantly improving the accuracy and reliability of the circuit board production monitoring system. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the intelligent monitoring method for PCB circuit board production based on machine vision in this invention;

[0021] Figure 2 This is a schematic diagram of a PCB circuit board in this invention;

[0022] Figure 3 This is a schematic illustration of the reconstructed image in this invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0025] This invention discloses an intelligent monitoring method for PCB circuit board production based on machine vision, referring to... Figure 1 This includes steps S1 to S6:

[0026] S1. Acquire images of the PCB circuit board and perform image preprocessing.

[0027] Obtain an image of the PCB circuit board after the components have been soldered. Convert the circuit board image from RGB color space to HSV color space to obtain the hue, saturation, and brightness of each pixel in the image. The hue of all pixels constitutes a hue map, the saturation of all pixels constitutes a saturation map, and the brightness of all pixels constitutes a brightness map.

[0028] S2. Obtain the gradient direction of pixels in the image, and obtain the structural homogeneity of pixels based on the difference in gradient direction between the pixel and its surrounding pixels.

[0029] It should be noted that in the traditional Retinex algorithm, a fixed filter size is used to filter the image to estimate the illumination component. This can cause artifacts in the edge region of the circuit board. The artifacts in the image are similar in shape to defects on the circuit board, so the artifacts can interfere with the identification of circuit board defects and thus affect the production monitoring of the circuit board. In order to avoid the artifacts in the edge region of the circuit board caused by using a fixed filter size to filter the image, this invention obtains the structural homogeneity of the pixels based on the difference between the pixels and the surrounding pixels in the gradient direction, and uses the structural homogeneity to adjust the size of the filter window.

[0030] Specifically, the PCB circuit board image is converted into a grayscale image. The Sobel operator is used to obtain the horizontal and vertical gradients of each pixel in the image. The horizontal gradient is used as the first directional feature of the pixel, and the vertical gradient is used as the second directional feature. The data point formed by the first and second directional features is used as the gradient direction feature of the pixel. This is then processed with each pixel as the center. The range is the neighborhood of a pixel. Pixels within this neighborhood are considered neighboring pixels. The average of the first-direction features and the average of the second-direction features of all neighboring pixels are obtained. These averages are then combined to form the neighborhood gradient direction feature of the pixel. The structural homogeneity of the pixel is determined based on the difference between its gradient direction feature and the gradient direction features of its neighboring pixels.

[0031] For example, The value is 9, and implementers can choose according to the actual situation. The value of .

[0032] Specifically, structural homogeneity satisfies the following relation:

[0033]

[0034] In the formula, For the circuit board image, the first Structural homogeneity of individual pixels For the circuit board image, the first The first pixel Gradient direction features of neighboring pixels For the circuit board image, the first The neighborhood gradient direction features of each pixel The side length of the neighborhood of a pixel. This is the cosine similarity function.

[0035] in, Represents the first The pixel and its first The similarity of gradient directions among the nth neighboring pixels; the larger this value, the stronger the similarity of the gradient directions among the nth neighboring pixels. The more consistent the gradient directions of pixels within the neighborhood of the nth pixel, the better. The greater the structural homogeneity of the nth pixel, the smaller the value indicates that the nth pixel has a higher structural homogeneity. The greater the difference in gradient directions among pixels within the neighborhood of the nth pixel, the better. The smaller the structural homogeneity of each pixel; since the range of cosine similarity is... To facilitate subsequent calculations, the formula is modified by adding 1 and dividing by 2. Normalize.

[0036] S3. Based on the structural homogeneity of the pixels, obtain the size of the filtering window used for each pixel, and filter the brightness map using the pixel filtering size to obtain the initial illumination component map.

[0037] It should be noted that pixels with greater structural homogeneity are more likely to be located in areas with simple structures in the circuit board image. Using a larger filter window can remove the influence of illumination on circuit board production monitoring. Pixels with less structural homogeneity are more likely to be located in areas with complex structures in the circuit board image. Using a smaller filter window can protect the structural areas of the circuit board in the circuit board image and avoid artifacts. Therefore, this invention obtains the filter window size used for each pixel based on the structural homogeneity of the pixels, and filters the brightness map using the filter size of the pixels to obtain the initial illumination component map.

[0038] Specifically, a minimum and a maximum filter window size are preset, and the filter window size used for each pixel is obtained based on the structural homogeneity of the pixels and the minimum and maximum filter window sizes.

[0039] For example, the minimum filter window size is 3 and the maximum filter window size is 11.

[0040] Furthermore, a Gaussian filter is applied to the brightness map to obtain the initial illumination component map, wherein the window size used for Gaussian filtering of each pixel is the pixel's filtering window size.

[0041] The filter window size satisfies the following relationship:

[0042]

[0043] In the formula, For the circuit board image, the first The size of the filtering window used per pixel Minimum filtering window For the maximum filtering window, For the circuit board image, the first Structural homogeneity of individual pixels For odd number transformation functions, where , This is an example input for the odd number conversion function.

[0044] Among them, the structural homogeneity of pixels The smaller the number, the more likely it is to be the first. The more complex the structure surrounding a pixel, the smaller the filter window size should be to avoid artifacts during subsequent Retinex algorithm-based circuit board image enhancement; the structural homogeneity of pixels... The larger the value, the more likely it is to be the first. The simpler the structure around a pixel, the larger the filter window size should be to fully smooth out unstructured noise and texture, thereby more accurately estimating the overall illumination component of the area and avoiding interference with circuit board defect detection.

[0045] S4. Obtain the saliency map of the PCB circuit board image, and obtain the detail retention coefficient based on the saliency map.

[0046] It should be noted that the traditional Retinex algorithm obtains the reflection component by logarithmically subtracting the illuminance component from the original image. The illuminance component represents the distribution of ambient light in the image, equivalent to the initial illumination component map mentioned above, while the reflection component is the image carrying detail information. This direct logarithmic subtraction may lead to the loss of details in some areas of the PCB circuit board image, such as solder joints and components. These areas are visually salient in the circuit board image. Therefore, this invention obtains the saliency map of the PCB circuit board image and obtains the detail preservation coefficient based on the saliency map. The detail preservation coefficient is used to avoid the loss of details on the circuit board.

[0047] A saliency map of a PCB circuit board image is obtained using a saliency detection algorithm. Each value in the saliency map is the saliency value of a pixel. The detail preservation coefficient of each pixel is obtained based on the saliency value of the pixel.

[0048] In one embodiment, the saliency detection algorithm is a frequency tuning saliency region detection algorithm.

[0049] In another embodiment, the significance detection algorithm is a spectral residual significance detection algorithm.

[0050] Specifically, the detail preservation coefficients satisfy the following relationship:

[0051]

[0052] In the formula, For the circuit board image, the first Detail preservation coefficient per pixel For the circuit board image, the first The significance value of each pixel. To adjust the parameters used to control the degree to which saliency affects detail preservation, in this example... The value is 1.5, and implementers can choose according to the actual situation. The value of .

[0053] in, The larger the value, the higher the value. The more visually prominent the region to which the first pixel belongs, the more prominent the region becomes. The more likely a pixel is to be in areas such as solder joints or components, the smaller the detail preservation coefficient should be used in order to enhance the details of this part of the circuit board image and avoid false detections during subsequent circuit board production monitoring. This will allow for a greater correction of the reflection component in subsequent calculations. The smaller the value, the higher the value. The less visually significant the region to which the first pixel belongs, the less significant the region to which the second pixel belongs. The more likely a pixel is to be in a simple structural area such as the solder mask layer, the more important it is to maintain the lighting correction effect in this area. A higher detail retention factor for each pixel helps avoid making excessive modifications to that pixel.

[0054] S5. Obtain the reflection component of the PCB circuit board image, correct the reflection component using the detail preservation coefficient, and obtain the corrected reflection component.

[0055] Specifically, the existing Retinex algorithm is used to logarithmically subtract the initial illumination component image from the PCB circuit board image to obtain the reflection component of the PCB circuit board image. The reflection component is then corrected using detail preservation coefficients to obtain the corrected reflection component. Each value in the reflection component is a reflection component value, and each value in the corrected reflection component is a corrected reflection component value.

[0056] Specifically, the corrected reflection component values ​​satisfy the following relationship:

[0057]

[0058] In the formula, For the first in the image Corrected reflection component values ​​for each pixel For the first in the image The reflection component value of each pixel For the first in the image The detail retention factor for each pixel.

[0059] Among them, the detail retention coefficient The larger, The closer it is to 1, the less need it is to modify the first value. The reflection component value of the nth pixel, then for the nth pixel The smaller the correction force for the reflection component value of the nth pixel, the better. The corrected reflection component value for each pixel is close to the reflection component value; detail preservation coefficient The smaller the value, the more correction is needed. The reflection component value of the nth pixel, then for the nth pixel The greater the correction force for the reflection component value of the nth pixel, the more likely the nth pixel will be to be corrected. The larger the corrected reflection component value of each pixel is based on the reflection component value, the more it can enhance and preserve the details of key areas such as solder joints.

[0060] S6. By correcting the reflection component, a reconstructed image is obtained, and the reconstructed image is used for PCB circuit board production monitoring.

[0061] It should be noted that the traditional Retinex algorithm processes each channel of the image separately, removing the illuminance component of each channel, obtaining the reflectance component of each channel, and then recombining the reflectance components of each channel to achieve image enhancement. However, this image enhancement may cause a shift in the proportional relationship between the pixels in the RGB channels of the image, resulting in a significant color shift in the enhanced image. To avoid this situation, this invention reconstructs the image in the HSV color space by correcting the reflectance component, obtaining a reconstructed image, and then using the reconstructed image for PCB circuit board production monitoring.

[0062] Specifically, a linear transformation is used to map all values ​​in the corrected reflection component to... The image composed of the corrected reflection components after mapping all pixels is used as the second brightness map. The hue map, saturation map and the second brightness map are recombined into an image in the HSV color space. The combined image is then converted into a PCB circuit board image in the RGB color space to obtain the reconstructed image. The PCB circuit board production is monitored through the reconstructed image.

[0063] It should be further explained that the correction of the reflection component includes the inherent structural characteristics of the PCB circuit board, while eliminating the interference of the illuminance component. Therefore, by obtaining the second luminance map through the correction of the reflection component, and then obtaining the reconstructed image from the second luminance map, the reconstructed image can only express the true structure of the PCB circuit board without bringing the illuminance estimation error back into the reconstructed image. At the same time, only the luminance map is modified during the acquisition of the reconstructed image, while the hue map and saturation map remain unchanged, which reduces the color shift in the reconstructed image and makes the monitoring of PCB circuit board production through the reconstructed image more reliable.

[0064] Furthermore, the PCB circuit board production monitoring through image reconstruction includes: acquiring a standard image of the PCB circuit board, acquiring an image of the PCB circuit board to be detected, and acquiring a reconstructed image of the PCB circuit board to be detected; using a template matching algorithm to obtain the similarity between the standard image and the reconstructed image; when the similarity is greater than or equal to a threshold, the PCB circuit board production process is normal; when the similarity is less than the threshold, the PCB circuit board production process is abnormal.

[0065] For example, the threshold is obtained by: obtaining a reconstructed image of a standard image, obtaining the similarity between the standard image and its reconstructed image through a template matching algorithm, and using the similarity as the threshold.

[0066] For example, Figure 2 This is an image of the PCB circuit board used in this invention. Figure 3 The reconstructed image in this invention, Figure 2 The upper right part of the image has a higher light intensity than other parts, while the light intensity is more uniform across the circuit board in the image.

Claims

1. A machine vision-based intelligent monitoring method for PCB circuit board production, characterized in that, include: The circuit board image is converted from the RGB color space to the HSV color space to obtain the hue map, saturation map, and brightness map. The structural homogeneity of a pixel is obtained based on the difference in gradient direction between the pixel and its surrounding pixels; the size of the filtering window used for each pixel is determined based on the structural homogeneity of the pixels, which satisfies the following relationship: In the formula, For the circuit board image, the first The size of the filtering window used per pixel For the minimum filtering window, For the maximum filtering window, For the circuit board image, the first Structural homogeneity of individual pixels For odd number transformation functions, where , As an example input for the odd-number transformation function, the brightness map is filtered using a window the size of the pixel's filter size as the filter window to obtain the initial illumination component map; The Retinex algorithm is used to logarithmically subtract the initial illumination component map from the circuit board image to obtain the reflection component of the circuit board image; a saliency detection algorithm is used to obtain the saliency map of the circuit board image, and the detail preservation coefficient is obtained based on the saliency map; the reflection component is then corrected using the detail preservation coefficient to obtain the corrected reflection component. The corrected reflection component satisfies the following relationship: In the formula, For the circuit board image, the first Corrected reflection component values ​​for each pixel For the circuit board image, the first The reflection component value of each pixel For the circuit board image, the first Detail retention factor per pixel; The detail preservation coefficients satisfy the following relationship: In the formula, For the circuit board image, the first Detail preservation coefficient per pixel For the circuit board image, the first The significance value of each pixel. The adjustment parameter controls the degree to which saliency affects detail preservation. The hue map, saturation map, and second brightness map are combined to form an HSV color space image. The HSV color space image is then converted to the RGB color space to obtain a reconstructed image. The second brightness map is an image composed of the normalized values ​​of the corrected reflection components of all pixels. Circuit board production monitoring through image reconstruction.

2. The intelligent monitoring method for PCB circuit board production based on machine vision according to claim 1, characterized in that, The structural homogeneity satisfies the following relation: ； In the formula, For the circuit board image, the first Structural homogeneity of individual pixels For the circuit board image, the first The first pixel Gradient direction features of neighboring pixels For the circuit board image, the first The neighborhood gradient direction features of each pixel The side length of the neighborhood of a pixel. This is the cosine similarity function.

3. The intelligent monitoring method for PCB circuit board production based on machine vision according to claim 1, characterized in that, The filtering of the luminance map includes: performing Gaussian filtering on the luminance map.

4. The intelligent monitoring method for PCB circuit board production based on machine vision according to claim 1, characterized in that, The saliency detection algorithm is a frequency tuning saliency region detection algorithm.

5. The intelligent monitoring method for PCB circuit board production based on machine vision according to claim 1, characterized in that, The significance detection algorithm is a spectral residual significance detection algorithm.

6. The intelligent monitoring method for PCB circuit board production based on machine vision according to claim 1, characterized in that, The method of monitoring PCB production by reconstructing images includes: acquiring a standard image of a PCB circuit board, acquiring an image of a PCB circuit board to be inspected, acquiring a reconstructed image of the image of the PCB circuit board to be inspected, using a template matching algorithm to obtain the similarity between the standard image and the reconstructed image, and when the similarity is greater than or equal to a threshold, the PCB circuit board production process is normal, and when the similarity is less than the threshold, the PCB circuit board production process is abnormal.

7. The intelligent monitoring method for PCB circuit board production based on machine vision according to claim 6, characterized in that, The method for obtaining the threshold includes: obtaining a reconstructed image of a standard image, obtaining the similarity between the standard image and its reconstructed image through a template matching algorithm, and using the similarity as the threshold.

Images