Intelligent monitoring method for PCB production based on machine vision

Abstract

The invention belongs to the technical field of image enhancement, and particularly relates to an intelligent monitoring method for PCB production based on machine vision, and the method comprises the steps: obtaining the structural homogeneity of a pixel point according to the difference between the pixel point and the surrounding pixel points in the gradient direction; obtaining the size of a filtering window used for each pixel point according to the structural homogeneity of the pixel points, and filtering the brightness image through the filtering size of the pixel points to obtain an initial illumination component image; obtaining a saliency map of the circuit board image, and obtaining a detail retention coefficient according to the saliency map; and obtaining a reflection component of the circuit board image through a Retinex algorithm, and correcting the reflection component through the detail retention coefficient to obtain a corrected reflection component. And obtaining a reconstructed image through the tone image, the saturation image and the corrected reflection component, and carrying out PCB production monitoring through the reconstructed image. According to the invention, PCB production monitoring is carried out through the reconstructed image, and defect misinformation is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image enhancement. More particularly, the present application relates to an intelligent monitoring method for PCB circuit board production based on machine vision. BACKGROUND

[0002] In the production process of PCB circuit board, the production of circuit board is usually monitored by collecting images of the circuit board. However, the surface of the circuit board is complex, and the surface of the circuit board has both high diffuse reflection of the solder mask and high specular reflection of the metal solder joints and pins. Therefore, the collected images of the circuit board may be locally overexposed due to uneven lighting, which may cover the defect features of the circuit board.

[0003] Retinex algorithm is usually used for image enhancement of the circuit board to reduce the influence of uneven lighting on the detection of defects of the circuit board and to avoid the production process monitoring of the circuit board being disturbed by lighting. However, the Retinex algorithm uses a fixed window Gaussian blur to estimate the illumination component, and the Gaussian blur will treat the high-frequency edges on the PCB circuit board, such as the edges of components and the boundaries of pads, without distinction. This phenomenon may cause artifacts in areas with sharp changes in brightness. These artifacts may be similar in shape to defects such as continuous soldering and excess glue, and this part of the area may be easily identified as a defect during subsequent production monitoring of the circuit board, resulting in false positives. At the same time, the Retinex algorithm removes the illumination component from each channel of the RGB three channels of the image for image enhancement, which may cause the ratio between the RGB three channels of the image to change and cause color shift of the image during image enhancement. Therefore, the existing technology of using the Retinex algorithm for image enhancement of the circuit board may interfere with the production monitoring of the circuit board to some extent. SUMMARY

[0004] In order to solve the technical problem that artifacts and color shift are easily generated when the circuit board image is enhanced, which interferes with the circuit board production monitoring, the present invention provides an intelligent monitoring method for PCB circuit board production based on machine vision, comprising: converting the circuit board image from RGB color space to HSV color space to obtain a hue map, a saturation map and a brightness map; obtaining the structural homogeneity of the pixel point based on the difference in the gradient direction between the pixel point and the surrounding pixels; obtaining the filter window size used for each pixel point based on the structural homogeneity of the pixel point, filtering the brightness map with a window of the filter size of the pixel point as the filter window, and obtaining an initial The invention discloses a method for monitoring circuit board production by using a saliency detection algorithm to obtain a saliency map of the circuit board image, a detail preservation coefficient according to the saliency map, and a correction of the reflection component. The hue map, the saturation map and the second brightness map are combined into an HSV color space image, and the HSV color space image is converted into an RGB color space to obtain a reconstructed image, wherein the second brightness map is an image composed of the normalized values ​​of the corrected reflection components of all pixel points. The reconstructed image is used to monitor circuit board production.

[0005] This method uses an image composed of the corrected reflected components after mapping all pixels as a second luminance map. This removes the original luminance information contaminated by uneven lighting and shadows, replacing it with a second luminance map that reflects the object's essential properties. Because the second luminance map has already undergone detail enhancement, it not only eliminates lighting interference and illumination estimation errors but also incorporates the enhancement effects on key features such as solder joints and components into the luminance channel of the final image. This creates an ideal image free of lighting artifacts and color distortion, providing reliable visual input for subsequent automated monitoring and enhancing the reliability of circuit board production monitoring.

[0006] Preferably, the structural homogeneity satisfies the relationship: Where, The first The structural homogeneity of pixels, The first The first pixel The gradient direction features of the neighboring pixels, The first The neighborhood gradient direction feature of pixels, is the side length of the pixel neighborhood, is the cosine similarity function.

[0007] This paper addresses the problem of artifacts generated at the edges of circuit boards when using a fixed filter window size in the traditional Retinex algorithm. By calculating structural homogeneity to guide adaptive filtering, a small window is used for complex structural areas, preserving edge details; a large window is used for flat areas, smoothing the effects of illumination. This results in more accurate illumination components, effectively suppressing artifacts, improving illumination estimation accuracy, and reducing interference in PCB production monitoring.

[0008] Preferably, the filter window size satisfies the relationship: Where, The first The filter window size used for pixels, is the minimum filtering window, is the maximum filter window, The first The structural homogeneity of pixels, is an odd transfer function, where , Example input for the odd conversion function.

[0009] Preferably, filtering the brightness image includes: performing Gaussian filtering on the brightness image.

[0010] Preferably, the saliency detection algorithm is a frequency-tuned saliency region detection algorithm.

[0011] Preferably, the saliency detection algorithm is a spectral residual saliency detection algorithm.

[0012] Preferably, the detail preservation coefficient satisfies the relationship: Where, The first The detail preservation coefficient of each pixel, The first The saliency value of each pixel, It is an adjustment parameter used to control the influence of saliency on detail preservation.

[0013] This invention constructs a detail preservation coefficient based on saliency. This coefficient identifies areas important for defect detection and enhances and corrects the reflection components in these areas. This method addresses the problem of losing key details, such as solder joint surface texture. By enhancing and preserving these key details, it facilitates more accurate identification of circuit board defects, thereby reducing the rate of missed detections.

[0014] Preferably, the corrected reflection component satisfies the relationship: Where, The first a corrected reflection component value of the pixel point, a reflection component value of a pixel point in the circuit board image, a reflection component value of a pixel point in the circuit board image, a detail preservation coefficient of a pixel point in the circuit board image.

[0015] The present application aims at the problem that the traditional Retinex algorithm may cause the details of key areas such as welding points and components to be blurred, the reflection component is corrected through the detail preservation coefficient, the details and contrast of important areas are selectively enhanced, the small defects are more highlighted in the enhanced image, and the sensitivity of detection is improved.

[0016] Preferably, the circuit board production monitoring through the reconstructed image comprises: acquiring a standard image of a PCB circuit board, collecting an image of a PCB circuit board to be detected, and acquiring a reconstructed image of the image of the PCB circuit board to be detected, using a template matching algorithm to acquire the similarity between the standard image and the reconstructed image, when the similarity is greater than or equal to a threshold value, the PCB circuit board production process is normal, and when the similarity is less than the threshold value, the PCB circuit board production process is abnormal.

[0017] The present application aims at the problem that the traditional Retinex algorithm directly processes RGB three channels and is easy to cause color deviation, processes the image in the HSV color space, reconstructs the image by modifying only the brightness component while keeping the hue and saturation unchanged, ensures that the color of the enhanced image is natural and without deviation, is more in line with the observation habit of human eyes, and provides a reliable basis for PCB circuit board production monitoring.

[0018] Preferably, the threshold value is obtained by: acquiring a reconstructed image of a standard image, and using a template matching algorithm to acquire the similarity between the standard image and the reconstructed image of the standard image, and taking the similarity as the threshold value.

[0019] The present application has the beneficial effects that: the present application suppresses artifacts by constructing a structural homogeneity index, and further reduces the interference of artifacts by using visual saliency to enhance the details of key areas such as welding points, effectively preserves the morphological characteristics of small defects, and avoids missed detection. At the same time, the present application independently operates on the brightness channel in the HSV space, ensures the high fidelity of the color of the finally reconstructed image, and avoids the influence of color deviation on circuit board production monitoring. The present application provides a high-quality image with uniform illumination, clear details and no color distortion for subsequent production monitoring, and significantly improves the accuracy and reliability of the circuit board production monitoring system. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a flow chart schematically showing the intelligent monitoring method for PCB circuit board production based on machine vision in the present application; ​Figure 2 is a schematic diagram illustrating a PCB image in the present application; Figure 3 is a schematic diagram illustrating a reconstructed image in the present application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0022] The specific embodiments of the present application will be described in detail below with reference to the drawings.

[0023] The embodiments of the present application disclose an intelligent monitoring method for PCB production based on machine vision, referring to Figure 1 , comprising steps S1 to S6: S1, collecting a PCB image and performing image preprocessing.

[0024] After obtaining the PCB image after component welding, the circuit board image is converted from the RGB color space to the HSV color space to obtain the hue, saturation and brightness of each pixel point in the image. The hue of all pixel points constitutes a hue map, the saturation of all pixel points constitutes a saturation map, and the brightness of all pixel points constitutes a brightness map.

[0025] S2, obtaining the gradient direction of the pixel points in the image, and obtaining the structural homogeneity of the pixel points according to the difference between the pixel points and the surrounding pixel points in the gradient direction.

[0026] It should be noted that in the traditional Retinex algorithm, a fixed filter size is used to filter the image to estimate the illumination component, which will cause artifacts in the edge region of the circuit board. The artifacts in the image are similar in shape to the defects on the circuit board, so the artifacts will interfere with the recognition of the circuit board defects, and thus affect the production monitoring of the circuit board. In order to avoid the use of a fixed filter size to filter the image to cause artifacts in the edge region of the circuit board, the present application obtains the structural homogeneity of the pixel points according to the difference between the pixel points and the surrounding pixel points in the gradient direction, and adjusts the size of the filter window using the structural homogeneity.

[0027] Specifically, the PCB circuit board image is converted into a grayscale image, and the horizontal gradient and vertical gradient of each pixel in the image are obtained by the Sobel operator. The horizontal gradient is used as the first directional feature of the pixel, and the vertical gradient is used as the second directional feature of the pixel. The data point composed of the first directional feature and the second directional feature is used as the gradient directional feature of the pixel. The range is the neighborhood range of the pixel point. The pixels within the pixel point's neighborhood range are the pixel's neighborhood pixels. The mean of the first directional feature and the mean of the second directional feature of all the pixel's neighborhood pixels are obtained. The mean of the first directional feature and the mean of the second directional feature are used to form the neighborhood gradient directional feature of the pixel point. The structural homogeneity of the pixel point is obtained based on the difference between the gradient directional feature of the pixel point and the gradient directional feature of its neighborhood.

[0028] For example, 9, implementation personnel can choose according to actual conditions The value of .

[0029] Specifically, structural homogeneity satisfies the relationship:

[0030] Where, The first The structural homogeneity of pixels, The first The first pixel The gradient direction features of the neighboring pixels, The first The neighborhood gradient direction feature of pixels, is the side length of the pixel neighborhood, is the cosine similarity function.

[0031] in, Represents the Pixels and their The similarity of the gradient direction between the neighboring pixels. The larger the value, the The more consistent the gradient directions of the pixels in the neighborhood of the first pixel are, the The greater the structural homogeneity of the pixels, the smaller the value is, the The greater the difference in the gradient direction of the pixels in the neighborhood of the first pixel, the The smaller the structural homogeneity of the pixels, the smaller the cosine similarity value range is. , in order to facilitate subsequent calculations, the formula is obtained by adding 1 and dividing by 2. Perform normalization.

[0032] S3. Obtain a filter window size for each pixel according to the structural homogeneity of the pixel points, and filter the brightness map using the filter size of the pixel points to obtain an initial illumination component map.

[0033] It should be noted that the pixel points with greater structural homogeneity are more likely to be located in the area of ​​simple structure in the circuit board image. Using a larger filter window can eliminate the impact of light on circuit board production monitoring; the pixel points with smaller structural homogeneity are more likely to be located in the area of ​​complex structure in the circuit board image. Using a smaller filter window can protect the structural area of ​​the circuit board in the circuit board image and avoid artifacts. Therefore, the present invention obtains the filter window size used for each pixel point according to the structural homogeneity of the pixel point, and filters the brightness map by the filter size of the pixel point to obtain the initial light component map.

[0034] Specifically, a minimum filter window size and a maximum filter window size are preset, and the filter window size used for each pixel is obtained according to the structural homogeneity of the pixel points and the minimum filter window size and the maximum filter window size.

[0035] Exemplarily, the minimum filter window size is 3 and the maximum filter window size is 11.

[0036] Furthermore, Gaussian filtering is performed on the brightness image to obtain an initial illumination component image, wherein the window size used for Gaussian filtering of each pixel is the filtering window size of the pixel.

[0037] The filter window size satisfies the relationship:

[0038] Where, The first The filter window size used for pixels, is the minimum filtering window, is the maximum filter window, The first The structural homogeneity of pixels, is an odd transfer function, where , Example input for the odd conversion function.

[0039] Among them, the structural homogeneity of pixels The smaller the time, the more The more complex the structure around each pixel, the smaller the filter window size should be used to avoid artifacts when using the Retinex algorithm to enhance the circuit board image; the structural homogeneity of the pixel The larger the The simpler the structure around a pixel, the larger the filter window size should be used to fully smooth out non-structural noise and texture, thereby more accurately estimating the overall illumination component of the area and avoiding interference in circuit board defect detection.

[0040] S4. Obtain a saliency map of the PCB circuit board image, and obtain a detail preservation coefficient based on the saliency map.

[0041] It should be noted that the traditional Retinex algorithm obtains the reflection component by performing logarithmic subtraction between the original image and the illumination component, where the illumination component represents the distribution of the ambient light of the image in the image, which is equivalent to the above-mentioned initial illumination component map, and the reflection component is an image that carries image detail information. This direct logarithmic subtraction method may cause the loss of details of some solder joints, components and other areas in the PCB circuit board image. However, solder joints, components and other areas in the circuit board image are visually significant. Therefore, the present invention obtains a saliency map of the PCB circuit board image, obtains a detail preservation coefficient based on the saliency map, and avoids the loss of details on the circuit board through the detail preservation coefficient.

[0042] A saliency map of the PCB circuit board image is obtained through a saliency detection algorithm. Each value in the saliency map is a saliency value of a pixel point. The detail preservation coefficient of each pixel point is obtained according to the saliency value of the pixel point.

[0043] In one embodiment, the saliency detection algorithm is a frequency tuned saliency region detection algorithm.

[0044] In another embodiment, the saliency detection algorithm is a spectral residual saliency detection algorithm.

[0045] Specifically, the detail preservation coefficient satisfies the relationship:

[0046] Where, The first The detail preservation coefficient of each pixel, The first The saliency value of each pixel, It is an adjustment parameter used to control the degree of influence of saliency on detail preservation. In this example, 1.5, the implementer can choose according to the actual situation The value of .

[0047] in, The larger the The more visually significant the area to which the pixel belongs, the The more likely a pixel is in an area such as a solder joint or component, the smaller the detail preservation coefficient should be used to enhance the details of this part of the PCB image and avoid false detection in the subsequent PCB production monitoring process. This will result in a stronger correction for the reflection component in the subsequent calculation process. The smaller the The less visually significant the area to which the pixel belongs, the The more likely the pixel is in a simple structure area such as the solder mask area, the better. In order to maintain the lighting correction effect in this area, the first pixel should be The larger the detail preservation coefficient of the pixels, the less likely it is to change too much.

[0048] S5. Obtain a reflection component of the PCB circuit board image, and correct the reflection component using a detail preservation coefficient to obtain a corrected reflection component.

[0049] Specifically, logarithmic subtraction is performed on the PCB circuit board image and the initial illumination component map using the existing technology of the Retinex algorithm to obtain a reflection component of the PCB circuit board image. The reflection component is corrected using a detail preservation coefficient to obtain a corrected reflection component, wherein each value in the reflection component is a reflection component value, and each value in the corrected reflection component is a corrected reflection component value.

[0050] Specifically, the corrected reflection component value satisfies the relationship:

[0051] Where, For the image The corrected reflection component value of each pixel, For the image The reflection component value of each pixel, For the image The detail preservation coefficient of each pixel.

[0052] Among them, the detail preservation coefficient The bigger, The closer it is to 1, the less need there is to correct the The reflection component value of the pixel point is The smaller the correction strength of the reflection component value of each pixel, the smaller the correction strength of the The corrected reflection component value of each pixel is close to the reflection component value; the detail preservation coefficient The smaller it is, the more it needs to be corrected. The reflection component value of the pixel point is The greater the correction strength of the reflection component value of each pixel, the greater the correction strength of the The correction reflection component value of each pixel point is larger than the reflection component value, thereby achieving enhancement and reservation of details of key areas such as welding points.

[0053] S6, obtaining a reconstruction image by correcting the reflection component, and monitoring PCB circuit board production by using the reconstruction image.

[0054] It should be noted that the traditional Retinex algorithm processes each channel of the image, removes the illumination component of each channel of the image respectively, obtains the reflection component of each channel, and recombines the reflection component of each channel to achieve image enhancement. However, such image enhancement may cause the proportional relationship between the pixel points of the RGB three channels of the image to deviate, resulting in obvious color deviation of the enhanced image. In order to avoid this situation, the present application reconstructs the image in the HSV color space by correcting the reflection component, obtains a reconstruction image, and monitors PCB circuit board production by using the reconstruction image.

[0055] Specifically, all values in the corrected reflection component are mapped to The image composed of the mapped corrected reflection component of all pixel points is taken as a second brightness map, the hue map, the saturation map and the second brightness map are recombined into an image in the HSV color space, and the combined image is converted into a PCB circuit board image in the RGB color space to obtain a reconstruction image, and the PCB circuit board production is monitored by using the reconstruction image.

[0056] It should be further noted that the corrected reflection component contains the inherent structural characteristics of the PCB circuit board, and the interference of the illumination component is excluded. Therefore, the second brightness map is obtained by using the corrected reflection component, and the reconstruction image is obtained by using the second brightness map, so that only the real structure of the PCB circuit board is expressed in the reconstruction image, and the illumination estimation error is not brought back to the reconstruction image. At the same time, only the brightness map is modified in the process of obtaining the reconstruction image, and the hue map and the saturation map remain unchanged, which reduces the color deviation in the reconstruction image and makes the PCB circuit board production monitoring by using the reconstruction image more reliable.

[0057] Further, the PCB circuit board production monitoring by using the reconstruction image comprises: obtaining a standard image of the PCB circuit board image, collecting an image of a PCB circuit board to be detected, obtaining a reconstruction image of the image of the PCB circuit board to be detected, using a template matching algorithm to obtain the similarity between the standard image and the reconstruction image, when the similarity is greater than or equal to a threshold value, the PCB circuit board production process is normal, and when the similarity is less than the threshold value, the PCB circuit board production process is abnormal.

[0058] Exemplarily, the threshold is obtained in the following manner: obtaining a reconstructed image of the standard image, obtaining a similarity between the standard image and the reconstructed image by a template matching algorithm, and taking the similarity as the threshold.

[0059] Exemplarily, Figure 2 For the PCB image in the application, Figure 3 For the reconstructed image in the application, Figure 2 The upper right part of the image has a higher light intensity relative to other parts, and the light intensity of each position of the circuit board in the image is more uniform.

Claims

1. An intelligent monitoring method for PCB circuit board production based on machine vision, characterized in that: include: Convert the circuit board image from RGB color space to HSV color space to obtain hue map, saturation map and brightness map; The structural homogeneity of the pixel is obtained based on the difference in the gradient direction between the pixel and its surrounding pixels. The filter window size used for each pixel is obtained based on the structural homogeneity of the pixel, and the brightness image is filtered using a window of the pixel filter size as the filter window to obtain an initial illumination component image. The circuit board image and the initial illumination component map are logarithmically subtracted using the Retinex algorithm to obtain the reflection component of the circuit board image. A saliency map of the circuit board image is obtained using a saliency detection algorithm, and a detail preservation coefficient is obtained based on the saliency map. The reflection component is corrected using the detail preservation coefficient to obtain a corrected reflection component. Combining the hue map, the saturation map, and the second brightness map into an HSV color space image, and converting the HSV color space image into an RGB color space to obtain a reconstructed image, wherein the second brightness map is an image composed of normalized values ​​of corrected reflection components of all pixels; Circuit board production monitoring by reconstructing images.

2. The intelligent monitoring method for PCB circuit board production based on machine vision according to claim 1, characterized in that: The structural homogeneity satisfies the relationship: ； Where, The first The structural homogeneity of pixels, The first The first pixel The gradient direction features of the neighboring pixels, The first The neighborhood gradient direction feature of pixels, is the side length of the pixel neighborhood, is the cosine similarity function.

3. The intelligent monitoring method for PCB circuit board production based on machine vision according to claim 1, characterized in that: The filter window size satisfies the relationship: ； Where, The first The filter window size used for pixels, is the minimum filtering window, is the maximum filter window, The first The structural homogeneity of pixels, is an odd transfer function, where , Example input for the odd conversion function.

4. The intelligent monitoring method for PCB circuit board production based on machine vision according to claim 1, characterized in that: The filtering of the brightness image includes: performing Gaussian filtering on the brightness image.

5. The intelligent monitoring method for PCB circuit board production based on machine vision according to claim 1, characterized in that: The saliency detection algorithm is a frequency-tuned saliency region detection algorithm.

6. The intelligent monitoring method for PCB circuit board production based on machine vision according to claim 1, characterized in that: The saliency detection algorithm is a spectral residual saliency detection algorithm.

7. The intelligent monitoring method for PCB circuit board production based on machine vision according to claim 1, characterized in that: The detail preservation coefficient satisfies the relationship: ； Where, The first The detail preservation coefficient of each pixel, The first The saliency value of each pixel, It is an adjustment parameter used to control the influence of saliency on detail preservation.

8. The intelligent monitoring method for PCB circuit board production based on machine vision according to claim 1, characterized in that: The corrected reflection component satisfies the relationship: ； Where, The first The corrected reflection component value of each pixel, The first The reflection component value of each pixel, The first The detail preservation coefficient of each pixel.

9. The intelligent monitoring method for PCB circuit board production based on machine vision according to claim 1, characterized in that: The circuit board production monitoring by reconstructing images includes: obtaining a standard image of a PCB circuit board, collecting an image of a PCB circuit board to be inspected, and obtaining a reconstructed image of the PCB circuit board image to be inspected; using a template matching algorithm to obtain the similarity between the standard image and the reconstructed image; when the similarity is greater than or equal to a threshold, the PCB circuit board production process is normal; when the similarity is less than the threshold, the PCB circuit board production process is abnormal.

10. The intelligent monitoring method for PCB circuit board production based on machine vision according to claim 9, characterized in that: The method for obtaining the threshold value includes: obtaining a reconstructed image of a standard image, obtaining the similarity between the standard image and the reconstructed image through a template matching algorithm, and using the similarity as the threshold value.

Images