Method and system for detecting edge deviation of appearance based on welding component and PCB (Printed Circuit Board)

Abstract

The invention relates to the technical field of PCB (Printed Circuit Board) detection, in particular to a method and a system for detecting the edge deviation of the appearance of a PCB based on a welding component. The polarization characteristic pattern is generated by acquiring the multi-spectral original image and the polarization state parameter of the to-be-detected area of the PCB, the reflection exposure abnormal area and the reflection exposure normal area can be effectively positioned, the fusion weight is automatically adjusted according to the spectral response characteristic of the signal area through the self-adaptive spectrum fusion enhancement technology, and the detection accuracy is improved. The problem that in a traditional method, image enhancement is not fine enough is solved, the adaptive fusion enhancement technology enables the multispectral image to better conform to the actual situation during enhancement, more detail information is reserved, key features can be accurately extracted by generating a semantic segmentation map and combining three-dimensional standard coordinate information, and the image enhancement accuracy is improved. The detection precision and reliability of the welding component and the appearance edge of the PCB are effectively improved, the deviation can be accurately positioned and quantified, and the quality control level in the production process of the PCB is improved.

Description

Technical Field

[0001] This invention relates to the field of PCB board inspection technology, and in particular to a method and system for detecting edge deviations of soldered components and PCB board shapes. Background Technology

[0002] The deviation of the soldered components and the PCB board's outer edge refers to the deviation between the component mounting position and the predetermined position on the PCB board, as well as the deviation between the outer edge of the PCB board itself and the design dimensions, caused by factors such as manufacturing process limitations, equipment errors, and material inhomogeneity during the PCB board design and soldering process.

[0003] Current technologies primarily involve using machine vision systems to acquire images of PCB boards via image acquisition devices. Image processing software is then used to analyze the images, extracting key information (such as component pin positions and PCB edge shapes) and comparing it with design data to determine if deviations exist. However, when using machine vision to inspect soldered components and PCB edge deviations, issues arise. The copper plating on the PCB surface and the metallic luster of solder joints can easily generate reflections. Furthermore, shadows from components or the texture of the PCB can cause localized overexposure / underexposure, resulting in blurred key features in the image that cannot be accurately extracted. Summary of the Invention

[0004] The main objective of this invention is to provide a method for detecting edge deviations of welded components and PCB boards, aiming to solve the technical problems in the prior art.

[0005] This invention proposes a method for detecting edge deviations in the shape of soldered components and PCB boards, comprising: The multispectral raw image and polarization state parameters of the area to be tested on the PCB board are acquired to generate a polarization feature map, and the abnormal and normal areas of reflective exposure are located based on the polarization feature map. The spectral response characteristics of the reflective exposure abnormal area and the image detail features of the normal area are obtained to determine the adaptive spectral fusion weights, and the multispectral original image is enhanced by adaptive spectral fusion according to the adaptive spectral fusion weights to generate a fused enhanced image; Obtain the semantic features and pixel distribution patterns of the fused and enhanced image, and generate a semantic segmentation map based on the semantic features and pixel distribution patterns; Key feature coordinate information is obtained from the semantic segmentation map, three-dimensional standard coordinate information in the PCB board design data is obtained, and three-dimensional deviation value is obtained from the key feature coordinate information and the three-dimensional standard coordinate information. Determine whether the three-dimensional deviation value exceeds a preset deviation threshold; If the three-dimensional deviation value does not exceed the preset deviation threshold, then it is determined that there is no deviation; If the three-dimensional deviation value exceeds the preset deviation threshold, a deviation is determined to exist, and the deviation location, deviation type, and deviation quantification result are output.

[0006] This application also provides a system for detecting edge deviations in the shape of soldered components and PCB boards, including: The positioning module is used to acquire the multispectral raw image and polarization state parameters of the area to be inspected on the PCB board to generate a polarization feature map, and to locate the abnormal and normal areas of reflective exposure based on the polarization feature map. The first acquisition module is used to acquire the spectral response characteristics of the reflective exposure abnormal area and the image detail features of the normal area to determine the adaptive spectral fusion weight, and to perform adaptive spectral fusion enhancement on the multispectral original image according to the adaptive spectral fusion weight to generate a fused enhanced image. The generation module is used to obtain the semantic features and pixel distribution patterns of the fused and enhanced image, and generate a semantic segmentation map based on the semantic features and pixel distribution patterns; The second acquisition module is used to acquire key feature coordinate information based on the semantic segmentation map, acquire three-dimensional standard coordinate information in the PCB board design data, and acquire three-dimensional deviation value based on the key feature coordinate information and the three-dimensional standard coordinate information. The judgment module is used to determine whether the three-dimensional deviation value exceeds a preset deviation threshold. If the three-dimensional deviation value does not exceed the preset deviation threshold, then it is determined that there is no deviation; If the three-dimensional deviation value exceeds the preset deviation threshold, a deviation is determined to exist, and the deviation location, deviation type, and deviation quantification result are output.

[0007] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method for detecting edge deviations of soldered components and PCB boards.

[0008] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described method for detecting edge deviations of soldered components and PCB boards.

[0009] The beneficial effects of this invention are as follows: By acquiring the multispectral original image and polarization state parameters of the area to be detected on the PCB board, this invention generates a polarization feature map, which can effectively locate abnormal and normal areas of reflective exposure. This not only reduces image distortion caused by metallic luster and surface reflection, but also accurately captures the spectral response characteristics of abnormal areas. Through adaptive spectral fusion enhancement technology, the fusion weights are automatically adjusted according to the spectral response characteristics of the signal area, solving the problem of insufficient image enhancement in traditional methods. The adaptive fusion enhancement technology makes the multispectral image enhancement more consistent with the actual situation, preserving more detailed information. This is achieved by generating a semantic segmentation map and combining it with… The three-dimensional standard coordinate information can accurately extract key features and calculate three-dimensional deviation values, providing more accurate spatial deviation detection. It avoids misjudgment caused by poor image quality, effectively improving the detection accuracy and reliability of soldered components and PCB board outline edges. It can accurately locate and quantify deviations, improving the quality control level in the PCB board production process. By combining multispectral and polarization features, spectral fusion enhancement technology, and three-dimensional deviation analysis, it overcomes many limitations of traditional image processing methods, significantly improving the accuracy and robustness of PCB board soldered components and outline edge deviation detection, and has significant technical advantages and application prospects. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of a method flow according to an embodiment of the present invention.

[0011] Figure 2 This is a schematic diagram of the system structure according to an embodiment of the present invention.

[0012] Figure 3 This is a schematic diagram of the internal structure of a computer device according to an embodiment of this application.

[0013] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0014] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0015] like Figure 1 As shown, this application provides a method for detecting edge deviations in the shape of soldered components and PCB boards, including: S1. Obtain the multispectral original image and polarization state parameters of the area to be tested on the PCB board to generate a polarization feature map, and locate the abnormal and normal areas of reflective exposure based on the polarization feature map; S2. Obtain the spectral response characteristics of the abnormal reflective exposure area and the image detail features of the normal area to determine the adaptive spectral fusion weight, and perform adaptive spectral fusion enhancement on the multispectral original image according to the adaptive spectral fusion weight to generate a fused enhanced image; S3. Obtain the semantic features and pixel distribution patterns of the fused and enhanced image, and generate a semantic segmentation map based on the semantic features and pixel distribution patterns; S4. Obtain key feature coordinate information based on the semantic segmentation map, obtain three-dimensional standard coordinate information in the PCB board design data, and obtain three-dimensional deviation value based on the key feature coordinate information and the three-dimensional standard coordinate information; S5. Determine whether the three-dimensional deviation value exceeds a preset deviation threshold; If the three-dimensional deviation value does not exceed the preset deviation threshold, then it is determined that there is no deviation; If the three-dimensional deviation value exceeds the preset deviation threshold, a deviation is determined to exist, and the deviation location, deviation type, and deviation quantification result are output.

[0016] As described in steps S1-S5 above, the steps following determining the existence of a deviation and outputting the deviation location, deviation type, and deviation quantification result include: obtaining the deviation impact range based on the deviation location and three-dimensional deviation value; retrieving the corresponding preset deviation correction rule library based on the deviation type; extracting suitable correction parameters and operation priorities; classifying emergency correction levels based on the deviation impact range and operation priorities; generating a targeted deviation correction plan based on the correction parameters; synchronizing the correction plan to the detection system; tracking the correction execution progress in real time; acquiring the re-inspection multispectral image and key feature coordinates of the PCB board after correction; calculating the corrected deviation value based on the re-inspection coordinates and three-dimensional standard coordinates; determining whether the corrected deviation value meets the preset qualification standard; if it meets the qualification standard, generating a deviation detection and correction report for archiving; if it does not meet the standard, re-optimizing the correction parameters and iteratively executing the correction process until the deviation meets the standard, thus achieving closed-loop control of detection-correction-re-inspection.

[0017] Existing technologies primarily rely on single-spectrum image acquisition equipment, which is easily affected by surface reflection and lighting conditions. This invention, however, combines multispectral images with polarization state parameters, not only acquiring more information but also effectively enhancing image depth and detail. By extracting reflection characteristics from the image using polarization state parameters, overexposure / underexposure problems caused by reflected light sources can be significantly reduced. Polarization feature maps locate abnormally exposed reflective areas, allowing for targeted processing of these areas and preventing the introduction of noise or errors in subsequent image processing. Adaptive spectral fusion weights automatically adjust the fusion process based on the spectral response characteristics of reflective areas and the detail features of normal areas, maximizing information extraction efficiency from different regions. Since the spectral response characteristics of reflective areas may differ significantly from the image details of normal areas, fixed weights cannot effectively optimize the image. Adaptive adjustment optimizes the fusion process based on the spectral characteristics of different regions, improving both global and local detail representation. Spectral fusion enhancement technology effectively fuses multiple spectral information, improving image performance at various spectral levels and reducing the negative impact of reflection or overexposure areas on image quality.

[0018] Current image processing technologies typically rely on traditional edge detection and template matching methods. These methods often fail to effectively identify varied features in complex backgrounds. However, extracting semantic features and pixel distribution patterns allows for a comprehensive understanding of image content from both the overall structure and local features, leading to more accurate extraction of key information. Semantic segmentation effectively distinguishes different regions in an image, such as soldered components or PCB board edges, avoiding misjudgments or omissions that may occur with traditional image processing methods. Through semantic segmentation, the system can automatically identify and label all key feature regions, improving the accuracy of feature extraction. By combining these two approaches, the semantic segmentation map better reflects the actual structure of the image, avoiding the loss of detail that may result from relying solely on pixel information. Furthermore, by combining three-dimensional design data... By combining data with image features, the position and deviation of each key feature can be calculated more accurately. By comparing the key feature coordinates obtained through semantic segmentation with the three-dimensional standard coordinates, more precise deviation values ​​can be obtained. Taking into account the three-dimensional structure of the PCB board, it can accurately reflect the deviation between the soldered components and the PCB edge. By comparing the three-dimensional coordinates, more comprehensive and accurate deviation analysis results can be provided, which helps to more precise process control and defect correction. By setting a preset deviation threshold and combining it with actual detection results, it can automatically determine whether there is a serious deviation and perform precise positioning and quantification. Once a situation exceeding the deviation threshold is detected, the system can immediately output the deviation position, type and quantification results, reducing the error of human judgment and improving detection efficiency and accuracy.

[0019] In one embodiment, step S1, which involves acquiring the multispectral raw image and polarization state parameters of the area to be inspected on the PCB board to generate a polarization feature map, and locating the abnormal and normal areas of reflective exposure based on the polarization feature map, includes: S11. Obtain image data of multiple bands based on the original multispectral image, and obtain corresponding light intensity data based on each image data; S12. Obtain the degree of polarization for each band based on the polarization state parameters, and obtain the corresponding polarization characteristic coefficient based on the light intensity data and degree of polarization for each band. S13. Each polarization feature coefficient is fused with the corresponding light intensity data at the pixel level to generate the corresponding band polarization feature component. S14. Obtain the corresponding spectral response overlap and detail complementarity coefficient according to each polarization feature component of the band, and determine the corresponding basic weight according to each spectral response overlap. S15. Adjust the corresponding basic weights according to each of the detailed complementarity coefficients to obtain the corresponding splicing weights, and perform pixel-level fusion of multiple band polarization feature components according to the splicing weights to obtain a polarization feature map. S16. Obtain the polarization degree threshold and gray value range of each pixel in the polarization feature map, and determine whether the polarization degree of each pixel exceeds the polarization degree threshold. If the polarization degree of a pixel exceeds the polarization degree threshold, then the area where the pixel is located is marked as a reflective area; S17. Analyze the relationship between the gray value and gray value range of each pixel; If the gray value of a pixel is greater than the upper limit of the gray value range, then the area where the pixel is located is marked as an overexposed area. If the gray value of a pixel is less than the lower limit of the gray value range, then the area where the pixel is located is marked as an underexposed area. S18. Integrate all reflective areas, overexposed areas, and underexposed areas to obtain reflective exposure abnormal areas, and determine the parts of the polarization feature map that are not marked as reflective exposure abnormal areas as normal areas.

[0020] As described in steps S11-S18 above, the polarization characteristic coefficient is obtained by acquiring the standard deviation and mean of multiple light intensity data, and then calculating it using the degree of polarization, the standard deviation of the light intensity data, and the mean of the light intensity data. The calculation formula is as follows: Where A represents the polarization characteristic coefficient, B represents the degree of polarization, C represents the standard deviation of light intensity data, and D represents the mean of light intensity data. The band polarization feature components are generated by pixel-level fusion through pixel-by-pixel multiplication of the light intensity data of each band with the corresponding polarization feature coefficient. The spectral response overlap is obtained by the cosine similarity of any two band polarization feature components. The sum of the detail complementarity coefficient and the spectral response overlap is 1. The formula for calculating the basic weight is as follows: Among them, E i F represents the i-th basic weight. i F represents the overlap of the i-th spectral response. n The formula above represents the overlap of the nth spectral response, where N represents the number of spectral response overlaps, and i and n represent the sequence numbers of the spectral response overlaps. The formula above refers to the ratio between the difference between 1 and the corresponding spectral response overlap for each basic weight, and the sum of the differences between 1 and the spectral response overlap for all bands.

[0021] This invention utilizes multispectral image acquisition to obtain more image information across different wavelengths. By extracting light intensity data, it can analyze image brightness variations, particularly in cases of reflection or exposure issues, enabling more effective identification of problem areas. Furthermore, by using polarization degree data across different wavelengths, it can effectively distinguish reflective areas caused by reflection and metallic luster. Compared to traditional techniques, polarization degree can filter out false signals caused by luster and reflection, improving the accuracy of extracting key information from the image. Combining light intensity data and polarization degree allows for the effective acquisition of polarization characteristic coefficients, enabling more precise detection of complex surface reflections or localized exposure problems. This surpasses the limitations of traditional methods that rely solely on brightness, allowing for more accurate analysis of various problem areas in images and reducing misjudgments caused by overexposure, underexposure, and reflection issues inherent in traditional image processing methods. By fusing polarization feature coefficients and light intensity data at the pixel level, the information of each pixel can be more accurately combined, thereby obtaining higher quality image features. By processing each band, the generation of polarization feature components can enhance the discrimination ability of reflective areas in the image, enabling the system to more clearly distinguish between reflective and overexposed areas in visual inspection, improving image quality and avoiding information loss. By calculating spectral response overlap and detail complementarity coefficient, image information of different bands can be comprehensively evaluated. The complementary information of multiple bands can enhance the details in the image, enabling image processing to avoid information loss caused by a single band and improving the overall image analysis effect. The basic weight is determined by the spectral response overlap, effectively combining the advantages of each band, so that the analysis of the entire image can achieve higher accuracy.

[0022] By adjusting the basic weights through detail complementarity coefficients and ultimately obtaining the stitching weights, the complementary information of each band can be maximized, effectively reducing information redundancy and enhancing key image features. Adjusting the stitching weights achieves precise fusion of information between different bands, effectively reducing the bias of single-band features and making the polarization feature map more representative. This ensures that the image is not interfered with by reflections or exposure issues during detection, improving detection accuracy. By setting a polarization degree threshold, areas with polarization degrees exceeding the normal range (such as reflective areas) can be effectively identified, and false signals caused by metal surface reflections can be removed. The polarization degree threshold judgment can accurately locate reflective areas, avoiding the influence of surface reflections or solder joint reflections on the detection results. Compared with traditional techniques, this reduces the possibility of false positives and false negatives. Through analysis... The relationship between the grayscale value of each pixel and the preset grayscale value range can effectively identify overexposed or underexposed areas, enabling more accurate extraction of key information from the image under different brightness conditions. This avoids visual errors caused by overexposure and underexposure, thereby improving the image's adaptability under complex lighting conditions and improving image quality issues caused by reflections and shadows. By integrating reflective, overexposed, and underexposed areas, it not only improves the detection accuracy of reflection and exposure problems but also ensures accurate identification of normal areas. This better avoids situations where individual problematic areas affect the overall image, ensuring more accurate final image analysis results. It also ensures accurate location and processing of all types of exposure anomalies (including reflections, overexposure, and underexposure), improving the reliability and accuracy of the entire image analysis process.

[0023] In one embodiment, step S2, which involves acquiring the spectral response characteristics of the reflective exposure aberration region and the image detail features of the normal region to determine the adaptive spectral fusion weights, includes: S21. Obtain the spectral absorption coefficient and reflectance coefficient of each band according to the spectral response characteristics of the reflective exposure abnormal region, and obtain the abnormal suppression weight of the corresponding band according to each spectral absorption coefficient and reflectance coefficient. S22. Obtain the edge sharpness, texture complexity, and contrast parameters of each band based on the image detail features of the normal region, and obtain the detail retention weight of the corresponding band based on each edge sharpness, texture complexity, and contrast parameter. S23. The weighted sum of the retention weight and anomaly suppression weight of each detail is used to obtain the adaptive spectral fusion weight of the corresponding band.

[0024] As described in steps S21-S23 above, the anomaly suppression weight is obtained by weighted summation of spectral absorption coefficient and reflectance coefficient to obtain a comprehensive value, and then by subtracting the comprehensive value from 1. The detail preservation weight is the average value after weighted summation of edge sharpness, texture complexity and contrast parameters. This invention, by acquiring the spectral response characteristics of reflective exposure anomalies, can accurately identify and quantify the absorption and reflection coefficients of different bands in reflective regions, significantly improving the accuracy and precision of reflective region analysis. By calculating the reflection and absorption coefficients, the reflection characteristics of each band can be derived, effectively identifying reflective regions. By calculating anomaly suppression weights, these regions are effectively suppressed, eliminating reflective interference and improving image analysis quality. By extracting detail retention weights for each band based on image detail features of normal regions (such as edge sharpness, texture complexity, and contrast), more accurate preservation of detail information in normal regions can be achieved. By combining three key parameters—edge sharpness, texture complexity, and contrast—detail information is extracted from multiple dimensions, effectively enhancing the richness of image details. The detail retention weight for each band is determined based on different image characteristics. The method is derived from the calculation of features, thus possessing adaptive capabilities. Under different lighting and reflection conditions, it can flexibly adjust the degree of image detail preservation, thereby improving the adaptability and stability of image processing. By weighted summation of detail preservation weights and anomaly suppression weights, an adaptive spectral fusion weight is generated, which allows the weights to be dynamically adjusted according to the needs of different regions in the image, thereby achieving more refined image optimization and avoiding the limitations of traditional methods. Through weighted summation, it can suppress interference from reflective areas while preserving image details, enabling each band of the image to retain effective information to the maximum extent while reducing the influence of irrelevant information. By introducing a dynamic adjustment mechanism for fusion weights, the weights can be adaptively adjusted according to the characteristics of different bands, thereby adapting to different types of PCB boards and different types of reflection and exposure problems, ensuring efficient and high-quality image processing results.

[0025] In one embodiment, step S2, which involves adaptively fusing and enhancing the multispectral original image according to the adaptive spectral fusion weights to generate a fused and enhanced image, includes: S24. Divide the original multispectral image into multiple single-band images according to the acquisition band channel, and obtain the pixel matrix of each single-band image; S25. Pixel-level weighted values ​​are assigned to the pixel matrix of the corresponding single-band image according to each of the adaptive spectral fusion weights to obtain the corresponding weighted pixel matrix; S26. A multi-scale fusion algorithm is used to decompose each weighted pixel matrix into multiple scales to obtain the corresponding low-frequency components and high-frequency components. Adaptive brightness equalization processing is then performed on each of the low-frequency components to obtain the corresponding optimized low-frequency components. S27. Perform gradient enhancement processing on each of the high-frequency components to obtain the corresponding optimized high-frequency components, and reconstruct all optimized low-frequency components and optimized high-frequency components to obtain a fused enhanced image.

[0026] As described in steps S24-S27 above, the pixel-level weighting is obtained by multiplying the elements of the single-band image pixel matrix with the corresponding adaptive spectral fusion weights element by element to obtain the weighted pixel matrix. The Laplacian pyramid algorithm is used for multi-scale decomposition. The low-frequency component is the result of the weighted pixel matrix after Gaussian filtering, and the high-frequency component is the weighted pixel matrix minus the low-frequency component. This invention divides multispectral images into multiple single-band images according to the acquisition band channels. Each band captures different spectral information, enabling more accurate feature extraction. It allows for more independent analysis and optimization across different bands, making subsequent image enhancement and analysis processes more targeted and flexible. Through adaptive spectral fusion weighting, different weight values ​​are assigned based on the characteristics of each single-band image. This allows the weights to adjust automatically according to the specific image quality or requirements, avoiding the fixed weight problem of traditional methods. The weighted assignment prioritizes high-clarity, information-rich bands based on their quality, thus reducing the influence of bands with greater interference. Multi-scale decomposition breaks down the image into different levels of detail, preserving low-frequency information (such as overall shape and contour) while enhancing high-frequency information (such as edges and textures), ensuring that features at different levels are fully utilized. Effective enhancement provides a richer information source for subsequent image processing and feature extraction. By adaptively equalizing the low-frequency components, uneven brightness areas can be effectively adjusted, improving the overall visual effect of the image and ensuring the clarity of key information in the image. By performing gradient enhancement on the high-frequency components, details in the image, such as texture and edges, can be enhanced, making the image more prominent in local details while maintaining global consistency. Gradient enhancement can enhance the sharpness and clarity of the image and reduce the impact of these problems, especially in areas with rich details (such as component pins and edges), where they can be presented more clearly. Reconstructing the optimized low-frequency and high-frequency components can better preserve the global and local information of the image, so that the final fused and enhanced image has both high-quality low-frequency background information and enhanced detail.

[0027] In one embodiment, step S3, which generates a semantic segmentation map based on the semantic features and pixel distribution patterns, includes: S31. Obtain the shape topology features, material spectral features, and straightness features of the PCB board edge of the welding components based on the semantic features; S32. Obtain pixel grayscale cluster centers and neighborhood correlation parameters according to the pixel distribution pattern, and construct an initial region segmentation template according to the shape topology features and pixel grayscale cluster centers; S33. Optimize the boundary of the initial region segmentation template based on the material's spectral characteristics, linearity characteristics, and neighborhood correlation parameters to obtain the optimized segmentation boundary; S34. Input the initial segmentation template and optimized boundary of the region into the improved fully convolutional neural network, and fuse shallow texture features and deep semantic features through skip connections to obtain the network output feature map; S35. The network output feature map is binarized using an adaptive threshold segmentation algorithm, and the areas of soldered components, PCB board edges, and background are marked. Holes in the areas are filled using morphological closing operations to generate a semantic segmentation map.

[0028] As described in steps S31-S35 above, the step of constructing the initial region segmentation template includes obtaining the contour topology and connected region parameters based on shape topology features; obtaining the target grayscale range and background grayscale range based on pixel grayscale cluster centers; determining the region morphological constraints based on the contour topology; setting the minimum region area threshold based on the connected region parameters; marking connected regions within the target grayscale range that meet the morphological constraints and have an area greater than the minimum threshold as candidate target regions; and preliminarily delineating the boundaries of the candidate target regions based on the spatial distribution of grayscale cluster centers to generate the initial region segmentation template.

[0029] The steps for generating a semantic segmentation map using an adaptive threshold segmentation algorithm and morphological closing operation include: obtaining the category response value of each pixel based on the network output feature map; determining the adaptive segmentation threshold interval based on the statistical distribution of the category response values, wherein the threshold values ​​for the soldering component region, the PCB board edge region, and the background region do not overlap; determining the threshold interval to which the category response value of each pixel belongs, and marking it as the corresponding target region pixel and background region pixel respectively, and completing the binarization process; constructing elliptical structural elements, and using morphological closing operation to traverse the binarized image to fill the small holes in the target region; removing isolated noise points and redundant regions with areas smaller than a preset threshold in the image after the closing operation, correcting discontinuities at the region boundaries, and obtaining the semantic segmentation map.

[0030] This invention analyzes the geometry of welded components, enabling accurate positioning based on their unique topological relationships. It maintains high accuracy even under conditions of reflection or overexposure. The spectral characteristics of different materials help the system effectively distinguish between metallic and non-metallic regions, reducing the impact of surface reflection on image analysis. The linearity of PCB board edges aids in identifying and locating PCB board boundaries, maintaining high accuracy even under deformation and irregular conditions. By extracting pixel grayscale cluster centers and neighborhood correlation parameters, the segmentation process more accurately reflects the true physical characteristics. Cluster analysis of pixel grayscale effectively identifies boundaries and transition zones of different materials. Combining shape topological features with pixel distribution patterns, an initial region segmentation template is constructed, effectively overcoming errors caused by noise, illumination variations, and other factors in traditional image segmentation methods. This makes the segmentation more aligned with practical needs. The different spectral characteristics of materials facilitate more accurate boundary identification of different objects, avoiding… Missegmentation, such as distinguishing metal solder joints from the edges of the PCB board itself, can be addressed by optimizing the segmentation boundary through linear features. This ensures that the straight edges of the PCB board are not missegmented into other regions, greatly improving the accuracy of boundary detection. Using skip connections in FCN can fuse shallow texture features with deep semantic features, enabling the network to capture both detailed information and understand global semantics. Compared to traditional neural networks, skip connections significantly improve the fine-grained accuracy of segmentation in image segmentation tasks. Fully convolutional networks can process input images of different sizes while reducing the number of parameters and improving computational efficiency. By employing an adaptive threshold segmentation algorithm on the feature map output by the network, the segmentation threshold can be automatically selected more accurately based on the actual grayscale distribution of the image, avoiding the poor adaptability of fixed threshold segmentation methods to different images. Morphological closing operations are used to fill in region holes, further correcting the segmentation results, eliminating noise or small holes, and ensuring accurate labeling of soldered components and PCB board areas.

[0031] In one embodiment, step S4, which involves obtaining the three-dimensional deviation value based on the key feature coordinate information and the three-dimensional standard coordinate information, includes: S41. Based on the key feature coordinate information and the three-dimensional standard coordinate information, obtain the actual three-dimensional coordinate set and the standard three-dimensional coordinate set respectively, and perform homologous registration on the actual three-dimensional coordinate set and the standard three-dimensional coordinate set to establish a one-to-one corresponding feature point pair; S42. Obtain the actual three-dimensional coordinates and standard three-dimensional coordinates of each of the feature point pairs, and obtain the X-axis deviation value, Y-axis deviation value and Z-axis deviation value based on each of the actual three-dimensional coordinates and standard three-dimensional coordinates; S43. Obtain the corresponding X-axis deviation weight, Y-axis deviation weight and Z-axis deviation weight according to each X-axis deviation value, Y-axis deviation value and Z-axis deviation value, and perform a weighted summation according to each X-axis deviation value, Y-axis deviation value, Z-axis deviation value, X-axis deviation weight, Y-axis deviation weight and Z-axis deviation weight to obtain the comprehensive deviation value of the corresponding feature point pair; S44. Obtain the mean deviation value based on the multiple comprehensive deviation values, and obtain the three-dimensional deviation value based on the mean deviation value and the preset deviation influence factor.

[0032] As described in steps S41-S44 above, before establishing a one-to-one correspondence of feature point pairs, it is necessary to unify the coordinates of the actual acquired key feature coordinates and the three-dimensional standard coordinates, and transform the two into the same three-dimensional coordinate system. The X-axis weight is set according to the tolerance of the PCB board length direction, the Y-axis weight is set according to the tolerance of the width direction, and the Z-axis weight is set according to the tolerance of the soldering height. By employing a registration process between the actual 3D coordinate set and the standard 3D coordinate set, the alignment of points from different data sources is ensured. This allows subsequent analysis to be performed based on a unified coordinate system, eliminating errors caused by different data acquisition methods. By acquiring 3D coordinates and directly calculating deviation values, this invention can comprehensively and accurately capture the positional errors of PCB components. It provides a more precise measurement, especially for deep errors that cannot be identified through 2D images (such as component positional deviations and PCB warping), offering significant advantages. The weighted summation of deviations can be personalized based on errors on different coordinate axes, reflecting the contribution of errors in each direction to the overall deviation. By assigning different weights to each direction, the system can more accurately reflect the impact of errors in each direction on the total deviation. By calculating the mean and adjusting it in conjunction with the deviation influence factor, the influence of different points can be comprehensively considered, allowing for a more scientific assessment of the overall 3D deviation. This effectively improves the stability and accuracy of the detection results, avoiding inaccurate deviation judgments of the entire product due to anomalies in a few feature points.

[0033] like Figure 2 As shown, this application also provides a system for detecting edge deviations in the shape of soldered components and PCB boards, including: The positioning module is used to acquire the multispectral raw image and polarization state parameters of the area to be inspected on the PCB board to generate a polarization feature map, and to locate the abnormal and normal areas of reflective exposure based on the polarization feature map. The first acquisition module is used to acquire the spectral response characteristics of the reflective exposure abnormal area and the image detail features of the normal area to determine the adaptive spectral fusion weight, and to perform adaptive spectral fusion enhancement on the multispectral original image according to the adaptive spectral fusion weight to generate a fused enhanced image. The generation module is used to obtain the semantic features and pixel distribution patterns of the fused and enhanced image, and generate a semantic segmentation map based on the semantic features and pixel distribution patterns; The second acquisition module is used to acquire key feature coordinate information based on the semantic segmentation map, acquire three-dimensional standard coordinate information in the PCB board design data, and acquire three-dimensional deviation value based on the key feature coordinate information and the three-dimensional standard coordinate information. The judgment module is used to determine whether the three-dimensional deviation value exceeds a preset deviation threshold. If the three-dimensional deviation value does not exceed the preset deviation threshold, then it is determined that there is no deviation; If the three-dimensional deviation value exceeds the preset deviation threshold, a deviation is determined to exist, and the deviation location, deviation type, and deviation quantification result are output.

[0034] In one embodiment, the first acquisition module includes: The first acquisition unit is used to acquire the spectral absorption coefficient and reflectance coefficient of each band according to the spectral response characteristics of the reflective exposure abnormal region, and to acquire the abnormal suppression weight of the corresponding band according to each spectral absorption coefficient and reflectance coefficient. The second acquisition unit is used to acquire the edge sharpness, texture complexity and contrast parameters of each band based on the image detail features of the normal region, and to acquire the detail retention weight of the corresponding band based on each edge sharpness, texture complexity and contrast parameter. The third acquisition unit is used to perform a weighted summation of the detail retention weight and the anomaly suppression weight for each of the aforementioned bands to obtain the adaptive spectral fusion weights for the corresponding bands.

[0035] It should be noted that each module and unit in the system for detecting edge deviations of welded components and PCB boards corresponds one-to-one with the steps in the method for detecting edge deviations of welded components and PCB boards.

[0036] like Figure 3 As shown, this application also provides a computer device, which can be a server, and its internal structure can be as follows: Figure 3As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores all data required for the process of detecting edge deviations in soldered components and PCB boards. The network interface allows communication with external terminals via a network connection. When executed by the processor, the computer program implements the method for detecting edge deviations in soldered components and PCB boards.

[0037] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment on which the present application is applied.

[0038] An embodiment of this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements any of the above-described methods for detecting edge deviations of soldered components and PCB boards.

[0039] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in this application and in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0040] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0041] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for detecting edge deviations of soldered components and PCB boards, characterized in that, include: The multispectral raw image and polarization state parameters of the area to be tested on the PCB board are acquired to generate a polarization feature map, and the abnormal and normal areas of reflective exposure are located based on the polarization feature map. The spectral response characteristics of the reflective exposure abnormal area and the image detail features of the normal area are obtained to determine the adaptive spectral fusion weights, and the multispectral original image is enhanced by adaptive spectral fusion according to the adaptive spectral fusion weights to generate a fused enhanced image. Obtain the semantic features and pixel distribution patterns of the fused and enhanced image, and generate a semantic segmentation map based on the semantic features and pixel distribution patterns; Based on the semantic segmentation map, key feature coordinate information is obtained, three-dimensional standard coordinate information in the PCB board design data is obtained, and three-dimensional deviation value is obtained based on the key feature coordinate information and the three-dimensional standard coordinate information. Determine whether the three-dimensional deviation value exceeds a preset deviation threshold; If the three-dimensional deviation value does not exceed the preset deviation threshold, then it is determined that there is no deviation; If the three-dimensional deviation value exceeds the preset deviation threshold, a deviation is determined to exist, and the deviation location, deviation type, and deviation quantification result are output.

2. The method for detecting edge deviations of welded components and PCB boards according to claim 1, characterized in that, The steps of acquiring the multispectral raw image and polarization state parameters of the area to be inspected on the PCB board to generate a polarization feature map, and locating the abnormal and normal areas of reflective exposure based on the polarization feature map, include: Image data of multiple bands are obtained from the original multispectral image, and corresponding light intensity data is obtained from each of the image data. The degree of polarization of each band is obtained based on the polarization state parameters, and the corresponding polarization characteristic coefficient is obtained based on the light intensity data and degree of polarization of each band. Each polarization feature coefficient is fused with the corresponding light intensity data at the pixel level to generate the corresponding band polarization feature component. The corresponding spectral response overlap and detail complementarity coefficient are obtained based on the polarization feature components of each band, and the corresponding basic weight is determined based on the spectral response overlap. The corresponding basic weights are adjusted according to each of the detailed complementarity coefficients to obtain the corresponding splicing weights, and the multiple polarization feature components of the bands are fused at the pixel level according to the splicing weights to obtain the polarization feature map. Obtain the polarization degree threshold and grayscale value range of each pixel in the polarization feature map, and determine whether the polarization degree of each pixel exceeds the polarization degree threshold; If the polarization degree of a pixel exceeds the polarization degree threshold, then the area where the pixel is located is marked as a reflective area; Analyze the relationship between the grayscale value and the grayscale value range of each pixel; If the gray value of a pixel is greater than the upper limit of the gray value range, then the area where the pixel is located is marked as an overexposed area. If the gray value of a pixel is less than the lower limit of the gray value range, then the area where the pixel is located is marked as an underexposed area. All reflective areas, overexposed areas, and underexposed areas are integrated to obtain reflective exposure abnormal areas, and the parts of the polarization feature map that are not marked as reflective exposure abnormal areas are determined to be normal areas.

3. The method for detecting edge deviations of welded components and PCB boards according to claim 1, characterized in that, The step of obtaining the spectral response characteristics of the reflective exposure aberration region and the image detail features of the normal region to determine the adaptive spectral fusion weights includes: The spectral absorption coefficient and reflectance coefficient of each band are obtained based on the spectral response characteristics of the reflective exposure abnormal region, and the abnormality suppression weight of the corresponding band is obtained based on each spectral absorption coefficient and reflectance coefficient. The edge sharpness, texture complexity, and contrast parameters of each band are obtained based on the image detail features of the normal region, and the detail retention weight of the corresponding band is obtained based on each edge sharpness, texture complexity, and contrast parameter. The adaptive spectral fusion weights for the corresponding bands are obtained by weighted summation of the weights for each detail retention weight and the anomaly suppression weight.

4. The method for detecting edge deviations of welded components and PCB boards according to claim 1, characterized in that, The step of performing adaptive spectral fusion enhancement on the multispectral original image according to the adaptive spectral fusion weights to generate a fused and enhanced image includes: The original multispectral image is divided into multiple single-band images according to the band, and the pixel matrix of each single-band image is obtained. The pixel matrix of the corresponding single-band image is weighted at the pixel level according to each of the adaptive spectral fusion weights to obtain the corresponding weighted pixel matrix; A multi-scale fusion algorithm is used to decompose each weighted pixel matrix into multiple scales to obtain corresponding low-frequency components and high-frequency components. Adaptive brightness equalization processing is then performed on each low-frequency component to obtain corresponding optimized low-frequency components. Gradient enhancement processing is performed on each of the high-frequency components to obtain the corresponding optimized high-frequency components. All optimized low-frequency components and optimized high-frequency components are then reconstructed to obtain a fused enhanced image.

5. The method for detecting edge deviations of welded components and PCB boards according to claim 1, characterized in that, The step of generating a semantic segmentation map based on the semantic features and pixel distribution patterns includes: Based on the semantic features, the shape topology features, material spectral features, and straightness features of the PCB board edges of the welding components are obtained; The pixel grayscale cluster center and neighborhood correlation parameters are obtained according to the pixel distribution pattern, and an initial region segmentation template is constructed according to the shape topology features and pixel grayscale cluster centers. The boundary of the initial region segmentation template is optimized based on the material's spectral characteristics, linearity characteristics, and neighborhood correlation parameters to obtain the optimized segmentation boundary; The initial segmentation template and optimized boundary of the region are input into the improved fully convolutional neural network, and shallow texture features and deep semantic features are fused through skip connections to obtain the network output feature map; An adaptive threshold segmentation algorithm is used to binarize the network output feature map, and the areas of soldered components, PCB board edges, and background are marked. The holes in the areas are filled by morphological closing operations to generate a semantic segmentation map.

6. The method for detecting edge deviations of welded components and PCB boards according to claim 1, characterized in that, The step of obtaining the three-dimensional deviation value based on the key feature coordinate information and the three-dimensional standard coordinate information includes: Based on the key feature coordinate information and the three-dimensional standard coordinate information, the actual three-dimensional coordinate set and the standard three-dimensional coordinate set are obtained respectively, and the actual three-dimensional coordinate set and the standard three-dimensional coordinate set are registered from the same source to establish a one-to-one corresponding feature point pair; Obtain the actual three-dimensional coordinates and standard three-dimensional coordinates of each feature point pair, and obtain the X-axis deviation value, Y-axis deviation value and Z-axis deviation value based on each actual three-dimensional coordinate and standard three-dimensional coordinate; Based on each X-axis deviation value, Y-axis deviation value, and Z-axis deviation value, obtain the corresponding X-axis deviation weight, Y-axis deviation weight, and Z-axis deviation weight, respectively, and perform a weighted summation based on each X-axis deviation value, Y-axis deviation value, Z-axis deviation value, X-axis deviation weight, Y-axis deviation weight, and Z-axis deviation weight to obtain the comprehensive deviation value of the corresponding feature point pair; The mean deviation is obtained based on the multiple comprehensive deviation values, and the three-dimensional deviation value is obtained based on the mean deviation and the preset deviation influence factor.

7. A system for detecting edge deviations of welded components and PCB boards, characterized in that, include: The positioning module is used to acquire the multispectral raw image and polarization state parameters of the area to be inspected on the PCB board to generate a polarization feature map, and to locate the abnormal and normal areas of reflective exposure based on the polarization feature map. The first acquisition module is used to acquire the spectral response characteristics of the reflective exposure abnormal area and the image detail features of the normal area to determine the adaptive spectral fusion weight, and to perform adaptive spectral fusion enhancement on the multispectral original image according to the adaptive spectral fusion weight to generate a fused enhanced image. The generation module is used to obtain the semantic features and pixel distribution patterns of the fused and enhanced image, and generate a semantic segmentation map based on the semantic features and pixel distribution patterns; The second acquisition module is used to acquire key feature coordinate information based on the semantic segmentation map, acquire three-dimensional standard coordinate information in the PCB board design data, and acquire three-dimensional deviation value based on the key feature coordinate information and the three-dimensional standard coordinate information. The judgment module is used to determine whether the three-dimensional deviation value exceeds a preset deviation threshold. If the three-dimensional deviation value does not exceed the preset deviation threshold, then it is determined that there is no deviation; If the three-dimensional deviation value exceeds the preset deviation threshold, a deviation is determined to exist, and the deviation location, deviation type, and deviation quantification result are output.

8. The edge deviation detection system based on welded components and PCB board outline as described in claim 7, characterized in that, The first acquisition module includes: The first acquisition unit is used to acquire the spectral absorption coefficient and reflectance coefficient of each band according to the spectral response characteristics of the reflective exposure abnormal region, and to acquire the abnormal suppression weight of the corresponding band according to each spectral absorption coefficient and reflectance coefficient. The second acquisition unit is used to acquire the edge sharpness, texture complexity and contrast parameters of each band based on the image detail features of the normal region, and to acquire the detail retention weight of the corresponding band based on each edge sharpness, texture complexity and contrast parameter. The third acquisition unit is used to perform a weighted summation of the detail retention weight and the anomaly suppression weight for each of the aforementioned bands to obtain the adaptive spectral fusion weight for the corresponding band.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images