Intelligent Detection Method and System for Surface Defects on Printed Circuit Boards Based on Dynamic Templates

Abstract

This invention belongs to the field of defect detection technology. It discloses an intelligent detection method and system for printed circuit board (PCB) surface defects based on a dynamic template. The method includes: acquiring the design file and surface image of the PCB to be inspected; performing hierarchical analysis to obtain multiple levels of information, aligning these levels to obtain standardized hierarchical data; performing feature enhancement based on the standardized hierarchical data, fusing the enhanced hierarchical data, and constructing a dynamic detection template based on this data; preprocessing the acquired surface image, and performing nonlinear registration on the preprocessed surface image using the dynamic detection template to obtain a corresponding registered image; and processing the obtained registered image to obtain defect detection and classification results. This invention ensures the accuracy of nonlinear registration by accurately establishing a local deformation model and feature point correlation matrix, effectively solving the problem of false detection caused by deformation in PCB inspection.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and more specifically, to a method and system for intelligent detection of surface defects on printed circuit boards based on dynamic templates. Background Technology

[0002] Printed circuit boards (PCBs) are core components of electronic products. As electronic devices become smaller and more integrated, the precision requirements for PCB manufacturing are constantly increasing, highlighting the growing importance of surface defect detection. Traditionally, surface defect detection on PCBs relies mainly on manual visual inspection or simple machine vision systems.

[0003] However, this method has certain limitations. On the one hand, identifying PCB surface defects requires specialized knowledge and extensive experience. The number of skilled inspectors is relatively limited, and the training period is long, making it difficult to meet the inspection needs of mass production in modern electronics manufacturing. On the other hand, as the complexity of printed circuit board structures continues to increase, circuit density grows, and defect features become more subtle, the accuracy and efficiency of manual inspection are difficult to guarantee. Although introducing computer vision and deep learning technologies can assist the inspection process to some extent, the nonlinear deformation problems that often occur during PCB manufacturing and use severely affect the accuracy of inspection results. Existing technologies struggle to effectively establish a precise correspondence between printed circuit board design documents and actual surface images, making it impossible to accurately distinguish between deformed areas and real defects, resulting in a high false detection rate and hindering defect detection and classification using deep learning models.

[0004] In view of this, the present invention proposes an intelligent detection method and system for surface defects of printed circuit boards based on dynamic templates to solve the above problems. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, the present invention provides the following technical solution:

[0006] A method for intelligent detection of surface defects on printed circuit boards based on dynamic templates includes:

[0007] Step S1: Obtain the design file and surface image of the printed circuit board to be tested. The design file contains the multilayer theoretical structure information of the printed circuit board.

[0008] Step S2: Perform hierarchical parsing on the acquired design files to obtain multiple levels of information, and align the multiple levels of information to obtain standardized hierarchical data;

[0009] Step S3: Perform feature enhancement based on standardized hierarchical data, fuse the enhanced hierarchical data, and construct a dynamic detection template based on it;

[0010] Step S4: Perform image preprocessing on the acquired surface image, and combine the dynamic detection template to perform nonlinear registration on the preprocessed surface image to obtain the corresponding registered image;

[0011] Step S5: Process the obtained registered images to obtain defect detection and classification results.

[0012] Furthermore, the process of parsing the acquired design documents to obtain multiple levels of information and aligning these multiple levels of information includes:

[0013] The acquired design files are parsed hierarchically according to preset rules to obtain multiple levels of information; and the multiple levels of information are aligned according to hierarchical characteristics to obtain standardized hierarchical data.

[0014] Furthermore, the process of aligning information at multiple levels according to hierarchical features includes:

[0015] Configure corresponding hierarchical matrices for multiple levels of information, and each hierarchical matrix includes multiple feature planes;

[0016] Establish hierarchical alignment reference points, which include multiple preset feature benchmarks. Extract features from hierarchical information according to the hierarchical alignment reference points to obtain multiple feature vectors that conform to the preset feature benchmarks. Optimize these vectors to obtain the first feature vector.

[0017] Multiple first feature vectors are placed one-to-one on a hierarchical matrix, and the feature vectors are outlined by the hierarchical matrix to obtain multiple standardized hierarchical data.

[0018] Furthermore, the process of placing multiple feature vectors one-to-one on a hierarchical matrix and recording the feature vectors' outlines using the hierarchical matrix includes:

[0019] Multiple first feature vectors are sequentially placed one-to-one on the level matrix according to the priority of their respective level types; the level matrix is ​​set sequentially.

[0020] Offset ranges of corresponding eigenvectors are set on multiple feature planes of the hierarchical matrix. The eigenvectors within the offset range are extracted and placed on multiple feature planes within the hierarchical matrix. The offsets of the eigenvectors are marked on the feature planes. Multiple calibration points are set on the feature planes. The offsets of the eigenvectors and their corresponding directions are recorded through the calibration points to obtain standardized hierarchical data.

[0021] Furthermore, the process of feature enhancement based on standardized hierarchical data, fusing the enhanced hierarchical data, and constructing a dynamic detection template based on it includes:

[0022] Enhancement rules are formulated for each of the multiple levels of information; the calibration points in the standardized hierarchical data are adjusted based on the enhancement rules to obtain the adjusted hierarchical data.

[0023] The adjusted hierarchical data that meets the preset conditions is used as the simulated hierarchical data. The offset and direction of the corresponding calibration point of the simulated hierarchical data are assigned to the feature vector on the feature bit surface to obtain the target hierarchical data. Multiple target hierarchical data are overlapped to obtain target hierarchical information. The multiple target hierarchical information are fused according to their positional relationship in the printed circuit board design file, and a dynamic detection template is constructed based on it.

[0024] Furthermore, the process of adjusting the calibration points in the standardized hierarchical data based on enhancement rules to obtain the adjusted hierarchical data includes:

[0025] Based on the offset and direction of the feature vector recorded in the calibration point, the offset range corresponding to the feature vector and the existing feature enhancement difference value are obtained as the adjustment base, and the corresponding enhancement rule is matched according to the adjustment base.

[0026] The offset simulation is applied to the calibration points in the standardized hierarchical data according to the matching enhancement rules. The offset of the calibration point simulation is then applied to the standardized hierarchical data to obtain the hierarchical data after adjustment.

[0027] Simultaneously, the process of using the adjusted hierarchical data that meets the preset conditions as simulated hierarchical data, and assigning the offset and corresponding direction of the calibration points corresponding to the simulated hierarchical data to the feature vectors on the feature plane, to obtain the target hierarchical data includes:

[0028] Establish preset conditions, which are that the difference range of the overlap between the adjusted hierarchical data and the standardized hierarchical data meets the preset difference threshold.

[0029] The adjusted hierarchical data that meets the preset conditions is used as the simulated hierarchical data. The offset of the calibration point corresponding to the simulated hierarchical data and the feature vector on the corresponding feature plane are adjusted to obtain the target hierarchical data.

[0030] Furthermore, the process of preprocessing the acquired surface image and performing nonlinear registration of the preprocessed surface image using a dynamic detection template includes:

[0031] The acquired surface image is preprocessed to obtain a standardized image;

[0032] The standardized image is decomposed into multiple scales, and local deformation descriptors are constructed based on the scale features at each scale to establish a nonlinear deformation model of the printed circuit board.

[0033] Feature points are extracted from the standardized image, and a feature point correlation matrix is ​​constructed based on them;

[0034] Based on the nonlinear deformation model and feature point correlation matrix, and using thin plate spline transformation to perform nonlinear registration between the standardized image and the dynamic detection template, a registered image is obtained.

[0035] Furthermore, the process of performing multi-scale decomposition on the standardized image and constructing local deformation descriptors based on the scale features at each scale to establish a nonlinear deformation model of the printed circuit board includes:

[0036] Wavelet transform is used to decompose the standardized image into multiple scales, resulting in several frequency sub-band images at different scales;

[0037] Features of the directional gradient histogram corresponding to the frequency sub-band image at each scale are extracted, and a local deformation descriptor is constructed based on it. The local deformation descriptor contains the material property parameters of the printed circuit board.

[0038] A nonlinear deformation model of a printed circuit board is established based on a local deformation descriptor and a preset nonlinear mapping function.

[0039] The process of using thin-plate spline transform to perform nonlinear registration between the standardized image and the dynamic detection template to obtain the registered image includes:

[0040] Based on the feature point correlation matrix, a correspondence between feature points is established between the standardized image and the dynamic detection template to obtain a set of control point pairs;

[0041] Based on the set of control point pairs and the nonlinear deformation model, a thin plate spline transformation model is constructed.

[0042] The standardized image is nonlinearly transformed based on the thin-plate spline transformation model to obtain the registered image.

[0043] Furthermore, the process of processing the obtained registered images to obtain defect detection and classification results includes:

[0044] A pre-selected dual-branch convolutional neural network is trained using a large number of labeled surface images to obtain a trained defect detection model.

[0045] The calibrated and registered image is input into the defect detection model to obtain the defect detection and classification results.

[0046] A smart detection system for surface defects on printed circuit boards based on dynamic templates includes:

[0047] The data acquisition module acquires the design file and surface image of the printed circuit board to be inspected. The design file contains multilayer theoretical structural information of the printed circuit board.

[0048] The data processing module performs hierarchical parsing on the acquired design files to obtain multiple levels of information, and aligns the multiple levels of information to obtain standardized hierarchical data.

[0049] The template construction module enhances features based on standardized hierarchical data, merges the enhanced hierarchical data, and constructs a dynamic detection template based on it.

[0050] The image correction module preprocesses the acquired surface image and performs nonlinear registration on the preprocessed surface image using a dynamic detection template to obtain the corresponding registered image.

[0051] The defect detection module processes the obtained registered images to obtain defect detection and classification results.

[0052] The technical effects and advantages of the intelligent detection method and system for surface defects of printed circuit boards based on dynamic templates described in this invention are as follows:

[0053] This invention, through multi-level parsing and precise alignment of printed circuit board design files, enables more detailed and systematic feature extraction and enhancement of PCB structural information, ensuring the accuracy and completeness of deformation model construction. This makes nonlinear registration more accurate and reliable, facilitating efficient identification and classification of subsequent defect detection models. Specifically, it includes the following advantages:

[0054] 1. By designing the hierarchical matrix and feature planes, and combining the precise recording of feature vector offsets at calibration points, high-precision parsing and alignment of PCB design files were achieved, ensuring the integrity and consistency of standardized hierarchical data and laying a solid foundation for subsequent dynamic template generation.

[0055] 2. Targeted enhancement rules are used to regulate standardized hierarchical data, and dynamic detection templates are constructed based on regional adaptive characteristics. This enables accurate modeling of the feature differences in different regions of the PCB, effectively improving the matching degree between the template and the actual image, and significantly enhancing the sensitivity and accuracy of defect detection.

[0056] 3. By combining multi-scale decomposition and thin-plate spline transformation techniques, a nonlinear deformation model capable of accurately describing the local and global deformation of PCBs was established. This effectively overcomes the limitations of traditional linear registration methods in handling complex deformations, significantly improves registration accuracy, and provides a stable and reliable foundation for defect detection.

[0057] 4. By using a dual-branch convolutional neural network to process deformation-sensitive areas and defect features separately, high-precision defect detection and classification under complex backgrounds are achieved. It can adapt to the inspection needs of PCBs with different materials and different circuit densities, and has strong generalization ability and practical value. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of the intelligent detection method for surface defects of printed circuit boards based on dynamic templates according to the present invention.

[0059] Figure 2 This is a schematic diagram of the intelligent detection system for surface defects of printed circuit boards based on dynamic templates according to the present invention. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] Example 1

[0062] Please see Figure 1 As shown, this embodiment is an intelligent detection method for surface defects on printed circuit boards based on dynamic templates, including:

[0063] Step S1: Obtain the design file and surface image of the printed circuit board to be tested. The design file contains the multilayer theoretical structure information of the printed circuit board.

[0064] It should be further explained that, in the specific implementation process, the design documents of the printed circuit board to be tested contain multi-layered theoretical structural information, such as key feature information such as circuit layout, wiring diagram, component position, and pad shape; the surface image is obtained by acquiring images of the surface of the corresponding printed circuit board to be tested through a high-resolution industrial camera or scanning equipment.

[0065] Step S2: Perform hierarchical parsing on the acquired design files to obtain multiple levels of information, and align the multiple levels of information to obtain standardized hierarchical data;

[0066] It should be further explained that, in the specific implementation process, step S2 includes the following steps:

[0067] Step S21: Perform hierarchical parsing on the acquired design files according to preset rules to obtain multiple levels of information;

[0068] Step S22: Align the information of multiple levels according to the hierarchical features to obtain standardized hierarchical data;

[0069] In one embodiment of the present invention, step S22, which involves aligning multiple levels of information according to hierarchical features to obtain standardized hierarchical data, specifically includes the following implementation process:

[0070] S221: Configure multiple hierarchical matrices corresponding to multiple hierarchical information, wherein the hierarchical matrix includes multiple feature planes;

[0071] S222: Define hierarchical alignment reference points. The hierarchical alignment reference points include multiple preset feature benchmarks. Extract features from the hierarchical information according to the hierarchical alignment reference points to obtain feature vectors corresponding to the multiple preset feature benchmarks.

[0072] S223: Place multiple feature vectors one-to-one on the hierarchical matrix, and record the contours of the feature vectors through the hierarchical matrix to obtain multiple standardized hierarchical data.

[0073] In one embodiment of the present invention, step S223, which involves placing multiple feature vectors one-to-one on a hierarchical matrix and recording the contours of the feature vectors using the hierarchical matrix to obtain multiple standardized hierarchical data, includes:

[0074] S2231: Place multiple feature vectors one-to-one on the hierarchical matrix in order of priority of their hierarchical type. The hierarchical matrix is ​​set sequentially.

[0075] S2232; Set the offset range of corresponding eigenvectors on multiple feature planes of the hierarchical matrix, extract and layer the eigenvectors within the offset range and place them on multiple feature planes, mark the offset of the eigenvectors on the feature planes, wherein multiple calibration points are set on multiple feature planes, and the offset of the eigenvectors and the corresponding direction are recorded through the calibration points.

[0076] As can be seen from steps S21 and S22 above, the printed circuit board design file that needs to be inspected for defects is determined. Before inspecting the printed circuit board image for defects, the design file needs to be parsed and aligned to improve the accuracy and recognizability of dynamic template generation. First, the format type of the obtained design file (e.g., Gerber, ODB++, etc.) is identified based on the file header information or extension. Based on the identified format type, a pre-built parsing module is called to perform hierarchical parsing of the multi-layer theoretical structure information in the design file, obtaining multiple levels of information. The parsing module parses according to preset rules (e.g., parsing Gerber format files according to the RS-274X standard). Then, the obtained multiple levels of information are configured accordingly. The corresponding hierarchical matrix is ​​composed of multiple feature planes. Each feature plane uses a blank database as a spatial layer (i.e., the blank database is spatially divided to obtain multiple spatial layers as feature planes, used for subsequent storage and display of aligned hierarchical information). Simultaneously, key alignment feature points within each layer are extracted, including pad centers, via centers, and graphic corner points, and these are used as hierarchical alignment reference points. Corresponding feature bases are set for each hierarchical alignment reference point. Feature bases are reference standards used in the hierarchical alignment process, primarily for extracting feature vectors from the hierarchical information of the PCB design file. (For example, feature bases can be key geometric shapes and feature points, component placement positions, or board edge contours in the design file).

[0077] Feature extraction of hierarchical information is performed based on hierarchical alignment reference points and feature benchmarks. This feature extraction process involves extracting feature vectors that conform to the feature benchmarks within the hierarchical information, resulting in feature vectors corresponding to multiple feature benchmarks. The feature vectors include feature contours and corresponding initial offsets. Iterative least squares method is used to continuously optimize the obtained initial offsets until the calculation results converge (i.e., the optimization process of iterative least squares method reaches a stable state), obtaining the optimal offset parameters. The optimal offset parameters and feature contours are integrated to obtain the first feature vector, and the obtained multiple first feature vectors are placed one-to-one on the hierarchical matrix. Then, multiple calibration points are set on the feature bit surface, and the first feature vectors on the hierarchical matrix are recorded based on them to obtain standardized hierarchical data. Among them, the calibration points are evenly and closely distributed on the feature bit surface to record the offset and corresponding direction of the corresponding feature vector. The position of the feature vector on the feature bit surface corresponds to the original position on the hierarchical information, with a more detailed division, which enables more detailed analysis and alignment of hierarchical information, making the details of the parsed printed circuit board design file clearer and facilitating the provision of more accurate defect detection information in the future.

[0078] Step S3: Perform feature enhancement based on standardized hierarchical data, fuse the enhanced hierarchical data, and generate a dynamic detection template based on it;

[0079] It should be further explained that, in the specific implementation process, step S3 includes the following steps:

[0080] S31: Formulate corresponding enhancement rules for information at multiple levels, including a strategy for completing missing information and corresponding feature enhancement difference values.

[0081] S32: Adjust the calibration points in the standardized hierarchical data based on the enhancement rules to obtain the adjusted hierarchical data;

[0082] S33: Use the adjusted hierarchical data that meets the preset conditions as the simulated hierarchical data, and assign the offset of the calibration point corresponding to the simulated hierarchical data and the corresponding direction to the feature vector on the feature plane to obtain the target hierarchical data.

[0083] S34: Overlap multiple target level data to obtain target level information, fuse multiple target level information according to their positional relationship in the printed circuit board design file, and build a dynamic detection template based on it;

[0084] In one embodiment, step S32, which modifies the calibration points in the standardized hierarchical data based on enhancement rules to obtain the modulated hierarchical data, includes:

[0085] S321: Based on the offset and corresponding direction of the feature vector recorded in the calibration point, obtain the offset range of the feature vector and the existing feature enhancement difference value as the adjustment base, and match the corresponding enhancement rule according to the adjustment base.

[0086] S322: Perform offset simulation adjustment on the calibration points in the standardized hierarchical data according to the matching enhancement rules, and load the offset of the calibration point simulation adjustment into the standardized hierarchical data to obtain the adjusted hierarchical data.

[0087] In one embodiment, step S33, which uses the adjusted hierarchical data that meets preset conditions as simulated hierarchical data, and assigns the offset of the calibration point corresponding to the simulated hierarchical data and the corresponding direction to the feature vector on the feature bit surface to obtain the target hierarchical data, includes:

[0088] S331: Set preset conditions, wherein the preset conditions are that the difference range of contour overlap between the adjusted hierarchical data and the standardized hierarchical data meets the preset difference threshold.

[0089] S332: The adjusted hierarchical data that meets the preset conditions is used as the simulated hierarchical data. The offset of the calibration point corresponding to the simulated hierarchical data and the feature vector on the corresponding feature plane of the corresponding direction are adjusted to obtain the target hierarchical data.

[0090] As can be seen from steps S31 to S34 above, corresponding enhancement rules are formulated for the obtained hierarchical information. The enhancement rules include the missing information completion strategy and the corresponding feature enhancement difference value. The feature enhancement difference value refers to the amount of offset to be adjusted within the corresponding offset range. Moreover, the enhancement rules have some differences for different hierarchical information. For example, for missing solder mask color information, it is inferred and completed by analyzing the material property identifiers in the design file or industry default standards; for missing font and position information of silkscreen text, it is predicted and completed using a deep learning-based text feature recognition algorithm; for missing inter-layer connection relationship information, it is inferred and completed using a graph theory-based network topology analysis method.

[0091] Furthermore, the offset and corresponding direction of the feature vector recorded at each calibration point are obtained, and the offset range and corresponding feature enhancement difference value corresponding to the first feature vector are estimated based on them, and used as the adjustment base. Based on the adjustment base, the enhancement rules are matched from the established enhancement rules to obtain the enhancement rules matched by the corresponding adjustment base. Based on these rules, the offset of the calibration points in the standardized hierarchical data is simulated and adjusted, and the offset of the calibration points is loaded into the standardized hierarchical data to obtain the adjusted hierarchical data. Because the feature vector is set on the feature plane, the calibration point is the unit for adjusting the offset, which is a single processing node. This processing node is connected to the corresponding hierarchical data, and the hierarchical data is the data layer. All calibration points on a single feature plane are... The calibration points are interconnected. Multiple calibration points are responsible for recording and adjusting the offset of their corresponding ranges. All storage ranges on the feature plane are covered by calibration points. In this way, after the feature vectors are laid out on the feature plane, they can be managed by the calibration points. The offset can be directly determined through the calibration points, and the offset of the responsible feature vectors can be adjusted. Moreover, in a hierarchical data, the feature vectors do not cover the entire area. Only the calibration points corresponding to the covered positions will have offset adjustment operations. Simulation adjustment can be performed without directly loading it into the hierarchical data. Because there is a corresponding relationship between the calibration points and the responsible feature vectors, the display of other ranges and offsets is the same. Therefore, the hierarchical data can be represented by calibration points as the hierarchical data after simulation.

[0092] The preset conditions are set, which refer to the fact that the contour overlap between the hierarchical data after simulation and the standardized hierarchical data meets the preset difference threshold. Since each calibration point is responsible for managing a certain area, the adjustment may cause local areas to exceed the original contour line. Therefore, it is necessary to evaluate the contour consistency between the adjusted hierarchical data and the standardized hierarchical data. Thus, when the difference range of contour overlap after adjustment through the calibration point meets the preset difference threshold, it indicates that the corresponding calibration point adjustment process is effective.

[0093] Then, the adjusted hierarchical data that meets the preset conditions is used as the simulated hierarchical data; the first feature vector on the feature plane corresponding to the calibration point in the simulated hierarchical data is adjusted to obtain the target hierarchical data; multiple target hierarchical data are overlapped to obtain the target hierarchical information; multiple target hierarchical information are fused according to their original positions in the printed circuit board design file to obtain the corrected multilayer theoretical structure information; the corrected multilayer theoretical structure information is converted into a precise three-dimensional model, which includes the geometry, material properties, and circuit layout information of the printed circuit board; then, according to the actual lighting conditions of the testing environment (such as the position and intensity of the light source)... Using color and camera parameters (such as focal length, viewing angle, and resolution), an ideal printed circuit board surface image is simulated and generated. Based on color features, texture features, or morphological features, the ideal printed circuit board surface image is precisely divided into multiple blocks such as functional blocks, pad blocks, circuit blocks, and background blocks. Finally, according to the characteristics of each block and the types of defects that may occur, corresponding detection parameters and thresholds are assigned to each block, such as brightness tolerance, edge sharpness, and shape deviation, forming a dynamic detection template with regional adaptive characteristics. The dynamic detection template can adapt to the detection needs of different areas, improving the accuracy and efficiency of defect detection.

[0094] Step S4: Perform image preprocessing on the acquired surface image, and combine the dynamic detection template to perform nonlinear registration on the preprocessed surface image to obtain the corresponding registered image;

[0095] It should be further explained that, in the specific implementation process, step S4 includes the following steps: Step S41: Perform image preprocessing on the acquired surface image to obtain a standardized image;

[0096] Step S42: Perform multi-scale decomposition on the standardized image and construct local deformation descriptors based on the scale features at each scale to establish a nonlinear deformation model of the printed circuit board.

[0097] Step S43: Extract feature points within the standardized image based on the adaptive threshold segmentation algorithm, and construct a feature point correlation matrix based on them;

[0098] Step S44: Based on the nonlinear deformation model and feature point correlation matrix, and using thin plate spline transformation, nonlinear registration is performed on the standardized image and the dynamic detection template to obtain the registered image;

[0099] In one embodiment of the present invention, step S41, which involves preprocessing the acquired surface image to obtain a standardized image, includes:

[0100] Step S411: Perform grayscale processing on the acquired surface image of the printed circuit board to obtain a grayscale image;

[0101] Step S412: Perform equalization processing on the grayscale image using a homomorphic filtering algorithm;

[0102] Step S413: Perform adaptive contrast enhancement on the image after equalization processing according to the material characteristics of the printed circuit board to obtain a standardized image.

[0103] In one embodiment of the present invention, step S42, which involves multi-scale decomposition of a standardized image and construction of local deformation descriptors based on scale features at each scale to establish a nonlinear deformation model of a printed circuit board, includes:

[0104] S421: Wavelet transform is used to decompose the standardized image into multiple scales to obtain several frequency sub-band images at different scales;

[0105] S422: Extract the features of the directional gradient histogram corresponding to the frequency sub-band image at each scale, and construct a local deformation descriptor based on it. The local deformation descriptor contains the material property parameters of the printed circuit board.

[0106] S423: Based on the local deformation descriptor and the preset nonlinear mapping function, establish a nonlinear deformation model of the printed circuit board.

[0107] In one embodiment of the present invention, step S43, which involves extracting feature points within a standardized image based on an adaptive threshold segmentation algorithm and constructing a feature point correlation matrix based thereon, includes:

[0108] S431: Based on the Otsu adaptive threshold segmentation algorithm, perform region segmentation on the standardized image and extract the salient regions in the image, including solder joints, wires and component edges on the circuit board;

[0109] S432: In the segmented salient region, feature points are extracted using the FAST corner detection algorithm;

[0110] S433: Filter and optimize the extracted feature points to obtain discriminative feature points, and construct a feature point correlation matrix based on them.

[0111] As described in steps S41 to S43 above, the acquired surface image is first converted into a grayscale image. Then, a homomorphic filtering algorithm is used to perform illumination equalization on the grayscale image. Homomorphic filtering can separate the corresponding illumination and reflection components from the grayscale image and suppress the illumination component, thereby eliminating the influence of uneven illumination. Finally, adaptive contrast enhancement is performed based on the characteristics of different printed circuit board materials (such as FR4, ceramic substrates, etc.) to highlight defect features and obtain a standardized image. Wavelet transform is used to decompose the standardized image into multiple scales, obtaining frequency sub-band images at different scales. Orientation gradient statistical analysis is performed on each frequency sub-band image to extract the texture features of the printed circuit board surface. Based on the texture features, deformation-sensitive features of local areas are constructed for subsequent deformation model establishment.

[0112] Furthermore, local deformation descriptors for key regions in each frequency sub-band image are calculated, and these descriptors contain printed circuit board material property parameters. A deformation parameter database for printed circuit boards of different materials (including FR4 and ceramic substrates) is established. Based on the deformation parameter database and local deformation descriptors, and combined with a preset nonlinear mapping function, a pre-selected machine learning algorithm (such as support vector machine or random forest) is iteratively trained to construct a nonlinear deformation model that can adapt to different types of printed circuit boards. The preset nonlinear mapping function is implemented using a radial basis function network to capture various nonlinear deformations that may occur in printed circuit boards during production and use.

[0113] Furthermore, the Otsu algorithm is used to adaptively determine the image segmentation threshold. The Otsu algorithm determines the optimal threshold by maximizing the inter-class variance, effectively addressing image segmentation problems under different lighting conditions. Within the segmented salient regions, the FAST corner detection algorithm is used to detect feature points such as pad edges and line intersections. Then, local response values ​​are calculated for the detected feature points, reflecting the intensity of changes in pixels surrounding the feature point. A threshold is set to remove low-quality feature points. Finally, non-maximum suppression is used to optimize the feature point distribution, ensuring feature point coverage in key areas. Local descriptors for each feature point are obtained; these descriptors are feature vectors that integrate gradient and texture information, describing the unique image characteristics of the region surrounding the feature point. A feature point association matrix is ​​constructed by combining the association strength between feature points established by local region similarity measurement. The association matrix is ​​then sparsified to retain key association information, improving the computational efficiency of subsequent registration algorithms.

[0114] In one embodiment of the present invention, step S44, which involves nonlinear registration of a standardized image and a dynamic detection template using a thin-plate spline transformation based on a nonlinear deformation model and a feature point correlation matrix to obtain a registered image, includes:

[0115] S441: Based on the feature point correlation matrix, establish the correspondence between feature points between the standardized image and the dynamic detection template to obtain a set of control point pairs;

[0116] S442: Based on the set of control point pairs and the nonlinear deformation model, a thin plate spline transformation model is constructed. The thin plate spline transformation model defines the spatial mapping relationship from the standardized image to the dynamic detection template.

[0117] S443: Based on the thin plate spline transformation model, a nonlinear transformation is performed on the standardized image to achieve accurate registration between the standardized image and the dynamic detection template, resulting in a registered image.

[0118] As shown in steps S441 to S443 above, when establishing the correspondence between feature points, a bidirectional nearest neighbor matching strategy is used to remove erroneous matches and improve the robustness of registration. The thin plate spline transformation model can be used to simulate the deformation process of an elastic thin plate under external force, and can accurately describe the local and global deformation of the printed circuit board. The thin plate spline transformation model consists of two parts: a global affine transformation part and a local nonlinear transformation part. The global affine transformation part is used to handle the overall translation, rotation, and scaling, while the local nonlinear transformation part is used to handle local bending and torsional deformation. When applying the thin plate spline transformation model, a bilinear interpolation algorithm is used to calculate the pixel mapping relationship to ensure the smoothness and continuity of the transformed image. At the same time, it ensures that the registered image and the dynamic detection template are highly consistent in spatial position, providing a reliable basis for subsequent defect detection and quality assessment. Through this nonlinear registration method, various deformations that may occur during the manufacturing and use of printed circuit boards can be effectively handled, significantly improving the accuracy and reliability of the detection system.

[0119] Step S5: Process the obtained registered images to obtain defect detection and classification results;

[0120] It should be further explained that, in the specific implementation process, step S5 includes the following steps:

[0121] S51: Train a pre-selected dual-branch convolutional neural network using a large number of labeled surface images to obtain a trained defect detection model.

[0122] S52: Input the calibrated and registered image into the defect detection model to obtain the defect detection and classification results.

[0123] As described in steps S51 and S52 above, a defect detection model is obtained by training a dual-branch convolutional neural network with a large number of labeled surface images. This process includes collecting a large number of surface image samples from optical inspection equipment on the printed circuit board production line, such as AOI equipment and high-precision cameras; ensuring that the images contain various types of printed circuit board materials, different line densities, and various common defect types to improve the model's generalization ability; accurately labeling the defect areas in the collected surface image samples, marking the defect categories at the pixel level, such as short circuits, open circuits, pinholes, and copper foil residues; and preprocessing the images, including normalization, image enhancement, and data augmentation. Normalization can make the model training more stable; image enhancement can improve the visibility of minor defects; and data augmentation (such as rotation, scaling, and adding noise) can increase data diversity and improve the model's adaptability to various imaging conditions. The corresponding accurate labeling process is existing technology and will not be elaborated on in this application.

[0124] The preprocessed surface image samples are input into the designed dual-branch convolutional neural network. One branch is responsible for feature extraction of deformation-sensitive areas, and the other branch is responsible for extracting defect features. Finally, high-precision defect detection is achieved through a feature fusion layer. The difference between the predicted results and the true labels is calculated through a loss function, and the network parameters are updated through an optimization algorithm. During the training process, the model performance is monitored using a validation set. When the performance indicators on the validation set no longer improve, the training is completed and the final defect detection model is obtained.

[0125] Then, the registered image is input into the defect detection model to obtain defect detection and classification results. The defect detection and classification results include: a defect location map, a binary image used to accurately display the location and extent of each defect on the printed circuit board, where 1 in the binary image represents a defect area and 0 represents a normal area; defect classification results: each detected defect area is classified, such as short circuit, open circuit, pinhole, copper foil residue, etc., and a confidence score for the classification is given; defect statistics: including statistical information such as the quantity, area, and severity of various defects, which can be used for production quality control and trend analysis; defect feature description: key feature parameters such as shape, size, and edge characteristics are extracted for each defect area to provide a basis for subsequent defect cause analysis.

[0126] This method can effectively solve the problems of difficulty in modeling local deformation and insufficient robustness of feature point extraction in printed circuit board inspection, thereby improving the accuracy and reliability of PCB surface defect detection.

[0127] Example 2

[0128] Please see Figure 2As shown, parts not described in detail in this embodiment are described in Embodiment 1. A smart detection system for surface defects on printed circuit boards based on dynamic templates is provided, including:

[0129] The data acquisition module acquires the design file and surface image of the printed circuit board to be inspected. The design file contains information on the multi-layer theoretical structure of the printed circuit board.

[0130] The data processing module performs hierarchical parsing on the acquired design files to obtain multiple levels of information, and aligns the multiple levels of information to obtain standardized hierarchical data.

[0131] The template construction module enhances features based on standardized hierarchical data, merges the enhanced hierarchical data, and constructs a dynamic detection template based on it.

[0132] The image correction module preprocesses the acquired surface image and performs nonlinear registration on the preprocessed surface image using a dynamic detection template to obtain the corresponding registered image.

[0133] The defect detection module processes the obtained registered images to obtain defect detection and classification results.

[0134] The modules are connected via wired and / or wireless means to enable data transmission between them.

[0135] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0136] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0137] In the description of this invention, it should be understood that the terms "first," "second," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0138] In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0139] In the description of this invention, "several" means one or more, and "a large number" means two or more.

[0140] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0141] All formulas in this manual are dimensionless and calculated numerically. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0142] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for intelligent detection of surface defects on printed circuit boards based on dynamic templates, characterized in that, include: Step S1: Obtain the design file and surface image of the printed circuit board to be tested. The design file contains the multilayer theoretical structure information of the printed circuit board. Step S2: Perform hierarchical parsing on the obtained design file to obtain multiple hierarchical information, and configure the corresponding hierarchical matrix for each of the multiple hierarchical information. The hierarchical matrix includes multiple feature planes. Define hierarchical alignment reference points, which include multiple preset feature benchmarks. Extract features from hierarchical information according to the hierarchical alignment reference points to obtain feature vectors corresponding to the multiple preset feature benchmarks. Multiple feature vectors are placed one-to-one on the hierarchical matrix according to the priority of their hierarchical type, and the hierarchical matrix is ​​set sequentially. The offset range of the corresponding eigenvector is set on multiple feature planes of the hierarchical matrix. The eigenvectors within the offset range are extracted, layered, and placed on multiple feature planes. The offset of the eigenvector is marked on the feature plane. Multiple calibration points are set on multiple feature planes. The offset of the eigenvector and its corresponding direction are recorded through the calibration points to obtain standardized hierarchical data. Step S3: Perform feature enhancement based on standardized hierarchical data, fuse the enhanced hierarchical data, and construct a dynamic detection template based on it; The process of constructing a dynamic detection template includes: For information at multiple levels, corresponding enhancement rules are formulated. Based on the enhancement rules, the calibration points in the standardized hierarchical data are adjusted. By following the offset and direction of the feature vector recorded in the calibration points, the deviation range of the feature vector and the existing feature enhancement difference value are obtained as the adjustment base. The corresponding enhancement rules are matched according to the adjustment base. The offset simulation is applied to the calibration points in the standardized hierarchical data according to the matching enhancement rules. The offset of the calibration point simulation is then applied to the standardized hierarchical data to obtain the hierarchical data after adjustment. Establish preset conditions, which are that the difference range of the overlap between the adjusted hierarchical data and the standardized hierarchical data meets the preset difference threshold. The hierarchical data after adjustment that meets the preset conditions is used as the simulated hierarchical data. The offset and direction corresponding to the calibration point in the simulated hierarchical data are assigned to the feature vector on the corresponding feature plane to obtain the target hierarchical data. The target level information is obtained by overlapping multiple target level data, and the target level information is fused according to the positional relationship in the printed circuit board design file. A dynamic detection template is then constructed based on this information. Step S4: Perform image preprocessing on the acquired surface image, and combine the dynamic detection template to perform nonlinear registration on the preprocessed surface image to obtain the corresponding registered image; Step S5: Process the obtained registered images to obtain defect detection and classification results.

2. The intelligent detection method for surface defects of printed circuit boards based on dynamic templates according to claim 1, characterized in that, The process of preprocessing the acquired surface image and then performing nonlinear registration of the preprocessed surface image using a dynamic detection template includes: The acquired surface image is preprocessed to obtain a standardized image; The standardized image is decomposed into multiple scales, and local deformation descriptors are constructed based on the scale features at each scale to establish a nonlinear deformation model of the printed circuit board. Feature points are extracted from the standardized image, and a feature point correlation matrix is ​​constructed based on them; Based on the nonlinear deformation model and feature point correlation matrix, and using thin plate spline transformation to perform nonlinear registration between the standardized image and the dynamic detection template, a registered image is obtained.

3. The intelligent detection method for surface defects of printed circuit boards based on dynamic templates according to claim 2, characterized in that, The process of decomposing a standardized image into multiple scales and constructing a local deformation descriptor based on the scale features at each scale to establish a nonlinear deformation model for a printed circuit board includes: Wavelet transform is used to decompose the standardized image into multiple scales, resulting in several frequency sub-band images at different scales; Features of the directional gradient histogram corresponding to the frequency sub-band image at each scale are extracted, and a local deformation descriptor is constructed based on it. The local deformation descriptor contains the material property parameters of the printed circuit board. A nonlinear deformation model of a printed circuit board is established based on a local deformation descriptor and a preset nonlinear mapping function. The process of using thin-plate spline transform to perform nonlinear registration between the standardized image and the dynamic detection template to obtain the registered image includes: Based on the feature point correlation matrix, a correspondence between feature points is established between the standardized image and the dynamic detection template to obtain a set of control point pairs; Based on the set of control point pairs and the nonlinear deformation model, a thin plate spline transformation model is constructed. The standardized image is nonlinearly transformed based on the thin-plate spline transformation model to obtain the registered image.

4. The intelligent detection method for surface defects of printed circuit boards based on dynamic templates according to claim 3, characterized in that... It lies in, The process of processing the obtained registered images to obtain defect detection and classification results includes: A pre-selected dual-branch convolutional neural network is trained using a large number of labeled surface images to obtain a trained defect detection model. The calibrated and registered image is input into the defect detection model to obtain the defect detection and classification results.

5. A dynamic template-based intelligent detection system for surface defects on printed circuit boards, used to implement the dynamic template-based intelligent detection method for surface defects on printed circuit boards according to any one of claims 1 to 4, comprising: The data acquisition module acquires the design file and surface image of the printed circuit board to be inspected. The design file contains multilayer theoretical structural information of the printed circuit board. The data processing module performs hierarchical parsing on the acquired design files to obtain multiple levels of information, and aligns the multiple levels of information to obtain standardized hierarchical data. The template construction module enhances features based on standardized hierarchical data, merges the enhanced hierarchical data, and constructs a dynamic detection template based on it. The image correction module preprocesses the acquired surface image and performs nonlinear registration on the preprocessed surface image using a dynamic detection template to obtain the corresponding registered image. The defect detection module processes the obtained registered images to obtain defect detection and classification results.

Images