Printed circuit board surface flaw intelligent detection method and system based on dynamic template

Abstract

The invention belongs to the technical field of flaw detection, and discloses an intelligent detection method and system for surface flaws of a printed circuit board based on a dynamic template, and the method comprises the steps: obtaining a design file and a surface image of a to-be-detected printed circuit board; performing hierarchical analysis on the data to obtain multiple pieces of hierarchical information, and performing alignment processing on the multiple pieces of hierarchical information to obtain standardized hierarchical data; performing feature enhancement based on the standardized hierarchical data, fusing the enhanced hierarchical data, and constructing a dynamic detection template based on the fused data; carrying out image preprocessing on the collected surface image, and carrying out nonlinear registration on the surface image after image preprocessing in combination with a dynamic detection template to obtain a corresponding registration image; processing the obtained registration image to obtain a flaw detection and classification result; according to the method, the local deformation model and the feature point incidence matrix are accurately established, so that the accuracy of nonlinear registration is ensured, and the problem of false detection caused by deformation in printed circuit board detection is effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of defect detection, more particularly, the present application relates to a printed circuit board surface defect intelligent detection method and system based on a dynamic template. BACKGROUND

[0002] A printed circuit board (PCB) is a core component of electronic products. With the development of electronic devices towards miniaturization and high integration, the manufacturing precision of PCBs is continuously improved, and the importance of surface defect detection is increasingly highlighted. Traditional printed circuit board surface defect detection mainly relies on manual visual inspection or simple machine vision systems for judgment.

[0003] However, this method has certain limitations. On the one hand, the recognition of PCB surface defects requires professional knowledge and rich experience, and the number of skilled detection personnel is relatively limited, and the training cycle is long, which is difficult to meet the detection needs of mass production in modern electronic manufacturing industry; on the other hand, with the increasing complexity of printed circuit board structure and the increasing line density, the defect features are more subtle, and the accuracy and efficiency of manual detection are difficult to guarantee. Although the introduction of computer vision and deep learning technology can assist the detection process to a certain extent, the non-linear deformation problem often occurs in the manufacturing and use of PCBs, which seriously affects the accuracy of the detection results. The existing technology cannot effectively establish an accurate correspondence between the printed circuit board design file and the actual surface image, and cannot accurately distinguish between the deformation area and the real defect, resulting in a high false detection rate and inconvenience for the defect detection and classification of the deep learning model.

[0004] In view of this, the present application proposes a printed circuit board surface defect intelligent detection method and system based on a dynamic template to solve the above problems. SUMMARY

[0005] In order to overcome the above-mentioned defects of the prior art, in order to achieve the above-mentioned purposes, the present application provides the following technical solutions: The printed circuit board surface defect intelligent detection method based on a dynamic template comprises: Step S1: obtaining a design file and a surface image of a printed circuit board to be detected, the design file containing multi-layer theoretical structure information in the printed circuit board; Step S2: performing hierarchical analysis on the obtained design file to obtain a plurality of hierarchical information, and performing alignment processing on the plurality of hierarchical information to obtain standardized hierarchical data; Step S3: performing feature enhancement based on the standardized hierarchical data, fusing the enhanced hierarchical data, and constructing a dynamic detection template based thereon; Step S4: image pre-processing is performed on the collected surface image, and non-linear registration is performed on the surface image after image pre-processing in combination with a dynamic detection template to obtain a corresponding registration image; Step S5: processing is performed on the obtained registration image to obtain a flaw detection and classification result.

[0006] Further, the process of performing hierarchical analysis on the obtained design file to obtain a plurality of hierarchical information and performing alignment processing on the plurality of hierarchical information includes: The obtained design file is analyzed according to a preset rule to obtain a plurality of hierarchical information, and the plurality of hierarchical information is aligned according to hierarchical characteristics to obtain standardized hierarchical data.

[0007] Further, the process of performing alignment processing on the plurality of hierarchical information according to hierarchical characteristics includes: A corresponding hierarchical matrix is configured for each of the plurality of hierarchical information, and the hierarchical matrix includes a plurality of characteristic bit planes; A hierarchical alignment reference point is formulated, the hierarchical alignment reference point includes a plurality of preset characteristic bases, a feature extraction is performed on the hierarchical information according to the hierarchical alignment reference point to obtain a plurality of characteristic vectors conforming to the preset characteristic bases, and an optimization is performed on the plurality of characteristic vectors to obtain a first characteristic vector; The plurality of first characteristic vectors are respectively placed one-to-one on the hierarchical matrix, and the characteristic vectors are profiled by the hierarchical matrix to obtain a plurality of standardized hierarchical data.

[0008] Further, the process of placing the plurality of characteristic vectors one-to-one on the hierarchical matrix and profiling the characteristic vectors by the hierarchical matrix includes: The plurality of first characteristic vectors are sequentially placed one-to-one on the hierarchical matrix according to the priority of the hierarchical type; wherein the hierarchical matrix is sequentially arranged; An offset range of the corresponding characteristic vector is set on the plurality of characteristic bit planes of the hierarchical matrix, the characteristic vectors within the offset range are extracted and placed on the plurality of characteristic bit planes in the hierarchical matrix, and the offset amount of the characteristic vector is marked on the characteristic bit plane, wherein a plurality of calibration points are arranged on the characteristic bit plane, the offset value and the corresponding direction of the characteristic vector are recorded by the calibration points to obtain standardized hierarchical data.

[0009] Further, the process of performing feature enhancement based on the standardized hierarchical data, fusing the hierarchical data after enhancement, and constructing a dynamic detection template based on the same includes: A corresponding enhancement rule is formulated for each of the plurality of hierarchical information; the calibration points in the standardized hierarchical data are regulated based on the enhancement rule to obtain the standardized hierarchical data after regulation; The hierarchical data after regulation meeting preset conditions is taken as simulation hierarchical data, offset values and directions of the simulation hierarchical data corresponding to the calibration points are assigned to the feature vectors on the feature bit surface, and target hierarchical data is obtained; the plurality of target hierarchical data is superimposed to obtain target hierarchical information, the plurality of target hierarchical information is correspondingly fused according to the positional relationship in the printed circuit board design file, and a dynamic detection template is constructed based on the same.

[0010] Further, the process of regulating the calibration points in the standardized hierarchical data based on the enhanced rules to obtain the hierarchical data after simulation includes: According to the offset values and directions of the feature vectors recorded in the calibration points, offset ranges and existing feature enhancement difference values corresponding to the feature vectors are obtained as adjustment bases, and the corresponding enhanced rules are matched according to the adjustment bases; The calibration points in the standardized hierarchical data are simulated and regulated according to the matched enhanced rules, and the offset of the calibration points after simulation and regulation is loaded in the standardized hierarchical data to obtain the hierarchical data after simulation; Meanwhile, the process of obtaining the target hierarchical data by assigning the offset values and corresponding directions of the simulation hierarchical data corresponding to the calibration points to the feature vectors on the feature bit surface includes: A preset condition is formulated, and the preset condition is that the difference range of the profile coincidence between the hierarchical data after simulation and the standardized hierarchical data meets a preset difference threshold; The hierarchical data after simulation meeting the preset condition is taken as simulation hierarchical data, and the offset values and corresponding directions of the simulation hierarchical data corresponding to the calibration points are adjusted to the feature vectors on the feature bit surface to obtain target hierarchical data.

[0011] Further, the process of performing image preprocessing on the collected surface image and combining the dynamic detection template to perform nonlinear registration on the surface image after image preprocessing includes: The obtained surface image is image preprocessed to obtain a standardized image; The standardized image is multiscale decomposed, and a local deformation descriptor is constructed based on the scale features under each scale to establish a nonlinear deformation model of the printed circuit board; Feature points in the standardized image are extracted, and a feature point correlation matrix is constructed based on the same; Based on the nonlinear deformation model and the feature point correlation matrix, and by using a thin plate spline transformation, the standardized image and the dynamic detection template are nonlinearly registered to obtain a registration image.

[0012] Furthermore, the standardized image is decomposed into multiple scales, and local deformation descriptors are constructed based on the scale features at each scale. The process of establishing a nonlinear deformation model of the printed circuit board includes the following steps: Wavelet transform is used to perform multi-scale decomposition on the standardized image to obtain frequency sub-band images at several different scales; Extract the features of the oriented gradient histogram corresponding to the frequency subband image at each scale, and construct a local deformation descriptor based on it. The local deformation descriptor contains the material characteristic parameters of the printed circuit board; Based on the local deformation descriptor and the preset nonlinear mapping function, a nonlinear deformation model of the printed circuit board is established; Thin plate spline transform is used to perform nonlinear registration between the standardized image and the dynamic detection template. The process of obtaining the registered image includes: Based on the feature point association matrix, the feature point correspondence relationship is established between the standardized image and the dynamic detection template to obtain a set of control point pairs; Based on the set of control point pairs and the nonlinear deformation model, a thin plate spline transformation model is constructed; The standardized image is nonlinearly transformed based on the thin plate spline transformation model to obtain the registered image.

[0013] Furthermore, the process of processing the obtained registered image to obtain defect detection and classification results includes: A pre-selected two-branch convolutional neural network is trained using a large number of labeled surface images to obtain a trained defect detection model. The calibrated registered image is input into the defect detection model to obtain the defect detection and classification results.

[0014] The intelligent detection system for printed circuit board surface defects based on dynamic templates includes: A data acquisition module is configured to acquire a design file and a surface image of a printed circuit board to be inspected, wherein the design file includes information on a multi-layer theoretical structure within the printed circuit board; The data processing module performs hierarchical analysis on the acquired design files to obtain multiple hierarchical information, and aligns the multiple hierarchical information to obtain standardized hierarchical data; The template construction module performs feature enhancement based on standardized hierarchical data, fuses the enhanced hierarchical data, and constructs a dynamic detection template based on it; The image correction module performs image preprocessing on the collected surface image, and performs nonlinear registration on the surface image after image preprocessing in combination with the dynamic detection template to obtain the corresponding registered image; The defect detection module processes the obtained registered images to obtain defect detection and classification results.

[0015] The technical effects and advantages of the dynamic template-based printed circuit board surface flaw intelligent detection method and system of the present application are as follows: The present application can more meticulously and systematically extract and enhance the PCB structure information by multi-level analysis and accurate alignment of the printed circuit board design file, ensure the accuracy and integrity of the deformation model construction, make the nonlinear registration more accurate and reliable, and facilitate the efficient identification and classification of the subsequent flaw detection model. 1. Through the design of hierarchical matrix and feature bit plane, combined with the accurate recording of feature vector offset by calibration points, high-precision analysis and alignment of PCB design files are realized, ensuring the integrity and consistency of standardized hierarchical data, and laying a solid foundation for subsequent dynamic template generation. 2. The standardized hierarchical data is regulated by targeted enhancement rules, and a dynamic detection template is constructed based on regional adaptive characteristics, which can accurately model the feature differences of different regions of the PCB, effectively improve the matching degree of the template and the actual image, and greatly improve the sensitivity and accuracy of flaw detection. 3. Combined with multi-scale decomposition and thin plate spline transformation technology, a nonlinear deformation model is established which can accurately describe the local and global deformation of the PCB, effectively overcoming the limitations of traditional linear registration methods that cannot handle complex deformation, significantly improving the registration accuracy, and providing a stable and reliable basis for flaw detection. 4. The double-branch convolutional neural network is used to process deformation-sensitive areas and flaw features respectively, realizing high-precision flaw detection and classification in complex backgrounds, and being able to adapt to PCB detection requirements of different materials and different line densities, with strong generalization ability and practical value. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 The figure is a schematic diagram of the dynamic template-based printed circuit board surface flaw intelligent detection method of the present application. Figure 2 The figure is a schematic diagram of the dynamic template-based printed circuit board surface flaw intelligent detection system of the present application. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0018] Example 1

[0019] Please refer to Figure 1As shown, the embodiment is a dynamic template-based printed circuit board surface flaw intelligent detection method, which comprises the following steps: Step S1: obtaining the design file and surface image of the printed circuit board to be detected, the design file containing the multi-layer theoretical structure information in the printed circuit board; It should be further pointed out that, in the specific implementation process, the design file of the printed circuit board to be detected contains multi-layer theoretical structure information, for example: circuit layout, wiring diagram, component position, pad shape and other key feature information; the surface image is obtained by image acquisition of the surface of the corresponding printed circuit board to be detected through a high-resolution industrial camera or a scanning device.

[0020] Step S2: performing hierarchical analysis on the obtained design file to obtain a plurality of hierarchical information, and performing alignment processing on the plurality of hierarchical information to obtain standardized hierarchical data; It should be further pointed out that, in the specific implementation process, the implementation process of step S2 comprises the following steps: Step S21: performing hierarchical analysis on the obtained design file according to a preset rule to obtain a plurality of hierarchical information; Step S22: performing alignment processing on the plurality of hierarchical information according to hierarchical features to obtain standardized hierarchical data; In an embodiment of the present application, the step S22 of performing alignment processing on the plurality of hierarchical information according to hierarchical features to obtain standardized hierarchical data comprises: S221: configuring a plurality of hierarchical matrices corresponding to the plurality of hierarchical information, wherein the hierarchical matrix comprises a plurality of feature bit planes; S222: formulating a hierarchical alignment reference point, the hierarchical alignment reference point comprising a plurality of preset feature references, performing feature extraction on the hierarchical information according to the hierarchical alignment reference point to obtain feature vectors corresponding to the plurality of preset feature references; S223: placing the plurality of feature vectors one by one on the hierarchical matrix, and recording the feature vectors through the hierarchical matrix to obtain a plurality of standardized hierarchical data; In an embodiment of the present application, the step S223 of placing the plurality of feature vectors one by one on the hierarchical matrix and recording the feature vectors through the hierarchical matrix to obtain a plurality of standardized hierarchical data comprises: S2231: placing the plurality of feature vectors one by one on the hierarchical matrix according to the priority of the hierarchical type, and the hierarchical matrix is sequentially arranged; S2232;set the offset range of the corresponding feature vector on the multiple feature bit planes of the hierarchical matrix, extract the feature vectors within the offset range and place them on the multiple feature bit planes, and mark the offset of the feature vector on the feature bit plane, wherein multiple calibration points are arranged on the multiple feature bit planes, and the offset value and the corresponding direction of the feature vector are recorded through the calibration points; From the above steps S21 and S22, it can be known that the design file of the printed circuit board which needs to be detected is determined, and the design file needs to be parsed and aligned before the printed circuit board image is detected, so as to better improve the accuracy and recognition of the dynamic template generation. First, the format type (such as Gerber, ODB++, etc.) of the obtained design file is identified based on the file header information or the extension name; the pre-constructed parsing module is called based on the identified format type to parse the multi-layer theoretical structure information in the design file to obtain multiple hierarchical information, and the parsing module is parsed according to the preset rule (for example: the Gerber format file is parsed according to the RS-274X standard); further, the obtained multiple hierarchical information is respectively configured with corresponding hierarchical matrices, and the hierarchical matrix is composed of multiple feature bit planes, and the feature bit plane is a space layer of a blank database as a data page (that is, the blank database is spatially divided to obtain multiple space layers as feature bit planes, which are used for subsequent storage and display of the aligned hierarchical information); at the same time, the key alignment feature points in each layer are extracted, and the key alignment feature points include the center of the pad, the center of the via and the corner point of the figure; and the key alignment feature points are used as the hierarchical alignment reference points; the corresponding feature reference is set for the corresponding hierarchical alignment reference points, and the feature reference is a reference standard used in the hierarchical alignment process, which is mainly used to extract the feature vector from the hierarchical information of the PCB design file (for example: the feature reference can be the key geometric shape and feature point, element placement position or board edge contour in the design file, etc.); The hierarchical information is feature-extracted based on the hierarchical alignment reference points and the feature benchmarks, the feature-extraction process is used for extracting feature vectors conforming to the feature benchmarks in the hierarchical information, and the feature vectors corresponding to the multiple feature benchmarks are obtained; the feature vectors include feature profiles and corresponding initial offset values; the obtained initial offset values are continuously optimized by using an iterative least square method until the calculation result converges (i.e. the optimization process of the iterative least square method reaches a stable state), and the optimal offset parameters are obtained; the optimal offset parameters and the feature profiles are integrated to obtain first feature vectors, and the obtained multiple first feature vectors are respectively and one-to-one placed on a hierarchical matrix; further, multiple calibration points are set on a feature plane, and the first feature vectors on the hierarchical matrix are recorded based on the calibration points to obtain standardized hierarchical data; wherein the calibration points are uniformly and closely distributed on the feature plane, are used for recording offset values and corresponding directions of the corresponding feature vectors, positions of the feature vectors on the feature plane are corresponding to original positions on the hierarchical information, are finely divided, and can more finely analyze and align the hierarchical information, so that details of the analyzed printed circuit board design file are more clearly displayed, and more accurate defect detection information can be subsequently given.

[0021] Step S3: feature enhancement is performed based on the standardized hierarchical data, the hierarchical data after the enhancement is fused, and a dynamic detection template is generated based on the hierarchical data after the fusion; It needs to be further explained that, in the specific implementation process, the specific implementation process of step S3 includes the following steps: S31: corresponding enhancement rules are respectively formulated for the multiple hierarchical information, wherein the enhancement rules include a missing information completion strategy and corresponding feature enhancement difference values; S32: the calibration points in the standardized hierarchical data are regulated based on the enhancement rules to obtain the standardized hierarchical data after the regulation; S33: the hierarchical data after the regulation that meets a preset condition is taken as simulation hierarchical data, offset values and corresponding directions of the calibration points corresponding to the simulation hierarchical data are assigned to the feature vectors on the feature plane, and target hierarchical data is obtained; S34: the multiple target hierarchical data are superimposed to obtain target hierarchical information, the multiple target hierarchical information are correspondingly fused according to position relationships in the printed circuit board design file, and a dynamic detection template is constructed based on the target hierarchical information; In one embodiment, the step S32 of regulating the calibration points in the standardized hierarchical data based on the enhancement rules to obtain the hierarchical data after the simulation includes: S321: the offset range of the feature vectors and the existing feature enhancement difference values are obtained as adjustment bases according to the offset values and corresponding directions of the feature vectors recorded in the calibration points, and the corresponding enhancement rules are matched according to the adjustment bases. S322: offset simulation regulation is performed on the calibration points in the standardized hierarchical data according to the matched enhancement rule, offset load of the calibration points after simulation regulation is borne in the standardized hierarchical data, and hierarchical data after simulation is obtained; In one embodiment, the hierarchical data after regulation meeting the preset condition is taken as simulation hierarchical data, offset values and corresponding directions of the calibration points corresponding to the simulation hierarchical data are assigned to the feature vectors on the feature bit surface, and step S33 of obtaining target hierarchical data includes: S331: a preset condition is formulated, wherein the preset condition is that a difference range of profile coincidence between the hierarchical data after simulation and the standardized hierarchical data meets a preset difference threshold; S332: the hierarchical data after simulation meeting the preset condition is taken as simulation hierarchical data, offset values and corresponding directions of the calibration points corresponding to the simulation hierarchical data are adjusted to the feature vectors on the feature bit surface, and target hierarchical data is obtained.

[0022] As can be seen from the above steps S31 to S34, corresponding enhancement rules are formulated for the obtained hierarchical information, the enhancement rules include a missing information completion strategy and corresponding feature enhancement difference values, the feature enhancement difference values refer to offset amounts to be adjusted under corresponding offset ranges; and the enhancement rules exist partial differences for different hierarchical information; for example: for missing color information of the solder mask layer, the missing color information is inferred and completed by analyzing material attribute identification in the design file or industry default standards; for missing font and position information of the silk screen text, the missing font and position information is predicted and completed by using a text feature recognition algorithm based on deep learning; for missing interlayer connection relationship information, the missing interlayer connection relationship information is inferred and completed by using a network topology analysis method based on graph theory; Further, the offset value and the corresponding direction of the feature vector recorded by each calibration point are obtained, and the offset range corresponding to the first feature vector and the existing feature enhancement difference value are estimated based on the same, and they are used as the adjustment basis; based on the adjustment basis, the matching is performed from the formulated enhancement rules, the corresponding adjustment basis is matched with the enhancement rule, and the calibration points in the standardized hierarchical data are simulated and regulated based on the same, and the offset of the calibration point model is loaded in the standardized hierarchical data to obtain the hierarchical data after regulation; because the feature vector is set on the feature bit plane, the calibration point is the unit of adjustment offset, which is a single processing node, the processing node is connected with the corresponding hierarchical data, the hierarchical data is the data layer, all calibration points on the single feature bit plane are connected with each other, multiple calibration points are responsible for recording and regulating the offset of the corresponding range, and all storage ranges on the feature bit plane are covered by the calibration points. After the feature vector is laid on the feature bit plane, it can be managed by the calibration point, so that the offset of the calibration point can be directly determined to adjust the offset of the responsible feature vector; and in one hierarchical data, the feature vector is not fully covered, only the calibration point corresponding to the covered position can perform offset adjustment operation, and simulation adjustment can be performed without being directly loaded in the hierarchical data. Because the calibration point and the responsible feature vector have a corresponding relationship, the display of other ranges and offsets is the same, so the calibration point can be used to represent the hierarchical data as the simulated hierarchical data; A preset condition is set, which refers to the profile coincidence degree between the hierarchical data after simulation regulation and the standardized hierarchical data satisfying the preset difference threshold; because each calibration point is responsible for managing a certain area range, the local area may exceed the original contour line during adjustment, so it is necessary to evaluate the profile consistency between the adjusted hierarchical data and the standardized hierarchical data, and therefore, when the profile coincidence difference range after adjustment by the calibration point satisfies the preset difference threshold, it indicates that the corresponding calibration point adjustment process is effective; Further, the hierarchical data after regulation meeting the preset condition is taken as simulation hierarchical data; a first feature vector on a feature bit surface where the calibration point is located in the simulation hierarchical data is adjusted to obtain target hierarchical data, a plurality of target hierarchical data are superimposed to obtain target hierarchical information, and the plurality of target hierarchical information are correspondingly fused according to original positions thereof in the printed circuit board design file to obtain corrected multi-layer theoretical structure information; the corrected multi-layer theoretical structure information is converted into an accurate three-dimensional model, and the three-dimensional model contains geometric shape, material attribute and circuit layout information of the printed circuit board; then, according to light conditions (such as light source position, intensity and color) and camera parameters (such as focal length, viewing angle and resolution) of an actual detection environment, a printed circuit board surface image in an ideal state is simulated and generated; the corresponding printed circuit board surface image in the ideal state is accurately divided into a plurality of blocks such as a functional block, a pad block, a circuit block and a background block based on color features, texture features or morphological features; finally, according to characteristics of each block and possible defect types, corresponding detection parameters and threshold values such as brightness tolerance, edge definition and shape deviation are allocated to each block to form a dynamic detection template with regional adaptive characteristics, and the dynamic detection template can adapt to detection requirements of different regions to improve accuracy and efficiency of defect detection.

[0023] Step S4: image pre-processing is performed on the collected surface image, and the surface image after image pre-processing is combined with the dynamic detection template to perform nonlinear registration to obtain a corresponding registration image; It needs to be further explained that, in the specific implementation process, the specific implementation process of step S4 includes the following steps: Step S41: image pre-processing is performed on the collected surface image to obtain a standardized image; Step S42: multi-scale decomposition is performed on the standardized image, and a local deformation descriptor is constructed based on scale features at each scale to establish a nonlinear deformation model of the printed circuit board; Step S43: feature points in the standardized image are extracted based on an adaptive threshold segmentation algorithm, and a feature point correlation matrix is constructed based on the feature points; Step S44: the standardized image and the dynamic detection template are nonlinearly registered by adopting a thin-plate spline transformation based on the nonlinear deformation model and the feature point correlation matrix to obtain a registration image; In an embodiment of the present application, the step S41 of performing image pre-processing on the collected surface image to obtain a standardized image includes: Step S411: the surface image of the printed circuit board is subjected to grayscale processing to obtain a grayscale image; Step S412: the grayscale image is subjected to equalization processing by adopting a homomorphic filtering algorithm; Step S413: performing adaptive contrast enhancement on the image after the equalization processing according to the material characteristics of the printed circuit board, to obtain a standardized image.

[0024] In an embodiment of the present application, the step S42 of performing multi-scale decomposition on the standardized image and constructing a local deformation descriptor based on the scale features at each scale to establish a nonlinear deformation model of the printed circuit board comprises: S421: performing multi-scale decomposition on the standardized image by using wavelet transform to obtain frequency sub-band images at different scales; S422: extracting features of a direction gradient histogram corresponding to each frequency sub-band image at each scale, and constructing a local deformation descriptor based thereon, the local deformation descriptor containing material characteristic parameters of the printed circuit board; S423: establishing a nonlinear deformation model of the printed circuit board based on the local deformation descriptor and a preset nonlinear mapping function.

[0025] In an embodiment of the present application, the step S43 of extracting feature points in the standardized image based on an adaptive threshold segmentation algorithm and constructing a feature point correlation matrix based thereon comprises: S431: performing region segmentation on the standardized image based on an Otsu adaptive threshold segmentation algorithm, and extracting significant regions in the image, including solder joints, wires and component edges on the circuit board; S432: performing feature point extraction in the segmented significant regions by using a FAST corner detection algorithm; S433: screening and optimizing the extracted feature points to obtain discriminative feature points, and constructing a feature point correlation matrix based thereon.

[0026] As can be seen from the above steps S41 to S43, first, the collected surface image is converted into a grayscale image, and the grayscale image is subjected to illumination equalization processing based on a homomorphic filtering algorithm. The homomorphic filtering technology can separate the corresponding illumination component and reflection component from the corresponding grayscale image, and suppress the illumination component, thereby eliminating the influence of uneven illumination. Finally, adaptive contrast enhancement is performed according to the characteristics of different material printed circuit boards (such as FR4, ceramic substrate, etc.), to highlight the flaw features, to obtain a standardized image. The standardized image is subjected to multi-scale decomposition by using wavelet transform, to obtain frequency sub-band images at different scales. Direction gradient statistical analysis is performed on each frequency sub-band image, to extract texture features on the surface of the printed circuit board. Deformation sensitive features of local regions are constructed based on the texture features, for subsequent deformation model establishment. Further, the local deformation descriptors of the key regions in each frequency sub-band image are calculated, the descriptors containing the printed circuit board material characteristic parameters; a deformation parameter database of printed circuit boards with different materials (including FR4 and ceramic substrates) is established; based on the deformation parameter database and the local deformation descriptors, and in combination with a preset nonlinear mapping function, a machine learning algorithm (such as a support vector machine or a random forest) selected in advance is iteratively trained to construct a nonlinear deformation model that can adapt to different types of printed circuit boards; wherein the preset nonlinear mapping function is implemented by a radial basis function network to capture various nonlinear deformations that may occur in the production and use of the printed circuit board. Further, the Otsu algorithm is used to adaptively determine the image segmentation threshold, the Otsu algorithm determines the optimal threshold by maximizing the inter-class variance, and can effectively deal with the image segmentation problem under different light conditions; in the significant region obtained by segmentation, the FAST corner detection algorithm is used to detect feature points such as pad edges and line intersection points; then the local response value of the detected feature points is calculated, the local response value reflects the change intensity of the pixels around the feature point; a threshold is set to remove low-quality feature points; finally, the non-maximum suppression technique is used to optimize the feature point distribution and ensure the coverage of the feature points in the key region; the local descriptor of each feature point is obtained, the local descriptor is a feature vector that fuses gradient and texture information, and is used to describe the unique image characteristics of the region around the feature point; and in combination with the correlation strength between the feature points established by the local region similarity measurement, a feature point correlation matrix is constructed; the correlation matrix is sparsified to retain key correlation information and improve the calculation efficiency of the subsequent registration algorithm; In an embodiment of the present application, based on the nonlinear deformation model and the feature point correlation matrix, and using thin plate spline transformation, the standardized image and the dynamic detection template are nonlinearly registered to obtain a registration image in step S44, which includes: S441: based on the feature point correlation matrix, establishing a feature point correspondence relationship between the standardized image and the dynamic detection template to obtain a control point pair set; S442: based on the control point pair set and the nonlinear deformation model, constructing a thin plate spline transformation model, the thin plate spline transformation model defining a spatial mapping relationship from the standardized image to the dynamic detection template; S443: based on the thin plate spline transformation model, nonlinearly transforming the standardized image to realize accurate registration of the standardized image and the dynamic detection template, and obtaining a registration image.

[0027] As can be known from the steps S441 to S443, in establishing the feature point correspondence, a bidirectional nearest neighbor matching strategy is adopted to remove false matches and improve the robustness of registration; the thin plate spline transformation model can be used to simulate the deformation process of an elastic thin plate under external force and accurately describe the local and global deformation of the printed circuit board; the thin plate spline transformation model is composed of two parts: a global affine transformation part and a local nonlinear transformation part, the global affine transformation part is used to process the translation, rotation and scaling of the whole, and the local nonlinear transformation part is used to process the bending and twisting deformation of the local; in applying the thin plate spline transformation model, a bilinear interpolation algorithm is used to calculate the pixel mapping relationship, so as to ensure the smoothness and continuity of the transformed image; at the same time, the registered image and the dynamic detection template are highly consistent in spatial position, which provides a reliable basis for subsequent defect detection and quality evaluation; through this nonlinear registration method, various deformations that may occur in the manufacturing and use process of the printed circuit board can be effectively processed, and the accuracy and reliability of the detection system are significantly improved.

[0028] Step S5: processing the obtained registered image to obtain a defect detection and classification result; It needs to be further explained that, in the specific implementation process, the specific implementation process of step S5 includes the following steps: S51: training the pre-selected double-branch convolutional neural network through a large number of labeled surface images to obtain a trained defect detection model; S52: inputting the calibrated registered image into the defect detection model to obtain a defect detection and classification result.

[0029] As can be known from the steps S51 and S52, the double-branch convolutional neural network is trained through a large number of labeled surface images to obtain a trained defect detection model, which includes collecting a large number of surface image samples, the surface image samples coming from optical detection equipment on a printed circuit board production line, such as AOI equipment, high-precision cameras, etc.; ensuring that the images contain various types of printed circuit board materials, different line densities and various common defect types to improve the generalization ability of the model; accurately labeling the defect areas in the collected surface image samples and marking the pixel-level defect categories such as short circuit, open circuit, pinhole, copper foil residue, etc.; pre-processing the images, including normalization, image enhancement, data augmentation, etc.; wherein, normalization can make the model training more stable; image enhancement can improve the visibility of subtle defects; data augmentation (such as rotation, scaling, adding noise, etc.) can increase data diversity and improve the model's adaptability to various imaging conditions; wherein, the corresponding accurate labeling process is prior art, and the present application will not be described in detail; The pre-processed surface image sample is input into a designed double-branch convolutional neural network, one branch is responsible for feature extraction of the deformation sensitive area, and the other branch is responsible for flaw feature extraction, and finally high-precision flaw detection is realized through a feature fusion layer; the difference between the prediction result and the true label is calculated through the loss function, and the network parameters are updated through the optimization algorithm, and in the training process, the performance of the model is monitored using the validation set, and when the performance indicators on the validation set no longer improve, the training is completed to obtain the final flaw detection model; Further, the registration image is input into the flaw detection model to obtain the flaw detection and classification result, which includes: a flaw position map, which is a binary image for accurately displaying the position and range of each flaw on the printed circuit board, wherein 1 in the binary image represents a flaw area, and 0 represents a normal area; flaw classification result: each detected flaw area is classified, such as short circuit, open circuit, pinhole, copper foil residue, etc., and a confidence score of the classification is given; flaw statistical data: including the number, area, severity, etc. of various flaws, which can be used for production quality control and trend analysis; flaw feature description: key feature parameters such as shape, size, edge characteristics, etc. are extracted for each flaw area, which provides a basis for subsequent defect cause analysis.

[0030] By this method, the problems of difficulty in local deformation modeling and insufficient robustness of feature point extraction in printed circuit board detection can be effectively solved, and the accuracy and reliability of PCB surface flaw detection are improved.

[0031] Embodiment 2

[0032] Please refer to Figure 2 The embodiment not described in detail is described in the description of embodiment 1, and a printed circuit board surface flaw intelligent detection system based on a dynamic template is provided, which includes: A data acquisition module acquires a design file and a surface image of a printed circuit board to be detected, and the design file contains multi-layer theoretical structure information in the printed circuit board; A data processing module performs hierarchical parsing on the acquired design file to obtain a plurality of hierarchical information, and performs alignment processing on the plurality of hierarchical information to obtain standardized hierarchical data; A template construction module performs feature enhancement based on the standardized hierarchical data, fuses the enhanced hierarchical data, and constructs a dynamic detection template based thereon; An image correction module performs image preprocessing on the collected surface image, and performs nonlinear registration on the surface image after image preprocessing in combination with the dynamic detection template to obtain a corresponding registration image; A flaw detection module processes the obtained registration image to obtain a flaw detection and classification result.

[0033] The various modules are connected through wired and / or wireless means to achieve data transmission between the modules.

[0034] The above merely describes preferred embodiments of the present application, and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art will appreciate that the technical solutions described in the foregoing embodiments can be modified or some technical features thereof can be replaced equivalently. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall fall within the scope of protection of the present application.

[0035] It should be noted that, in this document, the terms "comprises", "comprising", or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the statement "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0036] In the description of the present application, it should be understood that the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0037] In the description of the present application, unless otherwise specified, "a plurality of" means two or more.

[0038] In the description of the present application, "several" means one or more, and "a large number" means two or more.

[0039] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0040] The formula of the present specification is a dimensionless value, and the formula is obtained by software simulation of a large amount of data to obtain a formula of the most recent real situation. The preset parameters and threshold values in the formula are set by those skilled in the art according to the actual situation.

[0041] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely divergences of the principles and application of the present application and that numerous modifications, changes, substitutions, and alterations can be made thereto without departing from the spirit and scope of the present application, which is defined by the following claims and their equivalents.

Claims

1. A method for intelligent detection of surface defects of printed circuit boards based on dynamic templates, characterized in that: include: Step S1: obtaining a design file and a surface image of a printed circuit board to be inspected, wherein the design file contains information on a multi-layer theoretical structure within the printed circuit board; Step S2: performing hierarchical parsing on the acquired design file to obtain multiple hierarchical information, and aligning the multiple hierarchical information to obtain standardized hierarchical data; Step S3: Perform feature enhancement based on the standardized hierarchical data, fuse the enhanced hierarchical data, and construct a dynamic detection template based on it; Step S4: performing image preprocessing on the collected surface image, and performing nonlinear registration on the preprocessed surface image in combination with the dynamic detection template to obtain a corresponding registered image; Step S5: Process the obtained registered image to obtain defect detection and classification results.

2. The method for intelligent detection of printed circuit board surface defects based on dynamic templates according to claim 1, characterized in that: The process of performing hierarchical parsing on the acquired design file to obtain multiple hierarchical information and aligning the multiple hierarchical information includes: The obtained design files are hierarchically parsed according to preset rules to obtain multiple hierarchical information; and the multiple hierarchical information are aligned according to the hierarchical features to obtain standardized hierarchical data.

3. The method for intelligent detection of printed circuit board surface defects based on dynamic templates according to claim 2, characterized in that: The process of aligning multiple levels of information according to their hierarchical features includes: A corresponding hierarchical matrix is ​​configured for each of the multiple hierarchical information, wherein the hierarchical matrix includes multiple feature planes; Establishing a hierarchical alignment reference point, the hierarchical alignment reference point includes a plurality of preset feature benchmarks, performing feature extraction on the hierarchical information according to the hierarchical alignment reference point, obtaining a plurality of feature vectors that meet the preset feature benchmarks, and optimizing the feature vectors to obtain a first feature vector; The plurality of first eigenvectors are placed one-to-one on the hierarchical matrix respectively, and the eigenvectors are outlined by the hierarchical matrix to obtain a plurality of standardized hierarchical data.

4. The method for intelligent detection of printed circuit board surface defects based on dynamic templates according to claim 3, characterized in that: The process of placing multiple eigenvectors one-to-one on a hierarchical matrix and outlining the eigenvectors through the hierarchical matrix includes: Placing the plurality of first eigenvectors one-to-one on the hierarchy matrix according to the priority of the hierarchy type; wherein the hierarchy matrix is ​​arranged in sequence; An offset range of corresponding eigenvectors is set on multiple eigenplanes of the hierarchical matrix, eigenvectors within the offset range are extracted and placed on multiple eigenplanes in the hierarchical matrix, and the offset of the eigenvector is marked on the eigenplanes. Multiple calibration points are set on the eigenplanes, and the offset value and corresponding direction of the eigenvector are recorded through the calibration points to obtain standardized hierarchical data.

5. The method for intelligent detection of printed circuit board surface defects based on dynamic templates according to claim 4, characterized in that: The process of enhancing features based on standardized hierarchical data, fusing the enhanced hierarchical data, and building a dynamic detection template based on it includes: Corresponding enhancement rules are formulated for multiple levels of information respectively; calibration points in the standardized level data are regulated based on the enhancement rules to obtain the regulated standardized level data; The hierarchical data after regulation that meets the preset conditions is used as the simulation hierarchical data, and the offset value and direction of the calibration point corresponding to the simulation hierarchical data are assigned to the feature vector on the feature plane to obtain the target hierarchical data; multiple target hierarchical data are overlapped to obtain the target hierarchical information, and the multiple target hierarchical information are correspondingly fused according to the positional relationship in the printed circuit board design file, and a dynamic detection template is constructed based on it.

6. The method for intelligent detection of printed circuit board surface defects based on dynamic templates according to claim 5, characterized in that: The process of regulating the calibration points in the standardized hierarchical data based on the enhancement rule to obtain the hierarchical data after simulation includes: According to the offset value and direction of the feature vector recorded in the calibration point, the offset range corresponding to the feature vector and the existing feature enhancement difference value are obtained as the adjustment base, and the corresponding enhancement rule is matched according to the adjustment base; Performing offset simulation control on the calibration points in the standardized hierarchical data according to the matched enhancement rules, and applying the offset load of the calibration point simulation control to the standardized hierarchical data to obtain the hierarchical data after simulation; At the same time, the hierarchical data after the control that meets the preset conditions is used as the simulated hierarchical data, and the offset value and the corresponding direction of the calibration point corresponding to the simulated hierarchical data are assigned to the feature vector on the feature plane. The process of obtaining the target hierarchical data includes: Formulate a preset condition, where the preset condition is that the difference range of the contour overlap between the simulated hierarchical data and the standardized hierarchical data meets a preset difference threshold; The hierarchical data after simulation that meets the preset conditions is used as the simulated hierarchical data, and the offset value of the calibration point corresponding to the simulated hierarchical data and the eigenvector on the corresponding direction corresponding feature plane are adjusted to obtain the target hierarchical data.

7. The method for intelligent detection of printed circuit board surface defects based on dynamic templates according to claim 5, characterized in that: The process of performing image preprocessing on the collected surface image and performing nonlinear registration on the preprocessed surface image in combination with the dynamic detection template includes: performing image preprocessing on the acquired surface image to obtain a standardized image; The standardized image is decomposed into multiple scales, and local deformation descriptors are constructed based on the scale features at each scale to establish a nonlinear deformation model of the printed circuit board. Extract feature points in the standardized image and construct a feature point correlation matrix based on them; Based on the nonlinear deformation model and feature point correlation matrix, the standardized image and the dynamic detection template are nonlinearly registered using thin plate spline transformation to obtain the registered image.

8. The method for intelligent detection of printed circuit board surface defects based on dynamic templates according to claim 7, characterized in that: The process of performing multi-scale decomposition on the standardized image and constructing a local deformation descriptor based on the scale features at each scale to establish a nonlinear deformation model of the printed circuit board includes: Wavelet transform is used to perform multi-scale decomposition on the standardized image to obtain frequency sub-band images at several different scales; Extract the features of the oriented gradient histogram corresponding to the frequency subband image at each scale, and construct a local deformation descriptor based on it. The local deformation descriptor contains the material characteristic parameters of the printed circuit board; Based on the local deformation descriptor and the preset nonlinear mapping function, a nonlinear deformation model of the printed circuit board is established; Thin plate spline transform is used to perform nonlinear registration between the standardized image and the dynamic detection template. The process of obtaining the registered image includes: Based on the feature point association matrix, the feature point correspondence relationship is established between the standardized image and the dynamic detection template to obtain a set of control point pairs; Based on the set of control point pairs and the nonlinear deformation model, a thin plate spline transformation model is constructed; The standardized image is nonlinearly transformed based on the thin plate spline transformation model to obtain the registered image.

9. The method for intelligent detection of printed circuit board surface defects based on dynamic templates according to claim 8, characterized in that: The process of processing the obtained registered images to obtain defect detection and classification results includes: A pre-selected two-branch convolutional neural network is trained using a large number of labeled surface images to obtain a trained defect detection model. The calibrated registered image is input into the defect detection model to obtain the defect detection and classification results.

10. A system for intelligently detecting surface defects of printed circuit boards based on a dynamic template, for implementing the method for intelligently detecting surface defects of printed circuit boards based on a dynamic template according to any one of claims 1 to 9, comprising: A data acquisition module is configured to acquire a design file and a surface image of a printed circuit board to be inspected, wherein the design file includes information on a multi-layer theoretical structure within the printed circuit board; The data processing module performs hierarchical analysis on the acquired design files to obtain multiple hierarchical information, and aligns the multiple hierarchical information to obtain standardized hierarchical data; The template construction module performs feature enhancement based on standardized hierarchical data, fuses the enhanced hierarchical data, and constructs a dynamic detection template based on it; The image correction module performs image preprocessing on the collected surface image, and performs nonlinear registration on the surface image after image preprocessing in combination with the dynamic detection template to obtain the corresponding registered image; The defect detection module processes the obtained registered images to obtain defect detection and classification results.

Images