A method and system for detecting defects in a flexible circuit board

Through multi-spectral imaging and deep learning technology, a high-resolution three-dimensional model is built, which solves the instability and complex texture interference problems of flexible circuit board defect detection under changes in lighting conditions, and achieves accurate defect detection.

CN118644483BActive Publication Date: 2024-11-15SHENZHEN CHANGFENG LASER SWORD MOULD CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411111176.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-14
Publication Date
2024-11-15
Estimated Expiration
2044-08-14

AI Technical Summary

Technical Problem

The existing flexible circuit board defect detection methods are sensitive to lighting conditions, difficult to detect complex textures and tiny defects, and the detection results are unstable under different lighting conditions.

Method used

Multi-channel image sequences are collected using a multi-spectral light source array, multi-spectral image cubes are constructed, material area segmentation and multi-feature image fusion are performed, high-resolution three-dimensional surface model is obtained, and defect identification and positioning is performed through multi-scale feature extraction and deep learning models.

Benefits of technology

It reduces the impact of light changes, overcomes complex texture interference, enhances the detection ability of tiny defects, and realizes accurate defect detection in complex lighting environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118644483B_ABST
    Figure CN118644483B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of flexible circuit board detection technology, and in particular to a method and system for detecting defects in flexible circuit boards. The method comprises the following steps: acquiring a multi-channel image sequence of a flexible circuit board according to a multi-spectral light source array to obtain a multi-channel image sequence; constructing a multi-spectral image cube for the multi-channel image sequence to obtain a multi-spectral image cube; performing material region segmentation according to the multi-spectral image cube to obtain a material region map; performing multi-feature image fusion according to the material region map to obtain a spectral feature map; and acquiring multi-angle photometric images of the flexible circuit board to obtain a multi-angle photometric image sequence. The present invention reduces the influence of illumination changes through multi-spectral imaging, overcomes the interference of complex textures through material region segmentation and spectral feature extraction, enhances the detection capability of tiny defects through three-dimensional surface reconstruction and multi-scale feature fusion, and finally realizes accurate defect detection in complex lighting environments.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of flexible circuit board detection, and in particular to a flexible circuit board defect detection method and system. Background Art

[0002] Flexible printed circuits (FPCBs) are increasingly used in electronic products due to their thinness, lightness, and bendability. However, during the production process, FPCBs are prone to various defects, such as line breakage, short circuits, and foreign matter, which can seriously affect the performance and reliability of the product. Existing methods are sensitive to lighting conditions and are easily affected by changes in light. Machine vision relies on the analysis of image information, and changes in lighting conditions directly affect the quality of the image. For example, too strong light can cause the image to be overexposed and details to be lost; too weak light can cause the image to be dim and the signal-to-noise ratio to be reduced. In addition, changes in ambient light sources and the presence of shadows can interfere with the image and affect the identification of defects. Under different lighting conditions, the detection results of the same defect can vary greatly, resulting in unreliable detection results. Existing methods are difficult to detect complex textures and tiny defects. The surface of FPCBs usually has complex textures, such as copper foil, covering layers, silk-screen characters, etc. These textures can interfere with the identification of defects. In addition, some tiny defects, such as fine cracks and burrs, are very small in size and only occupy a small area in the image, making it difficult for the algorithm to accurately identify them. Summary of the invention

[0003] Based on this, it is necessary to provide a flexible circuit board defect detection method and system to solve at least one of the above technical problems.

[0004] To achieve the above object, a method for detecting defects in a flexible circuit board comprises the following steps:

[0005] Step S1: acquiring a multi-channel image sequence of the flexible circuit board according to the multi-spectral light source array to obtain a multi-channel image sequence; constructing a multi-spectral image cube for the multi-channel image sequence to obtain a multi-spectral image cube;

[0006] Step S2: performing material region segmentation according to the multispectral image cube to obtain a material region map; performing multi-feature image fusion according to the material region map to obtain a spectral feature map;

[0007] Step S3: performing multi-angle photometric image acquisition on the flexible circuit board to obtain a multi-angle photometric image sequence; performing spectral feature height registration map extraction based on the multi-angle photometric image sequence to obtain a spectral feature height registration map; using the spectral feature height registration map to construct a material surface texture database for the material area map to obtain a material surface texture database; performing high-resolution model optimization based on the material surface texture database to obtain a high-resolution three-dimensional surface model;

[0008] Step S4: constructing a curvature feature map pyramid for the high-resolution three-dimensional surface model to obtain a curvature feature map pyramid; performing multi-scale feature fusion on the curvature feature map pyramid to obtain a multi-scale fusion feature map pyramid; performing multi-scale feature aggregation on the multi-scale fusion feature map pyramid to obtain a multi-scale feature pyramid;

[0009] Step S5: performing multi-scale feature encoding layer by layer on the multi-scale feature pyramid to obtain a multi-scale feature encoding pyramid; performing global feature pooling on the multi-scale feature encoding pyramid to obtain a global feature vector; performing nonlinear transformation on the global feature vector to obtain a depth defect feature vector;

[0010] Step S6: Generate a class activation map according to the deep defect feature vector to obtain a class activation map; perform defect data fusion on the class activation map to obtain a defect classification result and a positioning map;

[0011] Step S7: Segment and extract the defect area according to the defect classification result and the positioning map as well as the high-resolution three-dimensional surface model positioning map to obtain a three-dimensional defect model; superimpose three-dimensional defect information on the three-dimensional defect model to obtain a three-dimensional defect detection model; use the three-dimensional defect detection model to perform flexible circuit board defect detection analysis to obtain flexible circuit board defect detection data.

[0012] The present invention collects multi-channel image sequences through a multi-spectral light source array and constructs a multi-spectral image cube, which can obtain rich spectral information of the flexible circuit board and provide a more comprehensive data basis for subsequent material analysis and defect detection. Through material region segmentation and multi-feature image fusion through a multi-spectral image cube, different material regions can be distinguished, and spectral feature maps that highlight material differences can be generated, providing more accurate background information for subsequent defect detection. Through multi-angle photometric image acquisition and spectral feature height registration map extraction, the three-dimensional morphological information of the flexible circuit board can be obtained, and accurately associated with the spectral information, a high-resolution three-dimensional surface model containing rich texture information can be constructed, providing a more intuitive visual expression and a more accurate positioning basis for subsequent defect detection. By performing multi-scale feature extraction and fusion on the high-resolution three-dimensional surface model, the geometric shape and texture information of the flexible circuit board can be fully described, and a multi-scale feature pyramid containing rich detail information can be constructed, providing a more discriminative feature input for the subsequent deep learning model. By encoding the multi-scale feature pyramid layer by layer, global feature pooling and nonlinear transformation, a more abstract, compact and semantically informative deep defect feature vector can be extracted, providing a more effective feature representation for subsequent defect classification and positioning. Through the classification prediction of the deep learning model, the generation and mapping of the category activation map and the optimization of defect positioning, the defect category can be accurately identified and the defect area can be accurately located, providing a reliable data basis for subsequent defect analysis and processing. By mapping the defect information to a three-dimensional model, and performing three-dimensional defect modeling, feature calculation, information superposition and analysis, three-dimensional visualization, quantitative analysis and comprehensive evaluation of flexible circuit board defects can be achieved, providing a more powerful and efficient technical means for the quality control and defect management of flexible circuit boards. Therefore, the present invention provides a flexible circuit board defect detection method and system. The influence of illumination changes is reduced by multi-spectral imaging, the interference of complex textures is overcome by material region segmentation and spectral feature extraction, and the detection ability of tiny defects is enhanced by three-dimensional surface reconstruction and multi-scale feature fusion, and finally accurate defect detection in complex lighting environments is achieved.

[0013] Preferably, step S1 comprises the following steps:

[0014] Step S11: performing LED light source lighting control on the flexible circuit board through a multi-spectrum light source array to obtain a multi-spectrum lighting matrix;

[0015] Step S12: acquiring a multi-channel image sequence of the flexible circuit board using an industrial camera according to a multi-spectral illumination matrix to obtain a multi-channel image sequence;

[0016] Step S13: performing spectral response correction on the multi-channel image sequence to obtain spectral response correction data; performing accurate image registration on the multi-channel image sequence according to the spectral response correction data to obtain a multi-channel image correction and registration sequence;

[0017] Step S14: construct a multispectral image cube for the multi-channel image correction and registration sequence according to the band order to obtain a multispectral image cube.

[0018] The present invention controls the LED light source illumination of the flexible circuit board through a multi-spectral light source array, can flexibly adjust the spectral illumination conditions for different materials and defect types, form a multi-spectral illumination matrix, and provide richer and more targeted spectral information for subsequent multi-channel image acquisition, thereby improving the accuracy and reliability of defect detection. The industrial camera collects multi-channel image sequences of the flexible circuit board according to the multi-spectral illumination matrix, and can obtain image information of the flexible circuit board under different spectral illumination conditions, and provide multi-dimensional image data for subsequent spectral analysis and feature extraction, thereby improving the ability to distinguish different materials and defect types. Spectral response correction and image precise registration of the multi-channel image sequence can eliminate the influence of uneven spectral distribution of the camera sensor and the light source, and ensure that the image information of each pixel in different spectral channels is aligned, thereby improving the quality and accuracy of the multi-spectral image. The multi-spectral image cube is constructed according to the band order for the multi-channel image correction and registration sequence, and the multi-channel two-dimensional image sequence can be integrated into a three-dimensional data cube containing rich spectral information, which provides a more intuitive and comprehensive data expression form for subsequent material analysis, feature extraction and three-dimensional modeling, thereby improving the efficiency and accuracy of defect detection.

[0019] Preferably, step S2 comprises the following steps:

[0020] Step S21: constructing a multispectral pixel vector according to the multispectral image cube to obtain a multispectral pixel vector data set; performing spectral dimension reduction on the multispectral pixel vector data set to obtain a multispectral pixel vector dimension reduction data set;

[0021] Step S22: performing spectral clustering analysis on the multi-spectral pixel vector dimension reduction data set to obtain a spectral clustering result graph; performing clustering result spatial smoothing on the spectral clustering result graph to obtain a spectral clustering result smoothing graph;

[0022] Step S23: extracting the material region boundary from the spectral clustering result smoothing map to obtain a material region boundary map;

[0023] Step S24: performing material region segmentation on the spectral clustering result smoothing map according to the material region boundary map to obtain a material region map; performing spectral characteristic curve fitting on the material region map to obtain a material region spectral characteristic curve;

[0024] Step S25: Select characteristic bands for the material region spectral characteristic curve to obtain a characteristic band set; extract corresponding band images from the multispectral image cube according to the characteristic band set and construct characteristic images to obtain a characteristic image set;

[0025] Step S26: performing multi-feature image fusion on the feature image set to obtain a spectral feature map.

[0026] The present invention constructs a multispectral image cube into a multispectral pixel vector data set and performs spectral dimension reduction, which can effectively extract the spectral feature information of each pixel point and reduce data redundancy, providing a more concise and efficient data expression form for subsequent spectral clustering analysis. Performing spectral clustering analysis on the multispectral pixel vector dimension reduction data set and spatially smoothing the clustering results can classify pixels with similar spectral features into one category and effectively suppress the influence of noise and isolated points, thereby obtaining a more accurate and smoother material region division result. Extracting the material region boundary of the smoothed graph of the spectral clustering result can accurately locate the boundary between different material regions, providing accurate guidance information for subsequent material region segmentation, thereby improving the accuracy and reliability of the segmentation result. Performing material region segmentation on the smoothed graph of the spectral clustering result according to the material region boundary graph, and fitting the spectral feature curve for each material region, the image can be segmented into different material regions, and the spectral feature curve of each region can be extracted, providing basic data for subsequent feature band selection and feature image construction. Selecting characteristic bands for the spectral characteristic curves of the material area, and extracting the images of the corresponding bands based on the characteristic band set to construct a characteristic image set, can select the characteristic bands that can best distinguish different materials, and construct a characteristic image set containing richer information, thereby improving the accuracy and reliability of subsequent defect detection. Performing multi-feature image fusion on the characteristic image set to obtain a spectral characteristic map can effectively fuse the information of multiple characteristic bands into one image, highlighting the differences between different material areas, and providing more intuitive and valuable image information for subsequent defect detection.

[0027] Preferably, step S3 comprises the following steps:

[0028] Step S31: performing multi-angle photometric image acquisition on the flexible circuit board to obtain a multi-angle photometric image sequence; performing spectral feature height registration map extraction according to the multi-angle photometric image sequence to obtain a spectral feature height registration map;

[0029] Step S32: Optimizing the material region segmentation of the material region map using the spectral feature height registration map to obtain a material region segmentation optimization map;

[0030] Step S33: extracting height and spectral feature information of different material regions in the material region segmentation optimization map to obtain height and spectral feature parameters; constructing a material surface texture database according to the height and spectral feature parameters to obtain a material surface texture database;

[0031] Step S34: constructing a three-dimensional mesh model using the spectral feature height registration map to obtain a three-dimensional mesh model of the surface of the flexible circuit board;

[0032] Step S35: performing texture mapping on the three-dimensional mesh model of the surface of the flexible circuit board according to the material surface texture database to obtain a texture-mapped three-dimensional mesh model;

[0033] Step S36: Optimize the texture mapping three-dimensional mesh model to obtain a high-resolution three-dimensional surface model.

[0034] The present invention can obtain the three-dimensional morphological information and spectral information of the surface of the flexible circuit board through multi-angle photometric image acquisition and spectral feature height registration map extraction, and accurately associate the two to provide basic data for subsequent material area segmentation optimization, texture database construction and three-dimensional model construction. The material area map is optimized for material area segmentation using the spectral feature height registration map, and the material area boundary can be more accurately located in combination with the height information, improving the accuracy of material area segmentation, and providing more reliable regional division results for subsequent feature extraction and three-dimensional modeling. The height and spectral feature information of different material areas in the material area segmentation optimization map is extracted, and a material surface texture database is constructed. The height feature and spectral feature parameters of each material area can be extracted and stored in the database, providing rich texture information for subsequent three-dimensional model texture mapping, and improving the realism and detail expression of the three-dimensional model. The three-dimensional mesh model is constructed using the spectral feature height registration map, and the initial three-dimensional mesh model of the surface of the flexible circuit board can be constructed according to the height information and camera parameters. According to the material surface texture database, the texture mapping of the three-dimensional mesh model of the flexible circuit board surface can map the texture information of the material to the three-dimensional model, so that the model can truly reflect the visual differences of different material areas, and improve the realism and visualization of the model. The high-resolution model optimization of the texture-mapped three-dimensional mesh model can smooth the model and enhance the details, improve the resolution and accuracy of the model, and enable the model to more finely express the morphology and texture details of the flexible circuit board surface.

[0035] Preferably, step S4 comprises the following steps:

[0036] Step S41: constructing a Gaussian pyramid on the high-resolution three-dimensional surface model to obtain a Gaussian pyramid;

[0037] Step S42: Calculate the vertex normal direction of the Gaussian pyramid to obtain vertex normal direction data; perform range mapping according to the vertex normal direction data and construct a normal direction map pyramid to obtain a normal direction map pyramid;

[0038] Step S43: calculating the shape index according to the normal direction map pyramid and constructing the shape index map pyramid to obtain the shape index map pyramid;

[0039] Step S44: extracting curvature features from the shape index graph pyramid and constructing a curvature feature graph pyramid to obtain a curvature feature graph pyramid;

[0040] Step S45: performing two-dimensional plane mapping on the high-resolution three-dimensional surface model and extracting texture features to obtain a texture feature map pyramid;

[0041] Step S46: performing multi-scale feature fusion on the curvature feature map pyramid and the texture feature map pyramid to obtain a multi-scale fused feature map pyramid;

[0042] Step S47: perform local feature description on the multi-scale fusion feature map pyramid to obtain a local feature description map pyramid; perform saliency feature enhancement on the local feature description map pyramid to obtain a saliency enhanced feature map pyramid; perform multi-scale feature aggregation on the saliency enhanced feature map pyramid to obtain a multi-scale feature pyramid.

[0043] The present invention can generate a series of model versions with gradually reduced resolutions while retaining the original high-resolution three-dimensional surface model detail information by constructing a Gaussian pyramid, which is conducive to multi-scale feature extraction and improves the algorithm's ability to detect defects of different sizes. By calculating the normal vector direction of each vertex and constructing a normal direction map pyramid, the surface orientation information of the three-dimensional model can be converted into an image form, which is convenient for subsequent feature extraction and analysis using image processing technology. By calculating the shape index according to the normal direction map pyramid and constructing a shape index map pyramid, the local concave and convex degree of the surface can be quantified, which is helpful for identifying and distinguishing different types of surface defects. By extracting curvature features from the shape index map pyramid and constructing a curvature feature map pyramid, the curvature degree of the surface can be further described, which is helpful for detecting and identifying defects such as cracks and wrinkles on the surface. By mapping the high-resolution three-dimensional surface model to a two-dimensional plane and extracting texture features, and constructing a texture feature map pyramid, the texture information of the surface can be converted from three-dimensional space to a two-dimensional plane, which is convenient for feature extraction and analysis using mature image processing technology. The multi-scale feature fusion of the curvature feature map pyramid and the texture feature map pyramid can effectively combine the surface geometry information and texture information to obtain a feature representation containing richer surface information, thereby improving the accuracy and robustness of subsequent defect detection. The local feature description, saliency feature enhancement, and multi-scale feature aggregation of the multi-scale fusion feature map pyramid can further enhance the feature information related to defects, suppress the influence of background noise, and construct the final multi-scale feature pyramid to provide more discriminative feature input for subsequent deep learning models.

[0044] Preferably, step S5 comprises the following steps:

[0045] Step S51: performing data enhancement on the multi-scale feature pyramid to obtain a data enhanced feature pyramid;

[0046] Step S52: inputting the data enhancement feature pyramid into the pre-trained deep feature extractor model, encoding the multi-scale features layer by layer, and obtaining a multi-scale feature encoding pyramid;

[0047] Step S53: performing an attention mechanism introduction analysis on the multi-scale feature coding pyramid to obtain an attention enhancement feature pyramid; performing feature graph fusion on the attention enhancement feature pyramid to obtain a fused feature pyramid;

[0048] Step S54: performing global feature pooling on the fused feature pyramid to obtain a global feature vector; performing feature vector dimension reduction on the global feature vector to obtain a global feature dimension reduction vector;

[0049] Step S55: performing defect feature enhancement on the global feature dimension reduction vector to obtain a defect feature enhancement vector; performing defect differentiation optimization on the defect feature enhancement vector to obtain a defect feature optimization vector;

[0050] Step S56: Input the defect feature optimization vector into the fully connected layer and perform nonlinear transformation to obtain a deep defect feature vector.

[0051] The present invention can increase the quantity and diversity of training data by performing data enhancement on the multi-scale feature pyramid, by rotating, translating, scaling and other operations on the existing data, thereby improving the generalization ability of the deep learning model and reducing the risk of overfitting. The data enhancement feature pyramid is input into the pre-trained deep feature extractor model, and the multi-scale features are encoded layer by layer. The powerful feature extraction ability of the pre-trained model can be used to map the feature information of different scales to a more abstract and semantically informative feature space, providing a more discriminative feature representation for subsequent defect detection. The introduction of the attention mechanism to the multi-scale feature encoding pyramid and the fusion of feature maps can make the model pay more attention to the key features related to the defects, suppress the interference of background noise and irrelevant information, thereby improving the sensitivity and detection accuracy of the model to defects. Global feature pooling and feature vector dimensionality reduction are performed on the fused feature pyramid, which can aggregate the multi-scale and multi-channel feature information into a compact global feature vector, remove redundant information, and provide a more concise and effective feature input for the subsequent classifier. Defect feature enhancement and defect differentiation optimization of the global feature dimensionality reduction vector can further highlight the feature information related to defects and optimize the feature weights, making the feature vector more conducive to distinguishing different types of defects, thereby improving the accuracy of subsequent defect classification. The defect feature optimization vector is input into the fully connected layer and nonlinearly transformed to obtain a deep defect feature vector, which can map the feature information to a higher-dimensional feature space, enhance the nonlinear expression ability of the feature, and provide a more accurate and robust feature representation for subsequent defect classification and positioning.

[0052] Preferably, step S6 comprises the following steps:

[0053] Step S61: inputting the depth defect feature vector into the pre-trained multi-classifier model to perform classification prediction to obtain defect category probability distribution data;

[0054] Step S62: using a preset confidence threshold to determine the defect category of the defect category probability distribution data, and obtaining defect classification result data;

[0055] Step S63: generating a class activation map according to the defect classification result data to obtain a class activation map;

[0056] Step S64: Mapping the class activation map to a multi-scale feature pyramid to accurately locate defects and obtain a multi-scale defect location map;

[0057] Step S65: optimizing the defect area boundary of the multi-scale defect location map to obtain a defect location optimization map;

[0058] Step S66: performing defect data fusion on the defect location optimization map and the defect classification result data to obtain the defect classification result and the location map.

[0059] The present invention inputs the deep defect feature vector into the pre-trained multi-classifier model to perform classification prediction. The trained model can be used to classify the defect features and output the probability distribution data corresponding to each category, providing a basis for subsequent defect category determination. The defect category probability distribution data is screened using a preset confidence threshold, and the classification results with low confidence can be filtered out, and the results with high confidence are retained as the final defect category determination results, thereby improving the reliability of the classification results. According to the defect classification result data, a category activation map is generated, and the classification decision process of the model can be visualized, which feature areas contribute more to the final classification results intuitively, providing guidance information for subsequent defect location. The category activation map is mapped to a multi-scale feature pyramid for accurate defect location. The feature area information provided by the category activation map can be used to perform weighted fusion on feature maps of different scales, thereby achieving accurate location of the defect. The defect area boundary optimization of the multi-scale defect location map can be performed, and the defect area boundary can be refined using an image segmentation algorithm to obtain a more accurate and smoother defect boundary, thereby improving the accuracy of defect location. By fusing the defect location optimization map and the defect classification result data, it is possible to combine the defect category information and precise positioning information to obtain defect classification results and location maps containing rich information, providing more comprehensive data support for subsequent defect analysis and processing.

[0060] Preferably, step S7 comprises the following steps:

[0061] Step S71: Mapping the defect classification result and the positioning map to a high-resolution three-dimensional surface model, performing three-dimensional positioning of the defect area, and obtaining three-dimensional positioning defect area data;

[0062] Step S72: segmenting and extracting defect regions from the high-resolution three-dimensional surface model according to the three-dimensional positioning defect region data to obtain a defect three-dimensional model;

[0063] Step S73: Calculate the defect surface characteristics of the defect three-dimensional model to obtain defect surface characteristic data;

[0064] Step S74: Mapping the defect classification result and the positioning map to the defect three-dimensional model, performing defect category label matching, and obtaining a category label defect three-dimensional model;

[0065] Step S75: superimposing three-dimensional defect information on the category label defect three-dimensional model according to the defect surface feature data to obtain a three-dimensional defect detection model;

[0066] Step S76: using the three-dimensional defect detection model to perform flexible circuit board defect detection analysis to obtain flexible circuit board defect detection data.

[0067] The present invention maps the defect classification results and the positioning map to the high-resolution three-dimensional surface model to perform three-dimensional positioning of the defect area, and can correspond the defect information on the two-dimensional image with the real position in the three-dimensional space, providing accurate spatial positioning information for subsequent three-dimensional modeling and analysis of the defect. According to the three-dimensional positioning defect area data, the high-resolution three-dimensional surface model is segmented and extracted, and the defect area can be separated from the complete three-dimensional model to construct an independent defect three-dimensional model, providing a more intuitive and more accurate object for subsequent defect feature calculation and analysis. The defect surface feature calculation of the defect three-dimensional model can extract richer defect geometric feature information, such as the volume, area, depth, curvature, etc. of the defect. The defect classification results and the positioning map are mapped to the defect three-dimensional model, and the defect category label matching is performed, so that the defect category information on the two-dimensional image can be corresponded with the defect area in the three-dimensional model, providing a more intuitive and easier to understand information expression method for subsequent defect visualization and analysis. According to the defect surface feature data, the three-dimensional defect information is superimposed on the category label defect three-dimensional model, which can present the defect category, size, shape, location and other information in a more intuitive and easier to understand three-dimensional way, which is convenient for subsequent defect analysis. Using the three-dimensional defect detection model to perform defect detection and analysis on flexible circuit boards can provide a more comprehensive and in-depth analysis of defects, such as counting the number, size distribution, spatial position relationship of different types of defects, etc., to provide more accurate and reliable data support for quality control, defect diagnosis and maintenance of flexible circuit boards.

[0068] Preferably, the present invention further provides a flexible circuit board defect detection system for executing the flexible circuit board defect detection method as described above, the flexible circuit board defect detection system comprising:

[0069] The multi-spectral fusion imaging module is used to collect a multi-channel image sequence of the flexible circuit board according to the multi-spectral light source array to obtain a multi-channel image sequence; and construct a multi-spectral image cube for the multi-channel image sequence to obtain a multi-spectral image cube;

[0070] The spectral feature extraction module is used to perform material region segmentation based on the multi-spectral image cube to obtain a material region map; perform multi-feature image fusion based on the material region map to obtain a spectral feature map;

[0071] The surface texture reconstruction module is used to collect multi-angle photometric images of the flexible circuit board to obtain a multi-angle photometric image sequence; extract the spectral feature height registration map according to the multi-angle photometric image sequence to obtain a spectral feature height registration map; use the spectral feature height registration map to construct a material surface texture database for the material area map to obtain a material surface texture database; optimize the high-resolution model according to the material surface texture database to obtain a high-resolution three-dimensional surface model;

[0072] The multi-scale feature pyramid module is used to construct a curvature feature map pyramid for a high-resolution three-dimensional surface model to obtain a curvature feature map pyramid; to perform multi-scale feature fusion on the curvature feature map pyramid to obtain a multi-scale fused feature map pyramid; to perform multi-scale feature aggregation on the multi-scale fused feature map pyramid to obtain a multi-scale feature pyramid;

[0073] The deep defect feature learning module is used to encode the multi-scale features of the multi-scale feature pyramid layer by layer to obtain the multi-scale feature encoding pyramid; perform global feature pooling on the multi-scale feature encoding pyramid to obtain the global feature vector; perform nonlinear transformation on the global feature vector to obtain the deep defect feature vector;

[0074] The defect classification and positioning module is used to generate a class activation map based on the deep defect feature vector to obtain the class activation map; the defect data is fused on the class activation map to obtain the defect classification result and positioning map;

[0075] The three-dimensional defect detection module is used to segment and extract the defect area according to the defect classification results and positioning map as well as the high-resolution three-dimensional surface model positioning map to obtain a three-dimensional defect model; to superimpose three-dimensional defect information on the defect three-dimensional model to obtain a three-dimensional defect detection model; and to use the three-dimensional defect detection model to perform flexible circuit board defect detection analysis to obtain flexible circuit board defect detection data.

[0076] The present invention collects multi-channel image sequences through a multi-spectral light source array and constructs a multi-spectral image cube, which can obtain rich spectral information of the flexible circuit board and provide a more comprehensive data basis for subsequent material analysis and defect detection. Through material region segmentation and multi-feature image fusion through a multi-spectral image cube, different material regions can be distinguished, and spectral feature maps that highlight material differences can be generated, providing more accurate background information for subsequent defect detection. Through multi-angle photometric image acquisition and spectral feature height registration map extraction, the three-dimensional morphological information of the flexible circuit board can be obtained, and accurately associated with the spectral information, a high-resolution three-dimensional surface model containing rich texture information can be constructed, providing a more intuitive visual expression and a more accurate positioning basis for subsequent defect detection. By performing multi-scale feature extraction and fusion on the high-resolution three-dimensional surface model, the geometric shape and texture information of the flexible circuit board can be fully described, and a multi-scale feature pyramid containing rich detail information can be constructed, providing a more discriminative feature input for the subsequent deep learning model. By encoding the multi-scale feature pyramid layer by layer, global feature pooling and nonlinear transformation, a more abstract, compact and semantically informative deep defect feature vector can be extracted, providing a more effective feature representation for subsequent defect classification and positioning. Through the classification prediction of the deep learning model, the generation and mapping of the category activation map and the optimization of defect positioning, the defect category can be accurately identified and the defect area can be accurately located, providing a reliable data basis for subsequent defect analysis and processing. By mapping the defect information to a three-dimensional model, and performing three-dimensional defect modeling, feature calculation, information superposition and analysis, three-dimensional visualization, quantitative analysis and comprehensive evaluation of flexible circuit board defects can be achieved, providing a more powerful and efficient technical means for the quality control and defect management of flexible circuit boards. Therefore, the present invention provides a flexible circuit board defect detection method and system. The influence of illumination changes is reduced by multi-spectral imaging, the interference of complex textures is overcome by material region segmentation and spectral feature extraction, and the detection ability of tiny defects is enhanced by three-dimensional surface reconstruction and multi-scale feature fusion, and finally accurate defect detection in complex lighting environments is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 A schematic diagram of the steps of a method for detecting defects in a flexible circuit board;

[0078] Figure 2 for Figure 1 Detailed implementation steps of step S3 in FIG.

[0079] Figure 3 for Figure 1 Detailed implementation steps of step S5 are shown in the flowchart.

[0080] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0081] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0082] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0083] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0084] To achieve this, please refer to Figures 1 to 3 , a method for detecting defects in a flexible circuit board, comprising the following steps:

[0085] Step S1: acquiring a multi-channel image sequence of the flexible circuit board according to the multi-spectral light source array to obtain a multi-channel image sequence; constructing a multi-spectral image cube for the multi-channel image sequence to obtain a multi-spectral image cube;

[0086] Step S2: performing material region segmentation according to the multispectral image cube to obtain a material region map; performing multi-feature image fusion according to the material region map to obtain a spectral feature map;

[0087] Step S3: performing multi-angle photometric image acquisition on the flexible circuit board to obtain a multi-angle photometric image sequence; performing spectral feature height registration map extraction based on the multi-angle photometric image sequence to obtain a spectral feature height registration map; using the spectral feature height registration map to construct a material surface texture database for the material area map to obtain a material surface texture database; performing high-resolution model optimization based on the material surface texture database to obtain a high-resolution three-dimensional surface model;

[0088] Step S4: constructing a curvature feature map pyramid for the high-resolution three-dimensional surface model to obtain a curvature feature map pyramid; performing multi-scale feature fusion on the curvature feature map pyramid to obtain a multi-scale fusion feature map pyramid; performing multi-scale feature aggregation on the multi-scale fusion feature map pyramid to obtain a multi-scale feature pyramid;

[0089] Step S5: performing multi-scale feature encoding layer by layer on the multi-scale feature pyramid to obtain a multi-scale feature encoding pyramid; performing global feature pooling on the multi-scale feature encoding pyramid to obtain a global feature vector; performing nonlinear transformation on the global feature vector to obtain a depth defect feature vector;

[0090] Step S6: Generate a class activation map according to the deep defect feature vector to obtain a class activation map; perform defect data fusion on the class activation map to obtain a defect classification result and a positioning map;

[0091] Step S7: Segment and extract the defect area according to the defect classification result and the positioning map as well as the high-resolution three-dimensional surface model positioning map to obtain a three-dimensional defect model; superimpose three-dimensional defect information on the three-dimensional defect model to obtain a three-dimensional defect detection model; use the three-dimensional defect detection model to perform flexible circuit board defect detection analysis to obtain flexible circuit board defect detection data.

[0092] The present invention collects multi-channel image sequences through a multi-spectral light source array and constructs a multi-spectral image cube, which can obtain rich spectral information of the flexible circuit board and provide a more comprehensive data basis for subsequent material analysis and defect detection. Through material region segmentation and multi-feature image fusion through a multi-spectral image cube, different material regions can be distinguished, and spectral feature maps that highlight material differences can be generated, providing more accurate background information for subsequent defect detection. Through multi-angle photometric image acquisition and spectral feature height registration map extraction, the three-dimensional morphological information of the flexible circuit board can be obtained, and accurately associated with the spectral information, a high-resolution three-dimensional surface model containing rich texture information can be constructed, providing a more intuitive visual expression and a more accurate positioning basis for subsequent defect detection. By performing multi-scale feature extraction and fusion on the high-resolution three-dimensional surface model, the geometric shape and texture information of the flexible circuit board can be fully described, and a multi-scale feature pyramid containing rich detail information can be constructed, providing a more discriminative feature input for the subsequent deep learning model. By encoding the multi-scale feature pyramid layer by layer, global feature pooling and nonlinear transformation, a more abstract, compact and semantically informative deep defect feature vector can be extracted, providing a more effective feature representation for subsequent defect classification and positioning. Through the classification prediction of the deep learning model, the generation and mapping of the category activation map and the optimization of defect positioning, the defect category can be accurately identified and the defect area can be accurately located, providing a reliable data basis for subsequent defect analysis and processing. By mapping the defect information to a three-dimensional model, and performing three-dimensional defect modeling, feature calculation, information superposition and analysis, three-dimensional visualization, quantitative analysis and comprehensive evaluation of flexible circuit board defects can be achieved, providing a more powerful and efficient technical means for the quality control and defect management of flexible circuit boards. Therefore, the present invention provides a flexible circuit board defect detection method and system. The influence of illumination changes is reduced by multi-spectral imaging, the interference of complex textures is overcome by material region segmentation and spectral feature extraction, and the detection ability of tiny defects is enhanced by three-dimensional surface reconstruction and multi-scale feature fusion, and finally accurate defect detection in complex lighting environments is achieved.

[0093] In the embodiment of the present invention, reference Figure 1 The above is a schematic diagram of the steps of the flexible circuit board defect detection method of the present invention. In this example, the flexible circuit board defect detection method includes the following steps:

[0094] Step S1: acquiring a multi-channel image sequence of the flexible circuit board according to the multi-spectral light source array to obtain a multi-channel image sequence; constructing a multi-spectral image cube for the multi-channel image sequence to obtain a multi-spectral image cube;

[0095] In an embodiment of the present invention, a multi-spectral light source array is used to control the brightness and switching status of LED light sources in different bands to form multiple groups of spectral lighting conditions, and an industrial camera is used to capture images of the flexible circuit board under each spectral lighting condition to obtain a multi-channel image sequence. Finally, the images of all channels are arranged in band order and combined into a multi-spectral image cube.

[0096] Step S2: performing material region segmentation according to the multispectral image cube to obtain a material region map; performing multi-feature image fusion according to the material region map to obtain a spectral feature map;

[0097] In the embodiment of the present invention, the multispectral information component vector of each pixel point is extracted from the multispectral image cube, and the dimension reduction process is performed. Then, the data after dimension reduction is classified using a clustering algorithm to obtain a spectral clustering result map. After smoothing and boundary extraction, the image is segmented according to the material area boundary map to obtain a material area map. Finally, the spectral curve is fitted for each material area, the characteristic band is selected, and the corresponding image is extracted for fusion to obtain a spectral feature map.

[0098] Step S3: performing multi-angle photometric image acquisition on the flexible circuit board to obtain a multi-angle photometric image sequence; performing spectral feature height registration map extraction based on the multi-angle photometric image sequence to obtain a spectral feature height registration map; using the spectral feature height registration map to construct a material surface texture database for the material area map to obtain a material surface texture database; performing high-resolution model optimization based on the material surface texture database to obtain a high-resolution three-dimensional surface model;

[0099] In an embodiment of the present invention, an industrial camera is used to shoot a flexible circuit board from multiple angles to obtain a multi-angle photometric image sequence, and the photometric stereo vision technology and the Lambertian reflectance model are used to calculate the normal vector of each pixel point to generate a normal map, and then the height information of each pixel point is calculated according to the normal map to generate a height map. Finally, the height map is aligned with the spectral feature map to obtain a spectral feature height alignment map. The alignment map is used to optimize the material region segmentation results, and the height and spectral feature information of each material region is extracted to construct a material surface texture database. Finally, a three-dimensional mesh model is constructed according to the alignment map, and the material surface texture database is mapped to the model, and a high-resolution three-dimensional surface model is obtained after optimization.

[0100] Step S4: constructing a curvature feature map pyramid for the high-resolution three-dimensional surface model to obtain a curvature feature map pyramid; performing multi-scale feature fusion on the curvature feature map pyramid to obtain a multi-scale fusion feature map pyramid; performing multi-scale feature aggregation on the multi-scale fusion feature map pyramid to obtain a multi-scale feature pyramid;

[0101] In an embodiment of the present invention, a Gaussian pyramid is constructed for a high-resolution three-dimensional surface model, and the normal vector direction of each vertex is calculated to construct a normal direction map pyramid. Then, a shape index is calculated based on the normal direction map pyramid to construct a shape index map pyramid, and curvature features are further extracted to construct a curvature feature map pyramid. At the same time, the three-dimensional model is mapped to a two-dimensional plane and texture features are extracted to construct a texture feature map pyramid. Finally, the curvature feature map pyramid and the texture feature map pyramid are subjected to multi-scale feature fusion, and local feature description, saliency feature enhancement and multi-scale feature aggregation are performed to obtain the final multi-scale feature pyramid.

[0102] Step S5: performing multi-scale feature encoding layer by layer on the multi-scale feature pyramid to obtain a multi-scale feature encoding pyramid; performing global feature pooling on the multi-scale feature encoding pyramid to obtain a global feature vector; performing nonlinear transformation on the global feature vector to obtain a depth defect feature vector;

[0103] In the embodiment of the present invention, data enhancement is performed on the multi-scale feature pyramid, and then it is input into the pre-trained deep feature extractor model for layer-by-layer encoding of multi-scale features. Then, an attention mechanism is introduced to the encoded feature pyramid, and feature graph fusion is performed. Finally, global feature pooling and feature vector dimensionality reduction are performed, and defect feature enhancement and defect distinction optimization are performed on the feature vector after dimensionality reduction to obtain a deep defect feature vector.

[0104] Step S6: Generate a class activation map according to the deep defect feature vector to obtain a class activation map; perform defect data fusion on the class activation map to obtain a defect classification result and a positioning map;

[0105] In the embodiment of the present invention, the deep defect feature vector is input into the pre-trained multi-classifier model for classification prediction, and the defect category is determined according to the confidence threshold. Then, a category activation map is generated according to the classification result, and the category activation map is mapped to the multi-scale feature pyramid for accurate defect positioning. Finally, the positioning result is boundary optimized, and the optimized positioning result is fused with the classification result to obtain the final defect classification result and positioning map.

[0106] Step S7: performing defect area segmentation and extraction according to the defect classification result and the positioning map and the high-resolution three-dimensional surface model positioning map to obtain a defect three-dimensional model; superimposing three-dimensional defect information on the defect three-dimensional model to obtain a three-dimensional defect detection model; performing flexible circuit board defect detection analysis using the three-dimensional defect detection model to obtain flexible circuit board defect detection data;

[0107] In the embodiment of the present invention, the defect classification result and the positioning map are mapped to the high-resolution three-dimensional surface model, the defect area is positioned in three dimensions, and the defect three-dimensional model is segmented and extracted according to the positioning result. Then, the surface features of the defect three-dimensional model are calculated, and the defect classification result and the positioning map are mapped to the defect three-dimensional model for category label matching. Finally, the three-dimensional defect information is superimposed according to the defect surface feature data to obtain a three-dimensional defect detection model, and the model is used to perform flexible circuit board defect detection analysis to obtain the final detection result data.

[0108] Preferably, step S1 comprises the following steps:

[0109] Step S11: performing LED light source lighting control on the flexible circuit board through a multi-spectrum light source array to obtain a multi-spectrum lighting matrix;

[0110] Step S12: acquiring a multi-channel image sequence of the flexible circuit board using an industrial camera according to a multi-spectral illumination matrix to obtain a multi-channel image sequence;

[0111] Step S13: performing spectral response correction on the multi-channel image sequence to obtain spectral response correction data; performing accurate image registration on the multi-channel image sequence according to the spectral response correction data to obtain a multi-channel image correction and registration sequence;

[0112] Step S14: construct a multispectral image cube for the multi-channel image correction and registration sequence according to the band order to obtain a multispectral image cube.

[0113] The present invention controls the LED light source illumination of the flexible circuit board through a multi-spectral light source array, can flexibly adjust the spectral illumination conditions for different materials and defect types, form a multi-spectral illumination matrix, and provide richer and more targeted spectral information for subsequent multi-channel image acquisition, thereby improving the accuracy and reliability of defect detection. The industrial camera collects multi-channel image sequences of the flexible circuit board according to the multi-spectral illumination matrix, and can obtain image information of the flexible circuit board under different spectral illumination conditions, and provide multi-dimensional image data for subsequent spectral analysis and feature extraction, thereby improving the ability to distinguish different materials and defect types. Spectral response correction and image precise registration of the multi-channel image sequence can eliminate the influence of uneven spectral distribution of the camera sensor and the light source, and ensure that the image information of each pixel in different spectral channels is aligned, thereby improving the quality and accuracy of the multi-spectral image. The multi-spectral image cube is constructed according to the band order for the multi-channel image correction and registration sequence, and the multi-channel two-dimensional image sequence can be integrated into a three-dimensional data cube containing rich spectral information, which provides a more intuitive and comprehensive data expression form for subsequent material analysis, feature extraction and three-dimensional modeling, thereby improving the efficiency and accuracy of defect detection.

[0114] In an embodiment of the present invention, a multi-spectral light source array is installed above a flexible circuit board to ensure that light can be evenly irradiated to the area to be tested. According to a preset multi-spectral lighting scheme, the brightness and switch state of LED light sources of different bands in the multi-spectral light source array are controlled. By adjusting the combination of LED light sources of different bands, multiple groups of different spectral lighting conditions are formed, and the spectral information under each group of lighting conditions is recorded to form a multi-spectral lighting matrix. The industrial camera is fixed above the flexible circuit board and remains relatively still with the multi-spectral light source array. According to the multi-spectral lighting matrix set in step S11, different spectral lighting conditions are switched in sequence. A standard whiteboard or a color card with known spectral reflectance is used to shoot under the same conditions as the collected image to obtain a reference image. The difference between each pixel point in different spectral channels between the reference image and the actual collected image is calculated to obtain spectral response correction data. The multi-channel image sequence is corrected using the spectral response correction data to eliminate the influence of uneven spectral distribution of the camera sensor and the light source. An image registration algorithm, such as a feature point-based registration method, is used to accurately register the corrected multi-channel image sequence to ensure that the image information of each pixel point in different spectral channels is aligned, and a multi-channel image correction registration sequence is obtained. Arrange the corrected and registered images of each channel obtained in step S13 according to the preset spectral band order. Combine the arranged multi-channel image sequence into a three-dimensional data cube, in which two dimensions represent the spatial information of the image and the third dimension represents the spectral information, and finally obtain a multispectral image cube.

[0115] Preferably, step S2 comprises the following steps:

[0116] Step S21: constructing a multispectral pixel vector according to the multispectral image cube to obtain a multispectral pixel vector data set; performing spectral dimension reduction on the multispectral pixel vector data set to obtain a multispectral pixel vector dimension reduction data set;

[0117] Step S22: performing spectral clustering analysis on the multi-spectral pixel vector dimension reduction data set to obtain a spectral clustering result graph; performing clustering result spatial smoothing on the spectral clustering result graph to obtain a spectral clustering result smoothing graph;

[0118] Step S23: extracting the material region boundary from the spectral clustering result smoothing map to obtain a material region boundary map;

[0119] Step S24: performing material region segmentation on the spectral clustering result smoothing map according to the material region boundary map to obtain a material region map; performing spectral characteristic curve fitting on the material region map to obtain a material region spectral characteristic curve;

[0120] Step S25: Select characteristic bands for the material region spectral characteristic curve to obtain a characteristic band set; extract corresponding band images from the multispectral image cube according to the characteristic band set and construct characteristic images to obtain a characteristic image set;

[0121] Step S26: performing multi-feature image fusion on the feature image set to obtain a spectral feature map.

[0122] The present invention constructs a multispectral image cube into a multispectral pixel vector data set and performs spectral dimension reduction, which can effectively extract the spectral feature information of each pixel point and reduce data redundancy, providing a more concise and efficient data expression form for subsequent spectral clustering analysis. Performing spectral clustering analysis on the multispectral pixel vector dimension reduction data set and spatially smoothing the clustering results can classify pixels with similar spectral features into one category and effectively suppress the influence of noise and isolated points, thereby obtaining a more accurate and smoother material region division result. Extracting the material region boundary of the smoothed graph of the spectral clustering result can accurately locate the boundary between different material regions, providing accurate guidance information for subsequent material region segmentation, thereby improving the accuracy and reliability of the segmentation result. Performing material region segmentation on the smoothed graph of the spectral clustering result according to the material region boundary graph, and fitting the spectral feature curve for each material region, the image can be segmented into different material regions, and the spectral feature curve of each region can be extracted, providing basic data for subsequent feature band selection and feature image construction. Selecting characteristic bands for the spectral characteristic curves of the material area, and extracting the images of the corresponding bands based on the characteristic band set to construct a characteristic image set, can select the characteristic bands that can best distinguish different materials, and construct a characteristic image set containing richer information, thereby improving the accuracy and reliability of subsequent defect detection. Performing multi-feature image fusion on the characteristic image set to obtain a spectral characteristic map can effectively fuse the information of multiple characteristic bands into one image, highlighting the differences between different material areas, and providing more intuitive and valuable image information for subsequent defect detection.

[0123] In an embodiment of the present invention, the grayscale value of each pixel in the multispectral image cube under different spectral channels is extracted to form a multidimensional vector as the multispectral pixel vector of the pixel, and the multispectral pixel vectors of all pixels constitute a multispectral pixel vector data set. The multispectral pixel vector data set is subjected to spectral dimension reduction using a dimensionality reduction algorithm such as principal component analysis (PCA) or linear discriminant analysis (LDA) to reduce data redundancy and obtain a multispectral pixel vector dimensionality reduction data set. The multispectral pixel vector dimensionality reduction data set is subjected to spectral clustering analysis using a K-means clustering algorithm or a fuzzy C-means clustering algorithm, and pixels with similar spectral features are classified into one category to obtain a spectral clustering result graph, in which each pixel is represented by a category label. The spectral clustering result graph is smoothed using a spatial smoothing algorithm such as a mean filter or a Gaussian filter to eliminate the influence of image noise and small areas, so that the material region boundary is smoother, and a spectral clustering result smoothing graph is obtained. The spectral clustering result smoothing graph is subjected to edge detection to extract the boundaries between material regions to obtain a material region boundary graph, in which each pixel is represented by whether it is a boundary point. According to the material region boundary map, the smooth map of the spectral clustering results is segmented, and the pixels belonging to the same material region are divided together to obtain a material region map, in which each pixel is represented by the material region label. The multispectral pixel vectors of all pixels in each material region are extracted, and the spectral curve is fitted, such as using polynomial fitting or Gaussian fitting, to obtain the spectral characteristic curve of each material region. The spectral characteristic curves of different material regions are analyzed, and characteristic bands that can effectively distinguish different materials, such as peak bands, valley bands, or bands with obvious slope changes, are selected to form a characteristic band set. According to the characteristic band set, images of corresponding bands are extracted from the multispectral image cube, and these images are used to construct a characteristic image set, each characteristic image corresponding to a characteristic band. The characteristic image set is fused, and the information of multiple characteristic bands is fused into one image to obtain a spectral characteristic map, which highlights the differences between different material regions.

[0124] Preferably, step S3 comprises the following steps:

[0125] Step S31: performing multi-angle photometric image acquisition on the flexible circuit board to obtain a multi-angle photometric image sequence; performing spectral feature height registration map extraction according to the multi-angle photometric image sequence to obtain a spectral feature height registration map;

[0126] Step S32: Optimizing the material region segmentation of the material region map using the spectral feature height registration map to obtain a material region segmentation optimization map;

[0127] Step S33: extracting height and spectral feature information of different material regions in the material region segmentation optimization map to obtain height and spectral feature parameters; constructing a material surface texture database according to the height and spectral feature parameters to obtain a material surface texture database;

[0128] Step S34: constructing a three-dimensional mesh model using the spectral feature height registration map to obtain a three-dimensional mesh model of the surface of the flexible circuit board;

[0129] Step S35: performing texture mapping on the three-dimensional mesh model of the surface of the flexible circuit board according to the material surface texture database to obtain a texture-mapped three-dimensional mesh model;

[0130] Step S36: Optimize the texture mapping three-dimensional mesh model to obtain a high-resolution three-dimensional surface model.

[0131] The present invention can obtain the three-dimensional morphological information and spectral information of the surface of the flexible circuit board through multi-angle photometric image acquisition and spectral feature height registration map extraction, and accurately associate the two to provide basic data for subsequent material area segmentation optimization, texture database construction and three-dimensional model construction. The material area map is optimized for material area segmentation using the spectral feature height registration map, and the material area boundary can be more accurately located in combination with the height information, improving the accuracy of material area segmentation, and providing more reliable regional division results for subsequent feature extraction and three-dimensional modeling. The height and spectral feature information of different material areas in the material area segmentation optimization map is extracted, and a material surface texture database is constructed. The height feature and spectral feature parameters of each material area can be extracted and stored in the database, providing rich texture information for subsequent three-dimensional model texture mapping, and improving the realism and detail expression of the three-dimensional model. The three-dimensional mesh model is constructed using the spectral feature height registration map, and the initial three-dimensional mesh model of the surface of the flexible circuit board can be constructed according to the height information and camera parameters. According to the material surface texture database, the texture mapping of the three-dimensional mesh model of the flexible circuit board surface can map the texture information of the material to the three-dimensional model, so that the model can truly reflect the visual differences of different material areas, and improve the realism and visualization of the model. The high-resolution model optimization of the texture-mapped three-dimensional mesh model can smooth the model and enhance the details, improve the resolution and accuracy of the model, and enable the model to more finely express the morphology and texture details of the flexible circuit board surface.

[0132] As an example of the present invention, refer to Figure 2 As shown, in this example, step S3 includes:

[0133] Step S31: performing multi-angle photometric image acquisition on the flexible circuit board to obtain a multi-angle photometric image sequence; performing spectral feature height registration map extraction according to the multi-angle photometric image sequence to obtain a spectral feature height registration map;

[0134] In an embodiment of the present invention, an industrial camera equipped with an adjustable light source is used to photograph a flexible circuit board from multiple preset angles to obtain a multi-angle photometric image sequence, and each angle corresponds to an image. Based on the photometric stereo vision technology, the brightness difference of pixels at different angles in the multi-angle photometric image sequence is used to calculate the normal vector corresponding to each pixel and construct a normal map. Using the normal map, combined with the camera parameters and shooting angle information, the height information of each pixel is calculated to generate a height map. The spectral feature map is aligned with the height map to ensure that the spectral information and height information of each pixel are matched, and a spectral feature height alignment map is generated.

[0135] Step S32: Optimizing the material region segmentation of the material region map using the spectral feature height registration map to obtain a material region segmentation optimization map;

[0136] In an embodiment of the present invention, the spectral feature height registration map obtained in step S31 is used to correct the material region map. For example, the height information is compared with the material region boundary, and the boundary position is fine-tuned to obtain a material region segmentation optimization map, so that it can more accurately reflect the distribution of different material regions.

[0137] Step S33: extracting height and spectral feature information of different material regions in the material region segmentation optimization map to obtain height and spectral feature parameters; constructing a material surface texture database according to the height and spectral feature parameters to obtain a material surface texture database;

[0138] In the embodiment of the present invention, in the material region segmentation optimization map, the height information and spectral characteristic information of all pixels in each material region are extracted respectively. The parameters such as the height mean, height standard deviation, and spectral characteristic curve of each material region are calculated, and these parameters are stored to construct a material surface texture database.

[0139] Step S34: constructing a three-dimensional mesh model using the spectral feature height registration map to obtain a three-dimensional mesh model of the surface of the flexible circuit board;

[0140] In the embodiment of the present invention, the spectral feature height registration map is used in combination with camera calibration information, and triangulation or other three-dimensional reconstruction algorithms are used to construct an initial three-dimensional mesh model of the surface of the flexible circuit board.

[0141] Step S35: performing texture mapping on the three-dimensional mesh model of the surface of the flexible circuit board according to the material surface texture database to obtain a texture-mapped three-dimensional mesh model;

[0142] In the embodiment of the present invention, according to the material surface texture database constructed in step S33, the spectral feature information corresponding to each material region is mapped to the corresponding region in the three-dimensional mesh model to obtain a texture-mapped three-dimensional mesh model.

[0143] Step S36: performing high-resolution model optimization on the texture mapping three-dimensional mesh model to obtain a high-resolution three-dimensional surface model;

[0144] In the embodiment of the present invention, mesh optimization algorithms such as bilateral filtering and Poisson reconstruction are used to smooth the texture mapping three-dimensional mesh model, improve the model resolution, remove noise, and obtain a high-resolution three-dimensional surface model.

[0145] Preferably, step S31 includes the following steps:

[0146] Step S311: collecting multi-angle photometric images of the flexible circuit board according to the multi-spectral light source array to obtain a multi-angle photometric image sequence;

[0147] Step S312: performing image brightness difference analysis on the multi-angle photometric image sequence to obtain image brightness difference data; using the image brightness difference data and the Lambertian reflection model to calculate the direction of the normal vector of the pixel point on the multi-angle photometric image sequence to obtain a normal map;

[0148] Step S313: Calculating the surface height information of the flexible circuit board using the normal map to obtain a height map;

[0149] Step S314: performing precise image registration on the height map and the spectral feature map to obtain a spectral feature height registration map.

[0150] The present invention can obtain the surface reflection information of the flexible circuit board under different angles and spectral conditions by performing multi-angle photometric image acquisition under the illumination conditions of a multi-spectral light source array, and provide richer and more accurate raw data for subsequent surface normal and height information calculation. By performing image brightness difference analysis on the multi-angle photometric image sequence and combining the Lambertian reflection model, the normal vector direction of each pixel in three-dimensional space can be calculated, and a normal map can be generated, thereby accurately describing the orientation change of the surface of the flexible circuit board. Using the normal map information, combined with camera parameters and shooting angles, the height information of each pixel relative to the reference plane can be calculated, and a height map can be generated, thereby realizing the digital expression of the three-dimensional morphology of the surface of the flexible circuit board. The height map is accurately aligned with the spectral feature map to obtain a spectral feature height alignment map, which can associate the three-dimensional morphology information of each pixel with its corresponding spectral information, providing a more comprehensive and accurate data basis for subsequent material analysis, defect detection, etc.

[0151] In an embodiment of the present invention, while keeping the multi-spectral light source array turned on and the lighting conditions unchanged, the industrial camera is controlled to shoot the flexible circuit board from multiple preset angles to obtain a multi-angle photometric image sequence, and each angle corresponds to an image. The brightness difference of each pixel in the multi-angle photometric image sequence in the images at different angles is analyzed to obtain image brightness difference data. Using the image brightness difference data, combined with the Lambertian reflection model, the normal vector direction corresponding to each pixel is calculated to generate a normal map, in which each pixel is represented by a three-dimensional vector. Using the normal map, combined with camera parameters, shooting angle information and photometric stereo vision technology, the height information of each pixel is calculated to generate a height map, in which each pixel is represented by a numerical value relative to the reference plane. Using a feature point-based image registration algorithm, the height map and the spectral feature map are accurately registered to ensure that the height information and spectral information of each pixel are aligned in spatial position, and a spectral feature height registration map is obtained.

[0152] Preferably, step S4 comprises the following steps:

[0153] Step S41: constructing a Gaussian pyramid on the high-resolution three-dimensional surface model to obtain a Gaussian pyramid;

[0154] Step S42: Calculate the vertex normal direction of the Gaussian pyramid to obtain vertex normal direction data; perform range mapping according to the vertex normal direction data and construct a normal direction map pyramid to obtain a normal direction map pyramid;

[0155] Step S43: calculating the shape index according to the normal direction map pyramid and constructing the shape index map pyramid to obtain the shape index map pyramid;

[0156] Step S44: extracting curvature features from the shape index graph pyramid and constructing a curvature feature graph pyramid to obtain a curvature feature graph pyramid;

[0157] Step S45: performing two-dimensional plane mapping on the high-resolution three-dimensional surface model and extracting texture features to obtain a texture feature map pyramid;

[0158] Step S46: performing multi-scale feature fusion on the curvature feature map pyramid and the texture feature map pyramid to obtain a multi-scale fused feature map pyramid;

[0159] Step S47: perform local feature description on the multi-scale fusion feature map pyramid to obtain a local feature description map pyramid; perform saliency feature enhancement on the local feature description map pyramid to obtain a saliency enhanced feature map pyramid; perform multi-scale feature aggregation on the saliency enhanced feature map pyramid to obtain a multi-scale feature pyramid.

[0160] The present invention can generate a series of model versions with gradually reduced resolutions while retaining the original high-resolution three-dimensional surface model detail information by constructing a Gaussian pyramid, which is conducive to multi-scale feature extraction and improves the algorithm's ability to detect defects of different sizes. By calculating the normal vector direction of each vertex and constructing a normal direction map pyramid, the surface orientation information of the three-dimensional model can be converted into an image form, which is convenient for subsequent feature extraction and analysis using image processing technology. By calculating the shape index according to the normal direction map pyramid and constructing a shape index map pyramid, the local concave and convex degree of the surface can be quantified, which is helpful for identifying and distinguishing different types of surface defects. By extracting curvature features from the shape index map pyramid and constructing a curvature feature map pyramid, the curvature degree of the surface can be further described, which is helpful for detecting and identifying defects such as cracks and wrinkles on the surface. By mapping the high-resolution three-dimensional surface model to a two-dimensional plane and extracting texture features, and constructing a texture feature map pyramid, the texture information of the surface can be converted from three-dimensional space to a two-dimensional plane, which is convenient for feature extraction and analysis using mature image processing technology. The multi-scale feature fusion of the curvature feature map pyramid and the texture feature map pyramid can effectively combine the surface geometry information and texture information to obtain a feature representation containing richer surface information, thereby improving the accuracy and robustness of subsequent defect detection. The local feature description, saliency feature enhancement, and multi-scale feature aggregation of the multi-scale fusion feature map pyramid can further enhance the feature information related to defects, suppress the influence of background noise, and construct the final multi-scale feature pyramid to provide more discriminative feature input for subsequent deep learning models.

[0161] In an embodiment of the present invention, the high-resolution three-dimensional surface model obtained in step S36 is subjected to multiple Gaussian filtering and downsampling operations to construct a three-dimensional model pyramid containing multiple resolution versions, namely, a Gaussian pyramid. For each resolution version of the three-dimensional model in the Gaussian pyramid, the normal vector direction of each vertex is calculated to obtain vertex normal direction data. The value range of the vertex normal direction data is mapped to [0, 255], and the normal direction data of each resolution version is stored in the form of an image to construct a normal direction map pyramid. According to the normal vector direction corresponding to each pixel point in the normal direction map pyramid, the shape index of the pixel point is calculated, and a shape index map pyramid is constructed to describe the local concavity and convexity of the surface. For each pixel point in the shape index map pyramid, its gradient information is calculated using a Sobel operator, etc., the curvature feature is extracted, and a curvature feature map pyramid is constructed to describe the curvature of the surface. The high-resolution three-dimensional surface model is mapped onto a two-dimensional plane, for example, using spherical mapping or cylindrical mapping. The texture features of the mapped two-dimensional image are extracted, for example, by using gray level co-occurrence matrix (GLCM) or local binary pattern (LBP) and other methods to obtain a texture feature map pyramid for describing the texture information of the surface. The curvature feature map pyramid obtained in step S44 and the texture feature map pyramid obtained in step S45 are fused with multi-scale features using a feature pyramid fusion method to obtain a multi-scale fused feature map pyramid containing richer surface information. For each pixel in the multi-scale fused feature map pyramid, a local feature description is extracted using a local binary pattern (LBP) or histogram of oriented gradients (HOG) and other methods to obtain a local feature description map pyramid. The local feature description map pyramid is enhanced for significant features to highlight important defect features. Multi-scale feature aggregation is performed on the significant enhancement feature map pyramid using a pooling operation or feature connection method to obtain a final multi-scale feature pyramid for subsequent defect detection.

[0162] Preferably, step S5 comprises the following steps:

[0163] Step S51: performing data enhancement on the multi-scale feature pyramid to obtain a data enhanced feature pyramid;

[0164] Step S52: inputting the data enhancement feature pyramid into the pre-trained deep feature extractor model, encoding the multi-scale features layer by layer, and obtaining a multi-scale feature encoding pyramid;

[0165] Step S53: performing an attention mechanism introduction analysis on the multi-scale feature coding pyramid to obtain an attention enhancement feature pyramid; performing feature graph fusion on the attention enhancement feature pyramid to obtain a fused feature pyramid;

[0166] Step S54: performing global feature pooling on the fused feature pyramid to obtain a global feature vector; performing feature vector dimension reduction on the global feature vector to obtain a global feature dimension reduction vector;

[0167] Step S55: performing defect feature enhancement on the global feature dimension reduction vector to obtain a defect feature enhancement vector; performing defect differentiation optimization on the defect feature enhancement vector to obtain a defect feature optimization vector;

[0168] Step S56: Input the defect feature optimization vector into the fully connected layer and perform nonlinear transformation to obtain a deep defect feature vector.

[0169] The present invention can increase the quantity and diversity of training data by performing data enhancement on the multi-scale feature pyramid, by rotating, translating, scaling and other operations on the existing data, thereby improving the generalization ability of the deep learning model and reducing the risk of overfitting. The data enhancement feature pyramid is input into the pre-trained deep feature extractor model, and the multi-scale features are encoded layer by layer. The powerful feature extraction ability of the pre-trained model can be used to map the feature information of different scales to a more abstract and semantically informative feature space, providing a more discriminative feature representation for subsequent defect detection. The introduction of the attention mechanism to the multi-scale feature encoding pyramid and the fusion of feature maps can make the model pay more attention to the key features related to the defects, suppress the interference of background noise and irrelevant information, thereby improving the sensitivity and detection accuracy of the model to defects. Global feature pooling and feature vector dimensionality reduction are performed on the fused feature pyramid, which can aggregate the multi-scale and multi-channel feature information into a compact global feature vector, remove redundant information, and provide a more concise and effective feature input for the subsequent classifier. Defect feature enhancement and defect differentiation optimization of the global feature dimensionality reduction vector can further highlight the feature information related to defects and optimize the feature weights, making the feature vector more conducive to distinguishing different types of defects, thereby improving the accuracy of subsequent defect classification. The defect feature optimization vector is input into the fully connected layer and nonlinearly transformed to obtain a deep defect feature vector, which can map the feature information to a higher-dimensional feature space, enhance the nonlinear expression ability of the feature, and provide a more accurate and robust feature representation for subsequent defect classification and positioning.

[0170] As an example of the present invention, refer to Figure 3 As shown, in this example, step S5 includes:

[0171] Step S51: performing data enhancement on the multi-scale feature pyramid to obtain a data enhanced feature pyramid;

[0172] In an embodiment of the present invention, data enhancement operations are performed on the multi-scale feature pyramid, such as random rotation, translation, scaling, adding noise, etc., to increase the amount and diversity of training data and obtain a data enhanced feature pyramid.

[0173] Step S52: inputting the data enhancement feature pyramid into the pre-trained deep feature extractor model, encoding the multi-scale features layer by layer, and obtaining a multi-scale feature encoding pyramid;

[0174] In an embodiment of the present invention, the data augmentation feature pyramid is input into a pre-trained deep feature extractor model, such as ResNet, VGGNet, etc., and the multi-layer convolutional structure of the model is used to encode features of different scales layer by layer to obtain a multi-scale feature encoding pyramid containing more abstract and higher-level semantic information.

[0175] Step S53: performing an attention mechanism introduction analysis on the multi-scale feature coding pyramid to obtain an attention enhancement feature pyramid; performing feature graph fusion on the attention enhancement feature pyramid to obtain a fused feature pyramid;

[0176] In an embodiment of the present invention, an attention mechanism, such as a spatial attention mechanism or a channel attention mechanism, is introduced into the multi-scale feature encoding pyramid, so that the model pays more attention to the feature areas related to defects, and obtains an attention-enhanced feature pyramid. A feature pyramid fusion method is used, such as splicing or weighted summing feature maps of different scales, to fuse features of different levels in the attention-enhanced feature pyramid, and obtain a fused feature pyramid.

[0177] Step S54: performing global feature pooling on the fused feature pyramid to obtain a global feature vector; performing feature vector dimension reduction on the global feature vector to obtain a global feature dimension reduction vector;

[0178] In the embodiment of the present invention, operations such as global average pooling or global maximum pooling are applied to the fused feature pyramid to convert each feature map into a numerical value to obtain a global feature vector representing global information. Dimensionality reduction methods such as principal component analysis (PCA) or linear discriminant analysis (LDA) are used to reduce the dimension of the global feature vector, remove redundant information, and obtain a global feature dimension reduction vector.

[0179] Step S55: performing defect feature enhancement on the global feature dimension reduction vector to obtain a defect feature enhancement vector; performing defect differentiation optimization on the defect feature enhancement vector to obtain a defect feature optimization vector;

[0180] In an embodiment of the present invention, defect feature enhancement is performed on the global feature dimensionality reduction vector, for example, using feature weighting or feature selection methods to highlight features related to defects, and obtain a defect feature enhancement vector. The defect feature enhancement vector is optimized for defect differentiation, for example, using a classifier such as a support vector machine or logistic regression for preliminary classification, and adjusting feature weights according to the classification results to obtain a defect feature optimization vector that is more conducive to distinguishing different types of defects.

[0181] Step S56: inputting the defect feature optimization vector into the fully connected layer and performing nonlinear transformation to obtain a deep defect feature vector;

[0182] In the embodiment of the present invention, the defect feature optimization vector is input into the fully connected layer, and nonlinear transformation is performed using a nonlinear activation function such as ReLU or Sigmoid to obtain a deep defect feature vector for subsequent defect classification and positioning.

[0183] Preferably, step S6 comprises the following steps:

[0184] Step S61: inputting the depth defect feature vector into the pre-trained multi-classifier model to perform classification prediction to obtain defect category probability distribution data;

[0185] Step S62: using a preset confidence threshold to determine the defect category of the defect category probability distribution data, and obtaining defect classification result data;

[0186] Step S63: generating a class activation map according to the defect classification result data to obtain a class activation map;

[0187] Step S64: Mapping the class activation map to a multi-scale feature pyramid to accurately locate defects and obtain a multi-scale defect location map;

[0188] Step S65: optimizing the defect area boundary of the multi-scale defect location map to obtain a defect location optimization map;

[0189] Step S66: performing defect data fusion on the defect location optimization map and the defect classification result data to obtain the defect classification result and the location map.

[0190] The present invention inputs the deep defect feature vector into the pre-trained multi-classifier model to perform classification prediction. The trained model can be used to classify the defect features and output the probability distribution data corresponding to each category, providing a basis for subsequent defect category determination. The defect category probability distribution data is screened using a preset confidence threshold, and the classification results with low confidence can be filtered out, and the results with high confidence are retained as the final defect category determination results, thereby improving the reliability of the classification results. According to the defect classification result data, a category activation map is generated, and the classification decision process of the model can be visualized, which feature areas contribute more to the final classification results intuitively, providing guidance information for subsequent defect location. The category activation map is mapped to a multi-scale feature pyramid for accurate defect location. The feature area information provided by the category activation map can be used to perform weighted fusion on feature maps of different scales, thereby achieving accurate location of the defect. The defect area boundary optimization of the multi-scale defect location map can be performed, and the defect area boundary can be refined using an image segmentation algorithm to obtain a more accurate and smoother defect boundary, thereby improving the accuracy of defect location. By fusing the defect location optimization map and the defect classification result data, it is possible to combine the defect category information and precise positioning information to obtain defect classification results and location maps containing rich information, providing more comprehensive data support for subsequent defect analysis and processing.

[0191] In an embodiment of the present invention, the deep defect feature vector is input into a pre-trained multi-classifier model, such as a support vector machine, a random forest or a deep neural network, etc., for classification prediction, and probability distribution data corresponding to each category is obtained. According to a pre-set confidence threshold, the defect category probability distribution data is screened to determine which category of defect each pixel belongs to, and the determination result is stored as defect classification result data. Using methods such as gradient back propagation, the classification result is reversely mapped to the feature map of the multi-classifier model to generate a category activation map for visualizing the feature area corresponding to each category. The category activation map is upsampled to the same resolution as the multi-scale feature pyramid, and weighted fusion is performed in the multi-scale feature pyramid according to the activation value of each pixel in the category activation map to obtain a multi-scale defect location map for accurately locating the defect area. Image segmentation algorithms such as threshold segmentation, edge detection or region growing are applied to the multi-scale defect location map to further optimize the boundary of the defect area and obtain a more accurate defect location optimization map. The defect classification result data is integrated with the defect location optimization map to obtain the final defect classification result and location map, which contains the category information and precise boundary location information of each defect.

[0192] Preferably, step S7 comprises the following steps:

[0193] Step S71: Mapping the defect classification result and the positioning map to a high-resolution three-dimensional surface model, performing three-dimensional positioning of the defect area, and obtaining three-dimensional positioning defect area data;

[0194] Step S72: segmenting and extracting defect regions from the high-resolution three-dimensional surface model according to the three-dimensional positioning defect region data to obtain a defect three-dimensional model;

[0195] Step S73: Calculate the defect surface characteristics of the defect three-dimensional model to obtain defect surface characteristic data;

[0196] Step S74: Mapping the defect classification result and the positioning map to the defect three-dimensional model, performing defect category label matching, and obtaining a category label defect three-dimensional model;

[0197] Step S75: superimposing three-dimensional defect information on the category label defect three-dimensional model according to the defect surface feature data to obtain a three-dimensional defect detection model;

[0198] Step S76: using the three-dimensional defect detection model to perform flexible circuit board defect detection analysis to obtain flexible circuit board defect detection data.

[0199] The present invention maps the defect classification results and the positioning map to the high-resolution three-dimensional surface model to perform three-dimensional positioning of the defect area, and can correspond the defect information on the two-dimensional image with the real position in the three-dimensional space, providing accurate spatial positioning information for subsequent three-dimensional modeling and analysis of the defect. According to the three-dimensional positioning defect area data, the high-resolution three-dimensional surface model is segmented and extracted, and the defect area can be separated from the complete three-dimensional model to construct an independent defect three-dimensional model, providing a more intuitive and more accurate object for subsequent defect feature calculation and analysis. The defect surface feature calculation of the defect three-dimensional model can extract richer defect geometric feature information, such as the volume, area, depth, curvature, etc. of the defect. The defect classification results and the positioning map are mapped to the defect three-dimensional model, and the defect category label matching is performed, so that the defect category information on the two-dimensional image can be corresponded with the defect area in the three-dimensional model, providing a more intuitive and easier to understand information expression method for subsequent defect visualization and analysis. According to the defect surface feature data, the three-dimensional defect information is superimposed on the category label defect three-dimensional model, which can present the defect category, size, shape, location and other information in a more intuitive and easier to understand three-dimensional way, which is convenient for subsequent defect analysis. Using the three-dimensional defect detection model to perform defect detection and analysis on flexible circuit boards can provide a more comprehensive and in-depth analysis of defects, such as counting the number, size distribution, spatial position relationship of different types of defects, etc., to provide more accurate and reliable data support for quality control, defect diagnosis and maintenance of flexible circuit boards.

[0200] In an embodiment of the present invention, the defect classification result and the positioning map are mapped to the high-resolution three-dimensional surface model according to the previous registration relationship, and the corresponding position of the defect area in the three-dimensional model is obtained to generate three-dimensional positioning defect area data. According to the three-dimensional positioning defect area data, a three-dimensional model segmentation algorithm is used, such as a method based on region growing, edge detection or graph cutting, to segment and extract the part of the high-resolution three-dimensional surface model belonging to the defect area, and obtain a defect three-dimensional model. The surface feature calculation of the defect three-dimensional model is performed, such as calculating the geometric features such as the surface area, volume, curvature, roughness, and appearance features such as color and texture of the defect area, to obtain defect surface feature data. The defect classification result and the positioning map are mapped to the defect three-dimensional model, and each defect three-dimensional model is assigned a corresponding category label to obtain a category label defect three-dimensional model. According to the defect surface feature data, the category label defect three-dimensional model is superimposed with three-dimensional defect information, such as marking the category, area, volume, depth and other information of the defect on the three-dimensional model in the form of text, color or graphics, to obtain a three-dimensional defect detection model containing rich information. Using the three-dimensional defect detection model and combining it with expert knowledge, the defects of the flexible circuit board are analyzed. For example, the number, size distribution, and location information of different types of defects are counted to obtain the defect detection data of the flexible circuit board.

[0201] Preferably, the present invention further provides a flexible circuit board defect detection system for executing the flexible circuit board defect detection method as described above, the flexible circuit board defect detection system comprising:

[0202] The multi-spectral fusion imaging module is used to collect a multi-channel image sequence of the flexible circuit board according to the multi-spectral light source array to obtain a multi-channel image sequence; and construct a multi-spectral image cube for the multi-channel image sequence to obtain a multi-spectral image cube;

[0203] The spectral feature extraction module is used to perform material region segmentation based on the multi-spectral image cube to obtain a material region map; perform multi-feature image fusion based on the material region map to obtain a spectral feature map;

[0204] The surface texture reconstruction module is used to collect multi-angle photometric images of the flexible circuit board to obtain a multi-angle photometric image sequence; extract the spectral feature height registration map according to the multi-angle photometric image sequence to obtain a spectral feature height registration map; use the spectral feature height registration map to construct a material surface texture database for the material area map to obtain a material surface texture database; optimize the high-resolution model according to the material surface texture database to obtain a high-resolution three-dimensional surface model;

[0205] The multi-scale feature pyramid module is used to construct a curvature feature map pyramid for a high-resolution three-dimensional surface model to obtain a curvature feature map pyramid; to perform multi-scale feature fusion on the curvature feature map pyramid to obtain a multi-scale fused feature map pyramid; to perform multi-scale feature aggregation on the multi-scale fused feature map pyramid to obtain a multi-scale feature pyramid;

[0206] The deep defect feature learning module is used to encode the multi-scale features of the multi-scale feature pyramid layer by layer to obtain the multi-scale feature encoding pyramid; perform global feature pooling on the multi-scale feature encoding pyramid to obtain the global feature vector; perform nonlinear transformation on the global feature vector to obtain the deep defect feature vector;

[0207] The defect classification and positioning module is used to generate a class activation map based on the deep defect feature vector to obtain the class activation map; the defect data is fused on the class activation map to obtain the defect classification result and positioning map;

[0208] The three-dimensional defect detection module is used to segment and extract the defect area according to the defect classification results and positioning map as well as the high-resolution three-dimensional surface model positioning map to obtain a three-dimensional defect model; to superimpose three-dimensional defect information on the defect three-dimensional model to obtain a three-dimensional defect detection model; and to use the three-dimensional defect detection model to perform flexible circuit board defect detection analysis to obtain flexible circuit board defect detection data.

[0209] The present invention collects multi-channel image sequences through a multi-spectral light source array and constructs a multi-spectral image cube, which can obtain rich spectral information of the flexible circuit board and provide a more comprehensive data basis for subsequent material analysis and defect detection. Through material region segmentation and multi-feature image fusion through a multi-spectral image cube, different material regions can be distinguished, and spectral feature maps that highlight material differences can be generated, providing more accurate background information for subsequent defect detection. Through multi-angle photometric image acquisition and spectral feature height registration map extraction, the three-dimensional morphological information of the flexible circuit board can be obtained, and accurately associated with the spectral information, a high-resolution three-dimensional surface model containing rich texture information can be constructed, providing a more intuitive visual expression and a more accurate positioning basis for subsequent defect detection. By performing multi-scale feature extraction and fusion on the high-resolution three-dimensional surface model, the geometric shape and texture information of the flexible circuit board can be fully described, and a multi-scale feature pyramid containing rich detail information can be constructed, providing a more discriminative feature input for the subsequent deep learning model. By encoding the multi-scale feature pyramid layer by layer, global feature pooling and nonlinear transformation, a more abstract, compact and semantically informative deep defect feature vector can be extracted, providing a more effective feature representation for subsequent defect classification and positioning. Through the classification prediction of the deep learning model, the generation and mapping of the category activation map and the optimization of defect positioning, the defect category can be accurately identified and the defect area can be accurately located, providing a reliable data basis for subsequent defect analysis and processing. By mapping the defect information to a three-dimensional model, and performing three-dimensional defect modeling, feature calculation, information superposition and analysis, three-dimensional visualization, quantitative analysis and comprehensive evaluation of flexible circuit board defects can be achieved, providing a more powerful and efficient technical means for the quality control and defect management of flexible circuit boards. Therefore, the present invention provides a flexible circuit board defect detection method and system. The influence of illumination changes is reduced by multi-spectral imaging, the interference of complex textures is overcome by material region segmentation and spectral feature extraction, and the detection ability of tiny defects is enhanced by three-dimensional surface reconstruction and multi-scale feature fusion, and finally accurate defect detection in complex lighting environments is achieved.

[0210] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0211] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A method for detecting defects in a flexible circuit board, characterized in that: The following steps are involved: Step S1: acquiring a multi-channel image sequence of the flexible circuit board according to the multi-spectral light source array to obtain a multi-channel image sequence; constructing a multi-spectral image cube for the multi-channel image sequence to obtain a multi-spectral image cube; Step S2: perform material region segmentation according to the multispectral image cube to obtain a material region map; perform multi-feature image fusion according to the material region map to obtain a spectral feature map, wherein step S2 is specifically as follows: Step S21: constructing a multispectral pixel vector according to the multispectral image cube to obtain a multispectral pixel vector data set; performing spectral dimension reduction on the multispectral pixel vector data set to obtain a multispectral pixel vector dimension reduction data set; Step S22: performing spectral clustering analysis on the multi-spectral pixel vector dimension reduction data set to obtain a spectral clustering result graph; Perform clustering result spatial smoothing on the spectral clustering result graph to obtain a spectral clustering result smoothing graph; Step S23: extracting the material region boundary from the spectral clustering result smoothing map to obtain a material region boundary map; Step S24: performing material region segmentation on the spectral clustering result smoothing map according to the material region boundary map to obtain a material region map; performing spectral characteristic curve fitting on the material region map to obtain a material region spectral characteristic curve; Step S25: Select characteristic bands for the material region spectral characteristic curve to obtain a characteristic band set; extract corresponding band images from the multispectral image cube according to the characteristic band set and construct characteristic images to obtain a characteristic image set; Step S26: performing multi-feature image fusion on the feature image set to obtain a spectral feature map; Step S3: multi-angle photometric image acquisition is performed on the flexible circuit board to obtain a multi-angle photometric image sequence; a spectral feature height registration map is extracted according to the multi-angle photometric image sequence to obtain a spectral feature height registration map; a material surface texture database is constructed for the material area map using the spectral feature height registration map to obtain a material surface texture database; a high-resolution model is optimized according to the material surface texture database to obtain a high-resolution three-dimensional surface model, wherein step S3 is specifically as follows: Step S31: performing multi-angle photometric image acquisition on the flexible circuit board to obtain a multi-angle photometric image sequence; performing spectral feature height registration map extraction according to the multi-angle photometric image sequence to obtain a spectral feature height registration map, wherein step S31 is specifically as follows: Step S311: collecting multi-angle photometric images of the flexible circuit board according to the multi-spectral light source array to obtain a multi-angle photometric image sequence; Step S312: performing image brightness difference analysis on the multi-angle photometric image sequence to obtain image brightness difference data; The normal vector direction of the pixel point in the multi-angle photometric image sequence is calculated using the image brightness difference data and the Lambertian reflectance model to obtain a normal map. Step S313: Calculating the surface height information of the flexible circuit board using the normal map to obtain a height map; Step S314: performing precise image registration on the height map and the spectral feature map to obtain a spectral feature height registration map; Step S32: Optimizing the material region segmentation of the material region map using the spectral feature height registration map to obtain a material region segmentation optimization map; Step S33: extracting height and spectral feature information of different material regions in the material region segmentation optimization map to obtain height and spectral feature parameters; constructing a material surface texture database according to the height and spectral feature parameters to obtain a material surface texture database; Step S34: constructing a three-dimensional mesh model using the spectral feature height registration map to obtain a three-dimensional mesh model of the surface of the flexible circuit board; Step S35: performing texture mapping on the three-dimensional mesh model of the surface of the flexible circuit board according to the material surface texture database to obtain a texture-mapped three-dimensional mesh model; Step S36: performing high-resolution model optimization on the texture mapping three-dimensional mesh model to obtain a high-resolution three-dimensional surface model; Step S4: constructing a curvature feature map pyramid for the high-resolution three-dimensional surface model to obtain a curvature feature map pyramid; performing multi-scale feature fusion on the curvature feature map pyramid to obtain a multi-scale fusion feature map pyramid; performing multi-scale feature aggregation on the multi-scale fusion feature map pyramid to obtain a multi-scale feature pyramid, wherein step S4 is specifically as follows: Step S41: constructing a Gaussian pyramid on the high-resolution three-dimensional surface model to obtain a Gaussian pyramid; Step S42: Calculate the vertex normal direction of the Gaussian pyramid to obtain vertex normal direction data; perform range mapping according to the vertex normal direction data and construct a normal direction map pyramid to obtain a normal direction map pyramid; Step S43: calculating the shape index according to the normal direction map pyramid and constructing the shape index map pyramid to obtain the shape index map pyramid; Step S44: extracting curvature features from the shape index graph pyramid and constructing a curvature feature graph pyramid to obtain a curvature feature graph pyramid; Step S45: performing two-dimensional plane mapping on the high-resolution three-dimensional surface model and extracting texture features to obtain a texture feature map pyramid; Step S46: performing multi-scale feature fusion on the curvature feature map pyramid and the texture feature map pyramid to obtain a multi-scale fused feature map pyramid; Step S47: performing local feature description on the multi-scale fusion feature map pyramid to obtain a local feature description map pyramid; performing saliency feature enhancement on the local feature description map pyramid to obtain a saliency enhanced feature map pyramid; performing multi-scale feature aggregation on the saliency enhanced feature map pyramid to obtain a multi-scale feature pyramid; Step S5: performing multi-scale feature encoding layer by layer on the multi-scale feature pyramid to obtain a multi-scale feature encoding pyramid; performing global feature pooling on the multi-scale feature encoding pyramid to obtain a global feature vector; performing nonlinear transformation on the global feature vector to obtain a depth defect feature vector, wherein step S5 is specifically as follows: Step S51: performing data enhancement on the multi-scale feature pyramid to obtain a data enhanced feature pyramid; Step S52: inputting the data enhancement feature pyramid into the pre-trained deep feature extractor model, encoding the multi-scale features layer by layer, and obtaining a multi-scale feature encoding pyramid; Step S53: performing an attention mechanism introduction analysis on the multi-scale feature coding pyramid to obtain an attention enhancement feature pyramid; performing feature graph fusion on the attention enhancement feature pyramid to obtain a fused feature pyramid; Step S54: performing global feature pooling on the fused feature pyramid to obtain a global feature vector; performing feature vector dimension reduction on the global feature vector to obtain a global feature dimension reduction vector; Step S55: performing defect feature enhancement on the global feature dimension reduction vector to obtain a defect feature enhancement vector; performing defect differentiation optimization on the defect feature enhancement vector to obtain a defect feature optimization vector; Step S56: inputting the defect feature optimization vector into the fully connected layer and performing nonlinear transformation to obtain a deep defect feature vector; Step S6: Generate a class activation map according to the deep defect feature vector to obtain a class activation map; perform defect data fusion on the class activation map to obtain a defect classification result and a positioning map, wherein step S6 is specifically as follows: Step S61: inputting the depth defect feature vector into the pre-trained multi-classifier model to perform classification prediction to obtain defect category probability distribution data; Step S62: using a preset confidence threshold to determine the defect category of the defect category probability distribution data, and obtaining defect classification result data; Step S63: generating a class activation map according to the defect classification result data to obtain a class activation map; Step S64: Mapping the class activation map to a multi-scale feature pyramid to accurately locate defects and obtain a multi-scale defect location map; Step S65: optimizing the defect area boundary of the multi-scale defect location map to obtain a defect location optimization map; Step S66: performing defect data fusion on the defect location optimization map and the defect classification result data to obtain a defect classification result and a location map; Step S7: performing defect area segmentation and extraction according to the defect classification result and the positioning map as well as the high-resolution three-dimensional surface model positioning map to obtain a defect three-dimensional model; The three-dimensional defect information is superimposed on the defect three-dimensional model to obtain a three-dimensional defect detection model; the three-dimensional defect detection model is used to perform flexible circuit board defect detection analysis to obtain flexible circuit board defect detection data, wherein step S7 is specifically: Step S71: Mapping the defect classification result and the positioning map to a high-resolution three-dimensional surface model, performing three-dimensional positioning of the defect area, and obtaining three-dimensional positioning defect area data; Step S72: performing defect area segmentation and extraction on the high-resolution three-dimensional surface model according to the three-dimensional positioning defect area data to obtain a defect three-dimensional model; Step S73: Calculate the defect surface characteristics of the defect three-dimensional model to obtain defect surface characteristic data; Step S74: Mapping the defect classification result and the positioning map to the defect three-dimensional model, performing defect category label matching, and obtaining a category label defect three-dimensional model; Step S75: superimposing three-dimensional defect information on the category label defect three-dimensional model according to the defect surface feature data to obtain a three-dimensional defect detection model; Step S76: using the three-dimensional defect detection model to perform flexible circuit board defect detection analysis to obtain flexible circuit board defect detection data.

2. The method for detecting defects in a flexible circuit board according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: performing LED light source lighting control on the flexible circuit board through a multi-spectrum light source array to obtain a multi-spectrum lighting matrix; Step S12: acquiring a multi-channel image sequence of the flexible circuit board using an industrial camera according to a multi-spectral illumination matrix to obtain a multi-channel image sequence; Step S13: performing spectral response correction on the multi-channel image sequence to obtain spectral response correction data; performing accurate image registration on the multi-channel image sequence according to the spectral response correction data to obtain a multi-channel image correction and registration sequence; Step S14: construct a multispectral image cube for the multi-channel image correction and registration sequence according to the band order to obtain a multispectral image cube.

3. A flexible circuit board defect detection system, characterized in that: Used to perform the flexible circuit board defect detection method according to claim 1, the flexible circuit board defect detection system comprises: The multi-spectral fusion imaging module is used to collect a multi-channel image sequence of the flexible circuit board according to the multi-spectral light source array to obtain a multi-channel image sequence; and construct a multi-spectral image cube for the multi-channel image sequence to obtain a multi-spectral image cube; The spectral feature extraction module is used to perform material region segmentation based on the multi-spectral image cube to obtain a material region map; perform multi-feature image fusion based on the material region map to obtain a spectral feature map; The surface texture reconstruction module is used to collect multi-angle photometric images of the flexible circuit board to obtain a multi-angle photometric image sequence; extract the spectral feature height registration map according to the multi-angle photometric image sequence to obtain a spectral feature height registration map; use the spectral feature height registration map to construct a material surface texture database for the material area map to obtain a material surface texture database; optimize the high-resolution model according to the material surface texture database to obtain a high-resolution three-dimensional surface model; The multi-scale feature pyramid module is used to construct a curvature feature map pyramid for a high-resolution three-dimensional surface model to obtain a curvature feature map pyramid; to perform multi-scale feature fusion on the curvature feature map pyramid to obtain a multi-scale fused feature map pyramid; to perform multi-scale feature aggregation on the multi-scale fused feature map pyramid to obtain a multi-scale feature pyramid; The deep defect feature learning module is used to encode the multi-scale features of the multi-scale feature pyramid layer by layer to obtain the multi-scale feature encoding pyramid; perform global feature pooling on the multi-scale feature encoding pyramid to obtain the global feature vector; perform nonlinear transformation on the global feature vector to obtain the deep defect feature vector; The defect classification and positioning module is used to generate a class activation map based on the deep defect feature vector to obtain the class activation map; the defect data is fused on the class activation map to obtain the defect classification result and positioning map; The three-dimensional defect detection module is used to segment and extract the defect area according to the defect classification results and positioning map as well as the high-resolution three-dimensional surface model positioning map to obtain a three-dimensional defect model; to superimpose three-dimensional defect information on the defect three-dimensional model to obtain a three-dimensional defect detection model; and to use the three-dimensional defect detection model to perform flexible circuit board defect detection analysis to obtain flexible circuit board defect detection data.