Deep learning method and system for printed circuit board defect recognition

Abstract

The present invention relates to the field of image recognition technology, and in particular to a deep learning method and system for printed circuit board defect recognition. The method comprises the following steps: obtaining a set of original images of a printed circuit board and performing image enhancement to obtain a set of enhanced images of the main body of the circuit board; recognizing the set of enhanced images of the main body of the circuit board to obtain a set of hierarchical functional module images of the circuit board; performing convolution extraction based on the image set of hierarchical functional modules of the circuit board to obtain hierarchical feature data of defective area images; correcting the hierarchical functional module defect image set to obtain a set of actual defective images of the hierarchical functional modules; recognizing according to the actual defective image set of the hierarchical functional modules to obtain circuit board defect pattern data; performing defect pattern feature extraction based on the circuit board defect pattern data to obtain a defect pattern feature vector set, and constructing a classifier based on the defect pattern feature vector set to obtain a printed circuit board defect recognition model. The present invention can improve the efficiency of defect recognition.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a deep learning method and system for printed circuit board defect recognition. Background Art

[0002] With the rapid development of electronic products, the design and manufacturing complexity of printed circuit boards (PCBs), as the core components of electronic devices, is increasing. PCBs not only carry the connection of electronic components, but also serve as the basis for the normal operation of electronic devices. With the advancement of technology, the size of PCBs continues to shrink and the layout becomes more compact. However, due to the small size and high-density layout of PCBs, various defects are prone to occur during the production process, such as short circuits, open circuits, poor solder joints, etc. These defects seriously affect the reliability and performance of products. Traditional PCB defect detection methods mostly rely on manual visual inspection or rule-based automated detection. These methods are usually labor-intensive and subjective, easily affected by human factors, and have certain limitations in the recognition of complex defect patterns. Summary of the invention

[0003] Based on this, it is necessary for the present invention to provide a deep learning method and system for printed circuit board defect identification to solve at least one of the above technical problems.

[0004] To achieve the above purpose, a deep learning method for printed circuit board defect recognition includes the following steps:

[0005] Step S1: obtaining a set of original images of a printed circuit board, and performing background cleaning on the original image set of the printed circuit board, thereby obtaining a set of main images of the circuit board; performing circuit board image quality enhancement on the main image set of the circuit board, thereby obtaining a set of main enhanced images of the circuit board;

[0006] Step S2: performing circuit board multi-layer main body level integration on the circuit board main body enhanced image set, thereby obtaining a circuit board main body level integrated image set, and performing circuit board functional module recognition on the circuit board main body level integrated image set, thereby obtaining a circuit board level functional module image set;

[0007] Step S3: performing hierarchical functional module defect detection based on the hierarchical functional module image set of the circuit board, thereby obtaining a hierarchical functional module defect image set, and performing hierarchical feature convolution extraction of the defect area based on the hierarchical functional module defect image set, thereby obtaining hierarchical feature data of the defect area image;

[0008] Step S4: correcting the missing circuit board depth information of the hierarchical function module defect image set based on the hierarchical feature data of the defect area image, thereby obtaining the actual defect image set of the hierarchical function module; and performing circuit board defect pattern recognition according to the actual defect image set of the hierarchical function module, thereby obtaining circuit board defect pattern data;

[0009] Step S5: Defect pattern features are extracted based on the circuit board defect pattern data to obtain a defect pattern feature vector set, and a deep learning classifier is constructed based on the defect pattern feature vector set to obtain a printed circuit board defect recognition model.

[0010] The present invention can effectively remove unnecessary interference information in the image and highlight the main part of the circuit board by acquiring the original image set of the printed circuit board (PCB) and performing background cleaning. This process lays a solid foundation for subsequent image processing and ensures that the detection algorithm can focus on key information. The clarity and readability of the image can be improved through quality enhancement technology (such as contrast enhancement and noise removal processing). This enhancement can help the subsequent algorithm to more accurately identify the details of the circuit board, thereby improving the accuracy of defect detection. Integrating the multi-layer images of the PCB can provide a more comprehensive perspective and help analyze the relationship between the layers. This hierarchical integration enables subsequent analysis to better understand the structure of the entire circuit board, thereby improving the recognition rate of functional modules. Through functional module identification, the specific role of each functional area can be determined, which is convenient for locating potential defects. This step can improve the pertinence of defect detection and reduce the probability of false detection and missed detection. Defect detection on the hierarchical functional module can quickly identify defects that affect product quality, so that timely measures can be taken. This process improves the efficiency and accuracy of detection. By extracting the hierarchical features of the defect area through a convolutional neural network, the nature and severity of the defect can be analyzed more deeply. This provides more abundant data support for subsequent defect classification and processing. Correcting the lack of depth information based on the feature data of the defect area can ensure the reliability of the defect detection results. This correction process reduces the misjudgment caused by missing information and makes defect identification more accurate. By performing pattern recognition on the actual defect image, specific defect pattern data can be extracted. This recognition process provides a data basis for subsequent defect analysis and improvement, which helps to improve the quality control level of PCB production. Extracting feature vector sets from defect pattern data provides effective input data for deep learning models. This feature extraction improves the learning efficiency and accuracy of the classifier, enabling it to better adapt to complex detection tasks. Constructing a printed circuit board defect recognition model can realize automated defect detection and classification and reduce manual intervention. This intelligent detection system improves production efficiency, reduces labor costs, and improves product reliability and consistency. In summary, the specific beneficial effects of each step significantly improve the efficiency and accuracy of PCB defect detection, reduce human errors in the production process, and ultimately ensure the stability and reliability of product quality. Through systematic processing and analysis, efficient recognition and processing of complex defect patterns can be achieved, providing strong technical support for the production of electronic products.

[0011] Optionally, step S1 specifically includes:

[0012] Step S11: obtaining a set of original images of a printed circuit board, and performing grayscale image conversion on the set of original images of the printed circuit board, thereby obtaining a set of grayscale images of the printed circuit board;

[0013] Step S12: performing pixel grayscale value statistics on the grayscale circuit board image set to obtain image pixel grayscale value data, and performing low-frequency grayscale value statistics based on the image pixel grayscale value data to obtain low-frequency pixel grayscale value data;

[0014] Step S13: performing low-frequency pixel grayscale value area image cleaning on the grayscale circuit board image set according to the low-frequency pixel grayscale value data, thereby obtaining a circuit board foreground image set;

[0015] Step S14: performing morphological edge smoothing on the circuit board foreground image set, thereby obtaining the circuit board main body image set;

[0016] Step S15: performing circuit board image quality enhancement on the circuit board main body image set, thereby obtaining a circuit board main body enhanced image set.

[0017] The present invention can reduce the amount of image data and simplify subsequent processing by converting a color image into a grayscale image. Grayscale images help to highlight the contrast of different structures and facilitate subsequent analysis and processing. Grayscale provides a unified image format for subsequent steps to ensure consistency of processing. By counting the grayscale values ​​of pixels, important features in the image can be extracted, which is helpful for subsequent analysis. Low-frequency grayscale value statistics can help identify and remove high-frequency noise in the image and improve the clarity of the image. Obtaining low-frequency grayscale value data provides important data support for subsequent image processing and can better understand the overall characteristics of the image. By cleaning the low-frequency area, unnecessary background noise can be removed, the quality of the image can be enhanced, the main features and structures of the circuit board can be highlighted, and the foreground information can be clearly visible. The cleaned image is convenient for subsequent analysis and processing, and the algorithm complexity is reduced. Morphological edge smoothing technology can make the edge of the circuit board smoother and easier to identify, improving the visual effect of the image. Through smoothing, small noise and artifacts in the image can be eliminated, thereby obtaining more accurate edge information. The smoothed image helps to improve the accuracy of subsequent analysis and feature extraction and ensure the reliability of the recognition result. Image quality enhancement can improve the overall visualization of the PCB, making details clearer and easier to observe. The enhanced image facilitates subsequent automated analysis, such as defect detection and feature recognition, and reduces manual intervention. By enhancing image quality, the overall quality of the PCB can be better ensured, thereby improving the reliability and effectiveness of the manufacturing process.

[0018] Optionally, step S15 is specifically:

[0019] Step S151: performing RGB image format conversion on the circuit board main body image set to obtain a circuit board main body conversion image set, and performing color channel separation on the circuit board main body conversion image set to obtain a color channel separation image set;

[0020] Step S152: performing full-channel adaptive histogram equalization on the color channel separated image set, thereby obtaining an enhanced color channel separated image set;

[0021] Step S153: performing color channel merging on the enhanced color channel separation image set, thereby obtaining a circuit board enhanced image set;

[0022] Step S154: enhancing the circuit board structural features according to the circuit board enhanced image set, thereby obtaining a circuit board component enhanced image set and a circuit board assembly enhanced image set;

[0023] Step S155: spatially merging the circuit board image set of the circuit board component enhanced image set and the circuit board assembly enhanced image set, thereby obtaining a circuit board body enhanced image set.

[0024] The present invention converts the image set of the circuit board body into RGB format, ensuring that the image information is stored and processed in a standard format. This conversion can effectively improve the compatibility of subsequent processing. The image set of the circuit board body is converted into RGB format, ensuring that the image information is stored and processed in a standard format. This conversion can effectively improve the compatibility of subsequent processing. By independently performing histogram equalization on each color channel, the contrast and brightness of the image are enhanced. This process helps to improve the visibility of the image, making tiny details more obvious, especially easier to identify circuits and components against complex backgrounds. The enhanced color channels are re-merged into an image, and the previous enhancement effects are combined to obtain a clearer and more colorful circuit board image, which is conducive to improving the accuracy and efficiency of subsequent analysis. Based on the enhanced image set, the structural features of the circuit board are further extracted and enhanced. This step can highlight important components on the circuit board, such as circuits, solder joints and components, making them more conspicuous in the image, thereby supporting subsequent automatic detection and recognition tasks. The circuit board enhanced image set and the component enhanced image set are spatially merged to finally generate a comprehensive circuit board main body enhanced image set. This process can integrate different enhancement effects to make the overall view of the circuit board clearer and more complete, which is helpful for subsequent analysis and decision-making.

[0025] Optionally, step S154 is specifically:

[0026] Step S1541: performing image pixel value statistics according to the circuit board enhanced image set, thereby obtaining high-frequency image pixel value data and low-frequency image pixel value data;

[0027] Step S1542: dividing the circuit board enhanced image set into high-frequency image pixel value region images according to the high-frequency image pixel value data, thereby obtaining a board region image set; dividing the circuit board enhanced image set into low-frequency image pixel value region images according to the low-frequency image pixel value data, thereby obtaining a component region image set;

[0028] Step S1543: performing board structure detail sharpening filtering processing on the board area image set to obtain a circuit board enhanced image set; performing component structure detail smoothing contour enhancement on the component area image set to obtain a circuit board component enhanced image set.

[0029] In the present invention, high-frequency pixel values ​​represent details, edges and texture information in the image, which can help identify tiny defects on the circuit board. Low-frequency pixel values ​​reflect the overall structure and large-scale changes of the image, which helps to understand the basic shape and position of the board. By performing statistics on the image, a distribution model of high-frequency and low-frequency pixel values ​​can be established. This provides data support for subsequent image processing and ensures that the processing method is more targeted. The statistical process can identify and filter out noise in the image, effectively enhancing the stability and accuracy of subsequent processing. The high-frequency pixel value region division can highlight the details of the circuit board, making it easier to identify areas with complex structures (such as solder joints and pins) and facilitate defect detection. The low-frequency pixel value region division emphasizes the overall structure and helps identify the location and layout of the components. By dividing the image into a board area and a component area, specific processing algorithms can be applied to different areas, reducing unnecessary calculations and improving overall processing efficiency. The image after region division can perform more in-depth feature extraction, such as shape, size and position, thereby providing richer data for subsequent machine learning or automated detection. Sharpening and filtering the board area helps to highlight details, improve the visibility of circuit board defects, and make it easier for the automatic detection system to identify defects. Smoothing the component area can eliminate unnecessary texture noise, enhance component recognizability, and ensure the clarity of the overall layout. The enhanced image is visually clearer, which helps engineers perform manual inspections and is also beneficial for subsequent automated analysis. The enhanced image set can be used in a variety of subsequent applications, such as machine vision systems, intelligent detection, and subsequent quality analysis, to enhance overall production efficiency and product quality.

[0030] Optionally, step S2 specifically includes:

[0031] Step S21: calculating the circuit board main body structure similarity of the circuit board main body enhanced image set, thereby obtaining circuit board main body structure similarity data;

[0032] Step S22: clustering the circuit board main body structure of the circuit board main body enhanced image set according to the circuit board main body structure similarity data, thereby obtaining a circuit board main body structure clustered image set;

[0033] Step S23: performing cluster structure pattern recognition on the cluster structure image set of the main structure of the circuit board, thereby obtaining cluster structure pattern data, and performing circuit board structure hierarchy definition according to the cluster structure pattern data, thereby obtaining cluster structure hierarchy data;

[0034] Step S24: performing multi-layer circuit board hierarchical image integration on the circuit board main structure cluster image set according to the cluster structure hierarchical data, thereby obtaining a circuit board main body hierarchical integrated image set;

[0035] Step S25: performing circuit board functional module identification on the circuit board main body level integrated image set, thereby obtaining a circuit board level functional module image set.

[0036] The present invention can accurately identify the structural similarity between different circuit boards by calculating the similarity of the main body of the circuit board, thereby reducing the possibility of misclassification. Enhanced images can remove background noise, improve the accuracy of detection, and make similarity calculation more reliable. Similarity calculation can be processed automatically, reducing the burden of manual detection and reducing the probability of human error. Cluster analysis can identify the potential patterns of different circuit board structures, thereby providing strong support for subsequent defect detection and classification. By clustering circuit boards with similar structures, the amount of data to be processed can be effectively reduced and the detection efficiency can be improved. Through clustering, the detection system can focus on circuit boards of a specific category more quickly, thereby improving the overall detection speed. Cluster structure pattern recognition helps to identify specific defect types and provides a basis for subsequent defect processing. According to the hierarchical definition of cluster structure pattern data, different circuit board functional modules can be clearly divided, enhancing the operability of subsequent analysis. By identifying different clustering patterns, the system can provide more intelligent decision support, thereby responding quickly in complex environments. Multi-layer integrated images enable detection personnel to grasp the circuit board structure as a whole and improve the efficiency of understanding and analysis. Through hierarchical integration, complex circuit board designs can be simplified into visual models that are easier to understand and operate. The integrated images can help team members collaborate better and improve communication efficiency. Through functional module identification, modules that may have defects can be quickly located, shortening troubleshooting time. Through effective functional module detection, potential faults can be discovered in a timely manner, thereby improving product reliability and performance. By analyzing the performance of functional modules, data support can be provided for the optimization of circuit board design, which will help improve the competitiveness of future products.

[0037] Optionally, step S25 is specifically:

[0038] Step S251: acquiring historical circuit board functional module structure data, and performing module component feature extraction on the historical circuit board functional module structure data, thereby obtaining functional module component feature data;

[0039] Step S252: performing hierarchical circuit board structure integration on the circuit board main body hierarchical integration image set, thereby obtaining hierarchical wiring structure data and hierarchical component structure data;

[0040] Step S253: performing wiring density statistics according to the hierarchical wiring structure data, thereby obtaining hierarchical wiring density data, and dividing the hierarchical wiring structure data into hierarchical dense wiring areas according to the hierarchical wiring density data, thereby obtaining high-density wiring area data;

[0041] Step S254: performing a hierarchical component connectivity relationship analysis based on the hierarchical component structure data to obtain hierarchical component connectivity data, and performing high connectivity component statistics based on the hierarchical component connectivity data to obtain high connectivity component data;

[0042] Step S255: performing statistics on the proportion of high-connectivity components in the region according to the high-connectivity component data, thereby obtaining data on the region with a high proportion of high-connectivity components;

[0043] Step S256: performing a region intersection operation on the high-connectivity component high-proportion region data and the high-density wiring region data, thereby obtaining module component region data;

[0044] Step S257: Perform functional module identification on the module component area data according to the functional module component feature data to obtain the functional module component area data, and divide the circuit board main body level integrated image set into functional module areas according to the functional module component area data to obtain the circuit board level functional module image set.

[0045] The present invention can construct a rich data set by extracting the functional module structure data of the historical circuit board, providing a basis for subsequent analysis. Module component feature extraction helps to understand the functions and performance of different modules, and supports optimized design and fault analysis. By analyzing historical data, common problems in the design can be found and provided as a reference for new designs. The integrated image set presents the structure of the circuit board in a hierarchical form, which is convenient for subsequent analysis and processing. Visualizing the hierarchical wiring structure and component structure improves the understanding of the circuit board design and helps designers to quickly identify problems. Through hierarchical integration, redundant parts in the design can be found, which helps to optimize the wiring and component layout. Statistical wiring density can help evaluate the wiring efficiency of the circuit board and provide data support for optimized design. Identifying high-density wiring areas helps designers avoid overly complex wiring during the design process and reduce manufacturing difficulty. Understanding wiring density helps predict the performance of the circuit board under high load and reduce the failure rate. Analyzing the connectivity relationship between components helps to understand the overall function and interaction of the system and improve the reliability of the design. By counting high-connectivity components, key components can be identified, which helps to pay attention to them in the design and ensure their performance. Connectivity relationship analysis can provide a basis for optimizing component layout and improve the overall performance and efficiency of the circuit. High-proportion areas are usually key areas for functional implementation. Understanding these areas can help prioritize their design and testing. According to the proportion of high-connectivity components, resources can be reasonably allocated to ensure the design and manufacturing quality of key areas. By performing an intersection operation on the data of high-proportion areas of high-connectivity components and the data of high-density wiring areas, important module areas can be obtained for further analysis. Identifying module component areas can better assess potential design risks and formulate corresponding solutions. The intersection results help designers to perform targeted optimization during layout and reduce unnecessary complexity. Identifying functional modules in module component areas can help designers clarify the functions of each module and improve the accuracy of defect detection. Through the division of functional module areas, the functional consistency of the circuit board design can be ensured, which is helpful for subsequent testing and verification. The obtained circuit board-level functional module image set provides an important basis for subsequent defect detection and improves work efficiency.

[0046] Optionally, step S3 specifically includes:

[0047] Step S31: extracting hierarchical functional module component layout features from the circuit board hierarchical functional module image set, thereby obtaining hierarchical functional module component layout data;

[0048] Step S32: performing component transmission distance estimation according to the hierarchical functional module component layout data, thereby obtaining module component transmission distance data, and performing module relative transmission distance statistics according to the module component transmission distance data, thereby obtaining high relative transmission distance module data and low relative transmission distance module data;

[0049] Step S33: estimating the module component density of the hierarchical functional module component layout data, thereby obtaining module component density data, and performing module relative density statistics based on the module component density data, thereby obtaining high component density module data and low component density module data;

[0050] Step S34: performing module intersection operation on the high relative transmission distance module data and the high component density module data, thereby obtaining defective function module data; performing module intersection operation on the low relative transmission distance module data and the low component density module data, thereby obtaining normal function module data;

[0051] Step S35: hierarchical spatial merging of normal function module data and defective function module data to obtain hierarchical function module defect data, and extracting defective module image sets from hierarchical function module image sets of the circuit board according to the hierarchical function module defect data to obtain hierarchical function module defect image sets;

[0052] Step S36: performing defect area hierarchical feature convolution extraction based on the hierarchical function module defect image set, thereby obtaining defect area image hierarchical feature data.

[0053] The present invention can effectively identify the layout of each component and its relationship by extracting features from the image of the functional module at the circuit board level. This lays the foundation for subsequent analysis. The extracted layout data facilitates further quantitative analysis and can provide a quantitative basis for subsequent transmission distance and density evaluation. Accurate feature extraction helps to reduce the misjudgment rate and improve the accuracy and reliability of subsequent steps. By estimating the transmission distance between components, it can be identified which modules are more functionally close and which modules may have potential communication bottlenecks. The relative transmission distance statistics between modules can provide a reference for design optimization and improve the system transmission efficiency. The extraction of high relative transmission distance module data helps to identify possible defective areas and provide a basis for subsequent defect detection. The analysis of component density helps to understand the design complexity of the circuit board and the rationality of the component layout. By statistically analyzing the data of high and low component densities, data support can be provided for the design improvement of the circuit board, reducing component interference and improving heat dissipation performance. The analysis of low component density module data helps to identify design areas that may cause performance problems. Through the intersection operation, defective modules and normal modules can be clearly distinguished, providing a data basis for subsequent fault diagnosis. Modules with high relative transmission distances and high component density are usually potential fault areas. The source of the fault can be accurately locked through module intersection operations. Effective integration of different data types makes the data of defective modules and normal modules clearer and easier for subsequent analysis. Through hierarchical space merging, the data of defective and normal modules are effectively integrated to provide a global perspective for subsequent defect analysis. The image set extraction of defective modules helps to form targeted defect identification and analysis tools. The obtained hierarchical functional module defect image set provides a basis for subsequent deep learning and image recognition, and can significantly improve the accuracy of the automatic detection system. Through hierarchical feature convolution extraction, the ability to identify defective areas can be enhanced and the accuracy of defect detection can be improved. The extraction of hierarchical features allows the defective area to be analyzed from multiple levels, helping to locate problems and conduct more in-depth fault analysis. The extracted feature data can provide support for subsequent intelligent algorithms and further realize automated fault detection and identification.

[0054] Optionally, step S4 is specifically:

[0055] Step S41: constructing a feature descriptor for the hierarchical feature data of the defect area image, thereby obtaining a defect feature descriptor;

[0056] Step S42: inferring defect depth information from the defect feature descriptor, thereby obtaining defect depth information data;

[0057] Step S43: compensating the missing depth information of the defect area image hierarchical feature data according to the defect depth information data, so as to obtain optimized defect area image hierarchical feature data;

[0058] Step S44: performing defect area perspective transformation on the hierarchical function module defect image set according to the optimized defect area image hierarchical feature data, thereby obtaining the hierarchical function module actual defect image set;

[0059] Step S45: performing circuit board defect pattern recognition according to the actual defect image set of the hierarchical function module, thereby obtaining circuit board defect pattern data.

[0060] The present invention can effectively capture defect features of different scales and types by performing hierarchical feature extraction on the image of the defect area. The constructed feature descriptor can improve the stability of the defect feature and reduce the influence of external environment changes (such as illumination, angle, etc.) on feature recognition. The construction of the feature descriptor provides the necessary basic data for subsequent depth information inference and defect recognition. The inferred defect depth information provides key data for understanding the severity of the defect, which is helpful for subsequent compensation and processing. Depth information can more accurately locate the location and nature of the defect, making it easier to take corresponding repair or treatment measures. Through depth information, engineers or detection systems can better judge the processing priority of defects, thereby optimizing resource allocation. By compensating for the missing depth information, the image features of the defect area can be optimized, making subsequent analysis more accurate. The feature data obtained after compensation will be more complete, improving the accuracy and reliability of subsequent defect detection. The optimized data helps to reduce false positives and false negatives caused by insufficient depth information. Through perspective transformation, the defect area can be observed from different angles, improving the defect visualization effect, and helping to standardize the defect area image, making subsequent processing and analysis more consistent. Perspective transformed images can be compared more easily, helping to identify the characteristics and patterns of different defects. By performing pattern recognition on actual defect image sets, automated and intelligent defect detection can be achieved, reducing the need for manual intervention. Fast and accurate defect pattern recognition can improve the efficiency of the entire inspection process, thereby speeding up the pace of the production line. The identified circuit board defect pattern data provides an important basis for quality control and improvement, and can help companies develop more effective improvement measures.

[0061] Optionally, step S42 is specifically:

[0062] Step S421: performing principal component dimensionality reduction on the defect feature descriptor to obtain a principal component defect feature descriptor;

[0063] Step S422: performing layout defect descriptor feature selection on the main component defect feature descriptor, thereby obtaining a layout defect feature descriptor;

[0064] Step S423: performing image texture coherence evaluation on the hierarchical functional module defect image set to obtain image texture coherence data, and performing depth information missing region clustering on the hierarchical functional module defect image set according to the image texture coherence data to obtain a depth information missing image;

[0065] Step S424: Correlate the layout defect feature descriptor and the depth information missing image with the defect depth information, thereby obtaining defect-depth correlation data;

[0066] Step S425: constructing a convolutional neural network deep inference model based on the historical circuit board layout defect image set and defect-depth association data, thereby obtaining a layout defect deep inference model;

[0067] Step S426: performing layout defect depth information inference on the defect feature descriptor through the layout defect depth inference model, thereby obtaining defect depth information data.

[0068] The present invention can simplify high-dimensional data into main components in low-dimensional space by reducing the dimensionality of the principal components of defect feature descriptors. This dimensionality reduction not only helps to eliminate redundant features, but also improves the computational efficiency of subsequent analysis and reduces the complexity of the model. The result is that principal component defect feature descriptors are obtained, which retain the most important information and help to classify and identify defects more accurately. Layout defect descriptor feature selection is performed on the principal component defect feature descriptors, which can further optimize the feature set and ensure that only features related to layout defects are retained. This process reduces noise, enhances the generalization ability of the model, and helps to improve detection accuracy. The final layout defect feature descriptor is more focused on information related to actual defect features. By performing image texture coherence evaluation on the hierarchical functional module defect image set, the texture features of the image can be quantitatively analyzed. This evaluation helps to identify areas in the image where depth information is missing and provides a basis for subsequent defect analysis. Through cluster analysis, the acquired depth information missing images provide a clear target for subsequent processing, which helps to concentrate resources for defect repair or improvement. The defect-depth information association between the layout defect feature descriptor and the image with missing depth information is helpful to establish a more comprehensive defect description system, so that the defect features are combined with their performance in depth information, which is conducive to a better understanding of the nature and impact of the defect, thereby providing guidance for subsequent improvement measures. Building a convolutional neural network (CNN) deep inference model based on the historical circuit board layout defect image set and defect-depth association data is an important step in achieving automated detection. This model can perform efficient defect detection and classification by learning patterns in historical data, thereby improving overall detection efficiency and accuracy. In addition, the deep inference model can self-optimize and continuously improve performance through iterative learning. Through the layout defect deep inference model, the defect feature descriptor is inferred to perform deep information, and finally the defect depth information data is obtained. This process makes it possible to quantitatively analyze the defects, provides data support for subsequent decision-making, and helps to optimize the production process and reduce the defect rate. The depth information data can be directly applied to production monitoring, quality control and the formulation of defect repair strategies.

[0069] Optionally, the present specification also provides a deep learning system for printed circuit board defect recognition, which is used to execute the deep learning method for printed circuit board defect recognition as described above, and the deep learning system for printed circuit board defect recognition includes:

[0070] The image quality enhancement module is used to obtain the original image set of the printed circuit board, and perform circuit board background cleaning on the original image set of the printed circuit board, so as to obtain the main image set of the circuit board; and perform circuit board image quality enhancement on the main image set of the circuit board, so as to obtain the main enhanced image set of the circuit board;

[0071] The function module identification module is used to perform circuit board multi-layer main body level integration on the circuit board main body enhanced image set, thereby obtaining a circuit board main body level integrated image set, and perform circuit board function module identification on the circuit board main body level integrated image set, thereby obtaining a circuit board level function module image set;

[0072] A functional module defect detection module is used to perform hierarchical functional module defect detection based on a circuit board hierarchical functional module image set, thereby obtaining a hierarchical functional module defect image set, and perform hierarchical feature convolution extraction of a defect area based on the hierarchical functional module defect image set, thereby obtaining hierarchical feature data of the defect area image;

[0073] A depth information missing correction module is used to perform circuit board depth information missing correction on the hierarchical function module defect image set based on the hierarchical feature data of the defect area image, so as to obtain the actual defect image set of the hierarchical function module; and perform circuit board defect pattern recognition based on the actual defect image set of the hierarchical function module, so as to obtain circuit board defect pattern data;

[0074] The circuit board defect recognition module is used to extract defect pattern features based on the circuit board defect pattern data to obtain a defect pattern feature vector set, and to construct a deep learning classifier based on the defect pattern feature vector set to obtain a printed circuit board defect recognition model.

[0075] The deep learning system for printed circuit board defect identification of the present invention can implement any deep learning method for printed circuit board defect identification of the present invention, and is used to combine the operations between various modules with a medium for signal transmission to complete the deep learning method for printed circuit board defect identification. The internal modules of the system cooperate with each other to improve the efficiency of defect identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments thereof made with reference to the following drawings:

[0077] Figure 1 A schematic flow chart of the steps of the deep learning method for printed circuit board defect recognition of the present invention;

[0078] Figure 2 Detailed step flow diagram of step S1 in the present invention;

[0079] Figure 3 Detailed step flow diagram of step S2 in the present invention;

[0080] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0081] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0082] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0083] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0084] To achieve this, please refer to Figures 1 to 3 The present invention provides a deep learning method for printed circuit board defect recognition, the method comprising the following steps:

[0085] Step S1: obtaining a set of original images of a printed circuit board, and performing background cleaning on the original image set of the printed circuit board, thereby obtaining a set of main images of the circuit board; performing circuit board image quality enhancement on the main image set of the circuit board, thereby obtaining a set of main enhanced images of the circuit board;

[0086] In this embodiment, in the process of obtaining the original image set of the printed circuit board, a high-resolution camera can be used to capture the circuit board image to ensure that the captured details are rich. Next, through image processing techniques such as image filtering and edge detection, background noise and interference are removed, and the coating, background impurities, etc. are removed, thereby obtaining a clean circuit board main body image set. Next, image enhancement algorithms, such as histogram equalization and local contrast enhancement, are applied to improve the visualization quality of the circuit board body, so that the details such as the components of the circuit board are more clearly visible. The goal of this step is to use the processed circuit board main body image for subsequent analysis.

[0087] Step S2: performing circuit board multi-layer main body level integration on the circuit board main body enhanced image set, thereby obtaining a circuit board main body level integrated image set, and performing circuit board functional module recognition on the circuit board main body level integrated image set, thereby obtaining a circuit board level functional module image set;

[0088] In this embodiment, for the enhanced image set of the circuit board body, a multi-layer convolutional neural network (CNN) is constructed to achieve hierarchical integration of the multi-layer body to better extract the relationship between different functional modules of the circuit board. This integration process can adopt a specific hierarchical structure, such as using a feature map fusion algorithm to merge the features output by different convolutional layers to form a more concentrated hierarchical feature set. In the functional module recognition stage, the labeled images are used for training so that the model can recognize various functional modules on the circuit board and output an image set containing module labels, thereby improving the model accuracy by increasing the sample data of different modules.

[0089] Step S3: performing hierarchical functional module defect detection based on the hierarchical functional module image set of the circuit board, thereby obtaining a hierarchical functional module defect image set, and performing hierarchical feature convolution extraction of the defect area based on the hierarchical functional module defect image set, thereby obtaining hierarchical feature data of the defect area image;

[0090] In this embodiment, based on the hierarchical functional module image set of the circuit board, the defect detection algorithm in computer vision (such as YOLO or Faster R-CNN) can be used to perform defect detection of the hierarchical functional modules, and the detected defects will be aggregated into a hierarchical functional module defect image set. Afterwards, a convolutional neural network (CNN) is used to extract multi-level features in the defect area image, which can be achieved by constructing a convolution structure of a specific level to extract local and global features, form hierarchical feature data of the defect area image, and further enhance the detection accuracy.

[0091] Step S4: correcting the missing circuit board depth information of the hierarchical function module defect image set based on the hierarchical feature data of the defect area image, thereby obtaining the actual defect image set of the hierarchical function module; and performing circuit board defect pattern recognition according to the actual defect image set of the hierarchical function module, thereby obtaining circuit board defect pattern data;

[0092] In this embodiment, hierarchical feature data is used to perform circuit board depth information missing correction, and principal component analysis (PCA) or autoencoder technology can be used to complete the missing parts by learning the features of non-missing data. The corrected data forms a hierarchical functional module actual defect image set, and then edge detection and contour analysis technology are applied to classify and mark the defect mode, and finally the circuit board defect mode data is output so as to design specific repair solutions for different types of defects.

[0093] Step S5: Defect pattern features are extracted based on the circuit board defect pattern data to obtain a defect pattern feature vector set, and a deep learning classifier is constructed based on the defect pattern feature vector set to obtain a printed circuit board defect recognition model.

[0094] In this embodiment, in the process of defect pattern feature extraction, a deep learning algorithm is applied to generate a defect pattern feature vector set, and the extracted features are learned through a convolutional neural network or a long short-term memory network (LSTM). Through training, a deep learning classifier is constructed. This model can identify different types of defects in the circuit board. When inferring new images in the future, the model can be used for real-time defect detection and classification. Finally, the classifier can be deployed on the circuit board production line to realize automatic defect detection to improve production efficiency and product quality.

[0095] Optionally, step S1 specifically includes:

[0096] Step S11: obtaining a set of original images of a printed circuit board, and performing grayscale image conversion on the set of original images of the printed circuit board, thereby obtaining a set of grayscale images of the printed circuit board;

[0097] In this embodiment, a high-resolution camera is used to capture multiple viewing angles of the printed circuit board to ensure that the image clarity reaches more than 600 DPI to capture the details of the circuit board. Next, the color image is converted into a grayscale image through grayscale processing using image processing software (such as OpenCV), and the formula Y = 0.299R + 0.587G + 0.114*B is used to calculate the brightness value of each pixel. The final output grayscale circuit board image set will include all converted grayscale images for use in subsequent steps.

[0098] Step S12: performing pixel grayscale value statistics on the grayscale circuit board image set to obtain image pixel grayscale value data, and performing low-frequency grayscale value statistics based on the image pixel grayscale value data to obtain low-frequency pixel grayscale value data;

[0099] In this embodiment, each grayscale image is statistically analyzed. The Python programming language and NumPy library are used to read the grayscale image data and calculate the number of pixels for each grayscale value (0-255). It is also necessary to screen the obtained grayscale value histogram, count the low-frequency grayscale values ​​(for example, grayscale values ​​below 128), and generate a corresponding data set for subsequent processing. Such statistics can help identify dark areas or low-contrast areas in the image and provide a basis for subsequent grayscale processing.

[0100] Step S13: performing low-frequency pixel grayscale value area image cleaning on the grayscale circuit board image set according to the low-frequency pixel grayscale value data, thereby obtaining a circuit board foreground image set;

[0101] In this embodiment, the low-frequency grayscale area is determined by image thresholding technology based on the obtained low-frequency grayscale value data. Specifically, the Otsu algorithm is used to automatically calculate a suitable threshold, and the pixels in the low-frequency area are set as the background, thereby achieving image cleaning. After processing, the obtained circuit board foreground image set will only contain the high-frequency information part, remove noise and background interference, and improve the usability of the image.

[0102] Step S14: performing morphological edge smoothing on the circuit board foreground image set, thereby obtaining the circuit board main body image set;

[0103] In this embodiment, a morphological method is applied to process the foreground image. Specifically, dilation and erosion operations are used, and a 3x3 structural element is used to smooth the edge. First, the small noise points in the foreground image are removed by the erosion operation, and then the dilation operation is applied to restore the object boundary and remove the small fractures. The final output of the circuit board main body image set will show clearer edges, which is conducive to the subsequent quality enhancement processing.

[0104] Step S15: performing circuit board image quality enhancement on the circuit board main body image set, thereby obtaining a circuit board main body enhanced image set.

[0105] In this embodiment, image enhancement technology is used to improve the visual effect of the main image of the circuit board. Specifically, histogram equalization is used to enhance the image contrast, and a sharpening filter (such as a Laplace filter) is combined to further improve edge clarity. In addition, gamma correction can be used to adjust the image brightness to make the image details more prominent. The final enhanced image set of the main body of the circuit board has a higher visual quality and is suitable for further analysis or production inspection.

[0106] Optionally, step S15 is specifically:

[0107] Step S151: performing RGB image format conversion on the circuit board main body image set to obtain a circuit board main body conversion image set, and performing color channel separation on the circuit board main body conversion image set to obtain a color channel separation image set;

[0108] In this embodiment, the original circuit board main body image set is converted from grayscale format to RGB format, which means that the color mode of each image is adjusted to three channels of red, green and blue. The image format conversion can be realized by calling the corresponding function using OpenCV or PIL library. The resolution and size of the image should also be considered to ensure that the converted image remains high quality. After completion, color channel separation is performed, and each image can be split into three separate channels using NumPy arrays, and the red, green and blue channels are extracted respectively, and saved as three different image sets for subsequent processing.

[0109] Step S152: performing full-channel adaptive histogram equalization on the color channel separated image set, thereby obtaining an enhanced color channel separated image set;

[0110] In this embodiment, the obtained color channel separated image set is subjected to full channel adaptive histogram equalization (CLAHE). This technique can enhance the contrast of the image, especially in low contrast areas. By using the CLAHE function in the OpenCV library, local histogram equalization can be performed on each separated color channel. The parameter setting can consider the size of each small block (e.g., 8x8 or 16x16 pixels) and contrast limit to prevent over-enhancement. After the operation is completed, an image set with more obvious visual effects for each channel will be obtained.

[0111] Step S153: performing color channel merging on the enhanced color channel separation image set, thereby obtaining a circuit board enhanced image set;

[0112] In this embodiment, the processed enhanced color channel separation image set is then subjected to color channel merging. The enhanced red, green, and blue channels can be recombined into a new color image using the image operation functions of OpenCV or PIL. In this process, the consistency of the channel order must be ensured to avoid color deviation. The resulting circuit board enhanced image set should be clearer and more layered than the original image in terms of overall visual effect.

[0113] Step S154: enhancing the circuit board structural features according to the circuit board enhanced image set, thereby obtaining a circuit board component enhanced image set and a circuit board assembly enhanced image set;

[0114] In this embodiment, the circuit board structural features are enhanced based on the circuit board enhanced image set. Edge detection algorithms (such as Canny or Sobel operators) can be applied to extract edge information of boards and components. In addition, morphological operations (such as expansion and corrosion) can be applied to better highlight structural features. By analyzing the features in the image, two sets of new image sets with enhanced color palette board and component features are generated. This process can be achieved through a custom function, and the sensitivity of the algorithm can be adjusted using specific parameters.

[0115] Step S155: spatially merging the circuit board image set of the circuit board component enhanced image set and the circuit board assembly enhanced image set, thereby obtaining a circuit board body enhanced image set.

[0116] In this embodiment, the circuit board image space is merged for the circuit board panel enhanced image set and the circuit board component enhanced image set. By merging the two enhanced image sets at the pixel level, for example, using a weighted average or maximization method, useful information can be better retained. The circuit board main body enhanced image set obtained after merging will combine all the previous enhancement processing results to generate an image that is more complete in details and structure. The final enhancement effect can be evaluated through visualization tools, and the merging strategy can be further adjusted as needed.

[0117] Optionally, step S154 is specifically:

[0118] Step S1541: performing image pixel value statistics according to the circuit board enhanced image set, thereby obtaining high-frequency image pixel value data and low-frequency image pixel value data;

[0119] In this embodiment, by collecting and analyzing all image data in the circuit board enhanced image set, it is first necessary to pre-process the image, such as denoising, standardization, etc., to improve the accuracy of subsequent pixel value statistics. Image processing algorithms (such as histogram analysis) are used to count the brightness and color distribution of each pixel, especially high-frequency and low-frequency pixel value data. High-frequency pixel values ​​usually represent the details in the image (such as circuits and solder joints), while low-frequency pixel values ​​mainly represent large plane areas (such as the background of the PCB substrate). In order to separate these two types of information accordingly, a threshold value can be set, for example, a high-frequency pixel value is defined as a pixel greater than a certain brightness value, and a low-frequency pixel value is defined as a pixel less than the value. The final result is the statistical result of the high-frequency image pixel value data and the low-frequency image pixel value data, which is used for subsequent area division.

[0120] Step S1542: dividing the circuit board enhanced image set into high-frequency image pixel value region images according to the high-frequency image pixel value data, thereby obtaining a board region image set; dividing the circuit board enhanced image set into low-frequency image pixel value region images according to the low-frequency image pixel value data, thereby obtaining a component region image set;

[0121] In this embodiment, the obtained high-frequency image pixel value data is used to divide the circuit board enhanced image set into regions. By setting the threshold of the high-frequency region, the part of the image that mainly contains high-frequency details is extracted to form a board area image set. This can be achieved through a contour detection algorithm (such as Canny edge detection) to ensure that the extracted area has a good boundary definition. Similarly, based on the low-frequency image pixel value data, a smoothing filtering technique (such as mean filtering or Gaussian filtering) is used to identify and extract component areas to obtain a component area image set. For each area image set, the regional growth method is used to further optimize the connectivity of the regional contour, thereby ensuring the accuracy and completeness of the regional division and preparing for subsequent enhancement processing.

[0122] Step S1543: performing board structure detail sharpening filtering processing on the board area image set to obtain a circuit board enhanced image set; performing component structure detail smoothing contour enhancement on the component area image set to obtain a circuit board component enhanced image set.

[0123] In this embodiment, for the panel area image set, a sharpening filter (such as UnsharpMasking or Laplacian filtering) is first applied to enhance the structural details of the circuit board. This process blurs the input image twice and subtracts it to highlight the details. The key parameters of this operation (such as the degree of sharpening) are adjusted according to the characteristics of the specific circuit board to prevent excessive sharpening from causing distortion of details. At the same time, for the component area image set, smooth contour enhancement technology, such as morphological closure processing, is applied to eliminate noise and smooth component edges. By carefully adjusting the size and shape of the structural elements, it can be ensured that the enhanced image maintains the original shape characteristics while significantly improving the visual effect. Finally, the two enhanced image sets generated will be used for subsequent analysis and detection to improve the accuracy and efficiency of circuit board fault diagnosis.

[0124] Optionally, step S2 specifically includes:

[0125] Step S21: calculating the circuit board main body structure similarity of the circuit board main body enhanced image set, thereby obtaining circuit board main body structure similarity data;

[0126] In this embodiment, the enhanced image set of the circuit board body needs to be processed first. The convolutional neural network (CNN) model in deep learning can be used to extract the feature vector of each image. The structural similarity between images is quantified by using methods such as cosine similarity calculation or Euclidean distance. For example, the circuit board image is feature extracted using a pre-trained ResNet network to obtain a 128-dimensional feature vector, and then the similarity between any two images is calculated to generate a similarity matrix.

[0127] Step S22: clustering the circuit board main body structure of the circuit board main body enhanced image set according to the circuit board main body structure similarity data, thereby obtaining a circuit board main body structure clustered image set;

[0128] In this embodiment, based on the obtained similarity data, a clustering algorithm (such as K-means, DBSCAN or hierarchical clustering) is used to group the main structure of the circuit board. An appropriate number of clusters (k value) is selected, such as k=5 based on the elbow rule, and then the circuit board images are divided into 5 clusters according to similarity. Each cluster contains circuit board images with similar structures, and the clustering results can be displayed through visualization (such as t-SNE).

[0129] Step S23: performing cluster structure pattern recognition on the cluster structure image set of the main structure of the circuit board, thereby obtaining cluster structure pattern data, and performing circuit board structure hierarchy definition according to the cluster structure pattern data, thereby obtaining cluster structure hierarchy data;

[0130] In this embodiment, pattern recognition is performed on the clustered images, and techniques such as row and column detection and shape analysis are used to identify different clustering structure patterns. Pattern recognition algorithms such as HOG (Histogram of Oriented Gradients) feature extraction and support vector machine (SVM) classification methods can be introduced, and the defined patterns can be "layered", "interlaced", etc., and corresponding structural levels are defined for each pattern. For example, the identified "layered" circuit board can be defined as having a three-layer structure from high to low.

[0131] Step S24: performing multi-layer circuit board hierarchical image integration on the circuit board main structure cluster image set according to the cluster structure hierarchical data, thereby obtaining a circuit board main body hierarchical integrated image set;

[0132] In this embodiment, image integration is performed based on the obtained cluster structure hierarchical data. Through image stitching technology, the circuit board images of the same level are synthesized to generate a multi-level circuit board image set. The image stitching function in the OpenCV library can be used to set fusion parameters, such as transparency, to ensure that the connection between different levels is natural and smooth. For example, the circuit board images classified into the "common" level are merged to form an integrated image containing all the features of this level.

[0133] Step S25: performing circuit board functional module identification on the circuit board main body level integrated image set, thereby obtaining a circuit board level functional module image set.

[0134] In this embodiment, after completing the hierarchical integration image set of the main body of the circuit board, the functional modules are identified. The classification function of the convolutional neural network can be used to perform functional analysis on the integrated image to detect functional nodes such as power modules and signal modules. For example, a deep learning model is trained to identify modules such as power supplies and amplifiers, and finally a hierarchical functional module image set is generated, each identified module and its position are marked, and a visual functional layout diagram is formed.

[0135] Optionally, step S25 is specifically:

[0136] Step S251: acquiring historical circuit board functional module structure data, and performing module component feature extraction on the historical circuit board functional module structure data, thereby obtaining functional module component feature data;

[0137] In this embodiment, a computer vision algorithm is used to extract functional module structure data from a stored and defined circuit board image. The shape, size and connection point of each module are extracted through a deep learning model, and the features of the module components are identified using a convolutional neural network (CNN) to form a database containing component features, such as the area, boundary, connection point location and other information of each module.

[0138] Step S252: performing hierarchical circuit board structure integration on the circuit board main body hierarchical integration image set, thereby obtaining hierarchical wiring structure data and hierarchical component structure data;

[0139] In this embodiment, image processing techniques such as edge detection and image segmentation are used to integrate the hierarchical structure of the circuit board body. Components at different levels are combined, and the accuracy of structural recognition at different resolutions is improved by generating an image pyramid, and a data set of hierarchical wiring structure is generated to record wiring and component information at each level.

[0140] Step S253: performing wiring density statistics according to the hierarchical wiring structure data, thereby obtaining hierarchical wiring density data, and dividing the hierarchical wiring structure data into hierarchical dense wiring areas according to the hierarchical wiring density data, thereby obtaining high-density wiring area data;

[0141] In this embodiment, the wiring density of each area is calculated using a point density statistical method based on the integrated hierarchical wiring structure. For example, a density map is formed by counting the number of lines per square centimeter. Afterwards, these data are input into a clustering algorithm to divide high-density wiring areas, such as marking areas with wiring density higher than a set threshold as high-density areas.

[0142] Step S254: performing a hierarchical component connectivity relationship analysis based on the hierarchical component structure data to obtain hierarchical component connectivity data, and performing high connectivity component statistics based on the hierarchical component connectivity data to obtain high connectivity component data;

[0143] In this embodiment, the connectivity algorithm in graph theory is used to analyze the connection relationship between components. A connectivity graph between components is constructed, the number of connection points of each component is counted, and components with a number of connections exceeding a certain value are identified, such as those that are connected to five or more other components. In this way, a list of components with high connectivity is formed, which can provide data support for subsequent analysis.

[0144] Step S255: performing statistics on the proportion of high-connectivity components in the region according to the high-connectivity component data, thereby obtaining data on the region with a high proportion of high-connectivity components;

[0145] In this embodiment, a statistical method is used to analyze the proportion of high-connectivity components. First, the proportion of high-connectivity components in the entire component is calculated. Assuming that among 100 components, 20 are high-connectivity components, then their proportion is 20%. This data will be used to evaluate the impact of a specific area on the overall performance of the circuit board.

[0146] Step S256: performing a region intersection operation on the high-connectivity component high-proportion region data and the high-density wiring region data, thereby obtaining module component region data;

[0147] In this embodiment, spatial data analysis technology is used to perform intersection operations on high-connectivity component high-proportion areas and high-density wiring areas. GIS (Geographic Information System) software is used to create a regional map containing the overlapping parts of the two, identify which part of the components and wiring density is the highest, and form a data layer of the module component area.

[0148] Step S257: Perform functional module identification on the module component area data according to the functional module component feature data to obtain the functional module component area data, and divide the circuit board main body level integrated image set into functional module areas according to the functional module component area data to obtain the circuit board level functional module image set.

[0149] In this embodiment, the intersection area is analyzed by a machine learning algorithm in combination with the functional module component feature data to identify specific functional modules. For example, based on the defined functional module features, a classification algorithm is used to identify a region of a certain shape as a power module, and automatic labeling is achieved. Subsequently, labeling is performed on the original circuit board image to generate a new set of functional module area images, providing a basis for circuit board design.

[0150] Optionally, step S3 specifically includes:

[0151] Step S31: extracting hierarchical functional module component layout features from the circuit board hierarchical functional module image set, thereby obtaining hierarchical functional module component layout data;

[0152] In this embodiment, an image processing algorithm is used to process the image set of the circuit board to extract the layout features of each functional module. Edge detection and region segmentation techniques can be used to identify the edges in the image using the Canny algorithm, and then combined with the image contour extraction technology to obtain the shape information of each module. Next, a machine learning model (such as a convolutional neural network) is used to classify and identify these modules, thereby forming a data set containing module size, position, shape and other features. For example, if a group of resistor modules with a spacing of 2mm is detected in a circuit board, the system will record its position coordinates (x, y) and its size characteristics. It is also possible to extract the size, position, shape, and component distribution of the hierarchical functional modules contained in the defined circuit board hierarchical functional module image set.

[0153] Step S32: performing component transmission distance estimation according to the hierarchical functional module component layout data, thereby obtaining module component transmission distance data, and performing module relative transmission distance statistics according to the module component transmission distance data, thereby obtaining high relative transmission distance module data and low relative transmission distance module data;

[0154] In this embodiment, based on the obtained module layout data, the transmission distance between each module is calculated. Specifically, a geometric distance formula (such as Euclidean distance) can be used to calculate each pair of modules to evaluate the relative transmission distance. For example, if module A is at position (10, 15) and module B is at (30, 35), the transmission distance between the two can be calculated as Through these data, the relative transmission distance is counted, and a threshold (such as 5mm) is set to distinguish modules with high relative transmission distance (>5mm) and low relative transmission distance (≤5mm), forming a corresponding data set.

[0155] Step S33: estimating the module component density of the hierarchical functional module component layout data, thereby obtaining module component density data, and performing module relative density statistics based on the module component density data, thereby obtaining high component density module data and low component density module data;

[0156] In this embodiment, the extracted layout data is used to estimate the density of module components. The density can be determined by calculating the number of modules per unit area. For example, in a certain area, if there are 10 modules distributed in 1 square centimeter, the density is 10 / cm². Then, the density data is counted, and thresholds of high density (>15 / cm²) and low density (≤15 / cm²) are set to form two types of module data sets for subsequent processing.

[0157] Step S34: performing module intersection operation on the high relative transmission distance module data and the high component density module data, thereby obtaining defective function module data; performing module intersection operation on the low relative transmission distance module data and the low component density module data, thereby obtaining normal function module data;

[0158] In this embodiment, the data of the high relative transmission distance module and the high component density module are used to perform an intersection operation to identify the potential defective functional modules. This can be achieved through set operations (such as set operations in Python), for example, performing an intersection operation on the module ID of the high relative transmission distance and the module ID of the high component density. At the same time, the intersection operation is also performed on the low relative transmission distance module and the low component density module to identify the normal functional modules. The result of this process is a specific list of defective and normal modules.

[0159] Step S35: hierarchical spatial merging of normal function module data and defective function module data to obtain hierarchical function module defect data, and extracting defective module image sets from hierarchical function module image sets of the circuit board according to the hierarchical function module defect data to obtain hierarchical function module defect image sets;

[0160] In this embodiment, the normal function module data and the defective function module data are hierarchically spatially merged. Specifically, data fusion technology is used, and the weighted average method or principal component analysis (PCA) is used to merge the module features. After the merger, the spatial position of the defective module contained in the hierarchical function module defect data is used, and the defective module image is extracted from the circuit board image set through the aforementioned image feature extraction method to obtain a hierarchical function module defect image set. For example, the defective module extracted from the merged data will show obvious damage or missing solder joints in the image.

[0161] Step S36: performing defect area hierarchical feature convolution extraction based on the hierarchical function module defect image set, thereby obtaining defect area image hierarchical feature data.

[0162] In this embodiment, based on the hierarchical functional module defect image set, a convolutional neural network (CNN) in deep learning is used for feature extraction. A pre-trained model (such as VGG16 or ResNet) can be used to extract the hierarchical features of the defect area, and a multi-layer feature map (feature maps) is obtained by performing a convolution operation on each defect image. For example, if a solder joint defect of a certain module is identified, the shape, size, and position of the defect can be analyzed through feature extraction of multi-layer convolution, forming detailed hierarchical feature data of the defect area image.

[0163] Optionally, step S4 is specifically:

[0164] Step S41: constructing a feature descriptor for the hierarchical feature data of the defect area image, thereby obtaining a defect feature descriptor;

[0165] In this embodiment, hierarchical feature data can be extracted from the defect area image using a convolutional neural network (CNN). First, the defect area image is preprocessed, including denoising and normalization. Then, a CNN model is constructed using a deep learning framework (such as TensorFlow or PyTorch), and multiple convolutional layers and pooling layers are designed to capture features at different levels. The model is trained with a large number of annotated defect images to extract features such as texture, shape, and color, and finally generates a defect feature descriptor to represent the unique feature information of each defect.

[0166] Step S42: inferring defect depth information from the defect feature descriptor, thereby obtaining defect depth information data;

[0167] In this embodiment, deep learning technology is applied to infer the depth information of the defect feature descriptor. Specifically, a regression model, such as a long short-term memory network (LSTM), is used to input the defect feature descriptor, and the depth information of the defect is learned through training data. The model needs to be trained with a labeled training data set, defect instances containing various depth information, and output defect depth information data, such as the actual depth of the defect and the distance from the surface of the circuit board. The accuracy of the model is optimized by calculating the loss function to ensure the reliability of the depth information inference.

[0168] Step S43: compensating the missing depth information of the defect area image hierarchical feature data according to the defect depth information data, so as to obtain optimized defect area image hierarchical feature data;

[0169] In this embodiment, the hierarchical feature data of the defect area image is compensated for the defect depth information data. Specifically, an interpolation algorithm (such as bilinear interpolation or cubic spline interpolation) is used to reconstruct the depth data of the defect area. On this basis, combined with the defect feature descriptor, the depth compensation method is used to generate optimized defect area feature data. The compensation process involves using known depth values ​​to infer the depth information of the unmeasured area, ensuring that the optimized feature data finally generated can more accurately reflect the actual state of the defect, thereby improving the accuracy of subsequent analysis.

[0170] Step S44: performing defect area perspective transformation on the hierarchical function module defect image set according to the optimized defect area image hierarchical feature data, thereby obtaining the hierarchical function module actual defect image set;

[0171] In this embodiment, the perspective transformation of the defect area is performed based on the optimized hierarchical feature data of the defect area image. Specifically, the perspective transformation algorithm (such as cv2.warpPerspective in OpenCV) is applied to calculate the transformation matrix based on the known defect feature descriptor and depth information. By inputting the defect area image and the transformation matrix, an actual defect image set is generated, and the relative position and shape of the defect in space are maintained, which is convenient for subsequent analysis and processing.

[0172] Step S45: performing circuit board defect pattern recognition according to the actual defect image set of the hierarchical function module, thereby obtaining circuit board defect pattern data.

[0173] In this embodiment, the actual defect image set of the hierarchical function module is used to identify the defect pattern of the circuit board. Specifically, a classification algorithm (such as a support vector machine SVM or a deep learning model) is used to train and test the defect images. Prepare a labeled defect image data set, including various common circuit board defect types, such as short circuits, open circuits, and solder joint defects. By extracting image features (such as edges, shapes, and textures), input them into the classification model for learning, and adjust the model parameters through the verification set. Finally, the new defect image is identified through the model to obtain the circuit board defect pattern data for subsequent processing and decision support.

[0174] Optionally, step S42 is specifically:

[0175] Step S421: performing principal component dimensionality reduction on the defect feature descriptor to obtain a principal component defect feature descriptor;

[0176] In this embodiment, data is extracted from the original defect feature descriptor. Taking a PCB (printed circuit board) sample as an example, the original features include the size, shape, color distribution, etc. of the defect. These high-dimensional features are reduced in dimension using principal component analysis (PCA). In specific implementation, the number of principal components can be set to 5 through Python's sklearn.decomposition.PCA module, thereby reducing the descriptor from 100 dimensions to 5 dimensions. After dimensionality reduction, the principal component defect feature descriptor can more effectively represent defect features and reduce computational complexity.

[0177] Step S422: performing layout defect descriptor feature selection on the main component defect feature descriptor, thereby obtaining a layout defect feature descriptor;

[0178] In this embodiment, after obtaining the principal component defect feature descriptor, the next step is to select the layout defect descriptor features. Specifically, LASSO regression can be used for feature selection, and the importance of features can be evaluated through cross-validation. Features that are strongly correlated with layout defects (such as circuit shorts or opens) are selected, such as the spatial distribution characteristics of defects and the electrical characteristics of adjacent components. This process will produce a set of optimized layout defect feature descriptors for subsequent analysis.

[0179] Step S423: performing image texture coherence evaluation on the hierarchical functional module defect image set to obtain image texture coherence data, and performing depth information missing region clustering on the hierarchical functional module defect image set according to the image texture coherence data to obtain a depth information missing image;

[0180] In this embodiment, image processing technology is used to evaluate the consistency of image texture for the defective image set of the hierarchical functional module. Taking the gray level co-occurrence matrix (GLCM) of the image as an example, texture features (such as contrast and homogeneity) are calculated to evaluate the consistency of the texture. This process can be implemented using OpenCV and Scikit-image libraries. With these data, the defective images of the hierarchical functional module are clustered in areas where depth information is missing (such as K-means clustering), thereby identifying images with missing depth information, such as detecting defective images at the edge or in areas with insufficient lighting.

[0181] Step S424: Correlate the layout defect feature descriptor and the depth information missing image with the defect depth information, thereby obtaining defect-depth correlation data;

[0182] In this embodiment, the layout defect feature descriptor is associated with the image with missing depth information. By building a multi-level perceptron model, the feature descriptor is combined with the depth information of the defect image to obtain defect-depth association data. Specifically, TensorFlow or PyTorch framework can be used to build a neural network with hidden layers to optimize the correlation between defect features and depth information. This process can generate specific data on the depth information corresponding to various types of defects, providing a more in-depth analysis basis.

[0183] Step S425: constructing a convolutional neural network deep inference model based on the historical circuit board layout defect image set and defect-depth association data, thereby obtaining a layout defect deep inference model;

[0184] In this embodiment, a convolutional neural network (CNN) deep inference model is constructed based on the historical circuit board layout defect image set and defect-depth association data. Specifically, the ResNet or VGG architecture can be selected, and pre-trained on ImageNet, and then fine-tuned on the own defect image set. The accuracy of the model can be improved by setting appropriate hyperparameters, such as learning rate, batch size, number of convolution layers, etc. After training, the model is saved and performance verification is performed to ensure its generalization ability on unknown data.

[0185] Step S426: performing layout defect depth information inference on the defect feature descriptor through the layout defect depth inference model, thereby obtaining defect depth information data.

[0186] In this embodiment, the constructed layout defect depth inference model is used to infer the depth information of the defect feature descriptor. Specifically, the model can be input into the preprocessed defect feature data, and forward propagation can be performed to obtain the depth information data of each defect. Evaluation indicators (such as mean square error MSE) can be used to verify the accuracy of the inference results to ensure that the inferred depth information can effectively reflect the actual layout defect depth.

[0187] Optionally, the present specification also provides a deep learning system for printed circuit board defect recognition, which is used to execute the deep learning method for printed circuit board defect recognition as described above, and the deep learning system for printed circuit board defect recognition includes:

[0188] The image quality enhancement module is used to obtain the original image set of the printed circuit board, and perform circuit board background cleaning on the original image set of the printed circuit board, so as to obtain the main image set of the circuit board; and perform circuit board image quality enhancement on the main image set of the circuit board, so as to obtain the main enhanced image set of the circuit board;

[0189] The function module identification module is used to perform circuit board multi-layer main body level integration on the circuit board main body enhanced image set, thereby obtaining a circuit board main body level integrated image set, and perform circuit board function module identification on the circuit board main body level integrated image set, thereby obtaining a circuit board level function module image set;

[0190] A functional module defect detection module is used to perform hierarchical functional module defect detection based on a circuit board hierarchical functional module image set, thereby obtaining a hierarchical functional module defect image set, and perform hierarchical feature convolution extraction of a defect area based on the hierarchical functional module defect image set, thereby obtaining hierarchical feature data of the defect area image;

[0191] A depth information missing correction module is used to perform circuit board depth information missing correction on the hierarchical function module defect image set based on the hierarchical feature data of the defect area image, so as to obtain the actual defect image set of the hierarchical function module; and perform circuit board defect pattern recognition based on the actual defect image set of the hierarchical function module, so as to obtain circuit board defect pattern data;

[0192] The circuit board defect recognition module is used to extract defect pattern features based on the circuit board defect pattern data to obtain a defect pattern feature vector set, and to construct a deep learning classifier based on the defect pattern feature vector set to obtain a printed circuit board defect recognition model.

[0193] The deep learning system for printed circuit board defect identification of the present invention can implement any deep learning method for printed circuit board defect identification of the present invention, and is used to combine the operations between various modules with a medium for signal transmission to complete the deep learning method for printed circuit board defect identification. The internal modules of the system cooperate with each other to improve the efficiency of defect identification.

[0194] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0195] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A deep learning method for printed circuit board defect recognition, characterized in that: The following steps are involved: Step S1: obtaining a set of original images of a printed circuit board, and performing background cleaning on the original image set of the printed circuit board, thereby obtaining a set of main images of the circuit board; Performing circuit board image quality enhancement on the circuit board main body image set, thereby obtaining a circuit board main body enhanced image set; Step S2: performing circuit board multi-layer main body level integration on the circuit board main body enhanced image set, thereby obtaining a circuit board main body level integrated image set, and performing circuit board functional module recognition on the circuit board main body level integrated image set, thereby obtaining a circuit board level functional module image set; Step S3: Performing hierarchical functional module defect detection based on the hierarchical functional module image set of the circuit board, thereby obtaining a hierarchical functional module defect image set, and performing hierarchical feature convolution extraction of the defect area based on the hierarchical functional module defect image set, thereby obtaining hierarchical feature data of the defect area image. Step S3 is specifically as follows: Step S31: extracting hierarchical functional module component layout features from the circuit board hierarchical functional module image set, thereby obtaining hierarchical functional module component layout data; Step S32: performing component transmission distance estimation according to the hierarchical functional module component layout data, thereby obtaining module component transmission distance data, and performing module relative transmission distance statistics according to the module component transmission distance data, thereby obtaining high relative transmission distance module data and low relative transmission distance module data; Step S33: estimating the module component density of the hierarchical functional module component layout data, thereby obtaining module component density data, and performing module relative density statistics based on the module component density data, thereby obtaining high component density module data and low component density module data; Step S34: performing module intersection operation on the high relative transmission distance module data and the high component density module data, thereby obtaining defective function module data; performing module intersection operation on the low relative transmission distance module data and the low component density module data, thereby obtaining normal function module data; Step S35: hierarchical spatial merging of normal function module data and defective function module data to obtain hierarchical function module defect data, and extracting defective module image sets from hierarchical function module image sets of the circuit board according to the hierarchical function module defect data to obtain hierarchical function module defect image sets; Step S36: performing defect region hierarchical feature convolution extraction based on the hierarchical function module defect image set, thereby obtaining defect region image hierarchical feature data; Step S4: correcting the missing circuit board depth information of the hierarchical function module defect image set based on the hierarchical feature data of the defect area image, thereby obtaining the actual defect image set of the hierarchical function module; and performing circuit board defect pattern recognition according to the actual defect image set of the hierarchical function module, thereby obtaining circuit board defect pattern data; Step S5: Defect pattern features are extracted based on the circuit board defect pattern data to obtain a defect pattern feature vector set, and a deep learning classifier is constructed based on the defect pattern feature vector set to obtain a printed circuit board defect recognition model.

2. The deep learning method for printed circuit board defect recognition according to claim 1, characterized in that: Step S1 is specifically as follows: Step S11: obtaining a set of original images of a printed circuit board, and performing grayscale image conversion on the set of original images of the printed circuit board, thereby obtaining a set of grayscale images of the printed circuit board; Step S12: performing pixel grayscale value statistics on the grayscale circuit board image set to obtain image pixel grayscale value data, and performing low-frequency grayscale value statistics based on the image pixel grayscale value data to obtain low-frequency pixel grayscale value data; Step S13: performing low-frequency pixel gray value area image cleaning on the grayscale circuit board image set according to the low-frequency pixel gray value data, thereby obtaining a circuit board foreground image set; Step S14: performing morphological edge smoothing on the circuit board foreground image set, thereby obtaining the circuit board main body image set; Step S15: performing circuit board image quality enhancement on the circuit board main body image set, thereby obtaining a circuit board main body enhanced image set.

3. The deep learning method for printed circuit board defect recognition according to claim 2, characterized in that: Step S15 is specifically as follows: Step S151: performing RGB image format conversion on the circuit board main body image set to obtain a circuit board main body conversion image set, and performing color channel separation on the circuit board main body conversion image set to obtain a color channel separation image set; Step S152: performing full-channel adaptive histogram equalization on the color channel separated image set, thereby obtaining an enhanced color channel separated image set; Step S153: performing color channel merging on the enhanced color channel separation image set, thereby obtaining a circuit board enhanced image set; Step S154: enhancing the circuit board structural features according to the circuit board enhanced image set, thereby obtaining a circuit board component enhanced image set and a circuit board assembly enhanced image set; Step S155: spatially merging the circuit board image set of the circuit board component enhanced image set and the circuit board assembly enhanced image set, thereby obtaining a circuit board body enhanced image set.

4. The deep learning method for printed circuit board defect recognition according to claim 2, characterized in that: Step S154 is specifically as follows: Step S1541: performing image pixel value statistics according to the circuit board enhanced image set, thereby obtaining high-frequency image pixel value data and low-frequency image pixel value data; Step S1542: dividing the circuit board enhanced image set into high-frequency image pixel value region images according to the high-frequency image pixel value data, thereby obtaining a board region image set; Dividing the circuit board enhanced image set into low-frequency image pixel value region images according to the low-frequency image pixel value data, thereby obtaining a component region image set; Step S1543: performing board structure detail sharpening filtering processing on the board area image set to obtain a circuit board enhanced image set; performing component structure detail smoothing contour enhancement on the component area image set to obtain a circuit board component enhanced image set.

5. The deep learning method for printed circuit board defect recognition according to claim 1, characterized in that: Step S2 is specifically as follows: Step S21: calculating the circuit board main body structure similarity of the circuit board main body enhanced image set, thereby obtaining circuit board main body structure similarity data; Step S22: clustering the circuit board main body structure of the circuit board main body enhanced image set according to the circuit board main body structure similarity data, thereby obtaining a circuit board main body structure clustered image set; Step S23: performing cluster structure pattern recognition on the cluster structure image set of the main structure of the circuit board, thereby obtaining cluster structure pattern data, and performing circuit board structure hierarchy definition according to the cluster structure pattern data, thereby obtaining cluster structure hierarchy data; Step S24: performing multi-layer circuit board hierarchical image integration on the circuit board main structure cluster image set according to the cluster structure hierarchical data, thereby obtaining a circuit board main body hierarchical integrated image set; Step S25: performing circuit board functional module identification on the circuit board main body level integrated image set, thereby obtaining a circuit board level functional module image set.

6. The deep learning method for printed circuit board defect recognition according to claim 5, characterized in that: Step S25 is specifically as follows: Step S251: acquiring historical circuit board functional module structure data, and performing module component feature extraction on the historical circuit board functional module structure data, thereby obtaining functional module component feature data; Step S252: performing hierarchical circuit board structure integration on the circuit board main body hierarchical integration image set, thereby obtaining hierarchical wiring structure data and hierarchical component structure data; Step S253: performing wiring density statistics according to the hierarchical wiring structure data, thereby obtaining hierarchical wiring density data, and dividing the hierarchical wiring structure data into hierarchical dense wiring areas according to the hierarchical wiring density data, thereby obtaining high-density wiring area data; Step S254: performing a hierarchical component connectivity relationship analysis based on the hierarchical component structure data to obtain hierarchical component connectivity data, and performing high connectivity component statistics based on the hierarchical component connectivity data to obtain high connectivity component data; Step S255: performing statistics on the proportion of high-connectivity components in the region according to the high-connectivity component data, thereby obtaining data on the region with a high proportion of high-connectivity components; Step S256: performing a region intersection operation on the high-connectivity component high-proportion region data and the high-density wiring region data, thereby obtaining module component region data; Step S257: Perform functional module identification on the module component area data according to the functional module component feature data to obtain the functional module component area data, and divide the circuit board main body level integrated image set into functional module areas according to the functional module component area data to obtain the circuit board level functional module image set.

7. The deep learning method for printed circuit board defect recognition according to claim 1, characterized in that: Step S4 is specifically as follows: Step S41: constructing a feature descriptor for the hierarchical feature data of the defect area image, thereby obtaining a defect feature descriptor; Step S42: inferring defect depth information from the defect feature descriptor, thereby obtaining defect depth information data; Step S43: compensating the missing depth information of the defect area image hierarchical feature data according to the defect depth information data, so as to obtain optimized defect area image hierarchical feature data; Step S44: performing defect area perspective transformation on the hierarchical function module defect image set according to the optimized defect area image hierarchical feature data, thereby obtaining the hierarchical function module actual defect image set; Step S45: performing circuit board defect pattern recognition according to the actual defect image set of the hierarchical function module, thereby obtaining circuit board defect pattern data.

8. The deep learning method for printed circuit board defect recognition according to claim 7, characterized in that: Step S42 is specifically as follows: Step S421: performing principal component dimensionality reduction on the defect feature descriptor to obtain a principal component defect feature descriptor; Step S422: performing layout defect descriptor feature selection on the main component defect feature descriptor, thereby obtaining a layout defect feature descriptor; Step S423: performing image texture coherence evaluation on the hierarchical functional module defect image set to obtain image texture coherence data, and performing depth information missing region clustering on the hierarchical functional module defect image set according to the image texture coherence data to obtain a depth information missing image; Step S424: Correlate the layout defect feature descriptor and the depth information missing image with the defect depth information, thereby obtaining defect-depth correlation data; Step S425: constructing a convolutional neural network deep inference model based on the historical circuit board layout defect image set and defect-depth association data, thereby obtaining a layout defect deep inference model; Step S426: performing layout defect depth information inference on the defect feature descriptor through the layout defect depth inference model, thereby obtaining defect depth information data.

9. A deep learning system for printed circuit board defect recognition, characterized in that: A deep learning method for printed circuit board defect recognition as claimed in claim 1, wherein the deep learning system for printed circuit board defect recognition comprises: The image quality enhancement module is used to obtain the original image set of the printed circuit board, and perform circuit board background cleaning on the original image set of the printed circuit board, so as to obtain the main image set of the circuit board; and perform circuit board image quality enhancement on the main image set of the circuit board, so as to obtain the main enhanced image set of the circuit board; The function module identification module is used to perform circuit board multi-layer main body level integration on the circuit board main body enhanced image set, thereby obtaining a circuit board main body level integrated image set, and perform circuit board function module identification on the circuit board main body level integrated image set, thereby obtaining a circuit board level function module image set; A functional module defect detection module is used to perform hierarchical functional module defect detection based on a circuit board hierarchical functional module image set, thereby obtaining a hierarchical functional module defect image set, and perform hierarchical feature convolution extraction of a defect area based on the hierarchical functional module defect image set, thereby obtaining hierarchical feature data of the defect area image; A depth information missing correction module is used to perform circuit board depth information missing correction on the hierarchical function module defect image set based on the hierarchical feature data of the defect area image, so as to obtain the actual defect image set of the hierarchical function module; and perform circuit board defect pattern recognition based on the actual defect image set of the hierarchical function module, so as to obtain circuit board defect pattern data; The circuit board defect recognition module is used to extract defect pattern features based on the circuit board defect pattern data to obtain a defect pattern feature vector set, and to construct a deep learning classifier based on the defect pattern feature vector set to obtain a printed circuit board defect recognition model.

Images