PCB intelligent detection method and system fusing multispectral light source and deep learning

Abstract

The invention relates to a multi-spectral light source and deep learning integrated PCB intelligent detection method and system, and belongs to the technical field of printed circuit board detection. The method comprises the following steps: acquiring a PCB image, and processing the PCB image through multispectral light source fusion to obtain a multispectral high-definition PCB image; comparing and processing the multispectral high-definition PCB image through a difference template to obtain geometric defect information of the PCB substrate; processing the multispectral high-definition PCB image through component defect detection to obtain PCB component defect information; and processing the geometric defect information of the PCB substrate and the defect information of the PCB component through a multi-source data fusion decision to obtain PCB defect judgment information, thereby realizing efficient and accurate PCB intelligent detection.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of printed circuit board detection, and particularly relates to a PCB intelligent detection method and system fusing a multi-spectrum light source and deep learning. BACKGROUND

[0002] As a core carrier of electronic products, the quality detection of printed circuit boards (PCBs) is crucial. Although automatic optical inspection (AOI) technology has been widely used in PCB production processes for many years, the current mainstream equipment still has significant bottlenecks and cannot meet the actual needs of modern efficient, high-precision, and low-cost manufacturing.

[0003] Existing PCB intelligent detection methods often rely on fixed rule algorithms, which have low tolerance for reasonable changes in component appearance and are prone to high misjudgment rates due to rigid rules. Moreover, it is difficult to effectively distinguish subtle differences between acceptable images and defective images, which can easily lead to missed detection and cannot meet standard requirements. The algorithm has poor robustness and is difficult to adapt to changes in component suppliers or natural process changes. In addition, existing PCB detection is slow, and the single board detection time is long, which seriously restricts the efficiency of online batch detection. The iteration cycle of PCB intelligent detection products cannot meet the mass detection of PCBs.

[0004] Therefore, it is urgent to develop an efficient intelligent PCB detection method and system to solve the above problems. SUMMARY

[0005] To solve the above problems in the prior art, the application provides a PCB intelligent detection method and system fusing a multi-spectrum light source and deep learning.

[0006] The object of the application can be achieved by the following technical solutions: A PCB intelligent detection method fusing a multi-spectrum light source and deep learning, comprising: S1: obtaining a PCB image, and obtaining a multi-spectrum high-definition PCB image by fusing the PCB image through a multi-spectrum light source; S2: obtaining PCB substrate geometric defect information by difference template comparison processing of the multi-spectrum high-definition PCB image; S3: obtaining PCB component defect information by component defect detection processing of the multi-spectrum high-definition PCB image; S4: obtaining PCB defect judgment information by multi-source data fusion decision processing of the PCB substrate geometric defect information and the PCB component defect information.

[0007] Preferably, the generation process of the multi-spectrum high-definition PCB image in step S1 is: S101: obtaining a PCB image through a CMOS industrial camera; S102: obtaining a uniformly-illuminated PCB image by peripheral brightness enhancement processing of the PCB image through the light source; S103: obtaining a distortion-corrected uniformly-illuminated PCB image by distortion correction processing of the uniformly-illuminated PCB image; S104: obtaining a multi-spectral high-definition PCB image by multi-channel image fusion processing of the distortion-corrected uniformly-illuminated PCB image.

[0008] Preferably, the process of generating the uniformly-illuminated PCB image in step S102 is as follows: S102-1: obtaining an adapted target PCB illumination spectrum by loading a preset spectral waveform through a spectrally programmable light source; S102-2: obtaining a high-stability output light intensity by spectral amplitude modulation processing of the adapted target PCB illumination spectrum; S102-3: obtaining a uniformly-illuminated PCB image according to the high-stability output light intensity.

[0009] Preferably, the process of outputting the PCB substrate geometric defect information in step S2 is as follows: S201: obtaining a registered PCB image by template matching alignment processing of the multi-spectral high-definition PCB image; S202: loading a pre-trained PCB difference template; S203: obtaining a PCB original difference mask by double-threshold comparison processing of the registered PCB image and the pre-trained PCB difference template; S204: obtaining a set of PCB independent defect regions by connected domain analysis processing of the PCB original difference mask; S205: obtaining PCB substrate geometric defect information by feature screening rule processing of the set of PCB independent defect regions.

[0010] Preferably, the process of outputting the PCB component defect information in step S3 is as follows: S301: obtaining a multi-scale PCB shared feature map by multi-scale component feature extraction processing of the multi-spectral high-definition PCB image; S302: obtaining a PCB component defect candidate region by RPN neural network processing of the multi-scale PCB shared feature map; S303: obtaining PCB component defect information by component feature mapping processing of the PCB component defect candidate region.

[0011] Preferably, the process of component feature mapping processing in step S303 is as follows: S303-1: obtaining a PCB component defect sub-region by spatial coordinate mapping processing of the multi-scale PCB shared feature map and the PCB component defect candidate region; S303-2: obtaining a PCB component feature tensor by performing a sampling pooling process on the PCB component defect sub-region; S303-3: obtaining PCB component defect information by performing a parallel branch process on the PCB component feature tensor.

[0012] Preferably, the output process of the PCB defect judgment information in step S4 is as follows: S401: obtaining a substrate defect feature vector by performing a feature vectorization encoding process on the PCB substrate geometric defect information; S402: obtaining a component defect feature vector by performing a multi-dimensional information extraction process on the PCB component defect information; S403: obtaining a PCB fusion feature matrix by performing a feature level dimension fusion splicing process on the substrate defect feature vector and the component defect feature vector; S404: obtaining PCB defect judgment information by performing a deep decision network process on the PCB fusion feature matrix.

[0013] A PCB intelligent detection system fusing a multi-spectral light source and deep learning, the system is applied to the PCB intelligent detection method, and comprises a multi-spectral light source fusion module, a PCB substrate defect detection module, a PCB component defect detection module, and a PCB intelligent detection module. The multi-spectral light source fusion module is configured to acquire a PCB image and obtain a multi-spectral high-definition PCB image by performing a multi-spectral light source fusion process. The PCB substrate defect detection module is configured to obtain PCB substrate geometric defect information by performing a difference template comparison process on the multi-spectral high-definition PCB image. The PCB component defect detection module is configured to obtain PCB component defect information by performing a component defect detection process on the multi-spectral high-definition PCB image. The PCB intelligent detection module is configured to obtain PCB defect judgment information by performing a multi-source data fusion decision process on the PCB substrate geometric defect information and the PCB component defect information.

[0014] An electronic device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the PCB intelligent detection method when executing the computer program.

[0015] A storage medium comprising computer executable instructions for executing the PCB intelligent detection method when executed by a computer processor.

[0016] The PCB intelligent detection method has the following advantages: By integrating difference template matching and deep learning judgment strategy, compared with the unified judgment standard of traditional AOI equipment, it is more flexible, can further reduce the misjudgment rate and the omission rate, and improve the PCB factory qualified rate.

[0017] By acquiring PCB images through high-resolution industrial cameras and low-distortion lens hardware, and then fusing the PCB images through a multi-spectrum light source to obtain a multi-spectrum high-definition PCB image, the subsequent single detection time can be greatly shortened, and the PCB production efficiency can be improved.

[0018] By fusing the PCB substrate geometric defect information and the PCB component defect information to obtain PCB defect judgment information, the accuracy of PCB intelligent detection can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to facilitate those skilled in the art to understand, the present application will be further described below in conjunction with the drawings.

[0020] Figure 1 A flowchart of a PCB intelligent detection method combining a multi-spectrum light source and deep learning according to the present application.

[0021] Figure 2 A diagram of a spectrum programmable control light source color random change according to the present application.

[0022] Figure 3 A structure diagram of an offline AOI intelligent detection system according to the present application.

[0023] Figure 4 A structure diagram of an online AOI intelligent detection system according to the present application.

[0024] Figure 5 A working flowchart of an offline PCB intelligent system according to the present application.

[0025] Figure 6 A working flowchart of an online PCB intelligent system according to the present application. DETAILED DESCRIPTION

[0026] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined application purposes, the specific embodiments, structures, features and effects according to the present application are described in detail below in conjunction with the drawings and preferred embodiments.

[0027] Please refer to Figure 1 A PCB intelligent detection method combining a multi-spectrum light source and deep learning, comprising: S1: acquiring a PCB image, and fusing the PCB image through a multi-spectrum light source to obtain a multi-spectrum high-definition PCB image; S2: obtaining PCB substrate geometric defect information by differential template matching processing on the multi-spectral high-definition PCB image; S3: obtaining PCB component defect information by component defect detection processing on the multi-spectral high-definition PCB image; S4: obtaining PCB defect judgment information by multi-source data fusion decision processing on the PCB substrate geometric defect information and the PCB component defect information.

[0028] Embodiment 1 In this embodiment, the PCB image is obtained, and the multi-spectral high-definition PCB image is obtained by multi-spectral light source fusion processing on the PCB image, which is specifically implemented by the following steps: S101: obtaining a PCB image by a CMOS industrial camera; It should be noted that the CMOS industrial camera uses a low-distortion pseudo-telecentric lens.

[0029] S102: obtaining a uniform illumination PCB image by light source peripheral brightness enhancement processing on the PCB image; S102-1: obtaining an adaptive target PCB illumination spectrum by loading a preset spectral waveform on a spectral programmable light source; S102-2: obtaining high-stability output light intensity by spectral amplitude modulation processing on the adaptive target PCB illumination spectrum; S102-3: obtaining a uniform illumination PCB image according to the high-stability output light intensity.

[0030] S103: obtaining a distortion-corrected uniform illumination PCB image by distortion correction processing on the uniform illumination PCB image; S104: obtaining a multi-spectral high-definition PCB image by multi-channel image fusion processing on the distortion-corrected uniform illumination PCB image.

[0031] In this embodiment, a LabVIEW-based software control platform is developed, and the spectral wavelength and amplitude are calibrated based on this platform. At the same time, a spectral amplitude modulation method based on lookup table and feedback adjustment is proposed, which effectively improves the output spectral accuracy compared to linear modulation. Finally, by completing the active Hadamard transform spectral imaging experiment, see Figure 2 , the performance advantages of multi-channel and high-throughput are verified, and the spectral reflectance reconstruction experiment is completed based on the linear finite-dimensional model, and the reflectance data with high accuracy is obtained. A color measurement model based on the programmable light source is established, and the target color measurement is completed through the fast tristimulus value imaging experiment, with very small measurement error.

[0032] In the embodiment, the obtaining of the PCB substrate geometric defect information by the difference template matching processing of the multi-spectrum high-definition PCB image is implemented through the following steps. S201: obtaining a registration PCB image by template matching alignment processing of the multi-spectrum high-definition PCB image; S202: loading a pre-trained PCB difference template; Specifically, the pre-trained PCB difference template includes a PCB standard image and a PCB difference image. S203: obtaining a PCB original difference mask by double-threshold matching processing of the registration PCB image and the pre-trained PCB difference template; S204: obtaining a PCB independent defect region set by connected domain analysis processing of the PCB original difference mask; S205: obtaining PCB substrate geometric defect information by feature screening rule processing of the PCB independent defect region set.

[0033] In the embodiment, the obtaining of the PCB component defect information by the component defect detection processing of the multi-spectrum high-definition PCB image is implemented through the following steps. S301: obtaining a multi-scale PCB shared feature map by multi-scale component feature extraction processing of the multi-spectrum high-definition PCB image; S302: obtaining a PCB component defect candidate region by RPN neural network processing of the multi-scale PCB shared feature map; S303: obtaining PCB component defect information by component feature mapping processing of the PCB component defect candidate region.

[0034] S303-1: obtaining a PCB component defect sub-region by spatial coordinate mapping processing of the multi-scale PCB shared feature map and the PCB component defect candidate region; The PCB component defect sub-region is a candidate sub-region on the multi-scale PCB shared feature map corresponding to the PCB component defect candidate region.

[0035] S303-2: obtaining a PCB component feature tensor by sampling pooling processing of the PCB component defect sub-region; S303-3: obtaining PCB component defect information by parallel branch processing of the PCB component feature tensor.

[0036] Specifically, the parallel branch processing includes a classification branch and a regression branch, the classification branch obtains a probability distribution of a candidate region belonging to each PCB defect category through a full connection layer and a Softmax activation function; the regression branch obtains a boundary box fine adjustment parameter of the candidate region relative to each defect category through a full connection layer. The network is trained in parallel branches, end-to-end back propagation and parameter optimization are performed; in the model inference stage, through confidence threshold screening, category determination, boundary box fine adjustment and non-maximum suppression processing, the final PCB component defect information is obtained, including PCB defect category, PCB defect confidence and PCB defect accurate boundary box position.

[0037] In the embodiment, the PCB defect judgment information is obtained by processing the PCB substrate geometric defect information and the PCB component defect information through multi-source data fusion decision, specifically through the following steps: S401: obtaining a substrate defect feature vector by processing the PCB substrate geometric defect information through feature vectorization coding; S402: obtaining a component defect feature vector by processing the PCB component defect information through multi-dimensional information extraction; S403: obtaining a PCB fusion feature matrix by processing the substrate defect feature vector and the component defect feature vector through feature level dimension fusion splicing; S404: obtaining the PCB defect judgment information by processing the PCB fusion feature matrix through a deep decision network.

[0038] It should be noted that the PCB defect judgment information includes PCB defect comprehensive confidence and PCB defect judgment label.

[0039] In the embodiment, the target recognition and comparison are completed based on a deep learning strategy, and a difference template comparison technology is used to realize a complete intelligent detection process of the PCB, and improve the detection efficiency and accuracy. Taking an offline detection AOI equipment as an example, see Figure 3 , mainly including a product moving module, a CCD detection module, an operation platform and the like.

[0040] In the embodiment, the online AOI intelligent detection system is as shown in Figure 4As shown, including the top detection area, the middle transmission belt and the bottom control cabinet; the top blue part is the installation position of the light source and the camera for optical detection of the PCB. The colored cylinder on the top is part of the light source or sensor; the middle green part is the PCB transmission belt for feeding the PCB to be detected into the detection area. The design of the transmission belt ensures the stability and accuracy of the PCB during detection; the bottom blue part is the control cabinet, which contains various control and processing units for controlling the detection process, processing image data and outputting detection results. The buttons and indicator lights on the control cabinet are used to operate and monitor the system status. The black area on the side is the display screen, which is used to display the detection results, system status and operation interface, making it convenient for operators to monitor and control the detection process. The main function of the online AOI intelligent detection system is to detect the missing, misplacement, polarity error, and poor welding of components on the PCB. Through complex image processing algorithms, the collected images are analyzed and processed to identify and judge the status of the components; record the detection results and data for statistical analysis to help improve production processes and improve product quality.

[0041] In this embodiment, the detection system in two different working modes can realize semi-automatic and full-automatic PCB intelligent detection tasks. Taking the specific working process of the offline detection AOI equipment as an example, see Figure 5 , including: the user selects the incoming model, scans the code, binds, feeds, CCD detects, decides whether to collect materials or manually rechecks according to the results, uploads test data to MES, etc.

[0042] In this embodiment, the working process of the online intelligent system is as shown in Figure 6 : Select the appropriate incoming model according to production needs. After selecting the incoming model, the system will automatically scan and bind the corresponding material information to ensure that the material matches the production task. The material is placed on the fixture and flows into the production line with the fixture, preparing for the next detection or processing. After the product flows into the fixture, it will pass through the CCD intelligent detection system. The CCD intelligent detection system will automatically detect the product to determine whether it meets the quality standards. If the CCD intelligent detection system detects that the product is unqualified, the product will be marked as unqualified and enter the manual inspection link. The manual inspection personnel will further inspect and handle the unqualified product. If the CCD intelligent detection system detects that the product is qualified, the product will flow out of the production line with the fixture and enter the next process or packaging link. Whether the product is qualified or unqualified, its detection data will be uploaded to the MES for subsequent data analysis and production management.

[0043] Embodiment 2 A PCB intelligent detection system integrating multi-spectral light source and deep learning includes a multi-spectral light source integration module, a PCB substrate defect detection module, a PCB component defect detection module, and a PCB intelligent detection module. The multispectral light source fusion module is used to acquire PCB images and obtain multispectral high-definition PCB images through multispectral light source fusion processing. The PCB substrate defect detection module is used to obtain geometric defect information of the PCB substrate by processing the multispectral high-definition PCB image through difference template comparison. The PCB component defect detection module is used to process the multispectral high-definition PCB image through component defect detection to obtain PCB component defect information. The PCB intelligent inspection module is used to obtain PCB defect judgment information by performing multi-source data fusion decision processing on the geometric defect information of the PCB substrate and the defect information of the PCB components.

[0044] Example 3 The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0045] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0046] The computer readable media on which the program code can be carried by any suitable medium, including but not limited to wireless, wired, optical fiber cable, RF, and the like, or any suitable combination of these. Computer program code for carrying out operations of the present application can be written in one or more programming languages, or combinations of languages, including object oriented, such as Java, Smalltalk, C++, and conventional procedural, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0047] The above description is only the preferred embodiment of the present application, and is not intended to limit the present application in any form. Although the present application has been disclosed as above with the preferred embodiment, it is not intended to limit the present application, and any person skilled in the art can make some changes or modifications to the equivalent embodiments within the scope of the technical solution of the present application, without departing from the technical solution of the present application. Any simple modification, equivalent change and modification of the above embodiments, which does not depart from the technical solution of the present application, and is based on the technical essence of the present application, still belongs to the scope of the technical solution of the present application.

Claims

1. A PCB intelligent inspection method integrating multispectral light source and deep learning, characterized in that, include: S1: Acquire a PCB image, and obtain a multispectral high-definition PCB image by multispectral light source fusion processing; S2: Obtain PCB substrate geometric defect information by processing the multispectral high-definition PCB image through differential template comparison; S3: Obtain PCB component defect information by processing the multispectral high-definition PCB image through component defect detection; S4: Obtain PCB defect determination information by performing multi-source data fusion decision processing on the geometric defect information of the PCB substrate and the defect information of the PCB components.

2. The PCB intelligent inspection method according to claim 1, characterized in that, The process of generating the multispectral high-definition PCB image in step S1 is as follows: S101: Acquire PCB images using a CMOS industrial camera; S102: Obtain a uniformly illuminated PCB image by enhancing the brightness of the light source's periphery; S103: Obtain a distortion-corrected uniformly illuminated PCB image by processing the uniformly illuminated PCB image; S104: Obtain a multispectral high-definition PCB image by performing multi-channel image fusion processing on the distortion-corrected uniformly illuminated PCB image.

3. The PCB intelligent inspection method according to claim 2, characterized in that, The process of generating the uniformly illuminated PCB image in step S102 is as follows: S102-1: Obtain the target PCB illumination spectrum by loading a preset spectral waveform using a spectral programmable light source; S102-2: High-stability output light intensity is obtained by processing the illumination spectrum of the target PCB through spectral amplitude modulation. S102-3: Obtain a uniformly illuminated PCB image based on the high-stability output light intensity.

4. The PCB intelligent inspection method according to claim 1, characterized in that, The process of outputting the geometric defect information of the PCB substrate in step S2 is as follows: S201: Obtain a registered PCB image by template matching and alignment of the multispectral high-definition PCB image; S202: Load the pre-trained PCB difference template; S203: Obtain the original PCB difference mask by processing the registered PCB image and the pre-trained PCB difference template through dual threshold comparison; S204: The original PCB difference mask is processed by connected component analysis to obtain a set of independent defect regions of the PCB; S205: Obtain PCB substrate geometric defect information by processing the PCB independent defect region set through feature filtering rules.

5. The PCB intelligent inspection method according to claim 1, characterized in that, The process of outputting PCB component defect information in step S3 is as follows: S301: The multi-spectral high-definition PCB image is processed by multi-scale component feature extraction to obtain a multi-scale PCB shared feature map; S302: Obtain PCB component defect candidate regions by processing the multi-scale PCB shared feature map through the RPN neural network; S303: Obtain PCB component defect information by processing the PCB component defect candidate region through component feature mapping.

6. The PCB intelligent inspection method according to claim 5, characterized in that, The component feature mapping process in step S303 is as follows: S303-1: Obtain PCB component defect sub-regions by processing the multi-scale PCB shared feature map and the PCB component defect candidate regions through spatial coordinate mapping; S303-2: Obtain the PCB component feature tensor by sampling pooling the defect sub-region of the PCB component; S303-3: Obtain PCB component defect information by processing the PCB component feature tensor through parallel branching.

7. The PCB intelligent inspection method according to claim 1, characterized in that, The process of outputting PCB defect determination information in step S4 is as follows: S401: Obtain the substrate defect feature vector by processing the geometric defect information of the PCB substrate through feature vectorization encoding; S402: Obtain the component defect feature vector by extracting and processing the PCB component defect information through multi-dimensional information extraction; S403: Obtain the PCB fusion feature matrix by fusing and splicing the feature vectors of the substrate defects and the component defects at the feature level. S404: Obtain PCB defect determination information by processing the PCB fusion feature matrix through a deep decision network.

8. A PCB intelligent inspection system integrating multispectral light source and deep learning, wherein the system is applied to the PCB intelligent inspection method as described in any one of claims 1-7, characterized in that, This includes a multispectral light source fusion module, a PCB substrate defect detection module, a PCB component defect detection module, and a PCB intelligent detection module; The multispectral light source fusion module is used to acquire PCB images and obtain multispectral high-definition PCB images through multispectral light source fusion processing. The PCB substrate defect detection module is used to obtain geometric defect information of the PCB substrate by processing the multispectral high-definition PCB image through difference template comparison. The PCB component defect detection module is used to process the multispectral high-definition PCB image through component defect detection to obtain PCB component defect information. The PCB intelligent inspection module is used to obtain PCB defect judgment information by performing multi-source data fusion decision processing on the geometric defect information of the PCB substrate and the defect information of the PCB components.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the PCB intelligent inspection method as described in any one of claims 1-7.

10. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the PCB intelligent inspection method as described in any one of claims 1-7.

Images