PCB defect detection method and device based on digital holography

Abstract

This application provides a PCB defect detection method and apparatus based on digital holography, belonging to the field of printed circuit boards. The method includes: acquiring multiple consecutive frames of interference fringe patterns; wherein, during the acquisition of the interference fringe patterns, the light intensities of the object beam and reference beam are acquired in real time and adjusted to make their intensities equal, thereby improving the quality of the interference fringe patterns; inter-frame alignment and temporal fusion are performed on the multiple frames of interference fringe patterns, followed by weak light enhancement processing to obtain enhanced interference fringe patterns; based on the enhanced interference fringe patterns, the three-dimensional topography of the PCB is reconstructed; and based on the reconstructed three-dimensional topography of the PCB, defect detection is performed on the PCB.

Description

Technical Field

[0001] This application belongs to the field of printed circuit boards, and in particular relates to a method and apparatus for PCB defect detection based on digital holography. Background Technology

[0002] Digital holography is an object detection and imaging technology based on the principles of holography. By recording and reproducing the light wave information (including amplitude and phase) of an object, it can acquire a three-dimensional image of the object and related information. Due to its ability to provide high-resolution, high-precision three-dimensional information, holographic detection technology has broad application prospects in materials science, biomedicine, industrial inspection, and other fields.

[0003] Holographic detection technology mainly includes the following methods: traditional holography, digital holography, phase holography, interferometric holography, and white light holography. However, these methods still face challenges in terms of operational complexity, cost, resolution, and environmental adaptability, and further improvements and innovations are needed to meet broader application needs in the future.

[0004] Chinese invention patent CN103822587A discloses an interferometric measurement system for testing three-dimensional deformation and displacement of microstructures. Using a piezoelectric ceramic phase-shifting device, it achieves measurement accuracy at the nanometer level. However, due to the system's sensitivity to external interference, it requires high stability of the measurement environment, necessitating operation in a vibration-proof workbench and darkroom to ensure the accuracy of the measurement results. Chinese invention patent CN108759709A discloses a white-light interferometric three-dimensional reconstruction method suitable for surface morphology detection. It obtains the zero optical path difference position by finding the maximum value of a continuous function, reducing the requirements for the testing environment. Compared to traditional methods, this method reduces the requirements for the testing environment and has strong anti-interference capabilities. However, this method does not fully address the influence of factors such as light intensity on the interferogram, which may limit the stability of the reconstruction accuracy.

[0005] In summary, existing methods and systems still have shortcomings in defect identification under complex testing environments. Summary of the Invention

[0006] In view of this, this application provides a PCB defect detection method and device based on digital holography, which aims to solve the technical problems of decreased interference fringe contrast and insufficient three-dimensional reconstruction accuracy of existing digital holographic detection technology under conditions of drastic changes in PCB surface reflectivity and weak light. It provides a high-precision PCB defect detection scheme that can adaptively match light intensity and combine deep learning for weak light enhancement.

[0007] Firstly, this application provides a PCB defect detection method based on digital holography, comprising: Acquire N consecutively acquired interference fringe patterns, where N≥2; during the acquisition of the interference fringe patterns, the light intensities of the object beam and the reference beam are acquired in real time and adjusted to make the light intensities of the object beam and the reference beam equal, thereby improving the quality of the interference fringe patterns. Multi-frame temporal fusion and low-light enhancement processing are performed on N frames of interference fringe patterns to obtain enhanced interference fringe patterns; Based on the enhanced interference fringe pattern, the three-dimensional morphology of the PCB is reconstructed; Based on the reconstructed three-dimensional morphology of the PCB, defect detection is performed on the PCB.

[0008] Optionally, the steps of performing multi-frame temporal fusion and low-light enhancement processing on N frames of interference fringe patterns to obtain enhanced interference fringe patterns include: The N frames of interferometric fringe patterns are input into the low-light enhancement model for inter-frame registration to eliminate inter-frame translation and rotation biases. The registered N frames are then weighted and averaged to obtain the fused interferometric fringe pattern. Finally, the fused interferometric fringe pattern is subjected to low-light enhancement processing to obtain the enhanced interferometric fringe pattern.

[0009] Optionally, the low-light enhancement model includes: The multi-frame temporal fusion module is used to receive N consecutively acquired interferometric fringe patterns, take one of the frames as a reference frame, estimate the optical flow field of the other frames relative to the reference frame through an optical flow estimation network, perform sub-pixel level spatial alignment of each frame according to the optical flow field, and perform adaptive weighted fusion of the aligned N frames through a temporal fusion network to output a high signal-to-noise ratio fused interferometric fringe pattern. A frequency-spatial dual-stream feature extractor includes a frequency domain branch, a spatial domain branch, and a cross-domain feature fusion module. The frequency domain branch is used to extract frequency-domain sensing features of the interference fringe pattern; the spatial domain branch is used to extract spatial features of the interference fringe pattern; and the cross-domain feature fusion module is used to fuse frequency-domain sensing features and spatial features to obtain multi-domain feature data. The physical constraint decomposition network includes a shared encoder, a reflectance decoding branch, an illumination decoding branch, a phase prior branch, and a contrast estimation branch. The shared encoder is used to perform multi-layer convolutional downsampling on multi-domain feature data to extract multi-scale shared feature representations. The reflectance decoding branch is used to reconstruct the reflectance component of the interferometric image based on the shared features through transposed convolutional upsampling. The illumination decoding branch is used to estimate the illumination component of the interferometric image based on the shared features through a global-local parallel architecture. The phase prior branch is used to estimate the phase prior information of the interference fringes based on the shared features through transposed convolutional upsampling. The contrast estimation branch is used to estimate the global contrast of the interference fringes based on the shared features through adaptive average pooling and fully connected layers. The progressive multi-scale fusion reconstructor is used to recover large-scale illumination distribution, mesoscale interference fringe structure and high-frequency fine texture in sequence through a three-stage progressive reconstruction strategy from coarse to fine based on reflectivity component, illumination component, phase prior information and global contrast, and output an enhanced interference fringe map.

[0010] Optionally, the progressive multi-scale fusion reconstructor includes: a first-stage reconstruction module, a second-stage reconstruction module, and a third-stage reconstruction module; the first-stage reconstruction module uses a holed residual block to recover the large-scale illumination distribution at a quarter resolution and performs feature weighting through a channel attention module; the second-stage reconstruction module uses a fringe structure sensitive module to recover the mesoscale interference fringe structure at a half resolution and performs feature weighting through a spatial attention module; the third-stage reconstruction module uses a texture refinement module to recover high-frequency fine textures and edge details at full resolution; information is transferred between the stages through bilinear interpolation upsampling and feature stitching.

[0011] Optionally, the loss function used in the weak light enhancement model is based on reconstruction loss, physical consistency loss, frequency domain fringe fidelity loss, and smoothing regularization loss. Among them, reconstruction loss is obtained by calculating the L1 norm distance and structural similarity loss between the enhanced interference fringe pattern and the ground truth interference fringe pattern; physical consistency loss is obtained by calculating the L2 norm distance between the product of reflectivity component and illumination component and the enhanced interference fringe pattern; frequency domain fringe fidelity loss is obtained by calculating the L1 norm distance between the spectral amplitudes of the enhanced interference fringe pattern and the ground truth interference fringe pattern after performing Fourier transforms on them respectively; smoothing regularization loss is obtained by calculating the L1 norm of the gradient of the illumination component in the horizontal and vertical directions, which is used to constrain the spatial smoothness of the illumination component.

[0012] Optionally, the steps for reconstructing the three-dimensional morphology of the PCB based on the enhanced interference fringe pattern include: performing a two-dimensional Fourier transform on the enhanced interference fringe pattern to extract the positive first-order spectral components; performing frequency domain filtering and frequency shifting on the positive first-order spectral components to obtain the complex amplitude information of the object light; extracting the phase from the complex amplitude information to obtain a wrapped phase map; performing phase unwrapping processing on the wrapped phase map to obtain a continuous phase distribution; and converting the phase information into height information based on the continuous phase distribution and system calibration parameters to reconstruct the three-dimensional morphology of the PCB.

[0013] Optionally, based on the reconstructed three-dimensional morphology of the PCB, the steps for defect detection of the PCB include: performing reference plane fitting and background removal processing on the three-dimensional morphology to obtain a defect height deviation map; setting a height threshold, performing threshold segmentation on the defect height deviation map, and extracting candidate defect regions; performing connected component analysis on the candidate defect regions to extract the geometric feature parameters of the defects, including at least one of defect area, depth, and volume; and classifying the defects based on the geometric feature parameters to determine the defect type.

[0014] Secondly, this application provides a PCB defect detection device based on digital holography, comprising: The acquisition module is used to acquire N consecutively collected interference fringe patterns, where N≥2. During the acquisition of the interference fringe patterns, the light intensities of the object light and the reference light are acquired in real time and adjusted to make the light intensities of the object light and the reference light equal, thereby improving the quality of the interference fringe patterns. The enhancement module is used to perform multi-frame temporal fusion and low-light enhancement processing on N frames of interference fringe patterns to obtain enhanced interference fringe patterns; The reconstruction module is used to reconstruct the three-dimensional topography of the PCB based on the enhanced interference fringe pattern. The defect detection module is used to detect defects in the PCB based on the reconstructed 3D topography.

[0015] Thirdly, this application provides an electronic device, including the PCB defect detection device based on digital holography as described above.

[0016] Fourthly, this application provides a computer-readable storage medium storing at least one piece of program code, which is executed by a processor to implement the PCB defect detection method based on digital holography as described in any of the preceding claims.

[0017] The technical solution provided in this application includes at least one of the following beneficial effects: (1) This invention proposes and implements an electronically controlled closed-loop adaptive light intensity matching device based on dual-path real-time light intensity detection and a hardware variable attenuator. Independent beam-sampling prisms and high-sensitivity light intensity detectors are inserted into the object arm and reference arm, respectively, to monitor the actual light intensity of the two arms in real time. The closed-loop control algorithm is run through an FPGA hardware controller to directly drive the liquid crystal variable attenuator component in the reference optical path, achieving real-time and accurate matching between the reference light intensity and the object light intensity across the entire dynamic range. The entire adjustment process is completed by the hardware closed loop, without relying on offline software processing or manual intervention. This hardware closed-loop design effectively solves the problem of severe fringe contrast reduction in traditional off-axis digital holographic systems when the reflectivity of the PCB surface changes drastically over a wide range. It significantly improves both the interference fringe contrast and the signal-to-noise ratio, ensuring that each frame of the hologram is in a good interference state, and significantly improving the robustness and stability of the system in complex industrial environments.

[0018] (2) This invention innovatively designs a deep learning-based low-light enhancement module for extremely weak light interferograms (photon count as low as 1 / 1000 of traditional methods). It uses a layer decomposition network to decouple the interferogram into reflection and illumination components, and then uses a progressive multi-scale fusion reconstructor to sequentially restore the large-scale illumination distribution, the medium-scale interference fringe structure, and the high-frequency fine texture for refined enhancement and fidelity reconstruction, ultimately outputting a high-quality interferogram. Under the premise of a single frame processing time of ≤15ms, this module can reduce the phase noise of the weak light interferogram by more than 70%, and reduce the RMS error of the reconstructed three-dimensional morphology from ≥80nm to ≤8nm under traditional methods. It can still stably and accurately resolve nanoscale defects (such as micro-bumps, pits, scratches, and residual adhesive) on the PCB surface under extremely low reflectivity (≤1%) or strong background stray light conditions, breaking through the detection limit of existing digital holography under low light conditions.

[0019] (3) This invention combines two core innovations: hardware adaptive light intensity matching and deep learning weak light enhancement. With the help of a multi-frame spatiotemporal fusion algorithm, the overall signal-to-noise ratio of the system is further improved to more than 45dB. The three-dimensional reconstruction accuracy reaches 5nm in the vertical direction and submicron level in the horizontal direction. The detection speed is ≥100frame / s and the field of view can reach 15mm×15mm. It successfully realizes 100% full-surface non-destructive rapid detection of 6 to 30 layers of high-density PCB boards (including black board, ultra-rough board and high-gloss board). The defect missed detection rate is reduced to less than 0.1% and the false detection rate is ≤0.5%.

[0020] (4) The system of the present invention has a compact structure and adopts off-axis optical path design. Only one high-resolution camera is needed to complete the acquisition of interferogram. It has fewer optical path components, higher light energy utilization, greater adjustment freedom, and is easy to integrate into existing PCB production lines or AOI equipment. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a PCB defect detection method based on digital holography provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of an interference fringe pattern acquisition system provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a low-light enhancement model provided in an embodiment of this application; Figure 4 A schematic diagram of the three-dimensional reconstruction result of a resolution plate provided in an embodiment of this application; Figure 5 This is a schematic diagram of the three-dimensional reconstruction result of a PCB foreign object defect provided in an embodiment of this application; Figure 6 A structural block diagram of a PCB defect detection device based on digital holography provided in an embodiment of this application; Figure 7 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] Figure 1 A flowchart illustrating a PCB defect detection method based on digital holography, provided as an embodiment of this application. See also... Figure 1 ,include: S101. Acquire N consecutively acquired interference fringe patterns, where N≥2; wherein, during the acquisition of the interference fringe patterns, the light intensities of the object light and the reference light are acquired in real time, and the light intensities of the object light and the reference light are adjusted to make the light intensities of the object light and the reference light equal, thereby improving the quality of the interference fringe patterns.

[0025] In some examples, in step S101, by means of... Figure 2The device shown continuously acquires N frames of interference fringe patterns of the same region at a high frame rate (N≥2, typically 4–8 frames). Since the camera frame rate is much higher than the PCB movement speed, there is only a tiny subpixel-level offset between consecutive N frames, making it suitable for inter-frame alignment and fusion processing.

[0026] To obtain high-quality interference fringe patterns, this application provides a schematic diagram of an interference fringe pattern acquisition system. See [link to relevant documentation]. Figure 2 The system includes: 1. Laser, 2. Collimator and beam expander, 3. First attenuator, 4. First reflector, 5. Second beam splitter, 6. First intensity detector, 7. Second attenuator, 8. Third beam splitter, 9. Second intensity detector, 10. Transimpedance amplifier, 11. FPGA hardware controller, 12. Second reflector, 14. Microscope lens, 15. Camera. The beam emitted by laser 1 is collimated and expanded by collimator 2 to form uniform parallel light. The overall light intensity is coarsely adjusted by the first attenuator 3, and then turned by the first reflector 4 to enter the first beam splitter 5, where it is split into transmitted object light and reflected reference light.

[0027] After the object light enters the second beam splitter 6, a small portion is reflected to the first light intensity detector 7 below. The weak photocurrent output by the first light intensity detector 7 is converted to voltage and amplified with low noise by the transimpedance amplifier 11, and then sent to the FPGA hardware controller 12 in digital form after analog-to-digital conversion as the real-time light intensity feedback signal of the object light arm. The main transmitted light illuminates the surface of the PCB 13, and the reflected scattered object light carrying defect information returns along the original path. After passing through the second beam splitter 6, the first beam splitter 5, and the microscope lens 15, it is projected onto the high-resolution camera 16. The reference light is reflected by the first beam splitter 5 and then enters the second attenuator 8, the third beam splitter 9, and the second reflector 14 in sequence. The third beam splitter 9 reflects a portion of the reference light to the second intensity detector 10. The photocurrent output by the second intensity detector 10 is also converted to voltage and amplified with low noise by the transimpedance amplifier 11. After analog-to-digital conversion, it is sent to the FPGA hardware controller 12 in digital form as a real-time intensity feedback signal for the reference beam arm. Another portion of the reference light passes through the third beam splitter 9 and is transmitted to the second reflector 14. After being reflected by the second reflector 14, it passes through the third beam splitter 9, the second attenuator 8, and the first beam splitter 5 in sequence and is transmitted to the microscope head 15 (used for wavefront cleaning and beam expansion of the reference light). Finally, it is superimposed on the target surface of the camera 16 with the object light at an off-axis angle of about 8 degrees to form an off-axis digital hologram.

[0028] The second attenuator 8 is a liquid crystal variable attenuation component, capable of continuously adjusting its optical transmittance according to the drive control signal output by the FPGA hardware controller 12. In each control cycle, the FPGA hardware controller 12 simultaneously reads the digital value of the object light intensity from the first intensity detector 7 and the digital value of the reference light intensity from the second intensity detector 10, calculates the intensity deviation between them, and compares this deviation with a preset target matching condition (i.e., equal intensity on both arms). Subsequently, the FPGA hardware controller 12 calculates the drive control amount for the second attenuator 8 based on this intensity deviation using a closed-loop control algorithm, and outputs a corresponding control signal to drive the second attenuator 8 to adjust its attenuation, making the intensity of the reference light approach the actual intensity of the object light. This closed-loop feedback process of "detection-comparison-adjustment-re-detection" runs continuously with a period much smaller than the camera frame interval, ensuring that the closed-loop control has completed multiple iterations and reached convergence between adjacent frame acquisitions, so that each frame of the interference fringe pattern is acquired under conditions where the intensity of the object light and the reference light are fully matched. When the reflectivity of the PCB surface changes significantly with the scanning area, this closed-loop mechanism can automatically track the changes in the intensity of the object light and drive the second attenuator 8 to make corresponding adjustments without manual intervention.

[0029] The obtained high-quality interference fringe pattern is input into the deep learning image processing module for low-light enhancement and multi-frame fusion processing. The enhanced interference pattern is output and reconstructed to obtain the accurate three-dimensional shape of the PCB surface.

[0030] S102. Perform time-series fusion and low-light enhancement processing on the multi-frame interference fringe patterns to obtain the enhanced interference fringe patterns.

[0031] In some examples, step S102 includes: The N frames of interference fringe patterns are input into the low-light enhancement model. First, the multi-frame temporal fusion module performs inter-frame alignment and temporal fusion to output a high signal-to-noise ratio fused interference fringe pattern. Then, the fused interference fringe pattern is sent to the subsequent low-light enhancement network for enhancement processing to obtain the enhanced interference fringe pattern.

[0032] See Figure 3 In some examples, low-light enhancement models include: The multi-frame temporal fusion module is used to receive N consecutively acquired interferometric fringe patterns, take one of the frames as a reference frame, estimate the optical flow field of the other frames relative to the reference frame through an optical flow estimation network, perform sub-pixel level spatial alignment of each frame according to the optical flow field, and perform adaptive weighted fusion of the aligned N frames through a temporal fusion network to output a high signal-to-noise ratio fused interferometric fringe pattern. The optical flow estimation network adopts a lightweight pyramid-shaped optical flow network structure, containing a 3-level feature pyramid. Each level contains 2 layers of Conv(3×3)-BN-LeakyReLU convolutional blocks with channel numbers of 32, 64, and 96 respectively. Using the middle frame of N frames as the reference frame, the two-dimensional dense optical flow field of the remaining N-1 frames relative to the reference frame is estimated. Subpixel-level spatial alignment (warping) of each frame is performed by bilinear interpolation to eliminate spatial offsets caused by vibration, sample micro-movement, etc. between frames.

[0033] The temporal fusion subnetwork stitches the aligned N-frame images together along the channel dimension to form an N-channel feature tensor, which is then fused temporally through three layers of convolution (Conv(3×3)-BN-ReLU, with 64, 64, and 1 channels respectively). An embedded channel attention module (Squeeze-and-Excitation structure) adaptively learns the fusion weights for the N frames: assigning larger weights to frames with higher quality and lower noise, and automatically reducing the weights of frames with motion blur or abnormal noise, outputting a fused interferometric fringe pattern.

[0034] In some examples, N typically takes the value of 4 to 8 frames. When N=1, the multi-frame temporal fusion module degenerates into an identity mapping, and the model automatically degrades to a single-frame processing mode, ensuring the system's compatibility in extremely high-speed scanning scenarios.

[0035] A frequency-spatial dual-stream feature extractor is used to receive fused interferometric fringe patterns output by a multi-frame temporal fusion module, including a frequency domain branch, a spatial domain branch, and a cross-domain feature fusion module; the frequency domain branch is used to extract frequency domain sensing features of the interferometric fringe pattern; the spatial domain branch is used to extract spatial features of the interferometric fringe pattern; and the cross-domain feature fusion module is used to fuse frequency domain sensing features and spatial features to obtain multi-domain feature data. The physical constraint decomposition network includes a shared encoder, a reflectance decoding branch, an illumination decoding branch, a phase prior branch, and a contrast estimation branch. The shared encoder is used to perform multi-layer convolutional downsampling on multi-domain feature data to extract multi-scale shared feature representations. The reflectance decoding branch is used to reconstruct the reflectance component of the interferometric image based on the shared features through transposed convolutional upsampling. The illumination decoding branch is used to estimate the illumination component of the interferometric image based on the shared features through a global-local parallel architecture. The phase prior branch is used to estimate the phase prior information of the interference fringes based on the shared features through transposed convolutional upsampling. The contrast estimation branch is used to estimate the global contrast of the interference fringes based on the shared features through adaptive average pooling and fully connected layers. The progressive multi-scale fusion reconstructor is used to recover large-scale illumination distribution, mesoscale interference fringe structure and high-frequency fine texture in sequence through a three-stage progressive reconstruction strategy from coarse to fine based on reflectivity component, illumination component, phase prior information and global contrast, and output an enhanced interference fringe map.

[0036] In some examples, the specific structure of the frequency-spatial dual-stream feature extractor 2 is as follows: Frequency domain branch 21 includes: The Fourier transform module 211, the first convolutional layer 212, the second convolutional layer 213, the third convolutional layer 214, and the inverse Fourier transform module 215 are connected in sequence. The first convolutional layer 212, the second convolutional layer 213, and the third convolutional layer 214 form a frequency domain filter, which is used to filter the spectrum.

[0037] In some examples, the kernel size of the first convolutional layer is 1×1.

[0038] In some examples, the kernel size of the second convolutional layer is 3×3.

[0039] In some examples, the kernel size of the third convolutional layer is 1×1.

[0040] In some examples, the activation function used in the first to third convolutional layers is the ReLU function.

[0041] In some examples, the frequency domain branch 21 first performs a 2D Fast Fourier Transform (FFT) on the input image, converting the image to the frequency domain and separating the real and imaginary parts. An adaptive filter (containing a Conv(1×1)-ReLU-Conv(3×3)-ReLU-Conv(1×1) structure) is used to filter the spectrum, enhancing the frequency components corresponding to the interference fringes and suppressing noise frequency components. After filtering, an inverse FFT is performed to transform the image back to the spatial domain, extracting the frequency domain perceptual features.

[0042] Spatial branch 22 adopts a multi-scale residual network structure, specifically including: The first residual block 221, the second residual block 222, the third residual block 223, and the fourth residual block 224 are connected in sequence.

[0043] The first to fourth residual blocks each include a convolutional layer and a batch normalization layer, and the activation function used is the ReLU function.

[0044] In some examples, the kernel size of the convolutional layer used for the first through fourth residual blocks is 3×3.

[0045] In some examples, the first residual block has 64 channels, the second residual block has 128 channels, the third residual block has 256 channels, and the fourth residual block has 256 channels.

[0046] The cross-domain feature fusion module (CDFM)23 includes: a channel splicing layer, a 1×1 convolutional dimensionality reduction layer, a channel attention module, and a spatial attention module; The channel stitching layer stitches together the frequency-domain perceptual features output from the frequency-domain branch and the spatial features output from the spatial-domain branch along the channel dimension; the 1×1 convolutional dimensionality reduction layer performs channel dimensionality reduction on the stitched features to reduce redundant information; the channel attention module performs adaptive weighting on the dimensionality-reduced features along the channel dimension to strengthen important channels related to interference fringes; the spatial attention module performs adaptive weighting on the channel-weighted features along the spatial dimension to highlight the spatial distribution area of ​​interference fringes and output fused multi-domain feature data.

[0047] In some examples, the Physically Constrained Layer Decomposition Network (PCLD-Net)3 is designed based on the interference intensity equation I(x,y) = R(x,y)·L(x,y)·[1 + γ·cos(φ(x,y))], where I(x,y) represents the image intensity of the interference fringe pattern at pixel coordinates (x,y); R(x,y) represents the reflectivity component of the object surface at that pixel; L(x,y) represents the illumination component of the incident light at that pixel; γ represents the global contrast of the interference fringes; and φ(x,y) represents the phase difference between the object light and the reference light at that pixel.

[0048] The specific structure of the Physical Constraint Layer Decomposition Network (PCLD-Net) 3 is as follows: The shared encoder 31 employs a 4-layer convolutional downsampling structure, specifically including: First convolutional downsampling structure, second convolutional downsampling structure, third convolutional downsampling structure, fourth convolutional downsampling structure; The first to fourth convolutional downsampling structures consist of sequentially connected convolutional layers and batch normalization layers, using the ReLU activation function.

[0049] The number of channels in the first to fourth layers of the convolutional downsampling structure are 64, 128, 256, and 256, respectively.

[0050] In the shared encoder, each convolutional downsampling layer is followed by a 2×2 stride downsampling step. This progressively reduces the spatial resolution of the feature map, expands the receptive field, and extracts multi-scale abstract feature representations from low-level texture to high-level semantics. Simultaneously, the features from each convolutional downsampling layer are preserved for skip connections with the transposed convolutional upsampling layers in the reflectivity decoding, illumination decoding, and phase prior branches to compensate for the spatial detail lost during downsampling.

[0051] Reflectance decoding branch 32 specifically includes: The first transposed convolutional upsampling structure 321, the second transposed convolutional upsampling structure 322, the third transposed convolutional upsampling structure 323, the fourth transposed convolutional upsampling structure 324, and the fourth convolutional layer 325 are connected in sequence.

[0052] Among them, the first transposed convolutional upsampling structure 321, the second transposed convolutional upsampling structure 322, the third transposed convolutional upsampling structure 323, and the fourth transposed convolutional upsampling structure 324 all include: transposed convolutional layers and batch normalization layers connected in sequence.

[0053] The transposed convolutional layer has a 4×4 kernel and uses the ReLU activation function. The fourth convolutional layer (325) has a 3×3 kernel and uses the Sigmoid activation function.

[0054] The number of channels in the first transposed convolutional upsampling structure 321, the second transposed convolutional upsampling structure 322, the third transposed convolutional upsampling structure 323, and the fourth transposed convolutional upsampling structure 324 are 256, 128, 64, and 32, respectively.

[0055] Four layers of transposed convolutional upsampling (TransConv(4×4)-BN-ReLU) are used, with the number of channels decreasing sequentially from 256 to 128 to 64 to 32. Each layer fuses the skip connection features of the corresponding coding layer, and finally outputs the reflectance map R∈[0,1] through Conv(3×3)+Sigmoid.

[0056] Lighting decoding branch 33 employs a multi-scale dilated convolutional architecture, specifically including: The first hollow convolutional layer 331, the second hollow convolutional layer 332, the third hollow convolutional layer 333, and the fourth hollow convolutional layer 334 are connected in sequence; In this layer, the kernel size of the first to fourth dilated convolutional layers is 3×3, the dilation rate is 2, 4, 8 and 16 respectively, the number of channels is 64, and the activation function used is ReLU function; each dilated convolutional layer is followed by a batch normalization layer. By gradually increasing the dilation rate, the receptive field of each dilated convolutional layer expands sequentially, enabling the capture of illumination distribution information from local to global scales without reducing resolution. The output of the fourth dilated convolutional layer is compressed to 1 channel by a convolutional layer with a kernel size of 1×1, and the illumination map L∈[0,1] is output through the Sigmoid activation function.

[0057] Phase prior branch 34 employs a 4-layer transposed convolutional upsampling structure, specifically including: The fifth transposed convolutional upsampling structure 341, the sixth transposed convolutional upsampling structure 342, the seventh transposed convolutional upsampling structure 343, the eighth transposed convolutional upsampling structure 344, and the fifth convolutional layer 345 are connected in sequence.

[0058] Among them, the kernel size of the transposed convolutional layer used in the fifth transposed convolutional upsampling structure 341, the sixth transposed convolutional upsampling structure 342, the seventh transposed convolutional upsampling structure 343, and the eighth transposed convolutional upsampling structure 344 is 4×4.

[0059] The activation function used in the fifth transposed convolutional upsampling structure 341, the sixth transposed convolutional upsampling structure 342, the seventh transposed convolutional upsampling structure 343, and the eighth transposed convolutional upsampling structure 344 is the ReLU function. The activation function used in the fifth convolutional layer 345 is the Tanh function.

[0060] Among them, the number of channels included in the fifth transposed convolutional upsampling structure 341, the sixth transposed convolutional upsampling structure 342, the seventh transposed convolutional upsampling structure 343, and the eighth transposed convolutional upsampling structure 344 are 256, 128, 64, and 32, respectively.

[0061] A 4-layer transposed convolution upsampling structure is adopted, and finally the phase prior map φ∈[-π,π] is output by Conv(3×3)+Tanh activation; Contrast estimation branch 35 includes: Global average pooling layer 351, data reshaping layer (Flatten layer) 352, first fully connected layer 353, and second fully connected layer 354; The contrast estimation branch uses adaptive average pooling to compress the feature map to a spatial resolution of 8×8. After being flattened into a one-dimensional vector by the data reshaping layer, the global stripe contrast γ∈[0,1] is estimated through two fully connected layers (the first layer has an output dimension of 256 and the second layer has an output dimension of 1). The first fully connected layer uses the ReLU activation function and the second fully connected layer uses the Sigmoid activation function, outputting the global stripe contrast γ∈[0,1].

[0062] In some examples, the progressive multiscale fusion reconstructor 4 employs a three-stage coarse-to-fine progressive reconstruction strategy, specifically including: First-stage reconstruction module 41, second-stage reconstruction module 42 and third-stage reconstruction module 43; The first-stage reconstruction module uses hollow residual blocks to recover the large-scale illumination distribution at a quarter resolution and performs feature weighting through the channel attention module. The second-stage reconstruction module uses a fringe structure sensitive module to recover the mesoscale interference fringe structure at half resolution, and performs feature weighting through a spatial attention module. The third-stage reconstruction module uses a texture refining module at full resolution to restore high-frequency fine textures and edge details; information is transferred between stages through bilinear interpolation upsampling and feature stitching.

[0063] The first-stage reconstruction module uses a large receptive field hollow residual block (dilation=4) with 128 channels, focusing on restoring large-scale illumination distribution and low-frequency background information. The output is then weighted by the channel attention module.

[0064] The second-stage reconstruction module uses a fringe structure sensitive module, which contains a 6-directional Gabor convolution group and a standard convolution, focusing on recovering the mesoscale interference fringe structure. The output is then weighted by a spatial attention module.

[0065] The third-stage reconstruction module uses a texture refinement block, which includes three layers of 3×3 convolutions and channel attention, focusing on restoring high-frequency fine textures and edge details. Finally, it outputs an enhanced interferogram via Conv(3×3)+Sigmoid.

[0066] Information is transferred between stages through bilinear interpolation upsampling and feature concatenation, enabling effective fusion of multi-scale features.

[0067] In some examples, the loss function used in the low-light enhancement model is based on reconstruction loss, physical consistency loss, frequency domain fringe fidelity loss, and smoothing regularization loss. Among them, reconstruction loss is obtained by calculating the L1 norm distance between the enhanced interference fringe pattern and the ground truth interference fringe pattern and the structural similarity loss; physical consistency loss is obtained by calculating the L2 norm distance between the product of the reflectivity component and the illumination component and the enhanced interference fringe pattern; frequency domain fringe fidelity loss is obtained by performing Fourier transforms on the enhanced interference fringe pattern and the ground truth interference fringe pattern respectively, and then calculating the L1 norm distance between their spectral amplitudes; smoothing regularization loss is obtained by calculating the L1 norm of the gradient of the illumination component in the horizontal and vertical directions, which is used to constrain the spatial smoothness of the illumination component.

[0068] Specifically, the loss function used in the low-light enhancement model is as follows:

[0069] In the formula: Indicates the overall loss; Indicates the losses incurred during reconstruction; This represents the loss of physical consistency. The weighting coefficients representing the physical consistency loss; This indicates the loss of fidelity in the frequency domain fringes; Weighting coefficients representing the frequency domain fringe fidelity loss; This represents the smoothing regularization loss; The weight coefficients represent the smoothing regularization loss.

[0070] As an example, , , The values ​​can be 0.1, 0.05, and 0.01, respectively.

[0071] The training strategy for the low-light enhancement model is as follows: The Adam optimizer is used, with an initial learning rate set to 1×10⁻⁶. -4 A cosine annealing learning rate scheduling strategy was adopted, with a total training epoch of 200 epochs and a batch size of 8. During training, data augmentation operations such as random cropping, random horizontal flipping, and random rotation were performed on the input images.

[0072] The training data for the low-light enhancement model comes from the following sources: The training data consists of interference fringe images acquired by an off-axis digital holographic interferometry system on different PCB sample surfaces. This includes high-quality interferograms acquired under normal illumination conditions as ground truth labels, and low-quality interferograms obtained by simulating low-light conditions through methods such as reducing laser power and increasing attenuator attenuation as input samples. Each training sample set includes N consecutively acquired low-light interference fringe images of the same PCB area as input, and the corresponding high-quality interferogram acquired under normal illumination conditions as ground truth labels. During training, random subpixel-level translation and rotation perturbations are applied to each set of N images to simulate inter-frame offsets in actual acquisition.

[0073] S103. Based on the enhanced interference fringe pattern, the three-dimensional morphology of the PCB is reconstructed.

[0074] In some examples, step S103 includes: The first step is to perform a two-dimensional fast Fourier transform on the enhanced interference fringe pattern to obtain the frequency domain distribution.

[0075] The second step is to locate the positive first-order spectral component in the frequency domain and use it as the center to perform bandpass filtering to extract the positive first-order spectrum.

[0076] The third step is to shift the extracted positive first-order spectrum to the center of the frequency domain; and to perform an inverse Fourier transform on the shifted spectrum to obtain the complex amplitude distribution of the object light.

[0077] Step 4: Extract the wrapped phase from the complex amplitude distribution to obtain the wrapped phase map.

[0078] Step 5: Use a phase unwrapping algorithm based on least squares or quality-oriented methods to unwrap the wrapped phase map and obtain a continuous phase distribution.

[0079] Step 6: Using the relationship between phase and height h(x,y) = φ(x,y)·λ / (4π) (where λ is the laser wavelength), the continuous phase distribution is converted into a surface height distribution, and the three-dimensional shape of the PCB is reconstructed.

[0080] S104. Based on the reconstructed three-dimensional morphology of the PCB, perform defect detection on the PCB.

[0081] In some examples, step S104 includes: The first step is to perform polynomial fitting or median filtering on the reconstructed PCB 3D topography to remove background trends such as surface tilt and bending, and obtain a height deviation map related to defects.

[0082] The second step involves setting an adaptive height threshold (e.g., ±3σ) based on the statistical distribution of height in flat areas of the PCB surface, performing threshold segmentation on the height deviation map, and initially extracting candidate defect areas.

[0083] The third step is to perform connected component labeling and morphological processing (such as opening operations to remove noise points) on the candidate defect regions, and extract the geometric feature parameters of each defect region, including defect area, maximum depth / height, average depth / height, and volume.

[0084] The fourth step is to classify defects into types such as micro-protrusions, pits, scratches, and residual adhesive based on their height direction (positive / negative) and geometric feature parameters, thus completing the PCB defect detection.

[0085] Figure 4 This is a schematic diagram of a resolution plate 3D reconstruction result provided in one embodiment of this application. See also... Figure 4 The three-dimensional reconstruction result is based on Figure 1 The method shown is used to obtain it.

[0086] in, Figure 4 (a) Optical image of the resolution plate. Figure 4 (b) in the figure represents the original off-axis holographic interference fringe pattern of the resolution plate; used to illustrate the original interference pattern acquired by the system. Figure 4 (c) in the figure represents the interference pattern of the three-dimensional reconstructed region. Figure 4 (d) in the diagram represents the continuous phase distribution after unwrapping; it is used to illustrate the effectiveness of the phase unwrapping process. Figure 4 (e) in the figure represents the reconstructed three-dimensional topography; it is used to illustrate the system's three-dimensional reconstruction capability of the resolution panel. Figure 4 (f) in the figure represents the height profile curve along the selected section; it is used to illustrate the consistency between the reconstructed height accuracy and the reference value.

[0087] Figure 5 This is a schematic diagram of a 3D reconstruction result of a foreign object defect on a PCB provided in one embodiment of this application. See also... Figure 5 ,in, Figure 5 (a) in the image represents an optical image of a foreign object defect on a PCB. Figure 5 (b) in the figure represents the original off-axis holographic interference fringe pattern of the PCB foreign object defect; used to illustrate the original interference pattern acquired by the system. Figure 5 (c) in the figure represents the interference pattern of the three-dimensional reconstruction region of this system. Figure 5 (d) in the figure represents the continuous phase distribution diagram after unwrapping in this system; it is used to illustrate the effectiveness of the phase unwrapping process. Figure 5 (e) Top view of the three-dimensional topography obtained by measuring with a commercial white light interferometer; Figure 5 (f) Side view of the three-dimensional topography obtained by measurement with a commercial white light interferometer; Figure 5In the figure, (g) and (h) represent the height profile curves along the selected section; the solid line represents the WLI measurement result and the dashed line represents the DHM reconstruction result; these are used to further verify the accuracy of the method in this application for the reconstruction of the three-dimensional morphology of foreign objects in PCBs.

[0088] Figure 6 This is a structural block diagram of a PCB defect detection device based on digital holography, provided as an embodiment of this application. See also... Figure 6 ,include: The acquisition module 41 is used to acquire N consecutively acquired interference fringe patterns, where N≥2; wherein, during the acquisition of the interference fringe patterns, the light intensity of the object light and the reference light is acquired in real time and the light intensity of the object light and the reference light is adjusted to make the light intensity of the object light and the reference light equal, thereby improving the quality of the interference fringe patterns. Enhancement module 42 is used to perform multi-frame temporal fusion and weak light enhancement processing on N frames of interference fringe patterns to obtain enhanced interference fringe patterns; Reconstruction module 43 is used to reconstruct the three-dimensional morphology of the PCB based on the enhanced interference fringe pattern; The defect detection module 44 is used to perform defect detection on the PCB based on the reconstructed three-dimensional morphology of the PCB.

[0089] Figure 7 This is a structural block diagram of an electronic device provided according to an embodiment of this application. See also... Figure 7 Electronic devices may include Figure 6The aforementioned PCB defect detection device based on digital holography typically includes a processor 51 and a memory 52. ​​The processor 51 may include one or more processing cores, such as a 4-core processor or an 8-core processor. The processor 51 can be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 51 may also include a main processor and a coprocessor. The main processor is used to process data in the wake-up state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. The memory 52 may include one or more computer-readable storage media, which may be non-transitory. The memory 52 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage medium in memory 52 is used to store at least one instruction, which is executed by processor 51 to implement the digital holographic-based PCB defect detection method performed by an electronic device provided in the method embodiments of this application.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A PCB defect detection method based on digital holography, characterized in that, include: Acquire N consecutively acquired interference fringe patterns, where N≥2; during the acquisition of the interference fringe patterns, the light intensities of the object beam and the reference beam are acquired in real time and adjusted to make the light intensities of the object beam and the reference beam equal, thereby improving the quality of the interference fringe patterns. Multi-frame temporal fusion and low-light enhancement processing are performed on N frames of interference fringe patterns to obtain enhanced interference fringe patterns; Based on the enhanced interference fringe pattern, the three-dimensional morphology of the PCB is reconstructed; Based on the reconstructed three-dimensional morphology of the PCB, defect detection is performed on the PCB.

2. The PCB defect detection method based on digital holography according to claim 1, characterized in that, The steps for performing multi-frame temporal fusion and low-light enhancement processing on N frames of interference fringe patterns to obtain the enhanced interference fringe pattern include: The N frames of interferometric fringe patterns are input into the low-light enhancement model for inter-frame registration to eliminate inter-frame translation and rotation biases. The registered N frames are then weighted and averaged to obtain the fused interferometric fringe pattern. Finally, the fused interferometric fringe pattern is subjected to low-light enhancement processing to obtain the enhanced interferometric fringe pattern.

3. The PCB defect detection method based on digital holography according to claim 1, characterized in that, Low-light enhancement models include: The multi-frame temporal fusion module is used to receive N consecutively acquired interferometric fringe patterns, take one of the frames as a reference frame, estimate the optical flow field of the other frames relative to the reference frame through an optical flow estimation network, perform sub-pixel level spatial alignment of each frame according to the optical flow field, and perform adaptive weighted fusion of the aligned N frames through a temporal fusion network to output a high signal-to-noise ratio fused interferometric fringe pattern. A frequency-spatial dual-stream feature extractor includes a frequency domain branch, a spatial domain branch, and a cross-domain feature fusion module. The frequency domain branch is used to extract frequency-domain sensing features of the interference fringe pattern; the spatial domain branch is used to extract spatial features of the interference fringe pattern; and the cross-domain feature fusion module is used to fuse frequency-domain sensing features and spatial features to obtain multi-domain feature data. The physical constraint decomposition network includes a shared encoder, a reflectance decoding branch, an illumination decoding branch, a phase prior branch, and a contrast estimation branch. The shared encoder is used to perform multi-layer convolutional downsampling on multi-domain feature data to extract multi-scale shared feature representations. The reflectance decoding branch is used to reconstruct the reflectance component of the interferometric image based on the shared features through transposed convolutional upsampling. The illumination decoding branch is used to estimate the illumination component of the interferometric image based on the shared features through a global-local parallel architecture. The phase prior branch is used to estimate the phase prior information of the interference fringes based on the shared features through transposed convolutional upsampling. The contrast estimation branch is used to estimate the global contrast of the interference fringes based on the shared features through adaptive average pooling and fully connected layers. The progressive multi-scale fusion reconstructor is used to recover large-scale illumination distribution, mesoscale interference fringe structure and high-frequency fine texture in sequence through a three-stage progressive reconstruction strategy from coarse to fine based on reflectivity component, illumination component, phase prior information and global contrast, and output an enhanced interference fringe map.

4. The PCB defect detection method based on digital holography according to claim 3, characterized in that, The progressive multiscale fusion reconstructor includes: The system consists of a first-stage reconstruction module, a second-stage reconstruction module, and a third-stage reconstruction module. The first-stage reconstruction module uses a hole residual block to recover the large-scale illumination distribution at one-quarter resolution and performs feature weighting through a channel attention module. The second-stage reconstruction module uses a fringe structure sensitive module to recover the mesoscale interference fringe structure at half resolution and performs feature weighting through a spatial attention module. The third-stage reconstruction module uses a texture refinement module to recover high-frequency fine textures and edge details at full resolution. Information is transferred between the stages through bilinear interpolation upsampling and feature stitching.

5. The PCB defect detection method based on digital holography according to claim 3, characterized in that, The loss function used in the low-light enhancement model is based on reconstruction loss, physical consistency loss, frequency domain fringe fidelity loss, and smoothing regularization loss. Among them, reconstruction loss is obtained by calculating the L1 norm distance between the enhanced interference fringe pattern and the ground truth interference fringe pattern and the structural similarity loss; physical consistency loss is obtained by calculating the L2 norm distance between the product of reflectivity component and illumination component and the enhanced interference fringe pattern; frequency domain fringe fidelity loss is obtained by performing Fourier transform on the enhanced interference fringe pattern and the ground truth interference fringe pattern respectively and then calculating the L1 norm distance between their spectral amplitudes; smoothing regularization loss is obtained by calculating the L1 norm of the gradient of the illumination component in the horizontal and vertical directions, which is used to constrain the spatial smoothness of the illumination component.

6. The PCB defect detection method based on digital holography according to claim 1, characterized in that, The steps for reconstructing the 3D topography of the PCB based on the enhanced interference fringe pattern include: A two-dimensional Fourier transform was performed on the enhanced interference fringe pattern to extract the positive first-order spectral components; Frequency domain filtering and frequency shifting are performed on the positive first-order spectral components to obtain the complex amplitude information of the object light; Phase extraction is performed on the complex amplitude information to obtain the package phase map; The wrapped phase map is unwrapped to obtain a continuous phase distribution; Based on continuous phase distribution and system calibration parameters, phase information is converted into height information, and the three-dimensional shape of the PCB is reconstructed.

7. The PCB defect detection method based on digital holography according to claim 1, characterized in that, Based on the reconstructed 3D morphology of the PCB, the steps for defect detection of the PCB include: The 3D topography is fitted with a reference plane and the background is removed to obtain a defect height deviation map; Set a height threshold, perform threshold segmentation on the defect height deviation map, and extract candidate defect regions; Connectivity analysis is performed on the candidate defect region to extract the geometric feature parameters of the defect, including at least one of the defect area, depth, and volume; Defects are classified and their types are determined based on geometric feature parameters.

8. A PCB defect detection device based on digital holography, characterized in that, include: The acquisition module is used to acquire N consecutively collected interference fringe patterns, where N≥2. During the acquisition of the interference fringe patterns, the light intensities of the object light and the reference light are acquired in real time and adjusted to make the light intensities of the object light and the reference light equal, thereby improving the quality of the interference fringe patterns. The enhancement module is used to perform multi-frame temporal fusion and low-light enhancement processing on N frames of interference fringe patterns to obtain enhanced interference fringe patterns; The reconstruction module is used to reconstruct the three-dimensional topography of the PCB based on the enhanced interference fringe pattern. The defect detection module is used to detect defects in the PCB based on the reconstructed 3D topography.

9. An electronic device, characterized in that, Including the PCB defect detection device based on digital holography as described in claim 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is executed by a processor to implement the PCB defect detection method based on digital holography as described in any one of claims 1 to 7.

Images