PCB intelligent online defect detection device based on multi-mode sensor fusion and edge calculation

The intelligent online defect detection device for PCB boards, which combines multimodal sensor fusion and edge computing, solves the problems of insufficient multimodal data fusion depth and poor real-time performance in existing technologies, achieves efficient and accurate detection of minor defects, adapts to the characteristics of different PCB materials, and meets the real-time detection needs of high-speed production lines.

CN120629161AActive Publication Date: 2025-09-12CHENGYI ELECTRONICS JIAXING

Patent Information

Application Number
CN202510743758.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-06-05
Publication Date
2025-09-12
Estimated Expiration
2045-06-05

AI Technical Summary

Technical Problem

Existing PCB board inspection technology has problems such as insufficient multimodal data fusion depth, poor real-time performance and weak algorithm adaptability, resulting in a high rate of missed detection of minor defects and large fluctuations in the false detection rate, which cannot meet the real-time inspection needs of high-speed production lines.

Method used

The intelligent online defect detection device for PCB boards adopts multimodal sensor fusion and edge computing, including visual acquisition, light source control, light intensity monitoring, displacement detection and ultrasonic detection modules. Combined with image preprocessing, processing acceleration and deep learning modules, it realizes efficient fusion and real-time analysis of multimodal data.

Benefits of technology

It achieves precise detection of tiny defects such as 0.05mm microcracks, reduces false detection and missed detection rates, meets the real-time detection needs of high-speed production lines, improves detection accuracy and production efficiency, and reduces hardware costs and manual intervention.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120629161A_ABST
    Figure CN120629161A_ABST
Patent Text Reader

Abstract

The invention discloses a PCB intelligent online defect detection device based on multi-modal sensor fusion and edge calculation, the PCB intelligent online defect detection device comprises a multi-modal module and an edge calculation module, the multi-modal module is used for collecting data of a PCB, and the edge calculation module is used for processing and analyzing the data. The multi-modal sensor array is fused with multiple detection modes such as vision, 3D laser and ultrasonic, data are comprehensively collected from surface details to an internal structure, a high-precision sensor and an advanced algorithm are matched, tiny defects such as 0.05 mm-level microcracks and various defects such as short circuits, open circuits and cavities can be accurately detected, the detection accuracy is high, and the detection precision is high. Through an edge calculation module and a hardware acceleration and optimization algorithm, rapid image preprocessing, traditional image processing acceleration and deep learning efficient operation are realized, the processing time of a single image is greatly shortened, the real-time detection requirement of a high-speed production line is met, and the production efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field, and in particular to an intelligent online defect detection device for PCB boards based on multimodal sensor fusion and edge computing. Background Art

[0002] A PCB, or printed circuit board, is an important electronic component that supports and electrically connects electronic components. Because it is manufactured using electronic printing technology, it is called a "printed" circuit board.

[0003] Existing detection technologies have the following bottlenecks:

[0004] 1. Insufficient multimodal data fusion depth: Traditional multi-sensor detection only achieves simple data superposition and lacks feature-level fusion, resulting in a high rate of missed detection of minor defects.

[0005] 2. Poor real-time performance: Cloud processing delay exceeds 50ms, which cannot meet the real-time detection requirements of high-speed production lines;

[0006] 3. Weak algorithm adaptability: Fixed parameter models are unable to cope with the differences in material properties of different batches of PCBs, and the false detection rate fluctuates by up to 15%. Summary of the Invention

[0007] The purpose of the present invention is to provide an intelligent online defect detection device for PCB boards based on multimodal sensor fusion and edge computing, so as to solve the problems of insufficient multimodal data fusion depth, poor real-time performance and weak algorithm adaptability proposed in the above background technology.

[0008] To achieve the above objectives, the present invention provides the following technical solutions: an intelligent online defect detection device for PCB boards based on multimodal sensor fusion and edge computing, comprising a multimodal module and an edge computing module. The multimodal module is used to collect data from PCB boards, and the edge computing module is used to process and analyze the data. The multimodal module includes a visual acquisition module, a light source control module, a light intensity monitoring module, a displacement detection module, and an ultrasonic detection module. The edge computing module includes an image preprocessing module, a processing acceleration module, and a deep learning module.

[0009] The visual acquisition module is used to collect images of the thermal distribution on the surface and inside of the PCB board;

[0010] The light source control module is used to adjust the light wavelength, intensity, angle and monitor stability;

[0011] The light intensity monitoring module is used to monitor light intensity, calculate uniformity and alarm abnormal conditions;

[0012] The displacement detection module is used to obtain PCB surface three-dimensional topography data and pre-process and visualize it;

[0013] The ultrasonic testing module is used to penetrate the PCB to detect internal defects and process signals and optimize parameters;

[0014] The image preprocessing module is used for image contrast enhancement, denoising, edge detection, and silkscreen removal;

[0015] The processing acceleration module is used to detect short circuit defects, calculate Euler numbers and area thresholds;

[0016] The deep learning module is used to accelerate model operations, optimize parameters and detection head performance.

[0017] Preferably, the visual acquisition module consists of an 8K linear array camera group, a high-precision line scanning CCD module, a positioning hole visual sensor, and a camera calibration and synchronization module.

[0018] Preferably, the 8K line array camera group uses 12 cameras to collaboratively collect data, with the visible light band capturing the surface details of the PCB board and the near-infrared band penetrating to detect internal thermal distribution anomalies. The resolution is 1920×1080 and the frame rate is 30FPS.

[0019] High-precision scanning CCD module adopts high pixel accuracy, can scan large-sized PCBs, and has low edge distortion rate to ensure image integrity and detail clarity;

[0020] The positioning hole vision sensor uses an industrial area array camera with a telecentric lens to accurately obtain the position of the positioning hole and provide a reference point for image registration;

[0021] The camera calibration and synchronization module regularly calibrates camera parameters to ensure image acquisition accuracy, achieves time synchronization between multiple cameras, and ensures the temporal and spatial consistency of the acquired images.

[0022] Preferably, the light source control module is composed of a multi-color LED lighting array, a light intensity adjustment module, a light angle adjustment mechanism, and a light source stability detection module;

[0023] The multi-color LED lighting array uses a multi-color LED integrated design to combine wavelengths according to detection requirements to enhance the contrast of specific areas;

[0024] The light intensity adjustment module uses a microcontroller to accurately control the LED current, with 0-50000 lux adaptive adjustment to adapt to different PCB reflective characteristics;

[0025] The illumination angle adjustment mechanism uses an adjustable bracket to achieve multi-angle lighting, vertically detect overall defects, and tilt to highlight edge features;

[0026] The light source stability monitoring module is used to monitor light source brightness fluctuations in real time, provide feedback and adjust to ensure lighting stability and reduce image acquisition errors.

[0027] Preferably, the light intensity monitoring module is composed of a light intensity sensor array, a light intensity data processing module and a light abnormality alarm module;

[0028] The light intensity sensor array uses multiple sensors evenly distributed to monitor the light intensity of the environment and the PCB surface in real time with high accuracy;

[0029] The light intensity data processing module is used to analyze and process light intensity data, calculate uniformity, and control the light intensity adjustment module to ensure uniform lighting;

[0030] The abnormal lighting alarm module is used to trigger an alarm when the lighting intensity or uniformity exceeds the set range, prompting you to check the lighting system.

[0031] Preferably, the displacement detection module is composed of a 3D laser displacement sensor, a 3D data acquisition and preprocessing module, and a 3D data visualization module;

[0032] 3D laser displacement sensor: Accuracy of ±1μm, 1kHz scanning frequency to obtain three-dimensional surface topography, measure circuit and solder joint height and shape;

[0033] The 3D data acquisition and preprocessing module is used to collect data in real time and perform preprocessing such as filtering and interpolation to improve data quality and provide reliable data for defect detection;

[0034] The 3D data visualization module is used to convert 3D data into a visual model, intuitively display the PCB surface morphology, and assist in analyzing defect characteristics.

[0035] Preferably, the ultrasonic detection module is composed of an air-coupled ultrasonic probe, an ultrasonic signal processing module and an ultrasonic detection parameter adjustment module;

[0036] The air-coupled ultrasonic probe uses a 50MHz frequency to penetrate the PCB and receive echoes to detect internal defects such as cracks, voids, and delamination;

[0037] The ultrasonic signal processing module is used to amplify, filter, and denoise the echo signal, extract defect features, and transmit them to the analysis module;

[0038] The ultrasonic detection parameter adjustment module is used to adjust the ultrasonic transmission frequency and power according to the PCB thickness, material, etc. to optimize the detection effect.

[0039] Preferably, the image preprocessing module is composed of a grayscale stretching processing module, a silk screen processing module, an edge detection module, an image noise reduction module and an image enhancement module;

[0040] The grayscale stretching processing module uses an improved algorithm to dynamically calculate the grayscale mean, enhance image contrast, and highlight defect areas;

[0041] The silk screen processing module is used to calculate the background color by statistical histogram, remove the logo, reduce the matching calculation amount, and avoid interference;

[0042] The edge detection module uses an 8-operator expansion algorithm to detect edges in multiple directions and an adaptive threshold to extract complete edge information;

[0043] The image noise reduction module uses median filtering, Gaussian filtering and other algorithms to remove image noise and improve image quality;

[0044] The image enhancement module uses a multi-scale detail enhancement algorithm to highlight the edge texture of circuits and pads, improving image clarity.

[0045] Preferably, the processing acceleration module consists of a connectivity analysis module, an Euler number calculation module, an area threshold analysis module, an image registration module and a binarization processing module;

[0046] The connectivity analysis module is used to quickly calculate the number of connected areas of an image based on FPGA hardware logic;

[0047] The Euler number calculation module is used to calculate the Euler number of the image and distinguish the defect types;

[0048] The area threshold analysis module is used to set the area threshold, identify size defects such as protrusions and depressions, and adjust the threshold adaptively;

[0049] The image registration module uses a template matching algorithm to ensure pixel-level alignment between the standard image and the image to be tested, with a registration error of ≤1 pixel;

[0050] The binarization processing module is used to perform image binarization by combining the maximum variance threshold method, etc., to highlight defective areas for subsequent analysis.

[0051] Preferably, the deep learning module consists of a cascade fusion network acceleration module, an efficient channel attention YOLO acceleration module, a YOLOX-like incremental learning acceleration module, a model parameter optimization module, a model compression and quantization module, a multimodal feature fusion acceleration module, and a detection head optimization acceleration module;

[0052] The cascade fusion network is used to accelerate the module by optimizing the hardware resource configuration, providing a high-speed computing environment for the cascade fusion network and accelerating the computing process of the multimodal feature fusion network;

[0053] The Efficient Channel Attention YOLO acceleration module is used to perform hardware acceleration on the Efficient Channel Attention and Feature Recombination modules of Efficient Channel Attention YOLO.

[0054] The YOLOX incremental learning acceleration module uses the YOLOX incremental detection model to accelerate the knowledge distillation calculation between the teacher model and the student model during the incremental learning process;

[0055] The model parameter optimization module is used to monitor the training process of the deep learning model in real time and dynamically adjust the model's hyperparameters based on changes in the loss function and detection accuracy indicators;

[0056] The model compression and quantization module is used to compress and quantize the trained deep learning model to reduce the model's storage space and computational complexity;

[0057] The multimodal feature fusion acceleration module accelerates the feature fusion process of multimodal data, optimizes data transmission and processing procedures, and uses hardware acceleration modules to quickly implement the splicing and fusion operations of different modal features;

[0058] The detection head optimization and acceleration module is used to optimize and accelerate the detection head of the deep learning model, thereby improving the detection head's classification and positioning accuracy for different types of defects.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] 1. In the present invention, a multimodal sensor array is used to integrate multiple detection methods such as vision, 3D laser, and ultrasound to comprehensively collect data from surface details to internal structures. Combined with high-precision sensors and advanced algorithms, it can accurately detect tiny defects such as 0.05mm microcracks, as well as various defects such as short circuits, open circuits, and voids with high detection accuracy. The edge computing module uses hardware acceleration and optimization algorithms to achieve rapid image preprocessing, traditional image processing acceleration, and efficient deep learning operations, greatly shortening the processing time of a single image, meeting the real-time detection needs of high-speed production lines, and improving production efficiency.

[0061] 2. In the present invention, the light intensity monitoring module monitors the lighting conditions in real time to ensure stable image acquisition; the light source control module automatically adjusts the lighting to adapt to the characteristics of different PCB boards; the model parameter optimization module dynamically adjusts the deep learning model parameters to enable the system to operate stably in different production environments, reducing false detections and missed detections; the model compression and quantization module reduces the volume of the deep learning model, lowers the requirements for hardware performance, and reduces hardware costs; automated detection reduces manual intervention, avoids the subjectivity and instability of manual detection, reduces labor costs and subsequent production losses caused by defective products; the YOLOX-type incremental learning acceleration module supports model incremental learning, and combined with the dynamic parameter optimization module, can automatically optimize the detection parameters and models according to historical data and real-time detection conditions, adapt to the detection of new defect types, and continuously improve detection capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a structural diagram of the intelligent online defect detection device for PCB boards based on multimodal sensor fusion and edge computing of the present invention;

[0063] Figure 2This is a structural schematic diagram of the PCB board intelligent online defect detection device based on multimodal sensor fusion and edge computing in the present invention.

[0064] In the picture:

[0065] 1. Multimodal module; 11. Visual acquisition module; 12. Light source control module; 13. Light intensity monitoring module; 14. Displacement detection module; 15. Ultrasonic detection module;

[0066] 2. Edge computing module; 21. Image preprocessing module; 22. Processing acceleration module; 23. Deep learning module. DETAILED DESCRIPTION

[0067] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the implementation regulations described are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0068] Example: Refer to Figure 1-Figure 2 As shown: An intelligent online defect detection device for PCB boards based on multimodal sensor fusion and edge computing includes a multimodal module 1 and an edge computing module 2. The multimodal module 1 is used to collect data from PCB boards, and the edge computing module 2 is used to process and analyze the data. The multimodal module 1 includes a visual acquisition module 11, a light source control module 12, a light intensity monitoring module 13, a displacement detection module 14, and an ultrasonic detection module 15. The edge computing module 2 includes an image preprocessing module 21, a processing acceleration module 22, and a deep learning module 23.

[0069] The visual acquisition module 11 is used to collect images of the surface and internal thermal distribution of the PCB board. It consists of an 8K linear array camera group, a high-precision line scan CCD module, a positioning hole visual sensor, and a camera calibration and synchronization module. The 8K linear array camera group uses 12 cameras to collaboratively collect data. The visible light band captures the surface details of the PCB board and the near-infrared band penetrates to detect internal thermal distribution anomalies. The resolution is 1920×1080 and the frame rate is 30FPS.

[0070] High-precision scanning CCD module adopts high pixel accuracy, can scan large-sized PCBs, and has low edge distortion rate to ensure image integrity and detail clarity;

[0071] The positioning hole vision sensor uses an industrial area array camera with a telecentric lens to accurately obtain the position of the positioning hole and provide a reference point for image registration;

[0072] The camera calibration and synchronization module regularly calibrates camera parameters to ensure image acquisition accuracy, achieves time synchronization between multiple cameras, and ensures the temporal and spatial consistency of the acquired images.

[0073] The light source control module 12 is used to adjust the light wavelength, intensity, angle and stability monitoring, and is composed of a multi-color LED lighting array, a light intensity adjustment module, a light angle adjustment mechanism, and a light source stability detection module;

[0074] The multi-color LED lighting array uses a multi-color LED integrated design to combine wavelengths according to detection requirements to enhance the contrast of specific areas;

[0075] The light intensity adjustment module uses a microcontroller to accurately control the LED current, with 0-50000 lux adaptive adjustment to adapt to different PCB reflective characteristics;

[0076] The illumination angle adjustment mechanism uses an adjustable bracket to achieve multi-angle lighting, vertically detect overall defects, and tilt to highlight edge features;

[0077] The light source stability monitoring module is used to monitor light source brightness fluctuations in real time, provide feedback and adjust to ensure lighting stability and reduce image acquisition errors.

[0078] The light intensity monitoring module 13 is used to monitor light intensity, calculate uniformity and alarm abnormalities. The light intensity monitoring module 13 consists of a light intensity sensor array, a light intensity data processing module and a light abnormality alarm module;

[0079] The light intensity sensor array uses multiple sensors evenly distributed to monitor the light intensity of the environment and the PCB surface in real time with high accuracy;

[0080] The light intensity data processing module is used to analyze and process light intensity data, calculate uniformity, and control the light intensity adjustment module to ensure uniform lighting;

[0081] The abnormal lighting alarm module is used to trigger an alarm when the lighting intensity or uniformity exceeds the set range, prompting you to check the lighting system.

[0082] The displacement detection module 14 is used to obtain the three-dimensional topography data of the PCB surface and pre-process and visualize it. The displacement detection module 14 consists of a 3D laser displacement sensor, a 3D data acquisition and pre-processing module, and a 3D data visualization module;

[0083] 3D laser displacement sensor: Accuracy of ±1μm, 1kHz scanning frequency to obtain three-dimensional surface topography, measure circuit and solder joint height and shape;

[0084] The 3D data acquisition and preprocessing module is used to collect data in real time and perform preprocessing such as filtering and interpolation to improve data quality and provide reliable data for defect detection;

[0085] The 3D data visualization module is used to convert 3D data into a visual model, intuitively display the PCB surface morphology, and assist in analyzing defect characteristics.

[0086] The ultrasonic detection module 15 is used to penetrate the PCB to detect internal defects and process signals and optimize parameters. The ultrasonic detection module 15 consists of an air-coupled ultrasonic probe, an ultrasonic signal processing module, and an ultrasonic detection parameter adjustment module;

[0087] The air-coupled ultrasonic probe uses a 50MHz frequency to penetrate the PCB and receive echoes to detect internal defects such as cracks, voids, and delamination;

[0088] The ultrasonic signal processing module is used to amplify, filter, and denoise the echo signal, extract defect features, and transmit them to the analysis module;

[0089] The ultrasonic detection parameter adjustment module is used to adjust the ultrasonic transmission frequency and power according to the PCB thickness, material, etc. to optimize the detection effect.

[0090] The image preprocessing module 21 is used to enhance image contrast, denoise, detect edges, and remove silk screen printing. The image preprocessing module 21 consists of a grayscale stretching processing module, a silk screen processing module, an edge detection module, an image noise reduction module, and an image enhancement module.

[0091] The grayscale stretching processing module uses an improved algorithm to dynamically calculate the grayscale mean, enhance image contrast, and highlight defect areas;

[0092] The silk screen processing module is used to calculate the background color by statistical histogram, remove the logo, reduce the matching calculation amount, and avoid interference;

[0093] The edge detection module uses an 8-operator expansion algorithm to detect edges in multiple directions and an adaptive threshold to extract complete edge information;

[0094] The image noise reduction module uses median filtering, Gaussian filtering and other algorithms to remove image noise and improve image quality;

[0095] The image enhancement module uses a multi-scale detail enhancement algorithm to highlight the edge texture of circuits and pads, improving image clarity.

[0096] The processing acceleration module 22 is used to detect short circuit defects, calculate the Euler number and area threshold, and the processing acceleration module 22 consists of a connectivity analysis module, an Euler number calculation module, an area threshold analysis module, an image registration module and a binarization processing module;

[0097] The connectivity analysis module is used to quickly calculate the number of connected areas of an image based on FPGA hardware logic;

[0098] The Euler number calculation module is used to calculate the Euler number of the image and distinguish the defect types;

[0099] The area threshold analysis module is used to set the area threshold, identify size defects such as protrusions and depressions, and adjust the threshold adaptively;

[0100] The image registration module uses a template matching algorithm to ensure pixel-level alignment between the standard image and the image to be tested, with a registration error of ≤1 pixel;

[0101] The binarization processing module is used to perform image binarization by combining the maximum variance threshold method, etc., to highlight defective areas for subsequent analysis.

[0102] The deep learning module 23 is used to accelerate model operations, optimize parameters and detection head performance. The deep learning module 23 consists of a cascade fusion network acceleration module, an efficient channel attention YOLO acceleration module, a YOLOX-like incremental learning acceleration module, a model parameter optimization module, a model compression and quantization module, a multimodal feature fusion acceleration module, and a detection head optimization acceleration module;

[0103] The cascaded fusion network is used to accelerate the module by optimizing hardware resource configuration, providing a high-speed computing environment for the cascaded fusion network and accelerating the computing process of the multimodal feature fusion network. With the collaborative work of the FPGA and NVIDIA Jetson AGX Orin module, data transmission delays are reduced, and the overall detection speed is improved, ensuring that the processing time of a single image meets the real-time detection requirements in high-speed production line scenarios.

[0104] The Efficient Channel Attention YOLO Accelerator module is used to perform hardware acceleration on the Efficient Channel Attention (ECA) and feature reconstruction modules of the Efficient Channel Attention YOLO algorithm. Leveraging the parallel computing capabilities of the FPGA, it quickly calculates channel weights and enhances the characteristic response of small target defects, increasing the model's attention to small targets by 30% and effectively improving the detection accuracy of small target defects.

[0105] The YOLOX incremental learning acceleration module uses the YOLOX incremental detection model to accelerate the knowledge distillation calculation between the teacher model and the student model during the incremental learning process. Leveraging the hardware acceleration engine, it quickly processes the distillation loss calculation of output features and intermediate features. This allows the model to better retain knowledge of old defect categories while learning new ones. In the two-stage incremental scenario, the average accuracy of old categories remains above 88.5%, reducing model training time by 30%.

[0106] The model parameter optimization module is used to monitor the training process of the deep learning model in real time. It dynamically adjusts the model's hyperparameters, such as the learning rate and weight decay coefficient, based on changes in the loss function and detection accuracy indicators. It uses an adaptive optimization algorithm to automatically find the optimal parameter combination, improve the model's convergence speed and detection performance, and ensure that the model maintains stable and efficient detection capabilities in different PCB production environments.

[0107] The model compression and quantization module is used to compress and quantize the trained deep learning model, reducing the model's storage space and computational complexity. It removes redundant connections and parameters through pruning technology and replaces floating-point data with low-precision data formats. Without significantly reducing detection accuracy, it improves the model's operating efficiency on edge computing devices, reduces hardware costs, and meets the strict storage and computing resource constraints of industrial equipment.

[0108] The multimodal feature fusion acceleration module accelerates the feature fusion process of multimodal data, optimizes data transmission and processing, and uses hardware acceleration modules to quickly implement splicing and fusion operations of different modal features, improving the efficiency of multimodal feature fusion, enhancing the model's ability to detect complex defects, and improving comprehensive detection accuracy by 5%;

[0109] The detection head optimization and acceleration module is used to optimize and accelerate the detection head of the deep learning model, improve the detection head's classification and positioning accuracy for different types of defects, achieve fast calculation of the convolutional layer and fully connected layer in the detection head through hardware acceleration, optimize the execution efficiency of the non-maximum suppression algorithm, reduce false detections and missed detections, and increase the detection accuracy of the detection head for common defects to more than 98%.

[0110] In the present invention, a multimodal sensor array is integrated with vision, 3D laser, ultrasound and other detection methods to comprehensively collect data from surface details to internal structures. With the help of high-precision sensors and advanced algorithms, it can accurately detect tiny defects such as 0.05mm microcracks, as well as various defects such as short circuits, open circuits, and voids. The detection accuracy is high. The edge computing module uses hardware acceleration and optimization algorithms to achieve fast image preprocessing, traditional image processing acceleration and deep learning efficient computing, greatly shortening the processing time of a single image, meeting the real-time detection needs of high-speed production lines, and improving production efficiency. The light intensity monitoring module monitors the lighting conditions in real time to ensure stable image acquisition; the light source control module automatically Adjust lighting to adapt to the characteristics of different PCB boards; the model parameter optimization module dynamically adjusts the deep learning model parameters to enable the system to run stably in different production environments, reducing false detections and missed detections. The model compression and quantization module reduces the size of the deep learning model, lowers hardware performance requirements, and reduces hardware costs. Automated detection reduces manual intervention, avoids the subjectivity and instability of manual detection, and reduces labor costs and subsequent production losses caused by defective products. The YOLOX-type incremental learning acceleration module supports model incremental learning. Combined with the dynamic parameter optimization module, it can automatically optimize detection parameters and models based on historical data and real-time detection conditions, adapt to the detection of new defect types, and continuously improve detection capabilities.

[0111] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. PCB board intelligent online defect detection device based on multimodal sensor fusion and edge computing, characterized by: The system comprises a multimodal module (1) and an edge computing module (2), wherein the multimodal module (1) is used to collect data of a PCB board, and the edge computing module (2) is used to process and analyze the data. The multimodal module (1) comprises a visual acquisition module (11), a light source control module (12), a light intensity monitoring module (13), a displacement detection module (14), and an ultrasonic detection module (15), and the edge computing module (2) comprises an image preprocessing module (21), a processing acceleration module (22), and a deep learning module (23); The visual acquisition module (11) is used to acquire the thermal distribution images of the surface and interior of the PCB board; The light source control module (12) is used to adjust the wavelength, intensity, angle and stability of the light; The light intensity monitoring module (13) is used to monitor light intensity, calculate uniformity and alarm abnormal conditions; The displacement detection module (14) is used to obtain the three-dimensional topography data of the PCB surface and perform pre-processing and visualization; The ultrasonic detection module (15) is used to penetrate the PCB to detect internal defects and process signals and optimize parameters; The image preprocessing module (21) is used for image contrast enhancement, noise removal, edge detection and silk screen removal; The processing acceleration module (22) is used to detect short circuit defects and calculate the Euler number and area threshold; The deep learning module (23) is used to accelerate model operations, optimize parameters and detection head performance.

2. The PCB intelligent online defect detection device based on multimodal sensor fusion and edge computing according to claim 1 is characterized in that: The visual acquisition module (11) is composed of an 8K linear array camera group, a high-precision line scanning CCD module, a positioning hole visual sensor, and a camera calibration and synchronization module.

3. The PCB intelligent online defect detection device based on multimodal sensor fusion and edge computing according to claim 2 is characterized in that: The 8K line array camera group uses 12 cameras to collaboratively collect data. The visible light band captures the surface details of the PCB board and the near-infrared band penetrates to detect internal thermal distribution anomalies. The resolution is 1920×1080 and the frame rate is 30FPS. High-precision scanning CCD module adopts high pixel accuracy; The positioning hole vision sensor uses an industrial area array camera with a telecentric lens; The camera calibration and synchronization module is used to calibrate camera parameters regularly.

4. The PCB intelligent online defect detection device based on multimodal sensor fusion and edge computing according to claim 1 is characterized in that: The light source control module (12) is composed of a multi-color LED lighting array, a light intensity adjustment module, a light angle adjustment mechanism, and a light source stability detection module; The multi-color LED lighting array uses a multi-color LED integrated design and combines wavelengths according to detection requirements; The light intensity adjustment module uses a microcontroller to accurately control the LED current, with 0-50000 lux adaptive adjustment to adapt to different PCB reflective characteristics; The illumination angle adjustment mechanism uses an adjustable bracket to achieve multi-angle lighting, vertically detect overall defects, and tilt to highlight edge features; The light source stability monitoring module is used to monitor light source brightness fluctuations in real time and provide feedback for adjustment.

5. The PCB intelligent online defect detection device based on multimodal sensor fusion and edge computing according to claim 1 is characterized in that: The light intensity monitoring module (13) is composed of a light intensity sensor array, a light intensity data processing module and a light abnormality alarm module; The light intensity sensor array uses multiple sensors evenly distributed to monitor the light intensity of the environment and the PCB surface in real time; The light intensity data processing module is used to analyze and process light intensity data, calculate uniformity, and control the light intensity adjustment module; The abnormal lighting alarm module is used to trigger an alarm when the lighting intensity or uniformity exceeds the set range, prompting you to check the lighting system.

6. The PCB intelligent online defect detection device based on multimodal sensor fusion and edge computing according to claim 1 is characterized in that: The displacement detection module (14) is composed of a 3D laser displacement sensor, a 3D data acquisition and preprocessing module, and a 3D data visualization module; 3D laser displacement sensor: Accuracy of ±1μm, 1kHz scanning frequency to obtain three-dimensional surface topography, measure circuit and solder joint height and shape; The 3D data acquisition and preprocessing module is used to collect data in real time and perform preprocessing such as filtering and interpolation to improve data quality; The 3D data visualization module is used to convert 3D data into a visual model, intuitively display the PCB surface morphology, and assist in analyzing defect characteristics.

7. The PCB intelligent online defect detection device based on multimodal sensor fusion and edge computing according to claim 1 is characterized in that: The ultrasonic detection module (15) is composed of an air-coupled ultrasonic probe, an ultrasonic signal processing module and an ultrasonic detection parameter adjustment module; The air-coupled ultrasonic probe uses a 50MHz frequency to penetrate the PCB and receive echoes to detect internal defects; The ultrasonic signal processing module is used to amplify, filter, and denoise the echo signal, extract defect features, and transmit them to the analysis module; The ultrasonic detection parameter adjustment module is used to adjust the ultrasonic transmission frequency and power according to the PCB thickness, material, etc.

8. The PCB intelligent online defect detection device based on multimodal sensor fusion and edge computing according to claim 1 is characterized in that: The image preprocessing module (21) is composed of a grayscale stretching processing module, a silk screen processing module, an edge detection module, an image noise reduction module and an image enhancement module; The grayscale stretching processing module uses an improved algorithm to dynamically calculate the grayscale mean, enhance image contrast, and highlight defect areas; The silk screen processing module is used to calculate the background color by statistical histogram and remove the logo; The edge detection module uses an 8-operator expansion algorithm to detect edges in multiple directions and an adaptive threshold to extract complete edge information; The image noise reduction module uses algorithms such as median filtering and Gaussian filtering to remove image noise; The image enhancement module uses a multi-scale detail enhancement algorithm to highlight the edge texture of circuits and pads.

9. The PCB intelligent online defect detection device based on multimodal sensor fusion and edge computing according to claim 1 is characterized in that: The processing acceleration module (22) is composed of a connectivity analysis module, an Euler number calculation module, an area threshold analysis module, an image registration module and a binarization processing module; The connectivity analysis module is used to quickly calculate the number of connected areas of an image based on FPGA hardware logic; The Euler number calculation module is used to calculate the Euler number of the image and distinguish the defect types; The area threshold analysis module is used to set the area threshold, identify size defects, and adjust the threshold adaptively; The image registration module uses a template matching algorithm to ensure pixel-level alignment between the standard image and the image to be tested, with a registration error of ≤1 pixel; The binarization processing module is used to perform image binarization in combination with the maximum variance threshold method to highlight defect areas.

10. The PCB intelligent online defect detection device based on multimodal sensor fusion and edge computing according to claim 1 is characterized in that: The deep learning module (23) consists of a cascade fusion network acceleration module, an efficient channel attention YOLO acceleration module, a YOLOX-like incremental learning acceleration module, a model parameter optimization module, a model compression and quantization module, a multimodal feature fusion acceleration module, and a detection head optimization acceleration module; The cascade fusion network is used to accelerate the module by optimizing the hardware resource configuration, providing a high-speed computing environment for the cascade fusion network and accelerating the computing process of the multimodal feature fusion network; The Efficient Channel Attention YOLO acceleration module is used to perform hardware acceleration on the Efficient Channel Attention and Feature Recombination modules of Efficient Channel Attention YOLO. The YOLOX incremental learning acceleration module uses the YOLOX incremental detection model to accelerate the knowledge distillation calculation between the teacher model and the student model during the incremental learning process; The model parameter optimization module is used to monitor the training process of the deep learning model in real time and dynamically adjust the model's hyperparameters based on changes in the loss function and detection accuracy indicators; The model compression and quantization module is used to compress and quantize the trained deep learning model to reduce the model's storage space and computational complexity; The multimodal feature fusion acceleration module accelerates the feature fusion process of multimodal data, optimizes data transmission and processing procedures, and uses hardware acceleration modules to quickly implement the splicing and fusion operations of different modal features; The detection head optimization and acceleration module is used to optimize and accelerate the detection head of the deep learning model.

Citation Information

Patent Citations

  • Cable head defect intelligent detection system

    CN117455843A

  • Metal material defect identification analysis method and system based on multiple detection technology

    CN119335157A

  • High-speed laser vision sensor system, high-speed image processing method and weld bead inspection mehtod using the same

    KR1020100002583A

Cited By

  • Robot laser vision three-dimensional scanning measurement system for large-scale structural member

    CN121026013A

  • Intelligent monitoring system for endoscopic injury detection of energy pipeline

    CN121275791A

  • Online intelligent monitoring system and method in electronic product production process

    CN121740299A

  • Defect detection method for semitransparent multilayer film material

    CN122089729A