Intelligent Online Defect Detection Device for PCB Boards Based on Multimodal Sensor Fusion and Edge Computing

Abstract

This invention discloses an intelligent online defect detection device for PCB boards based on multimodal sensor fusion and edge computing. It includes a multimodal module and an edge computing module. The multimodal module collects data from the PCB board, while the edge computing module processes and analyzes the data. This invention integrates multiple detection methods, including vision, 3D laser, and ultrasound, using a multimodal sensor array to comprehensively collect data from surface details to internal structures. Combined with high-precision sensors and advanced algorithms, it can accurately detect minute defects such as microcracks as small as 0.05mm, as well as various defects such as short circuits, open circuits, and voids, achieving high detection accuracy. The edge computing module, through hardware acceleration and algorithm optimization, enables rapid image preprocessing, accelerated traditional image processing, and efficient deep learning computation, significantly reducing the processing time per image, meeting the real-time detection needs of high-speed production lines, and improving production efficiency.

Description

Technical Field

[0001] This invention relates to the field of technology, specifically to an intelligent online defect detection device for PCB boards based on multimodal sensor fusion and edge computing. Background Technology

[0002] A PCB, or printed circuit board, is an important electronic component. It serves as the support for electronic components and the carrier for their electrical interconnections. Because it is manufactured using electronic printing techniques, it is called a "printed" circuit board.

[0003] Existing detection technologies suffer from the following bottlenecks:

[0004] 1. Insufficient depth of multimodal data fusion: Traditional multi-sensor detection only achieves simple data superposition and lacks feature-level fusion, resulting in a high rate of missed detection of minute defects;

[0005] 2. Poor real-time performance: Cloud processing latency exceeds 50ms, which cannot meet the real-time inspection requirements of high-speed production lines;

[0006] 3. Weak algorithm adaptability: Fixed parameter models are unable to cope with the differences in the characteristics of PCB materials from different batches, and the false detection rate fluctuates by up to 15%. Summary of the Invention

[0007] The purpose of this invention is to provide an intelligent online defect detection device for PCB boards based on multimodal sensor fusion and edge computing, so as to solve the problems of insufficient multimodal data fusion depth, poor real-time performance, and weak algorithm adaptability mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: an intelligent online defect detection device for PCB boards based on multimodal sensor fusion and edge computing, comprising a multimodal module and an edge computing module. The multimodal module is used to collect data from the PCB board, and the edge computing module is used to process and analyze the data. The multimodal module includes a visual acquisition module, a light source control module, a light intensity monitoring module, a displacement detection module, and an ultrasonic detection module. The edge computing module includes an image preprocessing module, a processing acceleration module, and a deep learning module.

[0009] The visual acquisition module is used to acquire images of the surface and internal heat distribution of the PCB board;

[0010] The light source control module is used to adjust the wavelength, intensity, and angle of the light and monitor its stability.

[0011] The light intensity monitoring module is used to monitor light intensity, calculate uniformity, and alarm for abnormal conditions;

[0012] The displacement detection module is used to acquire, preprocess, and visualize the three-dimensional topography data of the PCB surface;

[0013] The ultrasonic testing module is used to penetrate PCBs to detect internal defects and process signals and optimize parameters.

[0014] The image preprocessing module is used to enhance image contrast, denoise, perform edge detection, and remove silkscreen printing.

[0015] The processing acceleration module is used to detect short-circuit defects, calculate Euler numbers, and area thresholds;

[0016] The deep learning module is used to accelerate model computation, optimize parameters, and improve the performance of the detection head.

[0017] Preferably, the vision acquisition module consists of an 8K line scan camera group, a high-precision line scan CCD module, a positioning hole vision sensor, and a camera calibration and synchronization module.

[0018] Preferably, the 8K line array camera group uses 12 cameras to collect data in a coordinated manner. The visible light band captures the surface details of the PCB board, and the near-infrared band penetrates to detect internal heat distribution anomalies. The resolution is 1920×1080 and the frame rate is 30FPS.

[0019] The high-precision scanning CCD module uses high pixel precision, can scan large-size PCBs, and has low edge distortion rate to ensure image integrity and detail clarity;

[0020] The positioning hole vision sensor uses an industrial area array camera with a telecentric lens to accurately acquire the position of the positioning hole and provide a reference point for image registration.

[0021] The camera calibration and synchronization module periodically calibrates camera parameters to ensure image acquisition accuracy, achieves time synchronization among multiple cameras, and ensures spatiotemporal consistency of acquired images.

[0022] Preferably, the light source control module consists of a multi-color LED lighting array, a light intensity adjustment module, a light angle adjustment mechanism, and a light source stability detection module;

[0023] The multi-color LED lighting array adopts a multi-color LED integrated design, combining wavelengths according to detection requirements to enhance the contrast of specific areas;

[0024] The light intensity adjustment module uses a microcontroller to precisely control the LED current, with adaptive adjustment from 0-50000 lux to suit different PCB reflective characteristics;

[0025] The illumination angle adjustment mechanism uses an adjustable bracket to achieve multi-angle illumination, vertically inspecting overall defects, and tilting to highlight edge features;

[0026] The light source stability monitoring module is used to monitor light source brightness fluctuations in real time, provide feedback and adjustment, ensure stable illumination, and reduce image acquisition errors.

[0027] Preferably, the light intensity monitoring module consists of a light intensity sensor array, a light intensity data processing module, and a light abnormality alarm module;

[0028] The light intensity sensor array uses multiple sensors evenly distributed to monitor the light intensity of the environment and PCB surface in real time with high accuracy.

[0029] The light intensity data processing module is used to analyze and process light intensity data, calculate uniformity, and control the light intensity adjustment module to ensure uniform illumination.

[0030] The lighting anomaly alarm module is used to trigger an alarm and prompt the user to check the light source system when the light intensity or uniformity exceeds the set range.

[0031] Preferably, the displacement detection module consists of a 3D laser displacement sensor, a 3D data acquisition and preprocessing module, and a 3D data visualization module;

[0032] 3D laser displacement sensor: accuracy ±1μm, 1kHz scanning frequency to acquire three-dimensional surface morphology, and measure the height and shape of circuits and solder joints;

[0033] The 3D data acquisition and preprocessing module is used to acquire data in real time, perform preprocessing such as filtering and interpolation, improve data quality, and provide reliable data for defect detection.

[0034] The 3D data visualization module is used to transform 3D data into a visualization model, which can intuitively display the surface morphology of the PCB and help analyze defect characteristics.

[0035] Preferably, the ultrasonic testing module consists of an air-coupled ultrasonic probe, an ultrasonic signal processing module, and an ultrasonic testing parameter adjustment module;

[0036] The air-coupled ultrasonic probe uses a 50MHz frequency to penetrate the PCB and receive echoes to detect internal defects such as cracks, voids, and delamination.

[0037] The ultrasonic signal processing module is used to amplify, filter, and denoise echo signals, extract defect features, and transmit them to the analysis module.

[0038] The ultrasonic testing parameter adjustment module is used to adjust the ultrasonic emission frequency and power according to the PCB thickness, material, etc., to optimize the testing effect.

[0039] Preferably, the image preprocessing module consists of a grayscale stretching module, a screen printing module, an edge detection module, an image noise reduction module, and an image enhancement module;

[0040] The grayscale stretching module uses an improved algorithm to dynamically calculate the mean grayscale value, enhance image contrast, and highlight defective areas.

[0041] The silkscreen processing module is used to locate the background color by statistical histogram, remove the logo text, reduce the amount of matching calculations, and avoid interference;

[0042] The edge detection module uses an 8-operator extended algorithm to detect edges in multiple directions and uses adaptive thresholds to extract complete edge information.

[0043] The image noise reduction module uses algorithms such as median filtering and Gaussian filtering to remove image noise and improve image quality;

[0044] The image enhancement module uses a multi-scale detail enhancement algorithm to highlight the texture of circuit and pad edges, thereby improving image clarity.

[0045] Preferably, the processing acceleration module consists of a connectivity analysis module, an Euler number calculation module, an area threshold analysis module, an image registration module, and a binarization processing module;

[0046] The connectivity analysis module is used to quickly calculate the number of connected regions in an image based on FPGA hardware logic;

[0047] The Euler number calculation module is used to calculate the Euler number of an image and distinguish defect types.

[0048] The area threshold analysis module is used to set an area threshold and identify size-related defects such as protrusions and depressions. The threshold is automatically adjusted.

[0049] The image registration module uses a template matching algorithm to ensure pixel-level alignment between the standard image and the image under test, with a registration error of ≤1 pixel.

[0050] The binarization module is used to perform image binarization by combining methods such as the maximum variance thresholding method, highlighting defect areas and facilitating subsequent analysis.

[0051] Preferably, the deep learning module consists of a cascaded fusion network acceleration module, an efficient channel attention YOLO acceleration module, a YOLOX-like incremental learning acceleration module, a model parameter optimization module, a model compression and quantization module, a multimodal feature fusion acceleration module, and a detection head optimization acceleration module.

[0052] The cascaded fusion network is used to accelerate the module by optimizing hardware resource configuration, providing a high-speed computing environment for the cascaded fusion network, and accelerating the computing process of the multimodal feature fusion network;

[0053] The high-efficiency channel attention YOLO acceleration module is used to provide hardware acceleration for the high-efficiency channel attention and feature reconstruction modules of high-efficiency channel attention YOLO.

[0054] The YOLOX-type incremental learning acceleration module uses a YOLOX-type incremental detection model to accelerate knowledge distillation calculations between the teacher model and the student model during the incremental learning process.

[0055] The model parameter optimization module is used to monitor the training process of the deep learning model in real time and dynamically adjust the model's hyperparameters based on changes in the loss function and detection accuracy metrics.

[0056] The model compression and quantization module is used to compress and quantize trained deep learning models, reducing the model's storage space and computational load.

[0057] The multimodal feature fusion acceleration module accelerates the feature fusion process of multimodal data, optimizes data transmission and processing flow, and uses hardware acceleration modules to quickly realize the splicing and fusion operations of different modal features;

[0058] The detection head optimization and acceleration module is used to optimize and accelerate the detection head of the deep learning model, thereby improving the accuracy of the detection head in classifying and locating different types of defects.

[0059] Compared with the prior art, the beneficial effects of the present invention are:

[0060] 1. In this invention, a multimodal sensor array is used to fuse multiple detection methods such as vision, 3D laser, and ultrasound to comprehensively collect data from surface details to internal structure. Combined with high-precision sensors and advanced algorithms, it can accurately detect minute defects such as microcracks at the 0.05mm level, as well as various defects such as short circuits, open circuits, and voids. The detection accuracy is high. Through the edge computing module, hardware acceleration and algorithm optimization are used to achieve rapid image preprocessing, acceleration of traditional image processing, and efficient deep learning computation, which greatly shortens the processing time of a single image, meets the real-time detection needs of high-speed production lines, and improves production efficiency.

[0061] 2. In this invention, the light intensity monitoring module monitors the lighting conditions in real time to ensure stable image acquisition; the light source control module automatically adjusts the lighting to adapt to different PCB board characteristics; the model parameter optimization module dynamically adjusts the deep learning model parameters to ensure stable operation of the system under different production environments, reducing false positives and false negatives; the model compression and quantization module reduces the size of the deep learning model, lowers the hardware performance requirements, and reduces hardware costs; automated detection reduces manual intervention, avoids the subjectivity and instability of manual detection, and reduces labor costs and subsequent production losses caused by defective products; the YOLOX-type incremental learning acceleration module supports incremental model learning, and combined with the dynamic parameter optimization module, it can automatically optimize detection parameters and models based on historical data and real-time detection conditions, adapt to the detection of new defect types, and continuously improve detection capabilities. Attached Figure Description

[0062] Figure 1 This is a schematic diagram of the intelligent online defect detection device for PCB boards based on multimodal sensor fusion and edge computing according to the present invention;

[0063] Figure 2This is a schematic diagram of the intelligent online defect detection device for PCB boards based on multimodal sensor fusion and edge computing according to the present invention.

[0064] In the picture:

[0065] 1. Multimodal module; 11. Visual acquisition module; 12. Light source control module; 13. Light intensity monitoring module; 14. Displacement detection module; 15. Ultrasonic detection module;

[0066] 2. Edge computing module; 21. Image preprocessing module; 22. Processing acceleration module; 23. Deep learning module. Detailed Implementation

[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] Example: Refer to Figures 1-2 As shown: A smart online defect detection device for PCB boards based on multimodal sensor fusion and edge computing includes a multimodal module 1 and an edge computing module 2. The multimodal module 1 is used to collect data from the PCB board, and the edge computing module 2 is used to process and analyze the data. The multimodal module 1 includes a vision acquisition module 11, a light source control module 12, a light intensity monitoring module 13, a displacement detection module 14, and an ultrasonic detection module 15. The edge computing module 2 includes an image preprocessing module 21, a processing acceleration module 22, and a deep learning module 23.

[0069] The visual acquisition module 11 is used to acquire images of the surface and internal heat distribution of the PCB board. The visual acquisition module 11 consists of an 8K line scan camera group, a high-precision line scan CCD module, a positioning hole vision sensor, and a camera calibration and synchronization module. The 8K line scan camera group uses 12 cameras to acquire images in a coordinated manner. The visible light band captures the surface details of the PCB board, and the near-infrared band penetrates to detect internal heat distribution anomalies. The resolution is 1920×1080 and the frame rate is 30FPS.

[0070] The high-precision scanning CCD module uses high pixel precision, can scan large-size PCBs, and has low edge distortion rate to ensure image integrity and detail clarity;

[0071] The positioning hole vision sensor uses an industrial area array camera with a telecentric lens to accurately acquire the position of the positioning hole and provide a reference point for image registration.

[0072] The camera calibration and synchronization module periodically calibrates camera parameters to ensure image acquisition accuracy, achieves time synchronization among multiple cameras, and ensures spatiotemporal consistency of acquired images.

[0073] The light source control module 12 is used to adjust the wavelength, intensity, and angle of the light and monitor its stability. It consists of a multi-color LED lighting array, a light intensity adjustment module, a light angle adjustment mechanism, and a light source stability detection module.

[0074] The multi-color LED lighting array adopts a multi-color LED integrated design, combining wavelengths according to detection requirements to enhance the contrast of specific areas;

[0075] The light intensity adjustment module uses a microcontroller to precisely control the LED current, with adaptive adjustment from 0-50000 lux to suit different PCB reflective characteristics;

[0076] The illumination angle adjustment mechanism uses an adjustable bracket to achieve multi-angle illumination, vertically inspecting overall defects, and tilting to highlight edge features;

[0077] The light source stability monitoring module is used to monitor light source brightness fluctuations in real time, provide feedback and adjustment, ensure stable illumination, and reduce image acquisition errors.

[0078] The light intensity monitoring module 13 is used to monitor light intensity, calculate uniformity and alarm abnormal conditions. The light intensity monitoring module 13 consists of a light intensity sensor array, a light intensity data processing module and a light abnormality alarm module.

[0079] The light intensity sensor array uses multiple sensors evenly distributed to monitor the light intensity of the environment and PCB surface in real time with high accuracy.

[0080] The light intensity data processing module is used to analyze and process light intensity data, calculate uniformity, and control the light intensity adjustment module to ensure uniform illumination.

[0081] The lighting anomaly alarm module is used to trigger an alarm and prompt the user to check the light source system when the light intensity or uniformity exceeds the set range.

[0082] The displacement detection module 14 is used to acquire, preprocess, and visualize the three-dimensional topography data of the PCB surface. The displacement detection module 14 consists of a 3D laser displacement sensor, a 3D data acquisition and preprocessing module, and a 3D data visualization module.

[0083] 3D laser displacement sensor: accuracy ±1μm, 1kHz scanning frequency to acquire three-dimensional surface morphology, and measure the height and shape of circuits and solder joints;

[0084] The 3D data acquisition and preprocessing module is used to acquire data in real time, perform preprocessing such as filtering and interpolation, improve data quality, and provide reliable data for defect detection.

[0085] The 3D data visualization module is used to transform 3D data into a visualization model, which can intuitively display the surface morphology of the PCB and help analyze defect characteristics.

[0086] The ultrasonic testing module 15 is used to penetrate the PCB to detect internal defects and process signals and optimize parameters. The ultrasonic testing module 15 consists of an air-coupled ultrasonic probe, an ultrasonic signal processing module and an ultrasonic testing parameter adjustment module.

[0087] The air-coupled ultrasonic probe uses a 50MHz frequency to penetrate the PCB and receive echoes to detect internal defects such as cracks, voids, and delamination.

[0088] The ultrasonic signal processing module is used to amplify, filter, and denoise echo signals, extract defect features, and transmit them to the analysis module.

[0089] The ultrasonic testing parameter adjustment module is used to adjust the ultrasonic emission frequency and power according to the PCB thickness, material, etc., to optimize the testing effect.

[0090] The image preprocessing module 21 is used to enhance image contrast, denoise, detect edges, and remove silkscreen. The image preprocessing module 21 consists of a grayscale stretching module, a silkscreen processing module, an edge detection module, an image denoising module, and an image enhancement module.

[0091] The grayscale stretching module uses an improved algorithm to dynamically calculate the mean grayscale value, enhance image contrast, and highlight defective areas.

[0092] The silkscreen processing module is used to locate the background color by statistical histogram, remove the logo text, reduce the amount of matching calculations, and avoid interference;

[0093] The edge detection module uses an 8-operator extended algorithm to detect edges in multiple directions and uses adaptive thresholds to extract complete edge information.

[0094] The image noise reduction module uses algorithms such as median filtering and Gaussian filtering to remove image noise and improve image quality;

[0095] The image enhancement module uses a multi-scale detail enhancement algorithm to highlight the texture of circuit and pad edges, thereby improving image clarity.

[0096] The processing acceleration module 22 is used to detect short-circuit defects, calculate Euler number and area threshold. The processing acceleration module 22 consists of a connectivity analysis module, an Euler number calculation module, an area threshold analysis module, an image registration module and a binarization processing module.

[0097] The connectivity analysis module is used to quickly calculate the number of connected regions in an image based on FPGA hardware logic;

[0098] The Euler number calculation module is used to calculate the Euler number of an image and distinguish defect types.

[0099] The area threshold analysis module is used to set an area threshold and identify size-related defects such as protrusions and depressions. The threshold is automatically adjusted.

[0100] The image registration module uses a template matching algorithm to ensure pixel-level alignment between the standard image and the image under test, with a registration error of ≤1 pixel.

[0101] The binarization module is used to perform image binarization by combining methods such as the maximum variance thresholding method, highlighting defect areas and facilitating subsequent analysis.

[0102] Deep learning module 23 is used to accelerate model computation, optimize parameters and detector head performance. Deep learning module 23 consists of a cascaded fusion network acceleration module, an efficient channel attention YOLO acceleration module, a YOLOX-like incremental learning acceleration module, a model parameter optimization module, a model compression and quantization module, a multimodal feature fusion acceleration module, and a detector head optimization acceleration module.

[0103] The cascaded fusion network is used to accelerate the module by optimizing hardware resource configuration, providing a high-speed computing environment for the cascaded fusion network, accelerating the computing process of the multimodal feature fusion network, reducing data transmission latency and improving the overall detection speed under the collaborative work of FPGA and NVIDIA Jetson AGX Orin module, and ensuring that the processing time of a single image meets the real-time detection requirements in high-speed production line scenarios.

[0104] The high-efficiency channel attention YOLO acceleration module is used to accelerate the high-efficiency channel attention ECA and feature reorganization module of high-efficiency channel attention YOLO in hardware. It utilizes the parallel computing capabilities of FPGA to quickly calculate channel weights, enhance the feature response of small target defects, and increase the model's attention to small targets by 30%, effectively improving the detection accuracy of small target defects.

[0105] The YOLOX-type incremental learning acceleration module adopts the YOLOX-type incremental detection model. During the incremental learning process, it accelerates the knowledge distillation calculation between the teacher model and the student model. It utilizes a hardware acceleration engine to quickly process the distillation loss calculation of output features and intermediate features, enabling the model to better retain the knowledge of old defect categories while learning new defect categories. In the two-stage incremental scenario, the average accuracy of the old categories remains above 88.5%, reducing model training time by 30%.

[0106] The model parameter optimization module is used to monitor the training process of the deep learning model in real time. Based on the changes in the loss function and the detection accuracy index, it dynamically adjusts the model's hyperparameters, such as the learning rate and weight decay coefficient. It adopts an adaptive optimization algorithm to automatically find the optimal parameter combination, improve the model's convergence speed and detection performance, and enable the model to maintain stable and efficient detection capabilities in different PCB production environments.

[0107] The model compression and quantization module is used to compress and quantize the trained deep learning model, reducing the model's storage space and computational load. It removes redundant connections and parameters through pruning techniques and replaces floating-point data with low-precision data formats. Without significantly reducing detection accuracy, it improves the model's operating efficiency on edge computing devices, reduces hardware costs, and meets the strict limitations of industrial equipment on storage and computing resources.

[0108] The multimodal feature fusion acceleration module accelerates the feature fusion process of multimodal data, optimizes data transmission and processing flow, and uses hardware acceleration modules to quickly realize the splicing and fusion operations of different modal features, improves the efficiency of multimodal feature fusion, enhances the model's ability to detect complex defects, and improves the overall detection accuracy by 5%.

[0109] The detection head optimization and acceleration module is used to optimize and accelerate the detection head of the deep learning model, improve the classification and localization accuracy of the detection head for different types of defects, realize the rapid calculation of convolutional layers and fully connected layers in the detection head through hardware acceleration, optimize the execution efficiency of the nonmaximum suppression algorithm, reduce false detections and false negatives, and improve the detection accuracy of the detection head for common defects to over 98%.

[0110] In this invention, a multimodal sensor array integrates various detection methods such as vision, 3D laser, and ultrasound to comprehensively collect data from surface details to internal structures. Combined with high-precision sensors and advanced algorithms, it can accurately detect minute defects such as microcracks down to 0.05mm, as well as various defects such as short circuits, open circuits, and voids, achieving high detection accuracy. Through an edge computing module, hardware acceleration and algorithm optimization enable rapid image preprocessing, accelerated traditional image processing, and efficient deep learning computation, significantly reducing the processing time for a single image. This meets the real-time detection needs of high-speed production lines, improving production efficiency. A light intensity monitoring module monitors lighting conditions in real time to ensure stable image acquisition; the light source control module automatically... The system adjusts lighting to adapt to different PCB board characteristics; the model parameter optimization module dynamically adjusts deep learning model parameters to ensure stable operation in different production environments, reducing false positives and false negatives; the model compression and quantization module reduces the size of the deep learning model, lowers hardware performance requirements, and reduces hardware costs; automated detection reduces manual intervention, avoids the subjectivity and instability of manual detection, and reduces labor costs and subsequent production losses caused by defective products; the YOLOX-like incremental learning acceleration module supports incremental model learning, and combined with the dynamic parameter optimization module, it can automatically optimize detection parameters and models based on historical data and real-time detection conditions, adapt to the detection of new defect types, and continuously improve detection capabilities.

[0111] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A smart online defect detection device for PCB boards based on multimodal sensor fusion and edge computing, characterized in that, It includes a multimodal module (1) and an edge computing module (2). The multimodal module (1) is used to collect data from the PCB board, and the edge computing module (2) is used to process and analyze the data. The multimodal module (1) includes a visual acquisition module (11), a light source control module (12), a light intensity monitoring module (13), a displacement detection module (14), and an ultrasonic detection module (15). The edge computing module (2) includes an image preprocessing module (21), a processing acceleration module (22), and a deep learning module (23). The visual acquisition module (11) is used to acquire images of the surface and internal heat distribution of the PCB board; The light source control module (12) is used to adjust the wavelength, intensity, and angle of the light and monitor its stability; The light intensity monitoring module (13) is used to monitor light intensity, calculate uniformity and alarm abnormal conditions; The displacement detection module (14) is used to acquire, preprocess, and visualize the three-dimensional topography data of the PCB surface; The ultrasonic testing module (15) is used to penetrate the PCB to detect internal defects and process signals and optimize parameters; The image preprocessing module (21) is used to enhance image contrast, denoise, edge detection, and silkscreen removal; The processing acceleration module (22) is used to detect short-circuit defects, calculate Euler number and area threshold; The deep learning module (23) is used to accelerate model computation, optimize parameters, and improve the performance of the detection head; The visual acquisition module (11) consists of an 8K line array camera group, a high-precision line scan CCD module, a positioning hole visual sensor, and a camera calibration and synchronization module; The light source control module (12) consists of a multi-color LED lighting array, a light intensity adjustment module, a light angle adjustment mechanism, and a light source stability detection module; The multi-color LED lighting array adopts an integrated design of multiple color LEDs, and combines wavelengths according to detection requirements; The light intensity adjustment module uses a microcontroller to precisely control the LED current, with adaptive adjustment from 0-50000 lux to suit different PCB reflective characteristics; The illumination angle adjustment mechanism uses an adjustable bracket to achieve multi-angle illumination, vertically inspecting overall defects, and tilting to highlight edge features; The light source stability monitoring module is used to monitor light source brightness fluctuations in real time and provide feedback for adjustment. The light intensity monitoring module (13) consists of a light intensity sensor array, a light intensity data processing module, and a light abnormality alarm module; The light intensity sensor array uses multiple sensors evenly distributed to monitor the ambient light intensity and the light intensity on the PCB surface in real time. The light intensity data processing module is used to analyze and process light intensity data, calculate uniformity, and control the light intensity adjustment module. The lighting anomaly alarm module is used to trigger an alarm and prompt the user to check the light source system when the light intensity or uniformity exceeds the set range.

2. The intelligent online defect detection device for PCB boards based on multimodal sensor fusion and edge computing according to claim 1, characterized in that: The 8K line array camera group uses 12 cameras to collect data in a coordinated manner. The visible light band captures the surface details of the PCB board, and the near-infrared band penetrates to detect internal heat distribution anomalies. The resolution is 1920×1080 and the frame rate is 30FPS. The high-precision scanning CCD module employs high pixel precision; The positioning hole vision sensor uses an industrial area array camera paired with a telecentric lens; The camera calibration and synchronization module uses periodic calibration of camera parameters.

3. The intelligent online defect detection device for PCB boards based on multimodal sensor fusion and edge computing according to claim 1, characterized in that: The displacement detection module (14) consists of a 3D laser displacement sensor, a 3D data acquisition and preprocessing module, and a 3D data visualization module; 3D laser displacement sensor: accuracy ±1μm, 1kHz scanning frequency to acquire three-dimensional surface morphology, and measure the height and shape of circuits and solder joints; The 3D data acquisition and preprocessing module is used to acquire data in real time, perform filtering and interpolation preprocessing, and improve data quality. The 3D data visualization module is used to transform 3D data into a visualization model, which can intuitively display the surface morphology of the PCB and help analyze defect characteristics.

4. The intelligent online defect detection device for PCB boards based on multimodal sensor fusion and edge computing according to claim 1, characterized in that: The ultrasonic testing module (15) consists of an air-coupled ultrasonic probe, an ultrasonic signal processing module, and an ultrasonic testing parameter adjustment module; The air-coupled ultrasonic probe uses a 50MHz frequency to penetrate the PCB and receive echoes to detect internal defects. The ultrasonic signal processing module is used to amplify, filter, and denoise echo signals, extract defect features, and transmit them to the analysis module. The ultrasonic testing parameter adjustment module is used to adjust the ultrasonic emission frequency and power according to the PCB thickness and material.

5. The intelligent online defect detection device for PCB boards based on multimodal sensor fusion and edge computing according to claim 1, characterized in that: The image preprocessing module (21) consists of a grayscale stretching module, a silkscreen processing module, an edge detection module, an image noise reduction module, and an image enhancement module; The grayscale stretching module uses an improved algorithm to dynamically calculate the mean grayscale value, enhance image contrast, and highlight defective areas. The silkscreen processing module is used to locate the background color in the histogram and remove the logo text; The edge detection module uses an 8-operator extended algorithm to detect edges in multiple directions and uses adaptive thresholds to extract complete edge information; The image noise reduction module uses median filtering and Gaussian filtering algorithms to remove image noise; The image enhancement module uses a multi-scale detail enhancement algorithm to highlight the texture of circuit and pad edges.

6. The intelligent online defect detection device for PCB boards based on multimodal sensor fusion and edge computing according to claim 1, characterized in that: The processing acceleration module (22) consists of a connectivity analysis module, an Euler number calculation module, an area threshold analysis module, an image registration module, and a binarization processing module; The connectivity analysis module is used to quickly calculate the number of connected regions in an image based on FPGA hardware logic; The Euler number calculation module is used to calculate the Euler number of an image and distinguish defect types. The area threshold analysis module is used to set area thresholds, identify size-related defects, and the thresholds are adaptively adjusted. The image registration module uses a template matching algorithm to ensure pixel-level alignment between the standard image and the image under test, with a registration error of ≤1 pixel. The binarization module is used to perform image binarization by combining the maximum variance thresholding method to highlight defect areas.

7. The intelligent online defect detection device for PCB boards based on multimodal sensor fusion and edge computing according to claim 1, characterized in that: The deep learning module (23) consists of a cascaded fusion network acceleration module, an efficient channel attention YOLO acceleration module, a YOLOX-type incremental learning acceleration module, a model parameter optimization module, a model compression and quantization module, a multimodal feature fusion acceleration module, and a detection head optimization acceleration module. The cascaded fusion network is used to accelerate the module by optimizing hardware resource configuration, providing a high-speed computing environment for the cascaded fusion network, and accelerating the computing process of the multimodal feature fusion network; The high-efficiency channel attention YOLO acceleration module is used to provide hardware acceleration for the high-efficiency channel attention and feature reconstruction modules of high-efficiency channel attention YOLO. The YOLOX-type incremental learning acceleration module uses a YOLOX-type incremental detection model to accelerate knowledge distillation calculations between the teacher model and the student model during the incremental learning process. The model parameter optimization module is used to monitor the training process of the deep learning model in real time and dynamically adjust the model's hyperparameters based on changes in the loss function and detection accuracy metrics. The model compression and quantization module is used to compress and quantize trained deep learning models, reducing the model's storage space and computational load. The multimodal feature fusion acceleration module accelerates the feature fusion process of multimodal data, optimizes data transmission and processing flow, and uses hardware acceleration modules to quickly realize the splicing and fusion operations of different modal features; The detection head optimization and acceleration module is used to optimize and accelerate the detection head of deep learning models.

Images