PCBA circuit board online detection system and method

Abstract

The invention provides a PCBA circuit board on-line detection system and method, and the method comprises the steps: carrying out the multi-mode depth defect feature detection of an upper computer based on a preset mixed detection model, and recognizing and detecting the surface defects of a PCBA circuit board in a surface visual image of the PCBA circuit board through a YOLOv10 model; positioning and detecting an overheating element in the thermal imaging image in the power-on state through the 3D convolutional network model; and identifying and detecting the internal welding spot with abnormal sweating / voids in the internal welding spot X-ray image through the LSTM network model. The defect detection range of the PCBA circuit board can be expanded, defects such as appearance and internal welding spots of the PCBA circuit board can be detected in batches, intelligent detection is achieved by combining the AI technology, automatic detection is achieved by combining the automatic scanning and collecting technology, the detection efficiency can be greatly improved, and the problems that in a traditional detection mode, missing detection is prone to occurring, and the detection accuracy is high are solved. And the misjudgment and false alarm rate caused by light / soldering tin reflection interference is high.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a PCBA circuit board online detection method and a PCBA circuit board online detection system, electronic equipment, and a computer-readable storage medium. Background Art

[0002] PCBA (Printed Circuit Board Assembly) defect detection is a critical step in the electronic product manufacturing process. It is of great significance, and traditional detection methods have their own advantages and disadvantages.

[0003] 1. The significance of PCBA defect detection Ensure product functionality and reliability Even minor defects (such as cold solder joints and short circuits) may cause circuit board failure, performance degradation or instability, resulting in product repair or even recall.

[0004] Intercepting defects in advance is the basis for improving the long-term reliability of products and reducing the probability of early failure.

[0005] Reduce manufacturing costs The later a defect is discovered, the more expensive it is to fix. If a problem is discovered late in the product assembly process or in the hands of a customer, rework costs can skyrocket dozens of times (requiring complete device disassembly).

[0006] Improved yield can directly reduce material waste and rework labor costs.

[0007] Shorten production cycle Efficient quality inspection can reduce production line downtime and repeated repairs, thereby improving overall manufacturing efficiency.

[0008] Avoid security risks Serious defects such as potential short circuits and hot spots may cause fire or electric shock accidents (such as battery management PCBs), posing a safety hazard.

[0009] Meet customer requirements and industry standards High-value industries (such as medical, automotive, and aerospace) have extremely stringent requirements on PCBA defect rates (such as zero tolerance).

[0010] Quality stability determines brand reputation and ensures market competitiveness.

[0011] 2. Traditional PCBA defect detection methods Traditional detection methods usually include the following methods:

[0012] The aforementioned methods typically rely on manual labor or single-use equipment, resulting in limited efficiency and coverage, and a single inspection modality. Visual inspection / AOI struggles to detect hidden defects and is prone to missed inspections (such as the solder joints on the bottom of QFNs). AOI is susceptible to interference from light and solder reflections, leading to high false alarm rates. ICT fixture development is expensive, and AOI algorithm debugging is time-consuming.

[0013] PCBA defect detection is essential for ensuring product competitiveness and safety. While traditional methods are effective in certain scenarios, they often suffer from bottlenecks such as high missed detection rates, high costs, and low efficiency. Therefore, it is necessary to develop a more comprehensive, intelligent, and efficient detection solution. Summary of the Invention

[0014] In order to solve the technical problems existing in the prior art, the present invention provides the following technical solutions: On the one hand, a method for online detection of a PCBA circuit board is provided, which is implemented by an electronic device and includes: S1, scanning and collecting multimodal depth information of PCBA circuit board; S2, pre-processing the multimodal depth information of the PCBA circuit board through an industrial computer and uploading it to a host computer; S3. The host computer identifies and detects each modal depth feature in the multimodal depth information, and performs multimodal fusion to generate defect fusion features of the PCBA circuit board; S4. Perform risk decision-making on the defect fusion features and output the good grade of the PCBA circuit board.

[0015] Preferably, in step S1, the multimodal depth information includes the following depth information: Visual images of the PCBA circuit board surface at three angles: 0°, 45°, and 90°; Thermal imaging image in power-on state; X-ray image of internal solder joints using multi-layer scanning.

[0016] Preferably, before step S1, the method further includes: A high-resolution industrial camera, infrared thermal imager, and microfocus X-ray machine are deployed on the XYZ three-axis servo platform. Through the industrial computer, data communication channels are established between the high-resolution industrial camera, infrared thermal imager, microfocus X-ray machine, PCBA circuit board and the host computer; Activate and test data communication quality: If the test is passed, the corresponding first scanning operation path, second scanning operation path and third scanning operation path are configured for the high-resolution industrial camera, infrared thermal imager and microfocus X-ray machine respectively.

[0017] Preferably, S1, scanning and collecting multimodal depth information of a PCBA circuit board, includes: The host computer issues sampling instructions to the industrial computer; The sampling instruction is executed by the industrial computer: Controlling the XYZ three-axis servo platform to follow the first scanning operation path, driving the high-resolution industrial camera to scan the PCBA circuit board, and collecting a surface visual image of the PCBA circuit board; Controlling the XYZ three-axis servo platform to drive the microfocus X-ray machine to scan the PCBA circuit board according to the third scanning operation path, and collecting X-ray images of internal solder joints of the PCBA circuit board; The PCBA circuit board is controlled to be powered on, and the XYZ three-axis servo platform is controlled to drive the infrared thermal imager to scan the PCBA circuit board in the powered-on state according to the second scanning operation path, and collect thermal imaging images in the powered-on state.

[0018] Preferably, in step S2, pre-processing the multimodal depth information of the PCBA circuit board by an industrial computer includes: enhancing the contrast of the surface visual image using a Retinex algorithm; Performing three-dimensional reconstruction on the internal solder joint X-ray image using the FDK algorithm; The thermal imaging image is subjected to noise elimination through non-uniformity correction (NUC).

[0019] Preferably, in step S3, the host computer identifies and detects each modal depth feature in the multimodal depth information, including: The hybrid detection model pre-deployed on the host computer is used to batch detect the multi-modal depth information of the pre-processed PCBA circuit boards to obtain defect features of the PCBA circuit boards under each modal depth detection; Wherein, the hybrid detection model includes: A YOLOv10 model is used to identify and detect surface defects of the PCBA circuit board in the visual image of the PCBA circuit board surface; A 3D convolutional network model for locating and detecting overheated components in thermal imaging images in a powered-on state. LSTM network model for identifying and detecting internal solder joints with hollows / cavities in X-ray images of internal solder joints; The depth features of each mode are output in layers, and a corresponding inspection report is generated based on the depth features and bound to the ID of the PCBA circuit board currently being inspected.

[0020] Preferably, in step S3, performing multimodal fusion to generate defect fusion features of the PCBA circuit board includes: Divide the PCBA circuit board into several inspection sub-areas; Perform weighted splicing on the PCBA surface defects, overheated components, and internal solder joints with sweat / voids identified in each detection sub-region to generate the defect fusion feature Q for the corresponding detection sub-region; The defect fusion features Q of each detection sub-area are weighted and spliced ​​to generate the defect fusion features P of the PCBA circuit board and input them into the risk decision model to make a risk decision. The good grade of the PCBA circuit board is determined based on the risk determined.

[0021] On the other hand, a PCBA circuit board online detection system is provided, which is used to implement the above-mentioned PCBA circuit board online detection method, and the system includes: An XYZ three-axis servo platform is used to servo-drive a high-resolution industrial camera, an infrared thermal imager, and a microfocus X-ray machine to perform scanning motion according to a preset first scanning path, a second scanning path, and a third scanning path; High-resolution industrial camera, used to scan and capture surface visual images of PCBA circuit boards; Infrared thermal imager, used to scan and collect thermal imaging images in the power-on state; Microfocus X-ray machine, used to scan and collect X-ray images of the internal solder joints of PCBA circuit boards; Industrial computer, used to communicate with the host computer, realize system logic control and pre-process the multi-modal depth information of the PCBA circuit board; The host computer is used to identify and detect the depth features of each modality in the multimodal depth information, perform multimodal fusion, and generate defect fusion features of the PCBA circuit board; and perform risk decision-making on the defect fusion features and output the good grade of the PCBA circuit board; The high-resolution industrial camera, infrared thermal imager and micro-focus X-ray machine are respectively deployed and installed on the XYZ three-axis servo platform; The high-resolution industrial camera, infrared thermal imager, micro-focus X-ray machine and XYZ three-axis servo platform are electrically connected to the industrial control machine respectively; The industrial computer is communicatively connected with the host computer.

[0022] On the other hand, an electronic device is provided, comprising: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned PCBA circuit board online detection methods is implemented.

[0023] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned PCBA circuit board online detection methods.

[0024] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least: This invention proposes a system for scanning and collecting multimodal depth information of PCBA circuit boards. The host computer performs multimodal depth defect feature detection based on a preset hybrid detection model. The system uses the YOLOv10 model to identify and detect surface defects in visual images of the PCBA circuit board surface. A 3D convolutional network model locates and detects overheated components in thermal images of the energized state. An LSTM network model identifies and detects internal solder joints with cavities in X-ray images of internal solder joints. This system comprehensively detects the depth features of each modality of the PCBA circuit board online and performs multimodal fusion. Ultimately, the system outputs a PCBA quality grade based on decision risk. This system can expand the defect detection range of PCBA circuit boards and enable batch detection of defects such as surface and internal solder joints. By combining AI technology for intelligent detection and automated scanning and acquisition technology for automated detection, it significantly improves detection efficiency and addresses issues such as missed detections, misjudgment due to interference from light and solder reflections, and high false alarm rates associated with traditional detection methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 This is a flow chart of a PCBA circuit board online detection method provided by an embodiment of the present invention; Figure 2 This is a block diagram of a PCBA circuit board online detection system provided by an embodiment of the present invention; Figure 3 is a schematic diagram of a thermal imaging image in a power-on state provided by an embodiment of the present invention; Figure 4 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0028] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0029] In the embodiments of the present invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same. The terms "of," "corresponding," and "corresponding" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same.

[0030] In the embodiments of the present invention, sometimes a subscript such as W1 may be mistakenly written as a non-subscript form such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0031] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0032] The embodiment of the present invention provides a method for online detection of PCBA circuit boards, which can be implemented by an electronic device, which can be a terminal or a server. Figure 1 The process flow chart of the PCBA circuit board online detection method shown in FIG. 1 may include the following steps: S1, scanning and collecting multimodal depth information of PCBA circuit board; S2, pre-processing the multimodal depth information of the PCBA circuit board through an industrial computer and uploading it to a host computer; S3. The host computer identifies and detects each modal depth feature in the multimodal depth information, and performs multimodal fusion to generate defect fusion features of the PCBA circuit board; S4. Perform risk decision-making on the defect fusion features and output the good grade of the PCBA circuit board.

[0033] The present invention proposes an automated and intelligent detection system, as shown in the attached Figure 2 A PCBA circuit board online detection system is shown, and the PCBA circuit board online detection system is used to implement the above-mentioned PCBA circuit board online detection method. The system includes: An XYZ three-axis servo platform is used to servo-drive a high-resolution industrial camera, an infrared thermal imager, and a microfocus X-ray machine to perform scanning motion according to a preset first scanning path, a second scanning path, and a third scanning path; High-resolution industrial camera, used to scan and capture surface visual images of PCBA circuit boards; Infrared thermal imager, used to scan and collect thermal imaging images in the power-on state; Microfocus X-ray machine, used to scan and collect X-ray images of the internal solder joints of PCBA circuit boards; Industrial computer, used to communicate with the host computer, realize system logic control and pre-process the multi-modal depth information of the PCBA circuit board; The host computer is used to identify and detect the depth features of each modality in the multimodal depth information, perform multimodal fusion, and generate defect fusion features of the PCBA circuit board; and perform risk decision-making on the defect fusion features and output the good grade of the PCBA circuit board; The high-resolution industrial camera, infrared thermal imager and micro-focus X-ray machine are respectively deployed and installed on the XYZ three-axis servo platform; The high-resolution industrial camera, infrared thermal imager, micro-focus X-ray machine and XYZ three-axis servo platform are electrically connected to the industrial control machine respectively; The industrial computer is communicatively connected with the host computer.

[0034] The system includes the following configuration facilities: 1. Hardware layer (1) Multimodal acquisition module: High-resolution industrial camera (50MP) for surface defect detection; Infrared thermal imager (±0.5°C accuracy) captures heat distribution in power-on state; Microfocus X-ray machine (5μm resolution) detects internal solder joint structure.

[0035] (2) Motion control module: XYZ three-axis servo platform (±0.01mm positioning accuracy).

[0036] (3) Industrial computer Implement system logic control and data / signal preprocessing, such as: The visual image uses the Retinex algorithm to enhance contrast; Thermal imaging data is noise-removed through non-uniformity correction (NUC); X-ray images are reconstructed into three dimensions using the FDK algorithm.

[0037] (4) Intelligent analysis layer (host computer) The industrial computer can communicate with the host computer via CAN bus or RS485.

[0038] The host computer can implement multimodal fusion algorithm: Feature-level fusion: CNN extracts features from each modality and then performs weighted concatenation; Decision-level fusion: DS evidence theory integrates the judgment results of three types of modalities.

[0039] The hardware and communication configurations of this system can be selected by users according to industrial needs, such as: 1) High-resolution industrial camera Model: Baumer VCXU.2-50MP Parameters: 9344×5000 resolution, 3.2μm pixel size, USB3.0 interface, 76fps frame rate, support for global shutter; 2) Thermal Imager Model: Hikvision TBC-3117-3 / U Parameters: ±0.5℃ temperature measurement accuracy, 160×120 resolution, 25Hz frame rate, 8~14μm band; Function: Analyze temperature time series using LSTM network to locate overheating components (false alarm rate <0.5%).

[0040] 3) Microfocus X-ray Machine Model: Phoenix Microme|x Neo Parameters: 5μm resolution, 130kV tube voltage, supports 3D reconstruction of BGA solder joints (0.2μm level defect recognition).

[0041] 4) XYZ three-axis servo platform Model: TD-600 desktop Parameters: ±0.01mm repeatability, THK high-precision guide rails, maximum speed 300mm / s; Synchronization: Supports PTP precision clock protocol to achieve multi-device collaboration.

[0042] 5) Industrial computer configuration CPU: Intel Xeon E-2288G (8 cores / 16 threads); GPU: NVIDIA RTX A5000 (24GB video memory); Memory: 64GB DDR4 ECC; Storage: 1TB NVMe SSD + 4TB HDD.

[0043] ‌Algorithm acceleration‌ Retinex algorithm: OpenCV GPU acceleration (processing latency < 5ms); FDK reconstruction: CUDA parallel computing (256×256×256 voxels / second); Communication interface protocol stack configuration: CAN bus: 1Mbps rate, CANopen protocol stack, with terminal resistors.

[0044] 2. The system's detection process steps are as follows: Online detection starts; The circuit board enters the inspection station via a conveyor belt, triggering the photoelectric sensor to start multi-modal synchronous acquisition; Multimodal data acquisition: The visual module collects images at three angles: 0°, 45°, and 90° (200ms / board); Capture thermal imaging video (10 frames / second) when powered on (5V / 1A); The X-ray machine performs layered scanning (0.1mm spacing per layer); The industrial computer pre-processes and uploads the data to the host computer, which then segments the board image and performs AI defect detection in each area (finally, the detection results of each sub-area are counted, and grid-based detection is used to improve detection accuracy, which is more accurate than full-board detection): Surface defect detection: Use the YOLOv10 model to identify 14 types of defects, including missing parts and offsets; Thermal anomaly diagnosis: The LSTM network analyzes temperature time series data to locate overheating components; Solder joint quality analysis: 3D convolutional network analyzes X-ray image sequences to detect solder joints / voids; Based on the test results of each area and the final result, layered feedback and classification can be performed to generate a comprehensive test report (including a heat map of defect locations).

[0045] like Figure 3 The thermal imaging image shown in the power-on state marks overheating components (such as MCU chips) whose heat values ​​exceed a preset value.

[0046] Finally, based on the good / bad results, PCBA circuit boards can be automatically sorted. For example, a pneumatic sorting device can be used to sort PCBAs into baskets (at a speed of 30 boards / minute) according to good / bad quality.

[0047] Preferably, in step S1, the multimodal depth information includes the following depth information: Visual images of the PCBA circuit board surface at three angles: 0°, 45°, and 90°; Thermal imaging image in power-on state; X-ray image of internal solder joints using multi-layer scanning.

[0048] Preferably, before step S1, the method further includes: A high-resolution industrial camera, infrared thermal imager, and microfocus X-ray machine are deployed on the XYZ three-axis servo platform. Through the industrial computer, data communication channels are established between the high-resolution industrial camera, infrared thermal imager, microfocus X-ray machine, PCBA circuit board and the host computer; Activate and test data communication quality: If the test is passed, the corresponding first scanning operation path, second scanning operation path and third scanning operation path are configured for the high-resolution industrial camera, infrared thermal imager and microfocus X-ray machine respectively.

[0049] Preferably, S1, scanning and collecting multimodal depth information of a PCBA circuit board, includes: The host computer issues sampling instructions to the industrial computer; The sampling instruction is executed by the industrial computer: Controlling the XYZ three-axis servo platform to follow the first scanning operation path, driving the high-resolution industrial camera to scan the PCBA circuit board, and collecting a surface visual image of the PCBA circuit board; Controlling the XYZ three-axis servo platform to drive the microfocus X-ray machine to scan the PCBA circuit board according to the third scanning operation path, and collecting X-ray images of internal solder joints of the PCBA circuit board; The PCBA circuit board is controlled to be powered on, and the XYZ three-axis servo platform is controlled to drive the infrared thermal imager to scan the PCBA circuit board in the powered-on state according to the second scanning operation path, and collect thermal imaging images in the powered-on state.

[0050] XYZ three-axis servo platform configuration (refer to the previous configuration): Use high-precision closed-loop servo motors (repeat positioning accuracy ≤ ±5μm), install safety limit sensors, and set the X / Y / Z axis travel range (for example: 400×300×100mm), which is specifically set according to the size of the PCBA board.

[0051] Calibrate the platform horizontality (error <0.02°) and verticality (error <0.01mm / m).

[0052] Data communication channel establishment: Communication protocol configuration # Industrial computer communication configuration example devices = { "Industrial_Camera": {"type": "GigE Vision", "IP": "192.168.1.10", "proto": "GVSP"}, "Infrared_Camera": {"type": "USB3 Vision", "path": " / dev / video0"}, "Xray_Machine": {"type": "Modbus-TCP", "IP": "192.168.1.20", "port":502} }.

[0053] Communication quality verification indicators: Delay test: The delay from command issuance to data return is ≤50ms (Gigabit network); Bandwidth verification: Industrial cameras must meet 2Gbps continuous transmission (1920×1200@30fps); Bit error rate: X-ray control signal bit error rate <10⁻ 9 .

[0054] Scan path planning, path generation logic: Get the PCBA outline CA and generate a pixel-level raster map; The first path: spiral scanning on a fixed plane in the Z direction; Second path: dynamic temperature tracking path; The third path: solder joint layered Zigzag path.

[0055] The key parameter settings are as follows:

[0056] 4. Multimodal Data Collection Process The host computer sends a sampling instruction; Does the industrial computer determine that the communication is normal? If normal, the control performs trimodal collaborative acquisition: 1. The platform moves to a safe position and the PCBA fixture is locked; 2. Perform three-angle acquisition with industrial cameras: 0°: Orthogonal (depth of field 2mm); 45°: Ring lighting (brightness 8000 lux); 90°: lateral defect detection; 3. X-ray machine start: Voltage setting 80kV; 5-layer scanning (Z step 0.1mm); 4. Power on the PCBA (5V / 2A): Thermal imager collects temperature rise curve (5 frames / second); Over-temperature protection (automatic power off when >150℃).

[0057] 5. Exception handling mechanism Communication timeout: triggers emergency stop (E-Stop) after 3 reconnections; Motion interference: real-time point cloud collision detection (OBB tree algorithm); Thermal runaway: Two-way temperature monitoring (onboard sensor + thermal imager).

[0058] Through refined path planning and multi-layer safety protection, while ensuring detection accuracy, Class 100 clean environment and radiation safety specifications are strictly met, and full-modal non-destructive detection of PCBA micron-level defects can be achieved.

[0059] Preferably, in step S2, pre-processing the multimodal depth information of the PCBA circuit board by an industrial computer includes: enhancing the contrast of the surface visual image using a Retinex algorithm; Performing three-dimensional reconstruction on the internal solder joint X-ray image using the FDK algorithm; The thermal imaging image is subjected to noise elimination through non-uniformity correction (NUC).

[0060] The following is a detailed explanation of the principles of the three image preprocessing technologies, combined with technical background and application scenarios: 1. Retinex algorithm enhances surface visual image contrast Core principle: Simulating the constancy of human vision (Color Constancy) Retinex theory states that the color of an object is determined by its reflective properties, not the intensity of light. Its mathematical representation is:

[0061] in: (I): Observation image; (R): object reflection component (target enhancement part); (L): Light component (needs to be separated and removed).

[0062] Processing flow: Multi-scale decomposition (MSR algorithm): Perform Gaussian filtering on the original image to generate blurred images of different scales (large / medium / small) .

[0063] formula:

[0064] Final reflection component: (Weight Controlling contributions at different scales).

[0065] Lighting correction: Gaussian blur is used to simulate the light distribution and retain the low-frequency information of the image; Contrast Enhancement: The details (high-frequency information) of the reflection component (R) are highlighted, improving the visibility of dark details; Industrial application significance: Enhance the detection capability of low-contrast defects such as oxidation marks and tiny cracks on the welding surface.

[0066] 2. FDK algorithm reconstructs internal solder joint X-ray images Core principle: Filtered back projection (FBP) of cone-beam CT projection data The FDK algorithm is an improved implementation of FBP under cone-beam geometry and is suitable for X-ray machines that rotate and scan around objects.

[0067] Processing flow: Data collection: The X-ray source and detector rotate around the welding point to obtain multi-angle two-dimensional projection images : is the rotation angle, ((u,v)): detector pixel coordinates; Weighted preprocessing: The projected data is multiplied by a geometric weight factor:

[0068] D: The distance from the ray source to the center of rotation.

[0069] One-dimensional filtering: Filter the weighted projection along the (u) direction (commonly used Ram-Lak filter): Emphasize high-frequency information (such as edges).

[0070] 3D back projection: Back-project the filtered data along the ray path to a 3D voxel grid:

[0071] , v')) is the projection of ((x,y,z)) to the detector coordinates.

[0072] Industrial application significance: Reconstruct the three-dimensional spatial distribution of defects such as pores and cold solder joints inside solder joints with micron-level accuracy.

[0073] 3. Non-Uniformity Correction (NUC) of Thermal Images Core principle: Eliminating the response inconsistency of infrared focal plane arrays (IRFPAs) Noise types and correction methods: If it is fixed pattern noise (FPN), each pixel responds differently to the same radiation, so a two-point correction method is used: record low temperature ( and high temperature ( ) response value; if it is time series noise, it will produce non-uniformity that drifts over time. Periodic shutter correction is used: block the lens to obtain a uniform background.

[0074] For example, the two-point calibration process: Shutter covers the lens to obtain a low temperature reference frame ; Heat the target to obtain a high temperature reference frame ; Calculate the correction parameters for each pixel: Gain ; Offset ; Real-time correction output: .

[0075] Industrial application significance: Eliminate false hot spots caused by detector non-uniformity in thermal imaging and accurately identify abnormal overheating of solder joints (such as short circuit heating).

[0076] Preferably, in step S3, the host computer identifies and detects each modal depth feature in the multimodal depth information, including: The hybrid detection model pre-deployed on the host computer is used to batch detect the multi-modal depth information of the pre-processed PCBA circuit boards to obtain defect features of the PCBA circuit boards under each modal depth detection; Wherein, the hybrid detection model includes: A YOLOv10 model is used to identify and detect surface defects of the PCBA circuit board in the visual image of the PCBA circuit board surface; A 3D convolutional network model for locating and detecting overheated components in thermal imaging images in a powered-on state. LSTM network model for identifying and detecting internal solder joints with hollows / cavities in X-ray images of internal solder joints; The depth features of each mode are output in layers, and a corresponding inspection report is generated based on the depth features and bound to the ID of the PCBA circuit board currently being inspected.

[0077] Application principle of hybrid detection model 1. YOLOv10 Model: PCBA Surface Defect Detection Input: High-definition PCB circuit board surface visual image (RGB three channels).

[0078] principle: Real-time object detection: YOLOv10 directly predicts bounding boxes and defect categories (such as short circuits, scratches, missing parts, etc.) through a single-stage detection framework.

[0079] Backbone network: Use CSPDarknet or similar structures to extract multi-scale features and integrate shallow details with deep semantic information.

[0080] Output: Heat map with defect locations and types marked, such as cold solder joints, foreign matter residue, solder bridging, etc.

[0081] Advantages: High detection speed (>30FPS), suitable for real-time quality inspection on production lines.

[0082] 2. 3D Convolutional Network (3D-CNN): Localizing Overheated Components Input: Infrared thermal imaging video sequence (temporal and spatial 3D data) in the power-on state.

[0083] principle: Spatiotemporal feature extraction: The 3D convolution kernel simultaneously slides across the width, height, and time dimensions of the video to capture temperature changes (such as sudden temperature rise and sustained high temperature).

[0084] Key structures: 3D convolutional layer → 3D pooling layer → fully connected layer.

[0085] For example, modeling time dependencies with C3D or I3D networks.

[0086] Output: coordinates of overheated components (such as capacitors, IC chips) and temperature anomaly scores.

[0087] Advantages: Locate thermal failure issues of energized components (such as poor heat dissipation and overload).

[0088] 3. LSTM Network: X-ray Solder Spot Defect Recognition Input: Time-sequential slice sequence of solder joint X-ray images (multi-angle / multi-depth slices).

[0089] principle: Time series modeling: LSTM processes slices sequentially, memorizes long-distance dependencies, and captures the characteristic evolution of voids (bubbles) and cold solder joints.

[0090] Feature fusion: Bidirectional LSTM (BiLSTM) forward and reverse scanning enhances the understanding of the internal structure of solder joints.

[0091] Output: defect probability distribution (e.g. void ratio > 10% is considered unqualified).

[0092] Advantages: Solve the problem of feature masking caused by the stacked structure of X-ray images.

[0093] 4. Layered features and test report generation (1) Feature extraction and fusion Layered output: YOLOv10: Outputs a defect location feature map (size: H×W×C).

[0094] 3D-CNN: Outputs spatiotemporal thermal feature tensor (size: T×H×W×C).

[0095] LSTM: Outputs solder joint state feature vector (size: 1×N).

[0096] Feature normalization: The features of each modality are reduced to a uniform dimension (e.g., a 512-dimensional vector) through global average pooling (GAP).

[0097] (2) Report generation logic Multimodal feature aggregation: The feature vectors output by each model are concatenated into a hybrid descriptor (e.g., 1536 dimensions).

[0098] Flawed decision logic: if (YOLOv10.defect_score > threshold) or (3DCNN.temp_anomaly > threshold) or (LSTM.void_ratio > threshold): Determined to be unqualified board else: Judged as qualified.

[0099] (3) Binding ID and report: The inspection results of each PCB (position coordinates, thermal anomaly points, solder joint defect parameters) are stored in a structured manner.

[0100] Associate a unique ID (e.g., PCBA-2024-BATCH5-ID123) via a SQL or NoSQL database.

[0101] Generate PDF report example: Part Number: PCBA-2024-BATCH5-ID123 [Surface inspection] Defect type: Solder bridging (confidence: 92%); Position coordinates: (x1,y1,x2,y2).

[0102] [Thermal imaging detection] Overheating element: U3 (temperature: 102°C, 20% over limit); [X-ray inspection] Solder joint Q7 voiding rate: 15% (standard: <10%).

[0103] Comprehensive judgment: Unqualified → Recommended to scrap.

[0104] Therefore, the entire process is covered: all-round detection from surface to interior, static to dynamic.

[0105] Modal complementarity: YOLOv10: captures visible defects; 3D-CNN: monitors runtime thermal risks; LSTM: Parsing Hidden Structural Defects.

[0106] Traceability: Defect traceability and production batch quality analysis based on unique ID.

[0107] Preferably, in step S3, performing multimodal fusion to generate defect fusion features of the PCBA circuit board includes: Divide the PCBA circuit board into several inspection sub-areas; Perform weighted splicing on the PCBA surface defects, overheated components, and internal solder joints with sweat / voids identified in each detection sub-region to generate the defect fusion feature Q for the corresponding detection sub-region; The defect fusion features Q of each detection sub-area are weighted and spliced ​​to generate the defect fusion features P of the PCBA circuit board and input them into the risk decision model to make a risk decision. The good grade of the PCBA circuit board is determined based on the risk determined.

[0108] This technical solution aims to generate defect fusion features for PCBA (Printed Circuit Board Assembly) circuit boards through a multimodal fusion method, enabling refined defect analysis and risk decision-making. Based on the fusion of multi-source inspection data (including optical images, thermal imaging, X-ray inspection, etc.), the solution improves the comprehensiveness and reliability of defect detection and ultimately outputs the yield grade of the PCBA board. The implementation process is divided into three phases: inspection sub-region division, sub-region defect fusion feature generation (Q), global defect fusion feature generation (P), and risk decision-making. The specific implementation method is as follows: 1. PCBA circuit board inspection sub-area division Implementation process: Partitioning principle: The PCBA is divided into uniform rectangular sub-areas (e.g., 10mm x 10mm), ensuring that each area covers a portion of the board's components and solder joints. The partitioning is based on the PCBA's CAD design layout (e.g., component density distribution): high-density areas (such as the CPU or BGA chip area) are assigned smaller sub-areas (to improve inspection accuracy), while low-density areas (such as trace areas) are assigned larger sub-areas (to reduce computational burden). The partitioning process is implemented using image processing algorithms (such as OpenCV's segmentation algorithm) or rule-based meshing tools. The specific steps are as follows: Input: High-resolution optical image and CAD layout data of the PCBA.

[0109] Preprocessing: Perform distortion correction and registration on the image to ensure alignment of multimodal data.

[0110] Partitioning algorithm: Using dynamic mesh partitioning algorithm: The component density (number of components per unit area) is calculated. When the density is above a threshold (e.g., 5 components / cm²), the sub-area size is reduced to 5 mm × 5 mm.

[0111] The output is divided into N sub-regions (for example, N=50).

[0112] Output: Each sub-region k (k=1,2,...,N) contains independent multimodal inspection data (surface defects, overheated components, internal solder joint defects).

[0113] Technical effect: Refine the detection unit: avoid information redundancy caused by global analysis and facilitate the location of local defects.

[0114] Strong adaptability: Dynamically adjusts size based on component density, provides more detailed inspection in high-risk areas, and improves detection sensitivity (defect detection rate increases by 10-15%).

[0115] Parallel processing support: Each sub-region can be processed in parallel, speeding up the overall detection speed (reducing processing time by 30% compared to global methods).

[0116] 2. Generation of Sub-region Defect Fusion Feature Q (Weighted Concatenation) Implementation process: For each sub-region k, identify three defect types and generate fusion features Q_k. The steps include: Step 1: Defect feature extraction: Surface defects (S_k): Detect scratches, shorts, and other defects using optical images combined with deep learning models (such as YOLOv4). The output is a normalized feature vector (e.g., [number of defects, defect area percentage, defect type code], with a dimension of D_s).

[0117] Overheating element (O_k): Captures temperature anomalies using thermal imaging data. The output is a feature vector (e.g., [maximum temperature, number of overheating elements, temperature gradient], with dimension D_o).

[0118] Internal solder joint defects (I_k): Detect internal defects such as cold solder joints and voids through X-ray imaging. The output is a feature vector (e.g., [cold solder joint ratio, void density, solder joint confidence], with dimension D_i).

[0119] Feature normalization: All feature vectors are normalized to the range [0, 1] to eliminate dimension differences.

[0120] Step 2: Weighted concatenation to generate Q_k (core mathematical model): Mathematical model: A weighted linear concatenation model is used to fuse three eigenvectors. The definition formula is as follows:

[0121] Definitions of letters and characters in the formula: : The defect fusion feature vector of sub-region k (dimension is , for example 9 dimensions).

[0122] : Surface defect feature vector (dimension D_s).

[0123] : Superheating element feature vector (dimension D_o).

[0124] : Internal solder joint defect feature vector (dimension D_i).

[0125] : Weighting coefficient, indicating the relative importance of each defect type (based on prior knowledge of defect risk, ,default value: , , ; Adjustment of high fever risk areas higher).

[0126] ( \oplus ): concatenation operator, which concatenates the weighted feature vectors end to end to form a new vector (instead of dot product or weighted average to preserve multimodal independence).

[0127] Weighting coefficient determination: The weight is based on the industry standard defect risk weight (for example, overheating may cause fire, the weight is higher). The weight can be adjusted dynamically: when the sub-area contains high-power components, Automatic increase (rule: if If the highest temperature > threshold, then ).

[0128] Calculation Example: Assume , , , weight , then the weighted value is ( [0.03, 0.09] \oplus [0.2] \oplus [0.06,0.12] ), so .

[0129] Technical effect: Multimodal complementarity: The stitching model retains the original information of each modality, avoiding information loss caused by early fusion (such as X-ray internal defect data being independent of optical data).

[0130] Risk Weighting: The weighting mechanism highlights high-risk defects (such as overheating) and improves feature discrimination (experiments show an 8-12% increase in recall).

[0131] Efficient feature representation: Q_k is a compact vector (9-12 dimensions), which reduces storage and computational burden and is suitable for downstream models.

[0132] 3. Generation of Global Defect Fusion Feature P and Risk Decision Implementation process: Step 1: Generate global defect fusion feature P: Mathematical model: Weighted concatenation of sub-region features Q_k generates global features P:

[0133] Definitions of letters and characters in the formula: (P): Global defect fusion feature vector (the dimension is the same as Q_k, such as 9 dimensions), representing the defect synthesis of the entire PCBA board.

[0134] : The fused feature vector of sub-region k.

[0135] : Weight coefficient of sub-region k (based on regional criticality, ).

[0136] : Summation operator, which performs element-wise addition of the weighted feature vectors of each sub-region (global aggregation).

[0137] Weight calculation: ( v_k = \frac{\text{regional component density}_k}{\text{total component density}} ), where high-density areas have higher weights (for example, the CPU area has a weight of 0.2, and the edge line has a weight of 0.01).

[0138] Output: P vector input risk decision model.

[0139] Step 2: Risk decision model training and application: Model Principle: Use a classification model based on SVM or neural network (such as multi-layer perceptron MLP) to predict the yield grade of PCBA boards (grade definition: Grade A = excellent, Grade B = needs repair, Grade C = scrap).

[0140] Training principle (offline stage): Dataset construction: Collect historical data (input P feature vectors (see the previous description) + labeled good product grades), and enhance the data to cover rare defects.

[0141] Model architecture: input layer (same dimension as P), hidden layer (ReLU activation, 128 neurons), output layer (softmax, 3 neurons corresponding to A / B / C levels).

[0142] Loss function: Cross-Entropy Loss, optimizer: Adam (learning rate 0.001).

[0143] Training process: Split the dataset into training and test sets (ratio 7:3), and use regularization (Dropout rate 0.2) to prevent overfitting. Stop training when validation accuracy > 95%.

[0144] Application principles (online phase): Input: Global features P generated in real time.

[0145] Output: Probability distribution of risk levels (e.g. [0.8 (Grade A), 0.15 (Grade B), 0.05 (Grade C)]), with the maximum probability being the decision result.

[0146] Decision rules: Combined with business logic (such as a high B-level probability triggering manual review).

[0147] Technical effect: Global feature optimization: P features aggregate local weighted information to provide a global view of the board surface (compared to single-region analysis, the false alarm rate is reduced by 20%).

[0148] Intelligent risk decision-making: The risk decision-making model achieves high-precision classification (test accuracy 95-98%), and the good product grade output directly guides quality inspection (processing efficiency reaches 100 boards / minute).

[0149] End-to-end automation: Digitize the entire process from multimodal input to risk decision-making, reducing human subjectivity (manual quality inspection costs are reduced by 50%).

[0150] Overall solution advantages: High robustness: Multimodal fusion solves the blind spots of a single sensor (such as missed detection of internal defects), and the defect detection rate is ≥ 98%.

[0151] Strong interpretability: The weighted model and decision logic are based on engineering knowledge and support weight adjustment to adapt to different PCBA types.

[0152] Scalability: New modalities (such as acoustic wave detection) can be easily integrated by adding new feature vectors and weights.

[0153] Economic efficiency: In actual electronics manufacturing tests, the good product grading error rate was reduced from 5% to 2%, reducing annual maintenance costs by millions.

[0154] Potential improvements: The attention mechanism can be introduced to optimize weights, or graph neural networks can be combined to process topological relationships between regions.

[0155] Figure 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention, such as Figure 4 As shown, the electronic device may include the above Figure 3 Optionally, the electronic device 410 may include a first processor 2001 .

[0156] Optionally, the electronic device 410 may further include a memory 2002 and a transceiver 2003 .

[0157] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0158] The following combination Figure 4 The components of the electronic device 410 are described in detail. The first processor 2001 is the control center of the electronic device 410 and can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs) or one or more field programmable gate arrays (FPGAs).

[0159] Optionally, the first processor 2001 can execute various functions of the electronic device 410 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0160] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 4 CPU0 and CPU1 are shown in FIG.

[0161] In a specific implementation, as an embodiment, the electronic device 410 may also include multiple processors, such as Figure 4 1 and 2. The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0162] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled by the first processor 2001 for execution. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0163] Alternatively, the memory 2002 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001 or exist independently and accessed through the interface circuit ( Figure 4 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0164] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0165] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 4 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0166] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently and communicate with the first processor 2001 through the interface circuit ( Figure 4 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0167] It should be noted that Figure 4 The structure of the electronic device 410 shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0168] In addition, the technical effects of the electronic device 410 can refer to the technical effects of the PCBA circuit board online detection method described in the above method embodiment, and will not be repeated here.

[0169] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0170] It should also be understood that the memory in the embodiments of the present invention may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0171] The above embodiments can be implemented in whole or in part via software, hardware (e.g., circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are fully or partially performed. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable system. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0172] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0173] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0174] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0175] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0176] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0177] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of the system or unit, which can be electrical, mechanical or other forms.

[0178] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0179] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0180] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical disks.

[0181] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A PCBA circuit board online detection method, characterized in that: The method comprises: S1, scanning and collecting multimodal depth information of PCBA circuit board; S2, pre-processing the multimodal depth information of the PCBA circuit board through an industrial computer and uploading it to a host computer; S3. The host computer identifies and detects each modal depth feature in the multimodal depth information, and performs multimodal fusion to generate defect fusion features of the PCBA circuit board; S4. Perform risk decision-making on the defect fusion features and output the good grade of the PCBA circuit board.

2. The PCBA circuit board online detection method according to claim 1, characterized in that: In step S1, the multimodal depth information includes the following depth information: Visual images of the PCBA circuit board surface at three angles: 0°, 45°, and 90°; Thermal imaging image in power-on state; X-ray image of internal solder joints using multi-layer scanning.

3. The PCBA circuit board online detection method according to claim 1, characterized in that: Before step S1, the method further includes: A high-resolution industrial camera, infrared thermal imager, and microfocus X-ray machine are deployed on the XYZ three-axis servo platform. Through the industrial computer, data communication channels are established between the high-resolution industrial camera, infrared thermal imager, microfocus X-ray machine, PCBA circuit board and the host computer; Activate and test data communication quality: If the test is passed, the corresponding first scanning operation path, second scanning operation path and third scanning operation path are configured for the high-resolution industrial camera, infrared thermal imager and microfocus X-ray machine respectively.

4. The PCBA circuit board online detection method according to claim 3, characterized in that: S1. Scan and collect multimodal depth information of PCBA circuit boards, including: The host computer issues sampling instructions to the industrial computer; The sampling instruction is executed by the industrial computer: Controlling the XYZ three-axis servo platform to follow the first scanning operation path, driving the high-resolution industrial camera to scan the PCBA circuit board, and collecting a surface visual image of the PCBA circuit board; Controlling the XYZ three-axis servo platform to drive the microfocus X-ray machine to scan the PCBA circuit board according to the third scanning operation path, and collecting X-ray images of internal solder joints of the PCBA circuit board; The PCBA circuit board is controlled to be powered on, and the XYZ three-axis servo platform is controlled to drive the infrared thermal imager to scan the PCBA circuit board in the powered-on state according to the second scanning operation path, and collect thermal imaging images in the powered-on state.

5. The PCBA circuit board online detection method according to claim 1, characterized in that: In step S2, the multimodal depth information of the PCBA circuit board is pre-processed by an industrial computer, including: enhancing the contrast of the surface visual image using a Retinex algorithm; Performing three-dimensional reconstruction on the internal solder joint X-ray image using the FDK algorithm; The thermal imaging image is subjected to noise elimination through non-uniformity correction (NUC).

6. The PCBA circuit board online detection method according to claim 1, characterized in that: In step S3, the host computer identifies and detects each modal depth feature in the multimodal depth information, including: The hybrid detection model pre-deployed on the host computer is used to batch detect the multi-modal depth information of the pre-processed PCBA circuit boards to obtain defect features of the PCBA circuit boards under each modal depth detection; Wherein, the hybrid detection model includes: A YOLOv10 model is used to identify and detect surface defects of the PCBA circuit board in the visual image of the PCBA circuit board surface; A 3D convolutional network model for locating and detecting overheated components in thermal imaging images in a powered-on state. LSTM network model for identifying and detecting internal solder joints with hollows / cavities in X-ray images of internal solder joints; The depth features of each mode are output in layers, and a corresponding inspection report is generated based on the depth features and bound to the ID of the PCBA circuit board currently being inspected.

7. The PCBA circuit board online detection method according to claim 4, characterized in that: In step S3, the multimodal fusion is performed to generate defect fusion features of the PCBA circuit board, including: Divide the PCBA circuit board into several inspection sub-areas; Perform weighted splicing on the PCBA surface defects, overheated components, and internal solder joints with sweat / voids identified in each detection sub-region to generate the defect fusion feature Q for the corresponding detection sub-region; The defect fusion features Q of each detection sub-area are weighted and spliced ​​to generate the defect fusion features P of the PCBA circuit board and input them into the risk decision model to make a risk decision. The good grade of the PCBA circuit board is determined based on the risk determined.

8. A PCBA circuit board online detection system, wherein the PCBA circuit board online detection system is used to implement the PCBA circuit board online detection method according to any one of claims 1 to 7, characterized in that: The system comprises: An XYZ three-axis servo platform is used to servo-drive a high-resolution industrial camera, an infrared thermal imager, and a microfocus X-ray machine to perform scanning motion according to a preset first scanning path, a second scanning path, and a third scanning path; High-resolution industrial camera, used to scan and capture surface visual images of PCBA circuit boards; Infrared thermal imager, used to scan and collect thermal imaging images in the power-on state; Microfocus X-ray machine, used to scan and collect X-ray images of the internal solder joints of PCBA circuit boards; Industrial computer, used to communicate with the host computer, realize system logic control and pre-process the multi-modal depth information of the PCBA circuit board; The host computer is used to identify and detect the depth features of each modality in the multimodal depth information, perform multimodal fusion, and generate defect fusion features of the PCBA circuit board; and perform risk decision-making on the defect fusion features and output the good grade of the PCBA circuit board; The high-resolution industrial camera, infrared thermal imager and micro-focus X-ray machine are respectively deployed and installed on the XYZ three-axis servo platform; The high-resolution industrial camera, infrared thermal imager, micro-focus X-ray machine and XYZ three-axis servo platform are electrically connected to the industrial control machine respectively; The industrial computer is communicatively connected with the host computer.

9. An electronic device, characterized in that: The electronic device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images