A multi-modal interactive PCB defect detection system and a detection method thereof

Abstract

The application discloses a multi-modal interactive PCB defect detection system and a detection method thereof, and relates to the technical field of PCB detection.The system comprises an interactive fusion unit, an interactive fusion unit and a man-machine interaction unit.A multi-modal acquisition unit is used to synchronously acquire appearance visual data, depth transmission data and electrical parameter data of a PCB at the same detection point.The appearance, transmission and electrical data are synchronously acquired by the multi-modal acquisition unit, and are bound with the PCB identification, so that data islands are broken, multi-dimensional evidence chain fusion at the same point is realized, and single-mode misjudgment and missed detection are avoided.The interactive fusion unit performs feature fusion on multi-source heterogeneous data, constructs an index database and outputs a comprehensive judgment result after weighting.The man-machine interaction unit displays the appearance, transmission image and electrical curve of the same detection point in a multi-view linkage mode, receives a review operation and generates a feedback instruction, forms a man-machine collaborative review closed loop, and solves the problems of independent judgment of detection results and lack of fusion verification.

Description

Technical Field

[0001] This invention relates to the field of PCB inspection technology, and in particular to a multimodal interactive PCB defect detection system and its detection method. Background Technology

[0002] In PCB manufacturing and surface mount processes, defect detection is a crucial step in ensuring product yield and reliability. Currently, several single-modal inspection solutions have been developed: visual inspection uses high-definition cameras to acquire images of the PCB surface to identify surface defects such as exposed copper, scratches, solder balls, misalignment, and insufficient solder; depth / transmission inspection uses X-ray or transmission scanning to acquire inner layer circuits, vias, and buried / blind via structures to identify internal open circuits, cold solder joints, and hole wall defects; electrical inspection uses contact testing to acquire parameters such as resistance, impedance, voltage, and current to identify electrical defects such as open circuits, short circuits, and abnormal impedance.

[0003] The aforementioned known technologies each possess certain detection capabilities, but single visual or single electrical detection is insufficient to cover multi-dimensional defect features such as surface, inner layer, and electrical properties, resulting in significant risks of false detection and missed detection. Defect analysis lacks a multi-dimensional evidence chain, each modal detection system is independent of the others, data is difficult to correlate effectively, detection results are mostly independently determined, lack multi-feature fusion verification, and it is difficult to achieve rapid closed-loop debugging of process anomalies. Summary of the Invention

[0004] This invention provides a multimodal interactive PCB defect detection system and method, which can solve the problems in the prior art where each modal detection system is independent, the data is difficult to correlate effectively, the detection results are mostly independently judged, there is a lack of multi-feature fusion verification, and it is difficult to achieve rapid closed-loop debugging of process anomalies.

[0005] A multimodal interactive PCB defect detection system includes an interactive fusion unit, an interactive fusion unit, and a human-machine interaction unit. The multimodal acquisition unit is used to simultaneously acquire visual data, depth transmission data, and electrical parameter data of the PCB at the same inspection point, and binds and stores them with the PCB's unique identifier and the point identifier. The interactive fusion unit is connected to the multimodal acquisition unit and is used to fuse the multimodal data, extract multimodal features, construct a point index database, perform weighted fusion calculations, and output a comprehensive defect judgment result. The human-machine interaction unit is connected to the interactive fusion unit and is used to display the appearance image, transmission image, and electrical curve of the same inspection point in a multi-view linkage manner, receive review operations, and generate review feedback instructions to be sent back to the interactive fusion unit. The interactive fusion unit is also used to optimize the feature fusion algorithm according to the review feedback instructions and output equipment parameter adjustment instructions to the front-end process equipment according to the comprehensive defect judgment result.

[0006] A multi-modal interactive PCB defect detection system provided by the present invention has the following beneficial effects compared with the prior art, including but not limited to: The multi-modal interactive PCB defect detection system synchronously collects appearance, transmission, and electrical data through a multi-modal acquisition unit and binds them to the PCB identification, breaking data silos, realizing the fusion of multi-dimensional evidence chains at the same point, and avoiding missed detections and false positives in single-modal detection. The interactive fusion unit performs feature fusion on multi-source heterogeneous data, constructs an index database, and outputs a comprehensive judgment result with weighted values. The human-computer interaction unit displays the appearance image, transmission image, and electrical curve of the same detection point in a multi-view linkage manner, receives review operations, and generates feedback instructions, forming a human-computer collaborative review closed-loop, solving the problems of independent judgment of detection results and lack of fusion verification. The interactive fusion unit continuously optimizes the algorithm according to the review feedback, and outputs parameter adjustment instructions to the front-end process equipment based on the defect judgment result, realizing two-way closed-loop interaction between detection and process, instantly converting the detection result into process parameter adjustment, significantly shortening the process anomaly location and debugging cycle, and effectively solving the defect that it is difficult to achieve rapid closed-loop debugging of the process in the prior art.

[0007] Furthermore, the multi-modal acquisition unit includes an appearance vision acquisition module, a depth transmission acquisition module, an electrical detection module, a timing synchronization trigger module, and a point identification binding module. The appearance vision acquisition module is used to collect surface appearance images; the depth transmission acquisition module is used to obtain inner layer structures and three-dimensional data; the electrical detection module is used to collect electrical parameters; the timing synchronization trigger module is used to generate a unified trigger signal to enable the appearance vision acquisition module, the depth transmission acquisition module, and the electrical detection module to synchronously collect data at the same detection point; the point identification binding module is used to bind the collected data to the unique ID of the PCB and the point ID.

[0008] Furthermore, the interactive fusion unit includes a multi-modal feature fusion module, an index database construction module, a fusion weight configuration and judgment module, and a model iteration optimization module. The multi-modal feature fusion module is used to map the multi-modal data to a unified feature space; the index database construction module is used to establish an "point ID - multi-modal feature - defect type" index database; the fusion weight configuration and judgment module is used to perform weighted fusion according to the set weights and output a comprehensive judgment result; the model iteration optimization module is used to optimize the fusion algorithm according to the review feedback instructions.

[0009] Furthermore, the human-computer interaction unit includes a multi-view linkage interaction module, a defect interaction review module, and a full-link traceability interaction module. The multi-view linkage interaction module is used to display the appearance image, transmission image, and electrical curve of the same detection point in a linkage manner; the defect interaction review module is used to receive review operations and generate review feedback instructions; the full-link traceability interaction module is used to retrieve and display all the detection and review records of the PCB according to the unique ID of the PCB.

[0010] Furthermore, the timing synchronization triggering module synchronizes the sampling times of the appearance visual acquisition module, the depth transmission acquisition module, and the electrical detection module at the same detection point.

[0011] Furthermore, a bidirectional communication link is established between the interactive fusion unit and the front-end process equipment, and the equipment parameter adjustment instructions include printing parameter, patch parameter, or temperature parameter adjustment instructions.

[0012] Furthermore, the fusion weight configuration and determination module supports real-time modification of the fusion weights of the appearance, depth, and electrical three-modality through the human-computer interaction unit.

[0013] Furthermore, the model iteration optimization module supports two modes: online iteration and offline iteration. In online mode, the current detection parameters are updated in real time; in offline mode, samples are accumulated for the next batch of updates.

[0014] Furthermore, the end-to-end traceability interaction module is also used to construct the "defect type - process parameter - defect location" association.

[0015] A multimodal interactive PCB defect detection method, based on the aforementioned multimodal interactive PCB defect detection system, includes the following steps: S1, synchronously acquiring and binding multimodal data through a multimodal acquisition unit; S2, performing feature fusion and weighted judgment through an interactive fusion unit, and outputting a comprehensive defect result; S3, performing multi-view linkage verification through a human-computer interaction unit, and generating verification feedback instructions; S4, optimizing the algorithm based on the verification feedback through the interactive fusion unit, and outputting parameter adjustment instructions to the front-end process equipment based on the defect result. Attached Figure Description

[0016] Figure 1 This is a system flowchart of a multimodal interactive PCB defect detection system and method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating a multimodal interactive PCB defect detection system and method according to an embodiment of the present invention.

[0017] Explanation of reference numerals in the attached figures: 100. Multimodal Acquisition Unit; 200. Interactive Fusion Unit; 300. Human-Computer Interaction Unit; 400. Front-end Process Equipment; 110. Appearance Visual Acquisition Module; 120. Depth Transmission Acquisition Module; 130. Electrical Detection Module; 140. Timing Synchronization Trigger Module; 150. Point Identification Binding Module; 210. Multimodal Feature Fusion Module; 220. Index Database Construction Module; 230. Fusion Weight Configuration and Judgment Module; 240. Model Iteration Optimization Module; 310. Multi-view Linkage Interaction Module; 320. Defect Interaction Review Module; 330. Full-link Traceability Interaction Module. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application are described clearly and completely below with reference to the accompanying drawings. It should be understood that the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments described in this application without creative effort will fall within the scope of protection of this application.

[0019] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used in the specification of this application is for the purpose of describing specific embodiments only and is not intended to limit this application; the terms "comprising," "including," "having," "containing," "comprise," etc., in the specification, claims, and accompanying drawings of this application are open-ended terms, indicating that a method comprises one or more steps, or an apparatus comprises one or more elements, but do not exclude the inclusion of other steps or elements. The terms "first," "second," etc., in the specification, claims, or accompanying drawings of this application are used to distinguish different objects, not to describe a specific order or primary / secondary relationship. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0020] In the description of this application, it should be understood that the terms "upper," "lower," "left," "right," "front," and "rear," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Similarly, it should be noted that the meanings of the X-axis, Y-axis, and Z-axis are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0021] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," "linking," and "attachment" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0022] In this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, in this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0023] like Figure 1 As shown in the figure, an embodiment of the present invention provides a multimodal interactive PCB defect detection system, including an interactive fusion unit 200 and a human-machine interaction unit 300. The multimodal acquisition unit 100 is used to simultaneously acquire the appearance visual data, depth transmission data and electrical parameter data of the PCB at the same detection point, and bind and store them with the PCB unique identifier and the point identifier. The interactive fusion unit 200 is connected to the multimodal acquisition unit 100 and is used to fuse multimodal data, extract multimodal features, construct a point index database, and output a comprehensive defect judgment result after weighted fusion calculation. The human-machine interaction unit 300 is connected to the interactive fusion unit 200 and is used to display the appearance image, transmission image and electrical curve of the same detection point in a multi-view linkage mode, receive review operations and generate review feedback instructions to be sent back to the interactive fusion unit 200. The interactive fusion unit 200 is also used to optimize the feature fusion algorithm according to the review feedback instructions and output equipment parameter adjustment instructions to the front-end process equipment 400 according to the comprehensive defect judgment result.

[0024] In this embodiment, the multimodal acquisition unit 100 simultaneously acquires visual appearance data, depth transmission data, and electrical parameter data for the same inspection point and binds and stores them with the PCB unique identifier and the point identifier. The interactive fusion unit 200 performs feature fusion on these multi-source heterogeneous data, constructs a point index database, and outputs a weighted comprehensive defect judgment result. This breaks the limitations of each modal detection system being independent and data being difficult to correlate, and realizes multi-dimensional evidence chain fusion verification for the same inspection point, effectively avoiding false detections and missed detections caused by single modal judgment. At the same time, the human-computer interaction unit 300 displays the appearance of the same inspection point in a multi-view linkage manner. Images, transmission images, and electrical curves are received for verification operations, and verification feedback instructions are generated and sent back to the interactive fusion unit 200, forming a human-machine collaborative verification closed loop. This solves the problem of independent judgment of detection results and lack of fusion verification. The interactive fusion unit 200 also continuously optimizes the feature fusion algorithm based on the verification feedback instructions, and outputs equipment parameter adjustment instructions to the front-end process equipment 400 based on the comprehensive defect judgment results. This realizes two-way closed-loop interaction between the detection system and the process equipment, and instantly transforms the defect detection results into process parameter adjustments, significantly shortening the process anomaly location and debugging cycle, and solving the shortcomings of existing technologies that make it difficult to achieve rapid closed-loop debugging of the process.

[0025] like Figure 1 As shown, the multimodal acquisition unit 100 includes an appearance visual acquisition module 110, a depth transmission acquisition module 120, an electrical detection module 130, a timing synchronization trigger module 140, and a point identification binding module 150. The appearance visual acquisition module 110 is used to acquire surface appearance images; the depth transmission acquisition module 120 is used to acquire internal structure and three-dimensional data; the electrical detection module 130 is used to acquire electrical parameters; the timing synchronization trigger module 140 is used to generate a unified trigger signal so that the appearance visual acquisition module 110, the depth transmission acquisition module 120, and the electrical detection module 130 acquire data synchronously at the same detection point; and the point identification binding module 150 is used to bind the acquired data with the PCB unique ID and the point ID.

[0026] In this embodiment, the appearance visual acquisition module 110 acquires surface appearance images, the depth transmission acquisition module 120 acquires inner layer structure and three-dimensional data, and the electrical detection module 130 acquires electrical parameters. The timing synchronization triggering module 140 generates a unified trigger signal to enable the appearance visual acquisition module 110, the depth transmission acquisition module 120, and the electrical detection module 130 to synchronously acquire data at the same detection point. The point identification binding module 150 binds the acquired data with the PCB unique ID and the point ID, thereby achieving high-precision timing alignment and point-level association of the three heterogeneous data of appearance, inner layer, and electrical at the same detection point at the source of acquisition. This breaks the data silo situation in the traditional mode detection equipment where each mode acquires data independently and the data cannot be linked and linked, and provides a complete multi-dimensional evidence chain of the same point for subsequent multi-modal feature fusion and comprehensive judgment. It effectively overcomes the risk of false detection and missed detection caused by the difficulty of single mode detection in covering multi-dimensional defect features of surface, inner layer, and electrical.

[0027] like Figure 1 As shown, the interactive fusion unit 200 includes a multimodal feature fusion module 210, an index database construction module 220, a fusion weight configuration and judgment module 230, and a model iteration optimization module 240. The multimodal feature fusion module 210 is used to map multimodal data to a unified feature space; the index database construction module 220 is used to establish an index database of "location ID-multimodal feature-defect type"; the fusion weight configuration and judgment module 230 is used to perform weighted fusion according to the set weights and output the comprehensive judgment result; and the model iteration optimization module 240 is used to optimize the fusion algorithm according to the review feedback instructions.

[0028] In this embodiment, the multimodal feature fusion module 210 maps three heterogeneous data types—appearance visual, depth transmission, and electrical properties—to a unified feature space, achieving normalization of multi-dimensional features. The index database construction module 220 establishes an index database of "location ID-multimodal features-defect type," enabling rapid retrieval and linkage of detection data by location. The fusion weight configuration and judgment module 230 performs weighted fusion according to set weights and outputs a comprehensive judgment result, avoiding the risk of misjudgment based on a single modality. The model iteration optimization module 240 continuously optimizes the fusion algorithm based on the review feedback instructions, thereby achieving deep fusion and intelligent judgment of multimodal features at the core level of data processing, significantly improving the accuracy and reliability of defect detection. At the same time, the index database provides structured data support for defect tracing and process optimization, and the online model iteration continuously improves the detection performance with use.

[0029] like Figure 1As shown, the human-computer interaction unit 300 includes a multi-view linkage interaction module 310, a defect interaction verification module 320, and a full-link traceability interaction module 330. The multi-view linkage interaction module 310 is used to link and display the appearance image, transmission image, and electrical curve of the same inspection point; the defect interaction verification module 320 is used to receive verification operations and generate verification feedback instructions; and the full-link traceability interaction module 330 is used to retrieve and display all inspection and verification records of the PCB according to the PCB's unique ID.

[0030] In this embodiment, the multi-view linkage interaction module 310 displays the appearance image, transmission image, and electrical curve of the same inspection point in a linked manner. This allows the operator to intuitively compare the same suspected defect from three dimensions—surface, inner layer, and electrical properties—without switching between multiple systems, effectively solving the problem of information fragmentation caused by single-modal result display. The defect interaction verification module 320 receives verification operations such as operator confirmation, false alarm labeling, or defect level correction, and generates verification feedback instructions to be sent back to the interaction fusion unit 200, forming a collaborative closed loop of "intelligent judgment + manual calibration." This allows false detections to be corrected immediately, and verification data to directly drive model iteration and optimization. The full-link traceability interaction module 330 retrieves and displays all inspection and verification records of the PCB based on its unique PCB ID, enabling one-click backtracking of multi-dimensional evidence chains across nodes and processes. This quickly locates the process link and root cause of the defect, thereby systematically improving human-machine collaboration efficiency, verification accuracy, and defect tracing capabilities.

[0031] like Figure 1 As shown, the timing synchronization trigger module 140 synchronizes the sampling times of the appearance visual acquisition module 110, the depth transmission acquisition module 120, and the electrical detection module 130 at the same detection point.

[0032] In this embodiment, the timing synchronization triggering module 140 synchronizes the sampling times of the appearance visual acquisition module 110, the depth transmission acquisition module 120, and the electrical detection module 130 at the same detection point, thereby ensuring the precise alignment of the appearance image, inner structure data, and electrical parameters in the time dimension at the source of acquisition, and completely eliminating the data misalignment and point offset problems caused by independent triggering of each module and inconsistent sampling times.

[0033] like Figure 1 As shown, a bidirectional communication link is established between the interactive fusion unit 200 and the front-end process equipment 400. The equipment parameter adjustment commands include printing parameter, patch parameter, or temperature parameter adjustment commands.

[0034] In this embodiment, the detection system can automatically convert the comprehensive defect judgment result into specific executable process parameter adjustment instructions and send them to the front-end process equipment 400. At the same time, it can receive the equipment execution status feedback, thereby realizing a fully automatic two-way closed-loop interaction from "detection and judgment" to "process debugging". This breaks the data silo between the traditional detection system and process equipment, which is characterized by one-way data reporting and lack of feedback linkage. It significantly shortens the process anomaly location and debugging cycle caused by repeated defects, effectively avoids the continuous occurrence of the same type of defect in the same process, and greatly improves the process stability and debugging efficiency in PCB mass production.

[0035] like Figure 1 As shown, the fusion weight configuration and judgment module 230 supports real-time modification of the fusion weights of appearance, depth, and electrical three modalities through the human-computer interaction unit 300.

[0036] In this embodiment, the operator can dynamically adjust the contribution ratio of each mode in the comprehensive judgment according to different PCB product types, different manufacturing processes, or different defect sensitivity requirements. For example, the depth transmission weight of BGA devices can be increased, and the electrical weight of connectors can be increased. This gives the detection system a high degree of scene adaptability, avoids the missed detection or false alarm of specific defect types caused by fixed weight settings, significantly improves the accuracy and flexibility of the fusion judgment results, and meets the actual needs of batch inspection of diversified PCB products for personalized parameter configuration.

[0037] like Figure 1 As shown, the model iteration optimization module 240 supports two modes: online iteration and offline iteration. In online mode, the current detection parameters are updated in real time; in offline mode, samples are accumulated for the next batch of updates.

[0038] In this embodiment, the model iteration optimization module 240 supports both online and offline iteration modes: In online mode, the current detection parameters are updated in real time, so that the feedback results of manual review can take effect immediately, and the fusion judgment rules of subsequent detection points in the current batch are dynamically optimized, which greatly improves the response speed and accuracy of defect detection within the same batch; In offline mode, the accumulated samples are used for the next batch update, so that a large amount of review data forms a structured incremental training set, and the model depth optimization is completed without affecting the current production cycle, realizing the continuous evolution and generalization ability improvement of the detection algorithm, thereby balancing production real-time performance and model iteration depth, and ensuring that the performance of the defect detection system continuously improves with usage time and data accumulation.

[0039] like Figure 1 As shown, the end-to-end traceability interaction module 330 is also used to build the association between "defect type - process parameters - defect location".

[0040] In this embodiment, the detection system can structurally associate and intelligently map the specific defect type output by the multimodal acquisition unit 100 with the process parameters (such as printing pressure, chip coordinates, reflow soldering temperature, etc.) of the front-end process equipment 400 and the location of the defect point. When a certain detection point is determined to be a defect, the system can automatically retrieve and output the statistical correlation between the defect type and the corresponding process parameters and defect location in historical data, helping the operator to quickly locate the root process link and parameter deviation that caused the defect, thereby changing passive post-event analysis to proactive root cause tracing, significantly improving the efficiency of process anomaly location and the targeting of process optimization.

[0041] like Figure 2 As shown, a multimodal interactive PCB defect detection method, based on a multimodal interactive PCB defect detection system, includes the following steps: S1, synchronously acquiring and binding multimodal data through a multimodal acquisition unit 100; S2, performing feature fusion and weighted judgment through an interactive fusion unit 200, and outputting a comprehensive defect result; S3, performing multi-view linkage verification through a human-computer interaction unit 300, and generating verification feedback instructions; S4, optimizing the algorithm according to the verification feedback through the interactive fusion unit 200, and outputting parameter adjustment instructions to the front-end process equipment 400 according to the defect result.

[0042] In this embodiment, multimodal acquisition unit 100 synchronously acquires and binds multimodal data, breaking down data silos at the source and achieving precise alignment and associated storage of appearance, inner layer, and electrical data at the same location. Interactive fusion unit 200 performs feature fusion and weighted judgment, outputting comprehensive defect results to overcome the false detection and missed detection problems caused by single-modal detection. Human-machine interaction unit 300 performs multi-view linkage verification, generating verification feedback instructions to form a collaborative closed loop of "intelligent judgment + manual calibration," enabling verification data to drive model optimization in real time. Interactive fusion unit 200 optimizes the algorithm based on verification feedback and outputs parameter adjustment instructions to front-end process equipment 400 based on defect results, realizing bidirectional closed-loop debugging of detection and process. This systematically solves the problems of data isolation, lack of fusion, human-machine separation, and debugging lag in the prior art, significantly improving the accuracy of PCB defect detection, human-machine collaboration efficiency, and process response speed.

[0043] The above-disclosed embodiments are merely a few specific examples of the present invention. However, the embodiments of the present invention are not limited thereto, and any variations that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A multimodal interactive PCB defect detection system, characterized in that, include: The multimodal acquisition unit (100) is used to simultaneously acquire the appearance visual data, depth transmission data and electrical parameter data of the PCB at the same detection point, and bind and store them with the PCB unique identifier and the point identifier; The interactive fusion unit (200) is connected to the multimodal acquisition unit (100) and is used to fuse the multimodal data, extract multimodal features, construct a point index database, and output the comprehensive defect judgment result after weighted fusion calculation. The human-computer interaction unit (300) is connected to the interaction fusion unit (200) and is used to display the appearance image, transmission image and electrical curve of the same detection point in a multi-view linkage manner, receive the review operation and generate the review feedback instruction to be sent back to the interaction fusion unit (200). The interactive fusion unit (200) is also used to optimize the feature fusion algorithm according to the review feedback instruction, and output the equipment parameter adjustment instruction to the front-end process equipment (400) according to the defect comprehensive judgment result.

2. The multimodal interactive PCB defect detection system as described in claim 1, characterized in that, The multimodal acquisition unit (100) includes: The appearance visual acquisition module (110) is used to acquire surface appearance images; The depth transmission acquisition module (120) is used to acquire the inner structure and three-dimensional data; Electrical detection module (130) is used to collect electrical parameters; The timing synchronization trigger module (140) is used to generate a unified trigger signal so that the appearance visual acquisition module (110), the depth transmission acquisition module (120) and the electrical detection module (130) can synchronously acquire data at the same detection point. The point identification binding module (150) is used to bind the collected data with the PCB unique ID and the point ID.

3. The multimodal interactive PCB defect detection system as described in claim 1, characterized in that, The interactive fusion unit (200) includes: A multimodal feature fusion module (210) is used to map the multimodal data to a unified feature space; The index database construction module (220) is used to establish an index database of "location ID-multimodal features-defect type"; The fusion weight configuration and judgment module (230) is used to perform weighted fusion according to the set weights and output the comprehensive judgment result; The model iteration optimization module (240) is used to optimize the fusion algorithm according to the verification feedback instructions.

4. The multimodal interactive PCB defect detection system as described in claim 1, characterized in that, The human-computer interaction unit (300) includes: The multi-view linkage interaction module (310) is used to link and display the appearance image, transmission image and electrical curve of the same detection point; The defect interactive review module (320) is used to receive review operations and generate review feedback instructions; The end-to-end traceability interaction module (330) is used to retrieve and display all inspection and verification records of the PCB based on its unique PCB ID.

5. The multimodal interactive PCB defect detection system as described in claim 2, characterized in that, The timing synchronization trigger module (140) synchronizes the sampling times of the appearance visual acquisition module (110), the depth transmission acquisition module (120), and the electrical detection module (130) at the same detection point.

6. The multimodal interactive PCB defect detection system as described in claim 1, characterized in that, The interactive fusion unit (200) establishes a bidirectional communication link with the front-end process equipment (400), and the equipment parameter adjustment instructions include printing parameter, patch parameter or temperature parameter adjustment instructions.

7. The multimodal interactive PCB defect detection system as described in claim 3, characterized in that, The fusion weight configuration and judgment module (230) supports real-time modification of the fusion weights of appearance, depth and electrical three modes through the human-computer interaction unit (300).

8. The multimodal interactive PCB defect detection system as described in claim 3, characterized in that, The model iteration optimization module (240) supports both online and offline iteration modes: In online mode, the current detection parameters are updated in real time. In offline mode, accumulated samples are used for the next batch update.

9. The multimodal interactive PCB defect detection system as described in claim 4, characterized in that, The full-link traceability interaction module (330) is also used to construct the "defect type-process parameter-defect location" association.

10. A multimodal interactive PCB defect detection method, characterized in that, The multimodal interactive PCB defect detection system applied to any one of claims 1-9 includes the following steps: S1. Multimodal data is synchronously acquired and bound for storage through the multimodal acquisition unit (100); S2. Feature fusion and weighted judgment are performed through the interactive fusion unit (200), and the comprehensive defect result is output; S3. Perform multi-view linkage review through the human-computer interaction unit (300) and generate review feedback instructions; S4. The interactive fusion unit (200) optimizes the algorithm based on the review feedback and outputs parameter adjustment instructions to the front-end process equipment (400) based on the defect results.

Images