PCB detection method and device based on multi-modal fusion and adaptive algorithm

Abstract

The invention discloses a PCB detection method and device based on multi-modal fusion and an adaptive algorithm, and relates to the field of electronic manufacturing detection. The method comprises the following steps: acquiring optical imaging data by using an optical imaging unit, acquiring tomography data by using an X-ray unit, and acquiring three-dimensional point cloud data by using a laser unit; collecting a reference signal, and updating detection parameters of the optical imaging unit, the X-ray unit and the laser unit; performing feature level fusion to obtain multi-modal fusion data; and inputting the multi-modal fusion data into a detection model, and performing defect comparison in combination with a historical database to obtain a defect detection result of the PCB. According to the method, optical imaging, X-ray penetration and laser three-dimensional modeling technologies are combined, and full-dimensional defect detection of the surface, the interior and the three-dimensional structure of the PCB is covered, so that the defect accuracy is improved. Meanwhile, a traditional fixed parameter detection mode is broken through through a self-adaptive intelligent algorithm, dynamic parameter optimization and multi-scene self-adaption are achieved, and the detection efficiency is effectively improved.

Description

Technical Field

[0001] This application relates to the field of electronic manufacturing inspection technology, and particularly to a PCB inspection method and device based on multi-modal fusion and adaptive algorithms. Background Art

[0002] PCB (Printed Circuit Board) inspection is an important link to ensure the quality and performance of PCBs. The inspection contents include: electrical performance, ensuring that the circuits on the PCB are correctly connected without problems such as short circuits and open circuits, thus guaranteeing the normal operation of electronic devices; soldering quality, detecting whether the solder joints are full, without false soldering, solder balls and other defects to ensure a reliable connection between electronic components and the PCB; appearance, detecting whether there are scratches, stains, deformations and other problems on the surface of the PCB to avoid affecting the assembly and use of the PCB.

[0003] Traditional PCB inspection equipment usually adopts a single inspection technology, relying only on optical imaging or X-ray inspection, resulting in a limited inspection range. In addition, the traditional inspection has insufficient versatility and is difficult to adapt to PCBs of different models, materials and processes. Summary of the Invention

[0004] Based on this, it is necessary to provide a PCB inspection method and device based on multi-modal fusion and adaptive algorithms to improve the inspection accuracy and efficiency for the above technical problems.

[0005] In a first aspect, this application provides a PCB inspection method based on multi-modal fusion and adaptive algorithms. The method includes:

[0006] Obtaining optical imaging data of the PCB using an optical imaging unit, obtaining tomographic data of the PCB using an X-ray unit, and obtaining three-dimensional point cloud data of the PCB using a laser unit;

[0007] Collecting a reference signal of the PCB, and updating the detection parameters of the optical imaging unit, X-ray unit and laser unit according to the optical imaging data, tomographic data, three-dimensional point cloud data and reference signal using an adaptive intelligent algorithm;

[0008] Re-obtaining optical imaging data, tomographic data and three-dimensional point cloud data based on the updated detection parameters, and performing feature-level fusion to obtain multi-modal fusion data;

[0009] Inputting the multi-modal fusion data into a detection model, and comparing with a historical database to obtain a defect detection result of the PCB.

[0010] In one embodiment, the update of the detection parameters includes:

[0011] Obtain the errors between the optical imaging data, tomography data, and three-dimensional point cloud data and the reference signal respectively, and iteratively adjust the detection parameters according to the errors.

[0012] In one embodiment, the method further includes:

[0013] Take the detection parameters output in each iteration as the current parameters, calculate the performance metrics under the current parameters, and determine whether the adaptive intelligent algorithm converges according to the performance metrics; if not, continue to iteratively adjust the detection parameters; if so, stop the iteration when the iteration stop condition is met, and take the current detection parameters as the updated detection parameters.

[0014] In one embodiment, after re-obtaining the optical imaging data, tomography data, and three-dimensional point cloud data based on the updated detection parameters, the method further includes:

[0015] Perform image processing on the optical imaging data to identify the first defect feature;

[0016] Apply denoising and three-dimensional reconstruction algorithms to the tomography data, and combine threshold segmentation to locate the second defect feature;

[0017] After filtering and registering the three-dimensional point cloud data, extract the morphology parameters;

[0018] Combine the first defect feature, the second defect feature, and the morphology parameters to obtain multi-modal fusion data.

[0019] In one embodiment, obtaining the multi-modal fusion data includes:

[0020] Extract features from the optical imaging data, tomography data, and three-dimensional point cloud data, and introduce an attention mechanism to weight the first defect feature, the second defect feature, and the morphology parameters to obtain the corresponding target features;

[0021] Establish the causal relationship between multi-modal data through semantic alignment and temporal matching techniques to obtain multi-modal fusion data.

[0022] In one embodiment, the method further includes:

[0023] Adopt a spatio-temporal synchronization alignment mechanism to obtain the optical imaging data, tomography data, and three-dimensional point cloud data.

[0024] In a second aspect, the present application also provides a PCB detection device based on multi-modal fusion and adaptive algorithm. The device includes:

[0025] A data acquisition module, configured to obtain the optical imaging data of the PCB by using an optical imaging unit, obtain the tomography data of the PCB by using an X-ray unit, and obtain the three-dimensional point cloud data of the PCB by using a laser unit;

[0026] A parameter update module, configured to collect reference signals of the PCB, and update detection parameters of the optical imaging unit, the X-ray unit, and the laser unit according to the optical imaging data, the tomography data, the three-dimensional point cloud data, and the reference signals by using an adaptive intelligent algorithm;

[0027] A data update module, configured to re-acquire the optical imaging data, the tomography data, and the three-dimensional point cloud data based on the updated detection parameters, and perform feature-level fusion to obtain multi-modal fusion data;

[0028] A detection output module, configured to input the multi-modal fusion data into a detection model, compare defects in combination with a historical database, and obtain a defect detection result of the PCB.

[0029] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps in the above-mentioned PCB detection method based on multi-modal fusion and adaptive algorithm are implemented.

[0030] In a fourth aspect, the present application further provides a computer-readable storage medium. On the computer-readable storage medium, a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned PCB detection method based on multi-modal fusion and adaptive algorithm are implemented.

[0031] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned PCB detection method based on multi-modal fusion and adaptive algorithm are implemented.

[0032] The above-mentioned PCB detection method and device based on multi-modal fusion and adaptive algorithm utilize the optical imaging unit to obtain the optical imaging data of the PCB, utilize the X-ray unit to obtain the tomography data of the PCB, and utilize the laser unit to obtain the three-dimensional point cloud data of the PCB; collect reference signals of the PCB, and update detection parameters of the optical imaging unit, the X-ray unit, and the laser unit according to the optical imaging data, the tomography data, the three-dimensional point cloud data, and the reference signals by using an adaptive intelligent algorithm; re-acquire the optical imaging data, the tomography data, and the three-dimensional point cloud data based on the updated detection parameters, and perform feature-level fusion to obtain multi-modal fusion data; input the multi-modal fusion data into a detection model, compare defects in combination with a historical database, and obtain a defect detection result of the PCB. The present application combines optical imaging, X-ray penetration, and laser three-dimensional modeling technologies to cover full-dimensional defect detection of the PCB surface, internal, and three-dimensional structures, thereby being able to greatly reduce the probability of missed detection and false detection and more accurately identify various potential defects. At the same time, by using an adaptive intelligent algorithm to break through the traditional fixed-parameter detection mode, dynamic parameter optimization and multi-scenario adaptability are realized, effectively improving the detection efficiency. Description of the Drawings

[0033] Figure 1 It is a flowchart of a PCB detection method based on multi-modal fusion and adaptive algorithm in an embodiment;

[0034] Figure 2 It is a flowchart of an adaptive intelligent algorithm in an embodiment. Detailed Description of the Invention

[0035] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0036] An embodiment of the present application provides a PCB detection method based on multi-modal fusion and adaptive algorithm, as Figure 1 shown, including the following steps:

[0037] Step 102, obtaining optical imaging data of the PCB by using an optical imaging unit, obtaining tomographic data of the PCB by using an X-ray unit, and obtaining three-dimensional point cloud data of the PCB by using a laser unit.

[0038] Based on the optical principle, the optical imaging unit uses optical elements such as lenses and mirrors to refract and reflect light, so that the light emitted or reflected by the object is focused on the imaging surface, thereby forming an image of the object. Its essence is to convert the optical information of the object into a two-dimensional image signal that can be recorded and observed.

[0039] The X-ray unit generates X-rays by hitting a metal target with high-speed electrons. When the X-rays pass through an object, due to the different absorption degrees of different tissues or substances for X-rays, the intensity of the transmitted X-rays generates differences. These differences are received by the detector and converted into electrical signals, and after being processed by a computer, an image showing the inside of the object is obtained.

[0040] The laser unit determines the distance from each point on the surface of the object to the instrument by emitting a laser beam and measuring the time or phase change of the laser from emission to reflection back to the receiver. Combining the position and attitude information of the instrument, and using computer algorithms to process and reconstruct these point data, a three-dimensional model of the object can be generated.

[0041] The optical imaging unit uses optical imaging technology to clearly capture the fine defects on the surface of the PCB board, such as scratches and notches; the X-ray unit uses X-ray penetration technology to deeply detect the connection of internal circuits and find hidden dangers such as short circuits and open circuits; the laser unit uses laser three-dimensional modeling technology to detect the solder joint height of electronic components, the flatness of component mounting, and the warping of the substrate to ensure that the components meet the quality standards.

[0042] Step 104: Collect the reference signal of the PCB, and use an adaptive intelligent algorithm to update the detection parameters of the optical imaging unit, X-ray unit, and laser unit according to the optical imaging data, tomography data, 3D point cloud data, and reference signal.

[0043] Among them, the reference signal can be obtained through manual measurement, through the PCB production line, or through methods such as PCB design manuals. The reference signal gives a reference standard for information such as the model, specifications, and production process of the PCB. The adaptive intelligent algorithm evaluates the degree of deviation from the reference standard by comparing the optical imaging data, tomography data, and 3D point cloud data with the reference signal respectively, and uses this as the basis for adaptively adjusting the detection parameters of the optical imaging unit, X-ray unit, and laser unit. Finally, the detection parameters are adjusted so that the optical imaging unit, X-ray unit, and laser unit can accurately and completely obtain the optical imaging data, tomography data, and 3D point cloud data.

[0044] In this embodiment, according to different PCB models, specifications, and production processes, the detection parameters and strategies of the optical imaging unit, X-ray unit, and laser unit are automatically adjusted. The algorithm will analyze the detection data in real time and dynamically optimize the detection process. When a complex multi-layer PCB board is detected, the algorithm will automatically increase the number of scanning layers and accuracy of the X-ray unit to ensure that each layer of circuit can be accurately detected; for PCBs of different materials, the algorithm can intelligently match the most suitable detection mode to improve the detection efficiency and accuracy.

[0045] Step 106: Re-obtain the optical imaging data, tomography data, and 3D point cloud data based on the updated detection parameters, and perform feature-level fusion to obtain multi-modal fusion data.

[0046] After completing the optimization of the detection parameters of the optical imaging unit, X-ray unit, and laser unit through Step 104, the optical imaging unit, X-ray unit, and laser unit are configured according to the optimized detection parameters, and the PCB is detected again to obtain more reliable optical imaging data, tomography data, and 3D point cloud data.

[0047] Then, feature extraction is performed on the optical imaging data, tomography data, and 3D point cloud data respectively, and then the extracted features are fused to obtain multi-modal fusion data. Information contained in data of different modalities can complement each other, thus providing a more comprehensive perspective for subsequent defect detection.

[0048] Step 108: Input the multi-modal fusion data into the detection model, and perform defect comparison in combination with the historical database to obtain the defect detection result of the PCB.

[0049] Among them, the detection model can be an object detection network, such as Faster R-CNN, Mask R-CNN, etc., as well as single-stage YOLO series, SSD, etc. It can not only detect whether there are defects in the multi-modal fusion data, but also locate the positions of the defects. The historical database includes non-damaged PCBs and damaged PCBs, which are used as training data to train the detection model, enabling the detection model to learn defect features and better complete the defect detection task. In some cases where the number of defect samples is limited, a generative adversarial network can also be used to generate more simulated defect samples to improve the performance of the detection model.

[0050] This embodiment innovatively integrates various detection technologies such as optical imaging, X-ray penetration, and laser three-dimensional modeling. Optical imaging is used for surface defect recognition. By analyzing the image through an image processing algorithm and comparing it with a standard image, potential defects can be identified. X-ray penetration is for internal defect detection. Utilizing the penetrability and differential absorption characteristics of X-rays, abnormal internal structures of the product can be captured. Its advantage lies in making up for the limitation that optical imaging cannot penetrate the surface of the object. The laser unit generates three-dimensional point cloud data through laser scanning, quantifies the morphological parameters of the object, extracts features by combining algorithms, and can accurately describe the surface geometric characteristics, which is suitable for complex morphology analysis. At the same time, the detection parameters are adjusted efficiently and automatically through an adaptive intelligent algorithm to achieve all-round and high-precision detection of the PCB board, improving the detection efficiency and accuracy and reducing the cost of manual intervention.

[0051] In one embodiment, the complete detection process of the on-site PCB is as follows: The PCB board enters the detection station for positioning through a conveying device, and a four-axis structure moves the detection device including the optical imaging unit, X-ray unit, and laser unit to the detection station. The optical imaging unit scans for surface defects, and simultaneously the X-ray unit performs layer-by-layer imaging of the internal circuits. The laser unit scans the heights of solder joints and components to generate a three-dimensional model and compares it with the design drawing. The adaptive algorithm analyzes the multi-modal data, dynamically adjusts the subsequent detection parameters, and marks the positions of the defects. Qualified PCBs are conveyed to the next station, and defective PCBs enter the repair channel. Based on the defect detection results, the weak points in the production process are located, and a production line adjustment plan is proposed to improve the yield rate. Finally, a structured report integrating multi-source data and analysis conclusions can be generated, including a defect distribution map, quantified parameters, and optimization suggestions. It can also be presented in forms such as 3D rendering and heat maps to visually display the defect positions and the scope of process influence, supporting interactive data query.

[0052] In one embodiment, the update of the detection parameters includes: obtaining the errors between the optical imaging data, tomographic data, and three-dimensional point cloud data and the reference signals respectively, and iteratively adjusting the detection parameters according to the errors and the reference signals.

[0053] In this embodiment, the optical imaging data, tomography data, 3D point cloud data, and reference signal can be respectively converted into vector representations, and then by calculating the vector distances between the optical imaging data and the reference signal, between the tomography data and the reference signal, and between the 3D point cloud data and the reference signal, the corresponding errors can be obtained respectively, and the detection parameters of the optical imaging unit, X-ray unit, and laser unit can be optimized according to the errors.

[0054] Since the focuses of the optical imaging data, tomography data, and 3D point cloud data for detecting the PCB are not the same, the vectorization transformation of the parameter signals can be correspondingly designed to adapt to the optical imaging data, tomography data, and 3D point cloud data in vector representation, which is convenient for error analysis.

[0055] In one embodiment, as Figure 2 shown, it is the complete flowchart of the adaptive intelligent algorithm, and the specific steps are as follows:

[0056] Step 201, set the initial detection parameters of the algorithm and clarify the goal of minimizing the error of the algorithm.

[0057] Step 202, input the acquired data and the reference signal, where the acquired data includes optical imaging data, tomography data, and 3D point cloud data.

[0058] Step 203, calculate the error between the acquired data and the reference signal.

[0059] Step 204, update the detection parameters using the adaptive rule according to the error and the current detection parameters.

[0060] Step 205, calculate the performance index under the current detection parameters and determine whether the algorithm has converged. If not, return to Step 202 to continue the iteration; if converged, enter Step 206.

[0061] Step 206, output the current detection parameters after adaptive adjustment.

[0062] Step 207, according to the output current detection parameters and performance evaluation, decide whether further parameter or algorithm adjustment is needed. When the input data changes or the environment changes (such as the detection object changes, etc.), return to Step 201 to restart the adaptive process.

[0063] Step 208, judge whether the iteration terminates. The iteration termination conditions include: reaching the preset maximum number of iterations, the error being lower than the preset threshold, and the performance index reaching the preset goal.

[0064] Step 209, if the termination conditions are met, the algorithm ends; if the termination conditions are not met, return to Step 202 to continue the iteration.

[0065] In one embodiment, after re-acquiring the optical imaging data, tomographic data, and three-dimensional point cloud data based on the updated detection parameters, the method further includes: performing image processing on the optical imaging data to identify the first defect feature; applying denoising and three-dimensional reconstruction algorithms to the tomographic data, and combining threshold segmentation to locate the second defect feature; filtering and registering the three-dimensional point cloud data, and then extracting the topography parameters; and combining the first defect feature, the second defect feature, and the topography parameters to obtain multimodal fusion data.

[0066] Analyze the image through an image processing algorithm and compare it with a standard library to generate a detection result, which can preliminarily locate the defects in the optical imaging data.

[0067] Apply denoising and three-dimensional reconstruction algorithms to the tomographic data, and combine threshold segmentation to locate internal defects and output a three-dimensional topology analysis report. Threshold segmentation is an image segmentation method based on image gray values. In an image, different objects or regions usually have different gray value distributions. The basic idea of threshold segmentation is to select a suitable gray value as the threshold, and divide the pixel points in the image into two categories according to the relationship between their gray values and the threshold: pixel points greater than the threshold are classified into one category, and pixel points less than or equal to the threshold are classified into another category. Using threshold segmentation technology, according to the gray difference between the defect area and the normal area, a suitable threshold can be selected to segment the image, so as to separate the defect area from the normal area. Once the defect area is segmented, its position and range in the image can be determined, that is, the localization of the defect is achieved.

[0068] After filtering and registering the three-dimensional point cloud data, extract the key topography parameters and construct a quantitative feature library.

[0069] In one embodiment, obtaining multimodal fusion data includes: extracting features from the optical imaging data, tomographic data, and three-dimensional point cloud data, and introducing an attention mechanism to weight the first defect feature, the second defect feature, and the topography parameters to obtain corresponding target features; and establishing a causal relationship between multimodal data through semantic alignment and temporal matching techniques to obtain multimodal fusion data.

[0070] This embodiment integrates surface defect features, internal topological structures, and topography parameters, and uses the attention mechanism to weight different modal features, which can improve the defect recognition ability of the detection model.

[0071] Semantic alignment is an important step in multimodal fusion. By correlating and mapping the semantic information in different modal data, the data of different modalities can understand and correspond to each other at the semantic level, constructing a unified semantic representation space to break the semantic gap between different modal data. The temporal matching technology mainly focuses on the corresponding relationship of data in the time dimension. It aligns and matches data of different modalities and different time scales in chronological order to analyze the changes and mutual relationships of different data at the same time point or time period. For multimodal data with clear timestamps, alignment is directly performed according to the timestamps. When there are deviations in the clocks of different data sources, time calibration is required.

[0072] In one embodiment, the method further includes: obtaining optical imaging data, tomographic data, and three-dimensional point cloud data by using a spatio-temporal synchronization alignment mechanism.

[0073] In this embodiment, time synchronization adopts a combination of hardware triggering and software synchronization to ensure the time consistency of multimodal data. Specifically, the acquisition frequencies of different sensors are coordinated by a unified clock signal to avoid time drift.

[0074] Spatial alignment eliminates the spatial deviation between multimodal data through reference calibration, coordinate system transformation, and spatial interpolation techniques.

[0075] Reference calibration is to determine a unified reference standard, providing a basis for subsequent spatial alignment. In multimodal data, different data sources may collect data based on their respective local coordinate systems. The lack of a unified reference will lead to confusion in spatial position relationships. Through reference calibration, a common reference point or reference feature can be found, enabling data of different modalities to be compared and processed under the same reference framework. Common methods include the physical marker method and the feature matching method.

[0076] After determining the unified reference standard through reference calibration, it is necessary to convert the coordinate systems of different modal data into this unified coordinate system. Coordinate system transformation can unify the spatial positions of different data and eliminate the spatial deviation caused by coordinate system differences. Usually, rigid body transformation and affine transformation methods are used.

[0077] After coordinate system transformation, there may still be some discontinuous or missing situations in the space of different modal data. Spatial interpolation techniques can estimate the data values at unknown positions based on known data points, thereby achieving data smoothing and integrity. Commonly used spatial interpolation techniques such as nearest neighbor interpolation, linear interpolation, etc.

[0078] Through the spatio-temporal synchronization alignment mechanism, the data accuracy and reliability can be improved in this embodiment. During the data acquisition process of sensors of different modalities, due to their own hardware characteristics, sampling frequencies, and triggering mechanisms being different, there may be time differences. Time synchronization can ensure that data of different modalities are acquired at the same time point or time interval, avoiding data deviation caused by inconsistent time. The installation positions, angles, and coordinate systems of different sensors are also different, which will also cause spatial deviation in the acquired data. Spatial alignment can unify data of different modalities into the same spatial coordinate system, eliminating errors caused by spatial differences. After time synchronization and spatial alignment, the correlation between multi-modal data is closer, which helps to discover potential information that cannot be found in single-modal data.

[0079] The PCB automatic detection method based on multi-modal fusion and adaptive algorithm proposed by the present invention combines optical imaging, X-ray penetration, and laser three-dimensional modeling technologies to cover full-dimensional defect detection of the PCB surface, internal, and three-dimensional structure. At the same time, it breaks through the traditional fixed-parameter detection mode, realizes dynamic parameter optimization and multi-scenario adaptability, reduces manual intervention, meets the requirements of flexible production, and improves the detection efficiency by more than 30%. By connecting the detection data to the MES system, quality traceability and process optimization closed-loop can be achieved, further realizing the intelligent upgrade of the production line.

[0080] It should be understood that although the steps in the flowcharts involved in the above-mentioned embodiments are shown in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order limit for the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or in turn with at least some of the steps or stages in other steps or other steps.

[0081] Based on the same inventive concept, the embodiment of the present application also provides a PCB detection device based on multi-modal fusion and adaptive algorithm for implementing the above-mentioned PCB detection method based on multi-modal fusion and adaptive algorithm. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the following PCB detection devices based on multi-modal fusion and adaptive algorithm can refer to the limitations on the PCB detection method based on multi-modal fusion and adaptive algorithm in the above text, and will not be repeated here.

[0082] In one embodiment, a PCB detection device based on multimodal fusion and adaptive algorithms is provided, including: a data acquisition module, configured to obtain optical imaging data of a PCB using an optical imaging unit, obtain tomography data of the PCB using an X-ray unit, and obtain three-dimensional point cloud data of the PCB using a laser unit;

[0083] a parameter update module, configured to collect a reference signal of the PCB and update detection parameters of the optical imaging unit, the X-ray unit, and the laser unit according to the optical imaging data, the tomography data, the three-dimensional point cloud data, and the reference signal using an adaptive intelligent algorithm;

[0084] a data update module, configured to re-obtain the optical imaging data, the tomography data, and the three-dimensional point cloud data based on the updated detection parameters and perform feature-level fusion to obtain multimodal fusion data;

[0085] a detection output module, configured to input the multimodal fusion data into a detection model, perform defect comparison in combination with a historical database, and obtain a defect detection result of the PCB.

[0086] Each module in the above-mentioned PCB detection device based on multimodal fusion and adaptive algorithms can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or stored in a memory in a computer device in the form of software, so as to facilitate the processor to call and execute the operations corresponding to the above-mentioned modules.

[0087] In one embodiment, a computer device is provided, including a memory and a processor, where a computer program is stored in the memory, and when the processor executes the computer program, the steps in all the method embodiments described above are implemented.

[0088] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps in all the method embodiments described above are implemented.

[0089] In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in all the method embodiments described above are implemented.

[0090] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant laws, regulations, and standards in relevant countries and regions.

[0091] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0092] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0093] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A PCB detection method based on multimodal fusion and adaptive algorithm, characterized in that, The method includes: Obtaining optical imaging data of the PCB by using an optical imaging unit, obtaining tomographic data of the PCB by using an X-ray unit, and obtaining three-dimensional point cloud data of the PCB by using a laser unit; Collecting a reference signal of the PCB, and updating detection parameters of the optical imaging unit, the X-ray unit, and the laser unit by using an adaptive intelligent algorithm according to the optical imaging data, the tomographic data, the three-dimensional point cloud data, and the reference signal; Re-obtaining the optical imaging data, the tomographic data, and the three-dimensional point cloud data based on the updated detection parameters, and performing feature-level fusion to obtain multimodal fusion data; Inputting the multimodal fusion data into a detection model, and performing defect comparison in combination with a historical database to obtain a defect detection result of the PCB.

2. The method according to claim 1, wherein The update of the detection parameters includes: Obtaining errors between the optical imaging data, the tomographic data, the three-dimensional point cloud data and the reference signal respectively, and iteratively adjusting the detection parameters according to the errors.

3. The method according to claim 2, wherein The method further includes: Taking the detection parameters output in each iteration as current parameters, calculating a performance index under the current parameters, and judging whether the adaptive intelligent algorithm converges according to the performance index; if not, continue to iteratively adjust the detection parameters; if so, stop the iteration when the iteration stop condition is satisfied, and take the current detection parameters as the updated detection parameters.

4. The method according to claim 1, characterized in that After re-obtaining the optical imaging data, the tomographic data, and the three-dimensional point cloud data based on the updated detection parameters, the method further includes: Performing image processing on the optical imaging data to identify first defect features; Adopting a denoising and three-dimensional reconstruction algorithm for the tomographic data, and combining threshold segmentation to locate second defect features; Extracting morphological parameters from the three-dimensional point cloud data after filtering and registration; Combining the first defect features, the second defect features, and the morphological parameters to obtain the multimodal fusion data.

5. The method according to claim 4, characterized in that, The obtaining of the multimodal fusion data includes: Performing feature extraction on the optical imaging data, the tomographic data, and the three-dimensional point cloud data, and introducing an attention mechanism to weight the first defect features, the second defect features, and the morphological parameters to obtain corresponding target features; Establishing a causal relationship between multimodal data through semantic alignment and temporal matching techniques to obtain the multimodal fusion data.

6. The method according to claim 1, characterized in that, The method further includes: Adopting a spatio-temporal synchronization alignment mechanism to obtain the optical imaging data, the tomographic data, and the three-dimensional point cloud data.

7. A PCB detection device based on multimodal fusion and adaptive algorithm, characterized in that, The device includes: A data acquisition module, configured to obtain optical imaging data of the PCB by using an optical imaging unit, obtain tomographic data of the PCB by using an X-ray unit, and obtain three-dimensional point cloud data of the PCB by using a laser unit; A parameter update module, configured to collect a reference signal of the PCB, and update detection parameters of the optical imaging unit, the X-ray unit, and the laser unit by using an adaptive intelligent algorithm according to the optical imaging data, the tomographic data, the three-dimensional point cloud data, and the reference signal; A data update module, configured to re-acquire the optical imaging data, the tomographic data, and the three-dimensional point cloud data based on the updated detection parameters, and perform feature-level fusion to obtain multi-modal fusion data; A detection output module, configured to input the multi-modal fusion data into a detection model, perform defect comparison in combination with a historical database, and obtain a defect detection result of the PCB.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images