PCBA whole board dispensing colloid detection method and system based on multi-mode fusion

Abstract

The invention provides a PCBA whole board dispensing colloid detection method and system based on multi-modal fusion, and relates to the technical field of PCBA detection, registration is carried out through a two-dimensional image and three-dimensional point cloud data of a PCBA, RGBD data is obtained, the position of the PCBA is obtained from the RGBD data, point cloud data only containing components and colloid are obtained from the three-dimensional point cloud data, and the PCBA whole board dispensing colloid detection method and system based on multi-modal fusion are obtained. According to the contour and the position of a component and the contour and the position of a colloid obtained from RGBD data, the point cloud data of the component and the point cloud data of the colloid are segmented, the point cloud data of the colloid are clustered by taking the point cloud data center of the component as a clustering center, and the point cloud data of the colloid surrounding the component are obtained. According to the method, the point cloud data of the colloid surrounding the component is analyzed, the quality information of the colloid is obtained, whether the colloid is qualified or not is judged according to the quality information of the colloid, the problems that existing PCBA board dispensing colloid detection is low in precision and poor in effect are solved, and the method is suitable for PCBA board dispensing colloid detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of PCBA board detection, in particular to a PCBA board whole-board glue dispensing gel detection method and system based on multi-modal fusion. BACKGROUND

[0002] The PCBA board is a core component of electronic equipment, and its quality directly affects the performance of electronic products. In the manufacturing process, the reliability of the PCBA board is often improved through the glue dispensing process to ensure its stable operation. With the development of PCBA boards towards high density and multiple devices, the refinement and reliability requirements of glue dispensing gel are gradually increasing.

[0003] At present, the glue dispensing gel detection method of the PCBA board mostly relies on visible light image data. However, visible light imaging is easily affected by factors such as light and gel color, and high-transparency gel is difficult to clearly distinguish in the image. Two-dimensional data cannot obtain height and volume information, which easily leads to missed detection or false detection problems, thereby causing low detection precision and poor effect of the PCBA board glue dispensing gel. SUMMARY

[0004] The technical problem solved by the present application is to provide a PCBA board whole-board glue dispensing gel detection method and system based on multi-modal fusion, which solves the problems of low detection precision and poor effect of the existing PCBA board glue dispensing gel.

[0005] The technical solution adopted by the present application to solve the above technical problem is a PCBA board whole-board glue dispensing gel detection method based on multi-modal fusion, comprising the following steps: S1, acquiring two-dimensional image and three-dimensional point cloud data of the PCBA board; S2, registering the two-dimensional image and the three-dimensional point cloud data to obtain RGBD data; S3, obtaining the position of the PCBA board from the RGBD data; S4, obtaining the three-dimensional point cloud data corresponding to the position of the PCBA board from the three-dimensional point cloud data according to the position of the PCBA board, and eliminating the points corresponding to the PCBA board flat area to obtain the three-dimensional point cloud data with the points corresponding to the PCBA board flat area eliminated; S5, obtaining the contour and position of the component and the contour and position of the gel from the RGBD data; S6, segmenting the point cloud data of the component and the point cloud data of the gel from the three-dimensional point cloud data with the points corresponding to the PCBA board flat area eliminated according to the contour and position of the component and the contour and position of the gel around the component; S7, taking the center of the point cloud data of the component as the clustering center to cluster the point cloud data of the gel to obtain the gel point cloud data around the component; S8, analyze the colloid point cloud data around the component, obtain quality information of the colloid, and determine whether the colloid is qualified through the quality information of the colloid.

[0006] Further, in S2, for the unregistered pixel points in the two-dimensional image, the depth value of the unregistered pixel points is obtained based on the depth information of the adjacent points of the unregistered pixel points.

[0007] Further, in S3, the position of the PCBA board is calibrated by using the four corner points of the PCBA board.

[0008] Further, in S4, the obtaining method of the points corresponding to the PCBA board plane region includes: obtaining a fitting plane through plane fitting in the three-dimensional point cloud data corresponding to the position of the PCBA board, and the points contained in the fitting plane are the points corresponding to the PCBA board plane region.

[0009] Further, in S8, the quality information of the colloid includes colloid distribution, height, volume and surface area.

[0010] Further, the PCBA board whole board point gluing colloid detection method based on multi-modal fusion further includes: S9, displaying the result of determining whether the colloid is qualified through a display interface, and sorting or alarming the PCBA board with unqualified colloid.

[0011] The application also provides a PCBA board whole board point gluing colloid detection system based on multi-modal fusion, which realizes the PCBA board whole board point gluing colloid detection method based on multi-modal fusion as described above, and the system includes a data acquisition module, a data fusion module, a colloid point cloud data extraction module around components and a judgment module; the data acquisition module is used to acquire two-dimensional images and three-dimensional point cloud data of the PCBA board; the data fusion module is used to register the two-dimensional images and the three-dimensional point cloud data to obtain RGBD data; the colloid point cloud data extraction module around components is used to acquire the position of the PCBA board from the RGBD data, acquire the three-dimensional point cloud data corresponding to the position of the PCBA board from the three-dimensional point cloud data according to the position of the PCBA board, eliminate the points corresponding to the PCBA board plane region, obtain the three-dimensional point cloud data after eliminating the points corresponding to the PCBA board plane region, acquire the contour and position of the component and the contour and position of the colloid from the RGBD data, segment the point cloud data of the component and the point cloud data of the colloid from the three-dimensional point cloud data after eliminating the points corresponding to the PCBA board plane region according to the contour and position of the component and the contour and position of the colloid around the component, take the center of the point cloud data of the component as the clustering center, cluster the point cloud data of the colloid, and obtain the colloid point cloud data around the component; and the judgment module is used to analyze the colloid point cloud data around the component, obtain the quality information of the colloid, and determine whether the colloid is qualified through the quality information of the colloid.

[0012] Furthermore, the system also includes a display module, which is used to display the result of judging whether the colloid is qualified through a display interface.

[0013] Furthermore, the system also includes an alarm module, which is used to issue an alarm for PCBA boards with substandard colloids.

[0014] Furthermore, the system also includes a sorting module for sorting PCBA boards that do not meet the colloid quality standards.

[0015] The beneficial effects of this invention are as follows: This invention provides a method and system for detecting adhesive colloids on a PCBA board based on multimodal fusion. It registers two-dimensional images and three-dimensional point cloud data of the PCBA board to obtain RGBD data. The position of the PCBA board is obtained from the RGBD data, and the corresponding three-dimensional point cloud data is obtained from the three-dimensional point cloud data. Points corresponding to the planar areas of the PCBA board are removed, resulting in three-dimensional point cloud data containing only components and colloids. Based on the outlines and positions of the components and colloids obtained from the RGBD data, the point cloud data of the components and colloids are segmented. Using the point cloud data center of the components as the cluster center, the point cloud data of the colloids is clustered to obtain colloid point cloud data surrounding the components. The quality information of the colloids is obtained by analyzing the colloid point cloud data surrounding the components. The quality information of the colloids is used to determine whether the colloids are qualified, thus solving the problems of low accuracy and poor effect in existing PCBA board adhesive colloid detection methods. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a method for detecting adhesive colloids on a PCBA board based on multimodal fusion, provided by the present invention. Detailed Implementation

[0017] This invention addresses the problems of low accuracy and poor performance in existing PCBA board adhesive dispensing detection methods by providing a multi-modal fusion-based method for whole-board adhesive dispensing detection in PCBA boards. Figure 1 As shown, it includes the following steps: S1. Acquire two-dimensional images and three-dimensional point cloud data of the PCBA board.

[0018] S2. Register the 2D image with the 3D point cloud data to obtain RGBD data. Specifically, to improve data accuracy, a high-resolution camera can be used to acquire a 2D image of the PCBA board, and denoising, illumination compensation, and image enhancement can be performed to obtain a high-quality 2D image. A 3D scanning device can be used to acquire 3D point cloud data of the PCBA board, and stitching, denoising, and smoothing can be performed to obtain high-quality 3D point cloud data. For unregistered pixels in the 2D image, the depth value of the unregistered pixel is obtained based on the depth information of its neighboring points. For example, the average depth information of the neighboring points of the unregistered pixel is used as the depth value of the unregistered pixel. Therefore, the depth data in the RGBD data has a certain error.

[0019] S3. Obtain the position of the PCBA board from the RGBD data; specifically, the position of the PCBA board can be marked by the four corner points of the PCBA board.

[0020] S4. Based on the position of the PCBA board, obtain the 3D point cloud data corresponding to the position of the PCBA board from the 3D point cloud data, and remove the points corresponding to the planar area of ​​the PCBA board to obtain the 3D point cloud data with the points corresponding to the planar area of ​​the PCBA board removed. Specifically, since the 3D point cloud data may contain points not on the PCBA board, the points not on the PCBA board are removed by using the position of the PCBA board to obtain the 3D point cloud data corresponding to the position of the PCBA board, that is, the 3D point cloud data only includes the points on the PCBA board. The method of obtaining the points corresponding to the planar area of ​​the PCBA board includes: obtaining a fitted plane by performing plane fitting on the 3D point cloud data corresponding to the position of the PCBA board, and the points contained in the fitted plane are the points corresponding to the planar area of ​​the PCBA board.

[0021] S5. Obtain the outline and position of the components and the outline and position of the colloid from the RGBD data. Specifically, the position and size information of the components can be obtained by parsing the Gerber file and the corresponding BOM table of the PCBA board. The outline and position of the components can be verified. The colloid exists around the components, so the outline and position of the colloid can be verified by the position of the components.

[0022] S6. Based on the outline and position of the components and the outline and position of the colloid surrounding the components, segment the point cloud data of the components and the colloid from the 3D point cloud data of the points corresponding to the plane area of ​​the PCBA board. Specifically, since the depth data in the RGBD data has certain errors, it is converted to 3D point cloud data for subsequent processing to ensure the accuracy of the depth data.

[0023] S7. Using the point cloud data center of the components as the cluster center, cluster the point cloud data of the colloid to obtain the colloidal point cloud data surrounding the components.

[0024] S8. Analyze the colloidal point cloud data surrounding the component to obtain the quality information of the colloidal material. Determine whether the colloidal material is qualified based on the quality information. Specifically, the quality information of the colloidal material includes the distribution, height, volume, and surface area of ​​the colloidal material.

[0025] This invention addresses the problems of low accuracy and poor performance in existing PCBA board adhesive dispensing detection systems by providing a multimodal fusion-based whole-board adhesive dispensing detection system. This system implements the aforementioned multimodal fusion-based whole-board adhesive dispensing detection method for PCBA boards. The system includes a data acquisition module, a data fusion module, an adhesive point cloud data extraction module surrounding components, and a judgment module. The data acquisition module acquires two-dimensional images and three-dimensional point cloud data of the PCBA board. The data fusion module registers the two-dimensional images with the three-dimensional point cloud data to obtain RGBD data. The adhesive point cloud data extraction module extracts the position of the PCBA board from the RGBD data and, based on the position, extracts the appropriate position from the three-dimensional point cloud data. The 3D point cloud data corresponding to the location of the PCBA board is obtained, and the points corresponding to the planar area of ​​the PCBA board are removed to obtain the 3D point cloud data of the removed PCBA board planar area. The outline and position of the components and the outline and position of the colloid are obtained from the RGBD data. Based on the outline and position of the components and the outline and position of the colloid around the components, the point cloud data of the components and the point cloud data of the colloid are segmented from the 3D point cloud data of the removed PCBA board planar area. Using the point cloud data center of the components as the cluster center, the point cloud data of the colloid is clustered to obtain the colloid point cloud data surrounding the components. The judgment module is used to analyze the colloid point cloud data surrounding the components to obtain the quality information of the colloid, and to judge whether the colloid is qualified based on the quality information of the colloid.

[0026] Specifically, in order to facilitate the observation of substandard colloids or PCBA boards containing substandard colloids, the system also includes a display module, which is used to display the result of judging whether the colloid is qualified through a display interface.

[0027] To alert relevant personnel, the system also includes an alarm module, which is used to alert staff when PCBA boards have substandard colloids.

[0028] In order to automatically reject PCBA boards with unqualified colloids, the system also includes a sorting module, which is used to sort PCBA boards with unqualified colloids.

Claims

1. A PCBA board whole-board dispensing glue body detection method based on multi-modal fusion, characterized in that, The method comprises the following steps: S1, acquiring a two-dimensional image and three-dimensional point cloud data of a PCBA board; S2, registering the two-dimensional image and the three-dimensional point cloud data to obtain RGBD data; S3, obtaining the position of the PCBA board from the RGBD data; S4, obtaining three-dimensional point cloud data corresponding to the position of the PCBA board from the three-dimensional point cloud data according to the position of the PCBA board, and removing points corresponding to the flat area of the PCBA board to obtain three-dimensional point cloud data from which the points corresponding to the flat area of the PCBA board are removed; S5, obtaining the contour and position of the component and the contour and position of the glue from the RGBD data; S6, segmenting the point cloud data of the component and the point cloud data of the glue from the three-dimensional point cloud data from which the points corresponding to the flat area of the PCBA board are removed according to the contour and position of the component and the contour and position of the glue around the component; S7, taking the center of the point cloud data of the component as a clustering center to cluster the point cloud data of the glue to obtain glue point cloud data around the component; S8, analyzing the glue point cloud data around the component to obtain quality information of the glue, and determining whether the glue is qualified through the quality information of the glue.

2. The PCBA board whole-board dispensing adhesive detection method based on multi-modal fusion according to claim 1, characterized in that, In S2, for unregistered pixel points in the two-dimensional image, the depth value of the unregistered pixel points is obtained based on the depth information of the adjacent points of the unregistered pixel points.

3. The PCBA board whole-board glue dispensing gel detection method based on multi-modal fusion according to claim 1, characterized in that, In S3, the position of the PCBA board is calibrated by using the positions of the four corner points of the PCBA board.

4. The PCBA board whole-board dispensing adhesive detection method based on multi-modal fusion according to claim 1, characterized in that, In S4, the method for obtaining the points corresponding to the flat area of the PCBA board comprises: fitting a plane in the three-dimensional point cloud data corresponding to the position of the PCBA board to obtain a fitted plane, and the points contained in the fitted plane are the points corresponding to the flat area of the PCBA board.

5. The PCBA board whole-board glue dispensing gel detection method based on multi-modal fusion according to claim 1, characterized in that, In S8, the quality information of the glue comprises the distribution, height, volume and surface area of the glue.

6. The PCBA board whole-board glue body detection method based on multi-modal fusion according to claim 1, characterized in that, The method further comprises: S9, displaying the result of determining whether the glue is qualified through a display interface, and sorting or alarming the PCBA board with unqualified glue.

7. The PCBA board whole-board dispensing glue body detection system based on multi-modal fusion, realizing the PCBA board whole-board dispensing glue body detection method based on multi-modal fusion as claimed in claim 1, characterized in that, The system comprises a data acquisition module, a data fusion module, a colloid point cloud data extraction module surrounding components, and a judgment module; the data acquisition module is used to acquire two-dimensional images and three-dimensional point cloud data of the PCBA board; the data fusion module is used to register the two-dimensional images and the three-dimensional point cloud data to obtain RGBD data; the colloid point cloud data extraction module surrounding components is used to acquire the position of the PCBA board from the RGBD data, acquire the three-dimensional point cloud data corresponding to the position of the PCBA board from the three-dimensional point cloud data according to the position of the PCBA board, eliminate the points corresponding to the PCBA board plane area, obtain the three-dimensional point cloud data with the points corresponding to the PCBA board plane area eliminated, acquire the outline and position of the components and the outline and position of the colloid from the RGBD data, segment the point cloud data of the components and the point cloud data of the colloid from the three-dimensional point cloud data with the points corresponding to the PCBA board plane area eliminated according to the outline and position of the components and the outline and position of the colloid surrounding the components, take the point cloud data center of the components as a clustering center, cluster the point cloud data of the colloid, and obtain the colloid point cloud data surrounding the components; the judgment module is used to analyze the colloid point cloud data surrounding the components, obtain the quality information of the colloid, and determine whether the colloid is qualified through the quality information of the colloid.

8. The PCBA board whole-board glue-detecting system based on multi-modal fusion according to claim 7, characterized in that, The system further comprises a display module, which is used to display the result of determining whether the colloid is qualified through a display interface.

9. The PCBA board whole-board glue-detecting system based on multi-modal fusion according to claim 7, characterized in that, The system further comprises an alarm module, which is used to alarm the PCBA board with unqualified colloid.

10. The PCBA board whole-board glue-detecting system based on multi-modal fusion according to claim 7, characterized in that, The system further comprises a sorting module, which is used to sort the PCBA board with unqualified colloid. The system further comprises a sorting module, which is used to sort the PCBA board with unqualified colloid.

Images