A method for locating a region of interest for 3D dynamic scan imaging

Abstract

The application discloses a kind of region of interest positioning methods for 3D dynamic scanning imaging, and processing mechanism is obtained according to the 3D point cloud data obtained by dynamic scanning, after point cloud data filtering, fitting processing, respectively obtain 2D gray image and 3D depth map;2D gray image passes through UNET image segmentation model, for the pin and shell of the chip to be detected, realize the stable semantic segmentation between micro object such as chip pin and chip overall large plane;3D depth data is used to obtain 3D depth map, and the position of the region of interest segmented on the 2D gray image is superimposed, to obtain the depth information map with the region of interest positioning of chip region, pin and the like, facilitate subsequent selection of different positioning interest region to carry out coplanarity analysis and calculation one by one, make high-precision detection based on 3D dynamic scanning imaging adapt to more complex test application environment, improve the precision and stability of batch chip coplanarity detection under the customized transmission carrier of various transmission mechanisms, and tray, card slot and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to chip pin coplanarity detection, in particular to a method for positioning a region of interest for 3D dynamic scanning imaging. BACKGROUND

[0002] In the existing equipment production process, there is a certain probability of physical appearance defects in the packaging of chips and components during the second screening and transfer process, which may affect the quality of equipment and equipment manufacturing. According to long-term statistics in the production process, there is a large probability (more than 40% in the field) of pin warping defects. The lack of chip pins, abnormal pin spacing, and excessive warping may cause quality problems in the chip installation process, seriously affecting the quality and safety of the equipment. In addition, even if it enters the storage link, the long-term storage of plastic encapsulated devices has a "wet and humid" problem, causing pin tinning oxidation, and in severe cases, it may even cause serious safety problems such as bridging short circuit and chip burning. In the standardization documents such as "GJB3243-98 Electronic Component Surface Mounting Requirements", it is clearly stipulated that the pin lead inclination error of the component (pin) should not be greater than 0.08mm, and the coplanarity error of the component (pin) lead should not be greater than 0.1mm. Therefore, in the face of batch devices, how to effectively solve the problem of spatial defect detection of chip pins and confirm that the batch chips used for processing and manufacturing meet the quality standard requirements is of great significance, especially for application scenarios with high safety and reliability requirements.

[0003] Currently, chip pin coplanarity detection commonly uses three methods: human eye inspection, machine vision detection, and contact detection. Manual visual inspection of chip components for coplanarity detection in the production field not only easily leads to missed detection and false detection due to limited operator energy and inattention, but also slows down the detection speed, affecting production tasks. Although traditional machine vision detection is fast and does not miss detection, the detection precision is still relatively low compared to manual detection, and it cannot meet the high-precision requirements of today's chip detection industry. Contact detection not only has a slow detection speed, but also easily damages the chip pins during detection. In order to overcome the problems of low detection efficiency and low detection precision in existing chip coplanarity detection, chip manufacturers and users are seeking high-precision, high-efficiency chip pin coplanarity detection methods or equipment. To overcome the problems of low detection efficiency and low detection precision in existing chip coplanarity detection, chip manufacturers and users are seeking high-precision, batch, and high-efficiency chip pin coplanarity detection methods or equipment. Solving the problem of chip pin coplanarity detection precision is also the trend of the times and has far-reaching significance.

[0004] In actual engineering, for the high-precision and batch detection system of chip products, flexible docking is required in the original complete production process (such as device warehouse screening and manufacturing material preparation links) to adapt to various transmission mechanisms and non-standard detection scenes of application carriers such as trays, how to efficiently and accurately detect multiple interest regions within the field of view, and determine the position of each chip package and pin, is related to the success of batch chip pin coplanarity detection in non-standard application scenarios. SUMMARY

[0005] The purpose of the present application is to provide a method for positioning the region of interest for 3D dynamic scanning imaging, to solve the problem of segmenting and positioning the chip region and pin region of multiple detection interest regions after 3D dynamic scanning imaging, to adapt to batch chip coplanarity detection application scenarios under various transmission mechanisms and customized transmission carriers such as trays and card slots, to improve the precision and stability of chip pin coplanarity detection, and ultimately to improve the quality of chips in application.

[0006] The technical scheme of the present application is a method for positioning the region of interest for 3D dynamic scanning imaging, the principle of which is as follows: a processing mechanism obtains 2D grayscale images and 3D depth maps from 3D point cloud data obtained by dynamic scanning after point cloud data filtering and fitting processing; a UNET image segmentation model is used to realize stable semantic segmentation between micro objects such as chip pins and the overall large plane of the chip for the pins and shell of the chip to be detected; 3D depth data is used to obtain a 3D depth map, and the position of the region of interest segmented by semantic segmentation on the 2D grayscale image is superimposed to obtain a depth information map with chip region and pin region positioning, which facilitates subsequent selection of different positioned interest regions for coplanarity analysis and calculation, so that high-precision detection based on 3D dynamic scanning imaging can adapt to more complex test application environments, and the precision and stability of batch chip coplanarity detection under various transmission mechanisms and customized transmission carriers such as trays and card slots can be improved.

[0007] The method of the present application specifically comprises the following steps:

[0008] (1) calibrate the 3D camera to obtain the parameters (internal and external parameters) of the camera;

[0009] (2) the processing mechanism obtains 2D grayscale images and 3D depth maps from 3D point cloud data obtained by dynamic scanning after point cloud data filtering and fitting processing;

[0010] (3) save the 2D grayscale image as a PNG format picture; use a UNET image segmentation model to realize stable semantic segmentation between micro objects such as chip pins and the overall large plane of the chip for the pins and shell of the chip to be detected, and obtain a ROI (Region of Interest) mapping image according to the segmentation result;

[0011] (4) Obtain the 3D point cloud data of the chip by using the distance information of the chip and the point cloud reconstruction technology, then process and convert the point cloud data, and convert the 3D point cloud of the chip into a 3D depth map based on the calibration parameters (internal and external parameters) of the 3D camera;

[0012] (5) Fuse the 3D depth map and the ROI mapping map, each ROI mapping edge pixel, and according to the 3D depth information, perform moderate inflation under the selected threshold, and different interest areas cannot overlap;

[0013] (6) Output the fused target object depth map, and subsequent coplanarity calculation and defect detection can be performed on each target.

[0014] A computer storage medium, having a computer program stored thereon, the computer program being executed by a processor to implement the above-mentioned region of interest positioning method for 3D dynamic scanning imaging.

[0015] A computer device, comprising a storage, a processor and a computer program stored on the storage and executable on the processor, wherein the processor implements the above-mentioned region of interest positioning method for 3D dynamic scanning imaging when executing the computer program.

[0016] Advantages: Compared with the prior art, the present application has the following advantages:

[0017] 1. The present application adopts a trained UNET depth recognition segmentation model for semantic segmentation in 2D space, realizes region of interest positioning, cooperates with high-precision point cloud extraction of 3D dynamic scanning, realizes dynamic measurement of devices in non-standard regions such as tray grids and guide rails, and has the advantages of high speed and high precision compared with artificial detection, monocular or multi-lens 2D lens is difficult to realize 3D detection in a space occlusion environment, and cannot realize high-precision measurement; compared with contact detection, the speed is fast, and the chip pins are not damaged, and the environmental adaptability is strong, and is not affected by environmental light;

[0018] 2. The present application adopts a UNET depth recognition model for semantic segmentation, which is a relatively mature technology and has been well applied in vision, and the accuracy and feasibility have been verified; the developer can calibrate, train and develop the model for new devices;

[0019] 3, The 3D dynamic scanning of the interest region accurate positioning can be realized, so that the coplanarity of the chip pin under the free test environment is detected online; the test environment is allowed to have higher freedom, which is convenient for integrated into the batch patch process for manufacturing, and has strong adaptability to the use environment, and has wide application space; the batch chip spatial defect detection under the complex test environment including the tray with the slot is supported, the accurate and batch chip detection under the non-standard test tool for the complex application scene is realized, the safety quality detection demand of the chip for the manufacturing link is improved, and the integration of the existing application scene is facilitated. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 The step flow chart of the method of the application;

[0021] Figure 2 The schematic diagram of the target object depth map generated based on the interest region positioning of the 3D dynamic scanning imaging; wherein Figure 2 (a) is a 2D grayscale map, Figure 2 (b) is a target object depth map. DETAILED DESCRIPTION

[0022] The technical solutions of the application will be further described below with reference to the drawings.

[0023] As Figure 1 shown, an interest region positioning method for 3D dynamic scanning imaging includes the following steps:

[0024] (1) calibrate the 3D camera to obtain the parameters (internal and external parameters) of the camera;

[0025] (2) the processing mechanism obtains a 2D grayscale image and a 3D depth map after filtering and fitting processing of the 3D point cloud data obtained by dynamic scanning;

[0026] (3) save the 2D grayscale image as a PNG format picture; through the UNET image segmentation model, the stable semantic segmentation between the small object such as the pin of the chip and the whole large plane of the chip is realized for the pin and the shell of the chip to be detected, and the ROI (Region of Interest) mapping map is obtained according to the segmentation result;

[0027] (4) use the distance information of the chip and the point cloud reconstruction technology to obtain the 3D point cloud data of the chip, then process and convert the point cloud data, and convert the 3D point cloud of the chip into a 3D depth map based on the calibration parameters (internal and external parameters) of the 3D camera;

[0028] (5) The 3D depth map and the ROI mapping map are fused, and the pixels at the edges of each ROI mapping are moderately dilated according to the 3D depth information under a selected threshold, and no overlap can be generated between different interest regions;

[0029] (6) As shown in Figure 2 , the fused target object depth map is output, and subsequent coplanarity calculation and defect detection can be performed on each target.

Claims

1. A method for locating regions of interest in 3D dynamic scanning imaging, characterized in that, Includes the following steps: (1) Calibrate the 3D camera and obtain its parameters; (2) The processing unit obtains a 2D grayscale image and a 3D depth map respectively after filtering and fitting the 3D point cloud data obtained by dynamic scanning. (3) Save the 2D grayscale image as a PNG image; using the UNET image segmentation model, achieve stable semantic segmentation between the pins and the shell of the chip to be detected and the overall large plane of the chip, and obtain the ROI mapping map based on the segmentation results; (4) Obtain the 3D point cloud data of the chip by using the distance information of the chip and point cloud reconstruction technology, and then process and transform the 3D point cloud data obtained by dynamic scanning. Based on the calibration parameters of the 3D camera, convert the 3D point cloud of the chip into a 3D depth map. (5) The 3D depth map and the ROI mapping map are fused together. The pixels at the edge of each ROI mapping map are appropriately dilated according to the 3D depth information at a selected threshold. There should be no overlap between different regions of interest. (6) Output the depth map of the fused target object.

2. The method for locating the region of interest in 3D dynamic scanning imaging according to claim 1, characterized in that, The parameters mentioned in steps (1) and (4) include intrinsic and extrinsic parameters.

3. A computer storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements a region-of-interest localization method for 3D dynamic scanning imaging as described in any one of claims 1-2.

4. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements a region of interest localization method for 3D dynamic scanning imaging as described in any one of claims 1-2.

Images