Semiconductor package three-dimensional reconstruction method and device, electronic equipment and storage medium

Abstract

The application provides a semiconductor package three-dimensional reconstruction method and device, electronic equipment and storage medium, and belongs to the technical field of optical detection. The method comprises the following steps: time-division control is performed on multiple DLP projectors to project structured light on a semiconductor package to be measured, and a camera is controlled to collect a group of structured light images under a corresponding visual angle of each DLP projector; phase solving is performed on each group of structured light images to obtain a group of first point cloud data corresponding to the group of structured light images; each group of first point cloud data is segmented to obtain at least one cluster of single-solder-ball point clouds and one cluster of bottom plate point clouds; multiple-source solder-ball point clouds corresponding to each solder ball are fused to obtain first fused point clouds corresponding to each solder ball; each cluster of bottom plate point clouds is fused to obtain second fused point clouds; and three-dimensional reconstruction is performed on the semiconductor package to be measured based on the first fused point clouds corresponding to each solder ball and the second fused point clouds. The application can improve the accuracy of three-dimensional reconstruction of the semiconductor package.

Description

Technical Field

[0001] This application belongs to the field of optical inspection technology, and more specifically, relates to a method and apparatus for three-dimensional reconstruction of semiconductor packages, electronic devices, and storage media. Background Technology

[0002] With the rapid development of the electronics manufacturing industry, especially the widespread application of surface mount technology (SMT), ball grid array (BGA) packaging has become the mainstream form of integrated circuit packaging due to its advantages such as high pin count, small size, and high performance. The quality inspection of solder balls in BGA packaging is a key step in ensuring product reliability. Currently, the commonly used technology for solder ball inspection is structured light technology. Structured light technology projects a specific pattern onto the surface of the object being measured using a digital light processing projector (DLP projector). A camera captures the deformed pattern, and the three-dimensional shape of the object is reconstructed using the principle of triangulation. This technology has advantages such as non-contact, high precision, and fast measurement, and is suitable for detecting parameters such as solder ball height, coplanarity, and volume.

[0003] Due to the limited viewing angle of a single DLP projector, existing technologies typically employ multiple DLP projectors for detection from different angles. For example, CN121509635A discloses a structured light imaging system and method suitable for semiconductor SIP system-in-package. This system symmetrically arranges multiple DLP projectors and side cameras, and the control unit uses a weighted fusion strategy to fuse point clouds from multiple viewing angles to generate a three-dimensional topographic model of the die surface. However, due to the strong reflectivity of the solder ball's metal surface, there is diffuse reflection and superposition of reflected light between adjacent solder balls, resulting in increased noise and contour aliasing in the point cloud at the edge region. This makes it impossible to effectively distinguish the independent contours of adjacent solder balls during the weighted fusion process, leading to deviations in point cloud segmentation and registration. Ultimately, this results in low overall three-dimensional reconstruction accuracy and poor reconstruction effect for the solder ball array.

[0004] Therefore, how to improve existing multi-view point cloud fusion methods to enhance the accuracy of 3D reconstruction of semiconductor packages is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, electronic device, and storage medium for three-dimensional reconstruction of semiconductor packages, so as to achieve effective fusion of data from multiple perspectives and improve the accuracy of three-dimensional reconstruction of semiconductor packages.

[0006] A first aspect of this application is applied to a 3D reconstruction system for semiconductor packages. The system includes multiple DLP projectors, a camera, and a controller. The DLP projectors and the camera are respectively communicatively connected to the controller. The multiple DLP projectors have different projection angles. The method is executed by the controller and includes: The system controls multiple DLP projectors to project structured light onto the semiconductor package under test in a time-division manner, and controls the camera to acquire a set of structured light images from the corresponding viewpoint of each DLP projector. For each set of structured light images, phase calculation is performed based on the set of structured light images to obtain a set of first point cloud data corresponding to the set of structured light images; For each group of first point cloud data, perform point cloud segmentation to obtain at least one cluster of single tin ball point clouds and one cluster of base plate point clouds; Based on the three-dimensional position information corresponding to the single solder ball point cloud of each cluster, the single solder ball point clouds corresponding to the same solder ball are matched and grouped to obtain the multi-source solder ball point cloud corresponding to each solder ball; the multi-source solder ball point clouds corresponding to each solder ball are fused to obtain the first fused point cloud corresponding to each solder ball. The base point clouds of each cluster are fused to obtain a second fused point cloud; The semiconductor package under test is reconstructed in three dimensions based on the first fusion point cloud and the second fusion point cloud corresponding to each solder ball.

[0007] A second aspect of this application embodiment includes a controller disposed in a semiconductor package 3D reconstruction system. The system further includes multiple DLP projectors and cameras, each communicatively connected to the controller. The multiple DLP projectors have different projection angles. The device includes: The image acquisition module is used to control multiple DLP projectors to project structured light onto the semiconductor package under test in a time-division manner, and to control the camera to acquire a set of structured light images from the corresponding viewpoint of each DLP projector. The phase calculation module is used to perform phase calculation on each set of structured light images to obtain a set of first point cloud data corresponding to the set of structured light images. The point cloud segmentation module is used to segment each group of first point cloud data into point clouds to obtain at least one cluster of single solder ball point clouds and one cluster of base plate point clouds. The first fusion module is used to match and group the single solder ball point clouds corresponding to the same solder ball based on the three-dimensional position information corresponding to the single solder ball point clouds of each cluster, so as to obtain the multi-source solder ball point cloud corresponding to each solder ball; and to fuse the multi-source solder ball point clouds corresponding to each solder ball to obtain the first fused point cloud corresponding to each solder ball. The second fusion module is used to fuse the base point clouds of each cluster to obtain a second fused point cloud; The 3D reconstruction module is used to perform 3D reconstruction of the semiconductor package under test based on the first fusion point cloud and the second fusion point cloud corresponding to each solder ball.

[0008] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above-described semiconductor package three-dimensional reconstruction method.

[0009] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method for three-dimensional reconstruction of a semiconductor package.

[0010] The beneficial effects of the semiconductor package 3D reconstruction method and apparatus, electronic device, and storage medium provided in this application are as follows: This application first employs a time-division multiplexing method to project structured light onto the semiconductor package under test (SUT) to obtain multiple sets of first point cloud data from multiple perspectives. Then, each set of first point cloud data is segmented to separate the point cloud of a single solder ball and the point cloud of the substrate. The point clouds of the single solder ball and the point clouds of the substrate from multiple perspectives are then fused separately to avoid aliasing interference between adjacent single solder ball point clouds and mutual interference between single solder ball point clouds and substrate point clouds, resulting in accurate first fused point clouds and second fused point clouds. Based on the accurate first fused point clouds and second fused point clouds, the three-dimensional reconstruction of the SUT is performed, which improves the accuracy of the three-dimensional reconstruction of the SUT. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic flowchart of a three-dimensional reconstruction method for a semiconductor package provided in an embodiment of this application; Figure 2 A schematic diagram of a structured light image provided in an embodiment of this application; Figure 3 A partially enlarged schematic diagram of a set of first point cloud data provided in an embodiment of this application; Figure 4This is a schematic diagram of a single solder ball region and a base plate region obtained based on a second target brightness map, provided in an embodiment of this application. Figure 5 A schematic diagram of a multi-cluster single tin ball point cloud after data interpolation, provided in an embodiment of this application; Figure 6 This is a structural block diagram of a semiconductor package three-dimensional reconstruction device provided in an embodiment of this application; Figure 7 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0014] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0015] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a method for three-dimensional reconstruction of a semiconductor package according to an embodiment of this application. The method is applied to a three-dimensional reconstruction system for semiconductor packages, which includes multiple DLP projectors, a camera, and a controller. The DLP projectors and cameras are communicatively connected to the controller. The multiple DLP projectors have different projection angles. The method is executed by the controller and includes: S101: Time-division control of multiple DLP projectors to project structured light onto the semiconductor package under test, and control of cameras to acquire a set of structured light images from the corresponding viewpoint of each DLP projector.

[0016] In this embodiment, the controller can drive different DLP projectors to project coded structured light onto the semiconductor package under test (SUT) in a preset sequence at different time periods. During the projection of each DLP projector, the controller synchronously controls the camera to acquire the corresponding structured light image, completing multi-view image acquisition in a time-division manner. The time-division working mode ensures that only the structured light of one DLP projector is captured by the camera at any given time, effectively eliminating mutual interference between multiple light sources. The projection angles of each DLP projector are different, and multi-view acquisition can compensate for the data loss problem under a single viewpoint, providing multi-source data support for the subsequent 3D reconstruction of the SUT. S102: For each set of structured light images, perform phase calculation based on the set of structured light images to obtain a set of first point cloud data corresponding to the set of structured light images.

[0017] In this embodiment, based on the phase-height mapping principle of phase-shifted structured light 3D measurement, the coded structured light projected by the DLP projector is modulated by the surface of the semiconductor package under test, resulting in a phase shift. This shift corresponds one-to-one with the 3D height of the semiconductor package surface. Therefore, the DLP projector and camera can be jointly calibrated beforehand to establish a mapping relationship between pixels, phase, and 3D spatial coordinates.

[0018] Based on this, for each set of structured light images, existing methods can be used first. The phase-shifting method is used to calculate the phase of the structured light images to obtain the phase of each pixel.

[0019] For example, during each DLP projector projection, the following steps are performed sequentially. Secondary coded structured light projection, correspondingly, a set of structured light images acquired by the camera includes A structured light image. For example... Figure 2 The image shown is a schematic diagram of one of the structured light images. The white dotted areas represent solder balls, and the black areas represent the substrate.

[0020] Among them, the camera captured the first width ( The grayscale distribution of a structured light image can be represented as: ; in, Indicates the first A structured light image at pixel points grayscale value at that location Represents pixels Background light intensity, Represents pixels Modulated light intensity, This represents the wrapped phase to be solved. , for the first The phase shift corresponding to the structured light image.

[0021] Based on the least squares principle, wrapping phase The solution can be obtained using the following formula:

[0022] when (Four-step phase shift method) When the values ​​are 0, π / 2, π, and 3π / 2 respectively, the corresponding values ​​are... The corresponding values ​​are 0, 1, 0, and -1. Given 1, 0, -1, and 0 respectively, the above formula can be simplified to:

[0023] Furthermore, spatial phase unwrapping or temporal phase unwrapping algorithms are needed to refine the wrapped phase obtained above. The expansion process is performed to obtain a continuously distributed absolute phase. Then, based on the pre-defined pixel-phase-3D spatial coordinate mapping relationship, the pixel coordinates of each pixel are... and its absolute phase Convert to the corresponding three-dimensional coordinates of the spatial coordinates of the points ,in This is the height value of that point. Using the same method, traverse all pixels in the structured light image to obtain a set of first point cloud data. For example... Figure 3 The image shown is a magnified view of a portion of the first set of point cloud data.

[0024] Furthermore, the modulation index of each pixel can be calculated simultaneously with the phase calculation. This is used to characterize the modulation quality and signal-to-noise ratio of the structured light signal at that pixel. Modulation degree The calculation formula is:

[0025] when At that time, the formula for calculating the adjustment system can be simplified to:

[0026] Adjustment system Reflects pixels The higher the contrast of the structured light stripes at a point, the higher the signal-to-noise ratio of the structured light signal and the better the measurement reliability; conversely, if the modulation index is lower... If the modulation index is below a preset threshold (e.g., 0.5), it indicates that the pixel is located in a shadow area, a specular reflection area, or has significant noise, and the corresponding 3D coordinate measurement results are unreliable. In subsequent point cloud processing, the modulation index can be used as a reference. Confidence weighting or low-quality point removal is applied to each coordinate point in the first point cloud data to improve the accuracy and robustness of 3D reconstruction.

[0027] S103: Perform point cloud segmentation on each group of first point cloud data to obtain at least one cluster of single tin ball point clouds and one cluster of base plate point clouds.

[0028] In this embodiment, because the surface of the solder ball is made of a metallic mirror material, it is prone to specular reflection during structured light projection. When the projected light is incident at a specific angle, a bright area will be formed on the surface of the solder ball, causing overexposure of the structured light image captured by the camera. At the same time, the densely arranged solder balls will reflect the reflected light onto the measurement areas of other solder balls. All of the above problems will lead to phase distortion obtained from the structured light image, and thus the corresponding height value will be distorted. To solve the above problems, this embodiment utilizes the characteristic that the 3D height of the BGA package is a continuous curved surface. By calculating the curvature value of the point cloud and combining it with the height threshold, the first point cloud data can be preprocessed to effectively remove the phase distortion area caused by metallic mirror reflection in the first point cloud data, while retaining the real edge and feature information, to obtain the second point cloud data. Point cloud segmentation is performed on each group of second point cloud data to obtain accurate single solder ball point clouds and base plate point clouds.

[0029] For example, the step of preprocessing the first point cloud data based on point cloud curvature and height thresholds includes: Calculate the curvature value of each coordinate point in the first set of point cloud data; Calculate the average brightness of each pixel in a set of structured light images corresponding to the first set of point cloud data to obtain the first target brightness map corresponding to the first set of point cloud data. The first target brightness map is segmented based on a preset first brightness threshold to obtain the outline of the first solder ball region, and the solder ball radius is determined based on the outline of the first solder ball region. Based on the radius of the solder ball, the curvature threshold and the maximum height of the solder ball are determined. The coordinates of the points where the curvature value is greater than the curvature threshold and the height value is greater than the maximum height of the solder ball are taken as height distortion coordinates. The height values ​​corresponding to the height distortion coordinates are removed from the first set of point cloud data to obtain the second point cloud data.

[0030] The curvature value of each coordinate point in the first point cloud data can be calculated using the Local Quadric Surface Fitting Curvature Estimation method. Specifically, to avoid local noise, a quadratic function is fitted to the neighborhood points of each coordinate point in the first point cloud data to obtain the quadratic surface equation:

[0031] in These are the planar coordinates of the point where the curvature value is currently to be calculated. These are the planar coordinates of neighboring points. It is the height value; Calculating the first and second partial derivatives yields: ; Substituting the first and second partial derivatives into the Gaussian partial derivative calculation formula, we can obtain the curvature value. for:

[0032] Furthermore, for each set of first point cloud data, when determining the height threshold, the brightness difference between the solder ball and the base plate in the structured light image is taken into consideration. For example, the solder ball is made of metal, has strong reflectivity, and has a high overall brightness value (usually 180~255), while the base plate is made of plastic substrate, with a lower brightness value and uniform distribution (usually 40~100). Therefore, the first brightness threshold (e.g., 120) can be preset based on empirical values. At the same time, for a set of structured light images corresponding to this set of first point cloud data, the average brightness value of multiple structured light images at each pixel is calculated, and the average brightness value of each pixel is assigned to the corresponding pixel position to obtain the first target brightness map.

[0033] Based on this, the contour of the solder ball region (marked as the first solder ball region) is quickly segmented from the first target brightness map using a fixed threshold segmentation method. The contour diameter of the first solder ball region is measured, and the physical size k of a single pixel is obtained according to the camera resolution. The contour diameter of the first solder ball region is multiplied by k to obtain the physical diameter of the solder ball. The physical diameter of the solder ball is divided by 2 to obtain the radius r of the solder ball.

[0034] Furthermore, considering that the surface of the solder ball is close to a standard sphere, the curvature of the first solder ball region is always 1 / r, while the base plate region is almost flat, with a curvature of almost 0. Therefore, 1 / r can be used as the curvature threshold. Simultaneously, since the solder ball is spherical, using the base plate height of the semiconductor package under test as the reference height, the maximum solder ball height is 2r. Therefore, the maximum solder ball height can be used as the height threshold. When the height value of a certain coordinate point exceeds the maximum solder ball height (2r) and the curvature value exceeds the curvature threshold (1 / r), the coordinate point is determined to be a height-distorted coordinate point. The height value corresponding to the height-distorted coordinate point is removed from the first point cloud data to obtain the second point cloud data.

[0035] In summary, this embodiment first uses the brightness difference between the solder ball and the substrate in the first structured light image to perform simple fixed-threshold segmentation on the first target brightness map using a preset first brightness threshold, quickly extracting the first solder ball region. Then, based on the prior geometric features of the solder ball surface being close to a standard sphere, accurate curvature and height thresholds can be determined without complex calculations. Based on accurate curvature and height thresholds, height distortion points in the first point cloud data can be accurately removed, obtaining high-quality second point cloud data, thereby effectively improving the accuracy of subsequent point cloud segmentation, fusion, and 3D reconstruction.

[0036] For example, after preprocessing multiple sets of first point cloud data based on point cloud curvature values ​​and height thresholds to obtain multiple sets of second point cloud data, for each set of second point cloud data, the following steps can be used to segment the second point cloud data to obtain at least one cluster of single solder ball point clouds and one cluster of base plate point clouds: Remove the pixels corresponding to the height distortion coordinates from the first target brightness map to obtain the second target brightness map; The gray-level distribution histogram of the second target brightness map is statistically analyzed. The second brightness threshold is determined based on the gray-level distribution histogram. The second target brightness map is then segmented based on the second brightness threshold to obtain the second solder ball region and the base plate region. Based on the solder ball radius and the solder ball arrangement parameters, the second solder ball region is segmented to obtain at least one single solder ball region; each single solder ball region is mapped to the second point cloud data to obtain at least one cluster of single solder ball point clouds. Mapping the base plate area onto the second set of point cloud data yields a cluster of base plate point clouds.

[0037] In this embodiment, after removing the height values ​​corresponding to the height value distortion coordinates from the first point cloud data, the pixels corresponding to the height value distortion coordinates can be removed from the first target brightness map based on the mapping relationship between the first point cloud data and the first target brightness map to obtain the second target brightness map. This can remove invalid pixels caused by reflection and noise, thereby improving image quality.

[0038] Based on this, a grayscale distribution histogram of the second target brightness map is calculated, with grayscale levels from 0 to 255 as the horizontal axis and the number of pixels corresponding to each grayscale level as the vertical axis. The Otsu algorithm or an adaptive thresholding algorithm is used to traverse all grayscale levels, automatically selecting the optimal second brightness threshold based on maximizing the inter-class variance between the solder ball region and the substrate region. Thresholding segmentation is then performed on the second target brightness map based on this second brightness threshold to distinguish between the solder ball region (denoted as the second solder ball region) and the substrate region, thus achieving the initial division of the second target brightness map.

[0039] Furthermore, combining the solder ball radius *r* determined in the above steps, and the pre-saved geometric arrangement parameters such as the row and column spacing and array distribution pattern of the solder balls, the contiguous second solder ball region is morphologically refined and segmented using the solder ball radius *r* as a size constraint. The actual size of a single solder ball region is verified through radius constraints, while the minimum interval between adjacent solder balls is limited based on the row and column spacing, and the array distribution pattern is used to lock the solder ball arrangement position, accurately distinguishing the boundaries of adjacent solder balls and splitting them into independent single solder ball regions that do not overlap, effectively avoiding the overlap and adhesion of adjacent solder ball regions. Figure 4 As shown, after dividing the second solder ball region, the resulting individual solder ball regions have clear boundaries and are not connected to each other.

[0040] Finally, the segmented individual solder ball regions and base plate regions are mapped to the second point cloud data, and the regional information at the image level is mapped to the three-dimensional point cloud coordinates, thereby obtaining at least one independent cluster of individual solder ball point clouds and one cluster of base plate point clouds.

[0041] In summary, this embodiment, based on the mapping relationship between structured light images and point cloud data, firstly removes pixels corresponding to height value distortion coordinates from the first target brightness map to obtain a high-quality second target brightness map. Then, it uses the second target brightness map (two-dimensional image) to quickly segment the single solder ball region and the base plate region, and then maps each single solder ball region and base plate region to the second point cloud data to obtain independent single solder ball point clouds and a cluster of base plate point clouds. Compared with methods that directly perform clustering and segmentation on a large amount of three-dimensional point cloud data, the above method requires less computation and is faster.

[0042] S104: Based on the three-dimensional position information corresponding to the single solder ball point cloud of each cluster, the single solder ball point clouds corresponding to the same solder ball are matched and grouped to obtain the multi-source solder ball point cloud corresponding to each solder ball; the multi-source solder ball point clouds corresponding to each solder ball are fused to obtain the first fused point cloud corresponding to each solder ball.

[0043] In this embodiment, spatial registration can be performed on multiple sets of first point cloud data, that is, aligning multiple sets of first point cloud data to the same world coordinate system to eliminate overall attitude error. Then, the three-dimensional coordinates of the center point of each cluster of single solder ball point clouds are extracted. The distance between the three-dimensional coordinates of the center points of single solder ball point clouds under different viewpoints (i.e., in different sets of first point cloud data) is calculated. Multiple clusters of single solder ball point clouds with center point distances less than a preset distance threshold are selected as multi-source solder ball point clouds corresponding to the same solder ball. Then, the multi-source solder ball point clouds corresponding to each solder ball are fused to obtain the first fused point cloud corresponding to each solder ball. The distance threshold can be set according to an empirical value, for example, 0.05mm.

[0044] For example, there can be two DLP projectors, including a first DLP projector and a second DLP projector. Correspondingly, the multi-source solder ball point cloud for each solder ball includes a first solder ball point cloud corresponding to the first DLP projector and a second solder ball point cloud corresponding to the second DLP projector. Based on this, for each solder ball, the multi-source solder ball point clouds corresponding to that solder ball are fused to obtain a first fused point cloud corresponding to that solder ball, including: Register the first and second point clouds of the solder ball corresponding to the solder ball to obtain multiple coordinate point pairs; Calculate the distance between the two coordinate points in each coordinate point pair; Calculate the average distance of multiple coordinate points to their respective locations, and use this as the difference between the first and second tin sphere point clouds; If the difference value is less than or equal to the preset fusion threshold, the first solder ball point cloud and the second solder ball point cloud are weighted and fused to obtain the first fused point cloud corresponding to the solder ball. If the difference value is greater than the preset fusion threshold, then the point cloud belonging to the same solder ball as the first solder ball point cloud is extracted from the first point cloud data corresponding to the first DLP projector, and is used as the first original point cloud. The point cloud belonging to the same solder ball as the second solder ball point cloud is extracted from the first point cloud data corresponding to the second DLP projector, and is used as the second original point cloud. The signal-to-noise ratio of the first original point cloud and the signal-to-noise ratio of the second original point cloud are calculated. The original point cloud with the larger signal-to-noise ratio is selected as the first fused point cloud.

[0045] In this embodiment, the first solder ball point cloud and the second solder ball point cloud can be registered. Based on this, coordinate points with similar spatial positions in the two sets of point clouds are matched to obtain multiple coordinate point pairs. Taking one coordinate point pair as an example, assume that the two coordinate points in the coordinate pair are ( , , )and( , , ),in,( , )and( , All of these are planar coordinates. Represents planar coordinates ( , The corresponding height value, Represents planar coordinates ( , The corresponding height value indicates the distance between the two coordinate points. for: ; Furthermore, the average value of the distances between multiple coordinate points is calculated. This average value can quantify the overall difference between the first and second tin ball point clouds. The larger the average value, the greater the overall difference between the first and second tin ball point clouds.

[0046] The fusion process of the first and second tin sphere point clouds is described below in two scenarios: The first case: When the difference value is less than or equal to the preset fusion threshold (e.g., 0.02mm), it indicates that the overall shape deviation between the first solder ball point cloud and the second solder ball point cloud is small, the consistency of effective information is high, and the noise interference is weak. At this time, a weighted fusion method can be used to integrate the effective three-dimensional information of the two sets of point clouds, thereby making up for the problem of missing local point clouds under single view and improving the integrity of the three-dimensional reconstruction of solder balls.

[0047] For example, when performing weighted fusion of the first and second solder ball point clouds, the weights of the two point clouds can be determined based on their signal-to-noise ratios (SNR). Taking the first solder ball point cloud as an example, when calculating its SNR, the modulation index of the pixels corresponding to each coordinate point in the first solder ball point cloud can be obtained first. Multiple modulation schemes corresponding to the first tin sphere point cloud were obtained. Then calculate multiple modulation schemes. The average value is used as the signal-to-noise ratio of the first tin sphere point cloud.

[0048] Using the same method, the signal-to-noise ratio (SNR) of the second tin sphere point cloud can be calculated. A higher SNR for a given tin sphere point cloud indicates less noise and more abundant effective 3D information, thus requiring a larger weight. Conversely, a lower SNR indicates more significant interference from reflections and noise, resulting in a lower proportion of effective information, and thus requiring a smaller weight. This weighting method allows tin sphere point clouds with higher SNRs to have a larger proportion in the fusion result, thereby improving the reliability of the first fused point cloud.

[0049] For example, the weights corresponding to the first and second solder ball point clouds can be determined using the following formula: ; in, This indicates the weight corresponding to the first tin ball point cloud. This indicates the weight corresponding to the second tin ball point cloud. This represents the signal-to-noise ratio of the first tin sphere point cloud. This represents the signal-to-noise ratio of the second tin sphere point cloud.

[0050] For example, when performing weighted fusion of the first and second solder ball point clouds, the weights of the two point clouds can be determined based on their completeness. Taking the first solder ball point cloud as an example, the following steps can be used to calculate its completeness: (1) Count the number of coordinate points with missing height values ​​in the first solder ball point cloud and the total number of coordinate points in the first solder ball point cloud. Among them, in the process of converting the structured light image into the first point cloud data, some coordinate points will have missing height values, and in the process of preprocessing the first point cloud data, coordinate points with distorted height values ​​will be removed. These will all result in coordinate points with missing height values ​​in the first solder ball point cloud.

[0051] (2) Calculate the difference between the total number and the number of coordinate points with missing height values ​​as the number of valid coordinate points, and calculate the ratio between the number of valid coordinate points and the total number to obtain the completeness of the first tin ball point cloud.

[0052] Using the same method as described above, the completeness of the second solder ball point cloud can be calculated. A higher completeness indicates fewer missing points and more comprehensive morphological information, thus requiring a larger weight. Conversely, a lower completeness indicates more missing points due to reflections and phase distortion, resulting in less effective information, and thus a smaller weight. This weighting method allows for a higher proportion of more complete solder ball point clouds in the fusion result, improving the reliability of the first fused point cloud.

[0053] For example, the signal-to-noise ratio (SNR) and completeness of the first solder ball point cloud can be weighted and summed to obtain a first comprehensive evaluation value, and the SNR and completeness of the second solder ball point cloud can be weighted and summed to obtain a second comprehensive evaluation value. A higher comprehensive evaluation value for a given solder ball point cloud indicates less noise, more sufficient effective 3D information, and fewer shape defects, thus requiring a larger weight. Conversely, a lower comprehensive evaluation value indicates more significant interference from reflections and noise, a lower proportion of effective information, and more severe point cloud defects, thus requiring a smaller weight. This weighting method simultaneously considers both SNR and completeness, further improving the reliability of the first fused point cloud.

[0054] In summary, by using the above weight allocation method, the three-dimensional topographic information of high-quality point clouds can be preserved to the maximum extent, while taking into account the effective supplementary data of another set of tin ball point clouds, making up for the local deficiencies of single-view tin ball point clouds, and further improving the integrity and measurement accuracy of the first fused point cloud.

[0055] The second scenario: If the difference value is greater than the fusion threshold, it indicates that either the first or second solder ball point cloud is significantly distorted. Directly fusing the first and second solder ball point clouds would introduce erroneous data. In this case, we can backtrack to the first point cloud data, extract the first original point cloud corresponding to the first solder ball point cloud, and the second original point cloud corresponding to the second solder ball point cloud, and determine the first fused point cloud based on the first and second original point clouds.

[0056] For example, the same method as in the above embodiments can be used to calculate the signal-to-noise ratio (SNR) of the first and second original point clouds. A higher SNR in the original point cloud indicates a higher proportion of effective 3D information and less noise and distortion. Therefore, selecting the original point cloud with a higher SNR as the first fused point cloud can avoid interference from distorted point clouds and ensure the accuracy of the tin sphere 3D reconstruction.

[0057] For example, the same method as the above embodiments can be used to calculate the completeness of the first original point cloud and the second original point cloud respectively. The higher the completeness of the original point cloud, the fewer the missing areas of the point cloud and the more comprehensive the morphological information. Therefore, selecting the original point cloud with higher completeness as the first fused point cloud can reduce the impact of missing points on the three-dimensional reconstruction results and improve the completeness of the tin sphere morphology representation.

[0058] For example, the same method as in the above embodiments can be used to weighted sum the signal-to-noise ratio and completeness of the first original point cloud to obtain a third comprehensive evaluation value, and weighted sum the signal-to-noise ratio and completeness of the second original point cloud to obtain a fourth comprehensive evaluation value. The higher the comprehensive evaluation value of the original point cloud, the more it indicates that the original point cloud has the advantages of both low noise and high completeness. Therefore, selecting the original point cloud with a higher comprehensive evaluation value as the first fused point cloud can simultaneously take into account measurement accuracy and topographic completeness, further improving the overall quality of the tin sphere 3D reconstruction.

[0059] In summary, when determining the first fused point cloud based on the first solder ball point cloud and the second solder ball point cloud in this embodiment, the overall difference value between the first solder ball point cloud and the second solder ball point cloud is first quantified, and different fusion methods are selected based on the relative magnitude of the overall difference value and the fusion threshold to obtain the first fused point cloud. This can improve the accuracy of the first fused point cloud and thus improve the accuracy of the 3D reconstruction of the solder ball.

[0060] Furthermore, after obtaining the first fused point cloud, Gaussian smoothing can be applied to filter out the reconstructed height direction noise caused by camera random noise and thermal noise. After noise reduction, the height of the same solder ball can be repeatedly measured under the same measurement conditions, and the fluctuation error of the measurement results can be controlled within ±3μm.

[0061] S105: Merge the point clouds of each cluster of base plates to obtain the second merged point cloud.

[0062] In this embodiment, considering that the base plate is almost planar, the point clouds of each cluster of base plates can be weighted and fused to obtain a second fused point cloud. Further, the base plate plane can be fitted based on the second fused point cloud to obtain a theoretical reference height. The difference between the height value of each coordinate point in the second fused point cloud and the theoretical reference height is calculated. Coordinate points with a difference greater than a preset error threshold (e.g., 2~5μm) are identified as abnormal noise points with large height deviations. Mean filtering is used to smooth and reduce noise and remove these abnormal noise points. Finally, for coordinate points in the second fused point cloud that lack height values, interpolation is performed using the height values ​​of the effective point clouds in the neighborhood to fill in the missing height values, ultimately obtaining a flat and stable second fused point cloud.

[0063] It should be noted that step S104 can be performed before, after, or simultaneously with step S105. Figure 1 The execution method described is merely an example and is not intended to be limiting.

[0064] S106: Perform three-dimensional reconstruction of the semiconductor package under test based on the first fusion point cloud and the second fusion point cloud corresponding to each solder ball.

[0065] In this embodiment, the first fusion point cloud and the second fusion point cloud corresponding to each solder ball are spatially registered and stitched together. By integrating the three-dimensional shape of each solder ball and the reference plane information of the base plate, a complete and continuous three-dimensional model of the semiconductor package under test can be constructed.

[0066] As can be seen from the above, this embodiment first adopts a time-division projection of structured light onto the semiconductor package under test to obtain multiple sets of first point cloud data from multiple perspectives; then, each set of first point cloud data is segmented to separate the single solder ball point cloud and the substrate point cloud, and the single solder ball point cloud and the substrate point cloud from multiple perspectives are fused separately, which can avoid the aliasing interference between adjacent single solder ball point clouds and the mutual interference between the single solder ball point cloud and the substrate point cloud, and obtain accurate first fused point cloud and second fused point cloud; the accuracy of the three-dimensional reconstruction of the semiconductor package under test can be improved based on the accurate first fused point cloud and second fused point cloud.

[0067] In one embodiment of this application, before matching and grouping the point clouds of individual solder balls corresponding to the same solder ball, data interpolation can be performed on the missing height values ​​in the point cloud of the individual solder ball to fill in the point cloud holes caused by solder ball reflection and overexposure, making the point cloud of the individual solder ball more complete and avoiding center point offset caused by missing points, thereby improving the accuracy of matching and grouping.

[0068] For example, the step of data interpolation for missing height values ​​in a single tin ball point cloud may include: Identify the missing height values ​​in the point cloud of each cluster of individual tin spheres; For each cluster of single tin ball point clouds, a spherical model is obtained by performing spherical fitting on the coordinates of the points other than the missing height values ​​in the single tin ball point cloud of the cluster. Based on the spherical model, data interpolation is performed on the missing height values ​​in the single tin ball point cloud of the cluster to obtain the interpolated single tin ball point cloud.

[0069] In this embodiment, the point cloud of each cluster of single solder balls can be investigated first to identify the coordinate points with missing height values ​​and clarify the target points that need to be interpolated, i.e., the points with missing height values. Then, for each cluster of single solder ball point clouds, a spherical fitting is performed on the coordinate points other than the points with missing height values ​​(i.e., the effective point cloud) in the single solder ball point cloud of that cluster to obtain a spherical model that fits the actual shape of the solder ball. Based on the spherical model, the data interpolation of the missing height values ​​in the single solder ball point cloud of that cluster can be performed to obtain the single solder ball point cloud after data interpolation.

[0070] For example, the spherical equation can be used. Perform spherical fitting, where, ( Let be the three-dimensional coordinates of any point in the first tin ball point cloud. For planar coordinates, For height value, ( Let the coordinates be the center of the sphere. R Where is the radius of the sphere, ( and R There are four fitting parameters. For each cluster of single tin sphere point clouds, existing spherical least squares fitting algorithms can be used to solve the problem, resulting in four fitting parameters ( and The specific values ​​are then used to obtain the spherical model.

[0071] Based on the obtained spherical model, for each missing height value in the point cloud of a single tin ball in each cluster, data interpolation can be performed on the missing height value based on the spherical model. The specific steps include: If the radius of the sphere in the spherical model is not less than a preset first radius threshold and not greater than a preset second radius threshold, then the planar coordinates of the missing height value point are substituted into the spherical model to obtain the interpolated height value of the missing height value point; the first radius threshold is less than the second radius threshold. If the radius of the sphere in the spherical model is less than the preset first radius threshold, or the radius of the sphere in the spherical model is greater than the preset second radius threshold, then the average height value of the neighborhood coordinates of the missing height value is calculated and used as the interpolated height value of the missing height value. Data interpolation is performed on the missing height points based on the interpolated height values.

[0072] In this embodiment, a first radius threshold and a second radius threshold can be preset. If the spherical radius obtained by the above spherical fitting is... R If the value is between the first radius threshold and the second radius threshold, it indicates that the spherical fitting result is reliable. At this point, the planar coordinates of the missing height values ​​can be obtained (e.g., (…)). ， Substituting into the spherical model, we obtain the interpolated height value of the missing height point. Otherwise, if the spherical radius obtained by the above spherical fitting... R If the radius is less than the first radius threshold, it indicates that the fitted sphere radius is... R If the fitted radius of the sphere is too small, R A radius greater than the second radius threshold indicates that the fitted spherical radius is... R If the height is too large, the average height value of the neighborhood coordinates of the missing height point (e.g., the average height value of four neighborhood coordinates) can be calculated as the interpolated height value for the missing height point. Those skilled in the art can determine the first radius threshold and the second radius threshold based on the actual radius of the solder balls in the BGA package. For example, for a standard BGA solder ball with a nominal radius of 0.25mm, considering the solder ball processing tolerance and three-dimensional measurement error, the first radius threshold can be set to 0.20mm and the second radius threshold to 0.30mm.

[0073] As can be seen from the above, this embodiment performs threshold verification on the fitted spherical radius, distinguishes between effective fitting and distorted fitting based on the verification results, and adopts corresponding interpolation strategies for effective fitting and distorted fitting to interpolate the height values ​​of missing height points. This can effectively avoid the spherical fitting distortion problem caused by outliers, improve the robustness and accuracy of point cloud interpolation, and ensure the integrity of the point cloud morphology of a single tin ball and the stability of measurement accuracy.

[0074] In actual calculations, to improve interpolation efficiency, this embodiment first projects the point cloud of each individual solder ball vertically onto the XY plane, and then projects it onto each (…). The height is determined by representing the height in the z-direction using pixel grayscale values, resulting in a corresponding height map. Then, a least-squares interpolation algorithm based on the Graphics Processing Unit (GPU) is used to uniformly divide the height map into blocks of a predetermined size (e.g., 64×64 or 128×128 pixels), cached in the GPU cache, and the effective neighborhood is obtained using the Random Sample Consensus (RANSC) method. Finally, a spherical least-squares fitting algorithm is used to solve for the relationship between the sphere radius and z. Interpolating 1000 solder balls takes only 5 milliseconds. Figure 5 As shown, the interpolated single solder ball has a full spherical surface with virtually no holes.

[0075] In one embodiment of this application, when there are two DLP projectors, the two DLP projectors can be symmetrically arranged on both sides of the camera, and the projection optical axis of each DLP projector forms an angle of 15 to 30 degrees with the optical axis of the camera.

[0076] In this embodiment, a structured light hardware architecture with a single camera and dual DLP projectors is adopted. The two DLP projectors are symmetrically arranged on both sides of the camera, forming an angle of 15-30 degrees with the camera's optical axis to achieve triangulation. Compared with the traditional four-DLP architecture, the system complexity is reduced and the acquisition speed is improved, which can meet the requirements of high-speed detection of BGA chips (50,000-80,000 chips / hour).

[0077] In summary, this embodiment first employs a time-division multiplexing method to project structured light onto the semiconductor package under test (SUT) to obtain multiple sets of first point cloud data from multiple perspectives. Then, each set of first point cloud data is segmented to separate the point cloud of a single solder ball and the point cloud of the substrate. The point clouds of the single solder ball and the substrate from multiple perspectives are then fused separately. This avoids aliasing interference between adjacent single solder ball point clouds and mutual interference between the single solder ball point cloud and the substrate point cloud, resulting in accurate first and second fused point clouds. Based on these accurate first and second fused point clouds, the 3D reconstruction of the SUT is performed, improving the accuracy of the 3D reconstruction.

[0078] Furthermore, in the specific implementation process, this embodiment can also achieve the following beneficial effects: (1) In this embodiment, when performing point cloud segmentation on each group of first point cloud data, based on the brightness difference between the solder ball and the base plate, a preset first brightness threshold is used to perform simple fixed threshold segmentation on the first target brightness map, quickly extracting the first solder ball region. Then, based on the prior geometric features that the solder ball surface is close to a standard sphere, accurate curvature threshold and height threshold can be determined without complex calculations. Based on accurate curvature threshold and height threshold, height distortion points in the first point cloud data can be accurately removed to obtain high-quality second point cloud data. Point cloud segmentation based on high-quality second point cloud data can improve the accuracy of subsequent point cloud segmentation, fusion, and 3D reconstruction.

[0079] (2) In this embodiment, when fusing single solder ball point clouds (e.g., first solder ball point cloud and second solder ball point cloud) from multiple perspectives, the overall difference value of the first solder ball point cloud and the second solder ball point cloud is first quantified. Based on the relative size of the overall difference value and the fusion threshold, different fusion methods are selected to obtain the first fused point cloud, which can improve the accuracy of the first fused point cloud and thus improve the accuracy of the three-dimensional reconstruction of the solder ball.

[0080] (3) In this embodiment, before matching and grouping the single solder ball point cloud under multiple perspectives, the missing height values ​​in the single solder ball point cloud are interpolated by fitting a spherical model. During the interpolation process, the radius of the fitted sphere is threshold-checked, and the effective fitting and the distorted fitting are distinguished based on the check results. The corresponding interpolation strategy is adopted for the effective fitting and the distorted fitting, and the height values ​​of the missing height values ​​are interpolated. This can effectively avoid the spherical fitting distortion problem caused by abnormal points, improve the robustness and accuracy of point cloud interpolation, and ensure the integrity of the single solder ball point cloud morphology and the stability of measurement accuracy.

[0081] Corresponding to the semiconductor package 3D reconstruction method in the above embodiment, Figure 6 This is a structural block diagram of a semiconductor package 3D reconstruction apparatus according to an embodiment of this application. The apparatus is housed in a controller, which is integrated into a semiconductor package 3D reconstruction system. The system further includes multiple DLP projectors and cameras, each communicatively connected to the controller. The multiple DLP projectors have different projection angles. For ease of explanation, only the parts relevant to the embodiment of this application are shown. References Figure 6 The semiconductor package 3D reconstruction device 20 includes: an image acquisition module 21, a phase calculation module 22, a point cloud segmentation module 23, a first fusion module 24, a second fusion module 25, and a 3D reconstruction module 26.

[0082] Among them, the image acquisition module 21 is used to control multiple DLP projectors to project structured light onto the semiconductor package under test in a time-division manner, and to control the camera to acquire a set of structured light images from the corresponding viewpoint of each DLP projector. Phase calculation module 22 is used to perform phase calculation on each set of structured light images to obtain a set of first point cloud data corresponding to the set of structured light images. The point cloud segmentation module 23 is used to segment each group of first point cloud data into point clouds to obtain at least one cluster of single tin ball point clouds and one cluster of base plate point clouds. The first fusion module 24 is used to match and group the single solder ball point clouds corresponding to the same solder ball based on the three-dimensional position information corresponding to the single solder ball point clouds of each cluster, so as to obtain the multi-source solder ball point cloud corresponding to each solder ball; and to fuse the multi-source solder ball point clouds corresponding to each solder ball to obtain the first fused point cloud corresponding to each solder ball. The second fusion module 25 is used to fuse the point clouds of each cluster of base plates to obtain a second fused point cloud; The 3D reconstruction module 26 is used to perform 3D reconstruction of the semiconductor package under test based on the first fusion point cloud and the second fusion point cloud corresponding to each solder ball.

[0083] In one embodiment of this application, for each group of first point cloud data, when performing point cloud segmentation on the group of first point cloud data to obtain at least one cluster of single solder ball point clouds and one cluster of base plate point clouds, the point cloud segmentation module 23 is specifically used for: Calculate the curvature value of each coordinate point in the first set of point cloud data; Calculate the average brightness of each pixel in a set of structured light images corresponding to the first set of point cloud data to obtain the first target brightness map corresponding to the first set of point cloud data. The first target brightness map is segmented based on a preset first brightness threshold to obtain the outline of the first solder ball region, and the solder ball radius is determined based on the outline of the first solder ball region. Based on the radius of the solder ball, the curvature threshold and the maximum height of the solder ball are determined. The coordinates of the curvature values ​​that are greater than the curvature threshold and the height values ​​that are greater than the maximum height of the solder ball are taken as height distortion coordinates. The height values ​​corresponding to the height distortion coordinates are removed from the first set of point cloud data to obtain the second point cloud data. The second set of point cloud data was segmented to obtain at least one cluster of single tin ball point clouds and one cluster of base plate point clouds.

[0084] In one embodiment of this application, when performing point cloud segmentation on each group of second point cloud data to obtain at least one cluster of single solder ball point clouds and one cluster of base plate point clouds, the point cloud segmentation module 23 is further used for: Remove the pixels corresponding to the height distortion coordinates from the first target brightness map to obtain the second target brightness map; The gray-level distribution histogram of the second target brightness map is statistically analyzed. The second brightness threshold is determined based on the gray-level distribution histogram. The second target brightness map is then segmented based on the second brightness threshold to obtain the second solder ball region and the base plate region. Based on the solder ball radius and the solder ball arrangement parameters, the second solder ball region is segmented to obtain at least one single solder ball region; each single solder ball region is mapped to the second point cloud data to obtain at least one cluster of single solder ball point clouds. Mapping the base plate area onto the second set of point cloud data yields a cluster of base plate point clouds.

[0085] In one embodiment of this application, the first fusion module 24 is specifically used for: Identify the missing height values ​​in the point cloud of each cluster of individual tin spheres; For each cluster of single tin ball point clouds, a spherical model is obtained by performing spherical fitting on the coordinates of the points other than the missing height values ​​in the single tin ball point cloud of the cluster; and data interpolation is performed on the missing height values ​​in the single tin ball point cloud of the cluster based on the spherical model to obtain the interpolated single tin ball point cloud. The point clouds of individual solder balls, after data interpolation for the same solder ball, are matched and grouped.

[0086] In one embodiment of this application, when interpolating data for each missing height value point in the point cloud of each cluster of single tin spheres based on a spherical model, the first fusion module 24 is further configured to: If the radius of the sphere in the spherical model is not less than a preset first radius threshold and not greater than a preset second radius threshold, then the planar coordinates of the missing height value point are substituted into the spherical model to obtain the interpolated height value of the missing height value point; the first radius threshold is less than the second radius threshold. If the radius of the sphere in the spherical model is less than the preset first radius threshold, or the radius of the sphere in the spherical model is greater than the preset second radius threshold, then the average height value of the neighborhood coordinates of the missing height value is calculated and used as the interpolated height value of the missing height value. Data interpolation is performed on the missing height points based on the interpolated height values.

[0087] In one embodiment of this application, there are two DLP projectors, which are symmetrically arranged on both sides of the camera, and the projection optical axis of each DLP projector forms an angle of 15 to 30 degrees with the optical axis of the camera.

[0088] In one embodiment of this application, for each solder ball, when fusing the multi-source solder ball point clouds corresponding to that solder ball to obtain the first fused point cloud corresponding to that solder ball, the first fusion module 24 is specifically used for: Register the first and second point clouds of the solder ball corresponding to the solder ball to obtain multiple coordinate point pairs; Calculate the distance between the two coordinate points in each coordinate point pair; Calculate the average distance of multiple coordinate points to their respective locations, and use this as the difference between the first and second tin sphere point clouds; If the difference value is less than or equal to the preset fusion threshold, the first solder ball point cloud and the second solder ball point cloud are weighted and fused to obtain the first fused point cloud corresponding to the solder ball. If the difference value is greater than the preset fusion threshold, then the point cloud belonging to the same solder ball as the first solder ball point cloud is extracted from the first point cloud data corresponding to the first DLP projector, and is used as the first original point cloud. The point cloud belonging to the same solder ball as the second solder ball point cloud is extracted from the first point cloud data corresponding to the second DLP projector, and is used as the second original point cloud. The signal-to-noise ratio of the first original point cloud and the signal-to-noise ratio of the second original point cloud are calculated. The original point cloud with the larger signal-to-noise ratio is selected as the first fused point cloud.

[0089] See Figure 7 , Figure 7 This is a schematic block diagram of an electronic device provided in an embodiment of this application. The electronic device may be a controller. Figure 7 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of the modules in the aforementioned device embodiments, for example... Figure 6 The functions of the image acquisition module 21, phase calculation module 22, point cloud segmentation module 23, first fusion module 24, second fusion module 25, and three-dimensional reconstruction module 26 are shown.

[0090] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0091] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0092] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store preset constants such as a first brightness threshold, a first radius threshold, a second radius threshold, and a fusion threshold.

[0093] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation method described in the three-dimensional reconstruction method of semiconductor packages provided in the embodiments of this application, or they can execute the implementation method of the electronic device described in the embodiments of this application, which will not be repeated here.

[0094] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0095] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0096] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0097] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0098] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces or units, or they may be electrical, mechanical, or other forms of connection.

[0099] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0100] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0101] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for three-dimensional reconstruction of a semiconductor package, characterized in that, An application is made in a 3D reconstruction system for semiconductor packages. The system includes multiple DLP projectors, a camera, and a controller. The DLP projectors and the camera are communicatively connected to the controller. The multiple DLP projectors have different projection angles. The method is executed by the controller and includes: The system controls multiple DLP projectors to project structured light onto the semiconductor package under test in a time-division manner, and controls the camera to acquire a set of structured light images from the corresponding viewpoint of each DLP projector. For each set of structured light images, phase calculation is performed based on the set of structured light images to obtain a set of first point cloud data corresponding to the set of structured light images; For each group of first point cloud data, perform point cloud segmentation to obtain at least one cluster of single tin ball point clouds and one cluster of base plate point clouds; Based on the three-dimensional position information corresponding to the single solder ball point cloud of each cluster, the single solder ball point clouds corresponding to the same solder ball are matched and grouped to obtain the multi-source solder ball point cloud corresponding to each solder ball; the multi-source solder ball point clouds corresponding to each solder ball are fused to obtain the first fused point cloud corresponding to each solder ball. The base point clouds of each cluster are fused to obtain a second fused point cloud; The semiconductor package under test is reconstructed in three dimensions based on the first fusion point cloud and the second fusion point cloud corresponding to each solder ball.

2. The method for three-dimensional reconstruction of semiconductor packages as described in claim 1, characterized in that, For each set of first point cloud data, the first point cloud data of the set is segmented to obtain at least one cluster of single solder ball point clouds and one cluster of base plate point clouds, including: Calculate the curvature value of each coordinate point in the first set of point cloud data; Calculate the average brightness of each pixel in a set of structured light images corresponding to the first set of point cloud data to obtain the first target brightness map corresponding to the first set of point cloud data. The first target brightness map is segmented based on a preset first brightness threshold to obtain the outline of the first solder ball region, and the solder ball radius is determined based on the outline of the first solder ball region. Based on the solder ball radius, a curvature threshold and a maximum solder ball height are determined. Coordinate points with curvature values ​​greater than the curvature threshold and height values ​​greater than the maximum solder ball height are taken as height distortion coordinate points. The height values ​​corresponding to the height distortion coordinate points are removed from the first set of point cloud data to obtain the second point cloud data. The second set of point cloud data was segmented to obtain at least one cluster of single tin ball point clouds and one cluster of base plate point clouds.

3. The method for three-dimensional reconstruction of semiconductor packages as described in claim 2, characterized in that, The step of segmenting the second set of point cloud data to obtain at least one cluster of single tin ball point clouds and one cluster of base plate point clouds includes: The second target brightness map is obtained by removing the pixels corresponding to the height distortion coordinates from the first target brightness map; The grayscale distribution histogram of the second target brightness map is statistically analyzed. A second brightness threshold is determined based on the grayscale distribution histogram. The second target brightness map is then segmented based on the second brightness threshold to obtain the second solder ball region and the base plate region. Based on the solder ball radius and the solder ball arrangement parameters, the second solder ball region is segmented to obtain at least one single solder ball region; each single solder ball region is mapped to the group of second point cloud data to obtain the at least one cluster of single solder ball point clouds; The base plate region is mapped to the second set of point cloud data to obtain the cluster of base plate point clouds.

4. The method for three-dimensional reconstruction of semiconductor packages as described in claim 1, characterized in that, The step of matching and grouping the point clouds of individual solder balls corresponding to the same solder ball includes: Identify the missing height values ​​in the point cloud of each cluster of individual tin spheres; For each cluster of single solder ball point clouds, a spherical model is obtained by performing spherical fitting on the coordinates of the single solder ball point clouds in the cluster, excluding the missing height values. Based on the spherical model, data interpolation is performed on the missing height values ​​of the single solder ball point clouds in the cluster to obtain the interpolated single solder ball point cloud. The point clouds of individual solder balls, after data interpolation for the same solder ball, are matched and grouped.

5. The method for three-dimensional reconstruction of semiconductor packages as described in claim 4, characterized in that, For each missing height value point in the point cloud of a single tin ball in each cluster, the step of data interpolation based on the spherical model for that missing height value point includes: If the radius of the sphere in the spherical model is not less than a preset first radius threshold and not greater than a preset second radius threshold, then the planar coordinates of the missing height value point are substituted into the spherical model to obtain the interpolated height value of the missing height value point; the first radius threshold is less than the second radius threshold. If the radius of the sphere in the spherical model is less than a preset first radius threshold, or the radius of the sphere in the spherical model is greater than a preset second radius threshold, then the average height value of the neighborhood coordinates of the missing height value is calculated as the interpolated height value of the missing height value. Data interpolation is performed on the missing height points based on the interpolated height values.

6. The method for three-dimensional reconstruction of semiconductor packages as described in claim 1, characterized in that, There are two DLP projectors, which are symmetrically arranged on both sides of the camera, and the projection optical axis of each DLP projector forms an angle of 15 to 30 degrees with the optical axis of the camera.

7. The method for three-dimensional reconstruction of semiconductor packages as described in claim 2, characterized in that, The number of DLP projectors is two, namely a first DLP projector and a second DLP projector. The multi-source solder ball point cloud corresponding to each solder ball includes the first solder ball point cloud corresponding to the first DLP projector and the second solder ball point cloud corresponding to the second DLP projector. Specifically, for each solder ball, the multi-source solder ball point clouds corresponding to that solder ball are fused to obtain the first fused point cloud corresponding to that solder ball, including: Register the first and second point clouds of the solder ball corresponding to the solder ball to obtain multiple coordinate point pairs; Calculate the distance between the two coordinate points in each coordinate point pair; Calculate the average distance between multiple coordinate points and their respective corresponding distances, and use this as the difference value between the first solder ball point cloud and the second solder ball point cloud; If the difference value is less than or equal to a preset fusion threshold, the first solder ball point cloud and the second solder ball point cloud are weighted and fused to obtain the first fused point cloud corresponding to the solder ball. If the difference value is greater than a preset fusion threshold, then point clouds belonging to the same solder ball as the first solder ball point cloud are extracted from the first point cloud data corresponding to the first DLP projector as the first original point cloud, and point clouds belonging to the same solder ball as the second solder ball point cloud are extracted from the first point cloud data corresponding to the second DLP projector as the second original point cloud; the signal-to-noise ratio of the first original point cloud and the signal-to-noise ratio of the second original point cloud are calculated; and the original point cloud with the larger signal-to-noise ratio is selected as the first fused point cloud.

8. A three-dimensional reconstruction device for semiconductor packages, characterized in that, The device is configured in a controller, which is part of a semiconductor package 3D reconstruction system. The system also includes multiple DLP projectors and cameras, each communicatively connected to the controller. The multiple DLP projectors have different projection angles. The image acquisition module is used to control multiple DLP projectors to project structured light onto the semiconductor package under test in a time-division manner, and to control the camera to acquire a set of structured light images from the corresponding viewpoint of each DLP projector. The phase calculation module is used to perform phase calculation on each set of structured light images to obtain a set of first point cloud data corresponding to the set of structured light images. The point cloud segmentation module is used to segment each group of first point cloud data into point clouds to obtain at least one cluster of single solder ball point clouds and one cluster of base plate point clouds. The first fusion module is used to match and group the single solder ball point clouds corresponding to the same solder ball based on the three-dimensional position information corresponding to the single solder ball point clouds of each cluster, so as to obtain the multi-source solder ball point cloud corresponding to each solder ball; and to fuse the multi-source solder ball point clouds corresponding to each solder ball to obtain the first fused point cloud corresponding to each solder ball. The second fusion module is used to fuse the base point clouds of each cluster to obtain a second fused point cloud; The 3D reconstruction module is used to perform 3D reconstruction of the semiconductor package under test based on the first fusion point cloud and the second fusion point cloud corresponding to each solder ball.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images