A semiconductor package appearance detection angle alignment method and multi-view detection system

Abstract

The application relates to a semiconductor package appearance detection angle alignment method and a multi-view detection system. The method comprises the following steps: collecting an overhead image of a semiconductor device to be detected; performing distortion correction on the overhead image to obtain a corrected image; extracting a device contour from the corrected image to obtain a sub-pixel contour point set; obtaining a predicted rotation angle of the semiconductor device according to the sub-pixel contour point set; obtaining feature information reflecting a front direction on the surface of the device, determining an angle offset according to the predicted rotation angle and the feature information; controlling the relative rotation between the semiconductor device and an imaging unit according to the angle offset, so that the front of the device is consistent with a preset direction; collecting the overhead image again after the relative rotation, calculating a residual angle error, and performing supplementary rotation according to the angle difference until the angle error is not greater than a preset threshold. The above method improves the angle alignment accuracy and guarantees the geometric consistency of the images at each view before and after rotation. The application also comprises a detection system applying the above method.

Description

Technical Field

[0001] This invention belongs to the field of semiconductor packaging appearance inspection technology, specifically relating to a semiconductor packaging appearance inspection angle alignment method and multi-view inspection system, and more particularly to a semiconductor device angle alignment method and system based on visual feedback, used to achieve high-precision automatic correction of the front direction of the device in multi-view appearance inspection. Background Technology

[0002] Semiconductor package appearance inspection requires imaging multiple surfaces of the device to identify appearance defects such as damage, scratches, and dirt.

[0003] Existing testing systems typically place the device under test on a carrier, acquire images of the device using an imaging unit, determine the device's orientation deviation based on the images, adjust the device's orientation using a rotation mechanism, and then perform subsequent multi-view image acquisition and defect identification.

[0004] However, existing technologies generally suffer from low alignment accuracy and poor stability in the device orientation adjustment stage. In actual inspection, due to factors such as irregular device shape, image noise interference, and inherent errors in the rotating mechanism, the device often still exhibits orientation deviations after angle adjustment, resulting in inconsistent device position and orientation in images from different viewpoints. This affects the accuracy of subsequent defect detection and localization. These problems make it difficult for the overall accuracy and consistency of multi-view appearance inspection to meet increasingly stringent inspection requirements. Summary of the Invention

[0005] According to a first aspect of this application, a method for inspecting the angle alignment of a semiconductor package appearance is provided, comprising the following steps: Acquire a top-view image of the semiconductor device under test; The distortion of the top view image is corrected to obtain a corrected image; The device contour is extracted from the corrected image to obtain a sub-pixel contour point set; Based on the sub-pixel contour point set, the predicted rotation angle of the semiconductor device is obtained; Obtain feature information of the device surface reflecting the front direction, and determine the angle offset based on the predicted rotation angle and the feature information; Based on the angular offset, the relative rotation between the semiconductor device and the imaging unit is controlled so that the front side of the semiconductor device is aligned with a preset direction; After relative rotation, the top view image is acquired again, and the remaining angular error is calculated. The remaining angle error is compared with a preset angle error threshold to obtain an angle difference. Based on the angle difference, a supplementary rotation is performed. The steps of obtaining the angle difference and performing supplementary rotation based on the angle difference are repeated until the angle error is not greater than the angle error threshold.

[0006] Further, the step of extracting device contours from the corrected image to obtain a sub-pixel contour point set includes: The corrected image is preprocessed to obtain a device foreground mask; Edge detection is performed on the foreground mask to obtain pixel-level edges; Sub-pixel localization is performed on the pixel-level edges to obtain the sub-pixel contour point set.

[0007] Furthermore, the sub-pixel localization of the pixel-level edge includes using a gray-scale moment method to calculate the sub-pixel position using a Gaussian-weighted gray-scale moment along the edge normal direction; the sub-pixel position is calculated using the following formula:

[0008] in, For the sub-pixel position, These are Gaussian weighted coefficients. For pixel coordinates, The value is the grayscale value corresponding to the pixel coordinates.

[0009] Further, obtaining the predicted rotation angle of the semiconductor device based on the sub-pixel contour point set includes: An anti-outlier fitting algorithm is used to perform rotation rectangle fitting on the sub-pixel contour point set to obtain the predicted rotation angle.

[0010] Furthermore, the fitting algorithm for the anti-outlier points is RANSAC rotated rectangle fitting, which includes: Randomly select the minimum point set from the sub-pixel contour point set and fit a rotating rectangle; Calculate the distance from all contour points to the nearest side of the rotated rectangle; Mark contour points whose distance is less than the first threshold as interior points; Repeat the random sampling and fitting process a preset number of times, and select the rotating rectangle model with the most interior points. The rotating rectangle is refitted using all the interior points corresponding to the rotating rectangle model to obtain the final predicted rotation angle.

[0011] Further, the feature information includes silkscreen markings, positioning notches, or pin distribution differences; determining the angular offset based on the predicted rotation angle and the feature information includes: Based on the relative orientation of the feature information within the device contour, an angle consistent with the frontal direction of the device is selected from the candidate directions as the angle offset. The candidate directions include... , The predicted rotation angle is given.

[0012] Furthermore, the preset angle error threshold is 0.05° to 0.1°; the maximum number of times the step of repeatedly obtaining the angle difference and performing supplementary rotation based on the angle difference is 2 to 3 times.

[0013] Furthermore, the above method also includes an eccentricity estimation step, which includes: The predicted center position of the semiconductor device under test is obtained based on the sub-pixel contour point set; Obtain the coordinates of the rotation center in the image; The horizontal eccentricity is calculated based on the difference between the coordinates of the predicted center position and the rotation center in the image. Based on the horizontal eccentricity, the theoretical center position of the semiconductor device under test in the image is determined after a relative rotation of 180°. Using the theoretical center position as a reference, the side image acquired after a 180° relative rotation is translated and corrected to obtain the corrected side image.

[0014] According to a second aspect of this application, a semiconductor package appearance multi-view inspection system is provided, comprising: A hollow rotating gimbal has a detection window that runs vertically through its central area. The hollow rotating gimbal is driven by a drive motor to rotate around its central axis through a transmission mechanism. A transparent carrier is embedded in the detection window to support the semiconductor device under test; The upper imaging unit is fixedly installed above the hollow rotating gimbal, and its optical axis passes vertically downward through the center of the transparent carrier. The upper imaging unit is used to acquire a top view image of the semiconductor device under test. The lower imaging unit is fixedly disposed directly below the transparent carrier, and its optical axis coincides with the optical axis of the upper imaging unit; the lower imaging unit is used to acquire a bottom view image of the semiconductor device under test; A front-side camera is positioned on the front side of the hollow rotating gimbal, with its optical axis pointing horizontally toward the center of the transparent support component. The right-side camera is positioned to the right of the hollow rotating gimbal, with its optical axis pointing horizontally towards the center of the transparent support. The control module is connected to the drive motor via signals. The host computer is connected to the upper imaging unit, the lower imaging unit, the front camera, the right camera and the control module respectively, and is configured to execute the above-described semiconductor package appearance detection angle alignment method.

[0015] Furthermore, the host computer includes: The distortion correction module is used to correct radial and tangential distortion in top-view images; The contour extraction module is used to extract the device subpixel contours from the calibrated image; The angle determination module is used to obtain the predicted rotation angle based on the sub-pixel contour point set and determine the angle offset by combining feature information. The feedback control module is used to re-acquire the top view image after the rotation and determine whether fine-tuning is needed; An eccentricity estimation module is used to calculate the eccentricity between the center of the semiconductor device under test and the rotation center of the hollow rotating gimbal; The system also includes a display module connected to the host computer. The display module is used to display the detection results in real time. The detection results include images of the semiconductor device under test acquired by the upper imaging unit, the lower imaging unit, the front camera, and the right camera, as well as the real-time angular offset and eccentricity.

[0016] Compared with the prior art, this application has at least the following technical effects: On the one hand, the semiconductor package appearance inspection angle alignment method provided in this application obtains more accurate device contour data by performing distortion correction and sub-pixel contour extraction on the top view image. Based on this, the predicted rotation angle is obtained and the angle offset is determined by combining the feature information of the device surface reflecting the front direction, thereby uniquely determining the front direction of the device and avoiding angle ambiguity caused by the symmetry of the rectangular contour. After controlling the relative rotation according to the angle offset, the top view image is acquired again and the remaining angle error is calculated. The rotation is supplemented according to the error magnitude and the verification is repeated to compensate for the angle deviation caused by factors such as mechanical transmission clearance and step error, so that the front of the device is finally consistent with the preset direction, which effectively improves the accuracy and stability of angle alignment and ensures the consistency of device posture during subsequent multi-view image acquisition.

[0017] On the other hand, the multi-view semiconductor packaging appearance inspection system provided in this application, through the combination of a hollow gimbal, a transparent carrier, and a lower-mounted camera, achieves complete images of the top, bottom, and four sides of the semiconductor device without flipping it, ensuring no blind spots and completely replacing manual flipping operations. The images acquired by the upper-mounted imaging unit are used by the host computer to calculate angle deviations and instruct the control module to drive the gimbal to automatically rotate and align, ensuring the front of the device faces the front camera, eliminating the impact of manual placement angle deviations on inspection accuracy. The side cameras are arranged on the front and right sides, avoiding interference with the operator's loading and unloading actions, meeting ergonomic requirements. The host computer is responsible for image processing and defect identification, while the control module is responsible for gimbal motion control; the division of labor is clear, the system has a fast response speed, and high stability. Furthermore, the automatic inspection results are presented on the display module; the operator only needs to view and confirm before loading and unloading, significantly reducing the burden on the eyes and improving inspection efficiency. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a semiconductor package appearance inspection method provided in Example 1; Figure 2 This is a schematic diagram of the structure of a multi-view semiconductor packaging appearance inspection system provided in Example 2; Figure 3 This is a partial signal structure block diagram of a multi-view semiconductor packaging appearance inspection system provided in Example 2.

[0020] In the picture: 1-Upper imaging unit; 2-Front camera; 3-Right camera; 4-Transparent support bracket; 5-Transparent support; 6-Hollow rotating gimbal; 7-Lower imaging unit; 8-Substrate. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0022] In the description of the embodiments of this application, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the embodiments of this application and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention. The terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, the terms "installed," "connected," and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the embodiments of this application according to the specific circumstances.

[0023] Example 1 This embodiment provides a method for aligning the appearance of a semiconductor package at different angles, including the following steps: Acquire a top view image of the semiconductor device under test; perform distortion correction on the top view image to obtain a corrected image; The device contour is extracted from the corrected image to obtain a sub-pixel contour point set; Based on the sub-pixel contour point set, the predicted rotation angle of the semiconductor device is obtained; Obtain feature information of the device surface reflecting the front direction, and determine the angle offset based on the predicted rotation angle and the feature information; Based on the angular offset, the relative rotation between the semiconductor device and the imaging unit is controlled so that the front side of the semiconductor device is aligned with a preset direction; After relative rotation, the top view image is acquired again, and the remaining angular error is calculated. The remaining angle error is compared with a preset angle error threshold to obtain an angle difference. Based on the angle difference, a supplementary rotation is performed. The steps of obtaining the angle difference and performing supplementary rotation based on the angle difference are repeated until the angle error is not greater than the angle error threshold.

[0024] Specifically, regarding the above technical solution of this embodiment: This embodiment uses a top-view image as the adjustment standard. A top-view image is an image obtained by taking a picture vertically downwards from directly above the device, showing the device's projected shape in a horizontal plane, including information such as the device's outline, surface markings, and pin distribution. The main purpose of using a top-view image is: In practical applications, the side view of a semiconductor packaged device is usually rectangular, and the shape change of the side view due to angle variations is not as significant as that of the top view. As for the bottom view, it usually lacks the detailed outline, surface silkscreen, pin distribution, and other information found in the top view. Therefore, this embodiment, considering factors such as recognition difficulty, recognition accuracy, and work efficiency, preferably uses the top view as the adjustment standard.

[0025] After the operator places the semiconductor device under test (DUT) on the transparent carrier, the host computer triggers the upper imaging unit to acquire a top-view image of the device. The image is transmitted to the host computer via a data interface. The optical axis of the upper imaging unit passes vertically downward through the center of the transparent carrier to ensure that the device is located in the central area of ​​the field of view, avoiding contour distortion caused by excessive perspective angle. The upper imaging unit can be a planar array industrial camera, and the lens configuration is determined according to the detection accuracy requirements: in one exemplary implementation, a telecentric lens is used, which features constant magnification and small perspective error, suitable for scenarios with high geometric accuracy requirements; in another exemplary implementation, a common fixed-focus lens is used, which is lower in cost but has relatively obvious distortion, requiring compensation through subsequent distortion correction steps. The field of view should ideally completely cover the device's outline while retaining an appropriate margin, which is generally not less than 20% of the device's maximum size.

[0026] In this embodiment, after obtaining the top-view image, further distortion correction is required to obtain a corrected image. Distortion is an inherent imaging error of the lens optical system, manifested as curved lines and offset edges in the image, which directly affects the geometric accuracy of subsequent contour extraction, thus requiring correction. The parameters used for distortion correction are obtained through pre-calibration.

[0027] Regarding the aforementioned distortion correction, for example, calibration can be performed using a standard checkerboard calibration plate during the equipment installation and commissioning phase: the calibration plate is placed near the plane of the transparent support component, and multiple images of the calibration plate in different poses are acquired using the upper imaging unit. The distortion coefficients are then calculated using a calibration algorithm. The distortion coefficients include radial distortion coefficients. and tangential distortion coefficient During correction, the coordinates of each pixel are transformed using the following formula:

[0028]

[0029] In the formula, , These are the pixel coordinates in the original image, in pixels. Here are the corrected image coordinates, in pixels. Regarding the distortion coefficients in the above formula, their typical values ​​vary depending on the lens type; more specifically: For ordinary industrial lenses Typically between -0.5 and 0.5, , The absolute value is usually much smaller than For telecentric lenses, the distortion coefficients are usually close to zero, but they still need to be corrected to eliminate residual errors.

[0030] In another optional embodiment, distortion correction can also employ a lookup table method. This involves pre-establishing a mapping table from the original coordinates to the corrected coordinates based on the distortion model. During detection, the image is directly remapped pixel by pixel according to this table, avoiding redundant online calculations. This approach is suitable for scenarios with limited computing resources. The corrected image eliminates straight-line curvature, providing geometrically correct image data for subsequent contour extraction and angle fitting.

[0031] In this embodiment, the step of obtaining the sub-pixel contour point set more specifically includes three sub-steps: The corrected image is preprocessed to obtain the device foreground mask; Edge detection is performed on the foreground mask to obtain pixel-level edges; Sub-pixel localization is performed on pixel-level edges to obtain a sub-pixel contour point set.

[0032] The three sub-steps described above are explained in detail below: The purpose of preprocessing is to separate the device region from the background region. In one exemplary implementation, preprocessing includes grayscale conversion, Gaussian filtering, and adaptive thresholding. Grayscale conversion converts the color image into a single-channel grayscale image, reducing the computational load of subsequent processing; Gaussian filtering is used to suppress random noise in the image, and the filter kernel size is typically 3×3 or 5×5 pixels; adaptive thresholding determines the segmentation threshold based on the local grayscale distribution of the image. Compared to a globally fixed threshold, adaptive thresholding is more adaptable to uneven illumination and can obtain a more complete device foreground mask.

[0033] In another exemplary implementation, when the device surface texture is complex or the background is interfering, a lightweight semantic segmentation network can be used to extract the device region. The network weights are trained using pre-annotated device images, and the device foreground mask is directly output during inference. The advantage of this approach is its stronger adaptability to complex backgrounds.

[0034] Edge detection is used to locate the pixel-level position of a device contour on a foreground mask. In one exemplary implementation, the Canny edge detection algorithm is employed. The Canny algorithm calculates the magnitude and direction of the image gradient, applies non-maximum suppression and double-threshold hysteresis, and outputs a single-pixel-width edge response. The high and low thresholds in the double thresholding are typically set according to the image gradient magnitude distribution; the high threshold is used to identify strong edges, and the low threshold is used to connect edge breakpoints. The output of edge detection is pixel-level edge detection, meaning that edge positions can only be located to integer pixel coordinates.

[0035] However, the positioning accuracy of pixel-level edges is limited by pixel size. For small packaged devices, the actual physical size corresponding to a single pixel may reach several micrometers to tens of micrometers. A positioning error of a single pixel can cause a non-negligible angular deviation. Therefore, sub-pixel positioning is required to improve edge position accuracy to the sub-pixel level. In one exemplary implementation, sub-pixel positioning uses the gray-scale moment method. The principle is as follows: a gray-scale sequence of length 2k+1 pixels is taken along the normal direction of the edge pixel. This gray-scale sequence spans the transition region from the background to the device, and the center position of its gray-scale distribution corresponds to the actual edge position. The formula for calculating the sub-pixel position is:

[0036] In the formula, For the first in the window The coordinates of a pixel, in pixels; This is the grayscale value of the pixel, which is dimensionless. The calculated subpixel positions are in pixels; The Gaussian weighting coefficients are calculated using the following formula:

[0037] in This represents the standard deviation of the Gaussian kernel, expressed in pixels, typically between 1.0 and 2.0 pixels.

[0038] The Gaussian weighting coefficients are used to ensure that pixels closer to the edge center contribute more to the calculation results, while suppressing interference from pixels farther from the edge. (Window half-width) Generally, 3 to 5 pixels are used.

[0039] In another exemplary implementation, subpixel localization can also employ interpolation, i.e., performing bilinear or bicubic interpolation on the grayscale of the edge neighborhood and re-detecting the edge position on the interpolated high-resolution image; or employing curvature fitting, performing parabolic fitting on the grayscale gradient curve of the edge neighborhood and using the extreme points of the fitted curve as the subpixel edge positions.

[0040] Among the above methods, the gray-scale moment method has lower computational complexity and better noise resistance, making it suitable for online real-time processing; the interpolation method is simple to implement but is affected by the quality of the interpolation kernel; the fitting curvature method has higher accuracy but depends on the estimation of the edge normal direction. Designers can choose the appropriate method according to the actual situation. In this embodiment, the sub-pixel contour point set of the device is obtained after sub-pixel positioning through any selectable implementation method, denoted as . ,in The number of contour points, These are subpixel coordinates, in pixels.

[0041] Based on the sub-pixel contour point set, the predicted rotation angle of the semiconductor device is obtained. The predicted rotation angle refers to the current orientation angle of the device estimated through geometric analysis of the device contour, which is the basis for subsequently determining the angle offset. The top-view contour of a semiconductor device is usually approximately rectangular, so the predicted rotation angle can be obtained by rotating a rectangle. However, the actual device contour is not a perfect rectangle. Factors such as protruding pins, burrs, edge damage, and local reflections can introduce outliers into the contour point set, i.e., points that deviate from the true rectangular boundary. If all contour points are directly used for least squares fitting, outliers will significantly drag the fitting result, leading to deviations in angle estimation. Therefore, an outlier-resistant fitting algorithm is needed to perform rotating rectangle fitting on the sub-pixel contour point set. Specifically: In one exemplary implementation, the outlier-resistant fitting algorithm is RANSAC rotating rectangle fitting. The core idea of ​​RANSAC is to find the model parameters that can explain the most data points through repeated random sampling and consistency checks. Specifically, a minimum set of points is randomly selected from the sub-pixel contour point set to fit a rotating rectangle. For a rotating rectangle, the minimum set of points consists of 5 points, which determine five parameters: the rectangle's rotation angle, center coordinates, width, and length. After fitting the rotating rectangle, the distance from all contour points to the nearest edge of the rotating rectangle is calculated. Contour points with a distance less than a first threshold are marked as inliers, and the remaining contour points are marked as outliers. The first threshold is determined based on the imaging resolution and edge localization accuracy, in pixels, typically between 0.5 and 1.0 pixels. Inliers are considered valid contour points that conform to the true rectangular boundary of the device, while outliers are considered invalid contour points affected by pins, burrs, stains, etc. The random sampling and fitting process is repeated a preset number of times, typically 200 to 500 times, recording the number of inliers corresponding to each fitting iteration. After all iterations are completed, the rotating rectangle model with the most interior points is selected as the optimal model. Least square fitting is then performed again using all interior points corresponding to this model to obtain the final predicted rotation angle and center position. Since least square fitting only uses interior points, outlier interference is eliminated, and the fitting result more closely approximates the true orientation of the device.

[0042] In another exemplary implementation, the outlier-resistant fitting algorithm can also employ least median square fitting. This algorithm estimates model parameters by minimizing the median of the squared residuals, exhibiting low sensitivity to outliers and eliminating the need for manually setting an inlier threshold. It is suitable for scenarios with a high proportion of outliers or where determining a reasonable threshold is difficult. In yet another exemplary implementation, when the proportion of outliers in the device contour is low (e.g., below 10%), a weighted least squares method can be used. Each contour point is assigned a weight inversely proportional to its distance from the current fitted rectangle. The influence of outliers is gradually suppressed by iteratively updating the weights. This method has lower computational complexity and is suitable for online scenarios with high detection cycle requirements. Among the above methods, the RANSAC method has intuitive inlier determination and mature engineering implementation, making it the most commonly used choice. The least median square method does not require setting a threshold but requires more sampling times. The weighted least squares method is computationally efficient but only suitable for scenarios with a low proportion of outliers. Designers can choose the appropriate method based on device type, imaging quality, and cycle requirements.

[0043] In this embodiment, the significance of determining the angle offset lies primarily in the fact that, to ensure the front of the device aligns with the preset direction, a relative rotation angle is required based on the current orientation. The rotation angle of a rectangular packaged device exhibits a 90° periodic ambiguity, i.e., the predicted rotation angle... and , , Geometrically, these correspond to identical rectangular orientations, and the four directions cannot be distinguished solely by the rectangular outline. Therefore, it is necessary to obtain feature information from the device surface that reflects the frontal orientation to eliminate ambiguity. More specifically: Feature information refers to asymmetric markings on a device that uniquely indicate its frontal orientation. In one exemplary implementation, the feature information is silkscreen markings, i.e., text, graphics, or symbols printed on the device surface, whose position and orientation have a fixed correspondence with the frontal orientation of the device, and can be located in a top-view image through template matching or feature point detection. In another exemplary implementation, the feature information is a positioning notch, i.e., a notch or protrusion on the edge of the package, usually located at a specific corner of the package, and can be identified through contour shape analysis. In yet another exemplary implementation, the feature information is pin distribution differences, i.e., the number or arrangement of pins on different sides of the device is asymmetrical, and can be determined by analyzing the pin density and distribution pattern on each side through statistical analysis of the protrusion features around the contour. The above types of feature information are not mutually exclusive, and one or more combinations can be selected for determination based on the specific package type and process characteristics of the device under test.

[0044] Based on the relative orientation of the feature information within the device profile, from the candidate direction The angle consistent with the front direction of the device is selected as the angle offset, where To predict the rotation angle, for example, if the device surface has a circular positioning indentation located in the upper left corner of the device outline, and the front of the device is defined as the short side facing the observer, then the front orientation can be uniquely determined based on the relative angle between the indentation and the center of the rectangle. The determination rule can be preset according to the device model during system debugging and automatically executed by the host computer during the detection process.

[0045] Based on the angular offset, the relative rotation between the semiconductor device and the imaging unit is controlled to align the front of the semiconductor device with a preset direction. The preset direction refers to the direction in which the front of the device should be aligned, as defined in the detection process; typically, it's the direction where the front of the device faces the front camera, denoted as 0°. The host computer generates control commands based on the angular offset and sends them to the control module. The control module drives the hollow rotating gimbal to rotate, changing the relative rotation angle between the semiconductor device and the imaging unit by this angular offset. Since the device is placed on a transparent support, which is embedded within the detection window of the hollow rotating gimbal, the gimbal's rotation causes the device to rotate synchronously, thus adjusting the relative angle between the device and each imaging unit. In one possible implementation, the control module converts the angular offset into the number of steps or pulses for the drive motor and sends it to the motor driver. The motor then drives the gimbal to rotate via a transmission mechanism. The transmission mechanism can be a worm gear drive, a synchronous belt drive, or a direct-drive motor. Worm gear drives have self-locking characteristics but have backlash; synchronous belt drives offer high precision but have limited rigidity; and direct-drive motors have no backlash but are more expensive. The choice of transmission mechanism affects rotational accuracy and response speed. Designers can select one based on accuracy requirements and cost constraints.

[0046] After relative rotation, a top-view image is acquired again, and the remaining angular error is calculated. The remaining angular error is compared with a preset angular error threshold to obtain the angular difference. Based on the angular difference, supplementary rotation is performed. This process of obtaining the angular difference and performing supplementary rotation is repeated until the angular error is no greater than the angular error threshold. The purpose of this step is to verify and correct the rotation result. Due to factors such as transmission mechanism backlash, motor stepping error, and gimbal inertia, there may be a deviation between the actual rotation angle and the commanded angle. The device posture may not meet the accuracy requirements after a single rotation. Therefore, it is necessary to acquire a top-view image again after rotation for verification.

[0047] Specifically, after the hollow rotating gimbal has rotated to its position and undergone a stabilization delay, the host computer triggers the upper imaging unit to acquire a top-view image again. This process repeats the entire sequence of distortion correction, device contour extraction and sub-pixel positioning, predicted rotation angle acquisition, direction disambiguation, and angle offset determination, yielding the angular offset after rotation. This angular offset is the remaining angular error. The remaining angular error is compared with a preset angular error threshold to obtain the angular difference. The angular difference is the difference between the absolute value of the remaining angular error and the angular error threshold. If the angular difference is greater than zero, it indicates that the remaining angular error exceeds the threshold, and the alignment accuracy has not yet met the standard. Therefore, a fine-tuning control command needs to be generated based on the direction and magnitude of the remaining angular error. The control module drives the hollow rotating gimbal to perform a fine-tuning rotation, i.e., supplementary rotation. Then, the top-view image is acquired again, and the remaining angular error is calculated. The above process is repeated until the remaining angular error is no greater than the angular error threshold. In another possible implementation, for example, supplementary rotation can also be achieved by using an upper imaging unit capable of rotating the lens or the entire unit to adjust the top-view image acquisition angle. Supplementary rotation adjustment is performed through the rotation of the upper imaging unit rather than the entire gimbal.

[0048] The angle error threshold is a crucial parameter, and its selection requires balancing detection accuracy and detection cycle time. If the threshold is too small, the number of fine-tuning operations increases, leading to a longer detection cycle time; if the threshold is too large, the alignment accuracy is insufficient, resulting in significant device posture deviations in the side view image. In practical applications, the angle error threshold is generally set between 0.05° and 0.1°. For small-sized packaged devices or high-precision detection requirements, a smaller threshold can be selected; in this embodiment, the smaller threshold is 0.05°. For large-sized packaged devices or routine detection requirements, a larger threshold can be selected; in this embodiment, the larger threshold is 0.1°. The maximum number of times the fine-tuning process is repeated is 2 to 3. Setting a maximum number of times prevents equipment malfunctions or abnormal operating conditions from causing the fine-tuning to become stuck in a loop, thus affecting the detection cycle time. If the remaining angle error is still greater than the threshold after reaching the maximum number of times, an alignment anomaly is indicated on the host computer interface, requiring operator intervention to check for device stability, foreign objects on the surface of the support, and smooth gimbal movement.

[0049] Furthermore, the method also includes an eccentricity estimation step, which includes: The predicted center position of the semiconductor device is obtained from the sub-pixel contour point set; the coordinates of the rotation center in the image are obtained. Calculate the horizontal eccentricity based on the difference between the predicted center position and the rotation center coordinates; Based on the horizontal eccentricity, determine the theoretical center position of the semiconductor device in the image after a relative rotation of 180°; Using the theoretical center position as a reference, the side image acquired after a 180° relative rotation is translated and corrected to obtain the corrected side image.

[0050] The predicted center position is obtained simultaneously during the aforementioned rotating rectangle fitting process, i.e., in the fitting result. The coordinates of the rotation center in the image. Obtained through pre-calibration. For example, the calibration method is as follows: During the equipment debugging phase, place a standard device on a transparent carrier, and control the hollow rotating gimbal to rotate at multiple angles (e.g., 0°, 60°, 120°, 180°, 240°, 300°). After each rotation, acquire a top-view image and extract the device's center coordinates. Fit the device's center coordinates corresponding to each angle into a circular trajectory. The center coordinates of this circular trajectory are the coordinates of the rotation center in the image. The horizontal eccentricity is calculated as follows: All units are pixels. The existence of eccentricity is a common phenomenon because it is difficult for operators to ensure that the center of the device perfectly coincides with the axis of rotation when placing the device. If the eccentricity is not addressed, the device's position in the image will shift after a 180° rotation due to the eccentricity, resulting in different image coordinates for the same physical location in images acquired from different viewpoints before and after the rotation. This affects the accuracy of cross-surface defect merging and location coding.

[0051] Based on the horizontal eccentricity, the theoretical position of the device center in the image after a relative rotation of 180° is: The side image acquired after a 180° relative rotation undergoes translation correction. This involves translating the image based on the difference between the theoretical center position and the actual position of the device in the image, aligning the device center to the theoretical position, resulting in the corrected side image. The corrected side image eliminates the geometric positional offset caused by eccentricity, ensuring a consistent positional reference for images from all viewpoints before and after rotation. For example, if the rotation center coordinates are (500, 400) pixels, and the device center's position before rotation is (520, 380) pixels, then the horizontal eccentricity is (20, -20) pixels, and the theoretical position of the device center after a 180° rotation is (480, 420) pixels. If the device center in the actual side image acquired after rotation is located at (485, 415) pixels, then a translation of (-5, +5) pixels is needed to align the device center to the theoretical position.

[0052] Example 2 This embodiment provides a multi-view semiconductor package appearance inspection system, such as Figures 2-3As shown, the system includes a hollow rotating gimbal 6, a transparent carrier 5, an upper imaging unit 1, a lower imaging unit 7, a front camera 2, a right camera 3, a control module, and a host computer. The entire detection system is mounted on a substrate 8. The hollow rotating gimbal 6 has a vertically penetrating detection window in its central area. The hollow rotating gimbal 6 is driven by a drive motor to rotate around its central axis via a transmission mechanism. The transparent carrier 5 is embedded within the detection window and mounted on the hollow rotating gimbal 6 via a transparent carrier bracket 4, used to support the semiconductor device under test. The upper imaging unit 1 is fixedly positioned above the hollow rotating gimbal, its optical axis vertically downwards through the center of the transparent carrier 5, and is used to acquire a top-view image of the semiconductor device under test. The lower imaging unit 7 is fixedly positioned directly below the transparent carrier 5, its optical axis coinciding with the optical axis of the upper imaging unit 1. The front camera 2 is positioned on the front side of the hollow rotating gimbal 6, its optical axis pointing horizontally towards the center of the transparent carrier 5. The right-side camera 3 is positioned to the right of the hollow rotating gimbal 6, with its optical axis pointing horizontally towards the center of the transparent carrier 5. The control module is signal-connected to the drive motor. The host computer is signal-connected to the upper imaging unit 1, the lower imaging unit 7, the front camera 2, the right-side camera 3, and the control module, and is configured to execute the aforementioned semiconductor package appearance inspection method.

[0053] The multi-view semiconductor packaging appearance inspection system provided in this embodiment has the following operating procedure: The operator first places the semiconductor device under test in, such as Figure 1 The transparent carrier 5 is shown. Subsequently, the upper imaging unit 1 acquires a top view image of the device and sends it to the host computer. The host computer runs an image processing algorithm to identify the contour features of the device in the image, calculates the angular offset of the device, that is, the angle between the front of the device and the preset direction, and then generates a rotation command and sends it to the control module. The control module drives the motor to rotate the hollow rotating gimbal 6 by the corresponding angle so that the front of the device faces the front camera 2. Since the front camera 2 and the right camera 3 are at a fixed 90-degree angle, when the front of the device faces the front camera 2, its right side naturally faces the right camera 3. At the same time, the field of view of the side imaging unit, that is, the front camera 2 and the right camera 3, is sufficient. The small offset between the center of the device and the center of the rotation axis will not cause the device to go out of the field of view. However, the image position offset caused by the eccentricity still needs to be eliminated by eccentricity estimation and translation correction to ensure the accuracy of cross-surface defect merging and position coding.

[0054] After angle alignment is completed, the first imaging stage begins. The host computer triggers the upper imaging unit 1, lower imaging unit 7, front camera 2, and right camera 3 to acquire images, obtaining images of the top, bottom, front, and right sides of the device, and sending them to the internally integrated defect detection module for defect identification.

[0055] The rotation and re-image capture phase then begins. The host computer sends a rotation command to the control module, which drives the hollow rotating gimbal 6 to rotate 180 degrees. At this point, the original rear side of the device moves to the front of the front camera 2, and the original left side moves to the front of the right camera. The host computer then triggers the front camera 2 and the right camera 3 again to capture images of the rear and left sides, which are then sent to the defect detection module for defect identification. In total, the host computer obtains six complete images: top, bottom, front, right, rear, and left sides.

[0056] The defect detection module, based on a pre-trained object detection network, processes each image with a uniform input size and outputs the defect bounding box, defect category (including damage, scratches, dirt, etc.), and confidence score for each image. The display module receives the detection results from the host computer, arranges all six thumbnails in a grid, and overlays colored bounding boxes and labels on the images with detected defects.

[0057] The operator checks the test results on the monitor to determine if the defects identified by the machine are accurate. After confirmation, the operator manually removes the tested device, places the next device under test, and begins the next round of testing.

[0058] During the above operation, the rotation angle of the device is precisely controlled by combining the method described in Example 1 to achieve high-precision alignment and ensure the accuracy of image capture.

[0059] Furthermore, the host computer includes a distortion correction module, a contour extraction module, an angle determination module, a closed-loop control module, and an eccentricity estimation module. The distortion correction module performs radial and tangential distortion correction on the top-view image. It internally stores pre-calibrated distortion coefficients and performs pixel-by-pixel coordinate transformation on the input top-view image before outputting the corrected image. The contour extraction module extracts the sub-pixel contours of the device from the corrected image, sequentially performing preprocessing, edge detection, and sub-pixel localization, and outputting a set of sub-pixel contour points. The angle determination module obtains the predicted rotation angle based on the sub-pixel contour point set and determines the angle offset by combining feature information. The closed-loop control module re-acquires the image after relative rotation and determines whether fine-tuning is needed, specifically performing remaining angle error calculation, angle difference comparison, and fine-tuning instruction generation. The eccentricity estimation module calculates the eccentricity between the device center and the rotation center and outputs the theoretical center position after 180° rotation for subsequent image translation correction.

[0060] The implementation process of this detection method is further illustrated below through more specific embodiments.

[0061] Example 3 The device under test (DUT) is in a QFN package, with dimensions of 4mm × 4mm and a thickness of 0.85mm. The top-mounted imaging unit uses a 5-megapixel industrial camera with a 0.5x telecentric lens, providing a field of view of 12mm × 10mm. The actual size of a single pixel is approximately 0.005mm. The angle error threshold is set to 0.05°, and the maximum number of fine-tuning adjustments is set to 2.

[0062] The operator places the device on a transparent support, with the actual placement angle deviating approximately 15° from the preset direction. The top-mounted imaging unit acquires a top-view image, and after distortion correction, extracts the device's sub-pixel contour point set, containing approximately 1800 points. RANSAC rotation rectangle fitting is used, with 300 iterations and an interior point threshold of 0.8 pixels. The fitted result yields a predicted rotation angle of 14.87°, with center coordinates of (1243.2, 867.5). The angle offset of 14.87° is confirmed by detecting the direction of the silkscreen markings on the device surface. After the control module drives the hollow rotating gimbal to rotate 14.87°, a top-view image is acquired again, and the remaining angle error is calculated to be 0.12°, exceeding the threshold of 0.05°. A fine-tuning command is generated based on the angle difference. After the gimbal rotates 0.12°, the remaining angle error is verified to be 0.03°, less than the threshold, indicating alignment is complete. The eccentricity is estimated as follows. Pixels After rotating the pixel by 180°, the side image is translated and corrected based on the theoretical center position. After correction, the center positions of the device in the front and rear side images are aligned.

[0063] Example 4 The device under test (DUT) is in a QFP package, with dimensions of 14mm × 20mm and a thickness of 2.0mm. The top-mounted imaging unit uses a 2-megapixel industrial camera with a 0.3x lens, providing a field of view of 30mm × 24mm. The angle error threshold is set to 0.1°, and the maximum number of fine-tuning adjustments is set to 3.

[0064] The actual placement angle of the device deviates from the preset direction by approximately 32°. With RANSAC iterations set to 400 and an interior point threshold of 1.0 pixel, the predicted rotation angle obtained through fitting is 31.72°. After direction disambiguation, the angle offset is also 31.72°. The remaining angle error after rotation is 0.08°, less than the threshold of 0.1°, requiring no fine-tuning, and alignment is successful on the first attempt. The eccentricity estimation is obtained... Pixels The pixel positions of the side images are consistent after translation correction.

[0065] Example 5 The device under test (DUT) is in an SOP package, measuring 5mm × 6mm, with four pins on each side. The device surface has scratches and blemishes. RANSAC rotational rectangle fitting was used, with 350 iterations and an interior point threshold of 0.6 pixels. The fitting results showed 1450 interior points, approximately 230 outliers, and a predicted rotation angle of 8.35°. For comparison, least-squares fitting using all contour points yielded an angle of 8.71° with a deviation of 0.36°. After one closed-loop fine-tuning, the remaining angle error of this method was reduced to 0.02°, meeting the detection requirements.

[0066] The above embodiments demonstrate that this method can achieve high-precision angle alignment under different packaging types, sizes, placement deviations, and scenarios with outlier interference. Furthermore, the closed-loop fine-tuning mechanism effectively compensates for mechanical transmission errors, and the eccentricity estimation and translation correction ensure the geometric consistency of images from different viewpoints before and after rotation.

[0067] The various embodiments in this specification are described in a progressive manner. For the same or similar parts between the various embodiments, please refer to each other. Each embodiment focuses on describing the differences from other embodiments.

[0068] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of this application.

Claims

1. A method for aligning the appearance of a semiconductor package at different angles, characterized in that, Includes the following steps: Acquire a top-view image of the semiconductor device under test; The distortion of the top view image is corrected to obtain a corrected image; The device contour is extracted from the corrected image to obtain a sub-pixel contour point set; Based on the sub-pixel contour point set, the predicted rotation angle of the semiconductor device is obtained; Obtain feature information of the device surface reflecting the front direction, and determine the angle offset based on the predicted rotation angle and the feature information; Based on the angular offset, the relative rotation between the semiconductor device and the imaging unit is controlled so that the front side of the semiconductor device is aligned with a preset direction; After relative rotation, the top view image is acquired again, and the remaining angular error is calculated. The remaining angle error is compared with a preset angle error threshold to obtain an angle difference. Based on the angle difference, a supplementary rotation is performed. The steps of obtaining the angle difference and performing supplementary rotation based on the angle difference are repeated until the angle error is not greater than the angle error threshold.

2. The semiconductor package appearance inspection angle alignment method according to claim 1, characterized in that, The step of extracting device contours from the corrected image to obtain a sub-pixel contour point set includes: The corrected image is preprocessed to obtain a device foreground mask; Edge detection is performed on the foreground mask to obtain pixel-level edges; Sub-pixel localization is performed on the pixel-level edges to obtain the sub-pixel contour point set.

3. The semiconductor package appearance inspection angle alignment method according to claim 2, characterized in that, The sub-pixel localization of the pixel-level edge includes using a gray-scale moment method to calculate the sub-pixel position using a Gaussian weighted gray-scale moment along the edge normal direction; the sub-pixel position is calculated using the following formula: ； in, For the sub-pixel position, These are Gaussian weighted coefficients. For pixel coordinates, The value is the grayscale value corresponding to the pixel coordinates.

4. The semiconductor package appearance inspection angle alignment method according to claim 1, characterized in that, The step of obtaining the predicted rotation angle of the semiconductor device based on the sub-pixel contour point set includes: An anti-outlier fitting algorithm is used to perform rotation rectangle fitting on the sub-pixel contour point set to obtain the predicted rotation angle.

5. The semiconductor package appearance inspection angle alignment method according to claim 4, characterized in that, The fitting algorithm for the anti-outlier points is RANSAC rotated rectangle fitting, which includes: Randomly select the minimum point set from the sub-pixel contour point set and fit a rotating rectangle; Calculate the distance from all contour points to the nearest side of the rotated rectangle; Mark contour points whose distance is less than the first threshold as interior points; Repeat the random sampling and fitting process a preset number of times, and select the rotating rectangle model with the most interior points. The rotating rectangle is refitted using all the interior points corresponding to the rotating rectangle model to obtain the final predicted rotation angle.

6. The semiconductor package appearance inspection angle alignment method according to claim 1, characterized in that, The feature information includes silkscreen markings, positioning notches, or pin distribution differences; determining the angle offset based on the predicted rotation angle and the feature information includes: Based on the relative orientation of the feature information within the device contour, an angle consistent with the frontal direction of the device is selected from candidate directions as the angle offset. The candidate directions include... , The predicted rotation angle is given.

7. The semiconductor package appearance inspection angle alignment method according to claim 1, characterized in that, The preset angle error threshold is 0.05° to 0.1°; the maximum number of times the step of repeatedly obtaining the angle difference and performing supplementary rotation based on the angle difference is 2 to 3 times.

8. The semiconductor package appearance inspection angle alignment method according to claim 1, characterized in that, It also includes an eccentricity estimation step, which includes: The predicted center position of the semiconductor device under test is obtained based on the sub-pixel contour point set; Obtain the coordinates of the rotation center in the image; The horizontal eccentricity is calculated based on the difference between the coordinates of the predicted center position and the rotation center in the image. Based on the horizontal eccentricity, the theoretical center position of the semiconductor device under test in the image is determined after a relative rotation of 180°. Using the theoretical center position as a reference, the side image acquired after a 180° relative rotation is translated and corrected to obtain the corrected side image.

9. A multi-view inspection system for the appearance of semiconductor packages, characterized in that, include: A hollow rotating gimbal has a detection window that runs vertically through its central area. The hollow rotating gimbal is driven by a drive motor to rotate around its central axis through a transmission mechanism. A transparent carrier is embedded in the detection window to support the semiconductor device under test; The upper imaging unit is fixedly installed above the hollow rotating gimbal, and its optical axis passes vertically downward through the center of the transparent carrier. The upper imaging unit is used to acquire a top view image of the semiconductor device under test. The lower imaging unit is fixedly disposed directly below the transparent carrier, and its optical axis coincides with the optical axis of the upper imaging unit. The lower imaging unit is used to acquire a bottom view image of the semiconductor device under test; A front-side camera is positioned on the front side of the hollow rotating gimbal, with its optical axis pointing horizontally toward the center of the transparent support component. The right-side camera is positioned to the right of the hollow rotating gimbal, with its optical axis pointing horizontally towards the center of the transparent support. The control module is connected to the drive motor via signals. The host computer is connected to the upper imaging unit, the lower imaging unit, the front camera, the right camera and the control module respectively, and is configured to perform the method according to any one of claims 1 to 8.

10. The system according to claim 9, characterized in that, The host computer includes: The distortion correction module is used to correct radial and tangential distortion in top-view images; The contour extraction module is used to extract the device subpixel contours from the calibrated image; The angle determination module is used to obtain the predicted rotation angle based on the sub-pixel contour point set and determine the angle offset by combining feature information. The feedback control module is used to re-acquire the top view image after the rotation and determine whether fine-tuning is needed; An eccentricity estimation module is used to calculate the eccentricity between the center of the semiconductor device under test and the rotation center of the hollow rotating gimbal; The system also includes a display module connected to the host computer. The display module is used to display the detection results in real time. The detection results include images of the semiconductor device under test acquired by the upper imaging unit, the lower imaging unit, the front camera, and the right camera, as well as the real-time angular offset and eccentricity.

Images