Chip packaging test bubble rate analysis method and system based on image analysis
By employing image analysis methods and algorithms such as histogram equalization and median filtering, the bubble rate of chips is automatically identified and calculated, solving the problems of low efficiency and poor accuracy in existing technologies and achieving efficient and accurate bubble detection.
Patent Information
- Application Number
- CN202511486076.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-17
- Publication Date
- 2026-02-03
AI Technical Summary
Existing chip bubble detection technologies suffer from low efficiency, poor accuracy, difficulty in adapting to different scanning methods, errors in manual calculations, and difficulties and time-consuming deep learning annotation.
An image-based approach is employed, using preprocessing algorithms such as histogram equalization and median filtering, combined with multi-dimensional extraction and threshold adjustment, to automatically identify and calculate bubble area, thereby enabling bubble rate analysis in chip packaging and testing.
It improves the accuracy and calculation precision of bubble recognition, reduces labor costs and time, is suitable for large-scale chip production, and reduces calculation time and labeling errors.
Smart Images
Figure CN121458631A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of chip bubble detection, and particularly relates to a chip encapsulation bubble rate analysis method and system based on image analysis. BACKGROUND
[0002] In the chip manufacturing process, the presence of bubbles can have a serious impact on the performance and quality of the chip. Traditional chip bubble detection usually relies on manual observation and calculation, which is not only inefficient, but also difficult to ensure accuracy. With the continuous development of chip manufacturing technology, the accuracy and speed of bubble detection are increasingly demanding.
[0003] In the existing technology, X-Ray scanning and ultrasonic scanning are common detection methods, but due to the differences in resolution, clarity and contrast of the pictures obtained by these two scanning methods, it brings great challenges to the accurate identification and proportion calculation of bubbles. Some existing detection methods can only process a single scanning method, and cannot adapt to the characteristics of different scanning methods, resulting in insufficient reliability of the detection results. Moreover, the manual calculation of bubble proportion is not only time-consuming and laborious, but also prone to human error, which is difficult to meet the needs of large-scale chip production.
[0004] In the field of chip bubble detection, the conventional deep learning classification technology has certain limitations. Since the bubbles in the chip are usually small and irregular in shape, it is very difficult to label them. Labeling personnel are difficult to accurately outline the outline of the bubble and determine its category, which not only requires a lot of time and effort, but also easily introduces labeling errors, affecting the training effect and accuracy of the deep learning model. In contrast, the present technology is based on image recognition technology, using multi-dimensional extraction and comparative analysis, which can more effectively cope with the characteristics of chip bubbles and achieve more accurate bubble identification and proportion calculation. SUMMARY
[0005] The present application aims to overcome the shortcomings of the prior art and provide a chip encapsulation bubble rate analysis method and system based on image analysis.
[0006] To achieve the purpose of the present application, the following technical solutions are adopted.
[0007] A chip encapsulation bubble rate analysis method based on image analysis, comprising the following steps: S1, selecting X-Ray scanning images or ultrasonic scanning images from the imported board card original images, performing first preprocessing to obtain board card preprocessed images; S2. Convert the pre-processed image of the board into a grayscale histogram. Select the grayscale values to accumulate starting from 0. Use the grayscale value when the accumulated grayscale histogram value reaches 40% of the sum of all grayscale histogram values as the first threshold. Perform binarization processing on the pre-processed image of the board to identify and extract the outline of the chip in the board. Extract the original image of a single chip by using the minimum bounding rectangle of each chip outline. S3. Convert the original image of a single chip into a grayscale histogram. Starting from a grayscale value of 0, accumulate the values of the grayscale histogram. Use the grayscale value when the accumulated grayscale histogram value reaches 85% of the sum of all grayscale histogram values as the second threshold. Perform binarization processing on the original image of a single chip to identify the chip outline, and use the largest retrieved chip outline area as the chip area. At the same time, perform a second preprocessing on the original image of a single chip to obtain a preprocessed image of a single chip. Then, sequentially go through the bubble segmentation, bubble aggregation, and bubble extraction processes to obtain the bubble outline of each bubble. Use the largest area of the bounding rectangle of each bubble outline as the bubble outline area of each bubble. Calculate the area of all bubbles, i.e., the bubble area. S4. Calculate the percentage of the bubble area to the chip area based on the bubble area and the chip area; The first preprocessing step involves using a histogram equalization algorithm to enhance contrast if the original image of the imported board is an X-ray scan; and using a median filtering algorithm to remove noise if the original image of the imported board is an ultrasound scan. The second preprocessing step involves normalizing and Gaussian blurring the original image of a single chip to align the image brightness and reduce noise. Then, a combination of dilation and erosion operations are performed to eliminate holes and isolated points. Bubble segmentation employs a dual-thresholding approach to extract the edge contours of each bubble from a preprocessed image of a single chip using a hysteresis thresholding operation. Then, a closing operation is performed on each edge contour to ensure that each edge contour is coherent. Bubble aggregation involves taking an outer rectangle for the edge contour of each bubble, and determining whether each outer rectangle overlaps with the outer rectangles of all other bubbles. If they overlap, the two outer rectangles are merged, and the process continues until all outer rectangles no longer overlap. Bubble extraction involves taking the bounding rectangle of each aggregated image as a contour and overlaying it onto the original image of a single chip. The contour image of each bubble is then extracted. The average pixel grayscale of any two points on each side of each contour image is multiplied by a scaling factor to serve as the threshold for binarization, thereby extracting the bubble contour of each bubble.
[0008] Furthermore, the process of preprocessing the X-ray scan image using a histogram equalization algorithm includes the following steps: S21. Convert the X-Ray scan image to a grayscale image; S22. Perform histogram equalization using the cv2.equalizeHist() algorithm; S23. Convert back to BGR format to maintain channel consistency with the X-Ray scan.
[0009] Furthermore, the process of preprocessing the ultrasound scan using a median filtering algorithm includes the following steps: S31. Select the kernel size for median filtering. Since the ultrasonic scanning noise in this process is relatively low, the kernel size is set to 3x3. In cases with higher noise levels, it can be adjusted to 5x5.
[0010] S32. Perform median filtering using the cv2.medianBlur() algorithm.
[0011] Furthermore, the specific process of obtaining the original image of the single chip includes the following steps: S41. Binarize the preprocessed image of the board according to the first threshold to initially separate the chip area; S42. Use the OpenCV cv2.findContours function with the mode parameter RETR_EXTERNAL to detect only the outermost edge contour of the initially separated chip region, ignoring the inner contour, to identify the edge contour of the chip. S43. The parameter method uses CHAIN_APPROX_NONE to store all points of the edge contour without approximation, extracts the chip edge contour, and obtains the minimum bounding rectangle of each chip edge contour through boundingRect, cropping the original image of a single chip from the board preprocessing image.
[0012] Furthermore, the specific process of the second preprocessing includes the following steps: S51. Normalize and Gaussian blur the original image of the cropped individual chip to align the image brightness and reduce noise. S52. By combining expansion and erosion, voids and isolated points are eliminated, the integrity of the edge contour is optimized, and a preprocessed image of a single chip is obtained.
[0013] Furthermore, the specific process of bubble segmentation includes the following steps: S61. The Canny algorithm is used to perform hysteresis thresholding on the preprocessed image of a single chip using double thresholds to obtain the bubble edge contour. S62. Perform a closing operation on the bubble edge contour to make the bubble edge contour more coherent. The hysteresis thresholding operation selects two pixel thresholds, a low one and a high one. Pixels with values below the low pixel threshold are discarded, while those with values above the high pixel threshold are immediately retained. Pixels with values between the low and high pixel thresholds are only retained if they are connected to strong edges.
[0014] Furthermore, the specific process of bubble aggregation is as follows: the outer rectangle of the bubble edge contour obtained during the bubble segmentation process is taken, and the coordinates of the four corners of each outer rectangle are used to determine whether it overlaps with the outer rectangles of all other bubbles. If there is an overlap, the outer rectangles of the two bubbles are merged and then the judgment is continued to be made with the outer rectangles of all other bubbles. This operation is repeated until all outer rectangles no longer overlap.
[0015] Furthermore, the specific process of bubble extraction includes the following steps: S81. Using the bounding rectangle of the aggregated bubble as the outline, superimpose it onto the original image of a single chip, and extract the original image of the corresponding area on the image. S82. Take the average of the pixel grayscale values of any two points on each side of the bubble image in the original image and multiply it by the scaling factor to use as the threshold for binarization. Segment the bubble region from the original image of a single chip. S83. For the binarized image, use the cv2.findContours function, with the mode parameter set to RETR_EXTERNAL and the method parameter set to CHAIN_APPROX_NONE to extract the bubble edge contours, and calculate the area of each edge contour using cv2.contourArea. The proportional coefficient is configured through a configuration file, with a value between 0.6 and 0.9. The default value is 0.8, and it can be adjusted according to the extraction effect.
[0016] A chip packaging and testing bubble rate analysis system based on image analysis includes an image preprocessing module, a chip segmentation module, a bubble area calculation module, a chip area calculation module, and a bubble rate calculation module; wherein: The image preprocessing module uses a histogram equalization algorithm to preprocess the original image of the board if it is an X-ray scan image, thereby enhancing the contrast and generating a preprocessed image of the board. If the original image of the board is an ultrasound scan image, it uses a median filtering algorithm to preprocess the image, thereby removing noise and generating a preprocessed image of the board. The chip segmentation module converts the pre-processed board image into a grayscale histogram. It selects the grayscale values to accumulate starting from 0 and uses the grayscale value when the accumulated grayscale histogram value reaches 40% of the sum of all grayscale histogram values as the threshold. It then performs binarization processing on the generated pre-processed board image to identify the chip outline and uses the minimum bounding rectangle of each chip outline to crop the original image of a single chip. The bubble area calculation module includes a single-chip preprocessing unit, a bubble segmentation unit, a bubble aggregation unit, and a bubble extraction unit, wherein: The single-chip preprocessing unit normalizes and Gaussian blurs the original image of a single chip to align the image brightness and reduce noise. Then, it performs a combination of dilation and erosion operations to eliminate holes and isolated points, thus obtaining a preprocessed image of a single chip. The bubble segmentation unit uses a dual thresholding method to segment the edge contour of each bubble from the preprocessed image of a single chip through a hysteresis thresholding operation, and then performs a closing operation on the edge contour to make the edge contour coherent. The bubble aggregation unit takes an outer rectangle for the edge contour of each bubble, and determines whether each outer rectangle overlaps with the outer rectangles of all other bubbles. If they overlap, the two outer rectangles are merged, and the process continues to determine whether they overlap with the outer rectangles of all other bubbles until all outer rectangles no longer overlap. The bubble extraction unit takes each aggregated bounding rectangle as a contour, superimposes it onto the original image of a single chip, and extracts the contour image of each bubble. The average of the pixel grayscale values of any two points on each side of each contour image is multiplied by a scaling factor as the threshold for binarization processing. The bubble contour of each bubble is extracted, and the maximum area of the bounding rectangle of each bubble contour is taken as the bubble area of each bubble. The chip area calculation module converts the original image of a single chip into a grayscale histogram, accumulates the values of the grayscale histogram starting from grayscale 0, and uses the grayscale value when the accumulated grayscale histogram value reaches 85% of the sum of all grayscale histogram values as the threshold to perform binarization processing on the original image of a single chip, identify the chip outline, and retrieve the largest chip outline, using the area of the largest chip outline as the chip area. The bubble ratio calculation module calculates the bubble ratio, i.e., the bubble rate of the chip, based on the ratio of the bubble area to the chip area.
[0017] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: 1. Existing technologies lack targeted optimization processing when handling images obtained from different scanning methods (such as X-ray scanning and ultrasound scanning), resulting in low accuracy in bubble recognition. This patent effectively solves this problem by designing specific preprocessing algorithms for different scan images.
[0018] 2. Traditional methods for calculating bubble area are affected by factors such as light, making threshold setting difficult and resulting in poor calculation accuracy. This patent employs a series of innovative processing steps, including multiple edge extractions and threshold adjustments, and the overlay analysis of multiple extraction results, significantly improving the accuracy of bubble area calculation.
[0019] 3. Existing methods for calculating the total area of a chip have significant errors when dealing with irregularly shaped chips. This patent accurately calculates the total area of a chip by identifying the chip edges and determining the minimum circumscribed rectangle.
[0020] 4. In the past, the method of relying on manual chip bubble detection and calculation was costly and prone to errors. This patent realizes automated detection and calculation, reducing labor costs and errors.
[0021] 5. Compared to similar existing technologies or products, this patent has advantages such as faster speed than deep learning and no need for labeled data. In terms of speed, deep learning typically requires large amounts of data and a complex model training process, resulting in high computational costs and long processing times. This patent employs a unique algorithm and architecture that processes data more efficiently, significantly reducing computation time and enabling rapid results, thus improving work efficiency. Regarding data labeling, deep learning relies on large amounts of labeled data to train models. The labeling process is not only time-consuming and labor-intensive but also prone to introducing human error. This patent eliminates the need for data labeling, avoiding the tedious work and potential errors of the labeling process, saving labor and time costs, and making it more convenient to apply to real-world scenarios, especially suitable for fields where obtaining large amounts of labeled data is difficult. Attached Figure Description
[0022] Figure 1 This is a flowchart of the method described in this invention; Figure 2 This is a flowchart of the process for obtaining the preprocessed image of the board as described in this invention; Figure 3 This is a flowchart illustrating the process of obtaining the original image of a single chip as described in this invention. Figure 4 This is a flowchart of the process for obtaining the bubble area according to the present invention; Figure 5 This is a flowchart of the chip area acquisition process described in this invention; Figure 6 X-ray scan image; Figure 7 This is an ultrasound scan image; Figure 8 This is a diagram illustrating the bubble segmentation and recognition effect of the method described in this invention. Figure 9 This is a diagram showing the bubble segmentation and recognition results using the conventional threshold method. Detailed Implementation
[0023] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.
[0024] As an embodiment of the present invention, such as Figures 1 to 7As shown, a chip packaging and testing bubble rate analysis method based on image analysis includes the following steps: Step 1: As Figure 2 As shown, given a directory of board images to be processed, the type of board in the directory is determined based on its name. If the directory contains the keyword "Xray", it is identified as an X-Ray scanned board image. Figure 6 As shown; when the directory contains the keyword "UScan", it is considered an ultrasound scanning board image, such as... Figure 7 As shown. Note: Keywords used for type checking can be configured through a configuration file.
[0025] Step 2: Traverse the original images of the circuit boards in this directory, and preprocess the original images of the circuit boards according to the type of the directory: For X-ray scanned images of the board, a histogram equalization algorithm is used to enhance contrast, making the edges of bubbles clearer, resulting in a pre-processed image of the board. The specific implementation process includes the following steps: S21. Convert the X-Ray scan image to a grayscale image; S22. Perform histogram equalization using the cv2.equalizeHist() algorithm; S23. Convert back to BGR format to maintain channel consistency with the X-Ray scan.
[0026] For ultrasound scan images of the circuit board, median filtering is used to remove noise, resulting in a preprocessed image. The specific implementation process involves calling the cv2.medianBlur() algorithm for median filtering, with the kernel size set to 3, representing a 3x3 convolution kernel.
[0027] Step 3: Extract the original image of a single chip from the pre-processed image of the board, such as... Figure 3 The specific implementation process is shown below: 3.1. Pre-configuration Chip area threshold: The chip area threshold is the outline area of the chips to be processed that are statistically analyzed in advance. It can be set to 1 / 5 of the average chip area size to avoid invalid areas.
[0028] 3.2 Convert the pre-processed image of the board into a grayscale histogram. Select the grayscale values to accumulate starting from 0. Use the grayscale value when the accumulated grayscale histogram value reaches 40% of the sum of all grayscale histogram values as the first threshold. Perform binarization processing on the pre-processed image of the board to identify and extract the outline of the chip in the board. 3.3 Use the cv2.findContours function with the parameter mode set to RETR_EXTERNAL. This detects only the outermost contour and ignores the inner contour, which is suitable for extracting the outer contour of the chip. The parameter method set to CHAIN_APPROX_NONE stores all points of the contour without approximation and extracts the approximate contour of the chip. 3.4. Traverse each extracted contour, calculate the area of each contour, and filter out contours with an area less than the chip area threshold. 3.5. Obtain the minimum bounding rectangle of each chip from the filtered outlines using cv2.boundingRect; 3.6. Use the minimum bounding rectangle to crop the original image of a single chip from the preprocessed image of the board.
[0029] Step 4: Iterate through the original images of each cropped chip and preprocess each image, such as... Figure 4 The specific implementation process is shown below: 4.1 Normalization: Use cv2.normalize to normalize pixel values to the range of 0-255 to achieve brightness alignment of images from different chips; 4.2 Gaussian Blur: Reduce image noise using cv2.GaussianBlur, with a kernel size of 5x5 and a standard deviation of 1. (Kernel size and standard deviation can be adjusted based on the noise reduction effect.) 4.3 Morphological Operations: A combination of dilation and erosion is used to effectively eliminate small voids and isolated points, optimizing region integrity: First, dilate, setting the kernel size to 3 and the number of iterations to 1. Then erode, again with the kernel size set to 3 and the number of iterations to 1. Next, dilate again, setting the kernel size to 5 and the number of iterations to 1. Finally, erode again, with the kernel size set to 5 and the number of iterations to 1. This yields a preprocessed image of a single chip.
[0030] Step 5: On the pre-processed image of a single chip, bubble identification and extraction are performed. This process consists of three steps: bubble segmentation, bubble aggregation, and bubble extraction. Step 1: Bubble segmentation: (1) Input: A preprocessed image of a single chip; (2) Canny Edge Detection: The Canny algorithm is used to extract bubble edges, with dual threshold parameters: a low threshold of 55 and a high threshold of 100. Edge pixels are filtered through a hysteresis thresholding mechanism. When the pixel grayscale value is less than 55, it is judged as a non-edge and is discarded directly; When the pixel grayscale value is greater than 100, it is judged as a strong edge and is directly retained; When the pixel grayscale value is between 55 and 100, it is determined to be a weak edge and retained only if it is connected to a strong edge pixel; otherwise, it is discarded.
[0031] (3) Closing operation processing: A closing operation is performed on the Canny edge detection results. A 3×3 rectangular structuring element is selected (which can be adjusted to 5×5 according to the bubble size). Through a morphological transformation of first expansion and then erosion, the tiny breaks in the edge contour are filled, so that the bubble edge contour is continuous and complete.
[0032] Step 2: Bubble Polymerization (1) Extraction of bounding rectangle: For the bubble edge contour after the closing operation in step 1, calculate the minimum bounding rectangle of each contour using the cv2.boundingRect function, and record the coordinates of the four corners of each rectangle (x1,y1), (x2,y2), (x3,y3), (x4,y4).
[0033] (2) Overlap determination and merging: Traverse all circumscribed rectangles. For any two rectangles A and B, determine whether there is spatial overlap by using the coordinates of their four corners, i.e., whether the regions of rectangle A and rectangle B intersect. If there is overlap, merge the two rectangles into a new bounding rectangle. The boundary of the new rectangle is the smallest bounding rectangle of the two rectangles. Repeat the above process until all remaining bounding rectangles are no longer overlapping, resulting in a set of aggregated independent bubble bounding rectangles.
[0034] Step 3: Bubble Extraction (1) Region cropping: Each outer rectangle after aggregation in step 2 is superimposed on the original image of a single chip, and the image within the rectangular area is cropped as the bubble candidate region.
[0035] (2) Dynamic threshold calculation: Two sampling points are evenly selected on each side of the candidate region (a total of 8 points, located at 1 / 4 and 3 / 4 of the four sides of the rectangle, respectively). Calculate the average pixel grayscale value of the 8 sampling points, multiply the average value by a scaling factor, which is set to 0.8. The scaling factor can be adjusted between 0.6 and 0.9 according to the difference in grayscale between the bubble and the background to obtain the binarization threshold.
[0036] (3) Bubble region segmentation: The candidate region is binarized using the threshold obtained in step (2) to separate the bubble region, which is the foreground, from the background.
[0037] (4) Contour extraction and area calculation: The cv2.findContours function is used to extract contours from the binarized image. The parameters are set as follows: mode=RETR_EXTERNAL, to extract only the outermost contour, and method=CHAIN_APPROX_NONE, to retain all pixels of the contour. The area of each bubble outline is calculated using the cv2.contourArea function.
[0038] Step Six: Calculate the area of a single chip: Step 1: Obtain the original image and grayscale histogram of the chip. (1) Input: The original grayscale image of a single chip.
[0039] (2) Gray-level histogram calculation: The cv2.calcHist function is used to perform gray-level statistics on the original image and generate a histogram with gray levels of 0~255. The horizontal axis is the gray value (0 represents the darkest and 255 represents the brightest), and the vertical axis is the number of pixels corresponding to the gray value.
[0040] Step 2: Determine the binarization threshold based on the cumulative histogram (1) Accumulated pixel count: Starting from gray value 0, the number of pixels at each gray level in the histogram is accumulated sequentially.
[0041] (2) Threshold determination: When the cumulative number of pixels reaches 85% of the total number of pixels, the gray value at this time is determined as the binarization threshold. 。
[0042] Step 3: Binarization and Chip Contour Extraction (1) Binarization operation: The original image is binarized using the cv2.threshold function, with the parameter set to type=THRESH_BINARY, i.e.: When pixel grayscale value ≤ T When the area is identified as the foreground region of the chip, the pixel value is set to 255 (white). When pixel grayscale value > T When the pixel value is set to 0 (black), it is considered a background area.
[0043] The processed image yields a chip candidate region image containing only black and white binary values.
[0044] (2) Extraction of the outermost contour: The cv2.findContours function is used to extract contours from the binarized image. The parameters are set to mode=RETR_EXTERNAL, which only retrieves the outermost contour and ignores possible defect contours such as bubbles and scratches inside. The method=CHAIN_APPROX_SIMPLE compresses and stores the contour points to reduce redundant data.
[0045] Step 4: Calculate the total chip area (1) Contour filtering: Traverse all the extracted outermost contours and calculate the area of each contour (in pixels) using the cv2.contourArea function.
[0046] (2) Determine the chip outline: Since the chip body is the largest continuous area in the image, the outline with the largest area is selected. This outline is the complete boundary of the chip, which is the final total area of the chip.
[0047] Step 7: Determine the pass rate of the batch based on the ratio of the area of a single bubble to the chip area and the ratio of the total bubble area to the chip area, and export a detailed report as needed.
[0048] When the area of a single bubble exceeds the set threshold (1%, which can be modified according to quality standards), it is considered unqualified.
[0049] When the total area of bubbles on the chip exceeds the set threshold (3%, which can be modified according to quality standards), it is considered unqualified.
[0050] If one chip on a circuit board is defective, the entire circuit board is deemed defective.
[0051] Finally, the number of qualified and unqualified products is tallied, and the test results are output. For example, in this batch, there are a total of 12 boards and 152 chips, of which 9 boards are qualified and 3 are unqualified. The number of qualified chips is 139 and the number of unqualified chips is 13.
[0052] Finally, detailed data of the chip bubbles can be exported as needed for archiving.
[0053] As an embodiment of the present invention, a chip packaging and testing bubble rate analysis system based on image analysis includes an image preprocessing module, a chip segmentation module, a bubble area calculation module, a chip area calculation module, and a bubble rate calculation module; wherein: The image preprocessing module uses a histogram equalization algorithm to preprocess the original image of the board if it is an X-ray scan image, thereby enhancing the contrast and generating a preprocessed image of the board. If the original image of the board is an ultrasound scan image, it uses a median filtering algorithm to preprocess the image, thereby removing noise and generating a preprocessed image of the board. The chip segmentation module converts the pre-processed board image into a grayscale histogram. It selects the grayscale values to accumulate starting from 0 and uses the grayscale value when the accumulated grayscale histogram value reaches 40% of the sum of all grayscale histogram values as the threshold. It then performs binarization processing on the generated pre-processed board image to identify the chip outline and uses the minimum bounding rectangle of each chip outline to crop the original image of a single chip. The bubble area calculation module includes a single-chip preprocessing unit, a bubble segmentation unit, a bubble aggregation unit, and a bubble extraction unit, wherein: The single-chip preprocessing unit normalizes and Gaussian blurs the original image of a single chip to align the image brightness and reduce noise. Then, it performs a combination of dilation and erosion operations to eliminate holes and isolated points, thus obtaining a preprocessed image of a single chip. The bubble segmentation unit uses a dual thresholding method to segment the edge contour of each bubble from the preprocessed image of a single chip through a hysteresis thresholding operation, and then performs a closing operation on the edge contour to make the edge contour coherent. The bubble aggregation unit takes an outer rectangle for the edge contour of each bubble, and determines whether each outer rectangle overlaps with the outer rectangles of all other bubbles. If they overlap, the two outer rectangles are merged, and the process continues to determine whether they overlap with the outer rectangles of all other bubbles until all outer rectangles no longer overlap. The bubble extraction unit takes each aggregated bounding rectangle as a contour, superimposes it onto the original image of a single chip, and extracts the contour image of each bubble. The average of the pixel grayscale values of any two points on each side of each contour image is multiplied by a scaling factor as the threshold for binarization processing. The bubble contour of each bubble is extracted, and the maximum area of the bounding rectangle of each bubble contour is taken as the bubble area of each bubble. The chip area calculation module converts the original image of a single chip into a grayscale histogram, accumulates the values of the grayscale histogram starting from grayscale 0, and uses the grayscale value when the accumulated grayscale histogram value reaches 85% of the sum of all grayscale histogram values as the threshold to perform binarization processing on the original image of a single chip, identify the chip outline, and retrieve the largest chip outline, using the area of the largest chip outline as the chip area. The bubble ratio calculation module calculates the bubble ratio, i.e., the bubble rate of the chip, based on the ratio of the bubble area to the chip area.
[0054] The automated bubble detection of this system greatly improves the efficiency and accuracy of chip quality inspection, reducing the inspection time for a single board from about one minute to less than one second, while significantly improving accuracy and automatically generating inspection reports for archiving.
[0055] by Figure 7Taking the ultrasound scan image shown as an example, the effects of conventional thresholding methods and the method described in this invention on bubble segmentation and identification are compared. Figure 8 and Figure 9 As shown in the figure and the specific data given in the table below, it can be seen that the average bubble ratio measured based on binary segmentation (parameter values H: 0–180, S: 0–255, V: 130–255) is 0.907%, with an error of 0.351% compared to the true value of 0.556%. In contrast, the average bubble ratio measured by this method is 0.540%, with an error of only 0.016% compared to the true value of 0.556%, demonstrating a significant improvement. Furthermore, the degree of optimization in recognition performance is more intuitively apparent from the recognition image. (The table below shows the data.) Figure 8 The bubble proportion segmentation and recognition effect of the method described in this invention is demonstrated; Figure 9 The results of bubble proportion segmentation and recognition using the conventional thresholding method are demonstrated. The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.
Claims
1. A method for analyzing bubble rate in chip packaging based on image analysis, characterized in that: Includes the following steps: S1. Select X-Ray scan images or ultrasound scan images from the imported original images of the board, perform the first preprocessing, and obtain the preprocessed images of the board. S2. Convert the pre-processed image of the board into a grayscale histogram. Select the grayscale values to accumulate starting from 0. Use the grayscale value when the accumulated grayscale histogram value reaches 40% of the sum of all grayscale histogram values as the first threshold. Perform binarization processing on the pre-processed image of the board to identify and extract the outline of the chip in the board. Extract the original image of a single chip by using the minimum bounding rectangle of each chip outline. S3. Convert the original image of a single chip into a grayscale histogram. Starting from a grayscale value of 0, accumulate the values of the grayscale histogram. Use the grayscale value when the accumulated grayscale histogram value reaches 85% of the sum of all grayscale histogram values as the second threshold. Perform binarization processing on the original image of a single chip to identify the chip outline, and use the largest retrieved chip outline area as the chip area. At the same time, perform a second preprocessing on the original image of a single chip to obtain a preprocessed image of a single chip. Then, sequentially go through the bubble segmentation, bubble aggregation, and bubble extraction processes to obtain the bubble outline of each bubble. Use the largest area of the bounding rectangle of each bubble outline as the bubble outline area of each bubble. Calculate the area of all bubbles, i.e., the bubble area. S4. Calculate the percentage of the bubble area to the chip area based on the bubble area and the chip area; The first preprocessing step involves using a histogram equalization algorithm to enhance contrast if the original image of the imported board is an X-ray scan; and using a median filtering algorithm to remove noise if the original image of the imported board is an ultrasound scan. The second preprocessing step involves normalizing and Gaussian blurring the original image of a single chip to align the image brightness and reduce noise. Then, a combination of dilation and erosion operations are performed to eliminate holes and isolated points. Bubble segmentation employs a dual-thresholding approach to extract the edge contours of each bubble from a preprocessed image of a single chip using a hysteresis thresholding operation. Then, a closing operation is performed on each edge contour to ensure that each edge contour is coherent. Bubble aggregation involves taking an outer rectangle for the edge contour of each bubble, and determining whether each outer rectangle overlaps with the outer rectangles of all other bubbles. If they overlap, the two outer rectangles are merged, and the process continues until all outer rectangles no longer overlap. Bubble extraction involves taking the bounding rectangle of each aggregated image as a contour and overlaying it onto the original image of a single chip. The contour image of each bubble is then extracted. The average pixel grayscale of any two points on each side of each contour image is multiplied by a scaling factor to serve as the threshold for binarization, thereby extracting the bubble contour of each bubble.
2. The chip packaging and testing bubble rate analysis method based on image analysis according to claim 1, characterized in that: The process of preprocessing X-ray scan images using a histogram equalization algorithm includes the following steps: S21. Convert the X-Ray scan image to a grayscale image; S22. Perform histogram equalization using the cv2.equalizeHist() algorithm; S23. Convert back to BGR format to maintain channel consistency with the X-Ray scan.
3. The chip packaging and testing bubble rate analysis method based on image analysis according to claim 1, characterized in that: The process of preprocessing ultrasound scans using a median filtering algorithm includes the following steps: S31. Select the kernel size for median filtering; S32. Perform median filtering using the cv2.medianBlur() algorithm.
4. The chip packaging and testing bubble rate analysis method based on image analysis according to claim 3, characterized in that: The specific process of obtaining the original image of a single chip includes the following steps: S41. Binarize the preprocessed image of the board according to the first threshold to initially separate the chip area; S42. Use the OpenCV cv2.findContours function with the mode parameter RETR_EXTERNAL to detect only the outermost edge contour of the initially separated chip region, ignoring the inner contour, to identify the edge contour of the chip. S43. The parameter method uses CHAIN_APPROX_NONE to store all points of the edge contour without approximation, extracts the chip edge contour, and obtains the minimum bounding rectangle of each chip edge contour through cv2.boundingRect, cropping the original image of a single chip from the board preprocessing image.
5. The chip packaging and testing bubble rate analysis method based on image analysis according to claim 4, characterized in that: The specific process of the second preprocessing includes the following steps: S51. Normalize and Gaussian blur the original image of the cropped individual chip to align the image brightness and reduce noise. S52. By combining expansion and erosion, voids and isolated points are eliminated, the integrity of the edge contour is optimized, and a preprocessed image of a single chip is obtained.
6. The chip packaging and testing bubble rate analysis method based on image analysis according to claim 5, characterized in that: The specific process of bubble segmentation includes the following steps: S61. The Canny algorithm is used to perform hysteresis thresholding on the preprocessed image of a single chip using double thresholds to obtain the bubble edge contour. S62. Perform a closing operation on the bubble edge contour to make the bubble edge contour more coherent. The hysteresis thresholding operation selects two pixel thresholds, a low one and a high one. Pixels with values below the low pixel threshold are discarded, while those with values above the high pixel threshold are immediately retained. Pixels with values between the low and high pixel thresholds are only retained if they are connected to strong edges.
7. The chip packaging and testing bubble rate analysis method based on image analysis according to claim 6, characterized in that: The specific process of bubble aggregation is as follows: the bubble edge contour obtained during the bubble segmentation process is used to take the outer rectangle. Based on the coordinates of the four corners of each outer rectangle, it is determined whether it overlaps with the outer rectangles of all other bubbles. If there is an overlap, the outer rectangles of the two bubbles are merged and then the judgment is continued with the outer rectangles of all other bubbles. This operation is repeated until all outer rectangles no longer overlap.
8. The chip packaging and testing bubble rate analysis method based on image analysis according to claim 7, characterized in that: The specific process of bubble extraction includes the following steps: S81. Using the bounding rectangle of the aggregated bubble as the outline, superimpose it onto the original image of a single chip, and extract the original image of the corresponding area on the image. S82. Take the average of the pixel grayscale values of any two points on each side of the bubble image in the original image and multiply it by the scaling factor to use as the threshold for binarization. Segment the bubble region from the original image of a single chip. S83. For the binarized image, use the cv2.findContours function, with the mode parameter set to RETR_EXTERNAL and the method parameter set to CHAIN_APPROX_NONE to extract the bubble edge contours, and calculate the area of each edge contour using cv2.contourArea. The proportional coefficient is configured through a configuration file, with a value between 0.6 and 0.
9. The default value is 0.8, and it can be adjusted according to the extraction effect.
9. A chip packaging and testing bubble rate analysis system based on image analysis, characterized in that: The chip packaging and testing bubble rate analysis system includes an image preprocessing module, a chip segmentation module, a bubble area calculation module, a chip area calculation module, and a bubble rate calculation module; wherein: The image preprocessing module uses a histogram equalization algorithm to preprocess the original image of the board if it is an X-ray scan image, thereby enhancing the contrast and generating a preprocessed image of the board. If the original image of the board is an ultrasound scan image, it uses a median filtering algorithm to preprocess the image, thereby removing noise and generating a preprocessed image of the board. The chip segmentation module converts the pre-processed board image into a grayscale histogram. It selects the grayscale values to accumulate starting from 0 and uses the grayscale value when the accumulated grayscale histogram value reaches 40% of the sum of all grayscale histogram values as the threshold. It then performs binarization processing on the generated pre-processed board image to identify the chip outline and uses the minimum bounding rectangle of each chip outline to crop the original image of a single chip. The bubble area calculation module includes a single-chip preprocessing unit, a bubble segmentation unit, a bubble aggregation unit, and a bubble extraction unit, wherein: The single-chip preprocessing unit normalizes and Gaussian blurs the original image of a single chip to align the image brightness and reduce noise. Then, it performs a combination of dilation and erosion operations to eliminate holes and isolated points, thus obtaining a preprocessed image of a single chip. The bubble segmentation unit uses a dual thresholding method to segment the edge contour of each bubble from the preprocessed image of a single chip through a hysteresis thresholding operation, and then performs a closing operation on the edge contour to make the edge contour coherent. The bubble aggregation unit takes an outer rectangle for the edge contour of each bubble, and determines whether each outer rectangle overlaps with the outer rectangles of all other bubbles. If they overlap, the two outer rectangles are merged, and the process continues to determine whether they overlap with the outer rectangles of all other bubbles until all outer rectangles no longer overlap. The bubble extraction unit takes each aggregated bounding rectangle as a contour, superimposes it onto the original image of a single chip, and extracts the contour image of each bubble. The average of the pixel grayscale values of any two points on each side of each contour image is multiplied by a scaling factor as the threshold for binarization processing. The bubble contour of each bubble is extracted, and the maximum area of the bounding rectangle of each bubble contour is taken as the bubble area of each bubble. The chip area calculation module converts the original image of a single chip into a grayscale histogram, accumulates the values of the grayscale histogram starting from grayscale 0, and uses the grayscale value when the accumulated grayscale histogram value reaches 85% of the sum of all grayscale histogram values as the threshold. It then performs binarization processing on the original image of a single chip, identifies the chip outline, retrieves the largest chip outline, and uses the area of the largest chip outline as the chip area. The bubble ratio calculation module calculates the bubble ratio, i.e., the bubble rate of the chip, based on the ratio of the bubble area to the chip area.