Ball grid array packaging ball mounting visual detection method and system

Abstract

The invention relates to the field of integrated circuit packaging testing, in particular to a ball grid array packaging ball mounting visual detection method and system. The method comprises the following steps: acquiring a strip image; obtaining the boundary of each chip unit according to the mark points on the strip image; sequentially acquiring images of single chip units according to the boundary of each chip; sequentially sending the acquired images of the single chip units to a processing thread for visual detection processing; and after all the chip units on the strip are processed, splicing the images of the single chip units into an integral spliced strip image, and marking a visual detection and identification result on the integral spliced strip image. The chip units on the strips are acquired one by one in a virtual segmentation mode and are subjected to visual detection processing, and images of different chip units can be subjected to parallel processing through different threads. And the processing speed and precision are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit packaging and testing, and in particular to a visual inspection method and system for ball grid array packaging ball placement. Background Technology

[0002] Ball grid array (BGA) is a common form of chip packaging. In this packaging, one side of the substrate is soldered to the chip, and the other side of the substrate is connected to external circuitry via solder balls. The solder balls are mass-produced on the substrate through a ball-mounting process.

[0003] As integrated circuit packaging technology advances towards higher density and miniaturization, BGA ball placement, a critical step in the packaging process, directly determines the reliability of the chip's electrical connections and the yield of the final product. However, the solder balls are extremely small, typically 0.15–0.5 mm in diameter, and are produced in vast quantities, ranging from tens of thousands to hundreds of thousands in a single placement. Furthermore, the strip contains multiple units, making traditional manual visual inspection inefficient, subjective, and prone to visual fatigue leading to missed or incorrect inspections. Currently, the industry widely employs automated optical inspection (AOI) systems, which use an industrial camera to capture images of the entire strip or most of its area at once, and then utilize image processing algorithms to perform a global analysis of the solder balls within the image.

[0004] Due to the large area captured in a single shot and the large number of solder balls, each image has high resolution and a massive amount of data, resulting in an extremely high computational load during global image processing. This severely restricts the detection speed and makes it difficult to achieve the real-time or near-real-time detection cycle required by the production line. Secondly, there are issues with image distortion and depth of field: during large field-of-view shooting, distortion at the lens edges significantly affects the measurement accuracy of solder ball positions. Simultaneously, due to potential slight warping or unevenness in the strip, it is difficult to ensure that all solder balls are clearly imaged under large depth-of-field requirements, leading to inaccurate measurement of solder balls in the edge areas due to defocusing. Furthermore, existing systems typically inspect only a single item, have low levels of intelligence, and most can only perform basic binarization processing to detect obvious defects such as missing or extra balls. They lack the capability to detect complex defects such as minute changes in ball diameter and defects in the incoming substrate itself (e.g., pad contamination or damage). Finally, the system lacks flexibility; when the layout or size of units on the strip changes, the entire field of view needs to be recalibrated, resulting in poor adaptability and long changeover times. Summary of the Invention

[0005] Firstly, in order to solve the problems of large data processing volume and high computational load in the prior art, the embodiments of this application provide a visual inspection method for ball grid array packaging and ball placement.

[0006] The method includes: Obtain the strip image; Based on the marker points on the strip image, obtain the boundary of each chip cell; Images of individual chip units are sequentially obtained based on the boundaries of each chip; The images of each individual chip unit are sequentially sent to a processing thread for visual detection processing; After all the chip units on the strip are processed, the images of the individual chip units are stitched together to form a complete stitched strip image, and the visual detection and recognition results are marked on the complete stitched strip image.

[0007] By employing the above method, the ball-planting visual inspection method of this application acquires and performs visual inspection processing on chip units on the strip image by image through virtual segmentation, allowing images of different chip units to be processed in parallel using different threads. This significantly improves processing speed and accuracy.

[0008] In one possible implementation, acquiring the strip image includes: Divide the strip into multiple strip images according to integer multiples of the number of threads; Acquire and process a partial stripe image in a single step.

[0009] In one possible implementation, obtaining the boundary of each chip cell based on the marker points on the strip image includes: Identify the actual coordinates of a local strip based on some of the marker points on the strip; The XYR offset information of the local strip is obtained based on the actual coordinates of the local strip; The boundaries of chip cells are separated by a grid formed by horizontal and vertical lines connecting the marked points on the strip.

[0010] In one possible implementation, sequentially acquiring images of individual chip units based on the boundaries of each chip includes: The image of each chip unit is acquired line by line using a high-magnification telecentric lens for sequential scanning. The image of each chip unit is acquired based on the XYR offset information to perform offset correction.

[0011] In one possible implementation, sequentially sending the acquired images of individual chip units to a processing thread for visual detection processing includes: In one processing thread, the image of the individual chip unit is preprocessed; Locate the center coordinates and outline of the solder ball on the image of the individual chip unit; The parameters of the solder balls were measured and their quantities compared. The measured value is compared with the preset tolerance range to determine whether the chip unit is qualified. If it is unqualified, the defect type and location are recorded.

[0012] In one possible implementation, the step of sequentially sending the acquired images of individual chip units to a processing thread for visual detection processing includes: Acquire and analyze the relative position and shape of the solder ball and the pad below to detect pad defects.

[0013] In one possible implementation, measuring the solder ball parameters includes: Based on sub-pixel edge detection technology, the diameter of each solder ball is accurately calculated; Calculate the spacing between adjacent solder balls based on their center coordinates; The measured center coordinates of the solder ball are compared with the preset center coordinates of the pads on the standard template, and the offset is calculated.

[0014] In one possible implementation, measuring and comparing the solder ball parameters includes determining whether there are missing or extra balls by comparing them with a standard ball count.

[0015] Secondly, embodiments of this application also provide a visual inspection system for ball grid array packaging ball placement, the system comprising: The image acquisition module acquires the strip image; The marker recognition module obtains the boundary of each chip unit based on the markers on the strip image; The image scanning module sequentially acquires images of individual chip units based on the boundaries of each chip; Multiple thread processing modules, each sequentially performing visual detection processing on the images of a single chip unit acquired. Attached Figure Description

[0016] Figure 1 This is a flowchart of the method according to the first embodiment of the present invention; Figure 2 This is a schematic diagram of some of the strip images of the present invention; Figure 3 This is a schematic diagram of the marking points and chip unit areas on a portion of the strip of the present invention; Figure 4 This is a schematic diagram of the chip unit images stitched together according to an embodiment of the present invention. Detailed Implementation

[0017] The following detailed description, in conjunction with specific embodiments and accompanying drawings, clarifies that the described embodiments are only a portion, not all, of the embodiments. All other embodiments obtained by those skilled in the art based on the following embodiments without inventive effort are also within the scope of protection of this invention.

[0018] Because existing ball grid array (BGA) packaging ball placement visual inspection typically involves capturing images of the entire strip, it requires a wide-angle lens to obtain a larger field of view to encompass the entire strip. Furthermore, the large number of balls on the strip results in a very high computational load, slow processing speed, and low efficiency when visually processing numerous balls. These are the key technical problems that the following embodiments aim to solve.

[0019] like Figure 1 As shown, the first embodiment provides a visual inspection method for ball grid array packaging. The method includes the following steps: S1. Obtain the strip image; S2. Obtain the boundary of each chip unit based on the marked points on the strip image; S3. Sequentially acquire images of individual chip units based on the boundaries of each chip; S4. The images of the acquired individual chip units are sequentially sent to a processing thread for visual detection processing; S5. After processing all the chip units on the strip, the images of the individual chip units are stitched together into a whole stitched strip image, and the visual detection and recognition results are marked on the whole stitched strip image.

[0020] In the first embodiment, the imaging step S1 is mainly used to acquire the marker points in step S2. The marker points are preset marks on the strip. In the prior art, by acquiring the marker points, the XYR offset of the product can be obtained. Then, during chip unit cutting, the XYR offset can be used to adjust the offset so that the chip unit cutting can be performed accurately. Theoretically, by acquiring any three marker points, the overall XYR offset of the strip can be calculated based on the coordinate difference between the image position and the actual position.

[0021] Marker points are geometric feature points pre-designed and fabricated on the board during the manufacturing process of the packaging substrate strip. They are typically cross-shaped, T-shaped, circular, or have special patterns. They serve as the basis for coordinate reference and geometric correction by the machine vision system, used for global positioning and coordinate system establishment (reference function). Defining the global coordinate system, the marker points are the sole physical reference for the vision system to establish the strip's global coordinate system. The system first identifies marker points located at key positions on the strip (such as the four corners or edges) and determines their pixel coordinates. By comparing the actual pixel coordinates of the marker points with the theoretical standard coordinates, the system can accurately calculate the absolute position (X, Y coordinates) of the entire strip relative to the camera or motion platform. If the strip has a slight rotation or tilt when transported to the inspection station, the positioning frame alone cannot accurately correct it. By identifying at least two marker points and calculating the slope of the line connecting them, the system can accurately calculate the strip's rotation angle. Compensation and pre-correction: The calculated angle and position deviations are immediately used for pre-correction of the motion platform to ensure that the camera can accurately align with the target area during subsequent high-magnification scanning. Nonlinear Compensation for Geometric Distortion and Minor Deformation Correction: Ideally, a strip is a perfect rectangle, but actual substrate strips may experience minor nonlinear deformations (such as stretching or trapezoidal distortion) during manufacturing or transportation. By identifying multiple marker points (typically four or more), the system can establish a geometric correction model (such as an affine or perspective transformation matrix). Improved Accuracy: This model accurately maps pixel coordinates on the image to real physical coordinates, crucial for ensuring subsequent sub-pixel precision measurement accuracy. Precise Virtual Segmentation: Based on the precise coordinate system and geometric correction model established by the marker points, combined with chip cell dimensions and spacing stored in the product database, the system accurately calculates and segments the boundary region of each independent chip cell in the software. Parallel Processing Foundation: Only with accurate geometric segmentation provided by the marker points can each chip cell region be treated as an independent, parallelizable subtask, guiding the high-magnification camera to perform precise local scanning and data processing.

[0022] Preferably, in this embodiment, to address the distortion at the corners when acquiring the entire strip image using a wide-angle lens, distortion correction and other processing are required simultaneously with visual recognition, further reducing computational speed. Therefore, this embodiment employs a high-magnification telecentric lens for image acquisition. Since the field of view of a high-magnification telecentric lens is smaller than that of a wide-angle lens, it cannot acquire the entire strip image at once before segmentation. Therefore, to acquire the strip image, the acquisition of the strip image includes: Divide the strip into multiple strip images according to integer multiples of the number of threads; Acquire and process a partial stripe image in a single step.

[0023] The following section uses the processing of partial striped images to illustrate the processing method in detail, such as... Figure 2 As shown, the acquired partial strip image is a part of the overall strip. Images acquired through a high-magnification circular center lens can produce virtually no distortion. Identify the actual coordinates of a local strip based on some of the marker points on the strip; The XYR offset information of the local strip is obtained based on the actual coordinates of the local strip; The grid formed by connecting the horizontal and vertical lines of the marked points on the strip serves as the boundary for separating chip cells.

[0024] When proceeding to step S3, if Figure 3 As shown, in this embodiment, in one possible implementation, acquiring images of individual chip units sequentially according to the boundary of each chip includes acquiring images of each chip unit line by line using a high-magnification telecentric lens for sequential scanning.

[0025] Specifically, it includes: Based on the virtual boundary coordinates of the chip unit calculated in step S2, and the possible angle and position pre-correction amounts, the camera or strip is driven to perform precise relative movement. The camera or stage will: Move above the precise center position of the first chip unit.

[0026] Stop moving, perform precise focusing and image acquisition, and obtain a local high-magnification image of the first chip unit.

[0027] Quickly move to the center position above the next chip cell.

[0028] Stop moving, refocus and acquire data to obtain a local high-magnification image of the next chip unit.

[0029] This process involves scanning and imaging each chip unit region on the strip one by one, independently. Each acquired local image data is an independent, high-precision detection object.

[0030] The method of acquiring multiple portions of the entire strip using a high-magnification telecentric lens, and then sequentially scanning the chip unit image after each partial acquisition, differs from the traditional large-field-of-view single-shot imaging approach in three aspects: 1. Eliminate or greatly reduce geometric distortion to ensure positioning accuracy. Traditional wide-field lenses, in order to cover the entire strip in a single image, typically produce noticeable pincushion or barrel distortion at the edges of the field of view. This distortion causes a non-linear deviation between the image position of the solder ball and its actual physical position.

[0031] Telecentric lenses feature a unique parallel optical path design, maintaining a consistent magnification even at the edges of the field of view, thus eliminating or significantly reducing geometric distortion. Simultaneously, the high magnification means a smaller field of view, further ensuring near-perfect local image reproduction, which is a prerequisite for achieving sub-pixel and micrometer-level precision measurements (such as spherical offset and interspherical distance).

[0032] 2. Overcome depth-of-field issues and ensure image sharpness. Although the packaging substrate strip is rigid, it may still have slight warping or unevenness. If a wide-field single-shot imaging is used, it is difficult to ensure that all solder balls are in the optimal focal plane of the lens, and some solder balls will be out of focus and blurry.

[0033] When using a high-magnification lens for local scanning, the field of view of each chip unit area is very small. This allows the system to perform precise autofocus on each local area, ensuring that all solder balls in that area are in optimal clarity. This overcomes the impact of strip unevenness on detection accuracy.

[0034] 3. Supports independent processing of different regions to ensure computational efficiency. If the large image is simply sliced ​​by software, the computing resources are still wasted on processing the redundant information in the large image, and the distortion and blurring of the large image itself cannot be recovered.

[0035] Local scanning directly acquires high-quality, high-resolution, and low-distortion target data. These independent, high-quality local images can be directly fed into various independent parallel threads for precise calculations, thus ensuring the efficiency of precise measurement and multi-threaded regional parallel computing.

[0036] Preferably, since some steps in visual processing require comparing the image with a template to obtain the comparison result, in order to correct the offset of the individual chip unit image, the image of the individual chip unit is obtained sequentially according to the boundary of each chip, including: The image of each chip unit is acquired line by line using a high-magnification telecentric lens for sequential scanning. The image of each chip unit is acquired based on the XYR offset information to perform offset correction.

[0037] In this embodiment, sequentially sending the acquired images of individual chip units to a processing thread for visual detection processing includes: In one processing thread, the image of the individual chip unit is preprocessed; Locate the center coordinates and outline of the solder ball on the image of the individual chip unit; The parameters of the solder balls were measured and their quantities compared. The measured value is compared with the preset tolerance range to determine whether the chip unit is qualified. If it is unqualified, the defect type and location are recorded.

[0038] In this step, a key technical aspect of the present invention lies in transforming a large-scale, high-load global detection task into multiple completely independent, parallel-executable "unit tasks" through virtual segmentation and independent regional acquisition, thereby fully utilizing the parallel computing capabilities of multi-core processors.

[0039] The parallel capabilities of multi-core CPUs: Modern high-performance industrial computers are typically equipped with multi-core central processing units (CPUs), such as quad-core, hexa-core, or octa-core CPUs and more. In the relationship between cores and threads, a CPU core can independently execute one task. Through Hyper-Threading technology, one physical core can simulate two logical threads. Multi-core CPUs can process multiple independent instruction sequences simultaneously or alternately in a very short time; this is called parallel computing. In the traditional "whole strip single-image" scheme, the image processing of the entire strip (sub-pixel measurement of tens of thousands of solder balls) is considered a huge, single computational task that must be completed sequentially by one or a few threads of the CPU, resulting in slow processing speeds that cannot meet industrial cycle time requirements.

[0040] After the aforementioned virtual segmentation and chip-by-chip scanning, the image data of each chip unit is a complete and independent image file, which can be efficiently allocated to CPU resources along with the processing tasks.

[0041] In this embodiment, the multi-threaded task allocation process includes: after the image acquisition of a chip unit is completed and entered into memory, the main control program of the detection software immediately sends a request to the operating system to create a new, independent detection thread. The newly created thread is allocated two key elements: the memory address of the image data of a chip unit; and the precision measurement algorithm code block, including sub-pixel edge detection, tolerance comparison, etc. At this time, assuming the CPU has 8 cores, after the image data of eight chip units are acquired sequentially and threads are created, the operating system will allocate these 8 independent threads to 8 different CPU cores or logic processors. Since these 8 threads execute independent and independent calculations, they can run simultaneously on 8 cores, consuming CPU resources together. This parallel mechanism significantly reduces the total serial processing time of the original eight image processing steps to the time of processing a single chip unit image, which depends on the slowest thread. During processing, each thread only reads and writes its own chip unit image and result variables, avoiding resource contention and data conflicts between threads. The main control program continuously monitors the status of all unit task threads. Once a thread completes the inspection of its unit, it reports the result (OK / NG) to the main program and releases the computing resources it occupied. Only when all unit threads have completed the inspection does the main program proceed to summarize and output the results of each step and control the sorting mechanism.

[0042] Therefore, the improvement of this invention lies particularly in leveraging the multi-threaded processing capabilities of the processor to solve the problems of large data processing volume and low efficiency in the original single image processing task. Furthermore, based on the premise of simultaneous processing of multiple tasks, images from each chip unit are acquired independently. This independent acquisition by each chip unit solves the problem of requiring a wide-angle lens to capture the entire stripe image. Moreover, by using a high-magnification telecentric lens to complete the overall stripe image capture multiple times, the problems of image quality and image distortion are resolved. This results in a complete improvement and optimization of the entire process, including multi-threaded task allocation, image imaging logic, and camera hardware.

[0043] In this embodiment, the step of sequentially sending the acquired images of individual chip units to a processing thread for visual detection processing includes: Acquire and analyze the relative position and shape of the solder ball and the pad below to detect pad defects.

[0044] In one possible implementation, measuring the solder ball parameters includes: Based on sub-pixel edge detection technology, the diameter of each solder ball is accurately calculated; Calculate the spacing between adjacent solder balls based on their center coordinates; The measured center coordinates of the solder ball are compared with the preset center coordinates of the pads on the standard template, and the offset is calculated.

[0045] In this embodiment, the measurement and quantity comparison of the solder ball parameters includes determining whether there are missing or extra balls by comparing them with the standard ball count.

[0046] After the visual processing of the planted ball is completed, such as Figure 4 As shown, in order to demonstrate the situation on the entire strip and intuitively show the location of problematic solder balls on the strip, in this embodiment, step S5 further evaluates the images of all individual chip units. Specifically, after processing all chip units on the strip, the images of the individual chip units are stitched together into a complete stitched strip image, and the visual detection and recognition results are marked on the complete stitched strip image.

[0047] Additionally, embodiments of this application also provide a visual inspection system for ball grid array packaging ball placement, the system comprising: The image acquisition module acquires the strip image; The marker recognition module obtains the boundary of each chip unit based on the markers on the strip image; The image scanning module sequentially acquires images of individual chip units based on the boundaries of each chip; Multiple thread processing modules are used, each sequentially performing visual detection processing on the acquired images of a single chip unit. The above description is merely a preferred embodiment of this application and does not limit the scope of disclosure of the embodiments of this application. Any equivalent structural or procedural transformations made based on the description and drawings of the embodiments of this application, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope supported by the embodiments of this application.

Claims

1. A method for visually inspecting ball grid array package ball placement, the method comprising: include: Obtain the strip image; Based on the marker points on the strip image, obtain the boundary of each chip cell; Images of individual chip units are sequentially obtained based on the boundaries of each chip; The images of each individual chip unit are sequentially sent to a processing thread for visual detection processing; After all the chip units on the strip are processed, the images of the individual chip units are stitched together to form a complete stitched strip image, and the visual detection and recognition results are marked on the complete stitched strip image.

2. The method of claim 1, wherein the step of detecting the ball grid array package is performed by using a camera. The acquisition of the strip image includes: Divide the strip into multiple strip images according to integer multiples of the number of threads; Acquire and process a partial stripe image in a single step.

3. The method of claim 1, wherein the step of detecting the ball grid array package is performed by using a camera.

3. The method of claim 1, wherein the step of detecting the ball grid array package is performed by using a camera. The step of obtaining the boundary of each chip unit based on the marker points on the strip image includes: Identify the actual coordinates of a local strip based on some of the marker points on the strip; The XYR offset information of the local strip is obtained based on the actual coordinates of the local strip; The boundaries of chip cells are separated by a grid formed by horizontal and vertical lines connecting the marked points on the strip.

4. The method of claim 3, wherein the step of detecting the number of balls is performed by using a method of detecting a number of balls in a ball grid array package by using a camera, a light source, and a lens. The step of sequentially acquiring images of individual chip units based on the boundaries of each chip includes: The image of each chip unit is acquired line by line using a high-magnification telecentric lens for sequential scanning. The image of each chip unit is acquired based on the XYR offset information to perform offset correction.

5. The method of claim 1, wherein the step of detecting the ball grid array package is performed by using a camera. 5 The step of sequentially sending the acquired images of individual chip units to a processing thread for visual detection processing includes: In one processing thread, the image of the individual chip unit is preprocessed; Locate the center coordinates and outline of the solder ball on the image of the individual chip unit; The parameters of the solder balls were measured and their quantities compared. The measured value is compared with the preset tolerance range to determine whether the chip unit is qualified. If it is unqualified, the defect type and location are recorded.

6. The ball grid array package ball mounting visual inspection method of claim 5, wherein, The step of sequentially sending the acquired images of individual chip units to a processing thread for visual detection processing also includes: Acquire and analyze the relative position and shape of the solder ball and the pad below to detect pad defects.

7. The ball grid array package ball mounting visual inspection method according to claim 5 or 6, wherein The measurement of the solder ball parameters includes: Based on sub-pixel edge detection technology, the diameter of each solder ball is accurately calculated; Calculate the spacing between adjacent solder balls based on their center coordinates; The measured center coordinates of the solder ball are compared with the preset center coordinates of the pads on the standard template, and the offset is calculated.

8. The ball grid array package ball mounting visual inspection method of claim 5, wherein, The measurement and quantity comparison of the solder ball parameters includes determining whether there are missing or extra balls by comparing them with the standard ball count.

9. A ball grid array package ball placement vision inspection system implementing the method of any one of claims 1-8, characterized in that, The system includes: The image acquisition module acquires the strip image; The marker recognition module obtains the boundary of each chip unit based on the markers on the strip image; The image scanning module sequentially acquires images of individual chip units based on the boundaries of each chip; Multiple thread processing modules, each sequentially performing visual detection processing on the images of a single chip unit acquired.

Images