A chip package defect detection method

Abstract

The application discloses a kind of chip package defect detection methods, it is related to chip detection field, the application is by feeding position, visible light detection position, material handover position, infrared light detection position and discharge position are arranged side by side on rack, and by first conveying module in visible light detection position lifting carrier plate to be scanned head to each carrier plate unit is carried out line scanning imaging, second conveying module in infrared light detection position lifting carrier plate to be imaged by infrared system, the physical separation and process relay of visible light and infrared detection are realized;On this basis, based on the position offset of each carrier plate unit visible light image acquisition and surface defect, in combination with the internal hidden crack of each chip body infrared image recognition, and comprehensive multi-source results generate unit level determination, both avoid the efficiency bottleneck of traditional start-stop mode, and overcome the coverage blind area of single modal detection, significantly improve the beat, integrity and accuracy of high-capacity production line chip package defect detection.

Description

Technical Field

[0001] This invention relates to the field of chip inspection, and more particularly to a method for detecting defects in chip packages. Background Technology

[0002] In flip-chip (FC) packaging, diced wafers are mounted onto a carrier substrate capable of holding multiple chips, and then proceed to subsequent processes such as reflow soldering. To ensure mounting quality, an automated optical inspection (Die-bonding AOI) device is typically used to inspect the chips on the carrier substrate online after the FC equipment and before reflow soldering. Existing DB-AOI equipment is generally based on visible light area array imaging technology, using a white light coaxial light source and an area array camera. Driven by a motor, it aligns the chips one by one, requiring repeated deceleration, stopping, stabilization, imaging, and restarting processes. This results in long inspection cycles and low throughput, making it difficult to meet the needs of high-volume production lines. Furthermore, its inspection capabilities are limited to defects that can be imaged under visible light, such as positional misalignment, visible cracks, edge chipping, and surface scratches.

[0003] However, such equipment has a more serious functional limitation: it cannot detect microcracks (hidden cracks). Microcracks are usually introduced by improper parameter or conveyor adjustment in the FC process. They are located inside or on the bottom of the chip and are invisible under visible light, making them very easy to miss. Once a chip with a microcrack flows into the reflow soldering process, it may expand into a visible crack after a long period of thermal cycling, directly causing the chip to be scrapped. Even if it does not show a visible crack, the microcrack may have already damaged the internal circuitry and can only be exposed in the subsequent electrical testing stage, resulting in wasted production capacity, time delays, and delayed process adjustments to the FC equipment, which in turn leads to a wider range of batch defects. Therefore, there is an urgent need for a comprehensive inspection method that can efficiently cover the entire substrate and integrate visible light and infrared imaging to simultaneously solve the two major bottlenecks of insufficient detection capability and low detection efficiency. Summary of the Invention

[0004] To simultaneously detect visible surface defects and internal microcracks and improve detection efficiency, this invention proposes a chip package defect detection method, applied to a chip inspection device. The device includes a frame, a first conveying module, a second conveying module, a visible light imaging system, and an infrared light imaging system. The frame is equipped with a feed position, a material transfer position, a discharge position, a visible light detection position, and an infrared light detection position arranged side-by-side. The visible light detection position is located between the feed position and the material transfer position, and the infrared light detection position is located between the material transfer position and the discharge position.

[0005] The first conveying module is used to convey the carrier plate from the feeding position to the material transfer position, and lift the carrier plate through its lifting platform when the carrier plate reaches the visible light detection position. The visible light imaging system is located above the visible light detection position and is used to perform visible light imaging on the lifted carrier plate.

[0006] The second conveying module is used to convey the carrier plate from the material transfer position to the discharge position, and lifts the carrier plate through its lifting platform when the carrier plate reaches the infrared light detection position. The infrared light imaging system is located above the infrared light detection position and is used to perform infrared imaging on the lifted carrier plate.

[0007] The chip package includes a carrier board; the carrier board is provided with multiple carrier board units, and each carrier board unit is provided with alignment marks, components and a chip body; the detection method includes:

[0008] Visible light images of each carrier plate unit in the carrier plate are acquired using a visible light imaging system.

[0009] Based on visible light images, the positioning coordinates of alignment marks, components, and chip bodies in the corresponding carrier board unit are obtained through template matching. Based on the positioning coordinates, the positional offsets of the chip body and components relative to the alignment marks are calculated, and the positional offsets are used to determine whether there are positional assembly defects in the carrier board unit. The surface detection area corresponding to the chip body is delineated, and cracks and dirt defects are identified in the surface detection area through spot analysis.

[0010] Infrared images of each chip on the carrier board are acquired using an infrared imaging system; and for each chip's infrared image, a combination of image analysis and deep learning is used to detect hidden cracks and defects inside the corresponding chip.

[0011] By combining the identification results of assembly defects, cracks, and contamination defects at the corresponding locations of each carrier board unit, as well as the detection results of hidden crack defects inside the chip body, a comprehensive defect judgment result is generated for each carrier board unit.

[0012] Furthermore, the acquisition of visible light images corresponding to each carrier unit in the carrier board through the visible light imaging system specifically involves: using the visible light imaging system to perform line scanning imaging of each carrier unit on the carrier board along a preset Z-shaped path to acquire the corresponding visible light images;

[0013] The method involves obtaining the positioning coordinates of alignment marks, components, and the chip body in the corresponding carrier unit through template matching based on visible light images; specifically:

[0014] Obtain pre-constructed alignment mark templates, component templates, and chip body templates; wherein the alignment mark templates, component templates, and chip body templates are respectively extracted from the visible light image of a standard carrier unit, and correspond to reference images of a single alignment mark, a single component, and a single chip body;

[0015] The alignment mark template, chip body template, and component template are respectively matched with the entire visible light image corresponding to the carrier unit. The position with the highest similarity in each template matching result is determined as the positioning coordinate of the alignment mark, chip body, and component.

[0016] Furthermore, based on the positional offset, it is determined whether the carrier plate unit has a positional assembly defect; specifically:

[0017] The first net offset is obtained by subtracting the position offset of the chip body relative to the alignment mark from the standard position offset of the chip body; the second net offset is obtained by subtracting the position offset of the component relative to the alignment mark from the standard position offset of the component.

[0018] If the absolute value of the first net offset is greater than the first preset threshold, or the absolute value of the second net offset is greater than the second preset threshold, then it is determined that there is a positional assembly defect in the chip body or component.

[0019] Furthermore, the step of defining the surface detection area corresponding to the chip body specifically involves: determining the outline area of ​​the chip body in the visible light image based on the positioning coordinates of the chip body and the preset chip size, and expanding the outline area outward by a preset edge margin to obtain the surface detection area.

[0020] Furthermore, cracks and contamination defects are identified within the surface inspection area through spot analysis; specifically:

[0021] Within the surface detection area, pixels whose grayscale values ​​deviate from the preset normal range are extracted, and the pixels are clustered into multiple connected regions based on pixel connectivity, with each connected region serving as a candidate spot;

[0022] Calculate the attribute parameters of each candidate spot, and select the candidate spots that meet the defect judgment conditions according to the preset attribute threshold as significant defect spots;

[0023] Determine whether the significant defect spot is connected to the boundary of the surface detection area: if connected, it is determined to be a crack defect; if not connected, it is determined to be a dirt defect.

[0024] Furthermore, the attribute parameters of the candidate spots include: area, aspect ratio, roundness, average gray value, gray standard deviation, maximum gray value, and minimum gray value;

[0025] The step of selecting candidate spots that meet the defect determination criteria based on preset attribute thresholds as significant defect spots specifically involves:

[0026] Candidate spots that meet any of the following conditions are identified as significant defect spots:

[0027] The area is greater than the preset area threshold;

[0028] The aspect ratio is greater than the preset aspect ratio threshold;

[0029] The roundness is less than the preset roundness threshold;

[0030] The standard deviation of grayscale is greater than the preset standard deviation threshold for grayscale.

[0031] If any of the average grayscale value, maximum grayscale value, or minimum grayscale value exceeds the corresponding preset grayscale range.

[0032] Furthermore, the acquisition of infrared images of each chip on the carrier board via an infrared imaging system specifically involves:

[0033] Based on the physical size of the chip body and the preset effective field of view of the infrared imaging system, the number of scans required for each chip body in the horizontal and vertical directions is determined; wherein, the effective field of view is smaller than the actual imaging field of view of the infrared imaging system, so that there is an overlap between the image areas acquired by two adjacent scans, to ensure that a single chip body is completely covered.

[0034] Following a zigzag path starting from the lower right corner of the carrier board, each chip body is traversed sequentially. For each chip body, starting from its corresponding infrared scanning start point, the imaging of the entire scanning area of ​​the chip body is completed in a zigzag stepping manner according to the required number of horizontal and vertical scans, thereby obtaining the infrared image corresponding to the chip body.

[0035] Furthermore, the method for obtaining the infrared scanning starting point is as follows:

[0036] The first net offset of the corresponding chip body is compensated to the pre-set standard infrared scanning start point coordinates to obtain the infrared scanning start point corresponding to the chip body.

[0037] Furthermore, the infrared images of each chip body are used to detect hidden cracks and defects inside the corresponding chip body using a combination of image analysis and deep learning. Specifically:

[0038] For each chip body corresponding to the infrared image, spot analysis is performed to detect areas with gray values ​​lower than a preset threshold, and these areas are identified as serious hidden crack defects; wherein, the preset threshold is set according to the gray value statistics of the chip body area in the infrared image of a standard defect-free chip, and is less than the minimum gray value of the area.

[0039] Simultaneously, the infrared image corresponding to the chip body is input into a pre-trained deep learning model; the deep learning model is used to identify defect areas with gray values ​​within the normal gray range but exhibiting thin, elongated lines or fracture texture structures, and to determine them as minor hidden crack defects.

[0040] Furthermore, the method for obtaining the deep learning model is as follows:

[0041] Infrared images of multiple chip bodies were collected as raw samples;

[0042] Images containing hidden crack regions, whose gray values ​​are within the normal gray range and which exhibit thin, elongated lines or fracture texture structures, are selected from the original samples and used as target training samples.

[0043] The hidden crack regions in the target training samples are annotated at the pixel level or at the bounding box level to generate corresponding training labels;

[0044] The target training samples and their corresponding training labels are combined to form a training dataset, and the deep learning network is trained under supervision based on the training dataset to obtain a deep learning model for detecting minor hidden crack defects.

[0045] To address the aforementioned problems, another aspect of this invention provides an electronic device, including: a processor and a memory storing a program, the program including instructions that, when executed by the processor, cause the processor to perform the chip package defect detection method described above.

[0046] To address the aforementioned problems, in another aspect of this invention, a non-transitory machine-readable medium storing computer instructions is provided, the computer instructions being used to cause the computer to execute the chip package defect detection method described above.

[0047] Compared with the prior art, the present invention has at least the following beneficial effects:

[0048] (1) To address the problems of long detection cycle, low production capacity, and difficulty in simultaneously detecting surface and internal defects caused by the use of area array cameras to start and stop taking pictures one chip at a time in existing DB-AOI equipment, this invention sets up a feeding position, a visible light detection position, a material transfer position, an infrared light detection position, and a discharging position side by side on the rack. The first conveying module lifts the carrier plate at the visible light detection position so that the scanning head can perform line scanning imaging on each carrier plate unit, and the second conveying module lifts the carrier plate at the infrared light detection position so that the infrared system can perform imaging. This achieves physical separation and process relay of visible light and infrared detection. On this basis, the position offset and surface defects are obtained based on the visible light images of each carrier plate unit. The internal hidden cracks are identified by combining the infrared images of each chip body. The unit-level judgment is generated by integrating the results from multiple sources. This not only avoids the efficiency bottleneck of the traditional start-stop mode, but also overcomes the coverage blind spot of single-mode detection, which significantly improves the cycle time, completeness and accuracy of chip package defect detection on high-capacity production lines.

[0049] (2) The present invention enables the scanning head of the visible light imaging system to perform line scanning imaging of each carrier unit on the carrier along a preset Z-shaped path, avoiding the frequent acceleration and deceleration and stabilization waiting time caused by the "start-stop-take" mode of the traditional area array camera, realizing high-speed and continuous visible light image acquisition, effectively improving the detection capacity and meeting the needs of high-capacity production lines.

[0050] (3) In view of the problem of limited field of view of infrared imaging, the present invention adopts a Z-shaped path starting from the lower right corner of the carrier to traverse each chip body, and performs Z-shaped step scanning inside each chip body while ensuring that adjacent image areas overlap, thus ensuring that the large-size chip body is completely and without omission. At the same time, combined with the compensation of the infrared scanning starting point based on the visible light positioning results, the coverage and positioning accuracy of infrared microcrack detection are significantly improved.

[0051] (4) The present invention integrates the position offset in the visible light image, the surface spot analysis results and the hidden crack detection results in the infrared image, and integrates three-dimensional defect information of geometric assembly, surface appearance and internal structure. It overcomes the limitations of single-mode detection that is prone to missed detection or misjudgment, and greatly improves the reliability of chip package defect detection and yield control level. Attached Figure Description

[0052] Figure 1 This is a flowchart of a chip package defect detection method according to an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the chip detection device of the present invention;

[0054] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0055] The following are specific embodiments of the present invention, which are described in conjunction with the accompanying drawings. However, the present invention is not limited to these embodiments.

[0056] Example 1

[0057] To simultaneously achieve comprehensive detection of visible surface defects and internal microcracks, and to improve detection efficiency, such as... Figure 1 As shown, this invention proposes a chip package defect detection method, applied to a chip inspection device; the device includes a frame, a first conveying module, a second conveying module, a visible light imaging system, and an infrared light imaging system; the frame is provided with a feeding position, a material transfer position, a discharging position, a visible light detection position, and an infrared light detection position arranged side by side; the visible light detection position is located between the feeding position and the material transfer position, and the infrared light detection position is located between the material transfer position and the discharging position;

[0058] The first conveying module is used to convey the carrier plate from the feeding position to the material transfer position, and lift the carrier plate through its lifting platform when the carrier plate reaches the visible light detection position. The visible light imaging system is located above the visible light detection position and is used to perform visible light imaging on the lifted carrier plate.

[0059] The second conveying module is used to convey the carrier plate from the material transfer position to the discharge position, and lifts the carrier plate through its lifting platform when the carrier plate reaches the infrared light detection position. The infrared light imaging system is located above the infrared light detection position and is used to perform infrared imaging on the lifted carrier plate.

[0060] The chip package includes a carrier board; the carrier board is provided with multiple carrier board units, and each carrier board unit is provided with alignment marks, components and a chip body; the detection method includes:

[0061] Visible light images of each carrier plate unit in the carrier plate are acquired using a visible light imaging system.

[0062] The acquisition of visible light images corresponding to each carrier unit in the carrier board through the visible light imaging system specifically involves: using the visible light imaging system to perform line scanning imaging of each carrier unit on the carrier board along a preset Z-shaped path to acquire the corresponding visible light images;

[0063] This invention enables the scanning head of a visible light imaging system to perform line scanning imaging of each carrier unit on the carrier along a preset Z-shaped path, avoiding the frequent acceleration and deceleration and stabilization waiting time caused by the "start-stop-take" mode of traditional area array cameras. This achieves high-speed, continuous visible light image acquisition, effectively improving detection capacity and meeting the needs of high-capacity production lines.

[0064] Based on visible light images, the positioning coordinates of alignment marks, components, and chip bodies in the corresponding carrier board unit are obtained through template matching. Based on the positioning coordinates, the positional offsets of the chip body and components relative to the alignment marks are calculated, and the positional offsets are used to determine whether there are positional assembly defects in the carrier board unit. The surface detection area corresponding to the chip body is delineated, and cracks and dirt defects are identified in the surface detection area through spot analysis.

[0065] The method involves obtaining the positioning coordinates of alignment marks, components, and the chip body in the corresponding carrier unit through template matching based on visible light images; specifically:

[0066] Obtain pre-constructed alignment mark templates, component templates, and chip body templates; wherein the alignment mark templates, component templates, and chip body templates are respectively extracted from the visible light image of a standard carrier unit, and correspond to reference images of a single alignment mark, a single component, and a single chip body;

[0067] The alignment mark template, chip body template, and component template are respectively matched with the entire visible light image corresponding to the carrier unit. The position with the highest similarity in each template matching result is determined as the positioning coordinate of the alignment mark, chip body, and component.

[0068] In this embodiment, to achieve precise positioning of key structures within the carrier unit, the system employs a template matching method to identify the positions of alignment marks, chip bodies, and components. Specifically, firstly, the entire visible light image corresponding to the carrier unit is acquired; then, pre-constructed alignment mark templates, chip body templates, and component templates are sequentially matched against the visible light image. During the matching process, for each template, the system traverses all possible positions in the visible light image within a sliding window corresponding to its size, calculating the similarity between the template and the sub-image within the current window (e.g., using the normalized cross-correlation coefficient (NCC) as a similarity metric). Since each template is extracted from a standard defect-free carrier unit image, it has a clear geometric contour, edge features, and grayscale distribution. While the corresponding target object in the actual carrier unit may have slight offsets, its overall appearance remains consistent. Therefore, a significantly higher similarity peak than the background and other interfering areas will be generated near the actual location of the target. The system records the similarity values ​​of all positions and selects the position with the highest value as the optimal matching point. This position is then determined as the positioning coordinates of the corresponding target object (alignment mark, chip body, or component) in the visible light image. This method enables the stable and accurate acquisition of the spatial positions of key elements even in the presence of slight changes in lighting, surface reflection, or interference from adjacent structures, providing a reliable benchmark for subsequent position offset calculations, surface defect identification, and comprehensive judgment.

[0069] Based on the positional offset, it is determined whether the carrier plate unit has a positional assembly defect; specifically:

[0070] The first net offset is obtained by subtracting the position offset of the chip body relative to the alignment mark from the standard position offset of the chip body; the second net offset is obtained by subtracting the position offset of the component relative to the alignment mark from the standard position offset of the component.

[0071] If the absolute value of the first net offset is greater than the first preset threshold, or the absolute value of the second net offset is greater than the second preset threshold, then it is determined that there is a positional assembly defect in the chip body or component.

[0072] In this embodiment, to accurately determine whether there is an assembly position deviation between the chip body and the components, the system uses "net offset" as the core criterion. The specific process is as follows: First, based on the positioning coordinates identified in the visible light image, the actual position offset of the chip body relative to the alignment mark and the actual position offset of the components relative to the same alignment mark are calculated. This actual offset includes two components: a horizontal component (X-axis) and a vertical component (Y-axis), reflecting the measured relative position between each component in the current carrier unit and the alignment mark.

[0073] At the same time, the system pre-stores the "standard position offset" for the corresponding product model, which is the theoretical relative position of the chip body or component with respect to the alignment mark under ideal good condition. For multiple chip bodies on the same carrier board, the standard position offset is shared and consistent, without the need to set it separately for each chip, thereby simplifying the configuration and improving the detection efficiency.

[0074] Subsequently, the system subtracts the measured position offset from the corresponding standard position offset in each direction: for the chip body, the actual offset is subtracted from the standard offset of the chip body in both the X and Y directions to obtain the first net offset; for the component, the actual offset is similarly subtracted from the standard offset of the component in both the X and Y directions to obtain the second net offset. This "net offset" essentially eliminates the fixed bias introduced by the product design itself, retaining only the actual assembly deviation caused by mounting errors, equipment drift, or process fluctuations.

[0075] Finally, the system determines whether the absolute value of the first net offset in the X direction or the absolute value in the Y direction exceeds a first preset threshold, and whether the absolute value of the second net offset in the X direction or the absolute value in the Y direction exceeds a second preset threshold. If either direction exceeds the threshold, the corresponding component is determined to have a positional assembly defect. By introducing a net offset mechanism, misjudgments caused by design asymmetry are effectively avoided, significantly improving the accuracy and adaptability of the detection.

[0076] The process of defining the surface detection area corresponding to the chip body is as follows: based on the positioning coordinates of the chip body and the preset chip size, the outline area of ​​the chip body is determined in the visible light image, and the surface detection area is obtained by expanding the outline area outward by a preset edge margin.

[0077] Cracks and contamination defects are identified within the surface inspection area using spot analysis; specifically:

[0078] Within the surface detection area, pixels whose grayscale values ​​deviate from the preset normal range are extracted, and the pixels are clustered into multiple connected regions based on pixel connectivity, with each connected region serving as a candidate spot;

[0079] Calculate the attribute parameters of each candidate spot, and select the candidate spots that meet the defect judgment conditions according to the preset attribute threshold as significant defect spots;

[0080] The attribute parameters of the candidate spots include: area, aspect ratio, roundness, average gray value, gray standard deviation, maximum gray value, and minimum gray value.

[0081] The step of selecting candidate spots that meet the defect determination criteria based on preset attribute thresholds as significant defect spots specifically involves:

[0082] Candidate spots that meet any of the following conditions are identified as significant defect spots:

[0083] The area is greater than the preset area threshold;

[0084] The aspect ratio is greater than the preset aspect ratio threshold;

[0085] The roundness is less than the preset roundness threshold;

[0086] The standard deviation of grayscale is greater than the preset standard deviation threshold for grayscale.

[0087] If any of the average grayscale value, maximum grayscale value, or minimum grayscale value exceeds the corresponding preset grayscale range.

[0088] Determine whether the significant defect spot is connected to the boundary of the surface detection area: if connected, it is determined to be a crack defect; if not connected, it is determined to be a dirt defect.

[0089] Infrared images of each chip on the carrier board are acquired using an infrared imaging system; and for each chip's infrared image, a combination of image analysis and deep learning is used to detect hidden cracks and defects inside the corresponding chip.

[0090] The process of acquiring infrared images of each chip on the carrier board using an infrared imaging system specifically involves:

[0091] Based on the physical size of the chip body and the preset effective field of view of the infrared imaging system, the number of scans required for each chip body in the horizontal and vertical directions is determined; wherein, the effective field of view is smaller than the actual imaging field of view of the infrared imaging system, so that there is an overlap between the image areas acquired by two adjacent scans, to ensure that a single chip body is completely covered.

[0092] Following a zigzag path starting from the lower right corner of the carrier board, each chip body is traversed sequentially. For each chip body, starting from its corresponding infrared scanning start point, the imaging of the entire scanning area of ​​the chip body is completed in a zigzag stepping manner according to the required number of horizontal and vertical scans, thereby obtaining the infrared image corresponding to the chip body.

[0093] This invention addresses the problem of limited field of view in infrared imaging by employing a zigzag path starting from the lower right corner of the carrier board to traverse each chip body. It also performs a zigzag step scan inside each chip body while ensuring that adjacent image areas overlap, thus ensuring that the large-size chip body is completely and without omission. At the same time, it combines compensation for the infrared scanning starting point based on visible light positioning results, which significantly improves the coverage and positioning accuracy of infrared microcrack detection.

[0094] The method for obtaining the infrared scanning starting point is as follows:

[0095] The first net offset of the corresponding chip body is compensated to the pre-set standard infrared scanning start point coordinates to obtain the infrared scanning start point corresponding to the chip body.

[0096] In this embodiment, the pre-set standard infrared scanning starting point coordinates refer to the reference point used to initiate infrared imaging scanning in the theoretical position of the chip body under ideal assembly conditions. This reference point is usually selected as the lower right corner of the chip body outline to match the "Z-shaped path starting from the lower right corner of the carrier board" adopted by the overall system, ensuring that the scanning process is continuous, efficient, and without omissions.

[0097] When the chip body in the actual carrier board unit deviates from its theoretical position due to mounting deviation, using a fixed standard starting point for infrared scanning may lead to scan area offset, resulting in missed detection of some areas of the chip body. To address this, the system introduces a dynamic compensation mechanism: based on the first net offset of the chip body obtained during the visible light detection stage (i.e., its actual assembly deviation relative to the alignment mark), the coordinates of the standard infrared scanning starting point are corrected. Specifically, according to the horizontal and vertical components of the first net offset, the coordinate values ​​of the standard starting point are adjusted accordingly (for example, by adding the components of the first net offset in the corresponding directions to the X and Y values ​​of the standard infrared scanning starting point coordinates), thereby obtaining an infrared scanning starting point adapted to the actual position of the current chip. Through this compensation, the infrared imaging system can accurately align with the offset chip body and begin a zigzag step scan of the internal area from near its actual lower right corner, ensuring complete and accurate coverage of the entire chip body and effectively avoiding the problem of missed detection of microcracks due to positional deviation.

[0098] The infrared images of each chip body are used to detect hidden cracks and defects inside the corresponding chip body using a combination of image analysis and deep learning. Specifically:

[0099] For each chip body corresponding to the infrared image, spot analysis is performed to detect areas with gray values ​​lower than a preset threshold, and these areas are identified as serious hidden crack defects; wherein, the preset threshold is set according to the gray value statistics of the chip body area in the infrared image of a standard defect-free chip, and is less than the minimum gray value of the area.

[0100] Simultaneously, the infrared image corresponding to the chip body is input into a pre-trained deep learning model; the deep learning model is used to identify defect areas with gray values ​​within the normal gray range but exhibiting thin, elongated lines or fracture texture structures, and to determine them as minor hidden crack defects.

[0101] The method for obtaining the deep learning model (in this embodiment, the model is the YOLOv8 model) is as follows:

[0102] Infrared images of multiple chip bodies were collected as raw samples;

[0103] Images containing hidden crack regions, whose gray values ​​are within the normal gray range and which exhibit thin, elongated lines or fracture texture structures, are selected from the original samples and used as target training samples.

[0104] The hidden crack regions in the target training samples are annotated at the pixel level or at the bounding box level to generate corresponding training labels;

[0105] The target training samples and their corresponding training labels are combined to form a training dataset, and the deep learning network is trained under supervision based on the training dataset to obtain a deep learning model for detecting minor hidden crack defects.

[0106] By combining the identification results of assembly defects, cracks, and contamination defects at the corresponding locations of each carrier board unit, as well as the detection results of hidden crack defects inside the chip body, a comprehensive defect judgment result is generated for each carrier board unit.

[0107] In this embodiment, the comprehensive defect determination is implemented as follows: For each carrier board unit, the system acquires its corresponding positional assembly defect determination result (based on the positional offset of the chip body and components relative to the alignment mark), surface defect identification result (including defect type (crack or dirt) and defect location), and detection result of microcrack defects inside the chip body; then, the above three types of defect information are summarized, and the final comprehensive defect determination result of the carrier board unit is generated according to the preset comprehensive determination logic. The summarized defect information includes: whether the chip body or components have assembly position deviations, whether there are cracks or dirt on the surface, and whether there are microcracks inside the chip body; if any defect type is determined to be unqualified, the entire carrier board unit is marked as NG; otherwise, it is marked as OK. The final determination result and the corresponding defect information are output together for subsequent processing or recording. This process achieves the organic integration of multimodal detection results, ensuring the comprehensiveness and consistency of defect determination.

[0108] This invention comprehensively judges the positional offset in visible light images, the surface spot analysis results, and the hidden crack detection results in infrared images. It integrates three-dimensional defect information of geometric assembly, surface appearance, and internal structure, overcoming the limitations of single-modal detection that is prone to missed detection or misjudgment, and greatly improving the reliability of chip package defect detection and yield control.

[0109] To address the problems of long inspection cycles, low throughput, and difficulty in simultaneously detecting surface and internal defects in existing DB-AOI equipment due to the use of area array cameras for chip-by-chip start-stop imaging, this invention addresses these issues by arranging a feeding position, a visible light detection position, a material transfer position, an infrared light detection position, and a discharge position side-by-side on the rack. A first conveying module lifts the carrier plate at the visible light detection position for line scanning imaging of each carrier plate unit, while a second conveying module lifts the carrier plate at the infrared light detection position for infrared system imaging. This achieves physical separation and workflow relay between visible light and infrared detection. Furthermore, based on the visible light images of each carrier plate unit to obtain positional offsets and surface defects, combined with the infrared images of each chip body to identify internal hidden cracks, and integrating multi-source results to generate unit-level judgments, this invention avoids the efficiency bottleneck of traditional start-stop modes and overcomes the coverage blind spots of single-modal detection, significantly improving the cycle time, completeness, and accuracy of chip package defect detection on high-capacity production lines.

[0110] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, which, when executed by the at least one processor, causes the electronic device to perform the method of this invention.

[0111] The present invention also provides a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform the method of the present invention.

[0112] This invention also provides a computer program product, including a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform the method of this invention.

[0113] refer to Figure 3 The present invention will now describe a structural block diagram of an electronic device that can serve as a server or client in embodiments of the present invention, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0114] like Figure 3 As shown, the electronic device includes a computing unit 401, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. The RAM 403 may also store various programs and data required for the operation of the electronic device. The computing unit 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0115] Multiple components in the electronic device are connected to I / O interface 405, including: input unit 406, output unit 407, storage unit 408, and communication unit 409. Input unit 406 can be any type of device capable of inputting information into the electronic device. Input unit 406 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of the electronic device. Output unit 407 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 408 may include, but is not limited to, disks and optical discs. Communication unit 409 allows the electronic device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0116] The computing unit 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, CPUs, graphics processing units (GPUs), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 401 performs the methods and processes described above. For example, in some embodiments, the method embodiments of the present invention can be implemented as a computer program tangibly contained in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed on an electronic device via ROM 402 and / or communication unit 409. In some embodiments, the computing unit 401 can be configured to perform the methods described above by any other suitable means (e.g., by means of firmware).

[0117] Computer programs for implementing the methods of embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0118] In the context of embodiments of the present invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0119] Example 2

[0120] This embodiment provides a chip detection device, such as... Figure 2 As shown, it mainly includes:

[0121] The frame 1 has multiple feeding positions 101, material transfer positions 102 and discharge positions 103 arranged side by side. The carrier board 6 carrying the chip is fed in from the feeding position 101 and output from the discharge position 103 after passing through the material transfer position 102. The frame 1 has a first longitudinal track 11 and a second longitudinal track 12 along the arrangement direction of the multiple feeding positions 101.

[0122] The first conveying module 2 is slidably mounted on the first longitudinal track 11 to switch between multiple feeding positions 101. The first conveying module 2 includes a first lifting platform 20 and a first transverse track 21. A first conveying mechanism 22 is provided on the first transverse track 21. The first conveying mechanism 22 can convey the carrier plate 6 from the feeding position 101 along the first transverse track 21 to the material transfer position 102. The first lifting platform 20 can lift the carrier plate 6 to disengage the carrier plate 6 from the first transverse track 21.

[0123] The visible light imaging system 3 is mounted on the rack 1 and located above the first conveying module 2. It is configured to perform visible light detection on the chip on the carrier plate 6 supported by the first lifting platform 20.

[0124] The second conveying module 4 is slidably mounted on the second longitudinal track 12. The second conveying module 4 includes a second lifting platform 40 and a second transverse track 41. A second conveying mechanism 42 is provided on the second transverse track 41. The second conveying mechanism 42 can convey the carrier plate 6 from the material transfer position 102 to the discharge position 103 along the second transverse track 41. The second lifting platform 40 can lift the carrier plate 6 to disengage the carrier plate 6 from the second transverse track 41.

[0125] Specifically, the frame 1 is provided with multiple material transfer positions 102 and multiple discharge positions 103. The feeding position 101, the material transfer position 102 and the discharge position 103 correspond one to one. The second longitudinal track 12 is arranged along the arrangement direction of the multiple material transfer positions 102. The second conveying module 4 slides along the second longitudinal track 12 to switch between the multiple material transfer positions 102.

[0126] The infrared imaging system 5 is mounted on the frame 1 and located above the second conveying module 4. It is configured to perform infrared light detection on the chip on the carrier plate 6 supported by the second lifting platform 40.

[0127] In practical use, this embodiment introduces an infrared imaging system 5 into the chip inspection device. After the chip on the carrier board 6 undergoes visible light inspection by the visible light imaging system 3, the carrier board 6 carrying the chip flows into the second lifting platform 40. The second lifting platform 40 lifts the carrier board 6, bringing it close to the infrared imaging system 5. The infrared imaging system 5 can perform infrared inspection on the chip on the carrier board 6, imaging the hidden cracks (invisible cracks) inside the chip, and promptly detecting chips with hidden cracks, preventing them from flowing into subsequent processes and thus delaying time and wasting production capacity. Furthermore, the infrared imaging system 5 operates on the second lifting platform 40, while the visible light imaging system 3 operates on the first lifting platform 20; both can be performed independently, effectively improving inspection efficiency.

[0128] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0129] Furthermore, in this invention, descriptions involving terms such as "first," "second," and "a" are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0130] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0131] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

Claims

1. A method for detecting defects in a chip package, characterized in that, This device is used in chip testing equipment. It includes a frame, a first conveying module, a second conveying module, a visible light imaging system, and an infrared light imaging system. The frame is equipped with a feeding position, a material transfer position, a discharging position, a visible light detection position, and an infrared light detection position arranged side-by-side. The visible light detection position is located between the feeding position and the material transfer position, and the infrared light detection position is located between the material transfer position and the discharging position. The first conveying module is used to convey the carrier plate from the feeding position to the material transfer position, and lift the carrier plate through its lifting platform when the carrier plate reaches the visible light detection position. The visible light imaging system is located above the visible light detection position and is used to perform visible light imaging on the lifted carrier plate. The second conveying module is used to convey the carrier plate from the material transfer position to the discharge position, and lifts the carrier plate through its lifting platform when the carrier plate reaches the infrared light detection position. The infrared light imaging system is located above the infrared light detection position and is used to perform infrared imaging on the lifted carrier plate. The chip package includes a carrier board; the carrier board is provided with multiple carrier board units, and each carrier board unit is provided with alignment marks, components and a chip body; the detection method includes: Visible light images of each carrier plate unit in the carrier plate are acquired using a visible light imaging system. Based on visible light images, the positioning coordinates of alignment marks, components, and chip bodies in the corresponding carrier board unit are obtained through template matching. Based on the positioning coordinates, the positional offsets of the chip body and components relative to the alignment marks are calculated, and the positional offsets are used to determine whether there are positional assembly defects in the carrier board unit. The surface detection area corresponding to the chip body is delineated, and cracks and dirt defects are identified in the surface detection area through spot analysis. Infrared images of each chip on the carrier board are acquired using an infrared imaging system; and for each chip's infrared image, a combination of image analysis and deep learning is used to detect hidden cracks and defects inside the corresponding chip. By combining the identification results of assembly defects, cracks, and contamination defects at the corresponding locations of each carrier board unit, as well as the detection results of hidden crack defects inside the chip body, a comprehensive defect judgment result is generated for each carrier board unit.

2. The chip package defect detection method according to claim 1, characterized in that, The acquisition of visible light images corresponding to each carrier unit in the carrier board through the visible light imaging system specifically involves: using the visible light imaging system to perform line scanning imaging of each carrier unit on the carrier board along a preset Z-shaped path to acquire the corresponding visible light images; The method involves obtaining the positioning coordinates of alignment marks, components, and the chip body in the corresponding carrier unit through template matching based on visible light images; specifically: Obtain pre-constructed alignment mark templates, component templates, and chip body templates; wherein the alignment mark templates, component templates, and chip body templates are respectively extracted from the visible light image of a standard carrier unit, and correspond to reference images of a single alignment mark, a single component, and a single chip body; The alignment mark template, chip body template, and component template are respectively matched with the entire visible light image corresponding to the carrier unit. The position with the highest similarity in each template matching result is determined as the positioning coordinate of the alignment mark, chip body, and component.

3. The chip package defect detection method according to claim 1, characterized in that, Based on the positional offset, it is determined whether the carrier plate unit has a positional assembly defect; specifically: The first net offset is obtained by subtracting the position offset of the chip body relative to the alignment mark from the standard position offset of the chip body; the second net offset is obtained by subtracting the position offset of the component relative to the alignment mark from the standard position offset of the component. If the absolute value of the first net offset is greater than the first preset threshold, or the absolute value of the second net offset is greater than the second preset threshold, then it is determined that there is a positional assembly defect in the chip body or component.

4. The chip package defect detection method according to claim 1, characterized in that, The process of defining the surface detection area corresponding to the chip body is as follows: based on the positioning coordinates of the chip body and the preset chip size, the outline area of ​​the chip body is determined in the visible light image, and the surface detection area is obtained by expanding the outline area outward by a preset edge margin.

5. The chip package defect detection method according to claim 1, characterized in that, Cracks and contamination defects are identified within the surface inspection area using spot analysis; specifically: Within the surface detection area, pixels whose grayscale values ​​deviate from the preset normal range are extracted, and the pixels are clustered into multiple connected regions based on pixel connectivity, with each connected region serving as a candidate spot; Calculate the attribute parameters of each candidate spot, and select the candidate spots that meet the defect judgment conditions according to the preset attribute threshold as significant defect spots; Determine whether the significant defect spot is connected to the boundary of the surface detection area: if connected, it is determined to be a crack defect; if not connected, it is determined to be a dirt defect.

6. The chip package defect detection method according to claim 5, characterized in that, The attribute parameters of the candidate spots include: area, aspect ratio, roundness, average gray value, gray standard deviation, maximum gray value, and minimum gray value. The step of selecting candidate spots that meet the defect determination criteria based on preset attribute thresholds as significant defect spots specifically involves: Candidate spots that meet any of the following conditions are identified as significant defect spots: The area is greater than the preset area threshold; The aspect ratio is greater than the preset aspect ratio threshold; The roundness is less than the preset roundness threshold; The standard deviation of grayscale is greater than the preset standard deviation threshold for grayscale. If any of the average grayscale value, maximum grayscale value, or minimum grayscale value exceeds the corresponding preset grayscale range.

7. The chip package defect detection method according to claim 3, characterized in that, The process of acquiring infrared images of each chip on the carrier board using an infrared imaging system specifically involves: Based on the physical size of the chip body and the preset effective field of view of the infrared imaging system, the number of scans required for each chip body in the horizontal and vertical directions is determined; wherein, the effective field of view is smaller than the actual imaging field of view of the infrared imaging system, so that there is an overlap between the image areas acquired by two adjacent scans, to ensure that a single chip body is completely covered. Following a zigzag path starting from the lower right corner of the carrier board, each chip body is traversed sequentially. For each chip body, starting from its corresponding infrared scanning start point, the imaging of the entire scanning area of ​​the chip body is completed in a zigzag stepping manner according to the required number of horizontal and vertical scans, thereby obtaining the infrared image corresponding to the chip body.

8. The chip package defect detection method according to claim 7, characterized in that, The method for obtaining the infrared scanning starting point is as follows: The first net offset of the corresponding chip body is compensated to the pre-set standard infrared scanning start point coordinates to obtain the infrared scanning start point corresponding to the chip body.

9. The chip package defect detection method according to claim 1, characterized in that, The infrared images of each chip body are used to detect hidden cracks and defects inside the corresponding chip body using a combination of image analysis and deep learning. Specifically: For each chip body corresponding to the infrared image, spot analysis is performed to detect areas with gray values ​​lower than a preset threshold, and these areas are identified as serious hidden crack defects; wherein, the preset threshold is set according to the gray value statistics of the chip body area in the infrared image of a standard defect-free chip, and is less than the minimum gray value of the area. Simultaneously, the infrared image corresponding to the chip body is input into a pre-trained deep learning model; the deep learning model is used to identify defect areas with gray values ​​within the normal gray range but exhibiting thin, elongated lines or fracture texture structures, and to determine them as minor hidden crack defects. The method for obtaining the deep learning model is as follows: Infrared images of multiple chip bodies were collected as raw samples; Images containing hidden crack regions, whose gray values ​​are within the normal gray range and which exhibit thin, elongated lines or fracture texture structures, are selected from the original samples and used as target training samples. The hidden crack regions in the target training samples are annotated at the pixel level or at the bounding box level to generate corresponding training labels; The target training samples and their corresponding training labels are combined to form a training dataset, and the deep learning network is trained under supervision based on the training dataset to obtain a deep learning model for detecting minor hidden crack defects.

10. An electronic device, comprising: A processor and a memory storing a program, characterized in that the program includes instructions that, when executed by the processor, cause the processor to perform the chip package defect detection method according to any one of claims 1 to 9.

Images