An image recognition-based full-automatic bonding detection system and method

Abstract

The application discloses a kind of based on image recognition's bonding full-automatic detection system and method, belong to semiconductor packaging quality detection technical field, bonding area image is collected by multi-angle vision sensor, each bonding point is positioned and its coordinate, size and arc height are recorded by identification, and node network model is constructed according to spatial distribution structure;In model, select starting node to set reference, along preset path, reference value is sequentially transferred, each node is judged whether to update reference by comparing measured value and received value, and deviation state is recorded simultaneously;During the process, reference adjustment amplitude is recorded in real time, the accumulated adjustment amount of each path is calculated, abnormal transmission path is identified, and the density of deviated node is counted to mark abnormal area;Finally, according to the number of abnormal path and the area of abnormal area, bonding quality is judged, and the position of unqualified product is automatically calculated and laser marking is executed.

Description

Technical Field

[0001] This invention relates to the field of semiconductor packaging quality inspection technology, specifically a fully automated bonding inspection system and method based on image recognition. Background Technology

[0002] In semiconductor packaging manufacturing, bonding is one of the core processes, and its quality directly determines the electrical performance and reliability of the device. Bonding quality mainly depends on key parameters such as bonding ball size, arc height, crater morphology, and IMC layer integrity. Defects such as abnormal bonding ball size, lead arc deformation, excessively deep craters, or missing IMC layers can lead to open circuits, short circuits, or shortened device lifespan.

[0003] Currently, the industry commonly uses two main types of inspection methods: The first is traditional manual visual inspection, where operators examine each bonding point individually under a microscope. This method is not only inefficient, becoming a bottleneck for production line capacity, but more seriously, its inspection standards heavily rely on personal experience, resulting in significant subjectivity and non-repeatability, leading to high rates of misjudgment and missed detections, and it cannot accurately quantify the geometric parameters of the bonding points. The second type is automated inspection systems based on machine vision. These systems acquire bonding images through cameras and use simple image processing techniques or set fixed parameter thresholds to determine the pass / fail status of the bonding points. However, the judgment logic of such methods is relatively rigid, and they can only identify serious defects that are significantly beyond the preset tolerance range. They are powerless against hidden process anomalies where the parameters are within the individual tolerance but the relative relationship between multiple bonding points has shown abnormal consistency fluctuations. For example, when the bonding process begins to drift systematically, it may cause the size uniformity of an entire row of solder balls to deteriorate, but each solder ball may still be within the acceptable range when measured individually. Existing automatic detection systems cannot effectively capture such defects. Secondly, simple image comparison methods are extremely sensitive to factors such as ambient lighting and product position offset, and have poor robustness, which can easily generate a large number of false alarms on actual production lines. Summary of the Invention

[0004] The purpose of this invention is to provide an image recognition-based fully automated bonding detection system and method to solve the problems mentioned in the background art.

[0005] To address the aforementioned technical problems, this invention provides the following technical solution: a fully automated bonding detection method based on image recognition, comprising:

[0006] S100: Acquires images of the bonding region through a multi-angle vision sensor, identifies and locates each bonding point in the image; records the position coordinates, size and arc height of each bonding point; establishes a node network model based on the spatial distribution of the bonding points, where each node corresponds to a bonding point and stores its position information and measurement parameters;

[0007] S200: Select a starting node in the node network model and set its measurement value as the initial benchmark; transmit the benchmark value to adjacent nodes according to the preset transmission path; each receiving node compares its own measurement value with the received benchmark value: if the difference is within the allowable range, update the benchmark with its own measurement value and continue transmission; if the difference exceeds the range, maintain the original benchmark value and continue transmission, and record the deviation status of the node; this process continues in the network until all nodes have completed processing;

[0008] S300: During the reference transfer process, the adjustment range of the reference value at each node is recorded in real time; the cumulative adjustment of the reference value on each transfer path is calculated, and abnormal transfer paths in which the cumulative adjustment exceeds the path tolerance threshold are identified; at the same time, nodes recorded as deviating in the network are counted, and abnormal areas in which the density of deviating nodes exceeds the area density threshold are marked.

[0009] S400: The bonding quality is judged based on the number of abnormal transmission paths and the area of ​​abnormal regions identified. When the overall abnormality of the bonding meets the preset abnormality judgment conditions, the bonding quality of the product is judged to be unqualified. For unqualified products, the geometric center position of the main abnormal area is determined, the laser marking coordinates are calculated and the marking operation is performed.

[0010] Furthermore, S100 includes:

[0011] S101: Using a multi-angle vision sensor fixed above the production line, images of the bonding area of ​​the packaged product under inspection are simultaneously acquired from the top vertical view and the two side tilted view under preset multispectral illumination conditions. The multi-angle vision sensor system includes high-resolution industrial cameras and a 3D scanning device arranged in different orientations. The acquired multi-view images are registered and reconstructed in 3D to obtain 3D point cloud data of the bonding area. Based on the 3D point cloud data, the 3D contour of each bonding point is identified by a point cloud segmentation algorithm, and the 3D coordinates of each bonding point in the product coordinate system are determined. For each identified bonding point, the equivalent diameter is obtained as a size parameter based on its 3D contour, and the arc height is obtained by analyzing the vertical distance between the highest point of the lead point cloud and the pad reference plane.

[0012] S102: Define each bonding point as a detection node. The attribute data of the node includes at least its three-dimensional coordinates, equivalent diameter, and arc height. Based on the product's packaging design structure, obtain the electrical connection relationship and physical layout location information between the bonding points. Map the electrical connection relationship and physical layout location information to spatial topological connections between nodes, wherein a connection relationship is established between adjacent bonding points, and an association relationship is established between bonding points on the same conductive path. Based on the node attribute data and spatial topological connection relationship, construct a node network model of the product. This model is stored in the form of a graph structure, where nodes store attribute data and edges store topological connection relationships.

[0013] Furthermore, the S200 includes:

[0014] S201: In the node network model, the node whose position coordinates are closest to the geometric center of the product is selected as the starting node s, and its measurement value vector Ms is set as the initial reference value Bs, i.e., Bs=Ms, where the measurement value vector includes the equivalent diameter ds and the arc height hs; based on the spatial topology connection relationship in the node network model, a network traversal method is used to generate a transmission path sequence P={p1,p2,...,pn} with the starting node s as the root node, where p1 represents the starting node s and pn represents the last node; according to the transmission path sequence P, the reference value is transmitted from the starting node to the adjacent nodes in sequence;

[0015] S202: For each node pj (j≥2) in the transmission path sequence P, receive the reference value vector Bi from its predecessor node pi; compare the difference between the self-measured value vector Mj of node pj and the received reference value vector Bi, and calculate the relative deviation of each component: size deviation Δdj=|dj-di| / di, arc height deviation Δhj=|hj-hi| / hi; where dj represents the equivalent diameter measurement value of node pj; di represents the equivalent diameter reference value received from the predecessor node pi; hj represents the arc height measurement value of node pj; and hi represents the arc height reference value received from the predecessor node pi; if If the relative deviations of all components are less than or equal to the corresponding allowable range thresholds, i.e., Δdj≤δd and Δhj≤δh, where δd represents the allowable deviation threshold for size and δh represents the allowable deviation threshold for arc height, then the reference value vector is updated with its own measured value vector, i.e., Bj=Mj; otherwise, the received reference value vector remains unchanged, i.e., Bj=Bi, and the deviation status flag Fj=1 is marked in the node attributes; then the updated reference value vector Bj continues to be passed to the subsequent nodes of node pj in the transmission path; this reference transmission process continues in the network until all nodes in the transmission path sequence P have completed processing.

[0016] Furthermore, the S300 includes:

[0017] S301: During the reference transfer process, the reference value adjustment range ΔAj at each node pj is recorded in real time; the adjustment range ΔAj is obtained by calculating the relative changes of each component of the reference value vector Bj of node pj and the received reference value vector Bi, and the specific calculation formula is: size adjustment range ΔA_dj=|dj-di| / di×100%; arc height adjustment range ΔA_hj=|hj-hi| / hi×100%; where ΔA_dj represents the size adjustment range at node pj; ΔA_hj represents the arc height adjustment range at node pj; when node pj does not update the reference value, the adjustment range ΔAj is recorded as 0;

[0018] S302: Based on the transmission path sequence P, calculate the cumulative adjustment amount ΣΔAk of ​​the baseline value on each transmission path Lk; the transmission path Lk is defined as a complete transmission link from the starting node s to any node pj; the cumulative adjustment amount ΣΔAk is obtained by summing the adjustment magnitudes of all nodes on the path: ΣΔAk = Σ pj∈Lk (wd×ΔA_dj+wh×ΔA_hj); where wd and wh represent the weighting coefficients of size and arc height, respectively, and wd+wh=1; the cumulative adjustment amount ΣΔAk of ​​each transmission path is compared with the preset path tolerance threshold θpath. When ΣΔAk>θpath, the transmission path is marked as an abnormal transmission path; where the initial values ​​of the weighting coefficients wd and wh are set to 0.5, indicating that the size and arc height parameters are given equal importance in the initial stage; during operation, the weighting coefficients are periodically redistributed by statistically analyzing the Pearson correlation coefficient between the adjustment range of each parameter and the final quality defect in historical data: higher weights are assigned to parameters that show a stronger correlation with quality defects, thereby ensuring that the weighting coefficients always reflect the actual quality impact of the parameters;

[0019] S303: Based on the spatial distribution of the node network model, the bonding region is divided into several grid regions of equal area {C1, C2, ..., Cm}, where Cm represents the m-th grid region; the number of nodes in the deviated state in each grid region is counted, and the corresponding deviated node density ρm = Nm / Am is calculated, where Nm represents the number of nodes in the deviated state in the m-th grid region, and Am represents the area of ​​the m-th grid region; the deviated node density ρm of each grid region is compared with the preset region density threshold θarea, and when ρm > θarea, the grid region is marked as an abnormal region;

[0020] S304: Summarize the identification results of all abnormal transmission paths and abnormal regions to form a complete anomaly detection report, including the number of abnormal transmission paths, the total area of ​​abnormal regions, and the spatial location distribution information of each abnormal region.

[0021] Furthermore, the S400 includes:

[0022] S401: Based on the anomaly detection report generated in S304, extract the number of anomaly propagation paths Npath and the total area of ​​anomaly regions Sarea; calculate the overall bonding anomaly severity index E, the formula of which is: E=α×(Npath / Nt_path)+β×(Sarea / St_area); where Nt_path represents the total number of propagation paths, St_area represents the total area of ​​the bonding region, α represents the influence weight coefficient of the anomaly propagation path, β represents the influence weight coefficient of the anomaly region, and α+β=1; compare the overall bonding anomaly severity index E with the preset anomaly judgment threshold Et. If E≥E_t, the product bonding quality is judged to be unqualified; otherwise, the product bonding quality is judged to be qualified.

[0023] S402: For products deemed non-conforming, extract the spatial distribution information of all abnormal areas from the anomaly detection report; select the largest abnormal area as the main abnormal area, and calculate the geometric center coordinates O(xo, yo) of the main abnormal area, where xo represents the abscissa of the geometric center and yo represents the ordinate of the geometric center; based on the geometric center coordinates and the preset laser marking offset (Δx, Δy), calculate the laser marking coordinates G(xg, yg), where xg = xo + Δx, yg = yo + Δy; the preset laser marking offset is determined based on three key parameters: the minimum distance between the bonding point and the marking safety area specified in the product packaging design specification, the radius of the heat-affected zone of the laser marking process, and the comprehensive positioning error of the vision positioning system and the mechanical transmission system. Specifically, the values ​​of Δx and Δy are equal to the design distance from the bonding point to the safety area plus the radius of the laser heat-affected zone, and then superimposed with the standard deviation of the comprehensive positioning error of the vision positioning system and the mechanical transmission system;

[0024] S403: Send the laser marking coordinates M(xg,yg) to the laser marking equipment, control the laser marking equipment to perform marking operations at coordinates M(xg,yg) to mark unqualified products; at the same time, record the inspection results and marking information of the product, generate a final inspection report and store it in the database; for products that are judged to be qualified, generate a qualified inspection report and store it in the database; at the same time, control the production line conveyor to transfer qualified products to the next process, and complete the fully automatic bonding inspection process.

[0025] An image recognition-based fully automated bonding detection system includes an image acquisition and modeling module, a reference transfer analysis module, an anomaly analysis module, and a quality judgment module.

[0026] The image acquisition and modeling module acquires images of the bonding region through a multi-angle vision sensor, identifies and locates each bonding point in the image; for each bonding point, it records its position coordinates, size and arc height; based on the spatial distribution of the bonding points, it establishes a node network model, where each node corresponds to a bonding point and stores its position information and measurement parameters.

[0027] The reference transfer analysis module selects a starting node in the node network model and sets its measured value as the initial reference. Following a preset transfer path, the reference value is transferred to adjacent nodes. Each receiving node compares its own measured value with the received reference value: if the difference is within the allowable range, the reference is updated with its own measured value and the transfer continues; if the difference exceeds the range, the original reference value is maintained and the transfer continues, and the deviation status of that node is recorded. This process continues in the network until all nodes have completed processing.

[0028] During the baseline transfer process, the anomaly analysis module records the adjustment range of the baseline value at each node in real time; calculates the cumulative adjustment of the baseline value on each transfer path, identifies abnormal transfer paths where the cumulative adjustment exceeds the path tolerance threshold; and simultaneously counts the nodes recorded as deviating in the network, marking abnormal areas where the density of deviating nodes exceeds the regional density threshold.

[0029] The quality assessment module determines the bonding quality based on the number of identified abnormal transmission paths and the area of ​​abnormal regions. When the overall abnormality of the bonding meets the preset abnormality assessment conditions, the bonding quality of the product is determined to be unqualified. For unqualified products, the geometric center position of the main abnormal area is determined, the laser marking coordinates are calculated, and the marking operation is performed.

[0030] The image acquisition and modeling module includes a 3D acquisition unit and a network construction unit;

[0031] The 3D acquisition unit acquires multi-view images of the bonding region through multi-angle vision sensors, performs image registration and 3D reconstruction to obtain 3D point cloud data, identifies the 3D contour of each bonding point based on point cloud segmentation, and obtains its coordinates, equivalent diameter and arc height.

[0032] The network building unit defines each bonding point as a detection node, establishes spatial topology connections between nodes based on electrical connection relationships and physical layout location information, and constructs a node network model stored in the form of a graph structure.

[0033] The baseline transfer analysis module includes a path initialization unit and a baseline transfer unit;

[0034] The path initialization unit selects the node whose position coordinates are closest to the geometric center in the node network model as the starting node, generates a transmission path sequence with that node as the root node, and initializes the reference value vector.

[0035] The reference transmission unit transmits the reference value to the adjacent nodes in sequence according to the transmission path. By comparing the relative deviation between the node's own measurement value and the received reference value, it dynamically updates the reference value or marks the deviation status until all nodes have completed processing.

[0036] The anomaly analysis module includes an adjustment record unit, a path analysis unit, a region analysis unit, and a report generation unit;

[0037] The adjustment recording unit records the adjustment range of the reference value at each node in real time, including the size adjustment range and the arc height adjustment range;

[0038] The path analysis unit calculates the cumulative adjustment of the baseline value on each transmission path and identifies abnormal transmission paths by comparing it with the path tolerance threshold.

[0039] The region analysis unit divides the bonding region into grid regions, counts the density of off-nodes in each grid region, and marks abnormal regions by comparing it with the region density threshold.

[0040] The report generation unit summarizes the identification results of anomaly propagation paths and anomaly regions, and generates an anomaly detection report that includes the number of anomalies, total area, and location distribution.

[0041] The quality assessment module includes a quality assessment unit, a coordinate calculation unit, and an execution control unit;

[0042] The quality assessment unit calculates the overall abnormality index based on the abnormality detection report, and determines whether the product quality is qualified by comparing it with the abnormality assessment threshold.

[0043] The coordinate calculation unit determines the geometric center of the main abnormal area of ​​the non-conforming product and calculates the laser marking coordinates in combination with the preset laser marking offset.

[0044] The execution control unit controls the laser marking equipment to perform marking operations, generate and store inspection reports, and simultaneously controls the production line conveyor to achieve product diversion.

[0045] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0046] This invention uses multi-angle visual acquisition and 3D reconstruction technology to accurately obtain key parameters such as the position coordinates and size of each bonding point, and constructs a node network model that reflects the actual connection relationship of the bonding points. Compared with traditional manual sampling or single-parameter detection methods, this invention achieves full coverage and high-precision measurement of the bonding area, providing a complete and reliable data foundation for subsequent quality analysis.

[0047] This invention utilizes an innovative benchmark transfer mechanism to sequentially transmit quality standards throughout the network, starting from the central node. This effectively identifies consistency differences between bonding points. This method can not only detect obvious defects in individual bonding points but also uncover hidden problems such as overall inhomogeneity and gradual defects that are difficult to detect using traditional methods. By combining anomaly propagation path analysis and anomaly region density statistics, multi-dimensional quality inspection from local to global perspectives is achieved, significantly improving the accuracy and reliability of defect detection. Attached Figure Description

[0048] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0049] Figure 1 This is a flowchart of a fully automated bonding detection method based on image recognition. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Please see Figure 1 This invention provides a technical solution: a fully automated bonding detection method based on image recognition, comprising:

[0052] S100: Acquires images of the bonding region through a multi-angle vision sensor, identifies and locates each bonding point in the image; records the position coordinates, size and arc height of each bonding point; establishes a node network model based on the spatial distribution of the bonding points, where each node corresponds to a bonding point and stores its position information and measurement parameters;

[0053] S200: Select a starting node in the node network model and set its measurement value as the initial benchmark; transmit the benchmark value to adjacent nodes according to the preset transmission path; each receiving node compares its own measurement value with the received benchmark value: if the difference is within the allowable range, update the benchmark with its own measurement value and continue transmission; if the difference exceeds the range, maintain the original benchmark value and continue transmission, and record the deviation status of the node; this process continues in the network until all nodes have completed processing;

[0054] S300: During the reference transfer process, the adjustment range of the reference value at each node is recorded in real time; the cumulative adjustment of the reference value on each transfer path is calculated, and abnormal transfer paths in which the cumulative adjustment exceeds the path tolerance threshold are identified; at the same time, nodes recorded as deviating in the network are counted, and abnormal areas in which the density of deviating nodes exceeds the area density threshold are marked.

[0055] S400: The bonding quality is judged based on the number of abnormal transmission paths and the area of ​​abnormal regions identified. When the overall abnormality of the bonding meets the preset abnormality judgment conditions, the bonding quality of the product is judged to be unqualified. For unqualified products, the geometric center position of the main abnormal area is determined, the laser marking coordinates are calculated and the marking operation is performed.

[0056] S100 includes:

[0057] S101: Using a multi-angle vision sensor fixed above the production line, images of the bonding area of ​​the packaged product under inspection are simultaneously acquired from the top vertical view and the two side tilted view under preset multispectral illumination conditions. The multi-angle vision sensor system includes high-resolution industrial cameras and a 3D scanning device arranged in different orientations. The acquired multi-view images are registered and reconstructed in 3D to obtain 3D point cloud data of the bonding area. Based on the 3D point cloud data, the 3D contour of each bonding point is identified by a point cloud segmentation algorithm, and the 3D coordinates of each bonding point in the product coordinate system are determined. For each identified bonding point, the equivalent diameter is obtained as a size parameter based on its 3D contour, and the arc height is obtained by analyzing the vertical distance between the highest point of the lead point cloud and the pad reference plane.

[0058] S102: Define each bonding point as a detection node. The attribute data of the node includes at least its three-dimensional coordinates, equivalent diameter, and arc height. Based on the product's packaging design structure, obtain the electrical connection relationship and physical layout location information between the bonding points. Map the electrical connection relationship and physical layout location information to spatial topological connections between nodes, wherein a connection relationship is established between adjacent bonding points, and an association relationship is established between bonding points on the same conductive path. Based on the node attribute data and spatial topological connection relationship, construct a node network model of the product. This model is stored in the form of a graph structure, where nodes store attribute data and edges store topological connection relationships.

[0059] S200 includes:

[0060] S201: In the node network model, the node whose position coordinates are closest to the geometric center of the product is selected as the starting node s, and its measurement value vector Ms is set as the initial reference value Bs, i.e., Bs=Ms, where the measurement value vector includes the equivalent diameter ds and the arc height hs; based on the spatial topology connection relationship in the node network model, a network traversal method is used to generate a transmission path sequence P={p1,p2,...,pn} with the starting node s as the root node, where p1 represents the starting node s and pn represents the last node; according to the transmission path sequence P, the reference value is transmitted from the starting node to the adjacent nodes in sequence;

[0061] S202: For each node pj (j≥2) in the transmission path sequence P, receive the reference value vector Bi from its predecessor node pi; compare the difference between the self-measured value vector Mj of node pj and the received reference value vector Bi, and calculate the relative deviation of each component: size deviation Δdj=|dj-di| / di, arc height deviation Δhj=|hj-hi| / hi; where dj represents the equivalent diameter measurement value of node pj; di represents the equivalent diameter reference value received from the predecessor node pi; hj represents the arc height measurement value of node pj; and hi represents the arc height reference value received from the predecessor node pi; if If the relative deviations of all components are less than or equal to the corresponding allowable range thresholds, i.e., Δdj≤δd and Δhj≤δh, where δd represents the allowable deviation threshold for size and δh represents the allowable deviation threshold for arc height, then the reference value vector is updated with its own measured value vector, i.e., Bj=Mj; otherwise, the received reference value vector remains unchanged, i.e., Bj=Bi, and the deviation status flag Fj=1 is marked in the node attributes; then the updated reference value vector Bj continues to be passed to the subsequent nodes of node pj in the transmission path; this reference transmission process continues in the network until all nodes in the transmission path sequence P have completed processing.

[0062] The S300 includes:

[0063] S301: During the reference transfer process, the reference value adjustment range ΔAj at each node pj is recorded in real time; the adjustment range ΔAj is obtained by calculating the relative changes of each component of the reference value vector Bj of node pj and the received reference value vector Bi, and the specific calculation formula is: size adjustment range ΔA_dj=|dj-di| / di×100%; arc height adjustment range ΔA_hj=|hj-hi| / hi×100%; where ΔA_dj represents the size adjustment range at node pj; ΔA_hj represents the arc height adjustment range at node pj; when node pj does not update the reference value, the adjustment range ΔAj is recorded as 0;

[0064] S302: Based on the transmission path sequence P, calculate the cumulative adjustment amount ΣΔAk of ​​the baseline value on each transmission path Lk; the transmission path Lk is defined as a complete transmission link from the starting node s to any node pj; the cumulative adjustment amount ΣΔAk is obtained by summing the adjustment magnitudes of all nodes on the path: ΣΔAk = Σ pj∈Lk (wd×ΔA_dj+wh×ΔA_hj); where wd and wh represent the weighting coefficients for size and arc height, respectively, and wd+wh=1; the cumulative adjustment ΣΔAk of ​​each transmission path is compared with the preset path tolerance threshold θpath. When ΣΔAk>θpath, the transmission path is marked as an abnormal transmission path; the initial values ​​of the weighting coefficients wd and wh are set to 0.5, indicating that the size and arc height parameters are given equal importance in the initial stage; during operation, the weighting coefficients are periodically redistributed by statistically analyzing the Pearson correlation coefficient between the adjustment range of each parameter and the final quality defect in historical data: higher weights are assigned to the parameters related to quality defects. The parameters exhibiting stronger correlation are used to ensure that the weighting coefficients always reflect the actual quality impact of the parameters. The specific process is as follows: Regularly analyze the binarized results of the size adjustment range and arc height adjustment range of each bonding point in the recent product inspection records and whether the product is ultimately judged as unqualified. Qualified is 0, and unqualified is 1; calculate the Pearson correlation coefficients rd and rh of the size adjustment range, arc height adjustment range, and quality judgment results respectively; redistribute the weighting coefficients according to the following rules: wd_new=|rd| / (|rd|+|rh|), wh_new=|rh| / (|rd|+|rh|), and always satisfy wd_new+wh_new=1;

[0065] S303: Based on the spatial distribution of the node network model, the bonding region is divided into several grid regions of equal area {C1, C2, ..., Cm}, where Cm represents the m-th grid region; the number of nodes in the deviated state in each grid region is counted, and the corresponding deviated node density ρm = Nm / Am is calculated, where Nm represents the number of nodes in the deviated state in the m-th grid region, and Am represents the area of ​​the m-th grid region; the deviated node density ρm of each grid region is compared with the preset region density threshold θarea, and when ρm > θarea, the grid region is marked as an abnormal region;

[0066] S304: Summarize the identification results of all abnormal transmission paths and abnormal regions to form a complete anomaly detection report, including the number of abnormal transmission paths, the total area of ​​abnormal regions, and the spatial location distribution information of each abnormal region.

[0067] The S400 includes:

[0068] S401: Based on the anomaly detection report generated in S304, extract the number of anomaly propagation paths (Npath) and the total area of ​​anomaly regions (Sarea); calculate the overall bond anomaly severity index E, with the formula: E = α × (Npath / Nt_path) + β × (Sarea / St_area); where Nt_path represents the total number of propagation paths, St_area represents the total area of ​​bonded regions, α represents the influence weight coefficient of the anomaly propagation path, β represents the influence weight coefficient of the anomaly region, and α + β = 1; compare the overall bond anomaly severity index E with the preset anomaly judgment threshold Et. If E ≥ E_t, the product bond quality is determined to be unqualified; otherwise, the product bond quality is determined to be qualified. Initially set α = 0.5. β=0.5; Subsequently, historical test data of all products within a statistical window are collected periodically. Each data point includes the number of abnormal transmission paths Npath, the total area of ​​abnormal regions Sarea, and the final quality verification result of the product in subsequent testing stages. Qualified results are recorded as 0, and unqualified results due to bonding problems are recorded as 1. Based on this dataset, the Pearson correlation coefficient r_path between the abnormal transmission path number sequence and the quality result sequence, and the Pearson correlation coefficient r_area between the abnormal region area sequence and the quality result sequence are calculated respectively. Then, the weights are dynamically reallocated: α=|r_path| / (|r_path|+|r_area|), β=|r_area| / (|r_path|+|r_area|).

[0069] S402: For products deemed non-conforming, extract the spatial distribution information of all abnormal areas from the anomaly detection report; select the largest abnormal area as the main abnormal area, and calculate the geometric center coordinates O(xo, yo) of the main abnormal area, where xo represents the abscissa of the geometric center and yo represents the ordinate of the geometric center; based on the geometric center coordinates and the preset laser marking offset (Δx, Δy), calculate the laser marking coordinates G(xg, yg), where xg = xo + Δx, yg = yo + Δy; the preset laser marking offset is determined based on three key parameters: the minimum distance between the bonding point and the marking safety area specified in the product packaging design specification, the radius of the heat-affected zone of the laser marking process, and the comprehensive positioning error of the vision positioning system and the mechanical transmission system. Specifically, the values ​​of Δx and Δy are equal to the design distance from the bonding point to the safety area plus the radius of the laser heat-affected zone, and then superimposed with the standard deviation of the comprehensive positioning error of the vision positioning system and the mechanical transmission system;

[0070] S403: Send the laser marking coordinates M(xg,yg) to the laser marking equipment, control the laser marking equipment to perform marking operations at coordinates M(xg,yg) to mark unqualified products; at the same time, record the inspection results and marking information of the product, generate a final inspection report and store it in the database; for products that are judged to be qualified, generate a qualified inspection report and store it in the database; at the same time, control the production line conveyor to transfer qualified products to the next process, and complete the fully automatic bonding inspection process.

[0071] This invention uses a QFN32L-B packaged ASIC chip as the testing object. The chip's bonding area is 4×4mm, containing 32 matrix-layout bonding points (8 columns × 4 rows), divided into 4 independent conductive paths (each group has 8 bonding points connected in series to correspond to the chip's 4 core signals). The bonding points use gold wire bonding technology (25μm gold wire diameter), and must meet the requirements of an equivalent diameter of 30-50μm and an arc height of 15-25μm. The hardware includes a multi-angle vision sensor system (one 12-megapixel industrial camera on top, one 8-megapixel industrial camera on each side, and a 3D scanning device), a multispectral illumination system (a white ring light source on top and near-infrared strip light sources on both sides), and auxiliary positioning and marking equipment. The preset allowable deviation thresholds for size and arc height are 10% and 15%, respectively, and the weighting coefficients for size and arc height are dynamically adjusted to 0.6 and 0.4, respectively. The radius tolerance threshold is 30%, the grid area is 0.25 mm², the area density threshold is 40%, the anomaly judgment weights are α=0.5, β=0.5 and the threshold is 0.3, and the laser marking offset (Δx, Δy) is preset to (0.2 mm, 0.2 mm). This offset is determined comprehensively based on the product packaging design specifications, laser process characteristics and system positioning accuracy. Specifically, according to the product packaging design specifications, the minimum design spacing between the bonding point and the surrounding marking safety zone is 0.1 mm; the radius of the heat-affected zone of the laser marking process is 0.05 mm; after calibration, the comprehensive positioning error of the system's visual positioning and mechanical transmission is 0.05 mm, so the values ​​of Δx and Δy are: 0.1 mm (design spacing) + 0.05 mm (heat-affected zone radius) + 0.05 mm (positioning error) = 0.2 mm, hence the laser marking offset (0.2 mm, 0.2 mm).

[0072] First, the production line positioning device is activated to fix the chip under test using vacuum adsorption (positioning deviation ≤ 0.01mm). This triggers a multi-angle vision sensor to simultaneously acquire images under multispectral illumination. The top camera acquires vertical images with a 100μs exposure time and 1.0 gain to capture the top surface contour of the bonding points. The two side cameras acquire images from a 45° tilted angle with an 80μs exposure time and 1.2 gain to record the side curvature of the lead wires. Each acquisition takes 200ms, generating three images with matching resolution. Next, the ORB feature point matching algorithm is used for image registration. ≥50 bonding point Mark feature points are extracted from the top image as a reference, correcting for translation and rotation deviations in the two side images. The registration accuracy is ±0.005mm. Finally, the MVS algorithm is used, combining sensor intrinsic parameters (focal length 8mm, principal point coordinates 960×600) and extrinsic parameters (camera spacing 5). A 3D point cloud model with a density of 1000 points / mm² and an accuracy of ±0.5μm was generated (0mm, vertical height 100mm). Then, a region growing algorithm was used to segment the point cloud, selecting the centroid of the bright regions with grayscale values ​​of 200-255 as seed points (32 in total). Following the rule of normal vector angle ≤5° and point-to-point distance ≤0.5μm, 32 bonding point-lead combination point clouds were separated. Finally, a product coordinate system was established with the positioning hole at the lower left corner of the chip as the origin. The 3D coordinates were determined by the centroid of the point cloud (e.g., the corrected coordinates of the core bonding point P5 are (1.0, 1.0, 0.75)mm, with an accuracy of ±0.001mm). The RANSAC algorithm was used to fit the top contour of the bonding point to obtain the equivalent diameter (P5 is 42μm). After fitting the pad reference plane, the vertical distance between the highest point of the lead and the plane was calculated to obtain the arc height (P5 is 20μm).

[0073] After parameter extraction, the 32 bonding points are defined as detection nodes (attributes include 3D coordinates, equivalent diameter, and arc height; for example, the P5 node attributes are (1.0, 1.0, 0.75) mm, 42 μm, and 20 μm). The electrical connections (4 sets of conductive paths) and physical layout (16 nodes in the core area and 16 nodes in the edge area) of the bonding points are retrieved from the packaging design file and mapped to the node topology connections (spatial connections between adjacent nodes and connections between nodes on the same conductive path). A graph-structured node network model containing 32 nodes and 64 edges is constructed and stored.

[0074] In the node network, the node closest to the chip's geometric center (2.0, 2.0, 0.75) mm is selected as the starting node S. The measurement value vector (40μm, 22μm) is set as the initial reference Bs. Eight transmission path sequences are generated by traversing (each path covers 4 nodes, P1: S→P2→P3→P4). Then, the reference is transmitted according to the path. Taking P1 as an example, node P2 receives the reference (40μm, 22μm), and its own measurement value (43μm, 23μm) is calculated to have a size deviation of 7.5% and an arc height deviation of 4.5% (both less than the threshold). The reference is updated to (43μm, 23μm) and transmitted to P3. After receiving the reference, P3's own measurement value (48μm, 21μm) has a size deviation of 11.6% (exceeding the threshold). The original reference is maintained and the deviation status is marked. Finally, 6 edge region nodes out of 32 nodes are marked as deviation.

[0075] The adjustment range of each node was recorded in real time. P2, due to updating the baseline, had a weighted overall adjustment range of 6.3%, while nodes without updated baselines were recorded as 0 (average 5.2% in the core area and 8.1% in the edge area). The cumulative adjustment of the eight paths was then calculated. Path P5 was marked as abnormal because the adjustment ranges of three consecutive nodes were too large (6.8%, 7.5%, and 8.2%), with a cumulative value of 35.7% exceeding the threshold. Ultimately, two abnormal paths were identified. Simultaneously, the bonding region was divided into 64 0.25mm segments. 2 The mesh was statistically analyzed, and it was found that mesh C23 (X1.5-1.75mm, Y2.0-2.25mm) deviated from the node density by 50%, exceeding the threshold. It was marked as an abnormal region and four adjacent abnormal meshes (total area 1mm) were merged. 2 This process generates a comprehensive detection report that includes the number of abnormal paths, the area of ​​abnormal regions, and their locations.

[0076] The overall anomaly index E = 0.5 × (2 / 8) + 0.5 × (1 / 16) = 0.359 is calculated. If the value is greater than or equal to the threshold of 0.3, the product is deemed unqualified. The merged abnormal area is selected as the main abnormal area, and its geometric center (1.75, 2.25) mm is calculated. Combined with the offset, the marking coordinates (1.95, 2.45) mm are obtained and sent to the laser marking equipment to mark a 0.3 mm circular mark. At the same time, the parameters of 32 bonding points and the abnormal information are recorded to generate a final report for storage. The production line is then controlled to transfer the unqualified products to a dedicated workstation.

[0077] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. An image recognition-based full-automatic detection method for bonding, characterized in that: The method includes: S100: Acquires images of the bonding region through a multi-angle vision sensor, identifies and locates each bonding point in the image; records the position coordinates, size and arc height of each bonding point; establishes a node network model based on the spatial distribution of the bonding points, where each node corresponds to a bonding point and stores its position information and measurement parameters; S200: Select a starting node in the node network model and set its measurement value as the initial benchmark; transmit the benchmark value to adjacent nodes according to the preset transmission path; each receiving node compares its own measurement value with the received benchmark value: if the difference is within the allowable range, update the benchmark with its own measurement value and continue transmission; if the difference exceeds the range, maintain the original benchmark value and continue transmission, and record the deviation status of the node; this process continues in the network until all nodes have completed processing; S300: During the reference transfer process, the adjustment range of the reference value at each node is recorded in real time; the cumulative adjustment of the reference value on each transfer path is calculated, and abnormal transfer paths in which the cumulative adjustment exceeds the path tolerance threshold are identified; at the same time, nodes recorded as deviating in the network are counted, and abnormal areas in which the density of deviating nodes exceeds the area density threshold are marked. S400: The bonding quality is judged based on the number of abnormal transmission paths and the area of ​​abnormal regions identified. When the overall abnormality of the bonding meets the preset abnormality judgment conditions, the bonding quality of the product is judged to be unqualified. For unqualified products, the geometric center position of the main abnormal area is determined, the laser marking coordinates are calculated and the marking operation is performed.

2. The fully-automatic bonding detection method based on image recognition according to claim 1, characterized in that: S100 includes: S101: Using a multi-angle vision sensor fixed above the production line, images of the bonding area of ​​the packaged product under inspection are simultaneously acquired from the top vertical view and the two side tilted view under preset multispectral illumination conditions. The multi-angle vision sensor system includes high-resolution industrial cameras and a 3D scanning device arranged in different orientations. The acquired multi-view images are registered and reconstructed in 3D to obtain 3D point cloud data of the bonding area. Based on the 3D point cloud data, the 3D contour of each bonding point is identified by a point cloud segmentation algorithm, and the 3D coordinates of each bonding point in the product coordinate system are determined. For each identified bonding point, the equivalent diameter is obtained as a size parameter based on its 3D contour, and the arc height is obtained by analyzing the vertical distance between the highest point of the lead point cloud and the pad reference plane. S102: Define each bonding point as a detection node. The attribute data of the node includes at least its three-dimensional coordinates, equivalent diameter, and arc height. Based on the product's packaging design structure, obtain the electrical connection relationship and physical layout location information between the bonding points. Map the electrical connection relationship and physical layout location information to spatial topological connections between nodes, wherein a connection relationship is established between adjacent bonding points, and an association relationship is established between bonding points on the same conductive path. Based on the node attribute data and spatial topological connection relationship, construct a node network model of the product. This model is stored in the form of a graph structure, where nodes store attribute data and edges store topological connection relationships.

3. The fully automatic bonding detection method based on image recognition according to claim 2, characterized in that: S200 includes: S201: In the node network model, the node whose position coordinates are closest to the geometric center of the product is selected as the starting node s, and its measurement value vector Ms is set as the initial reference value Bs, i.e., Bs=Ms, where the measurement value vector includes the equivalent diameter ds and the arc height hs; based on the spatial topology connection relationship in the node network model, a network traversal method is used to generate a transmission path sequence P={p1,p2,...,pn} with the starting node s as the root node, where p1 represents the starting node s and pn represents the last node; according to the transmission path sequence P, the reference value is transmitted from the starting node to the adjacent nodes in sequence; S202: For each node pj (j≥2) in the transmission path sequence P, receive the reference value vector Bi from its predecessor node pi; compare the difference between the self-measured value vector Mj of node pj and the received reference value vector Bi, and calculate the relative deviation of each component: size deviation Δdj=|dj-di| / di, arc height deviation Δhj=|hj-hi| / hi; where dj represents the equivalent diameter measurement value of node pj; di represents the equivalent diameter reference value received from the predecessor node pi; hj represents the arc height measurement value of node pj; and hi represents the arc height reference value received from the predecessor node pi; if If the relative deviations of all components are less than or equal to the corresponding allowable range thresholds, i.e., Δdj≤δd and Δhj≤δh, where δd represents the allowable deviation threshold for size and δh represents the allowable deviation threshold for arc height, then the reference value vector is updated with its own measured value vector, i.e., Bj=Mj; otherwise, the received reference value vector remains unchanged, i.e., Bj=Bi, and the deviation status flag Fj=1 is marked in the node attributes; then the updated reference value vector Bj continues to be passed to the subsequent nodes of node pj in the transmission path; this reference transmission process continues in the network until all nodes in the transmission path sequence P have completed processing.

4. The fully-automatic bonding detection method based on image recognition according to claim 3, characterized in that: The S300 includes: S301: During the reference transfer process, the reference value adjustment range ΔAj at each node pj is recorded in real time; the adjustment range ΔAj is obtained by calculating the relative changes of each component of the reference value vector Bj of node pj and the received reference value vector Bi, and the specific calculation formula is: size adjustment range ΔA_dj=|dj-di| / di×100%; arc height adjustment range ΔA_hj=|hj-hi| / hi×100%; where ΔA_dj represents the size adjustment range at node pj; ΔA_hj represents the arc height adjustment range at node pj; when node pj does not update the reference value, the adjustment range ΔAj is recorded as 0; S302: Based on the transmission path sequence P, calculate the cumulative adjustment amount ΣΔAk of ​​the baseline value on each transmission path Lk; the transmission path Lk is defined as a complete transmission link from the starting node s to any node pj; the cumulative adjustment amount ΣΔAk is obtained by summing the adjustment magnitudes of all nodes on the path: ΣΔAk = Σ pj∈Lk (wd×ΔA_dj+wh×ΔA_hj); where wd and wh represent the weight coefficients of size and arc height, respectively, and wd+wh=1; the cumulative adjustment amount ΣΔAk of ​​each transmission path is compared with the preset path tolerance threshold θpath. When ΣΔAk>θpath, the transmission path is marked as an abnormal transmission path. S303: Based on the spatial distribution of the node network model, the bonding region is divided into several grid regions of equal area {C1, C2, ..., Cm}, where Cm represents the m-th grid region; the number of nodes in the deviated state in each grid region is counted, and the corresponding deviated node density ρm = Nm / Am is calculated, where Nm represents the number of nodes in the deviated state in the m-th grid region, and Am represents the area of ​​the m-th grid region; the deviated node density ρm of each grid region is compared with the preset region density threshold θarea, and when ρm > θarea, the grid region is marked as an abnormal region; S304: Summarize the identification results of all abnormal transmission paths and abnormal regions to form a complete anomaly detection report, including the number of abnormal transmission paths, the total area of ​​abnormal regions, and the spatial location distribution information of each abnormal region.

5. The fully automated bonding detection method based on image recognition according to claim 4, characterized in that: The S400 includes: S401: Based on the anomaly detection report generated in S304, extract the number of anomaly propagation paths Npath and the total area of ​​anomaly regions Sarea; calculate the overall bonding anomaly severity index E, the formula of which is: E=α×(Npath / Nt_path)+β×(Sarea / St_area); where Nt_path represents the total number of propagation paths, St_area represents the total area of ​​the bonding region, α represents the influence weight coefficient of the anomaly propagation path, β represents the influence weight coefficient of the anomaly region, and α+β=1; compare the overall bonding anomaly severity index E with the preset anomaly judgment threshold Et. If E≥E_t, the product bonding quality is judged to be unqualified; otherwise, the product bonding quality is judged to be qualified. S402: For products determined to be non-conforming, extract the spatial distribution information of all abnormal areas from the abnormality inspection report; select the abnormal area with the largest area as the main abnormal area, and calculate the geometric center coordinates O(xo, yo) of the main abnormal area, where xo represents the abscissa of the geometric center and yo represents the ordinate of the geometric center; based on the geometric center coordinates and the preset laser marking offset (Δx, Δy), calculate the laser marking coordinates G(xg, yg), where xg = xo + Δx and yg = yo + Δy. S403: Send the laser marking coordinates M(xg,yg) to the laser marking equipment, control the laser marking equipment to perform marking operations at coordinates M(xg,yg) to mark unqualified products; at the same time, record the inspection results and marking information of the product, generate a final inspection report and store it in the database; for products that are judged to be qualified, generate a qualified inspection report and store it in the database; at the same time, control the production line conveyor to transfer qualified products to the next process, and complete the fully automatic bonding inspection process.

6. An image recognition-based full-automatic detection system for bonding, characterized in that: The system includes an image acquisition and modeling module, a reference transfer analysis module, an anomaly analysis module, and a quality judgment module; The image acquisition and modeling module acquires images of the bonding region through a multi-angle vision sensor, identifies and locates each bonding point in the image; for each bonding point, it records its position coordinates, size and arc height; based on the spatial distribution of the bonding points, it establishes a node network model, where each node corresponds to a bonding point and stores its position information and measurement parameters. The reference transmission analysis module selects a starting node in the node network model and sets its measured value as the initial reference. According to the preset transmission path, the reference value is transmitted to the adjacent nodes. Each receiving node compares its own measured value with the received reference value: if the difference is within the allowable range, the reference is updated with its own measured value and the transmission continues; if the difference exceeds the range, the original reference value is maintained and the transmission continues, and the deviation status of the node is recorded. This process continues throughout the network until all nodes have completed their processing; During the benchmark transfer process, the anomaly analysis module records the adjustment range of the benchmark value at each node in real time. Calculate the cumulative adjustment of the baseline value on each transmission path, and identify abnormal transmission paths where the cumulative adjustment exceeds the path tolerance threshold; at the same time, count the nodes recorded as deviating in the network, and mark the abnormal regions where the density of deviating nodes exceeds the regional density threshold. The quality judgment module determines the bonding quality based on the number of identified abnormal transmission paths and the area of ​​abnormal regions. When the overall abnormality of the bonding meets the preset abnormality judgment conditions, the bonding quality of the product is determined to be unqualified. For unqualified products, the geometric center position of the main abnormal area is determined, the laser marking coordinates are calculated, and the marking operation is performed.

7. The fully automatic bonding detection system based on image recognition according to claim 6, characterized in that: The image acquisition and modeling module includes a 3D acquisition unit and a network construction unit; The three-dimensional acquisition unit acquires multi-view images of the bonding region through a multi-angle vision sensor, performs image registration and three-dimensional reconstruction to obtain three-dimensional point cloud data, identifies the three-dimensional contour of each bonding point based on point cloud segmentation and obtains its coordinates, equivalent diameter and arc height value. The network construction unit defines each bonding point as a detection node, establishes spatial topology connections between nodes based on electrical connection relationships and physical layout location information, and constructs a node network model stored in the form of a graph structure.

8. The fully automatic bonding detection system based on image recognition according to claim 6, characterized in that: The reference transfer analysis module includes a path initialization unit and a reference transfer unit; The path initialization unit selects the node whose position coordinates are closest to the geometric center in the node network model as the starting node, generates a transmission path sequence with the node as the root node, and initializes the reference value vector. The reference transmission unit transmits the reference value to the adjacent nodes in sequence according to the transmission path. By comparing the relative deviation between the node's own measurement value and the received reference value, the reference value is dynamically updated and the deviation status is marked until all nodes have completed processing.

9. The fully automatic bonding detection system based on image recognition according to claim 6, characterized in that: The anomaly analysis module includes an adjustment recording unit, a path analysis unit, a region analysis unit, and a report generation unit; The adjustment recording unit records the adjustment range of the reference value at each node in real time, including the size adjustment range and the arc height adjustment range; The path analysis unit calculates the cumulative adjustment of the baseline value on each transmission path and identifies abnormal transmission paths by comparing it with the path tolerance threshold. The region analysis unit divides the bonding region into grid regions, calculates the deviation node density of each grid region, and marks abnormal regions by comparing it with the region density threshold. The report generation unit summarizes the identification results of abnormal transmission paths and abnormal regions, and generates an abnormal detection report containing the number of abnormalities, total area, and location distribution.

10. The fully automatic bonding inspection system based on image recognition of claim 6, wherein: The quality determination module includes a quality determination unit, a coordinate calculation unit, and an execution control unit; The quality judgment unit calculates the overall abnormality index based on the abnormality detection report, and determines whether the product quality is qualified by comparing it with the abnormality judgment threshold. The coordinate calculation unit determines the geometric center position of the main abnormal area of ​​the defective product and calculates the laser marking coordinates in combination with the preset laser marking offset. The execution control unit controls the laser marking equipment to perform marking operations, generate and store inspection reports, and simultaneously controls the production line conveyor to achieve product diversion.

Images