Bonding full-automatic detection system and method based on image recognition

Abstract

The invention discloses a bonding full-automatic detection system and method based on image recognition, and belongs to the technical field of semiconductor packaging quality detection.The bonding area image is collected through a multi-angle visual sensor, bonding points are recognized and positioned, coordinates, sizes and arc heights of the bonding points are recorded, and a node network model is constructed according to a spatial distribution structure; selecting an initial node in the model to set a reference, transmitting reference values in sequence along a preset path, judging whether the reference is updated or not by each node by comparing a measured value with a received value, and recording a deviation state at the same time; in the process, the reference adjustment amplitude is recorded in real time, the accumulated adjustment amount of each path is calculated, abnormal transmission paths are identified, and deviation node density is counted to mark abnormal areas; and finally, according to the number of the abnormal paths and the area of the abnormal region, carrying out bonding quality judgment, automatically calculating the marking position of an unqualified product, and executing laser marking.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of semiconductor packaging quality detection, and particularly relates to a bonding full-automatic detection system and method based on image recognition. BACKGROUND

[0002] In the semiconductor packaging production process, bonding is one of the core processes, and the quality directly determines the electrical performance and reliability of the device. Bonding quality mainly depends on key parameters such as bonding ball size, arc height, crater shape, and IMC layer integrity. If there are defects such as abnormal bonding ball size, lead arc deformation, deep crater, or missing IMC layer, it will cause device open circuit, short circuit, or shortened service life.

[0003] At present, the detection methods commonly used in the industry mainly include two types: one is traditional manual visual inspection, that is, the operator observes and judges the bonding points one by one through a microscope. This method is not only inefficient and becomes a bottleneck of production capacity of the production line, but more seriously, the detection standard is heavily dependent on personal experience, and there is great subjectivity and non-repeatability, resulting in high misjudgment and missed detection rates, and the geometric parameters of the bonding points cannot be accurately quantified. The second is an automatic detection system based on machine vision, which captures bonding images through a camera and uses simple image processing techniques or sets fixed parameter thresholds to judge the quality of the bonding points. However, the determination logic of this method is relatively rigid, and it can only identify serious defects that are obviously outside the preset tolerance range, but it is powerless for hidden process abnormalities where the relative relationship between multiple bonding points has abnormal consistency fluctuations although the parameters are within the individual tolerance. For example, when the bonding process starts to drift systematically, it may cause the uniformity of the size of a whole row of solder balls to deteriorate, but each solder ball may still be within the acceptable range when measured individually. The existing automatic detection system cannot effectively capture such defects. Secondly, the simple image comparison method is very sensitive to environmental lighting, product position offset and other factors, and has poor robustness, which is prone to produce a large number of false positives in actual production lines. SUMMARY

[0004] The present application aims to provide a bonding full-automatic detection system and method based on image recognition to solve the problems raised in the background.

[0005] In order to solve the above technical problems, the present application provides the following technical scheme: a bonding full-automatic detection method based on image recognition, comprising: S100: acquiring bonding region images through a multi-angle vision sensor, identifying and positioning each bonding point in the image; for each bonding point, recording its position coordinates, size and arc height; according to the spatial distribution of the bonding points, establishing a node network model, wherein each node corresponds to a bonding point, and storing its position information and measurement parameters; S200: Select a starting node in the node network model and set its measurement value as the initial benchmark; transmit the benchmark value to adjacent nodes according to the preset transmission path; each receiving node compares its own measurement value with the received benchmark value: if the difference is within the allowable range, update the benchmark with its own measurement value and continue transmission; if the difference exceeds the range, maintain the original benchmark value and continue transmission, and record the deviation status of the node; this process continues in the network until all nodes have completed processing; S300: During the reference transfer process, the adjustment range of the reference value at each node is recorded in real time; the cumulative adjustment of the reference value on each transfer path is calculated, and abnormal transfer paths in which the cumulative adjustment exceeds the path tolerance threshold are identified; at the same time, nodes recorded as deviating in the network are counted, and abnormal areas in which the density of deviating nodes exceeds the area density threshold are marked. S400: The bonding quality is judged based on the number of abnormal transmission paths and the area of ​​abnormal regions identified. When the overall abnormality of the bonding meets the preset abnormality judgment conditions, the bonding quality of the product is judged to be unqualified. For unqualified products, the geometric center position of the main abnormal area is determined, the laser marking coordinates are calculated and the marking operation is performed.

[0006] Furthermore, S100 includes: S101: Using a multi-angle vision sensor fixed above the production line, images of the bonding area of ​​the packaged product under inspection are simultaneously acquired from the top vertical view and the two side tilted view under preset multispectral illumination conditions. The multi-angle vision sensor system includes high-resolution industrial cameras and a 3D scanning device arranged in different orientations. The acquired multi-view images are registered and reconstructed in 3D to obtain 3D point cloud data of the bonding area. Based on the 3D point cloud data, the 3D contour of each bonding point is identified by a point cloud segmentation algorithm, and the 3D coordinates of each bonding point in the product coordinate system are determined. For each identified bonding point, the equivalent diameter is obtained as a size parameter based on its 3D contour, and the arc height is obtained by analyzing the vertical distance between the highest point of the lead point cloud and the pad reference plane. S102: Define each bonding point as a detection node. The attribute data of the node includes at least its three-dimensional coordinates, equivalent diameter, and arc height. Based on the product's packaging design structure, obtain the electrical connection relationship and physical layout location information between the bonding points. Map the electrical connection relationship and physical layout location information to spatial topological connections between nodes, wherein a connection relationship is established between adjacent bonding points, and an association relationship is established between bonding points on the same conductive path. Based on the node attribute data and spatial topological connection relationship, construct a node network model of the product. This model is stored in the form of a graph structure, where nodes store attribute data and edges store topological connection relationships.

[0007] Furthermore, the S200 includes: S201: In the node network model, the node whose position coordinates are closest to the geometric center of the product is selected as the starting node s, and its measurement value vector Ms is set as the initial reference value Bs, i.e., Bs=Ms, where the measurement value vector includes the equivalent diameter ds and the arc height hs; based on the spatial topology connection relationship in the node network model, a network traversal method is used to generate a transmission path sequence P={p1,p2,...,pn} with the starting node s as the root node, where p1 represents the starting node s and pn represents the last node; according to the transmission path sequence P, the reference value is transmitted from the starting node to the adjacent nodes in sequence; S202: For each node pj (j≥2) in the transmission path sequence P, receive the reference value vector Bi from its predecessor node pi; compare the difference between the self-measured value vector Mj of node pj and the received reference value vector Bi, and calculate the relative deviation of each component: size deviation Δdj=|dj-di| / di, arc height deviation Δhj=|hj-hi| / hi; where dj represents the equivalent diameter measurement value of node pj; di represents the equivalent diameter reference value received from the predecessor node pi; hj represents the arc height measurement value of node pj; and hi represents the arc height reference value received from the predecessor node pi; if If the relative deviations of all components are less than or equal to the corresponding allowable range thresholds, i.e., Δdj≤δd and Δhj≤δh, where δd represents the allowable deviation threshold for size and δh represents the allowable deviation threshold for arc height, then the reference value vector is updated with its own measured value vector, i.e., Bj=Mj; otherwise, the received reference value vector remains unchanged, i.e., Bj=Bi, and the deviation status flag Fj=1 is marked in the node attributes; then the updated reference value vector Bj continues to be passed to the subsequent nodes of node pj in the transmission path; this reference transmission process continues in the network until all nodes in the transmission path sequence P have completed processing.

[0008] Furthermore, the S300 includes: S301: During the reference transfer process, the reference value adjustment range ΔAj at each node pj is recorded in real time; the adjustment range ΔAj is obtained by calculating the relative changes of each component of the reference value vector Bj of node pj and the received reference value vector Bi, and the specific calculation formula is: size adjustment range ΔA_dj=|dj-di| / di×100%; arc height adjustment range ΔA_hj=|hj-hi| / hi×100%; where ΔA_dj represents the size adjustment range at node pj; ΔA_hj represents the arc height adjustment range at node pj; when node pj does not update the reference value, the adjustment range ΔAj is recorded as 0; S302: Based on the transmission path sequence P, calculate the cumulative adjustment amount ΣΔAk of ​​the baseline value on each transmission path Lk; the transmission path Lk is defined as a complete transmission link from the starting node s to any node pj; the cumulative adjustment amount ΣΔAk is obtained by summing the adjustment magnitudes of all nodes on the path: ΣΔAk = Σ pj∈Lk (wd×ΔA_dj+wh×ΔA_hj); where wd and wh represent the weighting coefficients of size and arc height, respectively, and wd+wh=1; the cumulative adjustment amount ΣΔAk of ​​each transmission path is compared with the preset path tolerance threshold θpath. When ΣΔAk>θpath, the transmission path is marked as an abnormal transmission path; where the initial values ​​of the weighting coefficients wd and wh are set to 0.5, indicating that the size and arc height parameters are given equal importance in the initial stage; during operation, the weighting coefficients are periodically redistributed by statistically analyzing the Pearson correlation coefficient between the adjustment range of each parameter and the final quality defect in historical data: higher weights are assigned to parameters that show a stronger correlation with quality defects, thereby ensuring that the weighting coefficients always reflect the actual quality impact of the parameters; S303: Based on the spatial distribution of the node network model, the bonding region is divided into several grid regions of equal area {C1, C2, ..., Cm}, where Cm represents the m-th grid region; the number of nodes in the deviated state in each grid region is counted, and the corresponding deviated node density ρm = Nm / Am is calculated, where Nm represents the number of nodes in the deviated state in the m-th grid region, and Am represents the area of ​​the m-th grid region; the deviated node density ρm of each grid region is compared with the preset region density threshold θarea, and when ρm > θarea, the grid region is marked as an abnormal region; S304: Summarize the identification results of all abnormal transmission paths and abnormal regions to form a complete anomaly detection report, including the number of abnormal transmission paths, the total area of ​​abnormal regions, and the spatial location distribution information of each abnormal region.

[0009] Furthermore, the S400 includes: S401: Based on the anomaly detection report generated in S304, extract the number of anomaly propagation paths Npath and the total area of ​​anomaly regions Sarea; calculate the overall bonding anomaly severity index E, the formula of which is: E=α×(Npath / Nt_path)+β×(Sarea / St_area); where Nt_path represents the total number of propagation paths, St_area represents the total area of ​​the bonding region, α represents the influence weight coefficient of the anomaly propagation path, β represents the influence weight coefficient of the anomaly region, and α+β=1; compare the overall bonding anomaly severity index E with the preset anomaly judgment threshold Et. If E≥E_t, the product bonding quality is judged to be unqualified; otherwise, the product bonding quality is judged to be qualified. S402: For products deemed non-conforming, extract the spatial distribution information of all abnormal areas from the anomaly detection report; select the largest abnormal area as the main abnormal area, and calculate the geometric center coordinates O(xo, yo) of the main abnormal area, where xo represents the abscissa of the geometric center and yo represents the ordinate of the geometric center; based on the geometric center coordinates and the preset laser marking offset (Δx, Δy), calculate the laser marking coordinates G(xg, yg), where xg = xo + Δx, yg = yo + Δy; the preset laser marking offset is determined based on three key parameters: the minimum distance between the bonding point and the marking safety area specified in the product packaging design specification, the radius of the heat-affected zone of the laser marking process, and the comprehensive positioning error of the vision positioning system and the mechanical transmission system. Specifically, the values ​​of Δx and Δy are equal to the design distance from the bonding point to the safety area plus the radius of the laser heat-affected zone, and then superimposed with the standard deviation of the comprehensive positioning error of the vision positioning system and the mechanical transmission system; S403: Send the laser marking coordinates M(xg,yg) to the laser marking equipment, control the laser marking equipment to perform marking operations at coordinates M(xg,yg) to mark unqualified products; at the same time, record the inspection results and marking information of the product, generate a final inspection report and store it in the database; for products that are judged to be qualified, generate a qualified inspection report and store it in the database; at the same time, control the production line conveyor to transfer qualified products to the next process, and complete the fully automatic bonding inspection process.

[0010] An image recognition-based fully automated bonding detection system includes an image acquisition and modeling module, a reference transfer analysis module, an anomaly analysis module, and a quality judgment module. The image acquisition and modeling module acquires images of the bonding region through a multi-angle vision sensor, identifies and locates each bonding point in the image; for each bonding point, it records its position coordinates, size and arc height; based on the spatial distribution of the bonding points, it establishes a node network model, where each node corresponds to a bonding point and stores its position information and measurement parameters. The reference transfer analysis module selects a starting node in the node network model and sets its measured value as the initial reference. Following a preset transfer path, the reference value is transferred to adjacent nodes. Each receiving node compares its own measured value with the received reference value: if the difference is within the allowable range, the reference is updated with its own measured value and the transfer continues; if the difference exceeds the range, the original reference value is maintained and the transfer continues, and the deviation status of that node is recorded. This process continues in the network until all nodes have completed processing. During the baseline transfer process, the anomaly analysis module records the adjustment range of the baseline value at each node in real time; calculates the cumulative adjustment of the baseline value on each transfer path, identifies abnormal transfer paths where the cumulative adjustment exceeds the path tolerance threshold; and simultaneously counts the nodes recorded as deviating in the network, marking abnormal areas where the density of deviating nodes exceeds the regional density threshold. The quality assessment module determines the bonding quality based on the number of identified abnormal transmission paths and the area of ​​abnormal regions. When the overall abnormality of the bonding meets the preset abnormality assessment conditions, the bonding quality of the product is determined to be unqualified. For unqualified products, the geometric center position of the main abnormal area is determined, the laser marking coordinates are calculated, and the marking operation is performed.

[0011] The image acquisition and modeling module includes a 3D acquisition unit and a network construction unit; The 3D acquisition unit acquires multi-view images of the bonding region through multi-angle vision sensors, performs image registration and 3D reconstruction to obtain 3D point cloud data, identifies the 3D contour of each bonding point based on point cloud segmentation, and obtains its coordinates, equivalent diameter and arc height.

[0012] The network building unit defines each bonding point as a detection node, establishes spatial topology connections between nodes based on electrical connection relationships and physical layout location information, and constructs a node network model stored in the form of a graph structure. The baseline transfer analysis module includes a path initialization unit and a baseline transfer unit; The path initialization unit selects the node whose position coordinates are closest to the geometric center in the node network model as the starting node, generates a transmission path sequence with that node as the root node, and initializes the reference value vector. The reference transmission unit transmits the reference value to the adjacent nodes in sequence according to the transmission path. By comparing the relative deviation between the node's own measurement value and the received reference value, it dynamically updates the reference value or marks the deviation status until all nodes have completed processing.

[0013] The anomaly analysis module includes an adjustment record unit, a path analysis unit, a region analysis unit, and a report generation unit; The adjustment recording unit records the adjustment range of the reference value at each node in real time, including the size adjustment range and the arc height adjustment range; The path analysis unit calculates the cumulative adjustment of the baseline value on each transmission path and identifies abnormal transmission paths by comparing it with the path tolerance threshold. The region analysis unit divides the bonding region into grid regions, counts the density of off-nodes in each grid region, and marks abnormal regions by comparing it with the region density threshold. The report generation unit summarizes the identification results of anomaly propagation paths and anomaly regions, and generates an anomaly detection report that includes the number of anomalies, total area, and location distribution.

[0014] The quality assessment module includes a quality assessment unit, a coordinate calculation unit, and an execution control unit; The quality assessment unit calculates the overall abnormality index based on the abnormality detection report, and determines whether the product quality is qualified by comparing it with the abnormality assessment threshold. The coordinate calculation unit determines the geometric center of the main abnormal area of ​​the non-conforming product and calculates the laser marking coordinates in combination with the preset laser marking offset. The execution control unit controls the laser marking equipment to perform marking operations, generate and store inspection reports, and simultaneously controls the production line conveyor to achieve product diversion.

[0015] Compared with the prior art, the beneficial effects achieved by the present invention are: This invention uses multi-angle visual acquisition and 3D reconstruction technology to accurately obtain key parameters such as the position coordinates and size of each bonding point, and constructs a node network model that reflects the actual connection relationship of the bonding points. Compared with traditional manual sampling or single-parameter detection methods, this invention achieves full coverage and high-precision measurement of the bonding area, providing a complete and reliable data foundation for subsequent quality analysis. This invention utilizes an innovative benchmark transfer mechanism to sequentially transmit quality standards throughout the network, starting from the central node. This effectively identifies consistency differences between bonding points. This method can not only detect obvious defects in individual bonding points but also uncover hidden problems such as overall inhomogeneity and gradual defects that are difficult to detect using traditional methods. By combining anomaly propagation path analysis and anomaly region density statistics, multi-dimensional quality inspection from local to global perspectives is achieved, significantly improving the accuracy and reliability of defect detection. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a fully automated bonding detection method based on image recognition. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Please see Figure 1 This invention provides a technical solution: a fully automated bonding detection method based on image recognition, comprising: S100: Acquires images of the bonding region through a multi-angle vision sensor, identifies and locates each bonding point in the image; records the position coordinates, size and arc height of each bonding point; establishes a node network model based on the spatial distribution of the bonding points, where each node corresponds to a bonding point and stores its position information and measurement parameters; S200: Select a starting node in the node network model and set its measurement value as the initial benchmark; transmit the benchmark value to adjacent nodes according to the preset transmission path; each receiving node compares its own measurement value with the received benchmark value: if the difference is within the allowable range, update the benchmark with its own measurement value and continue transmission; if the difference exceeds the range, maintain the original benchmark value and continue transmission, and record the deviation status of the node; this process continues in the network until all nodes have completed processing; S300: During the reference transfer process, the adjustment range of the reference value at each node is recorded in real time; the cumulative adjustment of the reference value on each transfer path is calculated, and abnormal transfer paths in which the cumulative adjustment exceeds the path tolerance threshold are identified; at the same time, nodes recorded as deviating in the network are counted, and abnormal areas in which the density of deviating nodes exceeds the area density threshold are marked. S400: The bonding quality is judged based on the number of abnormal transmission paths and the area of ​​abnormal regions identified. When the overall abnormality of the bonding meets the preset abnormality judgment conditions, the bonding quality of the product is judged to be unqualified. For unqualified products, the geometric center position of the main abnormal area is determined, the laser marking coordinates are calculated and the marking operation is performed.

[0019] S100 includes: S101: Using a multi-angle vision sensor fixed above the production line, images of the bonding area of ​​the packaged product under inspection are simultaneously acquired from the top vertical view and the two side tilted view under preset multispectral illumination conditions. The multi-angle vision sensor system includes high-resolution industrial cameras and a 3D scanning device arranged in different orientations. The acquired multi-view images are registered and reconstructed in 3D to obtain 3D point cloud data of the bonding area. Based on the 3D point cloud data, the 3D contour of each bonding point is identified by a point cloud segmentation algorithm, and the 3D coordinates of each bonding point in the product coordinate system are determined. For each identified bonding point, the equivalent diameter is obtained as a size parameter based on its 3D contour, and the arc height is obtained by analyzing the vertical distance between the highest point of the lead point cloud and the pad reference plane. S102: Define each bonding point as a detection node. The attribute data of the node includes at least its three-dimensional coordinates, equivalent diameter, and arc height. Based on the product's packaging design structure, obtain the electrical connection relationship and physical layout location information between the bonding points. Map the electrical connection relationship and physical layout location information to spatial topological connections between nodes, wherein a connection relationship is established between adjacent bonding points, and an association relationship is established between bonding points on the same conductive path. Based on the node attribute data and spatial topological connection relationship, construct a node network model of the product. This model is stored in the form of a graph structure, where nodes store attribute data and edges store topological connection relationships.

[0020] S200 includes: S201: In the node network model, the node whose position coordinates are closest to the geometric center of the product is selected as the starting node s, and its measurement value vector Ms is set as the initial reference value Bs, i.e., Bs=Ms, where the measurement value vector includes the equivalent diameter ds and the arc height hs; based on the spatial topology connection relationship in the node network model, a network traversal method is used to generate a transmission path sequence P={p1,p2,...,pn} with the starting node s as the root node, where p1 represents the starting node s and pn represents the last node; according to the transmission path sequence P, the reference value is transmitted from the starting node to the adjacent nodes in sequence; S202: For each node pj (j≥2) in the transmission path sequence P, receive the reference value vector Bi from its predecessor node pi; compare the difference between the self-measured value vector Mj of node pj and the received reference value vector Bi, and calculate the relative deviation of each component: size deviation Δdj=|dj-di| / di, arc height deviation Δhj=|hj-hi| / hi; where dj represents the equivalent diameter measurement value of node pj; di represents the equivalent diameter reference value received from the predecessor node pi; hj represents the arc height measurement value of node pj; and hi represents the arc height reference value received from the predecessor node pi; if If the relative deviations of all components are less than or equal to the corresponding allowable range thresholds, i.e., Δdj≤δd and Δhj≤δh, where δd represents the allowable deviation threshold for size and δh represents the allowable deviation threshold for arc height, then the reference value vector is updated with its own measured value vector, i.e., Bj=Mj; otherwise, the received reference value vector remains unchanged, i.e., Bj=Bi, and the deviation status flag Fj=1 is marked in the node attributes; then the updated reference value vector Bj continues to be passed to the subsequent nodes of node pj in the transmission path; this reference transmission process continues in the network until all nodes in the transmission path sequence P have completed processing.

[0021] The S300 includes: S301: During the reference transfer process, the reference value adjustment range ΔAj at each node pj is recorded in real time; the adjustment range ΔAj is obtained by calculating the relative changes of each component of the reference value vector Bj of node pj and the received reference value vector Bi, and the specific calculation formula is: size adjustment range ΔA_dj=|dj-di| / di×100%; arc height adjustment range ΔA_hj=|hj-hi| / hi×100%; where ΔA_dj represents the size adjustment range at node pj; ΔA_hj represents the arc height adjustment range at node pj; when node pj does not update the reference value, the adjustment range ΔAj is recorded as 0; S302: Based on the transmission path sequence P, calculate the cumulative adjustment amount ΣΔAk of ​​the baseline value on each transmission path Lk; the transmission path Lk is defined as a complete transmission link from the starting node s to any node pj; the cumulative adjustment amount ΣΔAk is obtained by summing the adjustment magnitudes of all nodes on the path: ΣΔAk = Σ pj∈Lk (wd×ΔA_dj+wh×ΔA_hj); where wd and wh represent the weighting coefficients for size and arc height, respectively, and wd+wh=1; the cumulative adjustment ΣΔAk of ​​each transmission path is compared with the preset path tolerance threshold θpath. When ΣΔAk>θpath, the transmission path is marked as an abnormal transmission path; the initial values ​​of the weighting coefficients wd and wh are set to 0.5, indicating that the size and arc height parameters are given equal importance in the initial stage; during operation, the weighting coefficients are periodically redistributed by statistically analyzing the Pearson correlation coefficient between the adjustment range of each parameter and the final quality defect in historical data: higher weights are assigned to the parameters related to quality defects. The parameters exhibiting stronger correlation are used to ensure that the weighting coefficients always reflect the actual quality impact of the parameters. The specific process is as follows: Regularly analyze the binarized results of the size adjustment range and arc height adjustment range of each bonding point in the recent product inspection records and whether the product is ultimately judged as unqualified. Qualified is 0, and unqualified is 1; calculate the Pearson correlation coefficients rd and rh of the size adjustment range, arc height adjustment range, and quality judgment results respectively; redistribute the weighting coefficients according to the following rules: wd_new=|rd| / (|rd|+|rh|), wh_new=|rh| / (|rd|+|rh|), and always satisfy wd_new+wh_new=1; S303: Based on the spatial distribution of the node network model, the bonding region is divided into several grid regions of equal area {C1, C2, ..., Cm}, where Cm represents the m-th grid region; the number of nodes in the deviated state in each grid region is counted, and the corresponding deviated node density ρm = Nm / Am is calculated, where Nm represents the number of nodes in the deviated state in the m-th grid region, and Am represents the area of ​​the m-th grid region; the deviated node density ρm of each grid region is compared with the preset region density threshold θarea, and when ρm > θarea, the grid region is marked as an abnormal region; S304: Summarize the identification results of all abnormal transmission paths and abnormal regions to form a complete anomaly detection report, including the number of abnormal transmission paths, the total area of ​​abnormal regions, and the spatial location distribution information of each abnormal region.

[0022] The S400 includes: S401: Based on the anomaly detection report generated in S304, extract the number of anomaly propagation paths (Npath) and the total area of ​​anomaly regions (Sarea); calculate the overall bond anomaly severity index E, with the formula: E = α × (Npath / Nt_path) + β × (Sarea / St_area); where Nt_path represents the total number of propagation paths, St_area represents the total area of ​​bonded regions, α represents the influence weight coefficient of the anomaly propagation path, β represents the influence weight coefficient of the anomaly region, and α + β = 1; compare the overall bond anomaly severity index E with the preset anomaly judgment threshold Et. If E ≥ E_t, the product bond quality is determined to be unqualified; otherwise, the product bond quality is determined to be qualified. Initially set α = 0.5. β=0.5; Subsequently, historical test data of all products within a statistical window are collected periodically. Each data point includes the number of abnormal transmission paths Npath, the total area of ​​abnormal regions Sarea, and the final quality verification result of the product in subsequent testing stages. Qualified results are recorded as 0, and unqualified results due to bonding problems are recorded as 1. Based on this dataset, the Pearson correlation coefficient r_path between the abnormal transmission path number sequence and the quality result sequence, and the Pearson correlation coefficient r_area between the abnormal region area sequence and the quality result sequence are calculated respectively. Then, the weights are dynamically reallocated: α=|r_path| / (|r_path|+|r_area|), β=|r_area| / (|r_path|+|r_area|). S402: For products deemed non-conforming, extract the spatial distribution information of all abnormal areas from the anomaly detection report; select the largest abnormal area as the main abnormal area, and calculate the geometric center coordinates O(xo, yo) of the main abnormal area, where xo represents the abscissa of the geometric center and yo represents the ordinate of the geometric center; based on the geometric center coordinates and the preset laser marking offset (Δx, Δy), calculate the laser marking coordinates G(xg, yg), where xg = xo + Δx, yg = yo + Δy; the preset laser marking offset is determined based on three key parameters: the minimum distance between the bonding point and the marking safety area specified in the product packaging design specification, the radius of the heat-affected zone of the laser marking process, and the comprehensive positioning error of the vision positioning system and the mechanical transmission system. Specifically, the values ​​of Δx and Δy are equal to the design distance from the bonding point to the safety area plus the radius of the laser heat-affected zone, and then superimposed with the standard deviation of the comprehensive positioning error of the vision positioning system and the mechanical transmission system; S403: Send the laser marking coordinates M(xg,yg) to the laser marking equipment, control the laser marking equipment to perform marking operations at coordinates M(xg,yg) to mark unqualified products; at the same time, record the inspection results and marking information of the product, generate a final inspection report and store it in the database; for products that are judged to be qualified, generate a qualified inspection report and store it in the database; at the same time, control the production line conveyor to transfer qualified products to the next process, and complete the fully automatic bonding inspection process.

[0023] This invention uses a QFN32L-B packaged ASIC chip as the testing object. The chip's bonding area is 4×4mm, containing 32 matrix-layout bonding points (8 columns × 4 rows), divided into 4 independent conductive paths (each group has 8 bonding points connected in series to correspond to the chip's 4 core signals). The bonding points use gold wire bonding technology (25μm gold wire diameter), and must meet the requirements of an equivalent diameter of 30-50μm and an arc height of 15-25μm. The hardware includes a multi-angle vision sensor system (one 12-megapixel industrial camera on top, one 8-megapixel industrial camera on each side, and a 3D scanning device), a multispectral illumination system (a white ring light source on top and near-infrared strip light sources on both sides), and auxiliary positioning and marking equipment. The preset allowable deviation thresholds for size and arc height are 10% and 15%, respectively, and the weighting coefficients for size and arc height are dynamically adjusted to 0.6 and 0.4, respectively. The radius tolerance threshold is 30%, the grid area is 0.25 mm², the area density threshold is 40%, the anomaly judgment weights are α=0.5, β=0.5 and the threshold is 0.3, and the laser marking offset (Δx, Δy) is preset to (0.2 mm, 0.2 mm). This offset is determined comprehensively based on the product packaging design specifications, laser process characteristics and system positioning accuracy. Specifically, according to the product packaging design specifications, the minimum design spacing between the bonding point and the surrounding marking safety zone is 0.1 mm; the radius of the heat-affected zone of the laser marking process is 0.05 mm; after calibration, the comprehensive positioning error of the system's visual positioning and mechanical transmission is 0.05 mm, so the values ​​of Δx and Δy are: 0.1 mm (design spacing) + 0.05 mm (heat-affected zone radius) + 0.05 mm (positioning error) = 0.2 mm, hence the laser marking offset (0.2 mm, 0.2 mm). First, the production line positioning device is activated to fix the chip under test using vacuum adsorption (positioning deviation ≤ 0.01mm). This triggers a multi-angle vision sensor to simultaneously acquire images under multispectral illumination. The top camera acquires vertical images with a 100μs exposure time and 1.0 gain to capture the top surface contour of the bonding points. The two side cameras acquire images from a 45° tilted angle with an 80μs exposure time and 1.2 gain to record the side curvature of the lead wires. Each acquisition takes 200ms, generating three images with matching resolution. Next, the ORB feature point matching algorithm is used for image registration. ≥50 bonding point Mark feature points are extracted from the top image as a reference, correcting for translation and rotation deviations in the two side images. The registration accuracy is ±0.005mm. Finally, the MVS algorithm is used, combining sensor intrinsic parameters (focal length 8mm, principal point coordinates 960×600) and extrinsic parameters (camera spacing 5). A 3D point cloud model with a density of 1000 points / mm² and an accuracy of ±0.5μm was generated (0mm, vertical height 100mm). Then, a region growing algorithm was used to segment the point cloud, selecting the centroid of the bright regions with grayscale values ​​of 200-255 as seed points (32 in total). Following the rule of normal vector angle ≤5° and point-to-point distance ≤0.5μm, 32 bonding point-lead combination point clouds were separated. Finally, a product coordinate system was established with the positioning hole at the lower left corner of the chip as the origin. The 3D coordinates were determined by the centroid of the point cloud (e.g., the corrected coordinates of the core bonding point P5 are (1.0, 1.0, 0.75)mm, with an accuracy of ±0.001mm). The RANSAC algorithm was used to fit the top contour of the bonding point to obtain the equivalent diameter (P5 is 42μm). After fitting the pad reference plane, the vertical distance between the highest point of the lead and the plane was calculated to obtain the arc height (P5 is 20μm). After parameter extraction, the 32 bonding points are defined as detection nodes (attributes include 3D coordinates, equivalent diameter, and arc height; for example, the P5 node attributes are (1.0, 1.0, 0.75) mm, 42 μm, and 20 μm). The electrical connections (4 sets of conductive paths) and physical layout (16 nodes in the core area and 16 nodes in the edge area) of the bonding points are retrieved from the packaging design file and mapped to the node topology connections (spatial connections between adjacent nodes and connections between nodes on the same conductive path). A graph-structured node network model containing 32 nodes and 64 edges is constructed and stored. In the node network, the node closest to the chip's geometric center (2.0, 2.0, 0.75) mm is selected as the starting node S. The measurement value vector (40μm, 22μm) is set as the initial reference Bs. Eight transmission path sequences are generated by traversing (each path covers 4 nodes, P1: S→P2→P3→P4). Then, the reference is transmitted according to the path. Taking P1 as an example, node P2 receives the reference (40μm, 22μm), and its own measurement value (43μm, 23μm) is calculated to have a size deviation of 7.5% and an arc height deviation of 4.5% (both less than the threshold). The reference is updated to (43μm, 23μm) and transmitted to P3. After receiving the reference, P3's own measurement value (48μm, 21μm) has a size deviation of 11.6% (exceeding the threshold). The original reference is maintained and the deviation status is marked. Finally, 6 edge region nodes out of 32 nodes are marked as deviation. The adjustment range of each node was recorded in real time. P2, due to updating the baseline, had a weighted overall adjustment range of 6.3%, while nodes without updated baselines were recorded as 0 (average 5.2% in the core area and 8.1% in the edge area). The cumulative adjustment of the eight paths was then calculated. Path P5 was marked as abnormal because the adjustment ranges of three consecutive nodes were too large (6.8%, 7.5%, and 8.2%), with a cumulative value of 35.7% exceeding the threshold. Ultimately, two abnormal paths were identified. Simultaneously, the bonding region was divided into 64 0.25mm segments. 2 The mesh was statistically analyzed, and it was found that mesh C23 (X1.5-1.75mm, Y2.0-2.25mm) deviated from the node density by 50%, exceeding the threshold. It was marked as an abnormal region and four adjacent abnormal meshes (total area 1mm) were merged. 2 This process generates a comprehensive detection report that includes the number of abnormal paths, the area of ​​abnormal regions, and their locations. The overall anomaly index E = 0.5 × (2 / 8) + 0.5 × (1 / 16) = 0.359 is calculated. If the value is greater than or equal to the threshold of 0.3, the product is deemed unqualified. The merged abnormal area is selected as the main abnormal area, and its geometric center (1.75, 2.25) mm is calculated. Combined with the offset, the marking coordinates (1.95, 2.45) mm are obtained and sent to the laser marking equipment to mark a 0.3 mm circular mark. At the same time, the parameters of 32 bonding points and the abnormal information are recorded to generate a final report for storage. The production line is then controlled to transfer the unqualified products to a dedicated workstation.

[0024] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. An image recognition-based full-automatic detection method for bonding, characterized in that: The method comprises: S100: Collecting the bonding area image through a multi-angle visual sensor, identifying and positioning each bonding point in the image; for each bonding point, recording its position coordinates, size and arc height; according to the spatial distribution of the bonding points, establishing a node network model, wherein each node corresponds to a bonding point, storing its position information and measurement parameters; S200: Selecting a starting node in the node network model and setting its measurement value as an initial reference; according to a preset transmission path, transmitting the reference value to adjacent nodes; each receiving node compares its own measurement value with the received reference value: if the difference is within the allowable range, the reference value is updated with the own measurement value and the transmission continues; if the difference exceeds the range, the original reference value is maintained and the transmission continues, and the deviation state of the node is recorded; this process continues in the network until all nodes are processed; S300: During the reference transmission process, the adjustment amplitude of the reference value at each node is recorded in real time; the cumulative adjustment amount of the reference value on each transmission path is calculated, and the abnormal transmission path with a cumulative adjustment amount exceeding the path tolerance threshold is identified; at the same time, the nodes recorded in the deviation state in the network are counted, and the abnormal area with a deviation node density exceeding the area density threshold is marked; S400: According to the number of abnormal transmission paths and the area of abnormal areas identified, the bonding quality is judged, and when the overall abnormality degree of the bonding meets the preset abnormality judgment condition, the product bonding quality is judged as unqualified; for the unqualified product, the geometric center position of the main abnormal area is determined, the laser marking coordinates are calculated and the marking operation is performed.

2. The fully-automatic bonding detection method based on image recognition according to claim 1, characterized in that: The S100 comprises: S101: Under the condition of preset multi-spectrum illumination, the multi-angle visual sensor fixed above the production line synchronously collects the top vertical view, two side inclined view images of the bonding area of the packaged product to be detected; the multi-angle visual sensor system comprises high-resolution industrial cameras and three-dimensional scanning devices arranged at different positions; the collected multi-view images are image registered and three-dimensionally reconstructed to obtain three-dimensional point cloud data of the bonding area; based on the three-dimensional point cloud data, the three-dimensional profile of each bonding point is identified through a point cloud segmentation algorithm, and the three-dimensional coordinates of each bonding point in the product coordinate system are determined; for each identified bonding point, the equivalent diameter is obtained as the size parameter according to its three-dimensional profile, and the arc height value is obtained by analyzing the vertical distance between the highest point of the lead point cloud and the pad reference plane; S102: Each bonding point is defined as a detection node, and the attribute data of the node at least includes its three-dimensional coordinates, equivalent diameter and arc height value; based on the packaging design structure of the product, the electrical connection relationship and physical layout position information between the bonding points are obtained; the electrical connection relationship and physical layout position information are mapped into the spatial topological connection between the nodes, wherein the connection relationship between adjacent bonding points is established, and the bonding points on the same conductive path are associated; based on the node attribute data and the spatial topological connection relationship, the node network model of the product is constructed, which is stored in the form of a graph structure, wherein the node stores the attribute data and the edge stores the topological connection relationship.

3. The fully-automatic bonding detection method based on image recognition according to claim 2, characterized in that: The S200 comprises: S201: In the node network model, the node closest to the geometric center of the product in terms of position coordinates is selected as the starting node s, and its measurement vector Ms is set as the initial reference value Bs, i.e. Bs=Ms, wherein the measurement vector contains the equivalent diameter ds and the arc height hs; based on the spatial topological connection relationship in the node network model, a network traversal method is used to generate a transmission path sequence P={p1, p2,..., pn} with the starting node s as the root node, wherein p1 represents the starting node s and pn represents the last node; according to the transmission path sequence P, the reference value is transmitted from the starting node to the adjacent nodes in turn; S202: For each node pj (j≥2) in the transmission path sequence P, the reference value vector Bi is received from the previous node pi; the measurement value vector Mj of the node pj is compared with the received reference value vector Bi, and the relative deviations of each component are calculated: size deviation Δdj=|dj-di| / di, and arc height deviation Δhj=|hj-hi| / hi; wherein dj represents the equivalent diameter measurement value of the node pj; di represents the equivalent diameter reference value received from the previous node pi; hj represents the arc height measurement value of the node pj; hi represents the arc height reference value received from the previous node pi; if all component relative deviations are less than or equal to the corresponding allowable range threshold, i.e. Δdj≤δd and Δhj≤δh, wherein δd represents the size allowable deviation threshold and δh represents the arc height allowable deviation threshold, the reference value vector is updated with the measurement value vector, i.e. Bj=Mj; otherwise, the received reference value vector is kept unchanged, i.e. Bj=Bi, and the deviation state flag Fj=1 is marked in the node attribute; then the updated reference value vector Bj is transmitted to the subsequent nodes of the node pj in the transmission path; this reference transmission process continues in the network until all nodes in the transmission path sequence P are processed.

4. The fully-automatic bonding detection method based on image recognition according to claim 3, characterized in that: The S300 comprises: S301: In the reference transmission process, the reference value adjustment amplitude ΔAj at each node pj is recorded in real time; the adjustment amplitude ΔAj is obtained by calculating the relative change of each component of the reference value vector Bj of the node pj and the reference value vector Bi received by it, and the specific calculation formula is: size adjustment amplitude ΔA_dj=|dj-di| / di×100%; arc height adjustment amplitude ΔA_hj=|hj-hi| / hi×100%; wherein ΔA_dj represents the size adjustment amplitude at the node pj; ΔA_hj represents the arc height adjustment amplitude at the node pj; when the node pj does not update the reference value, the adjustment amplitude ΔAj is recorded as 0; S302: Based on the delivery path sequence P, the reference value cumulative adjustment amount ΣΔAk on each delivery path Lk is calculated; the delivery path Lk is defined as a complete delivery link from the starting node s to any node pj; the cumulative adjustment amount ΣΔAk is obtained by accumulating the adjustment amplitudes of all nodes on the path: ΣΔAk=Σ pj∈Lk (wd×ΔA_dj+wh×ΔA_hj); wherein, wd and wh represent the weight coefficients of the size and the arc height respectively, and wd+wh=1; the cumulative adjustment amount ΣΔAk of each delivery path is compared with the preset path tolerance threshold θpath, and when ΣΔAk>θpath, the delivery path is marked as an abnormal delivery path; S303: Based on the spatial distribution of the node network model, the bonding area is divided into a plurality of grid areas {C1, C2,..., Cm} with equal area, wherein Cm represents the mth grid area; the number of nodes in each grid area in the deviation state is counted, and the corresponding deviation node density ρm=Nm / Am is calculated, wherein Nm represents the number of nodes in the deviation state in the mth grid area, and Am represents the area of the mth grid area; the deviation node density ρm of each grid area is compared with the preset area density threshold θarea, and when ρm>θarea, the grid area is marked as an abnormal area; S304: The identification results of all abnormal transmission paths and abnormal areas are summarized to form a complete abnormal detection report, including the number of abnormal transmission paths, the total area of abnormal areas, and the spatial position distribution information of each abnormal area.

5. The fully automatic bonding detection method based on image recognition according to claim 4, characterized in that: The S400 comprises: S401: Based on the abnormal detection report generated in S304, the number of abnormal transmission paths Npath and the total area of abnormal areas Sarea are extracted; the bonding overall abnormality degree index E is calculated, and the calculation formula is: E=α×(Npath / Nt_path)+β×(Sarea / St_area); wherein Nt_path represents the total transmission path quantity, St_area represents the total bonding area, α represents the influence weight coefficient of abnormal transmission path, β represents the influence weight coefficient of abnormal area, and α+β=1; the bonding overall abnormality degree index E is compared with the preset abnormality judgment threshold Et, if E≥E_t, it is judged that the product bonding quality is unqualified; otherwise, it is judged that the product bonding quality is qualified; S402: For the product judged as unqualified, the spatial position distribution information of all abnormal areas is extracted from the abnormal detection report; the largest abnormal area is selected as the main abnormal area, and the geometric center position coordinates O(xo,yo) of the main abnormal area are calculated, wherein xo represents the horizontal coordinate of the geometric center position, and yo represents the vertical coordinate of the geometric center position; based on the geometric center position coordinates and the preset laser marking offset (Δx, Δy), the laser marking coordinates G(xg, yg) are calculated, wherein xg=xo+Δx, and yg=yo+Δy; S403: The laser marking coordinates M(xg, yg) are sent to the laser marking device, the laser marking device is controlled to perform marking operation at the coordinates M(xg, yg), and the unqualified product is marked; at the same time, the detection result and the marking information of the product are recorded, the final detection report is generated and stored in the database; for the product judged as qualified, the qualified detection report is generated and stored in the database; at the same time, the production line conveying device is controlled to transfer the qualified product to the next process, and the bonding full-automatic detection process is completed.

6. An image recognition-based full-automatic detection system for bonding, characterized in that: The system comprises an image acquisition and modeling module, a reference transmission analysis module, an abnormality analysis module, and a quality judgment module. The image acquisition and modeling module acquires images of the bonding area through multi-angle visual sensors, identifies and locates each bonding point in the images; for each bonding point, records its position coordinates, size and arc height; according to the spatial distribution of the bonding points, establishes a node network model, in which each node corresponds to a bonding point, and stores its position information and measurement parameters; The reference transmission analysis module selects a starting node in the node network model, sets its measurement value as the initial reference; according to the preset transmission path, transmits the reference value to the adjacent nodes; each receiving node compares its own measurement value with the received reference value: if the difference is within the allowable range, the reference value is updated with the own measurement value and the transmission continues; if the difference exceeds the range, the original reference value is maintained and the transmission continues, and the deviation state of the node is recorded; This process continues in the network until all nodes are processed; The abnormality analysis module records the adjustment amplitude of the reference value at each node in real time during the reference transmission process; The cumulative adjustment amount of the reference value on each transmission path is calculated, and the abnormal transmission path in which the cumulative adjustment amount exceeds the path tolerance threshold is identified; at the same time, the nodes recorded as the deviation state in the network are counted, and the abnormal area in which the deviation node density exceeds the area density threshold is marked; The quality determination module determines the bonding quality according to the number of identified abnormal transmission paths and the area of abnormal area, and determines that the product bonding quality is unqualified when the overall abnormality degree of the bonding meets the preset abnormality determination condition; for unqualified products, the geometric center position of the main abnormal area is determined, the laser marking coordinates are calculated and the marking operation is performed.

7. The fully automatic bonding detection system based on image recognition according to claim 6, characterized in that: The image acquisition and modeling module includes a three-dimensional acquisition unit and a network construction unit; The three-dimensional acquisition unit acquires multi-view images of the bonding area through multi-angle visual sensors, performs image registration and three-dimensional reconstruction to obtain three-dimensional point cloud data, identifies the three-dimensional profile of each bonding point based on point cloud segmentation, and obtains its coordinates, equivalent diameter and arc height value; The network construction unit defines each bonding point as a detection node, establishes the spatial topological connection between the nodes based on the electrical connection relationship and the physical layout position information, and constructs a node network model stored in the form of a graph structure.

8. The fully automatic bonding detection system based on image recognition according to claim 6, characterized in that: The reference transmission analysis module includes a path initialization unit and a reference transmission unit; The path initialization unit selects the node with the position coordinates closest to the geometric center as the starting node in the node network model, generates a transmission path sequence with the node as the root node, and initializes the reference value vector; The reference transmission unit transmits the reference value to the adjacent nodes in sequence according to the transmission path sequence, dynamically updates the reference value and marks the deviation state by comparing the relative deviation of the node's own measurement value and the received reference value, until all nodes are processed.

9. The fully automatic bonding detection system based on image recognition according to claim 6, characterized in that: The abnormality analysis module includes an adjustment recording unit, a path analysis unit, a region analysis unit and a report generation unit; The adjustment recording unit records the adjustment amplitude of the reference value at each node in real time, including the size adjustment amplitude and the arc height adjustment amplitude; The path analysis unit calculates the cumulative adjustment amount of the reference value on each transmission path, and identifies the abnormal transmission path by comparing with the path tolerance threshold; The area analysis unit divides the bonding area into grid areas, counts the deviated node density of each grid area, and marks the abnormal area by comparing with the area density threshold; The report generation unit summarizes the identification results of the abnormal transmission path and the abnormal area, and generates an abnormal detection report containing the number of abnormalities, the total area, and the location distribution.

10. The fully automatic bonding inspection system based on image recognition of claim 6, wherein: The quality determination module includes a quality determination unit, a coordinate calculation unit, and an execution control unit; The quality determination unit calculates the overall abnormality degree index of the bonding based on the abnormal detection report, and determines whether the product quality is qualified by comparing with the abnormality determination threshold; The coordinate calculation unit determines the geometric center position of the main abnormal area of the unqualified product, and calculates the laser marking coordinates combined with the preset laser marking offset; The execution control unit controls the laser marking device to perform the marking operation, generates the detection report and stores it, and controls the production line conveying device to realize product diversion.

Images