Chip welding spot quality evaluation method and system based on deep learning

Abstract

The invention relates to the technical field of electronic manufacturing automatic detection, and discloses a chip welding spot quality evaluation method and system based on deep learning. The method comprises the following steps: acquiring an X-ray perspective image of a printed circuit board assembly and an electronic design automation design file, completing coordinate registration, and constructing a multi-modal graph node containing visual, spatial and physical semantic features; then, a heterogeneous topological graph containing homogeneous neighbor edges, heterogeneous neighbor edges and long-range twin edges is constructed according to the heat capacity function type and physical distribution of the welding spots; feature aggregation is carried out by using a graph attention network with introduced heat capacity semantic bias terms, and weight deviation caused by heat capacity difference is corrected; and finally, inputting the updated features into a double-branch judgment module, and dynamically adjusting a judgment threshold value of local defect classification by utilizing a warping risk coefficient based on full-graph symmetry. According to the method, the problem of quality misjudgment caused by different heat dissipation conditions of the welding spots and packaging warping is effectively solved, and the detection robustness and accuracy are improved.

Description

Technical Field

[0001] This invention relates to the field of automated testing technology in electronic manufacturing, specifically to a method and system for evaluating chip solder joint quality based on deep learning. Background Technology

[0002] With the development of electronic assembly technology towards high density and miniaturization, ball grid array (BGA) and chip-scale package (CSP) devices have been widely used in printed circuit board assemblies. Since the solder joints of these types of packages are located under the chip body, traditional automated optical inspection equipment cannot directly observe their soldering quality. Therefore, automated X-ray inspection equipment using X-ray penetration imaging technology has become the main inspection method.

[0003] Existing automated X-ray inspection technologies primarily rely on computer vision algorithms to analyze solder joint images. Early inspection methods were mostly based on manually designed rules, calculating geometric parameters such as the area, roundness, and contrast of the solder joint projection and setting fixed threshold ranges to determine the presence of defects such as cold solder joints, bridging, or voids. However, this rule-based method is sensitive to imaging noise and struggles to extract robust features when faced with complex and varied circuit board wiring backgrounds, resulting in a high false alarm rate.

[0004] In recent years, deep learning technologies, represented by convolutional neural networks, have improved the accuracy of solder joint defect detection. These methods typically use a local image of a single solder joint as input, automatically extracting features and classifying them using neural networks. However, this approach often treats each solder joint on the chip as an independent entity, ignoring the physical connections and process logic between them. In reality, chip solder joints are not isolated; they are interconnected through copper traces within the circuit board and are subject to a shared thermodynamic environment during reflow soldering. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a chip solder joint quality assessment method and system based on deep learning. This solves the problems of existing X-ray inspection technologies ignoring the thermal capacity properties of solder joints and the geometric deformation characteristics of the packaging, which leads to the inability to distinguish between normal process grayscale fluctuations and real defects, as well as the difficulty in using global topological relationships to identify warp-related defects.

[0006] To achieve the above objectives, the present invention provides a chip solder joint quality assessment method based on deep learning, comprising the following steps:

[0007] S100: Acquire X-ray images of printed circuit board assemblies and electronic design automation design files, calculate and complete the registration of image pixel coordinates with physical design coordinates;

[0008] S200: Based on the registration results, solder joint diagram nodes are established, and visual, spatial location and physical semantic features are extracted and spliced ​​together to form an initial feature representation.

[0009] S300, construct a heterogeneous topology graph based on the heat capacity functional type and physical distribution of the solder joints. The heterogeneous topology graph includes homogeneous nearest neighbor edges, heterogeneous nearest neighbor edges, and long-range twin edges.

[0010] S400, the heterogeneous topology graph is input into the graph attention network, and a hot capacity semantic bias term is introduced according to the connection edge type to correct the attention weight coefficients. The updated node feature representation is obtained by aggregating based on the corrected attention weight coefficients.

[0011] S500, the updated node feature representation is input into the dual-branch decision module, the warping risk coefficient is generated using the feature distance between the nodes at both ends of the long-range twin edge, the decision threshold of the local defect classification branch is dynamically adjusted accordingly, and the chip solder joint quality evaluation result is output.

[0012] Preferably, the specific process of calculating the mapping relationship and registration in step S100 includes:

[0013] The theoretical physical coordinates of preset reference marker points are retrieved from the electronic design automation design file, and the image pixel center coordinates of the corresponding reference marker points are located in the X-ray fluoroscopic image.

[0014] Construct a source point set and a target point set containing multiple reference marker points;

[0015] A two-dimensional affine transformation model incorporating translation, rotation, and scaling factors is established. A loss function is constructed with the goal of minimizing the sum of squared coordinate transformation errors. The globally optimal affine transformation matrix is ​​then solved using the least squares method.

[0016] The theoretical physical coordinates of all solder joints in the electronic design automation design file are mapped to image pixel coordinates using the affine transformation matrix, and the network connection attributes parsed from the electronic design automation design file are bound to the mapped image pixel coordinates.

[0017] Preferably, the multimodal feature extraction process in step S200 includes:

[0018] The region of interest image with the physical center of the weld point as the origin is cropped from the X-ray fluoroscopic image, normalized, and then input into the backbone network of the convolutional neural network. The output of the global average pooling layer is extracted as the visual feature vector.

[0019] The relative offset and Euclidean distance of the solder joint relative to the geometric center of the chip are calculated, normalized, and then input into the multilayer perceptron position encoder to obtain the spatial position feature vector.

[0020] The network connection attributes in the electronic design automation design file are analyzed, and the solder joints are classified into discrete thermal capacity functional types according to the electrical network function of the solder joint connection and the copper foil area.

[0021] The physical semantic feature vector is obtained by searching for the row vector corresponding to the heat capacity function type in the preset entity embedding matrix.

[0022] Preferably, the specific process of constructing nearest neighbor connections and partitioning types in step S300 includes:

[0023] Calculate the Euclidean distance between the physical coordinates of all solder joints on the chip, and establish a nearest neighbor connection edge between two solder joint nodes whose Euclidean distance is less than the preset thermally affected radius threshold;

[0024] Retrieve the thermal capacity function type of the two solder joint nodes that establish the nearest neighbor connection edge;

[0025] If the solder joints at both ends of a connecting edge have the same heat capacity function type, the nearest connecting edge is marked as the homogeneous nearest neighbor edge;

[0026] If the two ends of the connecting edge have different heat capacity function types, the nearest connecting edge is marked as the heterogeneous nearest neighbor edge.

[0027] Preferably, the specific process of constructing long-range twin edges in step S300 includes:

[0028] Calculate the coordinates of the geometric center point of the chip package, traverse each solder joint node, and calculate its theoretical symmetric coordinates about the geometric center point;

[0029] Search for target nodes whose physical coordinates match the theoretical symmetric coordinates among all solder joint nodes. If the central symmetry error between two solder joint nodes is less than a preset spatial tolerance threshold, then determine that the two are twin nodes. Establish an undirected connection edge between the twin nodes and mark the undirected connection edge as the long-range twin edge.

[0030] Preferably, the specific process of introducing the heat capacity semantic bias term in step S400 includes:

[0031] Initialize a set of learnable bias parameters, the set of bias parameters containing bias values ​​corresponding to the homogeneous nearest neighbor edge, the heterogeneous nearest neighbor edge, and the long-range twin edge, respectively;

[0032] For any connection edge in the heterogeneous topology graph, identify the type of the connection edge and load the corresponding thermal capacity semantic bias value from the bias parameter set;

[0033] The bias value corresponding to the heterogeneous nearest neighbor edge is used to compensate for the difference in imaging grayscale caused by different thermal capacity functional types, and the bias value corresponding to the long-range twin edge is used to enhance the feature correlation weight of the chip's geometrically symmetrical position.

[0034] Preferably, the specific process of calculating the attention weight coefficient in step S400 includes:

[0035] The initial feature representations of graph nodes are mapped to a lower dimension using a shared linear transformation matrix;

[0036] For the target node and any of its neighboring nodes, their transformed feature vectors are concatenated and then multiplied with the attention weight vector to obtain the original relevance score.

[0037] The thermal capacity semantic bias value corresponding to the connection edge type is directly superimposed onto the original relevance score. The superimposed result is then subjected to nonlinear activation and exponential normalization operations to obtain the corrected attention weight coefficient.

[0038] Preferably, the specific process of step S500 includes:

[0039] Construct a local defect classification module containing a fully connected layer, map the updated node feature representation to a preset defect category space, and output the basic probability distribution of each solder joint belonging to each defect category;

[0040] Construct a globally symmetric consistent branch, traverse the twin node pairs connected by the long-range twin edges, and calculate the Euclidean distance between the twin node pairs in the feature space;

[0041] The Euclidean distance is mapped to a value between zero and one using a nonlinear mapping function, which serves as the warpage risk coefficient for the solder joint location. The larger the Euclidean distance, the higher the warpage risk coefficient.

[0042] Preferably, the specific process of dynamically adjusting the decision threshold in step S500 includes:

[0043] Set basic decision thresholds and sensitivity adjustment factors for defect categories that are sensitive to coplanarity;

[0044] Calculate the product of the sensitivity adjustment factor, the warpage risk coefficient, and the complement of the base probability that the solder joint is classified as normal.

[0045] Subtract the product from the basic decision threshold to obtain the dynamic decision threshold for the defect category; calculate the ratio of the probability value of each defect category in the basic probability distribution to the corresponding dynamic decision threshold, and select the category with the largest ratio as the final chip solder joint quality evaluation result.

[0046] A deep learning-based chip solder joint quality assessment system includes:

[0047] The data acquisition and registration module is used to acquire X-ray images of printed circuit board assemblies and electronic design automation design files, and to calculate the registration relationship between image pixel coordinates and physical design coordinates.

[0048] The multimodal feature extraction module is used to extract the visual feature vector, spatial location feature vector, and physical semantic feature vector of the solder joint, and generate the initial feature representation of the graph node;

[0049] The heterogeneous graph construction module is used to construct a heterogeneous topology graph containing homogeneous nearest neighbor edges, heterogeneous nearest neighbor edges, and long-range twin edges based on the physical distribution of solder joints and the heat capacity functional type.

[0050] The graph attention aggregation module is used to introduce heat capacity semantic bias terms corresponding to edge types into the graph attention network and update node features based on the corrected attention weights.

[0051] The dual-branch collaborative decision module is used to generate quality category probabilities using local defect classification branches, generate warpage risk coefficients using global symmetric consistency branches, and dynamically adjust the decision threshold based on the warpage risk coefficients to output evaluation results.

[0052] This invention provides a method and system for evaluating chip solder joint quality based on deep learning. It has the following beneficial effects:

[0053] 1. This invention addresses the problem of insufficient information from a single visual modality by integrating electronic design automation (EDA) documents and X-ray images. It utilizes an affine transformation model to achieve precise registration between physical design coordinates and image pixel coordinates, and extracts physical semantic features including thermal capacity functional type definitions. This ensures that the feature representation of solder joints not only includes imaging texture but also encompasses their electrical properties and heat dissipation conditions. This multimodal data enhances the model's ability to distinguish between normal process fluctuations and actual defects, especially in complex circuit environments, effectively eliminating background noise interference and improving the accuracy of feature extraction.

[0054] 2. This invention effectively suppresses imaging misjudgments caused by differences in solder joint thermal capacity by constructing a heterogeneous topology graph and introducing a thermal capacity semantic bias term. During feature aggregation in the graph attention network, a bias correction is applied to heterogeneous nearest neighbor edges connecting solder joints with different thermal capacity types, enabling the model to distinguish between abnormal features caused by defects and normal grayscale fluctuations caused by differences in thermal capacity properties. This mechanism enhances the robustness of the algorithm and significantly reduces the false alarm rate caused by different heat dissipation conditions when dealing with regions where grounding pins and signal pins are mixed.

[0055] 3. This invention generates a warpage risk coefficient by calculating the characteristic distance between long-range twin nodes, quantifying the specific impact of package deformation on the coplanarity of solder joints, and adaptively adjusting the decision threshold for local defect classification accordingly. The dynamic evaluation strategy breaks through the limitations of the traditional fixed threshold method, improving the detection accuracy of coplanarity-sensitive defects such as weak cold solder joints or bridging caused by package deformation while ensuring overall detection efficiency. Attached Figure Description

[0056] Figure 1 This is a flowchart of the method of the present invention;

[0057] Figure 2 This is a flowchart of the multi-source data acquisition and spatial registration process of the present invention;

[0058] Figure 3 This is a flowchart illustrating the multimodal node feature construction process of the present invention.

[0059] Figure 4 A logical flowchart for constructing the heterogeneous topology graph based on physical mechanisms in this invention;

[0060] Figure 5 This is a schematic diagram of the internal computational logic of the neural network of the present invention;

[0061] Figure 6 This is a schematic diagram of the dual-branch collaborative quality decision logic of the present invention;

[0062] Figure 7 This is a system framework diagram of the present invention. Detailed Implementation

[0063] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] Please see the appendix Figure 1 This invention provides a deep learning-based method for evaluating chip solder joint quality, comprising the following steps:

[0065] S100: Acquire X-ray fluoroscopic images or tomographic scan data of the printed circuit board assembly to be inspected, and read the electronic design automation (EDA) design file corresponding to the printed circuit board assembly; the EDA design file contains the theoretical physical coordinates and network connection attributes of each solder joint in the chip package; using the reference markers on the printed circuit board assembly, calculate the mapping relationship between the X-ray fluoroscopic image coordinate system and the physical coordinate system of the EDA design file, and register the image pixel coordinates with the physical design coordinates;

[0066] S200, based on the registered coordinate data, establishes a corresponding graph node for each solder joint on the chip and extracts multimodal features; for each solder joint, it extracts a region of interest image with the physical center of the solder joint as the origin from the X-ray fluoroscopic image, and uses a convolutional neural network to extract visual feature vectors; it calculates the relative position coordinates and distance of the solder joint with respect to the geometric center of the chip, and generates position feature vectors through a mapping function; it parses the network connection attributes in the electronic design automation design file, determines the thermal capacity function type of the solder joint, and generates physical semantic feature vectors; it concatenates the visual feature vectors, position feature vectors, and physical semantic feature vectors to form the initial feature representation of each node;

[0067] S300: Construct a heterogeneous topology graph based on the physical spatial distribution and thermal capacity function type of the solder joints; calculate the Euclidean distance between the solder joints, and establish nearest neighbor connections between solder joints whose distance is less than a preset thermally affected radius threshold; if the solder joints at both ends of the connection edge have the same thermal capacity function type, mark the connection edge as a homogeneous nearest neighbor edge; if the solder joints at both ends of the connection edge have different thermal capacity function types, mark the connection edge as a heterogeneous nearest neighbor edge; search for solder joint pairs in a centrally symmetric position based on the chip's geometric center coordinates, and establish long-range twin edges between solder joint pairs that satisfy symmetry constraints.

[0068] S400: Input the heterogeneous topological graph into the graph attention network and perform feature aggregation and update; when calculating the attention weight coefficients between nodes, introduce corresponding thermal capacity semantic bias terms according to the type of the connecting edge; for heterogeneous nearest neighbor edges, compensate for the imaging grayscale differences caused by different thermal capacity function types by loading preset thermal capacity semantic bias values; for long-range twin edges, strengthen the feature association of symmetrical positions by loading corresponding bias values; aggregate the features of neighboring nodes using the attention weight coefficients corrected by introducing bias terms to obtain the updated node feature representation;

[0069] The S500 inputs the updated node features into the dual-branch decision module for collaborative evaluation; the local defect classification branch outputs the quality category probability of each solder joint based on the node features to determine whether the solder joint has defects such as bubbles, bridging, or poor wetting; the global symmetry consistency branch calculates the feature distance between node pairs connected by long-range twin edges to evaluate the overall process consistency of the chip; when the statistical value of the feature distance exceeds the preset threshold, a global abnormal signal is generated and the decision threshold of the local defect classification branch is adjusted to output the final chip solder joint quality evaluation result.

[0070] The specific working principles and logical relationships of the above functional steps will be explained in detail below.

[0071] See attached document Figure 2Step S100 shown in the figure, the specific implementation process of multi-source data acquisition and spatial registration includes the following sub-steps:

[0072] Step S101: Obtain the fluoroscopic imaging data and design documents of the printed circuit board assembly. Scan the printed circuit board assembly containing the target chip using an automated X-ray inspection device to generate X-ray fluoroscopic images or three-dimensional tomographic slice data. The fluoroscopic imaging data is stored in the form of a two-dimensional grayscale matrix or a three-dimensional voxel array. The grayscale values ​​characterize the degree of X-ray absorption by the material in the solder joint area, and this value is directly related to the solder thickness and material density of the solder joint. Read the electronic design automation (EDA) design documents corresponding to the printed circuit board assembly. The file format should be GerberRS-274X, ODB++, or IPC-2581 industry standard format. Analyze the data structure of the design documents and extract the theoretical physical coordinates of each solder joint in the chip package. And the corresponding network connection attributes. The network connection attributes include the network identifier for signal, ground, power, or no-connection pins, which is used to subsequently determine the thermal capacity function type of the solder joint.

[0073] S102, Extract the coordinate pairs of the reference marker points. Retrieve the theoretical physical coordinates of the preset reference marker points (FiducialMark) in the electronic design automation (EDA) design file. In the X-ray fluoroscopic image, locate the corresponding reference marker points using a grayscale template matching algorithm or edge detection algorithm, and extract the pixel center coordinates of the marker points in the image coordinate system. Construct a system containing... The source point set and target point set of each reference marker point ,in For the first The physical coordinates of each marker point The corresponding image pixel coordinates, and the number of selected scattered marker points. There should be at least three degrees of freedom to meet the calculation requirements for plane transformation degrees of freedom.

[0074] S103, Calculate the cross-modal coordinate mapping matrix. Considering the thermal expansion and contraction of the printed circuit board during reflow soldering and the rotational deviation of the imaging device, an affine transformation model is established from the physical coordinate system to the image pixel coordinate system. The affine transformation model includes translation, rotation, and scaling transformation factors. A loss function is constructed with the objective of minimizing the sum of squared coordinate transformation errors, and the globally optimal affine transformation matrix is ​​solved using the least squares method. The coordinate mapping relationship is shown in the following formula:

[0075] ;

[0076] In the formula, This represents the x-coordinate of the pixel in the transformed image. This represents the ordinate of the pixels in the transformed image. Represents a two-dimensional affine transformation matrix, with dimensions... , Represents the coefficients of the linear part. Indicates the translation coefficient. This indicates the solder joint in the design physical coordinate system. coordinate, This indicates the solder joint in the design physical coordinate system. coordinate.

[0077] for The affine transformation matrix, where Characterize scaling and rotation parameters. Characterizing the translation parameters. The matrix obtained by solving the least squares method. It can obtain a linear fitting result with the minimum global residual even when there is local nonlinear deformation on the printed circuit board, thereby ensuring the alignment accuracy of the entire chip area.

[0078] S104 performs full-chip solder joint coordinate registration. This is done using the calculated affine transformation matrix. The algorithm iterates through all solder joints of the target chip in the electronic design automation (EDA) design file, batch mapping their theoretical physical coordinates to image pixel coordinates. For each solder joint, the calculated pixel coordinates are... The center index position of the solder joint in the image is marked, and the network connection attributes extracted from the design file are bound and stored with this index position to complete the spatial and attribute alignment of multi-source data.

[0079] See attached document Figure 3 The specific implementation process of multimodal node feature construction in step S200 shown includes the following sub-steps:

[0080] S201, Extract the local visual feature vector of the solder joint. Based on the image pixel center coordinates determined in step S100. Define the region of interest in the X-ray fluoroscopic image. Extract a fixed-size region using the pixel center coordinates as the center index. The local grayscale image patch of the pixel is used. To address the fluctuations in the exposure parameters of the imaging device, the local grayscale image patch is preprocessed by normalization, mapping the pixel grayscale values ​​to the standard range of [0,1]. A deep convolutional neural network model is constructed as the visual feature extractor, using ResNet, VGG, or MobileNet network structures. In the specific implementation, the fully connected classification layer at the end of the original convolutional neural network structure is removed, retaining the convolutional and pooling layers as the feature extraction backbone network. The preprocessed local grayscale image patch is input into the feature extraction backbone network, and forward propagation is performed. The output tensor of the terminal global average pooling layer is extracted as the visual feature vector of the weld point. This visual feature vector, in the form of high-dimensional hidden layer data, encodes the grayscale texture, edge gradient, and morphological distribution information within the solder joint area.

[0081] Step S202: Construct the spatial location encoding features of the solder joints. Traverse all solder joint coordinates in the electronic design automation design file and calculate the geometric center coordinates of the chip package. For each solder joint Calculate its theoretical physical coordinates Relative offset relative to the geometric center coordinates and the Euclidean distance from the solder joint to the geometric center To standardize the feature scale across different package sizes, the relative offset and Euclidean distance are normalized by dividing the physical length, width, and diagonal dimensions of the chip package, respectively. A multilayer perceptron position encoder is constructed, comprising at least two fully connected layers and a nonlinear activation function. The normalized coordinates and distance values ​​are input into the position encoder, mapping the low-dimensional geometric coordinate data into a high-dimensional spatial position feature vector. This feature vector provides the absolute orientation information of the solder joints on the chip for subsequent graph networks, which helps identify regional defects related to stress distribution.

[0082] Step S203: Generate physical semantic features based on thermal capacity attributes. Parse the netlist information in the electronic design automation (EDA) design file to obtain the name of the electrical network connected to each solder joint. Based on the circuit function of the electrical network and the corresponding copper foil area, classify the solder joints into discrete thermal capacity function categories. Specific categories include: grounding (connecting large areas of copper foil), power (connecting power planes), signal (connecting independent traces), and unconnected pins. Different thermal capacity function categories correspond to different heat dissipation rates, resulting in inherent differences in grayscale in X-ray imaging. Initialize an entity embedding matrix that can be updated via backpropagation, where the number of rows corresponds to the total number of thermal capacity function categories, and the number of columns corresponds to the dimension of the physical semantic features. Based on the thermal capacity function category index of the solder joint, search for and extract the corresponding row vector in the entity embedding matrix to obtain the physical semantic feature vector. Through the model training process, this feature vector maps discrete engineering design attributes into a continuous vector space representation, serving as a prior semantic condition for subsequent attention mechanism computation.

[0083] S204, Fusing multimodal features to generate initial node representations. The visual feature vectors extracted in the above steps... Spatial location feature vector and physical semantic feature vectors Perform channel-level concatenation to form the initial feature vector of the graph node corresponding to the solder joint. The feature fusion process is shown in the following equation:

[0084] ;

[0085] In the formula, Indicates the solder joint or graph node index. Indicates the first The initial feature vector of the graph node corresponding to each solder joint. This represents the initial layer features of a graph neural network. Indicates the first The visual feature vectors of each solder joint are extracted from image or X-ray data. Indicates the first The spatial location feature vector of each solder joint describes its geometric location information. Indicates the first The physical semantic feature vector of each solder joint contains design or process-related information. Represents vector concatenation operation , and The initial feature vector of the fused nodes The dimension is;

[0086] ;

[0087] In the formula, Represents visual feature vectors Dimensions Represents spatial location feature vector Dimensions Represents physical semantic feature vector Dimensions This represents the dimension of the fused feature vector, which satisfies the following condition.

[0088] The fusion of feature vectors aligns image information, spatial geometry information, and physical design information at the data level, providing a complete data foundation for subsequent inference of heterogeneous graph neural networks.

[0089] See attached document Figure 4 The specific implementation process of constructing a heterogeneous topology graph based on physical mechanisms, as shown in step S300, includes the following sub-steps;

[0090] S301, Calculate the Euclidean distance matrix of the physical coordinates of the solder joints. Based on all the physical coordinate data of the solder joints obtained in steps S100 and S200, construct a set of solder joint coordinates. ,in This refers to the total number of solder joints contained in the chip package. For the first Physical center coordinates of each solder joint Execute the distance calculation program to generate the Euclidean distance matrix between solder joints. Elements in the matrix Indicates solder joint With solder joints The straight-line physical distance between them is calculated using the following formula:

[0091] ;

[0092] In the formula, This indicates the total number of solder joints contained in the chip package. Represents the set of physical coordinates of the solder joints. Indicates the first The physical center coordinates of each solder joint Indicates the first The horizontal and vertical coordinates of each solder joint in the physical coordinate system This represents the Euclidean distance matrix between solder joints. Indicates solder joint With solder joints The straight-line physical distance between them Represents the L2 norm, Elements are real numbers Matrix space, Represents the solder joint index, satisfying .

[0093] For large-scale ball grid array packaged devices containing thousands of solder joints, in order to optimize computational efficiency, this embodiment adopts a neighborhood search algorithm based on spatial partitioning, such as kd-tree or quadtree algorithm, to calculate the distance only for nodes within local spatial partitions.

[0094] S302, set the thermally affected radius threshold and establish the basic adjacency relationship. Based on the package specifications of the chip under test, determine the distance threshold for establishing graph connections. This refers to the thermal influence radius. Distance threshold. The value is set to 1.2 to 1.5 times the minimum spacing of the chip pads. This range covers the significant coupling area where heat is conducted through the substrate during the reflow soldering process, and also encompasses the potential physical space range where bridging defects may occur. The distance matrix is ​​traversed for any two distinct solder joint nodes. and If their physical distance satisfies In this case, an undirected edge is created in the graph structure, forming a set of nearest neighbor edges. The connecting edges generated at this point only represent the proximity relationship between nodes in physical space, forming the data transmission channel for information transmission in the graph neural network.

[0095] S303, retrieve the node heat capacity attribute and classify the edge types. (For the set...) Each edge in The system retrieves nodes based on the network attributes in the electronic design automation design file. and nodes The corresponding heat capacity function type label. Let's assume... and They are nodes and nodes The heat capacity function type, where the type set These correspond to signal pins, ground pins, power pins, and unused pins, respectively. Based on the consistency of node attributes, nearest neighbor edges are further divided into homogeneous nearest neighbor edges and heterogeneous nearest neighbor edges to distinguish and model different thermodynamic interaction mechanisms.

[0096] S304, construct a homogeneous nearest neighbor subset. For any nearest neighbor connection edge... If the thermal capacity functional types of its two end nodes are the same, then the condition is satisfied. If so, then the edge is defined as a homogeneous nearest neighbor edge and included in the set of homogeneous edges. The mathematical definition of homogeneous nearest neighbor edges is as follows:

[0097] ;

[0098] In the formula, Indicates the solder joint or graph node index. Represents a node With nodes A nearest neighbor edge between them, This represents the set of nearest neighbor edges obtained based on spatial neighborhood search. Denotes the set of homogeneous nearest neighbor edges. Indicates the first The heat capacity function type label for each solder joint. Indicates the first The heat capacity function type label for each solder joint.

[0099] Homogeneous nearest-neighbor edges connect solder joints with the same thermal design background. During reflow soldering, solder joints in the same neighborhood and with similar properties have similar heat conduction rates and cooling solidification curves. Therefore, in a defect-free state, their X-ray imaging grayscale distribution characteristics should exhibit statistical similarity. In graph neural networks, homogeneous nearest-neighbor edges are used to transmit similar features, playing a role in feature smoothing and enhancing local consistency.

[0100] S305, construct a heterogeneous nearest neighbor subset. For any nearest neighbor connection edge... If the thermal capacity functional types of its two end nodes are different, then the following condition is met. If so, then the edge is defined as a heterogeneous nearest neighbor edge and included in the heterogeneous edge set. The mathematical definition of a heterogeneous nearest neighbor edge is as follows:

[0101] ;

[0102] In the formula, Indicates the solder joint or graph node index. Represents a node With nodes A nearest neighbor edge between them, This represents the set of nearest neighbor edges constructed based on spatial neighborhood search. Denotes the set of heterogeneous nearest neighbor edges. Indicates the first The heat capacity function type label for each solder joint. Indicates the first The heat capacity function type label for each solder joint.

[0103] Heterogeneous nearest neighbor edges connect physically neighboring nodes with significantly different thermal capacities. For example, when a low-thermal-capacity signal pin is adjacent to a high-thermal-capacity ground pin, the ground pin connects to a large area of ​​copper foil, resulting in a rapid heat dissipation rate and a temperature gradient during the solder melting stage. This leads to inherent physical differences in the final solder joint shape and X-ray grayscale value. The purpose of constructing heterogeneous nearest neighbor edges is to explicitly mark these non-defect-related differences, enabling subsequent algorithms to identify and load bias terms based on edge type, thereby offsetting grayscale inconsistencies caused by the design itself and avoiding misjudgments.

[0104] S306 calculates the geometric center symmetry of all chip solder joints. Based on the physical coordinate data of the chip solder joints, it determines the geometric center point of the chip package. Traverse every node in the graph. Calculate its value with respect to the geometric center point. The theoretical symmetric coordinates are used to search for nodes in the solder joint set whose physical coordinates match these theoretical symmetric coordinates. Considering the minute tolerances inherent in PCB manufacturing and mounting processes, a spatial tolerance threshold is introduced. Spatial tolerance threshold The value range is set to 10% to 20% of the diameter of a single solder pad to ensure that interference from adjacent solder pads is eliminated. When the node With nodes When the central symmetry constraint is satisfied, the two nodes are determined to be twin nodes. The formula for calculating the central symmetry constraint is as follows:

[0105] ;

[0106] In the formula, Indicates the solder joint or graph node index. Indicates the first The physical coordinate vector of each solder joint The first represents The physical coordinate vector of each solder joint This represents the coordinates of the geometric center point of the chip package. This represents the space tolerance threshold, used to accommodate manufacturing and mounting errors. 2-norm.

[0107] S307, establish a set of long-range twin edges and assign them physical meaning. For node pairs satisfying the above symmetry constraints... An undirected connection edge is established between them, defined as a long-range twin edge, and included in the twin edge set. Long-range twin edges span the local neighborhood of the chip, connecting corresponding points with the largest geometric span. By constructing long-range twin edges, the graph topology explicitly models the macroscopic deformation patterns of the chip. During the reflow soldering stage of surface mount technology, due to the mismatch in the coefficients of thermal expansion between the chip packaging substrate and the printed circuit board material, warping deformation may occur after heating, exhibiting a centrally symmetrical concave or convex bending shape. This macroscopic deformation causes solder joints in symmetrical positions to exhibit highly correlated quality states or complementary defect patterns (such as solder bridging under pressure on one side and poor solder joint under tension on the symmetrical side). Long-range twin edges provide graph neural networks with topological correlation paths to capture such global process drift.

[0108] See attached document Figure 5 Step S400 shown, the graph attention aggregation based on thermal capacity semantic bias, specifically includes the following sub-steps:

[0109] Step S401: Construct a graph attention network and perform a feature linear transformation. Input the heterogeneous topological graph containing nearest-neighbor edges and long-range twin edges constructed in step S300 into the graph attention network. The graph attention network contains at least one graph attention layer to process non-Euclidean space data between nodes. A shared linear transformation matrix is ​​defined. ,in The dimension of the initial feature vector of the node constructed in step S200. is the feature dimension of the hidden layer. For each node in the graph... , its initial feature vector The input is a linear transformation layer, which performs feature dimensionality reduction and spatial mapping operations to obtain the transformed feature vector. By uniformly mapping feature data from different modalities to the high-dimensional semantic space of graph neural networks, a unified metric benchmark is established for subsequent attention interaction computations.

[0110] S402, Initialize the thermal capacity semantic bias term parameters. To correct imaging biases caused by differences in physical properties during feature propagation, edge-type-specific thermal capacity semantic bias terms are introduced into the graph attention network. Define a set of learnable bias parameters. .in, Corresponding to homogeneous nearest neighbor edges, used to maintain the feature smoothness of nodes of the same type. Corresponding to heterogeneous nearest neighbor edges, this is used to compensate for differences in X-ray imaging grayscale caused by different thermal capacity functional types (such as ground pins and signal pins). Corresponding to long-range twin edges, feature association weights are used to enhance the geometric symmetry of the chip. The bias term, as part of the network parameters, is updated with gradients during model training to learn the contribution of different physical connections to the weld quality decision.

[0111] S403, calculate the attention weight coefficients incorporating physical bias. For any connecting edge in the graph structure... First, calculate the nodes. With nodes The original correlation in the transformed feature space is determined. A single-layer feedforward neural network is used as the scoring function to concatenate the transformed feature vectors of both entities and add them to the weight vector. Perform the dot product operation. Then, identify the edges. Type and from the parameter set Load the corresponding thermal capacity semantic bias value This is then directly superimposed onto the original relevance score. The superimposed score is then normalized using the LeakyReLU activation function and the Softmax function to obtain the corrected attention weight coefficients. The calculation formula is as follows:

[0112] ;

[0113] In the formula, Indicates the graph node index. Represents a node With nodes A connecting edge between them, This represents the input feature vector of a node. Represents a linear transformation matrix. Represents a node Representation in the transformed feature space This represents a vector concatenation operation. Represents the attention weight vector. This represents the transpose of a vector. Representing an edge Type identifier, This represents the physical semantic bias parameter corresponding to the edge type. Represents a node The set of adjacent nodes, This represents the modified linear unit activation function with leakage coefficient. Represents an exponential function. This represents the normalized attention weight coefficient.

[0114] For heterogeneous nearest neighbor edges It can offset the grayscale distribution shift caused by different heat dissipation rates when calculating correlations, preventing the network from erroneously reducing the weights of physically tightly coupled heterogeneous nodes; for long-range twin edges, It can enhance the influence of far-end symmetric nodes in local feature updates, enabling the model to capture the regular changes brought about by macroscopic warping.

[0115] S404 performs feature aggregation and updating based on physical mechanisms. It utilizes the calculated, corrected attention weight coefficients. For nodes All neighboring nodes Transformation eigenvectors Weighted summation is performed to generate aggregated feature representations. To enhance the model's expressive power, a multi-head attention mechanism is employed for parallel execution. The above calculation process is repeated, and the outputs of each attention head are concatenated. A residual connection and layer normalization mechanism are introduced to add and normalize the aggregated features to the original input features, resulting in the updated node feature vector. The feature update process is shown in the following equation:

[0116] ;

[0117] In the formula, Indicates the graph node index. Represents a node The set of adjacent nodes, Represents a node In the The input feature vector of the layer, Represents a node In the The input feature vector of the layer, Represents a node In the The updated feature vector of the layer, Indicates the number of attention heads. Indicates the attention head index. Indicates the first Under each attention head, the node For nodes Attention weights Indicates the first The linear transformation matrix corresponding to each attention head This represents the ELU activation function. Presentation layer normalization operation.

[0118] After this step, the updated node features not only integrate local visual texture information, but also embed the thermodynamic logic and geometric symmetry priors of the circuit board physical design, providing a robust high-order feature representation for subsequent bi-branch defect judgment.

[0119] See attached document Figure 6 The specific implementation process of the dual-branch collaborative quality decision shown in step S500 includes the following sub-steps:

[0120] S501 constructs a local defect classification branch and generates the basic posterior probability. The updated node feature vectors output from S400 are then used. A local defect classification module is constructed. This module consists of a multilayer perceptron, including fully connected layers and nonlinear activation functions, which map node features to a predefined defect category space. The predefined defect category set... These include: normal, bridging, open circuit, bubbles, and pillow effect.

[0121] Node The feature vector is input into the classification module, processed by the Softmax normalization function, and the basic probability distribution of the node belonging to each defect category is calculated. ;

[0122] In the formula, Represents a node The local defect prediction probability vector, This represents the predicted probability that the solder joint is in a normal state. This indicates the predicted probability that the solder joint has a bridging defect. This represents the predicted probability that the solder joint is an open-circuit defect. This represents the predicted probability that the solder joint has a bubble defect. This represents the predicted probability that the solder joint has a pillow effect defect.

[0123] The probability vector represents the welding quality state inferred solely from local image and text information. To optimize model parameters, this branch employs supervised learning using the cross-entropy loss function during the training phase.

[0124] S502, construct a globally symmetric consistency branch and calculate the warpage risk coefficient. This branch aims to assess the potential impact of macroscopic deformation of the chip package during reflow soldering on soldering quality. Iterate through the set of long-range twin edges constructed in step S300. For any pair of nodes with a symmetric relationship Extract its updated feature vector and First, calculate the Euclidean distance between the two in the feature space. Subsequently, a learnable nonlinear mapping function is used to calculate the local warping risk coefficient at that location. ;

[0125] ;

[0126] In the formula, Indicates the index of a solder joint or graph node that has a geometrically centrally symmetric relationship. Represents a pair of twin nodes. Represents a long-range twin edge set. Represents a node , In the The feature vector output by the layered graphical neural network. Represents a node With nodes Euclidean distance in feature space Represents the L2 norm, Represents a node The risk factor for local warping at the corresponding location, This represents the Sigmoid activation function, which maps the input to... interval, This represents a learnable parameter used to adjust risk sensitivity.

[0127] By statistically analyzing the average feature distance of all twin node pairs in the entire graph, a full-chip-level process drift index is calculated to monitor batch-specific process fluctuations.

[0128] S503 executes a collaborative evaluation strategy based on dynamic thresholds. To reduce misjudgments of specific defects caused by PCB-level warpage, the system utilizes the output of global branches. The decision logic for local branches is dynamically adjusted. For defect categories sensitive to coplanarity (especially open circuits and pillow effects), a basic decision threshold is set. Define categories Dynamic decision threshold The calculation logic is as follows;

[0129] ;

[0130] In the formula, Indicates the solder joint or graph node index. Indicates the defect category index. Indicates the basic decision threshold. Indicates the defect category The dynamic decision threshold, Represents a node The corresponding local warpage risk coefficient, Indicates the category of defect Sensitivity modulator (for open circuit and pillow effect, For Lian Xi, For the normal category, ), Represents a node The base probability of being classified as normal.

[0131] This enables the detection of high warpage risk (high Furthermore, when the confidence level of local features for normal judgment is insufficient, the detection threshold for latent defects is automatically lowered. The base probability for each defect category is calculated. Its corresponding dynamic threshold The ratio is used to determine the category with the largest ratio as the final defect judgment result. If the maximum ratio is still less than the preset confidence threshold, output an uncertain label and prompt for manual review.

[0132] The device of the deep learning-based chip solder joint quality assessment system described below can be referred to in correspondence with the deep learning-based chip solder joint quality assessment method described above.

[0133] Please see the appendix Figure 7 The present invention also provides a chip solder joint quality assessment system based on deep learning, comprising:

[0134] The data acquisition and registration module is used to acquire X-ray images of printed circuit board assemblies and electronic design automation design files, and to calculate the registration relationship between image pixel coordinates and physical design coordinates.

[0135] The multimodal feature extraction module is used to extract the visual feature vector, spatial location feature vector, and physical semantic feature vector of the solder joint, and generate the initial feature representation of the graph node;

[0136] The heterogeneous graph construction module is used to construct a heterogeneous topology graph containing homogeneous nearest neighbor edges, heterogeneous nearest neighbor edges, and long-range twin edges based on the physical distribution of solder joints and the heat capacity functional type.

[0137] The graph attention aggregation module is used to introduce heat capacity semantic bias terms corresponding to edge types into the graph attention network and update node features based on the corrected attention weights.

[0138] The dual-branch collaborative decision module is used to generate quality category probabilities using local defect classification branches, generate warpage risk coefficients using global symmetric consistency branches, and dynamically adjust the decision threshold based on the warpage risk coefficients to output evaluation results.

[0139] The system in this embodiment can be used to execute the above method embodiments, and its principle and technical effect are similar, so they will not be described again here.

Claims

1. A deep learning-based method for evaluating chip solder joint quality, characterized in that, Includes the following steps: S100: Acquire X-ray images of printed circuit board assemblies and electronic design automation design files, calculate and complete the registration of image pixel coordinates with physical design coordinates; S200: Based on the registration results, solder joint diagram nodes are established, and visual, spatial location and physical semantic features are extracted and spliced ​​together to form an initial feature representation. S300, construct a heterogeneous topology graph based on the heat capacity functional type and physical distribution of the solder joints. The heterogeneous topology graph includes homogeneous nearest neighbor edges, heterogeneous nearest neighbor edges, and long-range twin edges. S400, the heterogeneous topology graph is input into the graph attention network, and a hot capacity semantic bias term is introduced according to the connection edge type to correct the attention weight coefficients. The updated node feature representation is obtained by aggregating based on the corrected attention weight coefficients. S500, the updated node feature representation is input into the dual-branch decision module, the warping risk coefficient is generated using the feature distance between the nodes at both ends of the long-range twin edge, the decision threshold of the local defect classification branch is dynamically adjusted accordingly, and the chip solder joint quality evaluation result is output.

2. The chip solder joint quality assessment method based on deep learning according to claim 1, characterized in that, The specific process of calculating the mapping relationship and registration in step S100 includes: The theoretical physical coordinates of preset reference marker points are retrieved from the electronic design automation design file, and the image pixel center coordinates of the corresponding reference marker points are located in the X-ray fluoroscopic image. Construct a source point set and a target point set containing multiple reference marker points; A two-dimensional affine transformation model incorporating translation, rotation, and scaling factors is established. A loss function is constructed with the goal of minimizing the sum of squared coordinate transformation errors. The globally optimal affine transformation matrix is ​​then solved using the least squares method. The theoretical physical coordinates of all solder joints in the electronic design automation design file are mapped to image pixel coordinates using the affine transformation matrix, and the network connection attributes parsed from the electronic design automation design file are bound to the mapped image pixel coordinates.

3. The chip solder joint quality assessment method based on deep learning according to claim 1, characterized in that, The multimodal feature extraction process in step S200 includes: The region of interest image with the physical center of the weld point as the origin is cropped from the X-ray fluoroscopic image, normalized, and then input into the backbone network of the convolutional neural network. The output of the global average pooling layer is extracted as the visual feature vector. The relative offset and Euclidean distance of the solder joint relative to the geometric center of the chip are calculated, normalized, and then input into the multilayer perceptron position encoder to obtain the spatial position feature vector. The network connection attributes in the electronic design automation design file are analyzed, and the solder joints are classified into discrete thermal capacity functional types according to the electrical network function of the solder joint connection and the copper foil area. The physical semantic feature vector is obtained by searching for the row vector corresponding to the heat capacity function type in the preset entity embedding matrix.

4. The chip solder joint quality assessment method based on deep learning according to claim 1, characterized in that, The specific process of constructing nearest neighbor connections and classifying the types in step S300 includes: Calculate the Euclidean distance between the physical coordinates of all solder joints on the chip, and establish a nearest neighbor connection edge between two solder joint nodes whose Euclidean distance is less than the preset thermally affected radius threshold; Retrieve the thermal capacity function type of the two solder joint nodes that establish the nearest neighbor connection edge; If the solder joints at both ends of a connecting edge have the same heat capacity function type, the nearest connecting edge is marked as the homogeneous nearest neighbor edge; If the two ends of the connecting edge have different heat capacity function types, the nearest connecting edge is marked as the heterogeneous nearest neighbor edge.

5. The chip solder joint quality assessment method based on deep learning according to claim 4, characterized in that, The specific process of constructing long-range twin edges in step S300 includes: Calculate the coordinates of the geometric center point of the chip package, traverse each solder joint node, and calculate its theoretical symmetric coordinates about the geometric center point; Search for target nodes whose physical coordinates match the theoretical symmetric coordinates among all solder joint nodes. If the central symmetry error between two solder joint nodes is less than a preset spatial tolerance threshold, then determine that the two are twin nodes. Establish an undirected connection edge between the twin nodes and mark the undirected connection edge as the long-range twin edge.

6. The chip solder joint quality assessment method based on deep learning according to claim 1, characterized in that, The specific process of introducing the heat capacity semantic bias term in step S400 includes: Initialize a set of learnable bias parameters, the set of bias parameters containing bias values ​​corresponding to the homogeneous nearest neighbor edge, the heterogeneous nearest neighbor edge, and the long-range twin edge, respectively; For any connection edge in the heterogeneous topology graph, identify the type of the connection edge and load the corresponding thermal capacity semantic bias value from the bias parameter set; The bias value corresponding to the heterogeneous nearest neighbor edge is used to compensate for the difference in imaging grayscale caused by different thermal capacity functional types, and the bias value corresponding to the long-range twin edge is used to enhance the feature correlation weight of the chip's geometrically symmetrical position.

7. The chip solder joint quality assessment method based on deep learning according to claim 6, characterized in that, The specific process for calculating the attention weight coefficient in step S400 includes: The initial feature representations of graph nodes are mapped to a lower dimension using a shared linear transformation matrix; For the target node and any of its neighboring nodes, their transformed feature vectors are concatenated and then multiplied with the attention weight vector to obtain the original relevance score. The thermal capacity semantic bias value corresponding to the connection edge type is directly superimposed onto the original relevance score. The superimposed result is then subjected to nonlinear activation and exponential normalization operations to obtain the corrected attention weight coefficient.

8. The chip solder joint quality assessment method based on deep learning according to claim 1, characterized in that, The specific process of step S500 includes: Construct a local defect classification module containing a fully connected layer, map the updated node feature representation to a preset defect category space, and output the basic probability distribution of each solder joint belonging to each defect category; Construct a globally symmetric consistent branch, traverse the twin node pairs connected by the long-range twin edges, and calculate the Euclidean distance between the twin node pairs in the feature space; The Euclidean distance is mapped to a value between zero and one using a nonlinear mapping function, which serves as the warpage risk coefficient for the solder joint location. The larger the Euclidean distance, the higher the warpage risk coefficient.

9. The chip solder joint quality assessment method based on deep learning according to claim 8, characterized in that, The specific process of dynamically adjusting the decision threshold in step S500 includes: Set basic decision thresholds and sensitivity adjustment factors for defect categories that are sensitive to coplanarity; Calculate the product of the sensitivity adjustment factor, the warpage risk coefficient, and the complement of the base probability that the solder joint is classified as normal. Subtract the product from the basic decision threshold to obtain the dynamic decision threshold for the defect category; calculate the ratio of the probability value of each defect category in the basic probability distribution to the corresponding dynamic decision threshold, and select the category with the largest ratio as the final chip solder joint quality evaluation result.

10. A deep learning-based chip solder joint quality assessment system, applied to the deep learning-based chip solder joint quality assessment method according to any one of claims 1-9, characterized in that, include: The data acquisition and registration module is used to acquire X-ray images of printed circuit board assemblies and electronic design automation design files, and to calculate the registration relationship between image pixel coordinates and physical design coordinates. The multimodal feature extraction module is used to extract the visual feature vector, spatial location feature vector, and physical semantic feature vector of the solder joint, and generate the initial feature representation of the graph node; The heterogeneous graph construction module is used to construct a heterogeneous topology graph containing homogeneous nearest neighbor edges, heterogeneous nearest neighbor edges, and long-range twin edges based on the physical distribution of solder joints and the heat capacity functional type. The graph attention aggregation module is used to introduce heat capacity semantic bias terms corresponding to edge types into the graph attention network and update node features based on the corrected attention weights. The dual-branch collaborative decision module is used to generate quality category probabilities using local defect classification branches, generate warpage risk coefficients using global symmetric consistency branches, and dynamically adjust the decision threshold based on the warpage risk coefficients to output evaluation results.

Images