Method for comparing graphic data of front and back chips based on feature matching technology

Abstract

The invention discloses a method for comparing graphic data of a front chip and a rear chip based on a feature matching technology, and relates to the technical field of semiconductor chip manufacturing, and the method comprises the specific steps of data acquisition, model construction, feature matching, detection execution and result output. According to the method, an improved SuperPoint model is constructed, feature descriptors are introduced for normalization processing, effective key point pairs are screened in combination with a joint matching model, a projection transformation relation between a source image and a target image is established by using a homography matrix, and then full-region uniform sampling and feature comparison are realized through gridding difference detection, so that the target image is obtained. According to the process, by standardizing feature extraction standards, strengthening matching stability and guaranteeing sampling comprehensiveness, the specific type and range of a difference area are clarified in combination with clustering labeling, precise positioning and information quantification of chip layout differences are achieved, it is ensured that various subtle changes can be effectively recognized, and a reliable technical path is provided for chip layout consistency detection.

Description

Technical Field

[0001] This invention relates to the field of semiconductor chip manufacturing technology, specifically to a method for comparing front and rear chip graphic data based on feature matching technology. Background Technology

[0002] As semiconductor technology advances towards higher precision and higher density, chip structures are becoming increasingly complex. Different batches of the same model of chip may exhibit differences in layout structure, bonding status, and other aspects due to various reasons such as fluctuations in the precision of production equipment, fine-tuning of process parameters, and interference from environmental factors. If these differences are not detected in time, they may lead to inconsistent chip performance, decreased reliability, or even failure of end products. Therefore, accurate comparison of unpacking and inspection images of different batches of chips to identify potential differences is a key step in ensuring the stability of chip production quality and has become an important requirement in the semiconductor manufacturing industry.

[0003] However, existing solutions for chip image data comparison have some limitations. Traditional feature extraction methods lack standardized processing, resulting in insufficient matching stability and difficulty in accurately establishing correspondences between different images. Difference detection often uses local sampling, which easily misses subtle differences in effective matching areas. Solder ball deviation assessment relies on manual judgment or simple pixel statistics, which is not accurate enough. Bonding type identification does not form a complete logic of independent identification-confidence screening-batch comparison, resulting in low reliability of type change judgment. At the same time, existing solutions do not integrate multi-dimensional detection requirements and lack systematic integration and visualization of multi-dimensional detection results, making it impossible to achieve integrated comparison of layout differences, solder ball status, and bonding type simultaneously. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a method for comparing front- and back-end chip image data based on feature matching technology. This invention constructs an improved SuperPoint model and introduces feature descriptor normalization processing, combines a joint matching model to screen effective key point pairs, uses a homography matrix to establish the projection transformation relationship between the source and target images, and then achieves uniform sampling and feature comparison across the entire region through gridded difference detection. This process standardizes feature extraction criteria, enhances matching stability, ensures comprehensive sampling, and combines clustering annotation to clarify the specific types and ranges of difference regions, achieving precise positioning and information quantification of chip layout differences. This ensures that all kinds of subtle changes can be effectively identified, providing a basis for chip layout... Figure 1 Consistency testing provides a reliable technical approach.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a method for comparing front and rear chip image data based on feature matching technology, the specific steps of which are as follows:

[0006] Data acquisition: Collect unpacking inspection images of different batches of the same model of chip, covering chip layout images, bonding area images, and collect bonding solder ball images and ball bond, wedge bond, and crescent bond type images. Label the pads, solder ball areas and bonding types, and divide them into training set and test set.

[0007] Model construction: Based on the collected labeled images and the segmented training and test sets, an improved SuperPoint model is constructed. A joint matching model is built by integrating the improved SuperPoint model with the SuperGlue module. The BoTNet module is embedded in the YOLOv8-seg instance segmentation model to complete the optimization. A bonding type recognition model is built based on the YOLOv8 framework. The parameters of each model are iteratively optimized by inputting the training set.

[0008] Feature matching: The acquired chip layout image is converted into a tensor of a specified dimension, which is then input into the trained improved SuperPoint model to extract a feature subset. The feature subset is then input into the joint matching model to output key point pairs and confidence scores. Based on the key point pairs, the homography matrix is ​​estimated and valid point pairs are filtered.

[0009] Detection execution: Based on the homography matrix and the filtered valid point pairs, perform gridded difference detection on the chip layout. Input the bonding region image into the optimized YOLOv8-seg model to obtain pad and solder ball data and calculate the deviation ratio. Input the bonding images of the previous and next batches into the bonding type recognition model to output the type and confidence and determine whether there has been a change.

[0010] Results output: Summarize the difference point data, solder ball deviation ratio and judgment results, bonding type identification results and change judgment conclusions, cluster and label the difference points, generate various visualization images, and integrate them into a test report.

[0011] Furthermore, in the model construction, the improved SuperPoint model is constructed as follows: a deep learning framework is used to build the network structure, and a first convolutional layer, a second convolutional layer, and a linear rectified activation function are set sequentially. An arbitrary coordinate descriptor extraction function is added to support descriptor extraction from arbitrary coordinates. A feature descriptor convolutional layer is configured to generate feature descriptors. Non-maximum suppression radius and descriptor dimension parameters are set. In the descriptor output stage, the extracted feature descriptors are normalized using the L2 normalization formula. The labeled chip layout image training set is input for parameter iteration to complete the construction and optimization. The L2 normalization formula for the feature descriptor is: ,in, This is the normalized feature descriptor vector. This is the original feature descriptor vector, generated by the feature descriptor convolutional layer, containing keypoint feature information. The L2 norm of the original feature descriptor vector is obtained by calculating the square root of the sum of squares of all elements. This represents the feature descriptor dimension, with a value of 256. The first element of the original feature descriptor vector Each element.

[0012] Furthermore, in the model construction, the steps for building the joint matching model are as follows: initializing the configuration parameters of the joint matching model based on the deep learning framework, integrating the improved SuperPoint model as a feature extraction module into the joint matching model, loading the SuperGlue module as the feature matching core of the joint matching model, defining the input image data format, feature matching threshold, and output parameter type of the joint matching model, setting the computing device for inference of the joint matching model, inputting paired chip layout image training sets to jointly train the joint matching model, optimizing the adaptation parameters between the improved SuperPoint model and the SuperGlue module, so that the joint matching model outputs inference results containing key points, matching indices, and confidence scores.

[0013] Furthermore, in the model construction, the optimization and construction steps of the YOLOv8-seg instance segmentation model are as follows: A BoTNet module is inserted between the backbone network and the detection head of the YOLOv8-seg instance segmentation model. The BoTNet module is configured with three convolutional units and a sequence structure composed of bottleneck transformers. The number of input and output channels, the hidden channel expansion coefficients, the number of bottleneck transformer heads, and the resolution parameters of the BoTNet module are defined. The labeled pad and solder ball image training set is input into the YOLOv8-seg instance segmentation model, and the training batch size, iteration rounds, and learning rate are set. The weights of the YOLOv8-seg instance segmentation model are adjusted through backpropagation. The segmentation accuracy of the YOLOv8-seg instance segmentation model is verified using the test set. After iterative optimization, the construction of the YOLOv8-seg instance segmentation model is completed.

[0014] Furthermore, in the model construction, the bonding type recognition model is constructed and trained as follows: Bonding images of three types—spherical, wedge, and crescent—are collected; image annotation tools are used to annotate the bonding regions and types in the images; the training set and test set are divided in an 8:2 ratio; the input image size, anchor box parameters, and number of categories of the bonding type recognition model are configured based on the YOLOv8 framework; a bonding type recognition model network structure including feature extraction, feature fusion, and a detection head is constructed; the bonding type recognition model is trained using the training set; the learning rate and regularization parameters are adjusted based on the recognition accuracy of the validation set; and a confidence threshold is set. The generalization ability of the bonding type recognition model is verified through a test set, and the final weights are selected to complete the construction of the bonding type recognition model. When the trained bonding type recognition model is used, the bonding region images of the previous batch of chips and the bonding region images of the subsequent batch of chips are respectively input into the trained bonding type recognition model. The model outputs the bonding type and confidence level of the two batches of images respectively. If the bonding types of the two batches are inconsistent and the confidence levels of both are not lower than the set confidence level threshold, it is determined that the bonding type has changed. If the bonding types of the two batches are consistent, or the confidence level of either batch is lower than the confidence level threshold, it is determined that the bonding type has not changed effectively.

[0015] Furthermore, in the feature matching, the estimation process of the homography matrix is ​​as follows: The valid keypoint pairs output by the joint matching model are converted into numerical array format; the reprojection error threshold of the random sample consensus algorithm is set; a 3×3 homography matrix and its corresponding mask are obtained through iterative calculation; valid inliers in the mapping structure are selected using the mask, and unreasonable outliers are filtered out; the projection transformation relationship between the source image and the target image is established using the homography transformation projection formula. The homography matrix is ​​a 3×3 matrix describing the projection transformation relationship between the source image and the target image, and is the core parameter for realizing homography transformation projection. The random sample consensus algorithm is used to filter valid information from noisy data. Valid inliers refer to keypoints that conform to the projection transformation relationship between the source image and the target image; unreasonable outliers refer to keypoints that do not conform to the projection transformation relationship due to noise or mismatch. The homography transformation projection formula is: ,in, , These are the projected coordinates of key points in the target image, obtained from the coordinates of key points in the source image through matrix transformation. The matrix is ​​a 3×3 homography matrix, obtained by iteratively calculating valid keypoint pairs using a random sampling consensus algorithm. , Here are the original coordinates of the keypoints in the source image, and here are the valid keypoint pairs output by the joint matching model. The elements of the homography matrix, The value of is 0-2, with a total of 9 values, which are determined by iteratively optimizing the projection error of key point pairs.

[0016] Furthermore, in the detection process, the execution operation of the gridded difference detection is as follows: Effective inliers are projected onto the homography matrix to form corresponding point pairs between the source and target images. The extreme coordinates of the effective matching region are obtained based on the coordinate range of the corresponding point pairs. Grid control parameters are set to control the grid density and sampling interval. Uniformly distributed x and y coordinates are generated within the extreme coordinate range to form a uniform grid covering the effective matching region. Based on the homography matrix, grid points are projected onto the corresponding positions in the target image, and feature descriptors are extracted from the grid points in the source image and the projected grid points in the target image, respectively. The similarity distance between the descriptors of corresponding grid points in the image is calculated using the descriptor similarity measurement formula to quantify the feature differences. A threshold is set to filter out difference points greater than the threshold. The descriptor similarity measurement formula is: ,in, This represents the similarity distance between the descriptors of the source and target images. A larger value indicates a more significant difference in features. The normalized feature descriptor for the grid points of the source image is obtained by extracting the improved SuperPoint model and then performing L2 normalization. The normalized feature descriptor for the grid points corresponding to the target image is obtained by projecting the grid points from the source image and then extracting and normalizing them using the improved SuperPoint model. This represents the feature descriptor dimension, with a value of 256. These are the normalized feature descriptors of the source image grid points and the first... Each element.

[0017] Furthermore, in the detection process, the calculation steps for the solder ball deviation ratio are as follows: First, count the total number of valid pixels in the solder ball mask output by the YOLOv8-seg instance segmentation model to obtain the total number of solder ball pixels. Second, process the solder ball mask and the pad mask. The solder ball mask is constructed from the solder ball data output by the YOLOv8-seg instance segmentation model, and the pad mask is constructed from the pad data output by the YOLOv8-seg instance segmentation model. Count the total number of valid pixels in the processed result to obtain the number of pixels exceeding the pad area. Then, calculate the solder ball deviation ratio using the solder ball deviation ratio calculation formula. When it exceeds 50%, it is determined to be a solder ball deviation. The solder ball deviation ratio calculation formula is: ,in, For the solder ball deviation ratio, This refers to the number of pixels where the solder ball extends beyond the pad area. This represents the total number of pixels on the solder balls.

[0018] Furthermore, in the output results, during clustering and labeling, spatially adjacent differences are clustered into independent difference regions. Each difference region corresponds to a chip layout change type, and each difference region is marked to clearly indicate the specific location and range of each difference region.

[0019] Furthermore, the generated visualization images in the output include homography change images, effective detection range images, solder ball exceeding the pad area images, and bonding type annotation images. Through image color differentiation and labeling, the difference areas, solder ball deviation parts, and bonding type identification results are presented.

[0020] Compared with existing technologies, this method for comparing front and rear chip image data based on feature matching technology has the following advantages:

[0021] I. This invention constructs an improved SuperPoint model and introduces feature descriptor normalization processing. It combines a joint matching model to screen effective keypoint pairs, utilizes a homography matrix to establish the projection transformation relationship between the source and target images, and then achieves uniform sampling and feature comparison across the entire region through gridded difference detection. This process standardizes feature extraction criteria, enhances matching stability, ensures comprehensive sampling, and clarifies the specific types and ranges of difference regions through clustering annotation. This achieves precise localization and information quantification of chip layout differences, ensuring that all subtle changes can be effectively identified, thus providing a basis for chip layout... Figure 1 Consistency testing provides a reliable technical approach.

[0022] II. This invention optimizes segmentation by embedding a BoTNet module into the YOLOv8-seg instance segmentation model, accurately acquiring mask data of pads and solder balls and statistically analyzing relevant pixel information. Simultaneously, a bonding type recognition model is constructed based on the YOLOv8 framework to independently identify and screen bonding images from different batches, and then determines changes through batch-to-batch type comparison. This design, combined with a visual image generation design, presents the detection results in an intuitive form, achieving integrated and precise detection of solder ball deviation and bonding type changes, forming a multi-dimensional detection closed loop. This comprehensively covers batch differences in key chip components, facilitating rapid verification and traceability by operators, and providing comprehensive data support for chip production quality control.

[0023] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0025] Figure 1 A flowchart of a method for comparing front- and rear chip image data based on feature matching technology;

[0026] Figure 2 This is a framework diagram of a method for comparing front and rear chip image data based on feature matching technology.

[0027] Figure 3 This diagram illustrates the steps involved in the detection process of a method for comparing front- and back-end chip image data based on feature matching technology. Detailed Implementation

[0028] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0029] Example 1:

[0030] In the scenario of batch quality inspection of consumer electronics chips, graphic data comparison is performed between previous and current production batches of the same chip model to identify structural differences that may be caused by process fine-tuning. In a testing laboratory environment, industrial-grade imaging equipment is used to collect inspection images of two batches of unpacked chips, covering the core layout area and edge bonding area of ​​the chip. The focus is on collecting images of bonding balls and bonding images of ball bonds, wedge bonds, and crescent bonds. During the acquisition process, uniform lighting is ensured to avoid image distortion caused by reflections and shadows. Image annotation tools are used to accurately select and annotate the pad boundaries and bonding ball areas in each image, and the bonding types are classified and labeled. The training set and test set are divided proportionally to ensure that the dataset covers chip samples from different production periods, providing comprehensive support for the subsequent training of various models.

[0031] An improved SuperPoint model is built based on a deep learning framework. A first convolutional layer, a second convolutional layer, and a linear rectified activation function are sequentially configured. An arbitrary coordinate descriptor extraction function is added to adapt to the complex feature distribution of the chip layout. Feature descriptor convolutional layers are configured to generate feature descriptors. Non-maximum suppression radius and descriptor dimension parameters are set. During the descriptor output stage, the extracted feature descriptors are normalized using the L2 normalization formula to ensure the consistency of the feature vectors. The L2 normalization formula is as follows: ,in, This is the normalized feature descriptor vector. This is the original feature descriptor vector, generated by the feature descriptor convolutional layer, containing keypoint feature information. The L2 norm of the original feature descriptor vector is obtained by calculating the square root of the sum of squares of all elements. This represents the feature descriptor dimension, with a value of 256. The first element of the original feature descriptor vector Each element; the input is a labeled chip layout image training set, and the parameters are iterated to complete the construction and optimization of the SuperPoint model, such as... Figure 1 As shown;

[0032] The improved SuperPoint model is integrated with the SuperGlue module to build a joint matching model. The input image data format and feature matching threshold are defined, and the training set of paired layout images is used to complete joint training and optimize the adaptation parameters. A BoTNet module is inserted between the backbone network and the detection head of the YOLOv8-seg instance segmentation model. This BoTNet module contains three convolutional units and a sequence structure composed of bottleneck transformers. The training set of labeled pad and solder ball images is used as input, and the model weights are adjusted through backpropagation to complete the optimization. A bonding type recognition model is built based on the YOLOv8 framework. A network structure including feature extraction, feature fusion and detection head is built. The bonding image training set is used to complete the training and the confidence threshold is set.

[0033] The layout images of two batches of chips are converted into tensors of a specified dimension. These tensors are then input into the trained improved SuperPoint model to extract feature subsets containing chip layout texture and key structural features. The feature subsets from the previous and current batches are input into a joint matching model, outputting keypoint pairs and their corresponding confidence scores. The output valid keypoint pairs are converted into numerical arrays. A random sampling consensus algorithm is used, and a reprojection error threshold is set. A 3×3 homography matrix and its corresponding mask are obtained through iterative calculation. The homography transformation projection formula is then used to establish a precise projection transformation relationship between the two batches of layout images. The homography transformation projection formula is as follows: ,in, , These are the projected coordinates of key points in the target image, obtained from the coordinates of key points in the source image through matrix transformation. The matrix is ​​a 3×3 homography matrix, obtained by iteratively calculating valid keypoint pairs using a random sampling consensus algorithm. , Here are the original coordinates of the keypoints in the source image, and here are the valid keypoint pairs output by the joint matching model. The elements of the homography matrix, The value ranges from 0 to 2, with a total of 9 values. These values ​​are determined iteratively by optimizing the projection error of key point pairs. At the same time, a mask is used to filter out valid interior points and unreasonable exterior points caused by slight image contamination or noise interference.

[0034] Based on the obtained homography matrix and the filtered effective interior points, the extreme coordinates of the effective matching region of the two batches of layout images are determined. A uniform grid is generated by setting grid control parameters to ensure coverage of the core functional area of ​​the chip and surrounding key structures. The grid points are projected onto the corresponding positions of the current batch of images using the homography matrix. Feature descriptors at the grid points of the two batches of images are extracted respectively. The similarity distance between the corresponding grid point descriptors of the two images is calculated using the descriptor similarity measurement formula to quantify the feature differences. A threshold is set, and significantly different points exceeding the threshold are filtered out. The descriptor similarity measurement formula is as follows: ,in, This represents the similarity distance between the descriptors of the source and target images. A larger value indicates a more significant difference in features. The normalized feature descriptor for the grid points of the source image is obtained by extracting the improved SuperPoint model and then performing L2 normalization. The normalized feature descriptor for the grid points corresponding to the target image is obtained by projecting the grid points from the source image and then extracting and normalizing them using the improved SuperPoint model. This represents the feature descriptor dimension, with a value of 256. These are the normalized feature descriptors of the source image grid points and the first... Each element; the bonding area images of the two batches of chips are input into the optimized YOLOv8-seg instance segmentation model to obtain the pad and solder ball mask data. The pad mask is constructed from the pad data output by the model, and the solder ball mask is constructed from the solder ball data output by the model. The total number of effective pixels in the solder ball mask is counted to obtain the total number of solder ball pixels. By processing the solder ball mask and the pad mask, the number of effective pixels exceeding the pad area is counted. Then, the solder ball deviation ratio is calculated using the solder ball deviation ratio calculation formula. The solder ball deviation ratio calculation formula is: ,in, For the solder ball deviation ratio, This refers to the number of pixels where the solder ball extends beyond the pad area. The total number of pixels of the solder balls is used to determine whether the solder balls are deviated from the set standard. When the deviation exceeds 50%, it is determined that the solder balls are deviated. The bonding images of the two batches are input into the bonding type recognition model, and the bonding type and confidence level of each batch are output. The recognition results and confidence levels of the two batches are compared to see if they meet the set conditions, and it is determined whether the bonding type has changed.

[0035] This process summarizes data on significant differences, solder ball deviation ratios and judgment results, and bonding type identification and change judgment conclusions. Significant differences are clustered and labeled, grouping spatially adjacent differences into independent difference regions. The specific location, range, and corresponding chip layout change type of each difference region are clearly marked, such as missing resistor strips or changes in marking positions. Images of homography changes, effective detection range, solder balls extending beyond the pad area, and bonding type annotations are generated. Different colors are used to label the difference regions, solder ball deviations, and bonding type identification results. This information is integrated into a detailed inspection report, providing quality control departments with intuitive and accurate judgment criteria to ensure that non-conforming batches of chips do not enter the market.

[0036] In summary, for the quality inspection scenario of consumer electronics chips, images of the layout and bonding areas of two batches of chips were acquired through image acquisition in a laboratory environment, and then labeled and divided into datasets. Based on a deep learning framework, an improved SuperPoint model, a joint matching model, an optimized YOLOv8-seg instance segmentation model, and a bonding type recognition model were constructed. The projection transformation relationship between the two batches of images was established through feature matching. Then, through gridded difference detection, solder ball deviation ratio calculation, and bonding type comparison, a full-dimensional detection was completed. Finally, a test report was generated through clustering labeling and visualization, which accurately identified structural differences caused by process fine-tuning and provided reliable support for chip manufacturing quality control.

[0037] Example 2:

[0038] In the scenario of consistency verification during process switching of industrial control chips, graphical data comparison is performed on batches of the same chip from the old production line and batches from the new production line to confirm that the chip structure has not undergone unexpected changes after the replacement of process equipment and adjustment of processes. Inspection images of two batches of unpacked chips are collected at a dedicated inspection station in the industrial production workshop. This station is equipped with an anti-interference imaging system, which effectively avoids the impact of workshop vibration and electromagnetic interference on image acquisition. The acquisition range covers the chip logic layout area and power interface bonding area, focusing on collecting images of bonding balls and ball bonds, wedge bonds, and crescent bonds. High-precision image annotation tools are used to accurately annotate the contours of pads and solder balls, classify and annotate bonding types and bonding wire directions, and divide the training set and test set proportionally. Figure 2 As shown, this ensures that the training set contains typical samples of various types from two batches of chips, fully covering the chip characteristics under different production equipment and process parameters.

[0039] An improved SuperPoint model is built based on a deep learning framework. A first convolutional layer, a second convolutional layer, and a linear rectified activation function are configured sequentially. An arbitrary coordinate descriptor extraction function is added to adapt to the extraction requirements of dense features in industrial chip layouts. A feature descriptor convolutional layer is configured to generate feature descriptors. Non-maximum suppression radius and descriptor dimension parameters are set. In the descriptor output stage, the extracted feature descriptors are normalized using the L2 normalization formula. The layout annotation images in the training set are input to complete parameter iterative optimization. The L2 normalization formula for the feature descriptors is: The improved SuperPoint model was used as a feature extraction module and integrated with the SuperGlue module to build a joint matching model. The input image data format was defined, and paired layout training images were input to complete joint training, optimizing the adaptation parameters. The BoTNet module was embedded in the YOLOv8-seg instance segmentation model. The number of input and output channels and the hidden channel expansion coefficient of the BoTNet module were defined. The labeled pads and solder balls training set were input, and training-related parameters were set. The weights were adjusted through backpropagation, and the segmentation accuracy was verified using the test set before optimization. A bonding type recognition model was built based on the YOLOv8 framework. The anchor box parameters and number of categories adapted to the bonding features of industrial chips were configured. The bonding image training set was input to complete training, and the confidence threshold was set based on the accuracy of the validation set.

[0040] The layout images of two batches of chips are converted into tensors of a specified dimension and input into the trained improved SuperPoint model to extract feature subsets containing key information such as chip logic gate structure and wiring texture. The feature subsets of the old and new production line batches are input into a joint matching model, which outputs keypoint pairs and confidence data. The valid keypoint pairs are converted into numerical arrays, and the homography matrix solving logic is called. A 3×3 homography matrix and corresponding mask are obtained through iterative calculation using a random sampling consensus algorithm. Valid interior points conforming to the projection transformation relationship are selected, and mismatched exterior points caused by differences in production line equipment are filtered out. Finally, a precise mapping relationship between the two batches of images is established using the homography transformation projection formula, laying the foundation for subsequent difference detection. The homography transformation projection formula is as follows: .

[0041] Based on the homography matrix and effective interior points, such as Figure 3 As shown, the extreme coordinates of the effective matching area of ​​the two batches of layouts are determined, and grid control parameters are set to generate a uniform grid to ensure full coverage of key areas such as the chip logic core and power interface. The grid points are projected onto the corresponding positions of the new production line batch images, and feature descriptors at the grid points of the two batch images are extracted respectively. The similarity distance is calculated using the descriptor similarity measurement formula to quantify the feature differences. A threshold is set to filter out significantly different points with differences greater than the threshold. The descriptor similarity measurement formula is as follows: The two batches of bonding region images are input into the optimized YOLOv8-seg instance segmentation model to obtain the mask data of the pads and solder balls. The total number of pixels of the solder balls and the number of pixels exceeding the pad area are counted. The solder ball deviation ratio is calculated using the solder ball deviation ratio calculation formula, which is as follows: Combine quality standards to determine whether the solder balls deviate. When the deviation exceeds 50%, it is determined that the solder balls deviate. Input the bonding images of the two batches into the bonding type recognition model respectively, and output their respective bonding types and confidence levels. If the two batches are inconsistent in type and the confidence levels are both up to standard, it is determined that the bonding type has changed, and the bonding process parameters of the new production line need to be checked.

[0042] The system summarizes data on significant differences, solder ball deviation ratios and judgment results, and bond type change judgment conclusions. Difference points are clustered and labeled to clearly identify the specific location and range of changes such as missing resistor strips and altered bond wire orientation. Images of homography changes, effective detection range, solder balls extending beyond the pad area, and bond type labels are generated. The test results are presented intuitively through color differentiation and labeling. A detailed test report is then compiled and provided to the production line's process department as a core reference for adjusting new production line parameters and optimizing processes. This ensures that subsequent chip production maintains structural consistency with older batches, meeting the high reliability requirements of industrial control scenarios.

[0043] In summary, anti-interference images were acquired at a dedicated testing station in the industrial workshop, covering the chip logic layout and power interface bonding areas, and labeled and divided into datasets. By building and optimizing the SuperPoint model, joint matching model, YOLOv8-seg instance segmentation model with embedded BoTNet module, and bonding type recognition model, a precise mapping relationship was established through feature matching. Combined with gridded difference detection, solder ball deviation evaluation, and bonding type change judgment, the consistency of chip structure after process switching was fully verified. The generated visual inspection report provides core reference for parameter adjustment and process optimization of the new production line, ensuring the high reliability of chips in industrial control scenarios.

[0044] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for comparing front and rear chip image data based on feature matching technology, characterized in that, The specific steps of this method are as follows: Data acquisition: Collect unpacking inspection images of different batches of the same model of chip, covering chip layout images, bonding area images, and collect bonding solder ball images and ball bond, wedge bond, and crescent bond type images. Label the pads, solder ball areas and bonding types, and divide them into training set and test set. Model construction: Based on the collected labeled images and the segmented training and test sets, an improved SuperPoint model is constructed. A joint matching model is built by integrating the improved SuperPoint model with the SuperGlue module. The BoTNet module is embedded in the YOLOv8-seg instance segmentation model to complete the optimization. A bonding type recognition model is built based on the YOLOv8 framework. The parameters of each model are iteratively optimized by inputting the training set. Feature matching: The acquired chip layout image is converted into a tensor of a specified dimension, which is then input into the trained improved SuperPoint model to extract a feature subset. The feature subset is then input into the joint matching model to output key point pairs and confidence scores. Based on the key point pairs, the homography matrix is ​​estimated and valid point pairs are filtered. Detection execution: Based on the homography matrix and the filtered valid point pairs, perform gridded difference detection on the chip layout. Input the bonding region image into the optimized YOLOv8-seg model to obtain pad and solder ball data and calculate the deviation ratio. Input the bonding images of the previous and next batches into the bonding type recognition model to output the type and confidence and determine whether there has been a change. Results output: Summarize the difference point data, solder ball deviation ratio and judgment results, bonding type identification results and change judgment conclusions, cluster and label the difference points, generate various visualization images, and integrate them into a test report.

2. The method for comparing front and rear chip image data based on feature matching technology according to claim 1, characterized in that, In the model construction, the improved SuperPoint model is constructed as follows: a deep learning framework is used to build the network structure; a first convolutional layer, a second convolutional layer, and a linear rectified activation function are set sequentially; an arbitrary coordinate descriptor extraction function is added to support descriptor extraction at arbitrary coordinates; a feature descriptor convolutional layer is configured to generate feature descriptors; non-maximum suppression radius and descriptor dimension parameters are set; in the descriptor output stage, the extracted feature descriptors are normalized using the feature descriptor L2 normalization formula; and the parameters are iterated using a labeled chip layout image training set to complete the construction and optimization. The L2 normalization formula for descriptors is: ,in, This is the normalized feature descriptor vector. This is the original feature descriptor vector. The L2 norm of the original feature descriptor vector. This represents the feature descriptor dimension, with a value of 256. The first element of the original feature descriptor vector Each element.

3. The method for comparing front and rear chip image data based on feature matching technology according to claim 1, characterized in that, In the model construction, the steps for building the joint matching model are as follows: initializing the configuration parameters of the joint matching model based on the deep learning framework, integrating the improved SuperPoint model as a feature extraction module into the joint matching model, loading the SuperGlue module as the feature matching core of the joint matching model, defining the input image data format, feature matching threshold and output parameter type of the joint matching model, setting the computing device for inference of the joint matching model, inputting a pair of chip layout image training sets to jointly train the joint matching model, optimizing the adaptation parameters between the improved SuperPoint model and the SuperGlue module, so that the joint matching model outputs inference results containing key points, matching indices and confidence scores.

4. The method for comparing front and rear chip image data based on feature matching technology according to claim 1, characterized in that, The optimization and construction steps of the YOLOv8-seg instance segmentation model in the model construction are as follows: A BoTNet module is inserted between the backbone network and the detection head of the YOLOv8-seg instance segmentation model. The BoTNet module is configured with three convolutional units and a sequence structure composed of bottleneck transformers. The number of input and output channels, hidden channel expansion coefficients, number of bottleneck transformer heads, and resolution parameters of the BoTNet module are defined. The labeled pad and solder ball image training set is input into the YOLOv8-seg instance segmentation model, and the training batch size, iteration rounds, and learning rate are set. The weights of the YOLOv8-seg instance segmentation model are adjusted through backpropagation. The segmentation accuracy of the YOLOv8-seg instance segmentation model is verified using a test set. After iterative optimization, the construction of the YOLOv8-seg instance segmentation model is completed.

5. The method for comparing front and rear chip image data based on feature matching technology according to claim 1, characterized in that, In the model construction, the bonding type recognition model is built and trained as follows: Bonding images of three types—spherical, wedge, and crescent—are collected; image annotation tools are used to label the bonding regions and types in the images; the training and testing sets are divided in an 8:2 ratio; based on the YOLOv8 framework, the input image size, anchor box parameters, and number of categories for the bonding type recognition model are configured, and a network structure including feature extraction, feature fusion, and a detection head is built; the training set is used to train the bonding type recognition model; the learning rate and regularization parameters are adjusted based on the recognition accuracy on the validation set, and a confidence threshold is set; the model is then tested... The test set verifies the generalization ability of the bonding type recognition model, and the final weights are selected to complete the construction of the bonding type recognition model. When the trained bonding type recognition model is used, the bonding region images of the previous batch of chips and the bonding region images of the subsequent batch of chips are respectively input into the trained bonding type recognition model. The model outputs the bonding type and confidence level of the two batches of images respectively. If the bonding types of the two batches are inconsistent and the confidence levels of both are not lower than the set confidence level threshold, it is determined that the bonding type has changed. If the bonding types of the two batches are consistent, or the confidence level of either batch is lower than the confidence level threshold, it is determined that the bonding type has not changed effectively.

6. The method for comparing front and rear chip image data based on feature matching technology according to claim 1, characterized in that, In the feature matching process, the homography matrix estimation process is as follows: The effective keypoint pairs output by the joint matching model are converted into numerical array format; the reprojection error threshold of the random sampling consensus algorithm is set; a 3×3 homography matrix and its corresponding mask are obtained through iterative calculation; the mask is used to filter out effective interior points in the mapping structure and unreasonable exterior points; and the projection transformation relationship between the source image and the target image is established using the homography transformation projection formula. The homography transformation projection formula is: ,in, , These are the projected coordinates of key points in the target image. It is a 3×3 homography matrix. , These are the original coordinates of the key points in the source image. The elements of the homography matrix, The value can range from 0 to 2.

7. The method for comparing front and rear chip image data based on feature matching technology according to claim 1, characterized in that, In the detection process, the meshed difference detection operation is as follows: effective inliers are projected onto the homography matrix to form corresponding point pairs between the source image and the target image. The extreme coordinates of the effective matching region are obtained according to the coordinate range of the corresponding point pairs. Mesh control parameters are set to control the mesh density and sampling interval. Uniformly distributed x and y coordinates are generated within the extreme coordinate range to form a uniform mesh covering the effective matching region. Based on the homography matrix, grid points are projected onto corresponding positions in the target image, and feature descriptors at grid points in the source image and projected grid points in the target image are extracted respectively. The similarity distance between corresponding grid point descriptors in an image is calculated using a descriptor similarity metric formula. This quantifies the feature differences, and a threshold is set to filter out the difference points that exceed the threshold. The formula for measuring descriptor similarity is: ,in, The similarity distance between the corresponding descriptors of the source image and the target image. This is a normalized feature descriptor for the grid points of the source image. This refers to the normalized feature descriptor for the grid points corresponding to the target image. For feature descriptor dimension, These are the normalized feature descriptors of the source image grid points and the first... Each element.

8. The method for comparing front and rear chip image data based on feature matching technology according to claim 1, characterized in that, During the detection process, the calculation steps for the solder ball deviation ratio are as follows: The total number of valid pixels in the solder ball mask output by the YOLOv8-seg instance segmentation model is counted to obtain the total number of solder ball pixels; the solder ball mask and the pad mask are then processed. The solder ball mask is constructed from the solder ball data output by the YOLOv8-seg instance segmentation model, and the pad mask is constructed from the pad data output by the YOLOv8-seg instance segmentation model. The total number of valid pixels in the processed result is counted to obtain the number of pixels exceeding the pad area. Then, the solder ball deviation ratio is calculated using the solder ball deviation ratio calculation formula. When it exceeds 50%, it is determined to be a solder ball deviation. The formula for calculating the solder ball deviation ratio is: ,in, For the solder ball deviation ratio, This refers to the number of pixels where the solder ball extends beyond the pad area. This represents the total number of pixels on the solder balls.

9. The method for comparing front and rear chip image data based on feature matching technology according to claim 1, characterized in that, In the output results, during clustering and labeling, spatially adjacent differences are clustered into independent difference regions. Each difference region corresponds to a chip layout change type, and each difference region is marked to clearly indicate the specific location and range of each difference region.

10. The method for comparing front and rear chip image data based on feature matching technology according to claim 1, characterized in that, The output results include various visualization images such as homography change images, effective detection range images, solder ball exceeding the pad area images, and bonding type annotation images. By distinguishing and marking the images with colors, the results of identifying the difference areas, the solder ball deviation parts, and the bonding type are presented.

Images