A SAM model training method and device based on strip convolution and differential enhancement, and a semiconductor gold wire segmentation method and device based on the SAM model

Abstract

The application provides a SAM model training method and device based on strip convolution and differential enhancement, and a semiconductor gold wire segmentation method and device based on the SAM model. The training method comprises the following steps: constructing a chip gold wire segmentation data set; training the SAM model by using the data set, optimizing the segmentation loss of the gold wire region, and stopping until a preset training stop condition is reached; an improved low-rank adaptive block is arranged on each Transform block of an image encoder of the SAM model; in the model training process, the image encoder of the SAM model is frozen, and the parameters of the mask decoder and the low-rank adaptive block of the SAM model are fine-tuned. The application realizes explicit enhancement and background suppression of the semiconductor gold wire feature under the parameter efficient fine-tuning framework, and improves the segmentation precision.

Description

Technical Field

[0001] This application relates to the field of industrial inspection technology, specifically to a SAM model training method and apparatus based on strip convolution and differential enhancement, and a semiconductor gold wire segmentation method and apparatus based on the SAM model. Background Technology

[0002] In semiconductor integrated circuit packaging processes, the electrical interconnection between the chip and the packaging substrate or lead frame is typically achieved through gold wire bonding. Gold wires, as a critical interconnect material on the chip, are typically only 15–50 μm in diameter and connect chip pads and leads in an arc-shaped trajectory. Their morphological integrity determines the device's conductivity, signal transmission quality, and long-term reliability. However, during high-speed automated bonding processes, gold wires on the chip are prone to defects such as wire collapse, breakage, misalignment, abnormal curvature, and neck damage. Traditional manual visual inspection is inefficient and inconsistent, making it difficult to meet the real-time inspection requirements of production lines.

[0003] Deep learning-based image segmentation technology provides an effective approach for the automatic detection of gold wire defects on chips. Among them, the Segment Anything Model (SAM), as a fundamental large-scale vision model, has powerful zero-shot segmentation capabilities. However, its original design is geared towards general natural scenes, and it has significant limitations when directly applied to industrial micro-scale linear targets such as gold wires on chips. Gold wires on chips are characterized by extreme thinness, low contrast, and complex backgrounds (circuit textures, pad reflections, and molding compound interference on the chip surface). The general feature extraction of the SAM model is difficult to fully capture its directional edge information, and the computational cost of full parameter fine-tuning is large and prone to overfitting, which is not conducive to deployment in edge devices on semiconductor production lines.

[0004] Parameter-Efficient Fine-Tuning (PEFT) techniques achieve domain adaptation by injecting a small number of trainable parameters into a pre-trained model. Among these, Low-Rank Adaptation (LoRA) is widely used due to its advantages such as low training cost and zero inference latency. However, standard LoRA uses a fully connected low-rank decomposition and lacks the ability to explicitly model spatial structure and directional features. For microscale linear targets with significant directionality, such as gold wires on a chip, its feature enhancement effect is limited.

[0005] Furthermore, chip surfaces often contain interference information such as uniform illumination, substrate circuit textures, and pad reflections, which can easily lead to background noise being misidentified as foreground when directly segmented. Differential filtering can effectively highlight the contrast between linear targets and the background by suppressing low-frequency DC components and enhancing high-frequency edge responses, but existing LoRA frameworks have not yet organically integrated the differential mechanism with directional spatial modeling.

[0006] Therefore, there is an urgent need for a lightweight SAM fine-tuning segmentation method that is suitable for chip gold wire detection scenarios and has both direction-selective feature extraction and differential background suppression capabilities, so as to achieve high-precision and high-efficiency segmentation of gold wires on chips under the limited computing power of semiconductor production lines. Summary of the Invention

[0007] In view of this, this application proposes a SAM model training method and apparatus based on strip convolution and differential enhancement, and a semiconductor gold wire segmentation method and apparatus based on the SAM model, so as to achieve explicit enhancement of semiconductor gold wire features and background suppression under the framework of efficient parameter fine-tuning, thereby improving segmentation accuracy.

[0008] Specifically, this application is implemented through the following technical solution:

[0009] According to a first aspect of the embodiments of this specification, a method for training a SAM model based on strip convolution and differential augmentation is provided, the method comprising the following steps:

[0010] Step S11: Construct a chip gold wire segmentation dataset, which includes chip bonding region images and their corresponding pixel-level annotations of the gold wire regions.

[0011] Step S12: Train the SAM model using the dataset, optimizing the segmentation loss of the gold line region until a preset training stopping condition is reached; each Transformer block of the SAM model's image encoder is equipped with an improved low-rank adaptation block. During model training, the SAM model's image encoder is frozen, and the parameters of the SAM model's mask decoder and low-rank adaptation blocks are fine-tuned; wherein, the improved low-rank adaptation block is configured as follows:

[0012] The input features are linearly reduced in dimensionality using a learnable first low-rank decomposition matrix. The reduced features are then reshaped from a token sequence into a feature map. Spatial correlation calculations are performed with multiple learnable preset direction prototypes to generate directional weights for each preset direction. The multiple preset direction prototypes correspond to different extension directions of the gold wire in the chip bonding region image.

[0013] The reshaped feature map is subjected to depthwise separable convolution operation by strip convolution kernels corresponding to each preset direction. The convolution result is then multiplied by the directional weights of the corresponding directions to obtain the strip convolution feature map corresponding to each preset direction.

[0014] Based on the reshaped feature map, the sum of weights in all directions, and the learnable parameters, differential enhancement is performed on each strip convolutional feature map to obtain the enhanced strip convolutional feature map, and a fused feature map is obtained based on the enhanced strip convolutional feature map.

[0015] After flattening the fused feature map into a sequence, a linear transformation is performed using a learnable second low-rank decomposition matrix to obtain a low-rank increment matrix.

[0016] The low-rank increment matrix is ​​superimposed onto the attention weights of the Transformer block corresponding to the current improved low-rank adaptor block to complete the parameter fine-tuning.

[0017] According to a second aspect of the embodiments of this specification, a semiconductor gold wire segmentation method based on the SAM model is provided, comprising the following steps:

[0018] Step S21: Obtain an image of the chip bonding area to be detected;

[0019] Step S22: Input the image of the chip bonding region to be detected into the trained SAM model, and output the pixel-level segmentation mask of the gold line region through the SAM model;

[0020] The SAM model is trained using the method described in the first aspect.

[0021] According to a third aspect of the embodiments of this specification, a SAM model training apparatus based on strip convolution and difference enhancement is provided, comprising:

[0022] A dataset construction unit is used to construct a chip gold wire segmentation dataset, which includes chip bonding region images and their corresponding pixel-level annotations of the gold wire regions.

[0023] The model training unit is used to train the SAM model using the dataset, optimizing the segmentation loss of the gold line region until a preset training stopping condition is reached. Each Transformer block of the SAM model's image encoder is equipped with an improved low-rank adaptation block. During model training, the SAM model's image encoder is frozen, and the parameters of the SAM model's mask decoder and low-rank adaptation blocks are fine-tuned. The improved low-rank adaptation block is configured as follows:

[0024] The input features are linearly reduced in dimensionality using a learnable first low-rank decomposition matrix. The reduced features are then reshaped from a token sequence into a feature map. Spatial correlation calculations are performed with multiple learnable preset direction prototypes to generate directional weights for each preset direction. The multiple preset direction prototypes correspond to different extension directions of the gold wire in the chip bonding region image.

[0025] The reshaped feature map is subjected to depthwise separable convolution operation by strip convolution kernels corresponding to each preset direction. The convolution result is then multiplied by the directional weights of the corresponding directions to obtain the strip convolution feature map corresponding to each preset direction.

[0026] Based on the reshaped feature map, the sum of weights in all directions, and the learnable parameters, differential enhancement is performed on each strip convolutional feature map to obtain the enhanced strip convolutional feature map, and a fused feature map is obtained based on the enhanced strip convolutional feature map.

[0027] After flattening the fused feature map into a sequence, a linear transformation is performed using a learnable second low-rank decomposition matrix to obtain a low-rank increment matrix.

[0028] The low-rank increment matrix is ​​superimposed onto the attention weights of the Transformer block corresponding to the current improved low-rank adaptor block to complete the parameter fine-tuning.

[0029] According to a fourth aspect of the embodiments of this specification, a semiconductor gold wire dicing apparatus based on a SAM model is provided, comprising:

[0030] Image acquisition unit, used to acquire images of the bonding region of the chip to be detected;

[0031] The image segmentation unit is used to input the image of the chip bonding region to be detected into the trained SAM model, and output the pixel-level segmentation mask of the gold line region through the SAM model;

[0032] The SAM model is trained using the method described in the first aspect.

[0033] According to a fifth aspect of the embodiments of this specification, an electronic device is provided, comprising: a processor; and a computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in the first aspect.

[0034] The embodiments of this application have at least the following technical effects:

[0035] First, the embodiments of this application introduce learnable direction prototypes corresponding to different extension directions of gold wire (horizontal, vertical, diagonal, etc.) in the improved low-rank adaptation block, and use strip convolution kernels of the corresponding directions to perform depth-separable convolution, so that the model can selectively extract the slender and continuous edge features of gold wire in various directions, which significantly improves the ability to perceive micro-scale linear targets and overcomes the defect of standard LoRA lacking spatial orientation modeling.

[0036] Second, in this embodiment, the strip convolution feature map is differentially enhanced by the reshaped feature map, the sum of weights in all directions, and learnable parameters. This can adaptively remove low-frequency background interference such as uniform illumination on the chip surface and substrate circuit texture, while highlighting the high-frequency response of the gold line edge. Under low contrast conditions, this effectively enhances the distinction between the gold line and the background, and improves the recall and accuracy of the segmentation.

[0037] Third, by training only the improved low-rank adapter block and mask decoder and freezing the SAM image encoder backbone network, the number of parameters is much smaller than that of full parameter fine-tuning, which significantly reduces memory usage and training time, making it suitable for deployment on devices with limited computing power at the edge of semiconductor production lines.

[0038] Fourth, after spatial normalization, the weights of the multi-directional directions are weighted and fused to enhance the strip convolution feature map, enabling the SAM model to adapt to various complex shapes such as arcs, tilts, and neck bends of semiconductor gold wires, and to have a stronger generalization ability for defect types such as gold wire collapse, offset, and breakage.

[0039] Fifth, the SAM model only needs one forward propagation during the inference stage to output a pixel-level gold line segmentation mask, without the need for additional post-processing or multiple iterations, resulting in fast detection speed and meeting the real-time detection requirements of the production line. At the same time, it inherits the directional selectivity and differential enhancement capabilities from the training stage, and can still maintain high segmentation accuracy for narrow, low-contrast semiconductor gold lines. Attached Figure Description

[0040] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Some specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings in an exemplary and non-limiting manner. The same reference numerals in the drawings indicate the same or similar parts or components. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0041] Figure 1 This is a schematic flowchart illustrating an exemplary embodiment of the present application of a SAM model training method based on strip convolution and difference enhancement;

[0042] Figure 2 This is a schematic diagram illustrating the framework result of a SAM model according to an exemplary embodiment of this application;

[0043] Figure 3 This is a schematic flowchart illustrating a semiconductor gold wire segmentation method based on the SAM model, as shown in an exemplary embodiment of this application.

[0044] Figure 4 This is a schematic diagram illustrating the segmentation result of a semiconductor gold wire according to an exemplary embodiment of this application;

[0045] Figure 5 This is a schematic diagram showing the segmentation and comparison of a semiconductor gold wire according to an exemplary embodiment of this application;

[0046] Figure 6 This is a block diagram illustrating an electronic device according to an exemplary embodiment of this application;

[0047] Figure 7 This is a block diagram illustrating a SAM model training device based on strip convolution and difference enhancement, as shown in an exemplary embodiment of this application.

[0048] Figure 8 This is a block diagram illustrating a semiconductor gold wire splitting device based on the SAM model, as shown in an exemplary embodiment of this application. Detailed Implementation

[0049] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0050] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0051] To address the technical problems in existing technologies, such as the difficulty of SAM models in effectively segmenting micro-scale directional linear targets like gold wires on chips, the lack of spatial orientation modeling capabilities in standard LoRA, and insufficient segmentation accuracy due to industrial background interference, this application provides a training scheme for SAM models and a semiconductor gold wire segmentation scheme for chip gold wire detection. Through strip convolution and differential enhancement mechanisms, explicit enhancement of semiconductor gold wire features and background suppression are achieved within a framework of efficient parameter fine-tuning.

[0052] The embodiments described in this specification will now be described in detail.

[0053] This application provides a SAM model training method based on strip convolution and differential enhancement. This training method can be executed by a detection system, which can be deployed on an electronic device or in the cloud.

[0054] Figure 1 This is a schematic flowchart illustrating an exemplary embodiment of the present application of a SAM model training method based on striped convolution and difference enhancement, as shown below. Figure 1 As shown, the SAM model training method includes the following steps:

[0055] Step S11: Construct a chip gold wire segmentation dataset, which includes chip bonding region images and their corresponding pixel-level annotations of the gold wire regions.

[0056] This embodiment uses an industrial camera to acquire images of the chip after gold wire bonding. The acquired images are grayscale or color RGB images, and each image contains one or more chip bonding areas, with semiconductor gold wires connecting the chip pads and lead frame pins. After denoising and adaptive contrast enhancement, the acquired raw images are uniformly cropped or scaled to a fixed size to meet the input requirements of the SAM model. In cases where multiple chips are present in the image, individual chip bonding areas are extracted as independent samples through target detection or manual selection.

[0057] The gold line regions in the preprocessed image are labeled pixel by pixel. After the labeling is completed, a binary mask image corresponding to the original image is generated, in which the foreground pixel value is 1 (or 255) and the background pixel value is 0.

[0058] In some embodiments, in order to expand the dataset size and improve the generalization ability of the model, online or offline data augmentation is performed on the original image and its corresponding labeled mask to obtain a chip gold line segmentation dataset.

[0059] Step S12: Train the SAM model using the dataset, optimizing the segmentation loss of the gold line region until a preset training stopping condition is reached; each Transformer block of the SAM model's image encoder is equipped with an improved low-rank adaptation block. During model training, the SAM model's image encoder is frozen, and the parameters of the SAM model's mask decoder and low-rank adaptation blocks are fine-tuned; wherein, the improved low-rank adaptation block is configured as follows:

[0060] The input features are linearly reduced in dimensionality using a learnable first low-rank decomposition matrix. The reduced features are then reshaped from a token sequence into a feature map. Spatial correlation calculations are performed with multiple learnable preset direction prototypes to generate directional weights for each preset direction. The multiple preset direction prototypes correspond to different extension directions of the gold wire in the chip bonding region image.

[0061] The reshaped feature map is subjected to depthwise separable convolution operation by strip convolution kernels corresponding to each preset direction. The convolution result is then multiplied by the directional weights of the corresponding directions to obtain the strip convolution feature map corresponding to each preset direction.

[0062] Based on the reshaped feature map, the sum of weights in all directions, and the learnable parameters, differential enhancement is performed on each strip convolutional feature map to obtain the enhanced strip convolutional feature map, and a fused feature map is obtained based on the enhanced strip convolutional feature map.

[0063] After flattening the fused feature map into a sequence, a linear transformation is performed using a learnable second low-rank decomposition matrix to obtain a low-rank increment matrix.

[0064] The low-rank increment matrix is ​​superimposed onto the attention weights of the Transformer block corresponding to the current improved low-rank adaptor block to complete the parameter fine-tuning.

[0065] Specifically, in combination Figure 2 As shown, the input image of the SAM model is processed by the embedding layer, then encoded by the image encoder, and finally decoded by the mask decoder under the guidance of the mask embedding features, outputting the segmentation result performed by the semiconductor. Those skilled in the art can refer to relevant technical solutions for the model structure, such as the embedding layer and mask decoder; these details will not be elaborated upon in this embodiment.

[0066] During model training, the pre-trained SAM model is loaded, all parameters of the image encoder are frozen, low-rank adaptation blocks (i.e., LoRA adaptation blocks) are inserted into the attention layer of each Transformer block of the image encoder, and only the parameters of the mask decoder and LoRA adaptation blocks are trained.

[0067] Standard LoRA adapter block for the original weight matrix The update format is as follows:

[0068] (1)

[0069] in, It is the first learnable low-rank decomposition matrix. The learnable second low-rank decomposition matrix , This represents the output feature dimension of the Transformer block.

[0070] This embodiment improves the first low-rank decomposition matrix side, enabling the low-rank adaptation to have direction-aware capabilities.

[0071] Continue to refer to Figure 2 The low-rank adaptation block first reshapes the token features output by the Transformer block from a sequence form (BS, L, C) into a feature map form (BS, r, H, W). Then, it performs a linear transformation using a second low-rank decomposition matrix to obtain a low-rank increment matrix. This low-rank increment matrix is ​​then superimposed onto the attention weights of the Transformer block corresponding to the current improved low-rank adaptation block. This allows for fine-tuning of the parameters. Here, BS represents the batch size, indicating the number of samples processed simultaneously in one forward propagation; L represents the sequence length corresponding to the token feature; and C represents the number of channels, indicating the feature channel dimension of each token.

[0072] Assume the token characteristics output by the current Transformer block are as follows: After the first low-rank decomposition matrix Dimensional reduction After the reshape operation, the feature map is obtained. , .

[0073] In some embodiments, based on the extension direction of the gold wire in the chip bonding area image, the plurality of preset directions include a horizontal direction, a vertical direction, a main diagonal direction, and an anti-diagonal direction.

[0074] Specifically, for the horizontal and vertical directions, the strip convolution kernels are one-dimensional horizontal strip convolution kernels and one-dimensional vertical strip convolution kernels, respectively; for the main diagonal and anti-diagonal directions, the strip convolution kernels are two-dimensional convolution kernels constrained by a mask matrix, wherein the mask matrix is ​​used to retain the weights corresponding to the diagonal region while suppressing the off-diagonal region.

[0075] This embodiment defines four learnable directional prototypes. These correspond to the horizontal, vertical, main diagonal, and anti-diagonal directions, respectively, and the weights for each direction are generated through spatial correlation calculations.

[0076] (2)

[0077] Here, ⊙ represents the element-wise multiplication operator. For the first The direction weights corresponding to each direction For the first The prototypes corresponding to each direction The reshaped feature map, , To preset the number of directions, here .

[0078] Each direction's weight is depthwise separable convolutionally performed with the corresponding direction's strip convolution kernel to obtain a strip convolution feature map:

[0079] (3)

[0080] in, For the first The strip convolutional feature maps corresponding to each direction, This is a depthwise separable convolution operator. For the first Each direction corresponds to a strip convolution kernel.

[0081] In some embodiments, the one-dimensional horizontal strip convolution kernel is a 1×k convolution kernel; the one-dimensional vertical strip convolution kernel is a k×1 convolution kernel; where k is a preset value, for example, k=16.

[0082] The two-dimensional convolution kernel is ; For the mask matrix, .

[0083] After obtaining the strip convolutional feature map, a differential enhancement mechanism is introduced to enhance the strip convolutional feature map. The expression for the enhanced strip convolutional feature map is as follows:

[0084] (4)

[0085] in, For the first The enhanced strip convolutional feature maps corresponding to each direction For the first The strip convolutional feature maps corresponding to each direction, For learnable parameters, The reshaped feature map, For the first The direction weights of each direction The preset number of directions.

[0086] In some embodiments, learnable parameters Constrained by the Sigmoid function:

[0087] (5)

[0088] in, As a learnable variable, it is guaranteed by the Sigmoid function. It always stays in the interval [0, 1].

[0089] when When the value approaches 1, the differential term dominates the output, enhancing the high-frequency edge response to highlight the gold line outline; conversely, when... As the gold lines approach zero, some low-frequency information is retained to maintain their continuity. This effectively suppresses the uniform background on the chip surface and enhances the contrast between the gold lines and the background.

[0090] The enhanced strip convolutional feature map Then, a weighted sum is performed on all the enhanced strip convolutional feature maps to obtain the initial fused feature map. :

[0091] (6)

[0092] The initial fused feature map is residually connected to the reshaped feature map to obtain the final fused feature map. :

[0093] (7)

[0094] in, The preset residual weighting coefficient is set for extremely fine gold wire. >0 Prevents loss of feature information.

[0095] Fuse feature maps After being flattened, it was restored to a sequence form. The output of the first low-rank decomposition matrix and the second low-rank decomposition matrix, which are used as the LoRA adapter block. Multiplication:

[0096] (8)

[0097] Generate low-rank increment matrix And superimposed on the original Q / V attention weights. This enables efficient fine-tuning of the parameters for orientation perception.

[0098] In some embodiments, the configured SAM model is trained end-to-end using a dataset, optimizing the segmentation loss for the gold line region. For example, a joint form of cross-entropy loss and Dice loss is used.

[0099] (9)

[0100] in, Mark the gold line with a mask. Use it as a prediction mask for the model.

[0101] This application provides a semiconductor gold wire segmentation method based on the SAM model. This semiconductor gold wire segmentation method can be executed by a detection system, which can be deployed on an electronic device or in the cloud.

[0102] Figure 3 This is a schematic flowchart illustrating an exemplary embodiment of a semiconductor gold wire segmentation method based on the SAM model, as shown in this application. Figure 3 As shown, the semiconductor gold wire splitting method includes the following steps:

[0103] Step S21: Obtain an image of the chip bonding area to be detected;

[0104] Step S22: Input the image of the chip bonding region to be detected into the trained SAM model, and output the pixel-level segmentation mask of the gold line region through the SAM model;

[0105] The training method for the SAM model can be referred to... Figure 1 The embodiments described herein will not be repeated here.

[0106] In this embodiment, the image of the chip bonding region to be detected is input into the trained SAM model, and the pixel-level segmentation mask of the gold wire region is output to realize the automatic detection and location of semiconductor gold wire defects on the chip.

[0107] refer to Figure 4 and Figure 5 , Figure 4 The image shows various original images and their corresponding segmentation mask results. Figure 5 This shows the original gold line image, the baseline ground truth, the model prediction results, and the overlap between the model prediction results and the original image. Combined with... Figure 4 and Figure 5 As can be seen, the SAM model in this embodiment has excellent segmentation performance.

[0108] Furthermore, this embodiment quantitatively compares the SAM model of this embodiment with other segmentation models in terms of Dice coefficient, Boundary Intersection over Union (BIOU), Intersection over Union (IOU), and Pixel Accuracy (Acc). The comparison results are shown in Table 1. "Ours" in Table 1 refers to the SAM model of this embodiment. Comparison models include SAM_lora, deeplabv3+, Unet, SAM_Adapter, etc. The model structures of various comparison models can be found in relevant technical solutions, and will not be elaborated upon in this embodiment.

[0109] Table 1

[0110]

[0111] Based on the quantitative comparison results of each method on the same test set in Table 1, the following conclusions can be drawn:

[0112] The proposed model (Ours) significantly outperforms existing methods such as SAM_lora, Deeplabv3+, Unet, and SAM_Adapter in four metrics: Dice (93.41%), Biou (85.75%), IoU (86.48%), and Acc (98.02%). This demonstrates that the SAM model, incorporating strip convolution and differential enhancement mechanisms, exhibits stronger directional feature extraction and background suppression capabilities in semiconductor gold wire segmentation, particularly excelling in boundary preservation and pixel-level classification accuracy. This validates the effectiveness and superiority of our method in industrial microscale linear target segmentation.

[0113] In summary, the embodiments of this application have achieved at least the following beneficial effects:

[0114] First, directional selective strip convolution is introduced into the standard LoRA framework, which extends low-rank adaptation from a fully connected form to a spatially structured form, enhancing the feature extraction capability for directional linear targets such as gold wires on chips.

[0115] Second, the differential enhancement mechanism adaptively suppresses the uniform background and low-frequency noise on the chip surface, highlights the high-frequency information at the edge of the gold line, and improves the segmentation accuracy under low contrast conditions.

[0116] Third, by keeping the SAM image encoder frozen and training only a small number of LoRA parameters and Mask Decoder, the computational overhead and storage requirements are significantly reduced, meeting the real-time deployment requirements of edge devices on semiconductor production lines.

[0117] Fourth, the synergistic design of four-directional strip convolution and differential enhancement makes the model robust to the multi-angle morphology of gold wires on the chip and complex background interference, thereby improving the accuracy and recall of defect detection.

[0118] Figure 6 This is a schematic diagram of an electronic device illustrated in this specification according to an exemplary embodiment. Please refer to... Figure 6 At the hardware level, the device includes a processor 610, an internal bus 320, a network interface 630, memory 640, a hardware acceleration device 650, and non-volatile memory 660, and may also include other hardware required for its functions. One or more embodiments of this application can be implemented in software, for example, the processor 610 reads the corresponding computer program from the non-volatile memory 360 into the memory 640 and then runs it. Of course, in addition to software implementation, one or more embodiments of this application do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the above processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0119] Figure 7This is a structural block diagram illustrating an exemplary embodiment of a SAM model training device based on striped convolution and difference enhancement, which can be applied to, for example... Figure 6 The electronic device shown implements the technical solution of this application. The SAM model training device includes: a dataset construction unit 710 and a model training unit 720, wherein:

[0120] Dataset construction unit 710 is used to construct a chip gold line segmentation dataset, which includes chip bonding region images and their corresponding pixel-level annotations of the gold line regions.

[0121] The model training unit 720 is used to train the SAM model using the dataset, optimizing the segmentation loss of the gold line region until a preset training stopping condition is reached. Each Transformer block of the SAM model's image encoder is equipped with an improved low-rank adaptation block. During model training, the SAM model's image encoder is frozen, and the parameters of the SAM model's mask decoder and low-rank adaptation blocks are fine-tuned. The improved low-rank adaptation block is configured as follows:

[0122] The input features are linearly reduced in dimensionality using a learnable first low-rank decomposition matrix. The reduced features are then reshaped from a token sequence into a feature map. Spatial correlation calculations are performed with multiple learnable preset direction prototypes to generate directional weights for each preset direction. The multiple preset direction prototypes correspond to different extension directions of the gold wire in the chip bonding region image.

[0123] The reshaped feature map is subjected to depthwise separable convolution operation by strip convolution kernels corresponding to each preset direction. The convolution result is then multiplied by the directional weights of the corresponding directions to obtain the strip convolution feature map corresponding to each preset direction.

[0124] Based on the reshaped feature map, the sum of weights in all directions, and the learnable parameters, differential enhancement is performed on each strip convolutional feature map to obtain the enhanced strip convolutional feature map, and a fused feature map is obtained based on the enhanced strip convolutional feature map.

[0125] After flattening the fused feature map into a sequence, a linear transformation is performed using a learnable second low-rank decomposition matrix to obtain a low-rank increment matrix.

[0126] The low-rank increment matrix is ​​superimposed onto the attention weights of the Transformer block corresponding to the current improved low-rank adaptor block to complete the parameter fine-tuning.

[0127] Figure 8This is a structural block diagram of a semiconductor gold wire splitting device based on the SAM model, as illustrated in an exemplary embodiment of this application. The semiconductor gold wire splitting device can be applied to, for example... Figure 6 The electronic device shown implements the technical solution of this application. The semiconductor gold wire segmentation device includes: an image acquisition unit 810 and an image segmentation unit 820, wherein:

[0128] Image acquisition unit 810 is used to acquire an image of the bonding area of ​​the chip to be detected;

[0129] Image segmentation unit 820 is used to input the image of the chip bonding region to be detected into a trained SAM model, and output a pixel-level segmentation mask of the gold line region through the SAM model; wherein, the training method of the SAM model can refer to Figure 1 The embodiments described herein will not be repeated here.

[0130] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0131] Accordingly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the above embodiments.

[0132] Accordingly, embodiments of this application also provide a computer program product configured to perform the methods described in any of the above embodiments.

[0133] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0134] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0135] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0136] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0137] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0138] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0139] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0140] It should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0141] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A SAM model training method based on strip convolution and difference enhancement, characterized in that, The method includes the following steps: Step S11: Construct a chip gold wire segmentation dataset, which includes chip bonding region images and their corresponding pixel-level annotations of the gold wire regions. Step S12: Train the SAM model using the dataset, optimizing the segmentation loss of the gold line region until a preset training stopping condition is reached; each Transformer block of the SAM model's image encoder is equipped with an improved low-rank adaptation block. During model training, the SAM model's image encoder is frozen, and the parameters of the SAM model's mask decoder and low-rank adaptation blocks are fine-tuned; wherein, the improved low-rank adaptation block is configured as follows: The input features are linearly reduced in dimensionality using a learnable first low-rank decomposition matrix. The reduced features are then reshaped from a token sequence into a feature map. Spatial correlation calculations are performed with multiple learnable preset direction prototypes to generate directional weights for each preset direction. The multiple preset direction prototypes correspond to different extension directions of the gold wire in the chip bonding region image. The reshaped feature map is subjected to depthwise separable convolution operation by strip convolution kernels corresponding to each preset direction. The convolution result is then multiplied by the directional weights of the corresponding directions to obtain the strip convolution feature map corresponding to each preset direction. Based on the reshaped feature map, the sum of weights in all directions, and the learnable parameters, differential enhancement is performed on each strip convolutional feature map to obtain the enhanced strip convolutional feature map, and a fused feature map is obtained based on the enhanced strip convolutional feature map. After flattening the fused feature map into a sequence, a linear transformation is performed using a learnable second low-rank decomposition matrix to obtain a low-rank increment matrix. The low-rank increment matrix is ​​superimposed onto the attention weights of the Transformer block corresponding to the current improved low-rank adaptor block to complete the parameter fine-tuning.

2. The method according to claim 1, characterized in that, Based on the extension direction of the gold wire in the chip bonding area image, the plurality of preset directions include the horizontal direction, the vertical direction, the main diagonal direction, and the anti-diagonal direction; Specifically, for the horizontal and vertical directions, the strip convolution kernels are one-dimensional horizontal strip convolution kernels and one-dimensional vertical strip convolution kernels, respectively. For the main diagonal direction and the anti-diagonal direction, the strip convolution kernel is a two-dimensional convolution kernel constrained by a mask matrix. The mask matrix is ​​used to retain the weights corresponding to the diagonal region while suppressing the off-diagonal region.

3. The method according to claim 2, characterized in that: The one-dimensional horizontal strip convolution kernel is a 1×k convolution kernel; The one-dimensional vertical strip convolution kernel is a k×1 convolution kernel; The two-dimensional convolution kernel is ; Where k is a preset value and M is a mask matrix.

4. The method according to claim 1, characterized in that, The expression for the enhanced strip convolutional feature map is as follows: ； in, For the first The enhanced strip convolutional feature maps corresponding to each direction For the first The strip convolutional feature maps corresponding to each direction, For learnable parameters, The reshaped feature map, For the first The direction weights of each direction The preset number of directions.

5. The method according to claim 1, characterized in that, No. The expressions for the direction weights corresponding to each direction are as follows: ； in, For the first The direction weights corresponding to each direction For the first The prototypes corresponding to each direction The reshaped feature map, , The preset number of directions.

6. The method according to claim 1, characterized in that, The improved low-rank adaptor block is configured to obtain the fused feature map through the following steps: The weighted summation of all enhanced strip convolutional feature maps is used to obtain the initial fused feature map. The initial fused feature map is residually connected to the reshaped feature map to obtain the final fused feature map.

7. A semiconductor gold wire segmentation method based on the SAM model, characterized in that, Includes the following steps: Step S21: Obtain an image of the chip bonding area to be detected; Step S22: Input the image of the chip bonding region to be detected into the trained SAM model, and output the pixel-level segmentation mask of the gold line region through the SAM model; The SAM model is trained using the method described in any one of claims 1 to 6.

8. A SAM model training device based on strip convolution and difference enhancement, characterized in that, The device includes: A dataset construction unit is used to construct a chip gold wire segmentation dataset, which includes chip bonding region images and their corresponding pixel-level annotations of the gold wire regions. The model training unit is used to train the SAM model using the dataset, optimizing the segmentation loss of the gold line region until a preset training stopping condition is reached. Each Transformer block of the SAM model's image encoder is equipped with an improved low-rank adaptation block. During model training, the SAM model's image encoder is frozen, and the parameters of the SAM model's mask decoder and low-rank adaptation blocks are fine-tuned. The improved low-rank adaptation block is configured as follows: The input features are linearly reduced in dimensionality using a learnable first low-rank decomposition matrix. The reduced features are then reshaped from a token sequence into a feature map. Spatial correlation calculations are performed with multiple learnable preset direction prototypes to generate directional weights for each preset direction. The multiple preset direction prototypes correspond to different extension directions of the gold wire in the chip bonding region image. The reshaped feature map is subjected to depthwise separable convolution operation by strip convolution kernels corresponding to each preset direction. The convolution result is then multiplied by the directional weights of the corresponding directions to obtain the strip convolution feature map corresponding to each preset direction. Based on the reshaped feature map, the sum of weights in all directions, and the learnable parameters, differential enhancement is performed on each strip convolutional feature map to obtain the enhanced strip convolutional feature map, and a fused feature map is obtained based on the enhanced strip convolutional feature map. After flattening the fused feature map into a sequence, a linear transformation is performed using a learnable second low-rank decomposition matrix to obtain a low-rank increment matrix. The low-rank increment matrix is ​​superimposed onto the attention weights of the Transformer block corresponding to the current improved low-rank adaptor block to complete the parameter fine-tuning.

9. A semiconductor gold wire dicing device based on the SAM model, characterized in that, The device includes: Image acquisition unit, used to acquire images of the bonding region of the chip to be detected; The image segmentation unit is used to input the image of the chip bonding region to be detected into the trained SAM model, and output the pixel-level segmentation mask of the gold line region through the SAM model; The SAM model is trained using the method described in any one of claims 1 to 6.

10. An electronic device, characterized in that, include: processor; A computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 1 to 7.

Images