A SAM model training method and device based on strip convolution and differential enhancement, and a semiconductor gold wire segmentation method and device based on the SAM model

Abstract

The application provides a SAM model training method and device based on strip convolution and differential enhancement, and a semiconductor gold wire segmentation method and device based on the SAM model. The training method comprises the following steps: constructing a chip gold wire segmentation data set; training the SAM model by using the data set, optimizing the segmentation loss of the gold wire region, and stopping until a preset training stop condition is reached; an improved low-rank adaptive block is arranged on each Transform block of an image encoder of the SAM model; during the model training process, the image encoder of the SAM model is frozen, and the parameters of the mask decoder and the low-rank adaptive block of the SAM model are fine-tuned. The application realizes explicit enhancement and background suppression of the semiconductor gold wire features under the parameter efficient fine-tuning framework, and improves the segmentation precision.

Claims

1. A SAM model training method based on strip convolution and difference enhancement, characterized in that, The method includes the following steps: Step S11: Construct a chip gold wire segmentation dataset, which includes chip bonding region images and their corresponding pixel-level annotations of the gold wire regions. Step S12: Train the SAM model using the dataset, optimizing the segmentation loss of the gold line region until a preset training stopping condition is reached; each Transformer block of the SAM model's image encoder is equipped with an improved low-rank adaptation block. During model training, the SAM model's image encoder is frozen, and the parameters of the SAM model's mask decoder and low-rank adaptation blocks are fine-tuned; wherein, the improved low-rank adaptation block is configured as follows: The input features are linearly reduced in dimensionality using a learnable first low-rank decomposition matrix. The reduced features are then reshaped from a token sequence into a feature map. Spatial correlation calculations are performed with multiple learnable preset direction prototypes to generate directional weights for each preset direction. The multiple preset direction prototypes correspond to different extension directions of the gold wire in the chip bonding region image. The reshaped feature map is subjected to depthwise separable convolution operation by strip convolution kernels corresponding to each preset direction. The convolution result is then multiplied by the directional weights of the corresponding directions to obtain the strip convolution feature map corresponding to each preset direction. Based on the reshaped feature map, the sum of weights in all directions, and the learnable parameters, differential enhancement is performed on each strip convolutional feature map to obtain the enhanced strip convolutional feature map, and a fused feature map is obtained based on the enhanced strip convolutional feature map. After flattening the fused feature map into a sequence, a linear transformation is performed using a learnable second low-rank decomposition matrix to obtain a low-rank increment matrix. The low-rank increment matrix is ​​superimposed onto the attention weights of the Transformer block corresponding to the current improved low-rank adaptor block to complete the parameter fine-tuning.

2. The method according to claim 1, characterized in that, Based on the extension direction of the gold wire in the chip bonding area image, the plurality of preset directions include the horizontal direction, the vertical direction, the main diagonal direction, and the anti-diagonal direction; Specifically, for the horizontal and vertical directions, the strip convolution kernels are one-dimensional horizontal strip convolution kernels and one-dimensional vertical strip convolution kernels, respectively. For the main diagonal direction and the anti-diagonal direction, the strip convolution kernel is a two-dimensional convolution kernel constrained by a mask matrix. The mask matrix is ​​used to retain the weights corresponding to the diagonal region while suppressing the off-diagonal region.

3. The method according to claim 2, characterized in that: The one-dimensional horizontal strip convolution kernel is a 1×k convolution kernel; The one-dimensional vertical strip convolution kernel is a k×1 convolution kernel; The two-dimensional convolution kernel is ; Where k is a preset value and M is a mask matrix.

4. The method according to claim 1, characterized in that, The expression for the enhanced strip convolutional feature map is as follows: ； in, For the first The enhanced strip convolutional feature maps corresponding to each direction For the first The strip convolutional feature maps corresponding to each direction, For learnable parameters, The reshaped feature map, For the first The direction weights of each direction The preset number of directions.

5. The method according to claim 1, characterized in that, No. The expressions for the direction weights corresponding to each direction are as follows: ； in, For the first The direction weights corresponding to each direction For the first The prototypes corresponding to each direction The reshaped feature map, , The preset number of directions.

6. The method according to claim 1, characterized in that, The improved low-rank adaptor block is configured to obtain the fused feature map through the following steps: The weighted summation of all enhanced strip convolutional feature maps is used to obtain the initial fused feature map. The initial fused feature map is residually connected to the reshaped feature map to obtain the final fused feature map.

7. A semiconductor gold wire segmentation method based on the SAM model, characterized in that, Includes the following steps: Step S21: Obtain an image of the chip bonding area to be detected; Step S22: Input the image of the chip bonding region to be detected into the trained SAM model, and output the pixel-level segmentation mask of the gold line region through the SAM model; The SAM model is trained using the method described in any one of claims 1 to 6.

8. A SAM model training device based on strip convolution and difference enhancement, characterized in that, The device includes: A dataset construction unit is used to construct a chip gold wire segmentation dataset, which includes chip bonding region images and their corresponding pixel-level annotations of the gold wire regions. The model training unit is used to train the SAM model using the dataset, optimizing the segmentation loss of the gold line region until a preset training stopping condition is reached. Each Transformer block of the SAM model's image encoder is equipped with an improved low-rank adaptation block. During model training, the SAM model's image encoder is frozen, and the parameters of the SAM model's mask decoder and low-rank adaptation blocks are fine-tuned. The improved low-rank adaptation block is configured as follows: The input features are linearly reduced in dimensionality using a learnable first low-rank decomposition matrix. The reduced features are then reshaped from a token sequence into a feature map. Spatial correlation calculations are performed with multiple learnable preset direction prototypes to generate directional weights for each preset direction. The multiple preset direction prototypes correspond to different extension directions of the gold wire in the chip bonding region image. The reshaped feature map is subjected to depthwise separable convolution operation by strip convolution kernels corresponding to each preset direction. The convolution result is then multiplied by the directional weights of the corresponding directions to obtain the strip convolution feature map corresponding to each preset direction. Based on the reshaped feature map, the sum of weights in all directions, and the learnable parameters, differential enhancement is performed on each strip convolutional feature map to obtain the enhanced strip convolutional feature map, and a fused feature map is obtained based on the enhanced strip convolutional feature map. After flattening the fused feature map into a sequence, a linear transformation is performed using a learnable second low-rank decomposition matrix to obtain a low-rank increment matrix. The low-rank increment matrix is ​​superimposed onto the attention weights of the Transformer block corresponding to the current improved low-rank adaptor block to complete the parameter fine-tuning.

9. A semiconductor gold wire dicing device based on the SAM model, characterized in that, The device includes: Image acquisition unit, used to acquire images of the bonding region of the chip to be detected; The image segmentation unit is used to input the image of the chip bonding region to be detected into the trained SAM model, and output the pixel-level segmentation mask of the gold line region through the SAM model; The SAM model is trained using the method described in any one of claims 1 to 6.

10. An electronic device, characterized in that, include: processor; A computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 1 to 7.

Images