A high-temperature alloy microstructure segmentation method based on parameter efficient fine-tuning

Abstract

The application discloses a high-temperature alloy microstructure segmentation method based on parameter efficient fine-tuning, belongs to the technical field of computer vision and intelligent manufacturing, and is applied to automatic identification and segmentation of scanning electron microscope images. The method comprises the following steps: collecting scanning electron microscope images of high-temperature alloy microstructures, constructing a data set and performing image preprocessing and data enhancement; designing a fine-tuning strategy for a segmentation model, the fine-tuning method is a multi-scale interactive gate residual switch adapter, which is used for cross-scale feature extraction and detail enhancement, an adaptive amplitude modulation adapter is designed, which is used for amplitude self-adaption of the adapter output, dynamic matching of the main distribution and the task domain difference; the fine-tuning model is trained on the constructed data set to adapt to the characteristics of high-temperature alloy microstructure segmentation; and the trained fine-tuning model is used for identification and segmentation of the SEM images to be analyzed, and a segmentation mask of the target microstructure is output.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and intelligent manufacturing technology, specifically relating to a method for segmenting the microstructure of high-temperature alloys based on efficient parameter fine-tuning. Background Technology

[0002] The four essential elements of materials science are "preparation, composition and structure, properties, and service performance." The microstructure of materials, as a mesoscale structural feature, is a crucial link connecting the atomic structure and macroscopic properties of materials, directly influencing the "processing-structure-performance" relationship of engineering materials. In the field of materials science, microstructure plays a key role in the analysis, utilization, and improvement of material properties. It can reveal the synergistic effects of composition and processing, thereby customizing material properties.

[0003] Unlike natural image segmentation tasks, the segmentation of microscopic images of material microstructures is a binary segmentation task characterized by class imbalance, complex and irregular shapes. Unlike ordinary objects in the macroscopic world, the number of regions of interest (ROIs) in material microscopic images exhibits a dense and dispersed distribution pattern, and the microstructures typically possess diverse geometric morphologies; for example, grains may be irregular in shape, and grain boundaries may be curved or intersecting. Furthermore, the material microstructures in microscopic images are presented as grayscale images, and the image quality is affected by factors such as the quality of sample preprocessing before the experiment, human intervention during the scanning process, and the environment in which the experimental equipment is placed. This can lead to difficulties in accurately reflecting microstructural features, potentially resulting in images with high noise levels, low signal-to-noise ratios, and unclear microstructural features.

[0004] Traditional segmentation methods rely heavily on the experience of experimenters, manually segmenting microscopic images. This method is labor-intensive, time-consuming, and highly subjective, with segmentation results from different individuals potentially varying significantly, thus affecting the consistency of analysis results. To improve segmentation efficiency and make the results more convincing, it is necessary to introduce automated segmentation techniques. Classical segmentation methods include those based on intensity thresholding, edge or boundary detection, and region / similarity, which can process images through fixed logical rules. However, due to their reliance on pre-defined rules or fixed computational logic, they often suffer from drawbacks such as difficulty adapting to complex scenes and a lack of adaptability and learning ability. To handle diverse and dense structures under different imaging conditions, deep learning has been successfully applied to the analysis of material micrographs. With the emergence of basic models, it has ushered in the era of large-scale models in the field of image segmentation. The powerful generalization ability obtained through pre-training on massive cross-domain data enables it to achieve downstream tasks in various fields. By introducing structures or modules with a small number of parameters for fine-tuning, high-precision domain-specific segmentation performance can be achieved, significantly breaking through the data dependence and generalization bottlenecks of traditional models.

[0005] The Segement Anything Model (SAM), as the first general-purpose large-scale visual model for image-assisted segmentation, is trained on billions of image-mask pairs, exhibiting zero-shot learning capabilities and strong generalization abilities. Fine-tuning it using datasets from the target domain allows it to adapt well to the segmentation characteristics of high-temperature alloy microstructures. However, directly fine-tuning the 11 million-parameter SAM encoder using limited-scale material microscopy datasets (typically containing only hundreds to thousands of labeled images) is not only computationally expensive but also risks destroying the general representation capabilities acquired through pre-training due to over-adaptation to small sample sizes, leading to the catastrophic forgetting problem of the model. Therefore, establishing a high-temperature alloy microstructure segmentation method based on efficient parameter fine-tuning is of great significance in the field of material microstructure image segmentation. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, the present invention aims to provide a high-temperature alloy microstructure segmentation method based on efficient parameter fine-tuning. It proposes a multi-scale interactive gated residual switch adapter and an adaptive amplitude modulation adapter to fine-tune the model on a high-temperature alloy microstructure image dataset, thereby obtaining high-precision segmentation results.

[0007] The objective of this invention can be achieved through the following specific technical solutions:

[0008] The method for segmenting the microstructure of high-temperature alloys based on efficient parameter fine-tuning includes the following steps:

[0009] S1. Acquire scanning electron microscope images of the microstructure of high-temperature alloys, perform fine annotation of the target microstructure in the SEM images, construct a dataset, and perform image preprocessing and data augmentation.

[0010] S2. A parameter-efficient fine-tuning strategy is proposed, and a multi-scale interactive gated residual switch adapter and an adaptive amplitude modulation adapter are designed to achieve feature enhancement within the coding layer and attention module.

[0011] S3. Train the fine-tuned model on the constructed dataset to obtain the material microstructure segmentation model;

[0012] S4. The trained material microstructure segmentation model is used to identify and segment the SEM image to be analyzed, and the segmentation mask of the target microstructure is output.

[0013] Furthermore, step S1 specifically includes:

[0014] S1.1 Acquire scanning electron microscope images of the microstructure of high-temperature alloys;

[0015] S1.2 The labelme tool is used to perform fine annotation of the target microstructure in the SEM image. The annotation results are saved in JSON format. After exporting, the data format is converted from JSON to binary mask image, and the dataset is randomly divided into training set, validation set and test set in a 3:1:1 ratio.

[0016] S1.3 The image is preprocessed using grayscale conversion, histogram equalization, and edge sharpening, and then cropped to a size of 512×512.

[0017] S1.4 uses methods such as flipping, rotating, contrast adjustment, and random noise addition to enhance image data.

[0018] Furthermore, the specific content of step S2 includes:

[0019] S2.1 Design a multi-scale interactive gated residual switch adapter for cross-scale feature extraction and detail enhancement. The SAM image encoder sets the corresponding multi-scale interactive gated residual switch adapter layer by layer.

[0020] S2.2 Design an adaptive amplitude modulation adapter to adapt the output amplitude of the adapter and dynamically match the differences between the backbone distribution and the task domain. The SAM image encoder sets the corresponding adaptive amplitude modulation adapter layer by layer.

[0021] Furthermore, in the multi-scale interactive gating residual switch adapter module of step S2.1, firstly, the attention output is numerically stabilized and its distribution calibrated in the embedding dimension through layer normalization. Then, nonlinear activation is performed through linear downsampling and Gaussian error linear units. Subsequently, the bottleneck space processing is constructed by building three parallel convolutional branches, using 1×1, 3×3, and 5×5 convolutional kernels to achieve multi-scale response. A bidirectional gating interaction mechanism is introduced between adjacent branches, and the gating weights are controlled by learnable scalar parameters to simultaneously encode fine-grained boundary textures, local contours, and larger structural patterns. Directional coupling and stable fusion are achieved without introducing direct coupling at far scales. Then, the fused features are statistically normalized through batch normalization and nonlinear activation is performed using Gaussian error linear units. Then, the bottleneck output is restored to the embedding dimension of the image encoder through linear upsampling mapping. Finally, the integration of the backbone semantics is controlled by the residual switch.

[0022] Furthermore, when the multi-scale interactive gated residual switch adapter is located after the output projection of the multi-head attention mechanism and before the summation of the first residual, the adapter's internal residual is enabled; when the multi-scale interactive gated residual switch adapter is located in parallel with layer normalization → adaptive amplitude modulation adapter → multilayer perceptron, the input is the intermediate feature after the first residual, and it operates in a mode without enabling the internal residual. Its output is used as a parallel bypass and is added element-wise with the main output and the intermediate residual to form the final feature of the three-way summation.

[0023] Further, in step S2.2, the adaptive amplitude modulation adapter module is connected in series between layer normalization and the multilayer perceptron. In the adaptive amplitude modulation adapter module, the intermediate features after normalization are first received, and then the embedding dimension is compressed to the bottleneck dimension through linear downsampling mapping. A Gaussian error linear unit is used for nonlinear activation to establish a trainable lightweight representation in a low-dimensional space. Then, the bottleneck output is restored to the embedding dimension of the image encoder through linear upsampling mapping, forming a controllable reconstruction of the original features. Then, a dropout layer is applied at the upsampling output to increase randomness and robustness, suppress overfitting, and improve stability in imbalanced and noisy scenarios. Then, the adapter output is adaptively modulated with a learnable scaling factor α, starting with a small initial value and adaptively updated during training, so as to continuously and differentiably control the information injection intensity and achieve matching between the backbone distribution and the task domain difference. Finally, the residual is added to the input intermediate features element by element to inject a mild expression gain while ensuring the steady state of the backbone features. The fusion result is then sent to the subsequent multilayer perceptron to achieve semantic integration.

[0024] Furthermore, in step S3, the fine-tuned model is trained on the constructed dataset, specifically including:

[0025] S3.1 Model initialization uses the SAM image encoder weights pre-trained on a large-scale dataset for initialization, constructing an overall framework of image encoder, cue encoder, and mask decoder based on ViT-B; the multi-scale interactive gated residual switch adapter and adaptive amplitude modulation adapter are randomly initialized during the initialization phase; the cue encoder and mask decoder are loaded from pre-trained weights, wherein the mask label of the mask decoder and the output super-network multilayer perceptron are re-initialized according to the output dimension of this task to adapt to the new number of categories;

[0026] S3.2 Parameter Freeze: During the training phase, the non-adaptor parameters of the image encoder are frozen, and only the adapter parameters are retained for training; Note that the encoder and mask decoder parameters are not frozen and participate in training.

[0027] S3.3 Feature Extraction and Adapter Enhancement: Visual features of the input image are extracted by the SAM image encoder. After linear projection of the multi-head attention mechanism module, a multi-scale interactive gated residual switch adapter is connected to perform robust fusion with multi-scale parallel processing and bidirectional gating interaction between adjacent scales. The internal residual of the adapter is enabled by default. An adaptive amplitude modulation adapter is connected in series between the layer normalization and the multilayer perceptron. The main path features are conservatively enhanced with a linear bottleneck and a learnable scaling factor before being fed into the multilayer perceptron for integration. A bypass multi-scale interactive gated residual switch adapter is introduced in parallel at the tail of the converter module. The intermediate features after the first residual are used as input. The internal residual is explicitly disabled, and only the bypass gain is output. The bypass gain is then added element-wise with the output of the main multilayer perceptron and the intermediate residual.

[0028] S3.4 Calculates and optimizes the mixed loss consisting of binary cross-entropy, Dice and Tversky, and performs backpropagation and parameter updates based on the AdamW optimizer to update the parameters of four modules: multi-scale interactive gated residual switch adapter, adaptive amplitude modulation adapter, cue encoder and mask decoder.

[0029] S3.5 Repeat steps S3.2 to S3.4 using the training set data to train the model until it converges. A fixed learning rate of 1e-4 is used during the training process.

[0030] S3.6 The model's segmentation performance is validated using a validation set. During training, the optimal weights are saved using an optimal weight selection strategy based on the average intersection-union ratio of the validation set.

[0031] Furthermore, the prompt encoder outputs an empty sparse embedding and a dense embedding extended by the maskless embedding in the no-prompt mode, which, together with the image position encoding, are used by the mask decoder.

[0032] Furthermore, the specific content of step S4 includes:

[0033] S4.1 Input scanning electron microscope images of the microstructure of high-temperature alloys;

[0034] S4.2 Load the optimal weights and use the trained model to identify and segment the SEM image to be analyzed;

[0035] S4.3 After inference, output the segmentation mask of the target microstructure.

[0036] Compared with the prior art, the present invention has the following advantages:

[0037] This invention provides a high-temperature alloy microstructure segmentation method based on efficient parameter fine-tuning. By fine-tuning the SAM model, cross-scale feature fusion and detail enhancement are achieved with relatively small parameter overhead, resulting in accurate segmentation of high-temperature alloy microstructures. Through the design of a multi-scale interactive gated residual switch adapter and an adaptive amplitude modulation adapter, the model can be efficiently fine-tuned while freezing most of the SAM parameters. This solves the problems of high computational cost caused by comprehensive SAM fine-tuning, and the catastrophic forgetting of the model due to over-adaptation to small samples, which destroys the general representation ability obtained from pre-training. While preserving the original segmentation ability of the model, it can achieve accurate segmentation of microscopic images of high-temperature alloy microstructures. This method is suitable for the requirement of quantitative research on multiple microstructural features simultaneously in the context of high-throughput materials research, providing theoretical and methodological support for the characterization and analysis of high-temperature alloy microstructures, and has good application prospects. Attached Figure Description

[0038] Figure 1 This is a flowchart of the method of the present invention;

[0039] Figure 2 This is a schematic diagram of the overall architecture of the image segmentation model of the present invention;

[0040] Figure 3 This is a schematic diagram of the multi-scale interactive gated residual switch adapter structure of the present invention;

[0041] Figure 4 This is a schematic diagram of the adaptive amplitude modulation adapter structure of the present invention;

[0042] Figure 5 This is a schematic diagram of the recognition results of the present invention on the IN718 test set. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0044] The purpose of this invention is to provide a high-temperature alloy microstructure segmentation method based on efficient parameter fine-tuning. It employs a multi-scale interactive gated residual switch adapter and an adaptive amplitude modulation adapter to fine-tune the SAM model, thereby achieving accurate segmentation of the high-temperature alloy microstructure in scanning electron microscope images.

[0045] like Figure 1 As shown, a method for segmenting the microstructure of high-temperature alloys based on efficient parameter fine-tuning includes the following steps:

[0046] S1. Acquire scanning electron microscope images of the microstructure of high-temperature alloys, perform fine annotation of the target microstructure in the SEM images, construct a dataset, and perform image preprocessing and data augmentation.

[0047] This step specifically includes:

[0048] S1.1 Prepare IN718 specimens suitable for high-temperature in-situ tensile testing. The specimens are dog-bone shaped, 50 mm long, and contain a gauge length segment 1.5 mm long, 1.35 mm wide, and 1 mm thick. Heat the specimens to 1050 °C at a rate of 10 °C / min and hold for 1 hour. Then, water quench to room temperature. Subsequently, age the solution-treated specimens at 850 °C, 875 °C, 900 °C, 925 °C, 950 °C, and 975 °C for 1 hour, 3 hours, 5 hours, 8 hours, 12 hours, and 24 hours, respectively. Use 400 mesh, 600 mesh, 1200 mesh, 3000 mesh, and 5000 mesh screens sequentially. The cooled in-situ tensile specimen was mechanically polished with sandpaper until there were no visible scratches on the surface. Then, the polished specimen was coarsely polished and finely polished in sequence using a vibratory polishing machine. The polished specimen was then etched to focus on the surface microstructure.

[0049] S1.2 In-situ experiments were conducted at room temperature using a Tescan Clara scanning electron microscope. During the experiments, scanning electron microscope images of the microstructure of the high-temperature alloy were acquired under the same field of view.

[0050] S1.3 Labelme is used to perform fine annotation of the target microstructure in SEM images. The annotation results are saved in JSON format. After exporting, the data format is converted from JSON to binary mask image, and the dataset is randomly divided into training set, validation set and test set in a 3:1:1 ratio.

[0051] S1.4 The image is preprocessed using grayscale conversion, histogram equalization and edge sharpening, and then cropped to 512×512 pixels;

[0052] S1.5 uses methods such as flipping, rotating, contrast adjustment, and random noise addition to enhance image data in order to improve the robustness of the model.

[0053] S2. A parameter-efficient fine-tuning strategy is proposed, and a multi-scale interactive gated residual switch adapter and an adaptive amplitude modulation adapter are designed to achieve feature enhancement within the coding layer and attention module. This step specifically includes:

[0054] S2.1 Design a multi-scale interactive gated residual switch adapter for cross-scale feature extraction and detail enhancement. The SAM image encoder sets the corresponding multi-scale interactive gated residual switch adapter layer by layer.

[0055] As shown in Figure 3, in the multi-scale interactive gated residual switch adapter module, the attention output is first numerically stabilized and its distribution calibrated in the embedding dimension through layer normalization. Then, nonlinear activation is performed by linear downsampling and Gaussian error linear units. Subsequently, the bottleneck space processing is constructed by building three parallel convolutional branches, using 1×1, 3×3, and 5×5 convolutional kernels to achieve multi-scale response. A bidirectional gating interaction mechanism is introduced between adjacent branches, and the gating weights are controlled by learnable scalar parameters to simultaneously encode fine-grained boundary textures, local contours, and larger structural patterns. Directional coupling and stable fusion are achieved without introducing direct coupling at far scales. Then, the fused features are statistically normalized by batch normalization and nonlinear activation is performed by Gaussian error linear units. Then, the bottleneck output is restored to the embedding dimension of the image encoder by linear upsampling mapping. Finally, the integration of the backbone semantics is controlled by residual switching.

[0056] When the multi-scale interactive gated residual switch adapter is located after the output projection of the multi-head attention mechanism and before the summation of the first residual, the adapter's internal residual is enabled. When the multi-scale interactive gated residual switch adapter is located in parallel with layer normalization → adaptive amplitude modulation adapter → multilayer perceptron, the input is the intermediate feature after the first residual, and it operates in a mode without enabling the internal residual. Its output is used as a parallel bypass and is added element-wise with the main output and the intermediate residual to form the final feature of the three-way summation.

[0057] The interaction strength of the multi-scale interactive gated residual switch adapter module is controlled by a learnable scalar, with the initial value set to 0.1 to maintain a stable backbone distribution in the early stages of training.

[0058] S2.2 Design an adaptive amplitude modulation adapter to adapt the output amplitude of the adapter and dynamically match the differences between the backbone distribution and the task domain. The SAM image encoder sets the corresponding adaptive amplitude modulation adapter layer by layer.

[0059] As shown in Figure 4, in the adaptive amplitude modulation adapter module, the intermediate features after layer normalization are first received. Then, the embedding dimension is compressed to the bottleneck dimension through linear downsampling mapping, and Gaussian error linear units are used for nonlinear activation to establish a trainable lightweight representation in a low-dimensional space. Then, the bottleneck output is restored to the embedding dimension of the image encoder through linear upsampling mapping, forming a controllable reconstruction of the original features. Then, a dropout layer is applied at the upsampling output to increase randomness and robustness, suppress overfitting, and improve stability in imbalanced and noisy scenarios. Then, the adapter output is adaptively modulated with a learnable scaling factor α, starting with a small initial value and adaptively updating during training, so as to continuously and differentiably control the information injection intensity and achieve matching between the backbone distribution and the task domain difference. Finally, the residual is added to the input intermediate features element by element to inject a mild expression gain while ensuring the steady state of the backbone features. The fusion result is then sent to the subsequent multilayer perceptron to achieve semantic integration.

[0060] The adaptive amplitude modulation adapter module is connected in series between layer normalization and the multilayer perceptron. In the adaptive amplitude modulation adapter module, the dropout layer parameter is set to 0.1 to increase randomness and robustness and suppress overfitting; the learnable scaling factor α is initially set to 0.05 to maintain backbone statistical stability in the early stage of training and gradually release gain as training progresses.

[0061] S3. Train the fine-tuned model on the constructed dataset to obtain the material microstructure segmentation model. This step specifically includes:

[0062] S3.1 Model initialization: The SAM image encoder weights, pre-trained on a large-scale dataset, are used for initialization. The overall framework of the image encoder, cue encoder, and mask decoder based on ViT-B is constructed, as shown in Figure 2. The multi-scale interactive gated residual switch adapter and the adaptive amplitude modulation adapter are randomly initialized during the initialization phase. The cue encoder and mask decoder are loaded from the pre-trained weights. The mask label of the mask decoder and the output super-network multilayer perceptron are re-initialized according to the output dimension of this task to adapt to the new number of categories.

[0063] S3.2 Parameter Freeze: During the training phase, the non-adaptor parameters of the image encoder are frozen, and only the adapter parameters are retained for training; the encoder and mask decoder parameters are not frozen and participate in training.

[0064] S3.3 Feature Extraction and Adapter Enhancement: Visual features of the input image are extracted by the SAM image encoder. After linear projection of the multi-head attention mechanism module, a multi-scale interactive gated residual switch adapter is connected to perform robust fusion with multi-scale parallel processing and bidirectional gating interaction between adjacent scales. The internal residual of the adapter is enabled by default. An adaptive amplitude modulation adapter is connected in series between the layer normalization and the multilayer perceptron. The main path features are conservatively enhanced with a linear bottleneck and a learnable amplitude factor before being fed into the multilayer perceptron for integration. A bypass multi-scale interactive gated residual switch adapter is introduced in parallel at the tail of the converter module. The intermediate features after the first residual are used as input. The internal residual is explicitly disabled, and only the bypass gain is output. The bypass gain is then fused element-wise with the output of the main multilayer perceptron and the intermediate residual.

[0065] S3.4 Calculates and optimizes the mixed loss consisting of binary cross-entropy, Dice and Tversky, and performs backpropagation and parameter updates based on the AdamW optimizer, updating the parameters of four modules: multi-scale interactive gated residual switch adapter, adaptive amplitude modulation adapter, cue encoder, and mask decoder.

[0066] For the task of segmenting the microstructure of high-temperature alloys, a hybrid loss function consisting of binary cross-entropy, Dice, and Tversky is adopted: L = λ1 · + λ2 · + λ3 · λ1, λ2, and λ3 are the linear weighting coefficients of the three losses. Where L... BCE The binary cross-entropy loss is calculated using the following formula:

[0067] = - · ,

[0068] in, ∈ {0,1} represents a pixel binary label (0 for background, 1 for foreground); ∈ [0,1], representing the predicted probability that a pixel belongs to the foreground; if the model output is logits z i ,but = sigmoid( ) ;z i is the original output score of the model corresponding to pixel i without any probabilistic activation; N is the total number of pixels involved in the loss calculation.

[0069] The Dice loss is calculated using the following formula:

[0070] I = ,

[0071] Dice = (2 · I + ε) / ( + + ε),

[0072] = 1 - Dice

[0073] Here, ε is a smoothing term (e.g., 1e-6) used to avoid zero denominators.

[0074] It is the Tversky loss, calculated using the following formula:

[0075] TP = ,

[0076] FP = ,

[0077] FN = ,

[0078] T = (TP + ε) / (TP + α · FP + β · FN + ε) ,

[0079] = 1 - T,

[0080] Where TP represents the total number of "soft true positives" correctly predicted as foreground pixels; FP represents the total number of "soft false positives" incorrectly predicted as foreground pixels in the background; FN represents the total number of "soft false negatives" not predicted as foreground pixels in the foreground; α and β are Tversky coefficients that control the trade-off between FP and FN.

[0081] First, the model output Logits are obtained and activated by Sigmoid to obtain the foreground probability map. The original labels are inverted to match the foreground target. Then, the binary cross-entropy loss, which measures the difference in pixel-level distribution, the Dice loss, which measures the overlap of regions (with the smoothing term set to 1e-6), and the Tversky loss, which adjusts the false positive and false negative penalty weights by α = 0.4 and β = 0.6, are calculated and aggregated with weight coefficients of 0.2, 0.5, and 0.3 to obtain the total mixed loss. Finally, the total loss is backpropagated based on the AdamW optimizer, and gradient clipping with a maximum norm of 1.0 is implemented to prevent gradient explosion. Only the trainable parameters of the multi-scale interactive gated residual switch adapter, adaptive amplitude modulation adapter, cue encoder, and mask decoder are iteratively updated to complete the parameter optimization for a single training step.

[0082] S3.5 uses the training set data to repeat steps S3.2 to S3.4 to train the model. The batch size is set to 32, the total number of training epochs is set to 600, and the input image size is set to 512. Throughout the training process, no learning rate decay strategy is used, and a fixed learning rate of 1e-4 is always maintained to ensure that the model can continuously and stably update the relevant parameters of the adapter, cue encoder, and mask decoder in the parameter efficient fine-tuning mode.

[0083] S3.6 validates the model's segmentation performance using a validation set, and calculates and reports evaluation metrics including accuracy (Acc), mean intersection-over-union ratio (mIoU), recall, precision, and dice coefficient to monitor the model's convergence process and generalization performance.

[0084] During training, optimal weights are saved using an optimal weight selection strategy based on the average intersection-union ratio (IU) of the validation set: when the IU of the validation set in the current round is not lower than the historical best value, the weight file obtained from that round of training is saved to a specified directory (. / weight / epoch.pth), and the corresponding epoch number and the best average IU are recorded for subsequent inference and reproduction experiments. The cue encoder outputs empty sparse embeddings and dense embeddings expanded from unmasked embeddings in cueless mode, which, along with image position encoding, are used by the mask decoder.

[0085] S4. The trained material microstructure segmentation model is used to identify and segment the SEM image to be analyzed, and the segmentation mask of the target microstructure is output. This step specifically includes:

[0086] S4.1 Input scanning electron microscope images of the microstructure of high-temperature alloys;

[0087] S4.2 Load the saved optimal weights and use the trained model to identify and segment the SEM image to be analyzed;

[0088] After inference, the S4.3 model finally outputs a segmentation mask of the target's microstructure.

[0089] This section details the comparative implementation method of the present invention:

[0090] To verify the effectiveness of the proposed high-temperature alloy microstructure segmentation method based on efficient parameter fine-tuning, comparative experiments were conducted on the scanning electron microscope images of the high-temperature alloy microstructures acquired in step S1. Using SAMViT-B pre-trained weights as initialization, the non-adaptor parameters of the image encoder were frozen. The multi-scale interactive gated residual switch adapter, adaptive amplitude modulation adapter, cue encoder, and mask decoder were trained using the parameter configuration set in step S3.5. A hybrid loss consisting of binary cross-entropy, Dice, and Tversky was used, and the parameters provided in step S3.4 were set. Backpropagation and parameter updates were performed using the AdamW optimizer. During the inference phase, the single-channel mask was binarized using sigmoid+ with a unified threshold. Under the same test set, unified preprocessing and input size, and automatic segmentation settings without cueing, the segmentation results were compared with the SAM baseline model without neighborhood fine-tuning. The segmentation visualization results on the test set are reported.

[0091] In this embodiment, Figure 5 shows the image comparison results, which respectively show the original image, the SAM segmentation mask image, and the high-temperature alloy microstructure segmentation mask image based on efficient parameter fine-tuning proposed in this invention. It can be seen that the proposed method is better than the SAM model before fine-tuning in predicting image details, which demonstrates the reliability of the proposed method.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for segmenting the microstructure of high-temperature alloys based on efficient parameter fine-tuning, characterized in that, Includes the following steps: S1. Acquire scanning electron microscope images of the microstructure of high-temperature alloys, annotate the target microstructures in the SEM images, construct a dataset, and perform image preprocessing and data augmentation. S2. A parameter-efficient fine-tuning strategy is proposed, and a multi-scale interactive gated residual switch adapter and an adaptive amplitude modulation adapter are designed to achieve feature enhancement within the coding layer and attention module. The specific details of this step include: S2.1 Design a multi-scale interactive gated residual switch adapter for cross-scale feature extraction and detail enhancement. The SAM image encoder sets the corresponding multi-scale interactive gated residual switch adapter for each transform layer. In the multi-scale interactive gated residual switch adapter module, the attention output is numerically stabilized and its distribution calibrated in the embedding dimension through layer normalization. Then, it is non-linearly activated by linear downsampling and Gaussian error linear units. Subsequently, it enters the bottleneck space processing of constructing three parallel convolutional branches, using 1×1, 3×3 and 5×5 convolutional kernels to achieve multi-scale response. A bidirectional gated interaction mechanism is introduced between adjacent branches. The gate weights are controlled by learnable scalar parameters. Then, the fused features are statistically normalized by batch normalization and non-linearly activated by Gaussian error linear units. Then, the bottleneck output is restored to the embedding dimension of the image encoder by linear upsampling mapping. Finally, the residual switch controls whether to integrate the backbone semantics. S2.2 Design an adaptive amplitude modulation adapter to adapt the amplitude of the adapter output and dynamically match the differences between the backbone distribution and the task domain. The SAM image encoder sets the corresponding adaptive amplitude modulation adapter for each transformer layer. The adaptive amplitude modulation adapter module is connected in series between the layer normalization and the multilayer perceptron. In the adaptive amplitude modulation adapter module, the intermediate features after normalization are first received. Then, the embedding dimension is compressed to the bottleneck dimension through linear downsampling mapping and nonlinear activation is performed using Gaussian error linear units. Then, the bottleneck output is restored to the embedding dimension of the image encoder through linear upsampling mapping. Then, a dropout layer is applied at the upsampling output and the adapter output is adaptively modulated with an amplitude using a learnable scaling factor α. Finally, the residual is added element-wise with the input intermediate features and the fusion result is sent to the subsequent multilayer perceptron to realize semantic integration. S3. Train the fine-tuned model on the constructed dataset to obtain the material microstructure segmentation model; S4. The trained material microstructure segmentation model is used to identify and segment the SEM image to be analyzed, and the segmentation mask of the target microstructure is output.

2. The method for segmenting the microstructure of high-temperature alloys based on efficient parameter fine-tuning according to claim 1, characterized in that, The specific content of step S1 includes: S1.1 Acquire scanning electron microscope images of the microstructure of high-temperature alloys; S1.2 Labelme is used to annotate the target microstructure in SEM images. The annotation results are exported in JSON format, converted into binary mask images, and the dataset is divided. S1.3 The image is preprocessed using grayscale conversion, histogram equalization and edge sharpening, and then cropped to a size of 512×512. S1.4 uses methods such as flipping, rotating, contrast adjustment, and random noise addition to enhance image data.

3. The method for segmenting the microstructure of high-temperature alloys based on efficient parameter fine-tuning according to claim 1, characterized in that, When the multi-scale interactive gated residual switch adapter is located after the output projection of the multi-head attention mechanism and before the summation of the first residual, the adapter's internal residual is enabled. When the multi-scale interactive gated residual switch adapter is located in parallel with layer normalization → adaptive amplitude modulation adapter → multilayer perceptron, the input is the intermediate feature after the first residual, and it operates in a mode without enabling the internal residual. Its output is used as a parallel bypass and is added element-wise with the main output and intermediate residual to form the final feature of the three-way summation.

4. The method for segmenting the microstructure of high-temperature alloys based on efficient parameter fine-tuning according to claim 1, characterized in that, In step S3, the fine-tuned model is trained on the constructed dataset, specifically including: S3.1 Model initialization: The SAM image encoder weights, which are pre-trained on a large-scale dataset, are used for initialization to build an overall framework of image encoder, cue encoder and mask decoder based on ViT-B. S3.2 Parameter Freeze: During the training phase, the non-adaptor parameters of the image encoder are frozen, and only the adapter parameters are retained for training. This indicates that the encoder and mask decoder parameters are not frozen and are included in the training. S3.3 Feature Extraction and Adapter Enhancement: Visual features of the input image are extracted through the SAM image encoder, and the feature representation is enhanced through the adapter module. S3.4 Calculate and optimize the mixed loss consisting of binary cross-entropy, Dice and Tversky, and perform backpropagation and parameter update based on the AdamW optimizer; S3.5 Repeat steps S3.2 to S3.4 using the training set data to train the model until the model converges; S3.6 Validate the model's segmentation performance using a validation set.

5. The method for segmenting the microstructure of high-temperature alloys based on efficient parameter fine-tuning according to claim 4, characterized in that, The prompt encoder outputs empty sparse embeddings and dense embeddings extended by maskless embeddings in the non-prompt mode, which are used in conjunction with image position encoding for use by the mask decoder.

6. The method for segmenting the microstructure of high-temperature alloys based on efficient parameter fine-tuning according to claim 1, characterized in that, The specific contents of step S4 include: S4.1 Input scanning electron microscope images of the microstructure of high-temperature alloys; S4.2 Load the optimal weights and use the trained model to identify and segment the SEM image to be analyzed; S4.3 After inference, output the segmentation mask of the target's microstructure.

Images