Skin scar image segmentation method

Abstract

The invention relates to the field of medical image processing, in particular to a skin scar image segmentation method which is used for automatic skin scar area recognition and quantitative analysis. The problems of low segmentation precision, inaccurate boundary modeling and unreliable quantitative calculation caused by insufficient feature expression, weak generalization ability and manual dependence in the existing method are solved. The method comprises the following steps: S1, acquiring a skin scar image; s2, inputting a trained improved SAM2 segmentation network; and S3, outputting the segmentation mask. The improved SAM2 network is obtained by constructing and preprocessing a data set; the system comprises a trunk image encoder for freezing parameters, a prompt encoder, a medical image feature guide module (extracting texture details, color contrast and edge morphological features), a residual enhancement module (fusing general features, prompt vectors and medical features through residual connection) and a mask decoder. During training, backbone network parameters are frozen, and only the prompt encoder, the medical feature guide module, the residual enhancement module and the mask decoder are finely adjusted.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, specifically to a method for skin scar image segmentation, which is particularly suitable for automated skin scar region identification and quantitative analysis based on an improved SAM model. Background Technology

[0002] Skin scars are common conditions caused by burns, trauma, surgery, and other factors. Accurate identification and segmentation of scars are crucial for dermatopathological assessment, postoperative recovery analysis, and intelligent assisted treatment. Traditional scar image segmentation methods rely primarily on manual annotation or simple image processing techniques, which are insufficient to meet the robustness and accuracy requirements in complex backgrounds. In forensic practice, skin scars, as objective imprints of healed trauma, carry key information revealing the nature and extent of injury, making them indispensable physical evidence in judicial injury assessment. Specific segmentation (precisely depicting its boundaries) and calculation (measuring its length, area, and other parameters) of scars are fundamental and core tasks in forensic work. This is not a simple technical operation, as these geometric features of scars are directly related to the assessment of the severity of injury and the subsequent determination of disability levels, the results of which directly affect the vital rights of the parties involved—such as compensation amounts and social security benefits. Therefore, scar measurement must strive for extremely high precision to reflect the objectivity and impartiality of judicial assessment with the most rigorous standards.

[0003] Currently, manual segmentation and calculation by forensic experts remains the traditional and relatively accurate method widely used in the industry. Experts, relying on experience and a deep understanding of human anatomy, can make professional judgments. However, this method has significant limitations: firstly, the process is highly time-consuming and cumbersome, and inefficient when dealing with numerous complex scars; secondly, and more importantly, the measurement results rely heavily on the expert's subjective visual judgment—different individuals have naturally different visual sensitivities to subtle boundary changes, inevitably introducing subjective errors. This error makes it difficult for the final measurement data to fully meet the stringent requirements of judicial practice for the core standards of "objectivity and impartiality." Meanwhile, deep learning models are demonstrating strong vitality and vigorous development momentum in the field of intelligent medical image analysis. Especially in semantic segmentation of medical images, such as the precise location and delineation of brain tumor regions, deep learning technology has achieved remarkable results, with its accuracy and efficiency continuously improving. This wave of artificial intelligence (AI), driven by deep learning and advanced medical image processing technologies, is profoundly reshaping the landscape of medical diagnosis and research. It has already spurred groundbreaking progress in several key areas, including neuroscience exploration, early fetal developmental diagnosis and intervention, human emotion recognition and analysis, and even intelligent classification and quality enhancement of medical images of tissues such as the thyroid and breast. This undoubtedly provides a solid technological foundation and a highly attractive direction for using intelligent technologies to solve traditional pain points in forensic medicine, such as scar segmentation and computation.

[0004] Segment Anything Model 2 (SAM2), developed by Meta, is an advanced visual segmentation model and an upgraded version of SAM, specifically designed for object segmentation in images and videos. Built on the Transformer architecture, SAM2 achieves excellent segmentation performance through large-scale image data pre-training and utilizes a prompting mechanism for fast segmentation of various targets. Despite its strong versatility, SAM2 still suffers from insufficient feature adaptation in medical image segmentation, leading to inadequate feature representation and decoding accuracy. To address this issue, freezing the image encoder and fine-tuning the mask decoder and prompting encoder effectively improves SAM2's performance in medical image segmentation tasks.

[0005] Existing technology

[0006] Publication number: CN 116823852A

[0007] Invention Title: Method and System for Segmenting Strip-Shaped Skin Scar Images Based on Convolutional Neural Networks

[0008] This invention relates to a method and system for segmenting strip-shaped skin scars based on convolutional neural networks. The method includes: acquiring an image with strip-shaped skin scars; inputting the image into a U-shaped encoding / decoding network to segment the strip-shaped skin scars; wherein the U-shaped encoding / decoding network includes a SADC module and a GGMC module, the SADC module being used to enable the U-shaped encoding / decoding network to focus on the strip-shaped features of the skin scar, and the GGMC module being used to enhance the U-shaped encoding / decoding network's ability to perceive skin scars of different scales and lengths. However, this method has low segmentation accuracy when facing complex backgrounds, blurred boundaries, or heterogeneous scar regions, and cannot meet the needs of high-precision scar quantization in judicial or clinical scenarios; furthermore, due to the weak adaptability of traditional convolutional neural network models to scale and morphological changes in input images, the generalization ability is insufficient, making it difficult to effectively handle scars of different types and sizes. Therefore, this proposal aims to achieve high-precision, robust, and low-human-intervention-required automated segmentation and quantization of skin scar images by introducing an improved SAM2 model network structure to optimize the model architecture based on medical image characteristics. Summary of the Invention

[0009] The purpose of this invention is to solve the problems of low segmentation accuracy, inaccurate boundary modeling, unreliable quantization calculation, and poor medical image adaptability of existing skin scar image segmentation methods in complex backgrounds and heterogeneous scar regions due to insufficient feature expression, weak generalization ability, and dependence on manual interaction.

[0010] To achieve the above objectives, the present invention employs the following technical means:

[0011] This invention provides a method for skin scar image segmentation, comprising the following steps:

[0012] S1: Obtain the image of the skin scar to be segmented;

[0013] S2: Input the skin scar image to be segmented into the trained improved SAM2 segmentation network;

[0014] S3: The improved SAM2 segmentation network outputs a segmentation mask for the scar region in the image to be segmented;

[0015] The improved SAM2 segmentation network is trained in the following way:

[0016] Sa: Construct a dataset of skin scar images and perform preprocessing operations on the dataset;

[0017] Sb: Input the preprocessed dataset into the improved SAM2 segmentation network for training;

[0018] The improved SAM2 segmentation network includes:

[0019] A parameter-freezing image encoder is used to extract general visual features from the input image;

[0020] The image encoder, which adopts the Vision Transformer structure, mainly performs global feature extraction and characterization encoding on the input scar image.

[0021] A cue encoder is used to receive cue information and encode it into a high-dimensional semantic vector.

[0022] A medical image feature guidance module is used to extract and fuse scar-related medical features in the input image, including texture details, color contrast, and edge morphology features.

[0023] The residual enhancement module receives and fuses the general visual features extracted by the image encoder, the high-dimensional semantic vector output by the cue encoder, and the medical features output by the medical image feature guidance module, and enhances the features through residual connections and multi-source feature fusion mechanisms.

[0024] The mask decoder, based on the Transformer architecture, is used to receive the fused features output by the residual enhancement module and decode and output a multi-scale scar segmentation mask.

[0025] During training, the parameters of the image encoder are kept frozen, and only the prompt encoder, the medical image feature guidance module, the residual enhancement module, and the mask decoder are fine-tuned.

[0026] The original scar image is input into the image encoder, and a ViT structure is used for global visual feature extraction to obtain the general visual feature representation F of the image. backbone .

[0027] The human-generated prompts are input into the prompt encoder, and position embedding and multi-layer coding are performed to obtain the prompt guidance features F. prompt .

[0028] In the above scheme, the medical image feature guidance module is specifically used for:

[0029] Extract texture detail features from the input image;

[0030] Calculate the color contrast features of the input image;

[0031] Detect edge morphological features of the input image;

[0032] The texture detail features, color contrast features, and edge morphology features are fused to generate a medical feature guidance information map, which is then input into the residual enhancement module.

[0033] The specific implementation steps of the medical image feature guidance module in the above scheme are as follows:

[0034] Step b1: Extract texture detail features:

[0035] The Local Binary Pattern (LBP) method is used to extract local structural information from the image.

[0036] Step b2: Calculate color contrast characteristics:

[0037] Convert the image to Lab or HSV color space, extract the L, a, and b channels, and calculate the Euclidean distance between each pixel and its neighboring or all pixels in the image:

[0038]

[0039] D i C represents the contrast intensity of a pixel. i Represents the color characteristics of a pixel [L] i a i b i ], C j Indicates color characteristics [L] j a j b j ];

[0040] Step b3: Edge morphology detection:

[0041] The gradient map is calculated using the Sobel edge detection operator:

[0042] G x =I*K x G y =I*K y ,

[0043] Where I represents the input image, K x K y This represents the Sobel kernel, and G represents the edge intensity map.

[0044] Step b4: Feature fusion and generation of guiding information:

[0045] The features of the three channels L, a, and b are fused at multiple scales to generate medical feature guidance information, which is used to guide the network to better distinguish scar areas.

[0046] The three feature maps obtained in steps b1, b2, and b3 are concatenated along the channel dimension.

[0047] The structural medical feature representation F is obtained by channel compression and nonlinear mapping through a 3×3 convolution, batch normalization module, and ReLU module. guide The fused medical feature guide map is input into the residual enhancement module.

[0048] In the above scheme, the residual enhancement module adopts residual connections based on the ResNet structure and implements cross-layer connections to integrate the deep features extracted by the image encoder and the cue features F output by the cue encoder. prompt and the medical feature representation F output by the medical image feature guidance module guide Multi-scale context consistency modeling is performed.

[0049] In the above scheme, the specific implementation steps of the residual enhancement module are as follows:

[0050] Step 1: Preparation of input features

[0051] Input includes:

[0052] Visual features output by the image encoder: F backbone ;

[0053] Suggested semantic guidance features from the encoder output: F prompt ;

[0054] Structural medical features output by the medical image guidance module: F guide

[0055] Since these features come from different resolutions or semantic levels, they need to be aligned and fused.

[0056] Step 2: Channel Alignment and Blending

[0057] After concatenating the three features along the channel dimension, the channel dimension is unified by a 1×1 convolution, and a non-linear mapping is applied:

[0058] F concat =Concat(F backbone F prompr F guide )∈R H×W×3C

[0059] F fuesd =σ(BN(Connv) 3×3 (F concat )))∈R H×W×C

[0060] Where Conv is a 3×3 convolution operation, BN is batch normalization (BatchNorm), σ is the ReLU activation function, and Concat represents concatenation;

[0061] Step 3: Residual Connections Based on ResNet Structure

[0062] To enhance deep feature learning capabilities and prevent gradient vanishing, a standard residual block structure is introduced:

[0063]

[0064] The skip connection in the above structure It facilitates lossless information transmission and enhances the network's sensitivity to changes in texture and boundaries;

[0065] Step 4: Cross-layer connections enable multi-scale context modeling

[0066] Features from multiple layers (e.g., layers 4, 8, and 12) are extracted from the image encoder to represent the shallow, middle, and deep receptive fields, respectively. These cross-layer features are then combined with F... res_out After weighted fusion, they are input together into the cross-layer fusion module.

[0067]

[0068] Where α i F represents the weight coefficients of the feature map at layer i. bi Let F represent the feature map of the i-th layer of the backbone network image encoder, and β represent the residual output feature F. res_out Weighting coefficients The hierarchy of features is 2, 8, and 12;

[0069] Step 5: Output the fusion result

[0070] High semantic features F after cross-layer fusion module residual =F multi .

[0071] The final fused output enhancement feature F residual It will be passed to the mask decoder.

[0072] The prompting encoder receives prompting information including manually marked points or image prompt boxes, which are used to guide the improved SAM2 segmentation network to focus on the target scar area.

[0073] The preprocessing operations described in Sa include:

[0074] Data augmentation operations are performed on the original skin scar image, including one or more of image cropping, rotation, blurring, adding noise, and color perturbation.

[0075] The enhanced images are uniformly scaled or cropped to a preset size three-channel format;

[0076] The processed dataset is divided into training set, validation set and test set.

[0077] The image encoder adopts the Vision Transformer (ViT) structure.

[0078] The multi-scale segmentation mask output by the mask decoder is used to ensure accurate segmentation of complex scar structures.

[0079] Because the present invention employs the above-mentioned technical means, it has the following beneficial effects:

[0080] 1. This invention, by constructing a medical image-guided feature module (see step 3), integrates medically relevant visual features such as texture details, color contrast, and edge morphology, solving the technical problems of difficulty in distinguishing scars from the background and unclear boundaries in existing methods. This significantly improves the model's sensitivity and accuracy in locating scar regions. Traditional image segmentation models are prone to misjudging background areas or missing scar boundaries when faced with unstructured scars in medical images due to similar gray levels and insufficient contrast. The structural feature-guided mechanism introduced in this invention, through "splitting extraction + feature fusion," focuses on color-change regions, texture-abnormal regions, and edge features as key areas of interest for the model. This effectively improves the consistency between the segmented region and the actual scar morphology and enhances the network's adaptability to complex medical images.

[0081] 2. This invention uses a residual enhancement module (see step 4) to uniformly fuse three types of features from the image encoder, cue encoder, and medical feature module. It also introduces a ResNet-based residual connection and cross-layer feature integration mechanism, solving the problems of gradient decay and insufficient multi-source feature fusion in deep networks, thus improving the model's ability to express boundary details and context. Scar regions often have blurred and discontinuous boundaries, making them easily overlooked or locally lost. The multi-source feature residual fusion mechanism proposed in this invention retains the general semantic features extracted by the image encoder while integrating spatial guidance information and local structural cues. Through cross-scale information flow, the model maintains global consistency while considering local edge responses, thereby enhancing its ability to model the complex contour structure of scars and improving overall segmentation accuracy.

[0082] 3. This invention optimizes the mask decoder structure (see step 5) based on multi-scale...

[0083] The Transformer mechanism predicts scar masks and supports outputting mask results at different resolutions. This addresses the insensitivity of traditional methods to scar size and scale, ensuring accurate modeling and visualization of various scar types, including linear and blocky scars. By introducing a multi-scale feature processing mechanism, the model automatically adapts to scar regions of different sizes, making it suitable for different shooting scales and individual differences in clinical images. Simultaneously, the output mask image exhibits high edge fidelity and structural stability, providing a solid foundation for subsequent quantitative analysis and forensic measurements.

[0084] 4. This invention optimizes the training strategy by freezing the image encoder parameters and fine-tuning only the prompt encoder, mask decoder, and medical guidance module. This solves the problems of insufficient medical image sample size and low training efficiency, achieving high-performance transfer learning under low-sample conditions. Medical images are costly to acquire and have strong privacy requirements, often resulting in insufficient samples. The fine-tuning training strategy designed in this invention utilizes the general visual knowledge of the pre-trained image encoder and achieves excellent segmentation results with a small amount of medical image data through dedicated optimization of local modules. This significantly reduces training computational overhead and engineering deployment difficulty, demonstrating good transferability and engineering practicality. Attached Figure Description

[0085] Figure 1 SAM2 scar segmentation architecture diagram. Detailed Implementation

[0086] The embodiments of the present invention will be described in detail below. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, any modifications or equivalent substitutions made to the present invention should be covered within the scope of the claims of the present invention.

[0087] Furthermore, to better illustrate the present invention, numerous specific details are set forth in the following detailed embodiments. Those skilled in the art will understand that the present invention can be practiced without these specific details.

[0088] This invention aims to address the problems of existing skin scar image segmentation methods in the following aspects:

[0089] 1. Limited segmentation accuracy: Existing methods do not achieve high segmentation accuracy when faced with complex backgrounds, blurred boundaries, or heterogeneous scar areas, and cannot meet the needs of judicial or clinical scenarios for high-precision scar quantification.

[0090] 2. Insufficient generalization ability; traditional convolutional neural network models are weakly adaptable to scale and morphological changes in input images, making it difficult to effectively handle scars of different types and sizes.

[0091] 3. Lack of efficient automation mechanisms; the processing flow that relies on manual interaction is not only inefficient, but also susceptible to subjective judgment, and cannot meet the requirements of large-scale data processing and objective judicial assessment.

[0092] 4. Poor model versatility and poor matching with medical images; for example, although advanced segmentation models such as SAM2 have strong general performance, they have problems such as insufficient feature representation and insufficient decoding accuracy when applied in the field of medical images.

[0093] Therefore, this proposal aims to achieve high-precision, robust, and low-human-intervention-required automated segmentation and quantization of skin scar images by introducing an improved SAM2 structure and optimizing the model architecture in combination with medical image characteristics.

[0094] A method for segmenting skin scar images includes the following steps:

[0095] S1: Obtain the image of the skin scar to be segmented;

[0096] S2: Input the skin scar image to be segmented into the trained improved SAM2 segmentation network;

[0097] S3: The improved SAM2 segmentation network outputs a segmentation mask for the scar region in the image to be segmented;

[0098] The improved SAM2 segmentation network is trained in the following way:

[0099] Sa: Construct a dataset of skin scar images and perform preprocessing operations on the dataset;

[0100] Sb: Input the preprocessed dataset into the improved SAM2 segmentation network for training;

[0101] The improved SAM2 segmentation network includes:

[0102] A parameter-freezing image encoder is used to extract general visual features from the input image;

[0103] The image encoder, which adopts the Vision Transformer structure, mainly performs global feature extraction and characterization encoding on the input scar image.

[0104] A cue encoder is used to receive cue information and encode it into a high-dimensional semantic vector.

[0105] A medical image feature guidance module is used to extract and fuse scar-related medical features in the input image, including texture details, color contrast, and edge morphology features.

[0106] The residual enhancement module is used to receive and fuse the general visual features extracted by the image encoder, the high-dimensional semantic vector output by the prompt encoder, and the medical features output by the medical image feature guidance module, and perform feature enhancement through residual connection and multi-source feature fusion mechanism.

[0107] The mask decoder, based on the Transformer architecture, is used to receive the fused features output by the residual enhancement module and decode and output a multi-scale scar segmentation mask.

[0108] During training, the parameters of the image encoder are kept frozen, and only the prompt encoder, the medical image feature guidance module, the residual enhancement module, and the mask decoder are fine-tuned.

[0109] refer to Figure 1 The process logic within the improved SAM2 segmentation network data processing logic and its relationships are as follows:

[0110] Step a1: Input the original scar image into the image encoder (image encoder), and use the ViT structure to extract global visual features to obtain the general visual feature representation F of the image. backbone .

[0111] Step a2: Input the manually provided prompts (such as markers, bounding boxes, etc.) into the prompt encoder, perform position embedding and multi-layer encoding processing, and obtain the prompt guidance features F. prompt .

[0112] Step a3: Input the original scar image into the medical image guidance feature module, and sequentially perform texture feature extraction, color contrast calculation, and edge morphology detection, then fuse them into a guidance feature map to obtain the structural medical feature representation F. guide .

[0113] Step a4: Analyze the general visual features F obtained in step a1. backbone The cue feature F obtained in step 2 prompt and the medical feature F obtained in step 3 guide The input is fed into the residual enhancement module, where feature concatenation, 3×3 convolutional compression, residual connections, and cross-layer context fusion are performed to obtain a high semantic feature representation F after multi-source fusion. residual .

[0114] Step a5: Enhance the residual feature F obtained in step a4. residual The input mask decoder uses a multi-scale attention mechanism based on the Transformer structure for mask prediction to obtain a precisely segmented scar region mask.

[0115] Step 1 above specifically includes the following sub-steps:

[0116] Step 1.1: Collect a large number of real skin scar image samples to construct an original image database;

[0117] Step 1.2: Perform enhancement operations on the original image, including image cropping, rotation, blurring, adding noise, color perturbation, etc., to improve the robustness of the model;

[0118] Step 1.3: Divide the augmented dataset into training, validation, and test sets proportionally;

[0119] Step 2 above specifically includes the following sub-steps:

[0120] Step 2.1: Standardize the training set images to a three-channel input format of 512×512 pixels to reduce training costs and improve model learning efficiency;

[0121] Step 2.2: Input the image data into the improved SAM2 network for training. The improved SAM2 structure includes a prompt encoder, a mask decoder, and a feature guidance module. To improve the adaptability to medical images, the image encoder parameters are frozen, and only the prompt encoder and mask decoder are fine-tuned. A shared feature extraction layer and residual connection structure are introduced to enhance feature fusion capabilities.

[0122] The SAM2 backbone adopts a ViT structure, which has powerful visual feature extraction capabilities. To improve training efficiency and reduce computational costs, the image encoder parameters are kept frozen in this invention, with only subsequent modules being fine-tuned.

[0123] The prompt encoder receives manually labeled points or image cues to guide the model to focus on the scar region. The encoder converts the cues into high-dimensional semantic vectors, which are then passed to the mask decoder to enhance target perception. The specific steps are as follows:

[0124] Step 11: Provide manual prompts for the input image. The prompts include key points, bounding boxes, click points, etc., forming a prompt data structure.

[0125] Step e2: Coordinate encoding or spatial embedding is performed on the cue data structure to transform the position data into a cue feature map aligned with the image size.

[0126] Step e3: Input the cue feature map into a multi-layer convolutional or Transformer encoder to extract deep semantic features of the cue information and obtain a high-dimensional cue vector representation.

[0127] Step e4: Combine the high-dimensional cue vector with the global visual features output by the image encoder and input them into the residual enhancement module or mask decoder to guide the model to focus on the cue region and improve the spatial localization capability of scar recognition.

[0128] Medical Image Guided Feature Module: This module is designed specifically for scar images. It integrates medical-related features such as texture details, color contrast, and edge morphology of the image as prior information input models, effectively improving medical adaptability.

[0129] Residual Enhancement Module: Introduces residual connections based on the ResNet structure to improve the training stability of deep networks and enhance the accuracy of scar boundary modeling. At the same time, a multi-source feature fusion mechanism is introduced to integrate backbone features, cue features, and guiding features, and completes multi-scale, context-consistent modeling through cross-layer connections and the residual enhancement module.

[0130] Mask Decoder: The decoder is designed based on the Transformer architecture. It receives the fused global high-dimensional features and outputs a multi-scale segmentation mask to ensure accurate segmentation of complex scar structures.

[0131] Step 4: Segmentation Reasoning and Result Output

[0132] The image to be processed is input into the trained SAM2 network, and the system automatically segments the scar region. The output results, through subsequent calculations, can yield: an accurate scar region mask; quantitative indicators such as area, length, and contour complexity; and standardized visual results that can be used for clinical diagnosis and forensic evaluation.

[0133] Figure 1 The diagram shows the overall architecture of the improved SAM2 scar image segmentation system used in this invention, clearly illustrating the connection relationships and functional flow between some modules.

[0134] Example 1

[0135] This invention provides a method for skin scar image segmentation, comprising the following steps:

[0136] S1: Obtain the image of the skin scar to be segmented;

[0137] S2: Input the skin scar image to be segmented into the trained improved SAM2 segmentation network;

[0138] S3: The improved SAM2 segmentation network outputs a segmentation mask for the scar region in the image to be segmented;

[0139] The improved SAM2 segmentation network is trained in the following way:

[0140] Sa: Construct a dataset of skin scar images and perform preprocessing operations on the dataset;

[0141] Sb: Input the preprocessed dataset into the improved SAM2 segmentation network for training;

[0142] The improved SAM2 segmentation network includes:

[0143] A parameter-freezing image encoder is used to extract general visual features from the input image;

[0144] The image encoder, which adopts the Vision Transformer structure, mainly performs global feature extraction and characterization encoding on the input scar image.

[0145] A cue encoder is used to receive cue information and encode it into a high-dimensional semantic vector.

[0146] A medical image feature guidance module is used to extract and fuse scar-related medical features in the input image, including texture details, color contrast, and edge morphology features.

[0147] The residual enhancement module receives and fuses the general visual features extracted by the image encoder, the high-dimensional semantic vector output by the cue encoder, and the medical features output by the medical image feature guidance module, and enhances the features through residual connections and multi-source feature fusion mechanisms.

[0148] The mask decoder, based on the Transformer architecture, is used to receive the fused features output by the residual enhancement module and decode and output a multi-scale scar segmentation mask.

[0149] During training, the parameters of the image encoder are kept frozen, and only the prompt encoder, the medical image feature guidance module, the residual enhancement module, and the mask decoder are fine-tuned.

[0150] The original scar image is input into the image encoder, and a ViT structure is used for global visual feature extraction to obtain the general visual feature representation F of the image. backbone .

[0151] The human-generated prompts are input into the prompt encoder, and position embedding and multi-layer coding are performed to obtain the prompt guidance features F. prompt .

[0152] In the above scheme, the medical image feature guidance module is specifically used for:

[0153] Extract texture detail features from the input image;

[0154] Calculate the color contrast features of the input image;

[0155] Detect edge morphological features of the input image;

[0156] The texture detail features, color contrast features, and edge morphology features are fused to generate a medical feature guidance information map, which is then input into the residual enhancement module.

[0157] The specific implementation steps of the medical image feature guidance module in the above scheme are as follows:

[0158] Step b1: Extract texture detail features:

[0159] The Local Binary Pattern (LBP) method is used to extract local structural information from the image.

[0160] Step b2: Calculate color contrast characteristics:

[0161] Convert the image to Lab or HSV color space, extract the L, a, and b channels, and calculate the Euclidean distance between each pixel and its neighboring or all pixels in the image:

[0162]

[0163] D i C represents the contrast intensity of a pixel. i Represents the color characteristics of a pixel [L] i a i b i ], C j Indicates color characteristics [L] j a j b j ];

[0164] Step b3: Edge morphology detection:

[0165] The gradient map is calculated using the Sobel edge detection operator:

[0166] G x =I*K x G y =I*K y ,

[0167] Where I represents the input image, K x K y This represents the Sobel kernel, and G represents the edge intensity map.

[0168] Step b4: Feature fusion and generation of guiding information:

[0169] The features of the three channels L, a, and b are fused at multiple scales to generate medical feature guidance information, which is used to guide the network to better distinguish scar areas.

[0170] The three feature maps obtained in steps b1, b2, and b3 are concatenated along the channel dimension.

[0171] The structural medical feature representation F is obtained by channel compression and nonlinear mapping through a 3×3 convolution, batch normalization module, and ReLU module. guide The fused medical feature guide map is input into the residual enhancement module.

[0172] In the above scheme, the residual enhancement module adopts residual connections based on the ResNet structure and implements cross-layer connections to integrate the deep features extracted by the image encoder and the cue features F output by the cue encoder. prompt and the medical feature representation F output by the medical image feature guidance module guide Multi-scale context consistency modeling is performed.

[0173] In the above scheme, the specific implementation steps of the residual enhancement module are as follows:

[0174] Step 1: Preparation of input features

[0175] Input includes:

[0176] Visual features output by the image encoder: F backbone ;

[0177] Suggested semantic guidance features from the encoder output: F prompt :

[0178] Structural medical features output by the medical image guidance module: F guide

[0179] Since these features come from different resolutions or semantic levels, they need to be aligned and fused.

[0180] Step 2: Channel Alignment and Blending

[0181] After concatenating the three features along the channel dimension, the channel dimension is unified by a 1×1 convolution, and a non-linear mapping is applied:

[0182] F concat =Concat(F backbone F prompr F guide )∈R H×W×3C

[0183] F fuesd =σ(BN(Conv) 3×3 (F concat )))∈R H×W×C

[0184] Where Conv is a 3×3 convolution operation, BN is batch normalization (BatchNorm), v is the ReLU activation function, and Concat represents concatenation;

[0185] Step 3: Residual Connections Based on ResNet Structure

[0186] To enhance deep feature learning capabilities and prevent gradient vanishing, a standard residual block structure is introduced:

[0187]

[0188] The skip connection in the above structure It facilitates lossless information transmission and enhances the network's sensitivity to changes in texture and boundaries;

[0189] Step 4: Cross-layer connections enable multi-scale context modeling

[0190] Features from multiple layers (e.g., layers 4, 8, and 12) are extracted from the image encoder to represent the shallow, middle, and deep receptive fields, respectively. These cross-layer features are then combined with F... res_out After weighted fusion, they are input together into the cross-layer fusion module.

[0191]

[0192] Where α i F represents the weight coefficients of the feature map at layer i. bi Let F represent the feature map of the i-th layer of the backbone network image encoder, and β represent the residual output feature F. res_out The weight coefficients are 2, 8, and 12, representing the feature hierarchy.

[0193] Step 5: Output the fusion result

[0194] High semantic features F after cross-layer fusion module residual =F multi .

[0195] The final fused output enhancement feature F residual It will be passed to the mask decoder.

[0196] The prompting encoder receives prompting information including manually marked points or image prompt boxes, which are used to guide the improved SAM2 segmentation network to focus on the target scar area.

[0197] The preprocessing operations described in Sa include:

[0198] Data augmentation operations are performed on the original skin scar image, including one or more of image cropping, rotation, blurring, adding noise, and color perturbation.

[0199] The enhanced images are uniformly scaled or cropped to a preset size three-channel format;

[0200] The processed dataset is divided into training set, validation set and test set.

[0201] The image encoder adopts the Vision Transformer (ViT) structure.

[0202] The multi-scale segmentation mask output by the mask decoder is used to ensure accurate segmentation of complex scar structures. Key points of this application:

[0203] 1. Customized optimization of SAM2 network structure: In order to reduce training costs and improve efficiency, the image encoder structure of the SAM model is frozen, and the mask decoder and cue encoder are optimized to adapt to the scar image segmentation task.

[0204] 2. Design of a guidance mechanism for scar features: A feature guidance module is proposed to highlight the texture and boundary of scars, thereby improving the model's ability to identify scar targets.

[0205] 3. Introduce cross-layer feature fusion and sharing mechanism: Enhance the model's ability to model details of scar areas by using residual connections and shared feature extraction layers.

[0206] 4. Integrated design of the quantitative calculation module: realizes the automatic calculation of quantitative parameters such as scar length and area, and ensures reusability and accuracy in judicial and medical scenarios.

[0207] Compared with the closest prior art, the advantages of this application are as follows:

[0208] Compared with the scar segmentation method based on a U-shaped encoding and decoding structure described in CN116823852A, the present invention has the following technical advantages:

[0209] 1. Enhanced model versatility and adaptability: This invention is based on an improved SAM2 model, which has stronger image understanding and generalization capabilities, and can more effectively handle multiple types and morphologies of scars.

[0210] 2. Prompt-driven efficient interaction mechanism: SAM2's prompt mechanism allows for quick location of target areas through simple interactions, greatly improving efficiency and reducing human error.

[0211] 3. Enhanced ability to model boundary and structural information: The introduction of feature guidance modules and cross-layer connection structures makes the model better at capturing the detailed boundaries of complex scar morphology.

[0212] 4. Integrated Quantitative Capabilities: This method not only focuses on segmentation but also integrates a scar quantitative analysis module, providing a complete solution for forensic identification and clinical assessment.

[0213] 5. More efficient training transfer strategies: By freezing the image encoder and fine-tuning specific modules, efficient transfer learning can be achieved in small-sample medical image scenarios, reducing training costs.

Claims

1. A method of skin scar image segmentation, characterized by, The method comprises the following steps: Step S1: obtaining a skin scar image to be segmented; Step S2: inputting the skin scar image to be segmented into the improved SAM2 segmentation network trained; Step S3: the improved SAM2 segmentation network outputs a segmentation mask of the scar area in the image to be segmented; Wherein, the improved SAM2 segmentation network is trained by the following way: Sa: construct a skin scar image dataset, and perform a pretreatment operation on the dataset; Sb: input the pretreated dataset into the improved SAM2 segmentation network for training; The improved SAM2 segmentation network comprises: An image encoder with frozen parameters for extracting general visual features of an input image; The image encoder adopts a Vision Transformer structure, which mainly functions to extract and encode global features of an input scar image. A prompt encoder for receiving prompt information and encoding it into a high-dimensional semantic vector; A medical image feature guiding module for extracting and fusing medical features related to scars in an input image, including texture details, color contrast, and edge morphology features; A residual enhancement module for receiving and fusing the general visual features extracted by the image encoder, the high-dimensional semantic vector output by the prompt encoder, and the medical features output by the medical image feature guiding module, and performing feature enhancement through residual connection and multi-source feature fusion mechanism. A mask decoder based on a Transformer architecture for receiving the fused features output by the residual enhancement module and decoding and outputting multi-scale scar segmentation masks; During the training process, the parameters of the image encoder are kept frozen, and only the prompt encoder, the medical image feature guiding module, the residual enhancement module, and the mask decoder are fine-tuned.

2. The method of claim 1, wherein, The original scar image is input into an image encoder, global visual feature extraction is performed by using a ViT structure, and general visual feature representation of the image is obtained .

3. The method of claim 2, wherein, The artificial prompt information input prompt encoder is used for position embedding and multi-layer encoding processing to obtain prompt guide features .

4. The method of claim 3, wherein, The medical image feature guiding module is specifically used for: Extracting texture detail features of an input image; Calculating color contrast features of an input image; Detecting edge morphology features of an input image; Fusing the texture detail features, color contrast features, and edge morphology features to generate a medical feature guiding information map and input it into the residual enhancement module.

5. The method of claim 4, wherein, The medical image feature guiding module specifically implements the following steps: Step b1: extracting texture detail features: Using a local binary pattern (LBP) method to extract local structure information of an image; Step b2: calculating color contrast features: Converting the image to Lab or HSV color space, extracting L, a, and b channels, and calculating the Euclidean distance between each pixel and the neighborhood or the entire image pixels: a contrast intensity of the pixel, a color characteristic of the pixel ], a color characteristic ]; Step b3: edge morphology detection: Using a Sobel edge detection operator to calculate a gradient map: ， ， wherein represents an input image, , represents a Sobel kernel, represents an edge intensity map, Step b4: feature fusion and generating guiding information: Multi-scale fusing the features of the above three channels to generate medical feature guiding information for guiding the network to better distinguish scar areas; Concatenating the three feature maps obtained in steps b1, b2, and b3 in the channel dimension; The structural medical feature representation is obtained through a 3*3 convolution, a batch normalization module, and a ReLU module for channel compression and nonlinear mapping, and input into a residual enhancement module. , input into a residual enhancement module.

6. The method of claim 5, wherein, The residual enhancement module adopts residual connection based on ResNet structure and realizes cross-layer connection for integrating visual feature representation extracted by the image encoder , prompt features output by the prompt encoder , and medical feature representation output by the medical image feature guiding module , to perform multi-scale context consistency modeling.

7. The method of claim 6, wherein, The residual enhancement module specifically implements the following steps: Step c1: input feature preparation, including: Visual features output by the image encoder: ; Prompting the encoder to output semantic guiding features: ; Structural medical features output by the medical image guidance module: Step c2: Channel alignment and fusion After concatenating the three features in the channel dimension, a 1x1 convolution is used to unify the channel dimension, and a nonlinear mapping is applied: wherein, Conv is a 3x3 convolution operation, BN is batch normalization, and σ is a ReLU activation function, concatenation Step c3: Residual connection based on ResNet structure To improve the ability of deep feature learning and prevent gradient disappearance, the standard residual block structure is introduced: The jump connection in the above structure It helps to propagate information without loss and strengthens the sensitivity of the network to texture and boundary changes. Step c4: Multi-scale context modeling through cross-layer connection The multiple levels of features are extracted from the image encoder, respectively representing shallow, middle and deep receptive fields, and these cross-layer features are combined with The weighted fusion is input into the cross-layer fusion module: wherein represents the first layer feature map, represents the weight coefficient of the residual output feature map of the backbone network image encoder, represents the residual output feature map of the first layer, , 2, 8, 12 are the levels of the features; Step 5: Output fusion result Cross-layer fusion module post high semantic features ; Output enhanced features of final fusion will be passed to the mask decoder.

8. The method of claim 1, wherein, The prompt information received by the prompt encoder includes artificial annotation points or image prompt boxes, which are used to guide the improved SAM2 segmentation network to focus on the scar target area.

9. The method of claim 1, wherein, The preprocessing operation in Sa includes: Performing data enhancement operations on the original skin scar image, the data enhancement operations including one or more of image cropping, rotation, blurring, noise addition, color disturbance; Uniformly scaling or cropping the enhanced image into a three-channel format of a preset size; Divide the processed data set into training set, validation set and test set.

10. The method of claim 1, wherein, The multi-scale segmentation mask output by the mask decoder is used to ensure accurate segmentation of complex scar structures.

Images