Skin lesion image segmentation method and system based on multi-scale deformable aggregation

Abstract

The invention provides a skin lesion image segmentation method and system based on multi-scale deformable aggregation, and the method comprises the steps: obtaining a skin lesion image data set, dividing the data set into a training set and a test set, constructing an improved U-shaped segmentation network model, optimizing model parameters through training, applying the trained model to the test set, and obtaining a skin lesion image segmentation result. And generating a high-precision segmented image. According to the model, an innovative convolution-parallel vision Mangbar module is adopted in an encoder, and extraction and fusion of local details and global long-range dependency of an image are realized through cooperation of a CNN and a state space model; a deformable convolution attention gate is put forward in a decoder, a key area is accurately focused by using a deformable convolution adaptive lesion shape, and cross-level feature fusion and multi-scale refining are realized. Through the multi-scale deformable aggregation mechanism, the segmentation precision and robustness of the model to complex and various skin lesions are significantly improved, and reliable assistance is provided for clinical diagnosis.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for segmenting skin lesion images based on multi-scale deformable aggregation. Background Technology

[0002] Accurate image segmentation of skin lesions is crucial for computer-aided early screening and diagnosis of skin cancers such as melanoma. However, due to the high heterogeneity and complexity of skin lesions in terms of morphology, size, boundary, and contrast, achieving accurate and robust automatic segmentation remains a major challenge.

[0003] Currently, the field of medical image segmentation mainly employs two architectures: convolutional neural networks (CNNs) and Transformers. CNN models, represented by U-Net, excel at extracting local features, but their local receptive fields limit their ability to model long-range dependencies, making it difficult to fully understand the global context of the image. In contrast, Transformers achieve global modeling through a self-attention mechanism, but their computational complexity increases quadratically with image size, placing a heavy computational burden on processing high-resolution medical images. Furthermore, their block-based processing method can easily lose pixel-level details, affecting the accuracy of segmentation boundaries.

[0004] In recent years, structured state-space sequence models, represented by Mamba, have demonstrated the advantage of capturing long-range dependencies with linear computational complexity in sequence modeling, providing new possibilities for simultaneously overcoming the locality limitations of CNNs and the high computational overhead of Transformers. Therefore, there is an urgent need for a new network architecture that can effectively integrate local feature extraction and efficient global modeling, significantly improving the segmentation accuracy of complex skin lesion images while ensuring computational efficiency. Summary of the Invention

[0005] To address the efficiency limitations of existing skin lesion segmentation techniques, such as the difficulty in modeling long-range dependencies in local receptive fields of Convolutional Neural Networks (CNNs) and the high computational complexity of Visual Transformers (ViTs) due to their self-attention mechanisms, this invention provides a skin lesion image segmentation method and system based on multi-scale deformable aggregation. Specifically, this invention designs a novel multi-scale deformable aggregation U-Net, in which the Convolutional-Parallel Visual Mamba (Conv-PVM) module integrates the local feature extraction capabilities of CNNs with the global dependency modeling advantages of state-space models, enabling the network to simultaneously and efficiently capture local details and global contextual information. The deformable convolutional attention gate allows the network to adaptively fit the irregular shapes, varying sizes, and complex textures of lesions, significantly improving the model's ability to subdivide blurred and complex boundaries. The two working together significantly improve segmentation accuracy and robustness while maintaining computational efficiency.

[0006] This invention provides a method for skin lesion image segmentation based on multi-scale deformable aggregation, comprising: S1. Obtain a dataset of skin lesion images and preprocess the dataset to obtain a standardized image dataset; S2. Divide the standardized image dataset into a training set and a test set; S3. Construct a skin lesion image segmentation network model, input the training set to train the segmentation network model, and optimize the model parameters; S4. Apply the trained model to the test set to generate skin lesion segmentation images; In step S3, the skin lesion image segmentation network model is a symmetrical U-shaped encoding and decoding structure. The encoder extracts and fuses image features through multiple cascaded convolutional-parallel visual Mamba modules. The convolutional-parallel visual Mamba module adopts a dual-branch parallel architecture. It performs channel segmentation on the input features and performs local feature extraction and global dependency modeling respectively. It then fuses the outputs of the two branches to generate enhanced features that simultaneously contain local details and overall semantics. The decoder refines the fused features from the encoder by progressively upsampling and fusing them in multiple dimensions to generate segmentation results. The decoder includes: an efficient upconvolutional block for upsampling and scale alignment of deep semantic features; a deformable convolutional attention gate for dynamically generating attention weights based on the feature space distribution to enhance the target region and fuse cross-layer features; and a multi-scale convolutional attention module for serializing and refining features through attention mechanisms and multi-scale convolutions.

[0007] Furthermore, the preprocessing procedure for the skin lesion image dataset includes: Hair and artifact removal were performed on the original skin lesion images, and illumination inhomogeneity correction was applied. Adjust the size of the skin lesion images to be processed to the preset resolution; The image is normalized in size and then subjected to random data augmentation operations, including horizontal flipping, vertical flipping, and random rotation.

[0008] Furthermore, the composition and feature extraction process of the convolutional-parallel visual Mamba module include: Channel segmentation unit: Divides the input feature map into two parts evenly along the channel dimension; The dual-branch parallel processing unit includes a convolutional branch and a parallel visual Mamba branch. The convolutional branch receives the first part of the features and extracts local feature maps rich in fine-grained details, while the parallel visual Mamba branch receives the second part of the features and extracts global feature maps with global semantic information and long-range spatial dependencies. Feature fusion unit: The output features of the convolutional branch and the parallel visual Mamba branch are concatenated by channels, and the channel shuffling operation is used to promote the full exchange and integration of information across branches, and the output contains enhanced features that include both local details and global context.

[0009] Furthermore, the feature extraction process of the dual-branch parallel processing unit includes: The convolutional branch receives the first part of the features from the channel segmentation, and outputs the processed local feature map through batch normalization, standard 2D 3×3 convolutional layer, non-linear activation function, standard 2D 3×3 convolutional layer, non-linear activation function, and 1×1 convolution operation. The parallel visual Mamba branch receives the second part of the features from the channel segmentation, performs layer normalization on the input features X with C channels, and uniformly segments them into four parts along the channel dimension; The four segments of features are input into the Visual State Space (VSS) blocks for processing. Taking the i-th feature as an example, the i-th feature is input into the Visual State Space block and processed sequentially through a fully connected layer, a 3×3 depthwise convolution, batch normalization, ReLU activation, two-dimensional selective scanning, and layer normalization to obtain the output feature. A learnable scaling factor is applied to the output feature and residual concatenated with the input i-th feature to output the enhanced feature. The concat operation combines the four features obtained from visual state space block processing into a channel number of... C Feature X out The global feature map is processed by layer normalization and linear projection.

[0010] Furthermore, the two-dimensional selective scanning in the visual state space block comprises three sub-steps: Scan Unfolding: The 2D features are unfolded into four independent 1D sequences along the four diagonal directions; Selective scanning spatial state sequence model block processing: Each 1D sequence is processed by a selective scanning spatial state sequence model block. The selective scanning spatial state sequence model block decides which information to remember or ignore based on the current input, and through the passing of hidden states, allows each element in the sequence to interact with all previous elements in that direction, thereby capturing long-range dependencies with linear complexity. Scan merging: The four processed 1D sequences are merged back into a 2D feature map according to their original spatial correspondence.

[0011] Furthermore, the expression for the efficient upsampling and feature enhancement process of the convolutional block is as follows: ,in, x This represents the low-resolution feature map of the input. Up() represents an upsampling operation with a scaling factor of 2. DWC() represents a 3×3 depthwise separable convolution operation. BN() represents batch normalization. ReLU() represents a non-linear activation operation. C 1×1 () indicates a 1×1 convolution operation.

[0012] Furthermore, during the deformable convolutional attention-gated feature fusion process, it receives shallow detail features g from the encoder's skip connections and deep semantic features output from the decoder's previous layer after efficient upsampling of the convolutional block. x A spatial attention map is generated adaptively to the image content through deformable operations, and the deep semantic features are then applied based on the spatial attention map. x Perform weighted fusion and output calibrated features.

[0013] Furthermore, the multi-scale convolutional attention module composition and feature multi-dimensional refinement process include: Channel attention block: Performs channel attention transformation on the input feature map, which enhances discriminative features by aggregating global information of channels and recalibrating the weights of each channel; Spatial attention block: Performs spatial attention transformation on the feature map after channel attention transformation, and generates a spatial weight map by aggregating spatial context information to focus on the target region; Efficient multi-scale convolutional blocks: Perform multi-scale convolutional transformations on feature maps that have undergone spatial attention transformations. Extract multi-scale features through at least two parallel convolutional branches with different receptive fields and fuse them to output a refined feature map that simultaneously contains local details and global context.

[0014] Furthermore, each of the four stages of the decoder has a segmentation head at the end, which is used to map the refined features into a stage-wise segmentation prediction map. The segmentation prediction maps output by the four segmentation heads are fused together by weighted summation to obtain a skin lesion segmentation map.

[0015] Furthermore, the skin lesion image segmentation network model is trained using a multi-stage deep supervision strategy to optimize model parameters, including: Obtain a training dataset containing images of skin lesions and corresponding pixel-level annotations; The training image is input into the skin lesion image segmentation network model for forward propagation, and intermediate segmentation prediction maps from four different stages are obtained through the segmentation head set in the decoder. Calculate the loss value between each intermediate segmentation prediction map and the ground truth annotation map, and then sum them using predetermined weighting coefficients to obtain the joint loss. The calculation formula is as follows: ;in, loss p1 , loss p2 , lossp 3 and lossp 4 represents the loss of four different prediction heads. α, β, γ and ζ This represents the corresponding loss weight for a single predictor head; Based on the joint loss, the gradient descent algorithm is used to update the parameters of the skin lesion image segmentation network model, and the joint loss calculation process of the training images is iteratively executed until the model converges.

[0016] In a second aspect, this invention provides a skin lesion image segmentation system based on multi-scale deformable aggregation, comprising: Image input and preprocessing module: used to acquire and preprocess the original skin lesion images to obtain standardized images; Model computation module: Used to load and run the skin lesion image segmentation network model, process the standardized images output by the image input and preprocessing module, and generate the final segmentation map; The encoder of the skin lesion image segmentation network model includes a local detail and global long-range dependency convolution-parallel visual Mamba module for collaborative extraction of input features, and the decoder includes an efficient upconvolution block for upsampling the feature map, a deformable convolutional attention gate for adaptively fusing the upsampled features with features from the corresponding stage of the encoder, a multi-dimensional and multi-scale convolutional attention module for multi-scale refinement of the fused features, and a segmentation head for generating intermediate segmentation prediction maps; The results output module is used to receive the segmentation results generated by the model calculation module and render the segmentation results into a visual segmentation image for output or storage.

[0017] Compared with the prior art, the present invention has the following beneficial effects: 1) Through the innovative convolutional-parallel visual Mamba module, the local detail perception capability of convolutional neural networks and the efficient long-range dependency modeling advantage of state space models are effectively integrated. While maintaining linear computational complexity, it unifies the collaborative understanding of the micro-texture and macro-morphology of skin lesions, and overcomes the inherent contradiction between receptive field and computational efficiency in traditional models.

[0018] 2) In response to the high heterogeneity of lesion morphology, the innovative deformable convolutional attention gate enables the network to dynamically adapt to irregular contours and blurred boundaries. Through an adaptive spatial attention mechanism, it accurately enhances the features of the lesion region and significantly improves the fit and accuracy of the segmentation boundary in complex situations.

[0019] 3) The model employs a multi-scale feature extraction and deep supervision mechanism, which integrates semantic information and detailed features at different levels. This not only optimizes the stability of the training process but also results in segmentation results with excellent robustness, achieving leading performance metrics on multiple authoritative public datasets.

[0020] 4) The hybrid architecture framework constructed in this invention exhibits good versatility and scalability. Its core design concept provides a valuable technical path for solving the common problem of balancing accuracy and efficiency in other medical image analysis tasks. Attached Figure Description

[0021] Figure 1 The diagram shows the overall network architecture of MADU-Net provided in this embodiment of the invention.

[0022] Figure 2 This is a network architecture diagram for the Conv-PVM module.

[0023] Figure 3 This is a network structure diagram of the parallel vision Mamba layer.

[0024] Figure 4 This is a network structure diagram of the visual state space block.

[0025] Figure 5 This is a flowchart of the two-dimensional selective scanning technology operation.

[0026] Figure 6 This is a network structure diagram of the deformable convolutional attention gate module.

[0027] Figure 7 This is a diagram of the network structure of deformable convolutional blocks.

[0028] Figure 8 This is a network structure diagram of the channel attention block.

[0029] Figure 9 This is a network structure diagram of a spatial attention block.

[0030] Figure 10 This is a diagram of the network structure of a multi-scale convolutional block. Detailed Implementation

[0031] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0032] This invention provides a method for skin lesion image segmentation based on multi-scale deformable aggregation, comprising the following steps: S1. Obtain a dataset of skin lesion images and preprocess the dataset to obtain a standardized image dataset; S2. Divide the standardized image dataset into a training set and a test set; S3. Construct a skin lesion image segmentation network model, input the training set to train the segmentation network model, and optimize the model parameters; S4. Apply the trained model to the test set to generate skin lesion segmentation images.

[0033] In one specific embodiment of this example, the source of the skin lesion image dataset includes: The skin lesion image datasets used in this embodiment of the invention are from the ISIC2016, ISIC 2017, and ISIC2018 datasets provided by the International Skin Imaging Collaboration (ISIC), and the PH2 dataset provided by the Dermatology Department of Matosinhos Pedro Hispano Hospital in Portugal. Specifically, the ISIC 2016 dataset contains 1279 skin lesion images, the ISIC 2017 dataset contains 2150 images, the ISIC 2018 dataset contains 2594 training images, 100 validation images, and 1000 test images, and the PH2 dataset contains 200 skin lesion images.

[0034] In one specific embodiment of this example, the preprocessing process for the skin lesion image dataset includes: Hair and artifact removal and illumination non-uniformity correction are performed on the original skin lesion images in the skin lesion image dataset. The size of the skin lesion images to be processed is uniformly adjusted to a preset resolution. In this embodiment of the invention, the resolution of all images used for training, verification and testing is uniformly adjusted to 256×256 pixels. Random data augmentation operations are performed on images that have undergone size normalization. In this embodiment of the invention, the data augmentation operations include horizontal flipping, vertical flipping, and random rotation.

[0035] In one specific embodiment of this example, the skin lesion image segmentation network model based on multi-scale deformable aggregation comprises: The skin lesion image segmentation network model designed in this invention is a U-shaped network based on a symmetric encoder-decoder architecture; wherein, the encoder is configured to extract and fuse local features and global long-range dependent features through a convolutional-parallel visual Mamba module, and the decoder is configured to progressively upsample and fuse multi-scale features from the encoder, and refine the fused features in multiple dimensions. The skin lesion image segmentation network used in the embodiments of the present invention ( Figure 1 The model consists of the following components: Symmetric Encoder-Decoder Structure: The U-shaped network's encoder and decoder each consist of four stages, forming a mirror-symmetric structure. The encoder primarily extracts features through progressive downsampling operations, with the spatial resolution of the image halved and the number of feature channels doubling progressively in each stage. The decoder restores resolution through progressive upsampling, gradually reconstructing the segmentation map. Corresponding stages of the encoder and decoder are connected via skip connections. Taking the i-th stage as an example, the output features of the i-th stage of the encoder are directly passed to the i-th stage of the decoder for feature fusion. Furthermore, the four stages of the encoder network generate four predicted feature maps. These predicted feature maps are then weighted and aggregated to generate the final predicted feature map.

[0036] The encoder's first stage includes a patch embedding layer, which transforms the preprocessed image into a feature sequence. In this embodiment, the patch embedding layer divides the input image into non-overlapping 4×4 image patches and maps the image dimension to C (C is 96 by default in this embodiment) to obtain the embedded image. The latter three stages extract features through stacked convolutional-parallel visual mamba modules. The second and third stages each contain two convolutional-parallel visual mamba (Conv-PVM) modules, and the fourth stage contains nine convolutional-parallel visual mamba (Conv-PVM) modules. Each stage of the decoder mainly includes an efficient up-convolution block (EUCB), a deformable convolutional attention gate (DCAG), a multi-scale convolutional attention module (MSCAM), and a segmentation head.

[0037] Convolutional-Parallel Visual Mamba (Conv-PVM) module: It adopts a dual-branch parallel design of Convolutional Neural Network (CNN) and State Space Model (SSM). It outputs enhanced features containing local details and global context through the local feature extraction of CNN and the global dependency modeling of State Space Model (SSM). Efficient Upconvolutional Block (EUCB): Through lightweight depthwise separable convolution and upsampling, the deep, low-resolution semantic feature maps obtained by the multi-scale convolutional attention module are enlarged, the feature map resolution is gradually restored, and the size is aligned with the skip connection features. Deformable Convolutional Attention Gating (DCAG): By adapting to the irregular shape of the lesion through deformable convolution, a spatial attention map is dynamically generated to enhance the features of the lesion area, suppress irrelevant background, and achieve accurate cross-layer feature fusion. Multi-Scale Convolutional Attention Module (MSCAM): The fused features are comprehensively refined from the channel to the spatial and then to the multi-scale context through channel attention blocks, spatial attention blocks and multi-scale convolutional blocks in sequence; Segmentation Head: At the end of each decoding stage, a segmentation head is set up. It applies 1×1 convolution to the features refined by the multi-scale convolutional attention module and maps them into a preliminary segmentation prediction map. The outputs of the segmentation heads of the four stages of the decoder are fused with deep semantics and shallow details through weighted summation to obtain a more robust and accurate skin lesion segmentation map.

[0038] In one specific implementation of this embodiment, the composition and feature extraction process of the Convolutional-Parallel Visual Mamba (Conv-PVM) module include: The Conv-PVM module innovatively introduced in this invention is a hybrid architecture that combines convolutional neural networks (CNN) and state-space models (SSM). Figure 2 In skin lesion segmentation, it can effectively capture local and global features; the Conv-PVM module mainly includes a channel segmentation unit, a dual-branch parallel processing unit, and a feature fusion unit.

[0039] To further illustrate the innovation of the Conv-PVM module proposed in this invention, a comparative analysis with existing similar technologies is conducted. Currently, existing technologies using modules similar to the Conv-PVM module mentioned in this invention include fused convolutional Mamba modules (using a state-space model) and fused convolutional modules (using a dual-branch structure). Table 1 presents a comparative analysis of the innovative differences between the Conv-PVM module and existing technology architectures. This comparative analysis of existing similar technologies demonstrates the advantages of the Conv-PVM module proposed in this invention.

[0040] Table 1. Comparative Analysis of the Differentiated Innovations of Conv-PVM Modules and Existing Technology Architectures As shown in Table 1, while existing similar technologies using fused convolutional Mamba modules combine Mamba and convolution operations, their core logic still employs a full-channel serial coupling mechanism, deeply nesting Mamba features through convolutional residual blocks. Furthermore, although similar technologies using fused convolutional modules employ parallel branching structures, their focus is on the spatial complementarity of isomorphic features (parallel operation of regular and deformable convolutions), failing to overcome the local perception limitations of convolutional operations and lacking modeling of long-range dependencies.

[0041] Unlike the technical modules proposed in the above-mentioned schemes, the Conv-PVM module proposed in this invention adopts a unique channel-level decoupling architecture: firstly, the input features are physically split into two parts through channel partitioning technology, allowing the convolutional branches to independently process fine-grained local information, while the parallel visual Mamba branches specifically model global semantics and long-range spatial dependencies using a multi-branch Mamba extraction layer structure. This design is neither the "full-channel nested superposition" in the fused convolutional Mamba module nor the "pure convolutional branch parallelism" in the fused convolutional module, but rather achieves complete separation and parallel processing of heterogeneous features at the underlying level. Even a simple combination of the ideas of the former two cannot yield the specific technical path of "channel binary splitting—local / global heterogeneous parallelism—channel shuffling deep interaction" proposed in this invention. Therefore, the Conv-PVM module proposed in this invention has significant originality.

[0042] The following details the composition of the Conv-PVM module designed in this invention and the role of each unit in the feature extraction process: Channel segmentation unit: Divides the input feature map into two parts evenly along the channel dimension; Dual-branch parallel processing unit: includes a convolutional branch and a parallel visual Mamba branch; The convolutional branch receives the first part of the features segmented by the channel segmentation unit and extracts a local feature map rich in fine-grained details. The specific feature extraction process is as follows: receiving the first part of the features from the channel segmentation, batch normalizing the input features to stabilize the output data distribution, applying a standard two-dimensional 3×3 convolutional layer to extract local feature information, applying a non-linear activation function to introduce non-linear transformation capability, applying a standard two-dimensional 3×3 convolutional layer again to extract local feature information, applying a non-linear activation function again to introduce non-linear transformation capability, and finally adjusting and integrating the features through 1×1 convolution processing to output the processed local feature map. The parallel visual Mamba branch receives the second part of the segmented features from the channel segmentation unit and extracts a global feature map with global semantic information and long-range spatial dependencies. The deep feature process of the parallel visual Mamba layer (PVM layer) is described below. Figure 3 The specific operating procedure is as follows: First, the second part of the features from the channel segmentation is received. Layer normalization is then performed on the input features to stabilize the distribution of the input data. The normalized features (with C channels) are then uniformly divided into 4 parts along the channel dimension, with each part having C / 4 channels. Specifically, this is expressed as follows: ,in, X Indicates input features, C Indicates the number of channels. LN Representation layer normalization, Sp This indicates a splitting operation. This represents the four features obtained from the segmentation operation; Subsequently, the four segments of features are input into the VSS block respectively. A learnable scaling factor is applied to the output features of the VSS block, and residual concatenation is performed with the input features of the VSS block to optimize the ability to acquire long-range spatial information. Taking the i-th feature input as an example, the specific expression is as follows: ,in, , θ The adjustment factor for the residual connection is represented by Mamba(), which represents the VSS block. Then, the four features output by the VSS block are combined into a feature X with C channels by the concat operation. out The specific expression is as follows: Where Cat represents the concatenation operation; finally, the processed global feature map is output through layer normalization (LayerNorm) and linear projection (Projection) operations, specifically expressed as follows: Where Pro represents projection operation; The VSS block feature processing process in the parallel visual Mamba branch is shown below. Figure 4 The specific operating procedure is as follows: The input features are received and channel-transformed through a 1×1 convolution, followed by a 3×3 depthwise convolution, batch normalization, and ReLU activation. A two-dimensional selective scan (SS2D) is then performed. The output of the SS2D selective scan is processed by layer normalization to obtain the output features. The output features are then added element-wise to the original input features of the VSS block to output the enhanced feature map. Because there are significant differences between 2D visual data and 1D language sequences, in this embodiment of the invention, if the VSS block directly applies the Mamba model to a flattened 2D image, the network's receptive field will be limited due to the one-dimensional sequence modeling characteristics, making it difficult to effectively establish the spatial relationships between unscanned areas of the image. Therefore, in this embodiment of the invention, the VSS block utilizes 2D selective scanning (SS2D) technology... Figure 5 The specific steps for effectively processing 2D image data are as follows: First, the 2D features are expanded into four independent 1D sequences along the four diagonal directions. Then, each 1D sequence is processed by a selective scan spatial state sequence model (S6) block. The S6 block decides which information to remember or ignore based on the current input and allows each element in the sequence to interact with all previous elements in that direction through the propagation of hidden states, thereby capturing long-range dependencies with linear complexity. Finally, the four processed 1D sequences are merged back into a 2D feature map according to their original spatial correspondence.

[0043] Feature fusion unit: The output features of the convolutional branch and the parallel visual Mamba branch are concatenated by channels, and the channel shuffling operation is used to promote the full exchange and integration of information across branches, and the output contains enhanced features that include both local details and global context.

[0044] In one specific implementation of this embodiment, the efficient upsampling and feature enhancement process of the upper convolutional block in the decoder includes: This invention employs an efficient upconvolutional block to upsample the feature maps at each stage of the decoder, while simultaneously aligning the size and resolution of the feature maps. The expression for the efficient upconvolutional block upsampling and feature enhancement process is as follows: ,in, x This represents the low-resolution feature map of the input. Up() indicates an upsampling operation with a magnification factor of 2. DWC ( ) indicates a 3×3 depth separable convolution operation. BN () represents batch normalization, ReLU() represents nonlinear activation, and C 1×1 () indicates a 1×1 convolution operation.

[0045] In one specific embodiment of this example, the feature fusion process of the deformable convolutional attention gate (DCAG) module in the decoder includes: Skin lesions often exhibit extremely irregular shapes and blurred boundaries. Traditional standard convolution, due to its fixed geometry, struggles to accurately model these complex geometric variations. This invention proposes an innovative deformable convolutional attention gate (DCAG)... Figure 6 By introducing deformable convolution, the network is allowed to adaptively adjust the position of sampling points according to the actual shape of the lesion area, thus enabling it to capture the variable contours of lesions such as melanoma as flexibly as a "rubber band".

[0046] Furthermore, DCAG exhibits strong targeting in processing the information flow between the encoder and decoder. It can utilize features from higher levels as guiding signals and learn attention coefficients to regulate information transmission at different stages. This mechanism can significantly enhance task features related to lesions while effectively suppressing irrelevant noise interference from skin texture, hair, or low-contrast areas. This attention-guided gating strategy enables dynamic alignment of semantic and detail features at different scales, and is particularly effective for segmenting samples with blurred edges.

[0047] The deformable convolutional attention gate module proposed in this invention utilizes high-level semantic features from deep layers as gating signals to adaptively adjust the cross-level information flow between the encoder and decoder, thereby significantly improving the accuracy of skin lesion segmentation. The feature fusion process of the DCAG module is as follows: Receive gate signal g and input characteristics x Wherein, the gated signal g comes from the spatially detailed feature map of the encoder's skip connections; input features x Feature maps from the previous layer of the decoder, which are upsampled and rich in high-level semantics; Calculate attention weights for different feature maps The specific calculation method is as follows: DC g ( ) and DC x ( ) all represent deformable convolution processing. BN ( ) indicates batch normalization. R( ) indicates that the ReLU activation function is used for nonlinear activation; Acquiring attention-gated features The specific calculation method is as follows: ,in, C () represents 1×1 convolution, BN() represents batch normalization. σ ( ) represents the Sigmoid activation function. This indicates element-wise multiplication, which combines the generated attention coefficient map with the original input features. x Multiply.

[0048] Deformable convolutional blocks were used in the DCAG module. Figure 7 Through deformable convolution, the DCAG module can flexibly adapt to the irregular shape and size of skin lesions, and ultimately enhance the segmentation of skin lesions by dynamically aggregating features from different scales and spatial regions.

[0049] In one specific implementation of this embodiment, the composition of the multi-scale convolutional attention module (MSCAM) in the decoder and the multi-dimensional feature refinement process include: This invention utilizes an efficient multi-scale convolutional attention module (MSCAM) in the decoder, which consists of a channel attention block (CAB), a spatial attention block (SAB), and an efficient multi-scale convolutional block (MSCB). The MSCAM focuses on the input feature map... x The processing flow is as follows: By using depthwise convolutions at multiple scales, MSCAM can extract multi-scale features, improve segmentation accuracy, and reduce computational costs.

[0050] The roles of the Channel Attention Block (CAB), Spatial Attention Block (SAB), and Efficient Multi-Scale Convolutional Block (MSCB) in the multi-scale convolutional attention module are as follows: Channel Attention Block (CAB) network structure diagram as follows Figure 8As shown, the specific steps are as follows: Average-max pooling and average-average pooling strategies are applied simultaneously to the input features to generate vectors representing the global information of each channel. The compressed vectors are then fed into a small neural network (dimensionality reduction is performed through the first conv1×1, activated with PReLU, and dimensionality is restored through the second conv1×1) for interaction and information integration. The integrated information output is then processed by the Sigmoid function to generate final channel attention weights between 0 and 1. These weights are multiplied channel-wise with the original input features to output the enhanced feature map. In skin lesion image segmentation, lesions and normal skin differ significantly in color, texture, and other features. CAB, through a channel self-attention mechanism, helps the model focus on the most discriminative feature types.

[0051] The spatial attention block (SAB) network structure diagram is as follows: Figure 9 As shown, the specific steps are as follows: Average pooling and max pooling are applied simultaneously to the feature map after channel attention enhancement, and the results are concatenated to obtain a more information-rich feature map, resulting in a dual-channel feature map. This dual-channel feature map is then processed through a standard 7x7 convolutional layer to fuse spatial context information. The convolutional output is normalized to the [0, 1] interval using the Sigmoid function to obtain a spatial attention weight map. This weight map is multiplied positionally by position with the input feature map, enhancing features in spatially important regions (values ​​close to 1) and suppressing features in unimportant regions (values ​​close to 0), thereby focusing on the target region of the feature map and outputting the spatially enhanced feature map. In skin lesion image segmentation, the lesion region typically occupies only a small portion of the image, and its boundaries are blurred. SAB can guide the model to concentrate computational resources on suspicious regions, improving boundary localization accuracy.

[0052] The structure diagram of the efficient multi-scale convolutional block (MSCB) network is shown below. Figure 10 As shown, the specific steps are as follows: The feature map enhanced with spatial attention is subjected to 1×1 convolution for feature projection and channel adjustment, followed by batch normalization and ReLU6 activation to enhance numerical stability. Core features are extracted through Multi-Scale Depthwise Aggregation (MSDA). These core features are then processed using 1×1 convolution and batch normalization (BN). Finally, the output core features are stably fused with the original input features through element-wise addition, resulting in a refined feature map that simultaneously contains local details and global context. In skin lesion image segmentation, skin lesions vary in size and shape. MSCB ensures that the module can clearly see both tiny pigment dots (small receptive field) and large lesion areas (large receptive field).

[0053] In one specific embodiment of this example, the decoder output fuses deep semantics and shallow details through a weighted summation method to obtain a more robust and accurate skin lesion segmentation map, including: In the network designed in this embodiment of the invention, each of the four stages of the decoder produces a segmentation output. Each stage has a segmentation head, which generates a segmentation logic graph from the refined features through feature mapping. The segmentation output expression generated by a single segmentation head from the refined feature mapping is as follows: ,in, x This represents a refined feature mapping. Represents a 1×1 convolution; Finally, additive aggregation is used to calculate and fuse the intermediate segmentation prediction maps of the four segmentation heads to obtain a more robust and accurate skin lesion segmentation map, expressed as: ,in, p 1. p 2. p 3 and p 4 represents the feature mapping of the four-stage segmentation head. w, x, y and z represent the weights assigned to each corresponding splitting head, respectively; where, weight w, x, y and z All are set to 1.0.

[0054] In one specific embodiment of this example, the designed skin lesion segmentation network model is trained using a multi-stage deep supervision strategy, the specific process of which includes: A training dataset containing skin lesion images and corresponding pixel-level annotations was obtained. The training datasets used in this embodiment of the invention are ISIC2016, ISIC 2017, and ISIC2018 provided by the International Society for Imaging Skin (ISIC), and the PH2 dataset provided by the Dermatology Department of Matosinhos Pedro Hispano Hospital in Portugal. The training epochs for each dataset are set as follows: ISIC 2016 dataset 150 epochs, ISIC 2017 and ISIC 2018 datasets 200 epochs each, and PH2 dataset 50 epochs. The AdamW optimizer was used with an initial learning rate of 1e-3.

[0055] During training, the AdamW optimizer is used with an initial learning rate of 1e-3 to guide the updating of the MDAU-Net model parameters. Taking one training iteration as an example, the specific training process is as follows: the training image is input into the designed skin lesion segmentation network model for forward propagation. The segmentation head set in the decoder obtains intermediate segmentation prediction maps from four different stages. The loss value between each intermediate segmentation prediction map and the ground truth annotation map is calculated separately, and then weighted and summed using predetermined weighting coefficients to obtain the joint loss. The calculation formula is as follows: ;in, loss p1 , loss p2 , lossp 3 and lossp 4 represents the loss of four different prediction heads. α, β, γ and ζ The values ​​represent the corresponding individual predictor head loss weights (α, β, γ, and ζ are all set to 1.0 in this embodiment). Based on the obtained joint loss, the parameters of the skin lesion segmentation network model are updated using the gradient descent algorithm. The joint loss calculation process for the training images is iteratively executed until the model converges.

[0056] In one specific embodiment of this invention, a skin lesion image segmentation system based on multi-scale deformable aggregation is provided, comprising: Image input and preprocessing module: used to acquire and preprocess the original skin lesion images to obtain standardized images; The model computation module is used to load and run the skin lesion segmentation network model, and to process the standardized images input from the image input and preprocessing module. The skin lesion segmentation network model is a U-shaped network based on a symmetric encoder-decoder architecture. The encoder includes a local detail and global long-range dependency convolution-parallel visual Mamba module for collaboratively extracting input features. The decoder includes an efficient upconvolution block for upsampling feature maps, a deformable convolutional attention gate for adaptively fusing upsampled features with features from the corresponding stage of the encoder, and a multi-dimensional, multi-scale convolutional attention module for refining the fused features. Each stage of the decoder has a segmentation head at its end to generate intermediate segmentation prediction maps, and multiple intermediate segmentation prediction maps are aggregated to generate the final segmentation map. The results output module is used to receive the segmentation results generated by the model calculation module and render them as a visual segmentation image for output or storage.

[0057] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for segmentation of skin lesion images based on multiscale deformable aggregation, characterized in that, The method comprises the following steps: S1, obtaining a skin lesion image dataset and preprocessing the dataset to obtain a standardized image dataset; S2, dividing the standardized image dataset into a training set and a test set; S3, constructing a skin lesion image segmentation network model, inputting the training set to train the segmentation network model, and optimizing the model parameters; S4, applying the trained model to the test set to generate a skin lesion segmentation image; In step S3, the skin lesion image segmentation network model is a symmetrical U-shaped encoding and decoding structure; the encoder extracts and fuses image features through a plurality of cascaded convolution-parallel visual mamba modules, wherein the convolution-parallel visual mamba module adopts a double-branch parallel architecture, performs local feature extraction and global dependency modeling on the input features through channel segmentation, fuses the outputs of the two branches, and generates enhanced features containing local details and overall semantics at the same time; The decoder fuses multi-scale features from the encoder through step-by-step upsampling, and refines the fused features in multiple dimensions to generate a segmentation result; wherein the decoder includes: an efficient up-convolution block for upsampling and scale alignment of deep semantic features, a deformable convolution attention gate for dynamically generating attention weights according to feature space distribution to enhance target regions and fuse cross-layer features, and a multi-scale convolution attention module for serializing and refining features through attention mechanism and multi-scale convolution.

2. The method of claim 1, wherein, The skin lesion image dataset preprocessing process comprises: Removing hair and artifacts from the original skin lesion image and correcting uneven illumination; Uniformly adjusting the size of the skin lesion image to be processed to a preset resolution; Performing random data enhancement operation on the size-normalized image, and the data enhancement operation includes horizontal flipping, vertical flipping and random rotation.

3. The method of claim 1, wherein, The convolution-parallel visual mamba module composition and feature extraction process comprises: Channel segmentation unit: uniformly segmenting the input feature map into two parts in the channel dimension; Double-branch parallel processing unit: including a convolution branch and a parallel visual mamba branch; wherein the convolution branch receives the first part of the feature and extracts a local feature map rich in fine-grained detail information, and the parallel visual mamba branch receives the second part of the feature and extracts a global feature map with global semantic information and long-range spatial dependency; Feature fusion unit: channel splicing the output features of the convolution branch and the parallel visual mamba branch, and promoting sufficient exchange and integration of cross-branch information through channel shuffling operation, outputting enhanced features containing local details and global context at the same time.

4. The method of claim 3, wherein, The double-branch parallel processing unit feature extraction process comprises: The convolution branch receives the first part of the feature from the channel segmentation, and outputs the processed local feature map through batch normalization, standard 2D 3*3 convolution layer, nonlinear activation function, standard 2D 3*3 convolution layer, nonlinear activation function, and 1*1 convolution operation; The parallel visual mamba branch receives the second part of the feature from the channel segmentation, performs layer normalization on the input feature X with channel number C, and uniformly segments it into four parts in the channel dimension; The four-part features obtained by segmentation are respectively input into the visual state space block for processing. Taking the i-th part feature processing as an example, the i-th part feature is input into the visual state space block, sequentially passes through a full connection layer, a 3*3 deep convolution, a batch normalization, a ReLU activation, a two-dimensional selective scanning and a layer normalization processing, to obtain an output feature. A learnable scaling factor is applied to the output feature, and the i-th part feature is spliced with the output feature through a residual connection, to output an enhanced feature; The four-part features processed by the visual state space block are combined into a feature X with a channel number of 2048 through a concat operation C out , and the processed global feature map is output through layer normalization and linear projection.​ 5. The method of claim 1, wherein, The high-efficiency up-convolution block up-sampling and feature enhancement process expression is: wherein x represents an input low-resolution feature map, Up() represents an up-sampling operation with a magnification factor of 2, DWC() represents a 3x3 deep separable convolution operation, BN() represents a batch normalization operation, ReLU() represents a nonlinear activation operation, C 1×1 represents a 1x1 convolution operation.

6. The method of claim 1, wherein, The deformable convolution attention gate feature fusion process comprises: receiving shallow detail features g from an encoder skip connection and deep semantic features output by an efficient up-convolution block after up-sampling of a previous layer of a decoder x , generating a spatial attention map by a deformable operation adaptive to image content, and performing weighted fusion on the deep semantic features x according to the spatial attention map to output calibrated features.

7. The method of claim 1, wherein, The multi-scale convolution attention module group and the feature multi-dimensional refining process include: The channel attention block: performing channel attention transformation on the input feature map, aggregating global information of the channels and recalibrating the channel weights, to enhance the discriminative features; The spatial attention block: performing spatial attention transformation on the feature map after the channel attention transformation, aggregating spatial context information and generating a spatial weight map, to focus on the target region; The efficient multi-scale convolution block: performing multi-scale convolution transformation on the feature map after the spatial attention transformation, extracting multi-scale features through at least two parallel convolution branches with different receptive fields and fusing the multi-scale features, to output a refined feature map containing both local details and global context.

8. The method of claim 1, wherein, A segmentation head is arranged at the end of each of the four stages of the decoder, to map the refined features into a stage segmentation prediction map. The segmentation prediction maps output by the four segmentation heads are fused through weighted summation to obtain a skin lesion segmentation map, by fusing deep semantic features and shallow details.

9. The method of claim 1, wherein, The skin lesion image segmentation network model is trained using a multi-stage deep supervision strategy to optimize the model parameters, including: obtaining a training dataset containing skin lesion images and corresponding pixel-level labels; inputting the training images into the skin lesion image segmentation network model for forward propagation, and obtaining intermediate segmentation prediction maps from the four different stages through the segmentation heads arranged in the decoder; The loss value between each intermediate segmentation prediction map and the real labeled map is calculated respectively, and the weighted sum is obtained by a predetermined weighting coefficient to obtain a joint loss, and the calculation formula is ; wherein, loss p1 , loss p2 , lossp 3 and lossp 4 represent the loss of four different prediction heads, α, β, γ and ζ represent the corresponding single prediction head loss weight. According to the joint loss, the skin lesion image segmentation network model parameters are updated using a gradient descent algorithm, and the joint loss calculation process of the training images is iteratively performed until the model converges.

10. A multiscale deformable aggregation based skin lesion image segmentation system for implementing the method of any one of claims 1-9, characterized in that, including: The image input and preprocessing module: used for obtaining and preprocessing the original skin lesion image to obtain a standardized image; The model calculation module: used for loading and running the skin lesion image segmentation network model, processing the standardized image output by the image input and preprocessing module, and generating a final segmentation map; wherein the skin lesion image segmentation network model encoder includes a local detail and global long-range dependency convolution-parallel visual mamba module for cooperatively extracting input features, the decoder includes an efficient up-convolution block for upsampling features, a deformable convolution attention gate for adaptively fusing the upsampled features with features from the corresponding stage of the encoder, a multi-scale convolution attention module for multi-dimensional refining of the fused features, and a segmentation head for generating an intermediate segmentation prediction map; The result output module is used for receiving the segmentation result generated by the model calculation module, and rendering the segmentation result into a visualized segmentation image for output or storage.

Images