Skin lesion image segmentation method and system based on multi-scale deformable aggregation

Abstract

The application provides a skin lesion image segmentation method and system based on multi-scale deformable aggregation, including: acquiring a skin lesion image dataset, dividing the dataset into a training set and a test set, constructing an improved U-shaped segmentation network model, optimizing the model parameters through training, applying the trained model to the test set, and generating high-precision segmentation images. Among them, the model adopts an innovative convolution-parallel visual mamba module in the encoder, cooperates with CNN and state space model, realizes extraction and fusion of local details and global long-range dependence of the image; a deformable convolution attention gate is proposed in the decoder, which uses deformable convolution to adaptively focus on key areas and realize cross-level feature fusion and multi-scale refinement. Through the above multi-scale deformable aggregation mechanism, the segmentation accuracy and robustness of the model for complex and diverse skin lesions are significantly improved, providing reliable assistance for clinical diagnosis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a skin lesion image segmentation method and system based on multi-scale deformable aggregation. BACKGROUND

[0002] Precise image segmentation of skin lesions is crucial for computer-aided early screening and diagnosis of skin cancer (such as melanoma). However, due to the high heterogeneity and complexity of skin lesions in morphology, size, boundary and contrast, it is still a major challenge to achieve accurate and robust automatic segmentation.

[0003] Currently, two types of architectures, convolutional neural networks and Transformers, are mainly used in the field of medical image segmentation. Convolutional neural network (CNN) models represented by U-Net are good at extracting local features, but their local receptive fields limit the ability to model long-range dependencies, making it difficult to fully understand the global context of the image. In contrast, Transformers achieve global modeling through self-attention mechanisms, but their computational complexity grows quadratically with image size, making it computationally heavy to process high-resolution medical images; at the same time, their block processing approach also easily loses pixel-level details, affecting the accuracy of the segmentation boundary.

[0004] In recent years, structured state space sequence models represented by Mamba have shown the advantage of capturing long-range dependencies with linear computational complexity in sequence modeling, providing new possibilities for overcoming the limitations of CNN locality and high computational overhead of Transformers. Therefore, a new network architecture is needed that can effectively integrate local feature extraction and efficient global modeling, significantly improving the segmentation accuracy of complex skin lesion images while ensuring computational efficiency. SUMMARY

[0005] In view of the technical problems that the convolutional neural network (CNN) in the existing skin lesion segmentation technology is difficult to model long-range dependence due to local receptive field, and the visual Transformer (ViT) is limited in efficiency due to high computational complexity of the secondary calculation of the self-attention mechanism, the present application provides a skin lesion image segmentation method and system based on multi-scale deformable aggregation, specifically, a new multi-scale deformable aggregation U-Net is designed, wherein the Conv-PVM (Convolutional-Parallel Visual Mamba) module fuses the local feature extraction capability of the convolutional neural network and the global dependence modeling advantage of the state space model, so that the network can efficiently capture local details and global context information at the same time. The deformable convolution attention gate enables the network to adaptively fit the irregular shape, variable size and complex texture of the lesion, significantly improving the model's ability to subdivide fuzzy and complex boundaries. Both work together to significantly improve the segmentation accuracy and robustness while ensuring computational efficiency.

[0006] In one aspect, the present application provides a skin lesion image segmentation method based on multi-scale deformable aggregation, comprising:

[0007] S1, obtaining a skin lesion image dataset and preprocessing the dataset to obtain a standardized image dataset;

[0008] S2, dividing the standardized image dataset into a training set and a test set;

[0009] S3, constructing a skin lesion image segmentation network model, inputting the training set to train the segmentation network model and optimizing the model parameters;

[0010] S4, applying the trained model to the test set to generate a skin lesion segmentation image;

[0011] In step S3, the skin lesion image segmentation network model is a symmetrical U-shaped encoding and decoding structure; the encoder extracts and fuses image features through multiple cascaded Conv-PVM modules, wherein the Conv-PVM module adopts a double-branch parallel architecture, extracts local features and models global dependence by splitting the input features into channels and processing them respectively, and fuses the outputs of the two branches to generate enhanced features containing local details and overall semantics;

[0012] The decoder fuses multi-scale features from the encoder by stepwise upsampling, and performs multi-dimensional refinement on the fused features to generate a segmentation result; wherein the decoder comprises: an efficient up-convolution block for upsampling and scale alignment of deep semantic features, a deformable convolution attention gate for dynamically generating attention weights according to feature space distribution to enhance target regions and fuse cross-layer features, and a multi-scale convolution attention module for serializing and refining features through attention mechanism and multi-scale convolution.

[0013] Further, the skin lesion image dataset preprocessing process comprises:

[0014] Hair and artifact removal is performed on the original skin lesion image, and illumination non-uniformity correction is performed.

[0015] The size of the skin lesion image to be processed is uniformly adjusted to a preset resolution.

[0016] Random data augmentation is performed on the size-normalized image, and the data augmentation includes horizontal flipping, vertical flipping and random rotation.

[0017] Further, the convolution-parallel visual mamba module composition and feature extraction process comprises:

[0018] Channel segmentation unit: uniformly segment the input feature map into two parts in the channel dimension;

[0019] Double-branch parallel processing unit: including convolution branch and parallel visual mamba branch; wherein the convolution branch receives the first part of the feature and extracts the local feature map rich in fine-grained detail information, and the parallel visual mamba branch receives the second part of the feature and extracts the global feature map with global semantic information and long-range spatial dependency;

[0020] Feature fusion unit: channel concatenation of the output features of the convolution branch and the parallel visual mamba branch, and channel shuffling operation to promote full exchange and integration of cross-branch information, output enhanced features containing local details and global context.

[0021] Further, the double-branch parallel processing unit performs feature extraction process, comprising:

[0022] The convolution branch receives the first part of the feature from the channel segmentation, and outputs the processed local feature map through batch normalization, standard two-dimensional 3x3 convolution layer, nonlinear activation function, standard two-dimensional 3x3 convolution layer, nonlinear activation function, and 1x1 convolution operation;

[0023] The parallel visual mamba branch receives the second part of the feature from the channel segmentation, performs layer normalization on the input feature X with channel number C, and uniformly divides it into four parts in the channel dimension;

[0024] The four-part features obtained by segmentation are respectively input into a visual state space (VSS) block for processing. Taking the i-th part feature as an example, the i-th part feature is input into the visual state space block, sequentially passes through a fully connected layer, a 3x3 deep convolution, batch normalization, ReLU activation, two-dimensional selective scanning, and layer normalization processing to obtain output features. A learnable scaling factor is applied to the output features, and the i-th part feature is concatenated with the input to output enhanced features.

[0025] The four-part features processed by the visual state space block are combined into a feature X C with a channel number of 192 through a concat operation. The feature X out is output after layer normalization and linear projection.

[0026] Further, the two-dimensional selective scanning in the visual state space block includes three sub-steps:

[0027] Scanning expansion: The 2D feature is expanded into four independent 1D sequences along four diagonal directions.

[0028] Selective scanning space state sequence model block processing: Each 1D sequence is processed by a selective scanning space state sequence model block. The selective scanning space state sequence model block decides which information to remember or ignore according to the current input and allows each element in the sequence to interact with all previous elements in the direction through the transmission of hidden states, thereby capturing long-range dependencies with linear complexity.

[0029] Scanning merging: The four processed 1D sequences are merged into a 2D feature map according to the original spatial position correspondence.

[0030] Further, the efficient up-convolution block upsampling and feature enhancement process is expressed as: wherein, x represents the input low-resolution feature map, Up() represents an up-sampling operation with a magnification factor of 2, DWC() represents a 3x3 deep separable convolution operation, BN() represents a batch normalization operation, ReLU() represents a nonlinear activation operation, and C 1×1 () represents a 1x1 convolution operation.

[0031] Further, in the deformable convolution attention gate feature fusion process, shallow detail features g from the encoder skip connection and deep semantic features x from the output of the efficient up-convolution block upsampling of the previous layer of the decoder are received. A spatial attention map is generated through deformable operation adaptive to image content, and the deep semantic features xThe weighted fusion is performed, and a calibrated feature is output.

[0032] Further, the multi-scale convolution attention module is composed and a feature multi-dimensional refining process is included.

[0033] The channel attention block: performing channel attention transformation on the input feature map, aggregating global information of channels and recalibrating channel weights to enhance discriminative features;

[0034] The spatial attention block: performing spatial attention transformation on the feature map after channel attention transformation, aggregating spatial context information and generating a spatial weight map to focus on the target region;

[0035] The efficient multi-scale convolution block: performing multi-scale convolution transformation on the feature map after spatial attention transformation, extracting multi-scale features through at least two parallel convolution branches with different receptive fields and fusing them to output a refined feature map containing both local details and global context.

[0036] Further, a segmentation head is provided at the end of each of the four stages of the decoder, for mapping the refined features into a stage segmentation prediction map, and the segmentation prediction maps output by the four segmentation heads are fused by weighted summation to obtain a skin lesion segmentation map.

[0037] Further, the skin lesion image segmentation network model is trained using a multi-stage deep supervision strategy to optimize the model parameters, including:

[0038] Obtaining a training dataset containing skin lesion images and corresponding pixel-level labels;

[0039] Inputting the training images into the skin lesion image segmentation network model for forward propagation, and obtaining the intermediate segmentation prediction maps from the four different stages through the segmentation heads provided in the decoder;

[0040] Calculating the loss value between each intermediate segmentation prediction map and the true label map, and performing weighted summation through predetermined weighting coefficients to obtain a joint loss, the calculation formula being ; wherein, loss p1 , loss p2 , lossp 3 and lossp 4 represent the losses of the four different prediction heads, α, β, γ and ζ represent the loss weights of the corresponding single prediction head;

[0041] According to the joint loss, the gradient descent algorithm is used to update the skin lesion image segmentation network model parameters, and the joint loss calculation process of the training images is iteratively performed until the model converges.

[0042] The two aspects of the present application provide a skin lesion image segmentation system based on multi-scale deformable aggregation, comprising:

[0043] An image input and preprocessing module is configured to acquire and preprocess an original skin lesion image to obtain a standardized image.

[0044] A model calculation module is configured to load and run a skin lesion image segmentation network model to process the standardized image output by the image input and preprocessing module to generate a final segmentation map.

[0045] A result output module is configured to receive the segmentation result generated by the model calculation module and render the segmentation result as a visualized segmentation image for output or storage.

[0046] Compared with the prior art, the present application has the following beneficial effects:

[0047] 1) By means of the innovative convolution-parallel visual mamba module, the local detail perception ability of the convolutional neural network and the efficient long-range dependency modeling advantage of the state space model are effectively fused, the collaborative understanding of the microscopic texture and the macroscopic morphology of the skin lesion is unified under the premise of maintaining the linear computational complexity, and the inherent contradiction between the receptive field and the computational efficiency of the traditional model is overcome.

[0048] 2) In view of the high heterogeneity of the lesion morphology, the innovative deformable convolution attention gate enables the network to dynamically adapt to irregular contours and fuzzy boundaries, accurately enhances the lesion region features through the adaptive spatial attention mechanism, and significantly improves the fitting degree and accuracy of the segmentation boundary in complex situations.

[0049] 3) The multi-scale feature extraction and deep supervision mechanism adopted by the model optimizes the stability of the training process by fusing semantic information and detailed features at different levels, and the segmentation result has excellent robustness and achieves leading performance indicators on multiple authoritative public datasets.

[0050] 4) The hybrid architecture framework constructed in the present application exhibits good universality and scalability, and its core design idea provides a valuable technical path for solving the common problem of balancing precision and efficiency in other medical image analysis tasks. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 The overall network architecture diagram of the MADU-Net provided for the embodiment of the present application.

[0052] Figure 2 The network architecture diagram of the Conv-PVM module.

[0053] Figure 3 The network structure diagram of the parallel vision mamba layer.

[0054] Figure 4 The network structure diagram of the vision state space block.

[0055] Figure 5 The operation flowchart of the two-dimensional selective scanning technology.

[0056] Figure 6 The network structure diagram of the deformable convolution attention gate module.

[0057] Figure 7 The network structure diagram of the deformable convolution block.

[0058] Figure 8 The network structure diagram of the channel attention block.

[0059] Figure 9 The network structure diagram of the spatial attention block.

[0060] Figure 10 The network structure diagram of the multi-scale convolution block. DETAILED DESCRIPTION

[0061] In order to enable the person skilled in the art to better understand the technical solutions in the specification, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments. Obviously, the described embodiments are only a part of the embodiments of the specification, not all the embodiments. Based on the embodiments in the specification, all other embodiments obtained by the person skilled in the art without creative labor should belong to the protection scope of the specification.

[0062] The embodiment of the present application provides a skin lesion image segmentation method based on multi-scale deformable aggregation, which comprises the following steps:

[0063] S1, acquiring skin lesion image data set, and pre-processing the data set to obtain a standardized image data set;

[0064] S2, dividing the standardized image data set into a training set and a test set;

[0065] S3. Construct a skin lesion image segmentation network model, input the training set to train the segmentation network model, and optimize the model parameters;

[0066] S4. Apply the trained model to the test set to generate skin lesion segmentation images.

[0067] In one specific embodiment of this example, the source of the skin lesion image dataset includes:

[0068] The skin lesion image datasets used in this embodiment of the invention are from the ISIC2016, ISIC 2017, and ISIC2018 datasets provided by the International Skin Imaging Collaboration (ISIC), and the PH2 dataset provided by the Dermatology Department of Matosinhos Pedro Hispano Hospital in Portugal. Specifically, the ISIC 2016 dataset contains 1279 skin lesion images, the ISIC 2017 dataset contains 2150 images, the ISIC 2018 dataset contains 2594 training images, 100 validation images, and 1000 test images, and the PH2 dataset contains 200 skin lesion images.

[0069] In one specific embodiment of this example, the preprocessing process for the skin lesion image dataset includes:

[0070] Hair and artifact removal and illumination non-uniformity correction are performed on the original skin lesion images in the skin lesion image dataset.

[0071] The size of the skin lesion images to be processed is uniformly adjusted to a preset resolution. In this embodiment of the invention, the resolution of all images used for training, verification and testing is uniformly adjusted to 256×256 pixels.

[0072] Random data augmentation operations are performed on images that have undergone size normalization. In this embodiment of the invention, the data augmentation operations include horizontal flipping, vertical flipping, and random rotation.

[0073] In one specific embodiment of this example, the skin lesion image segmentation network model based on multi-scale deformable aggregation comprises:

[0074] The skin lesion image segmentation network model designed in this invention is a U-shaped network based on a symmetric encoder-decoder architecture; wherein, the encoder is configured to extract and fuse local features and global long-range dependent features through a convolutional-parallel visual Mamba module, and the decoder is configured to progressively upsample and fuse multi-scale features from the encoder, and refine the fused features in multiple dimensions.

[0075] The skin lesion image segmentation network used in the embodiments of the present invention ( Figure 1 The model consists of the following components:

[0076] Symmetrical encoding-decoding structure: the encoder and the decoder in the U-shaped network each contain four stages, forming mirror symmetry, the encoder mainly extracts features through progressive down-sampling operations, in each stage, the spatial resolution of the image is halved and the number of feature channels is multiplied; the decoder restores the resolution through progressive up-sampling and gradually reconstructs the segmentation map; the corresponding stages of the encoder and the decoder are connected through a skip connection, taking the i-th stage as an example, the output features of the i-th stage of the encoder are directly transmitted to the i-th stage of the decoder for feature fusion; in addition, the four stages of the encoder network generate four prediction feature maps. The final prediction feature map is generated after the weighted aggregation of the prediction feature maps.

[0077] The first stage of the encoder includes a patch embedding layer for converting the preprocessed image into a feature sequence, wherein the patch embedding layer of the embodiment of the application divides the input image into non-overlapping image blocks of size 4x4 and maps the dimensions of the image to C (C is 96 by default in the embodiment of the application), obtaining an embedded image; the last three stages extract features through stacked convolution-parallel visual mamba modules, wherein the second stage and the third stage each include 2 convolution-parallel visual mamba (Conv-PVM) modules, and the fourth stage includes 9 convolution-parallel visual mamba (Conv-PVM) modules; each stage of the decoder mainly includes an efficient up-convolution block (EUCB), a deformable convolution attention gate (DCAG), a multi-scale convolutional attention module (MSCAM), and a segmentation head.

[0078] Convolution-parallel visual mamba (Conv-PVM) module: adopts a dual-branch parallel design of convolutional neural network (CNN) and state space model (SSM), and outputs enhanced features containing local details and global context through local feature extraction of CNN and global dependency modeling of state space model (SSM);

[0079] Efficient up-convolution block (EUCB): through lightweight depth separable convolution and up-sampling, the deep and low-resolution semantic feature map obtained by the multi-scale convolutional attention module is enlarged, the feature map resolution is gradually restored, and the size is aligned with the skip connection features;

[0080] Deformable Convolution Attention Gate (DCAG): dynamically generate spatial attention map by deformable convolution to adapt to irregular shape of lesion, enhance lesion area features and suppress irrelevant background, and realize precise cross-layer feature fusion.

[0081] Multi-Scale Convolution Attention Module (MSCAM): sequentially pass through channel attention block, spatial attention block and multi-scale convolution block to comprehensively refine the fused features from channel to space and then to multi-scale context.

[0082] Segmentation head: a segmentation head is arranged at the end of each decoding stage, and 1x1 convolution is applied to the features refined by the multi-scale convolution attention module to map them to preliminary segmentation prediction maps; wherein the outputs of the segmentation heads of the four stages of the decoder are fused by weighted summation to obtain more robust and accurate skin lesion segmentation maps.

[0083] In one specific embodiment of the present embodiment, the Conv-PVM module composition and feature extraction process include:

[0084] The Conv-PVM module innovatively introduced by the present application is a hybrid architecture combining convolutional neural network (CNN) and state space model (SSM) Figure 2 , which can effectively capture local and global features in skin lesion segmentation; wherein the Conv-PVM module mainly includes a channel segmentation unit, a double-branch parallel processing unit and a feature fusion unit.

[0085] To further illustrate the innovation of the Conv-PVM module proposed by the present application, the existing similar technologies are compared and analyzed. The modules used in the prior art similar to the Conv-PVM module mentioned in the present application include fusion convolution Mamba module (using state space model) and fusion convolution module (using double-branch structure), and Table 1 is a comparison and analysis table of the differential innovation of the Conv-PVM module and the existing technology architecture. The advantages of the Conv-PVM module proposed by the present application are illustrated by comparing and analyzing the existing similar technologies.

[0086] Table 1 Comparison and analysis table of differential innovation of Conv-PVM module and existing technology architecture

[0087]

[0088] As can be seen from Table 1, in the prior art, although the similar technology using the convolution fusion mamba module combines the mamba and convolution operations, the core logic still adopts a full-channel serial coupling mechanism to perform deep nested processing on the mamba features through a convolution residual block. In addition, although the similar technology using the convolution fusion module adopts a parallel branch structure, the focus is on the spatial complementarity of isomorphic features (parallel of conventional convolution and deformable convolution), and it does not break through the local perception limitation of convolution operation and lacks modeling of long-range dependence.

[0089] Unlike the technical modules proposed in the above solutions, the Conv-PVM module proposed in the present application adopts a unique channel-level decoupling architecture: first, the input features are split into two parts in the physical dimension through a channel division technique, so that the convolution branch independently processes fine-grained local information, and the parallel visual mamba branch specially models global semantics and long-range spatial dependence with the help of a multi-branch mamba extraction layer structure. This design is neither the “full-channel nested superposition” in the convolution fusion mamba module nor the “pure convolution branch parallel” in the convolution fusion module, but rather a complete separation and parallel processing of heterogeneous features at the bottom. Even if the ideas of the former two are simply combined, the specific technical path of “channel bisection-local / global heterogeneous parallel-channel shuffling deep interaction” proposed in the present application cannot be obtained, so the Conv-PVM module proposed in the present application has significant originality.

[0090] The following specifically describes the composition of the Conv-PVM module designed in the present application and the role of each unit in the feature extraction process:

[0091] Channel segmentation unit: uniformly segmenting the input feature map in the channel dimension into two parts;

[0092] Double-branch parallel processing unit: including a convolution branch and a parallel visual mamba branch;

[0093] Among them, the convolution branch receives the first part of the features segmented by the channel segmentation unit and extracts a local feature map rich in fine-grained detail information; the feature extraction specific process is: receiving the first part of the features segmented by the channel segmentation unit, performing batch normalization on the input features to stabilize the data distribution of the output, applying a standard two-dimensional 3x3 convolution layer to extract local feature information, applying a nonlinear activation function to introduce nonlinear transformation capability, again applying a standard two-dimensional 3x3 convolution layer to extract local feature information, again applying a nonlinear activation function to introduce nonlinear transformation capability, and finally adjusting and integrating the features through a 1x1 convolution processing to output the processed local feature map;

[0094] The parallel visual mamba branch receives the second part of the features segmented by the channel segmentation unit and extracts a global feature map with global semantic information and long-range spatial dependence relationship, wherein the deep feature process of the parallel visual mamba layer (PVM layer) is as follows:Figure 3 The specific operation process is as follows:

[0095] First, the second part of the feature from the channel segmentation is received, the input feature is layer normalized to stabilize the distribution of the input data, and the layer normalized feature (C channels) is uniformly segmented into 4 parts in the channel dimension, and the channel number of each part of the feature is C / 4. The specific expression is as follows: Wherein, X represents the input feature, C represents the number of channels, LN represents layer normalization, Sp represents a segmentation operation, represents four parts of features obtained by the segmentation operation;

[0096] Then, the four part features segmented are input into the VSS block, a learnable scaling factor is applied to the output feature of the VSS block, and the input of the VSS block is residual concatenated to optimize the remote spatial information acquisition capability. Taking the i-th part of the feature input as an example, the specific expression is as follows: Wherein, , θ represents the adjustment factor of residual connection, and Mamba() represents the VSS block; then, the four features output by the VSS block are combined into a feature X with C channels through the concat operation out The specific expression is as follows: Wherein, Cat represents the concatenation operation; finally, the processed global feature map is output through layer normalization (LayerNorm) and linear projection (Projection) operation. The specific expression is as follows: Wherein, Pro represents the projection operation.

[0097] The VSS block feature processing process in the parallel visual Mamba branch is shown in Figure 4 The specific operation process is as follows:

[0098] The input feature is received, a 1x1 convolution is performed for channel transformation, 3x3 deep convolution, batch normalization and ReLU activation are performed in turn, two-dimensional selective scanning (SS2D, 2D Selective Scan) is performed, the output of the two-dimensional selective scanning is layer normalized to obtain the output feature, and the output feature is element-wise added to the original input feature of the VSS block to output the enhanced feature map.

[0099] Due to the significant difference between 2D visual data and 1D language sequence, in the embodiment of the present application, if the VSS block directly applies the Mamba model to the flattened two-dimensional image, the network receptive field will be limited due to the characteristics of one-dimensional sequence modeling, making it difficult to effectively establish the spatial relationship between the unscanned areas of the image. Therefore, in the embodiment of the present application, the VSS block effectively processes the 2D image data by using a 2D selective scanning (SS2D) technology Figure 5 ), and the specific operation steps are as follows:

[0100] First, the 2D features are unfolded into 4 independent 1D sequences along the four diagonal directions; then each 1D sequence is processed by a selective scanning spatial state sequence model (S6) block, which will decide to remember or ignore some information according to the current input, and through the transmission of hidden states, each element in the sequence can interact with all previous elements in this direction, so as to capture long-range dependencies with linear complexity; finally, the processed 4 1D sequences are merged into a 2D feature map according to the original spatial position correspondence.

[0101] Feature fusion unit: the output features of the convolution branch and the parallel visual Mamba branch are spliced in the channel, and the channel shuffling operation is performed to promote the full exchange and integration of cross-branch information, and the enhanced features containing local details and global context are output.

[0102] In one specific embodiment of the present embodiment, the efficient up-convolution block upsampling and feature enhancement process in the decoder includes:

[0103] In the embodiment of the present application, an efficient up-convolution block is used to perform upsampling operation on the feature maps of each stage of the decoder, and the size and resolution of the feature maps are aligned, and the efficient up-convolution block upsampling and feature enhancement process expression is: , wherein, x represents the input low-resolution feature map, Up represents the up-sampling operation with a magnification factor of 2, DWC represents a 3x3 depth separable convolution operation, BN represents a batch normalization operation, and ReLU() represents a nonlinear activation operation, C 1×1 represents a 1x1 convolution operation.

[0104] In one specific embodiment of the present embodiment, the feature fusion process of the deformable convolution attention gate (DCAG) module in the decoder includes:

[0105] Skin lesions usually have extremely irregular shapes and blurred boundaries. Traditional standard convolution is difficult to accurately model these complex geometric changes due to its fixed geometric structure. An innovative deformable convolution attention gate (DCAG) is proposed in the present applicationFigure 6 By introducing deformable convolution, the network can adaptively adjust the position of the sampling points according to the actual shape of the lesion area, so as to flexibly capture the variable contour of the melanoma lesion like a rubber band.

[0106] In addition, DCAG shows strong pertinence when processing the information flow between the encoder and the decoder. It can use the features from the higher level as a guide signal to learn the attention coefficient to adjust the information transmission at different stages. This mechanism can significantly enhance the task-related features related to the lesion, while effectively suppressing irrelevant noise interference generated by skin texture, hair or low-contrast areas. The above attention-guided gating strategy realizes the dynamic alignment of semantic features and detail features at different scales, which is particularly effective for samples with fuzzy edges.

[0107] The deformable convolution attention gate module proposed in the application can significantly improve the accuracy of skin lesion segmentation by using high-level semantic features from the deep layer as a gating signal to adaptively adjust the cross-level information flow between the encoder and the decoder. The feature fusion process of the DCAG module is as follows:

[0108] Receive gating signal g and input feature x : Wherein, the gating signal g comes from the feature map rich in spatial details of the encoder skip connection; the input feature x comes from the feature map rich in high-level semantics after upsampling of the previous layer of the decoder;

[0109] Calculate the attention weight of different feature maps , the specific calculation method is: , wherein, DC g () and DC x () both represent deformable convolution processing, BN () represents batch normalization, R () represents nonlinear activation using ReLU activation function;

[0110] Get attention gate features , the specific calculation method is: , wherein, C () represents 1x1 convolution, BN() represents batch normalization, σ () represents Sigmoid activation function, represents element-wise multiplication, and the generated attention coefficient map is multiplied with the original input feature x .

[0111] The deformable convolution block (DC Figure 7), through deformable convolution, the DCAG module can be flexibly adapted to the irregular shape and size of skin lesions, and finally through dynamic aggregation of features from different scales and spatial regions to enhance the segmentation of skin lesions.

[0112] In one specific embodiment of the present embodiment, the multi-scale convolution attention module (MSCAM) in the decoder is composed of and the feature multi-dimensional refining process includes:

[0113] The present application uses an efficient multi-scale convolution attention module (MSCAM) in the decoder, which is composed of a channel attention block (CAB), a spatial attention block (SAB), and an efficient multi-scale convolution block (MSCB); wherein the multi-scale convolution attention module (MSCAM) extracts multi-scale features from the input feature map x The processing flow is: Because of the use of deep convolution in multiple scales, MSCAM can extract multi-scale features, improve segmentation accuracy, and reduce computational cost.

[0114] The channel attention block (CAB), the spatial attention block (SAB), and the efficient multi-scale convolution block (MSCB) in the multi-scale convolution attention module have the following effects:

[0115] The network structure diagram of the channel attention block (CAB) is as Figure 8 shown, and the specific steps are: using average-maximum pooling and average-average pooling strategy on the input feature at the same time, generating a vector representing the global information of each channel, sending the compressed vector into a small neural network (reducing dimension through the first conv1×1, using PReLU activation, restoring dimension through the second conv1×1) for interaction and information integration, outputting the integrated information through the Sigmoid function to generate the final channel attention weight between 0 and 1, multiplying the weight with the original input feature by channel, and outputting the channel-enhanced feature map. In skin lesion image segmentation, the lesion and normal skin have significant differences in color, texture and other features, and CAB helps the model focus on the most discriminative feature types through channel self-attention mechanism.

[0116] The network structure diagram of the spatial attention block (SAB) is as Figure 9As shown, the specific steps are: the feature map after the channel attention enhancement is simultaneously subjected to average pooling and maximum pooling, and the results are spliced to obtain a feature map with more information, a double-channel feature map is obtained, a standard 7x7 convolution layer is used for processing to fuse the spatial context information; the convolution output is normalized to the [0, 1] interval through the Sigmoid function to obtain a spatial attention weight map, the weight map is multiplied with the input feature map at each position to enhance the features of the important regions (value close to 1) in space and suppress the features of the unimportant regions (value close to 0), so as to focus on the target region of the feature map and output the feature map after spatial enhancement. In the skin lesion image segmentation, the skin lesion region usually only occupies a very small part of the image, and the boundary is blurred. SAB can guide the model to concentrate computing resources in the suspicious region, thereby improving the boundary positioning accuracy.

[0117] The high-efficiency multi-scale convolution block (MSCB) network structure diagram is as shown in Figure 10 As shown, the specific steps are: the feature map after the spatial attention enhancement is subjected to feature projection and channel adjustment using 1x1 convolution, then the value stability is enhanced through batch normalization and ReLU6 activation, the core features are extracted through multi-scale depth aggregation (MSDA, Multi-Scale Depthwise Aggregation), the core features are processed through 1x1 convolution and batch normalization (BN), and the output core features are stably fused with the original input features through element-by-element addition to obtain a refined feature map containing both local details and global context. In the skin lesion image segmentation, the size and morphology of the skin lesion are variable. MSCB can ensure that the module can not only see the tiny pigment spots (small receptive field), but also grasp the large lesion area (large receptive field).

[0118] In one specific embodiment of the present embodiment, the output of the decoder is fused by weighted summation of deep semantic and shallow details to obtain a more robust and more accurate skin lesion segmentation map, including:

[0119] In the network designed by the embodiment of the present application, four stages of the decoder will produce segmentation outputs, and each stage has a segmentation head to generate a segmentation logic diagram from the refined features through feature mapping, and the expression of the segmentation output generated by a single segmentation head from the refined feature mapping is: wherein, x represents the refined feature mapping, represents 1x1 convolution;

[0120] Finally, the intermediate segmentation prediction map of the four segmentation heads is calculated and fused by additive aggregation to obtain a more robust and more accurate skin lesion segmentation map, and the expression is: wherein, p 1、 p 2、p 3 and p 4 represents the feature mapping of the four-stage segmentation head. w, x, y and z represent the weights assigned to each corresponding splitting head, respectively; where, weight w, x, y and z All are set to 1.0.

[0121] In one specific embodiment of this example, the designed skin lesion segmentation network model is trained using a multi-stage deep supervision strategy, the specific process of which includes:

[0122] A training dataset containing skin lesion images and corresponding pixel-level annotations was obtained. The training datasets used in this embodiment of the invention are ISIC2016, ISIC 2017, and ISIC2018 provided by the International Society for Imaging Skin (ISIC), and the PH2 dataset provided by the Dermatology Department of Matosinhos Pedro Hispano Hospital in Portugal. The training epochs for each dataset are set as follows: ISIC 2016 dataset 150 epochs, ISIC 2017 and ISIC 2018 datasets 200 epochs each, and PH2 dataset 50 epochs. The AdamW optimizer was used with an initial learning rate of 1e-3.

[0123] During training, the AdamW optimizer is used with an initial learning rate of 1e-3 to guide the updating of the MDAU-Net model parameters. Taking one training iteration as an example, the specific training process is as follows: the training image is input into the designed skin lesion segmentation network model for forward propagation. The segmentation head set in the decoder obtains intermediate segmentation prediction maps from four different stages. The loss value between each intermediate segmentation prediction map and the ground truth annotation map is calculated separately, and then weighted and summed using predetermined weighting coefficients to obtain the joint loss. The calculation formula is as follows: ;in, loss p1 , loss p2 , lossp 3 and lossp 4 represents the loss of four different prediction heads. α, β, γ and ζ The values ​​represent the corresponding individual predictor head loss weights (α, β, γ, and ζ are all set to 1.0 in this embodiment). Based on the obtained joint loss, the parameters of the skin lesion segmentation network model are updated using the gradient descent algorithm. The joint loss calculation process for the training images is iteratively executed until the model converges.

[0124] In one specific embodiment of this invention, a skin lesion image segmentation system based on multi-scale deformable aggregation is provided, comprising:

[0125] An image input and preprocessing module is configured to acquire and preprocess an original skin lesion image to obtain a standardized image.

[0126] A model calculation module is configured to load and run a skin lesion segmentation network model to process the standardized image input by the image input and preprocessing module. The skin lesion segmentation network model is a U-shaped network based on a symmetric encoder-decoder architecture. The encoder includes a local detail and global long-range dependence convolution-parallel visual mamba module for collaborative extraction of input features. The decoder includes an efficient up-convolution block for up-sampling feature maps, a deformable convolution attention gate for adaptive fusion of up-sampled features and features from the corresponding stage of the encoder, and a multi-dimensional refining multi-scale convolution attention module for refining the fused features. A segmentation head is provided at the end of each stage of the decoder to generate an intermediate segmentation prediction map, and multiple intermediate segmentation prediction maps are aggregated to generate a final segmentation map.

[0127] A result output module is configured to receive the segmentation result generated by the model calculation module and render it as a visualized segmentation image for output or storage.

[0128] The above-described embodiments are merely used to illustrate the technical solutions of the present application, rather than limit the same. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalent technical features. Such modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments.

Claims

1. A skin lesion image segmentation method based on multi-scale deformable aggregation, characterized in that, include: S1. Obtain a dataset of skin lesion images and preprocess the dataset to obtain a standardized image dataset; S2. Divide the standardized image dataset into a training set and a test set; S3. Construct a skin lesion image segmentation network model, input the training set to train the segmentation network model, and optimize the model parameters; S4. Apply the trained model to the test set to generate skin lesion segmentation images; In step S3, the skin lesion image segmentation network model is a symmetrical U-shaped encoding and decoding structure. The encoder extracts and fuses image features through multiple cascaded convolutional-parallel visual Mamba modules. The convolutional-parallel visual Mamba module adopts a dual-branch parallel architecture. It performs channel segmentation on the input features and performs local feature extraction and global dependency modeling respectively. It then fuses the outputs of the two branches to generate enhanced features that simultaneously contain local details and overall semantics. The decoder refines the fused features from the encoder by progressively upsampling and fusing them in multiple dimensions to generate segmentation results. The decoder includes: an efficient upconvolutional block for upsampling and scale alignment of deep semantic features; a deformable convolutional attention gate for dynamically generating attention weights based on the feature space distribution to enhance the target region and fuse cross-layer features; and a multi-scale convolutional attention module for serializing and refining features through attention mechanisms and multi-scale convolutions. In response to the high heterogeneity of skin lesion morphology, the feature fusion process of the deformable convolutional attention gate includes: receiving shallow detail features from the encoder's skip connections and deep semantic features from the decoder's previous layer after efficient upsampling of the upper convolutional block. x A spatial attention map is generated adaptively to the image content through deformable convolution operations, and deep semantic features are then applied based on the spatial attention map. x Weighted fusion is performed to dynamically adapt to the irregular contours and blurred boundaries of skin lesions, and the output is a calibration feature enhanced by a self-attention mechanism.

2. The method according to claim 1, characterized in that, The preprocessing process for skin lesion image datasets includes: Hair and artifact removal were performed on the original skin lesion images, and illumination inhomogeneity correction was applied. Adjust the size of the skin lesion images to be processed to the preset resolution; The image is normalized in size and then subjected to random data augmentation operations, including horizontal flipping, vertical flipping, and random rotation.

3. The method according to claim 1, characterized in that, The convolutional-parallel visual Mamba module composition and feature extraction process include: Channel segmentation unit: Divides the input feature map into two parts evenly along the channel dimension; The dual-branch parallel processing unit includes a convolutional branch and a parallel visual Mamba branch. The convolutional branch receives the first part of the features and extracts local feature maps rich in fine-grained details, while the parallel visual Mamba branch receives the second part of the features and extracts global feature maps with global semantic information and long-range spatial dependencies. Feature fusion unit: The output features of the convolutional branch and the parallel visual Mamba branch are concatenated by channels, and the channel shuffling operation is used to promote the full exchange and integration of information across branches, and the output contains enhanced features that include both local details and global context.

4. The method according to claim 3, characterized in that, The dual-branch parallel processing unit performs the feature extraction process, including: The convolutional branch receives the first part of the features from the channel segmentation, and outputs the processed local feature map through batch normalization, standard 2D 3×3 convolutional layer, non-linear activation function, standard 2D 3×3 convolutional layer, non-linear activation function, and 1×1 convolution operation. The parallel visual Mamba branch receives the second part of the features from the channel segmentation, performs layer normalization on the input features X with C channels, and uniformly segments them into four parts along the channel dimension; The four segments of features are input into the visual state space blocks for processing. Taking the i-th feature as an example, the i-th feature is input into the visual state space block and sequentially processed through a fully connected layer, a 3×3 depthwise convolution, batch normalization, ReLU activation, two-dimensional selective scanning, and layer normalization to obtain the output feature. A learnable scaling factor is applied to the output feature and residual concatenated with the input i-th feature to output the enhanced feature. The concat operation combines the four features obtained from visual state space block processing into a channel number of... C Feature X out The global feature map is processed by layer normalization and linear projection.

5. The method according to claim 1, characterized in that, The expression for the efficient upsampling and feature enhancement process of the convolutional block is as follows: ,in, x This represents the low-resolution feature map of the input. Up() represents an upsampling operation with a scaling factor of 2. DWC() represents a 3×3 depthwise separable convolution operation. BN() represents batch normalization. ReLU() represents a non-linear activation operation. C 1×1 () represents a 1×1 convolution operation.

6. The method according to claim 1, characterized in that, The multi-scale convolutional attention module composition and feature multi-dimensional refinement process include: Channel attention block: Performs channel attention transformation on the input feature map, which enhances discriminative features by aggregating global information of channels and recalibrating the weights of each channel; Spatial attention block: Performs spatial attention transformation on the feature map after channel attention transformation, and generates a spatial weight map by aggregating spatial context information to focus on the target region; Efficient multi-scale convolutional blocks: Perform multi-scale convolutional transformations on feature maps that have undergone spatial attention transformations. Extract multi-scale features through at least two parallel convolutional branches with different receptive fields and fuse them to output a refined feature map that simultaneously contains local details and global context.

7. The method according to claim 1, characterized in that, Each of the four stages of the decoder has a segmentation head at the end, which is used to map the refined features into a stage-specific segmentation prediction map. The segmentation prediction maps output by the four segmentation heads are fused together by weighted summation to obtain a skin lesion segmentation map.

8. The method according to claim 1, characterized in that, The skin lesion image segmentation network model is trained using a multi-stage deep supervision strategy to optimize model parameters, including: Obtain a training dataset containing images of skin lesions and corresponding pixel-level annotations; The training image is input into the skin lesion image segmentation network model for forward propagation, and intermediate segmentation prediction maps from four different stages are obtained through the segmentation head set in the decoder. Calculate the loss value between each intermediate segmentation prediction map and the ground truth annotation map, and then sum them using predetermined weighting coefficients to obtain the joint loss. The calculation formula is as follows: ;in, loss p1 , loss p2 , lossp 3 and lossp 4 represents the loss of four different prediction heads. α, β, γ and ζ This represents the corresponding loss weight for a single predictor head; Based on the joint loss, the gradient descent algorithm is used to update the parameters of the skin lesion image segmentation network model, and the joint loss calculation process of the training images is iteratively executed until the model converges.

9. A skin lesion image segmentation system based on multi-scale deformable aggregation, used to implement the method according to any one of claims 1-8, characterized in that, include: Image input and preprocessing module: used to acquire and preprocess the original skin lesion images to obtain standardized images; Model computation module: Used to load and run the skin lesion image segmentation network model, process the standardized images output by the image input and preprocessing module, and generate the final segmentation map; The encoder of the skin lesion image segmentation network model includes a local detail and global long-range dependency convolution-parallel visual Mamba module for collaborative extraction of input features, and the decoder includes an efficient upconvolution block for upsampling the feature map, a deformable convolutional attention gate for adaptively fusing the upsampled features with features from the corresponding stage of the encoder, a multi-dimensional and multi-scale convolutional attention module for multi-scale refinement of the fused features, and a segmentation head for generating intermediate segmentation prediction maps; The results output module is used to receive the segmentation results generated by the model calculation module and render the segmentation results into a visual segmentation image for output or storage.

Images