A lightweight diabetic foot ulcer image segmentation method and system

Abstract

The present invention discloses a lightweight diabetic foot ulcer image segmentation method and system, which relates to the field of medical image processing technology. The specific steps are: obtaining an image to be segmented, preprocessing the image to be segmented, and obtaining a preprocessed image; inputting the preprocessed image into a pre-trained diabetic foot ulcer image segmentation model to obtain a segmented image; wherein the diabetic foot ulcer image segmentation model adopts an MC-UNet network architecture, including an encoder, an LSDE module and a decoder connected in sequence, and the LSDE module introduces a channel and spatial attention mechanism and a multi-head cross-axis attention mechanism to capture long-distance dependencies in the preprocessed image and enhance feature expression. The present invention uses a visual state space module group to perform multi-scale feature extraction, and provides an adaptive solution for the diversity of diabetic foot ulcer area size and morphology, thereby significantly improving segmentation accuracy and stability.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and more particularly to a lightweight diabetic foot ulcer image segmentation method and system. Background Art

[0002] At present, early diagnosis and treatment of diabetic foot ulcers are crucial to prevent the disease from worsening. Medical image segmentation technology has made significant progress in clinical applications, especially driven by deep learning. Image segmentation methods provide strong support for solving medical image analysis problems. Deep learning-based methods can provide relatively accurate segmentation results by efficiently extracting and learning features from diabetic foot ulcer images, thereby assisting doctors to quickly and accurately detect and diagnose ulcer areas.

[0003] However, existing segmentation methods still face many challenges when processing diabetic foot ulcer images: the large changes in the size of the ulcer area make it difficult for traditional methods to adapt to these changes; the blurred boundaries make the ulcer area easily confused with the surrounding tissue, increasing the risk of missed segmentation or mis-segmentation; background interference and image quality differences reduce the robustness of existing methods; in addition, the large number of model parameters limits its application on mobile or embedded devices. These problems affect the accuracy and real-time performance of diabetic foot ulcer image segmentation.

[0004] Therefore, developing a segmentation method and system that can solve the shortcomings of existing methods in terms of segmentation accuracy, boundary clarity and robustness is an issue that needs to be urgently addressed by those skilled in the art. Summary of the invention

[0005] In view of this, the present invention provides a lightweight diabetic foot ulcer image segmentation method and system, which overcomes the above-mentioned defects.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A lightweight diabetic foot ulcer image segmentation method, the specific steps are:

[0008] Acquire an image to be segmented, and preprocess the image to be segmented to obtain a preprocessed image;

[0009] The preprocessed image is input into a pre-trained diabetic foot ulcer image segmentation model to obtain a segmented image; wherein the diabetic foot ulcer image segmentation model adopts an MC-UNet network architecture, including an encoder, an LSDE module and a decoder connected in sequence, and the LSDE module introduces a channel and spatial attention mechanism and a multi-head cross-axis attention mechanism to capture long-distance dependencies in the preprocessed image and enhance feature expression.

[0010] Optionally, the step of acquiring the pre-processed image is: using a bilinear interpolation method to process the image to be segmented into the pre-processed image of a preset resolution.

[0011] Optionally, the encoder includes a plurality of visual state space module groups and a plurality of block merging operations, and the processing steps are:

[0012] Performing multi-scale feature extraction on the pre-processed image based on the visual state space module group;

[0013] The extracted multi-scale features are downsampled according to the block merging operation to generate multi-scale extracted features.

[0014] Optionally, the visual state space module group includes at least one visual state space module, and the visual state space module includes layer normalization, two parallel branches and a linear layer connected in sequence. The preprocessed image enters the parallel branch after the layer normalization processing, and is sequentially processed by the first linear layer and the first activation function in the first branch to generate a first extracted feature; in the second branch, it is sequentially processed by the second linear layer, depthwise separable convolution, SS2D module, and layer normalization to generate a second extracted feature; the first extracted feature and the second extracted feature are multiplied element by element, and then processed by the linear layer and added to the preprocessed image to obtain the extracted feature.

[0015] Optionally, the data processing steps in the LSDE module are:

[0016] The multi-scale extracted features are weighted combined through the channel and spatial attention mechanism to further capture long-distance dependencies and generate a feature map ;

[0017] Based on the feature map by acting on the 1D convolution of the horizontal and vertical axes Extract multi-scale context information separately;

[0018] According to the multi-scale context information, the multi-head cross-axis attention mechanism is used to calculate the cross-axis attention along the x-axis and the y-axis to generate multi-scale features. and multi-scale features ;

[0019] For the multi-scale features And the multi-scale features After convolution processing, it is combined with the feature map Add them together to generate an enhanced feature map.

[0020] Optionally, the feature map The steps to obtain are:

[0021] The multi-scale extracted features are processed by the channel space attention module and the spatial attention module in sequence, and then normalized and 1×1 convolution is performed to unify the dimensions of the multi-scale extracted features to generate the feature map .

[0022] Optionally, the expression of the multi-scale context information is:

[0023] ;

[0024] ;

[0025] In the formula, Indicates normalization processing; In represents the convolution along the water direction, Indicates the size of the convolution kernel; In represents convolution along the vertical direction, Indicates the size of the convolution kernel; Indicates a 1×1 convolution.

[0026] Optionally, the expression of the multi-head cross-axis attention mechanism is:

[0027] ;

[0028] ;

[0029] In the formula, and Respectively Axis and Multi-head cross-axis attention calculation in the axis direction; It is the multi-scale context information in the vertical direction; It is the multi-scale context information in the horizontal direction.

[0030] Optionally, the data processing steps in the decoder are:

[0031] Splitting the enhanced feature graph into a plurality of enhanced feature sub-graphs;

[0032] Inputting the plurality of enhanced feature sub-graphs into different depth-separable convolution modules respectively to obtain local information and global information at different scales;

[0033] The local information and global information at different scales are cascaded along the channel dimension to form a stitched image;

[0034] The spliced ​​image is fused with the enhanced feature map to obtain a fused image.

[0035] A lightweight diabetic foot ulcer image segmentation system, comprising an image acquisition module, a model building module and a recognition module;

[0036] The image acquisition module is used to acquire diabetic foot ulcer images and construct a training data set;

[0037] The model building module is used to build an initial diabetic foot ulcer image segmentation model based on the MC-UNet network architecture, and iteratively train the initial diabetic foot ulcer image segmentation model using the training data set to obtain the diabetic foot ulcer image segmentation model; the initial diabetic foot ulcer image segmentation model includes an encoder, an LSDE module and a decoder connected in sequence, the LSDE module introduces a channel and spatial attention mechanism and a multi-head cross-axis attention mechanism to capture long-distance dependencies in the training data and enhance feature expression, and the training data comes from the training data set;

[0038] The recognition module is used to obtain the image to be recognized, input the image to be recognized into the diabetic foot ulcer image segmentation and recognition model, and obtain a binary segmentation result.

[0039] It can be seen from the above technical solutions that the present invention provides a lightweight diabetic foot ulcer image segmentation method and system, which has the following beneficial effects compared with the prior art:

[0040] Accurate multi-scale feature extraction: Utilize the visual state space module group to extract multi-scale features, providing an adaptive solution for the diversity of diabetic foot ulcer area size and morphology, thereby significantly improving segmentation accuracy and stability;

[0041] Effective capture of long-distance dependencies: The LSDE module combined with MSCA and MLLA can effectively address the limitations of traditional segmentation methods in dealing with complex boundaries, ensure clear definition of ulcer edges, and reduce segmentation errors;

[0042] Strengthening context perception: The introduction of the cross-axis attention mechanism enhances the understanding of image context, helps reduce the impact of background noise, and improves the model's adaptability and segmentation performance for complex and low-quality images;

[0043] Efficient and lightweight architecture: The linear computational complexity of the Mamba module and the lightweight design enable this method to significantly reduce the number of parameters and computational requirements without affecting the segmentation accuracy, making it particularly suitable for real-time applications in resource-constrained environments such as mobile terminals and embedded systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0045] Figure 1 A schematic diagram of the method provided by the present invention;

[0046] Figure 2 A schematic diagram of the structure of the diabetic foot ulcer image segmentation model provided by the present invention;

[0047] Figure 3 A schematic diagram of the structure of the LSDE module provided by the present invention;

[0048] Figure 4 A schematic diagram of the structure of the multi-scale cross-axis attention module provided by the present invention;

[0049] Figure 5 This is a schematic diagram of the structure of the decomposition large kernel convolution module provided by the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] On the one hand, the embodiment of the present invention discloses a lightweight diabetic foot ulcer image segmentation method, such as Figure 1 As shown, the specific steps are:

[0052] Step 1: Obtain an image to be segmented, preprocess the image to be segmented, and obtain a preprocessed image;

[0053] Step 2: Input the pre-processed image into the pre-trained diabetic foot ulcer image segmentation model to obtain a segmented image; wherein the segmentation model adopts the MC-UNet network architecture, such as Figure 2 As shown in the figure, it includes an encoder, a lightweight semantic and detail enhancement module, and a decoder connected in sequence. The lightweight semantic and detail enhancement module (Lightweight Semantic and Detail Enhancement Module) is referred to as the LSDE module. The LSDE module introduces channel and spatial attention mechanisms and multi-head cross-axis attention mechanisms to capture long-distance dependencies in preprocessed images and enhance feature expression.

[0054] In one embodiment, the step of obtaining the pre-processed image is: using a bilinear interpolation method to process the image to be segmented into a pre-processed image of a preset resolution.

[0055] In one embodiment, the encoder includes a plurality of visual state space module groups and a plurality of block merging operations, and the processing steps are:

[0056] Perform multi-scale feature extraction on preprocessed images based on the visual state space module group;

[0057] The extracted multi-scale features are downsampled according to the block merging operation to generate multi-scale extracted features.

[0058] Furthermore, in this embodiment, four downsamplings are performed, and each downsampling reduces the image size to 1 / 2 of the original size, the number of channels × 2, the encoder extracts image features through a multi-layer convolutional neural network, and uses the VSS module to perform multi-scale feature extraction, and its expression is:

[0059] ;

[0060] In the formula, VSS is the encoder module, F is the input feature, and Feat represents the feature output obtained after passing through the encoder.

[0061] In one embodiment, the visual state space module group includes at least one visual state space module, and the visual state space module includes layer normalization, two parallel branches and a linear layer connected in sequence. The preprocessed image enters the parallel branch after layer normalization processing, and is processed by the first linear layer and the first activation function in the first branch to generate a first extracted feature; in the second branch, it is processed by the second linear layer, depthwise separable convolution, SS2D module, and layer normalization to generate a second extracted feature; the first extracted feature and the second extracted feature are multiplied element by element, and then processed by the linear layer and added to the preprocessed image to obtain the extracted feature.

[0062] Furthermore, the preprocessed image is input into the encoder part of the model, and the visual state space module is used in the encoder to perform feature extraction and multi-scale information fusion. The four encoding layers in the model contain eight visual state space modules, and each layer contains two visual state space modules that output features of four different scales. , , , After passing through the visual state space module, a block merging operation is performed to downsample the input features. Each downsampling reduces the width and height of the image to 1 / 2 of the original, and the number of channels is doubled.

[0063] Furthermore, the H×W image is transformed into Feature map (The value of C is 48) and then enters two visual state space modules. In the visual state space module, First, after linear normalization, it enters two parallel branches. One is processed by a simple linear layer and activated with the silu activation function; the other is first processed by a linear layer, then activated with the silu activation function after passing through a depth-wise separable convolution, and then processed by linear normalization, multiplied with the feature result of the other parallel branch, and then processed by a linear layer and the original Add together to get the final result .

[0064] After obtaining the phased features, the block merging operation is used for downsampling, and the feature map is transformed into After that, the feature results are obtained by passing through two visual state space modules. , similarly we can get , .

[0065] In one embodiment, the data processing steps in the LSDE module are:

[0066] The multi-scale extracted features are weighted combined through channel and spatial attention mechanisms to further capture long-distance dependencies and generate feature maps. ;

[0067] Based on the feature map by 1D convolution acting on the horizontal and vertical axes Extract multi-scale context information separately;

[0068] According to the multi-scale context information, a multi-head cross-axis attention mechanism is used to Axis and Axis calculation cross-axis attention to generate multi-scale features and multi-scale features ;

[0069] Multi-scale features and multi-scale features After convolution, it is combined with the feature map Add them together to generate an enhanced feature map.

[0070] Furthermore, the LSDE module is an intermediate connection layer, such as Figure 3 As shown, the features extracted by the receiving encoder are effectively transmitted with low parameter and computational complexity, thus ensuring the effective fusion and preservation of multi-level features; including the channel and spatial attention module (CBAM) and the multi-scale cross-axis attention module (MSCA).

[0071] In one embodiment, the feature map The steps to obtain are:

[0072] The multi-scale extracted features are processed by the channel spatial attention module and the spatial attention module in turn, and then normalized and 1×1 convolution is performed to unify the dimensions of the multi-scale extracted features and generate a feature map. , the expression is:

[0073] ;

[0074] Where CBAM represents the channel and spatial attention mechanism in the LSDE module, Represents the output features after CBAM.

[0075] Furthermore, the encoding layer , , , It is sent to the intermediate connection layer, i.e., the lightweight semantic and detail enhancement module. In the intermediate connection layer, it first enters the channel spatial attention module in the channel and spatial attention module (CBAM), which is specifically expressed as follows: Represents the input features, AvgPool and MaxPool represent average pooling and maximum pooling respectively, MLP represents multi-layer perceptron, Represents the sigmoid activation function;

[0076]

[0077] ;

[0078] In the formula, and They represent the weights of the multi-layer perceptron, and They represent the results of input features after average pooling and maximum pooling respectively. It represents the interim results after the channel attention module. Indicates the final result.

[0079] This is followed by a spatial attention module, where Indicates a 7×7 convolution;

[0080] ;

[0081] ;

[0082] In the formula, Represents the interim results after the spatial attention module. Represents the final result after the feature passes through the channel space attention module.

[0083] The results obtained by the channel and spatial attention modules are normalized and processed with 1×1 convolution, and the number of channels of all features is unified to C to obtain the feature map .

[0084] In one embodiment, the obtained feature map Input the multi-scale cross-axis attention module for processing, and its structure is as follows Figure 4 As shown. First, the input feature map Then, the feature map is encoded by three parallel 1D convolutions of different scales, one horizontally ( axis) and vertical ( The convolution kernel size is set to 1×7, 1×11, and 1×21 to ensure the ability to capture spatial information at different scales. The specific operation can be described as follows:

[0085] ;

[0086] ;

[0087] In the formula, represents normalization processing, In represents the convolution along the water direction, represents the size of the convolution kernel, In represents convolution along the vertical direction, represents the size of the convolution kernel, Indicates a 1×1 convolution.

[0088] along Axis and The cross-axis attention is calculated to further fuse multi-scale features. In this process, The output of the branch is the key and value matrix, and The output of the branch is used as the query matrix. The multi-head cross-axis attention mechanism (MHCA) is used for attention calculation. The process is as follows:

[0089] ;

[0090] ;

[0091] In the formula, and Respectively Axis and The multi-head cross-axis attention calculation in the axis direction and the cross-attention mechanism explicitly model the dependencies along different spatial dimensions, effectively integrating contextual information and improving feature expression capabilities. and After two 1×1 convolutions, the feature map Add them together to get the final output of the module :

[0092] .

[0093] In one embodiment, the data processing steps in the decoder are:

[0094] Splitting the enhanced feature graph into multiple enhanced feature sub-graphs;

[0095] Multiple enhanced feature sub-graphs are input into different depth-wise separable convolution modules to obtain local and global information at different scales.

[0096] The local information and global information at different scales are cascaded along the channel dimension to form a stitched image;

[0097] The spliced ​​image is fused with the enhanced feature map to obtain a fused image.

[0098] Furthermore, the features processed by the intermediate connection layer are input into the decoder, and the decomposed large kernel convolution is used to expand the receptive field to cope with the situation of complex wounds and backgrounds.

[0099] The decomposed large kernel convolution is applied to the upsampling module. The features input after passing through the intermediate connection layer are decomposed through the large kernel convolution module and then deconvolved to double the width and height of the image and reduce the number of channels to half of the original.

[0100] Among them, the decomposed large kernel convolution includes depth-separable convolution, which can decompose the convolution kernel into multiple convolution kernels, and then use the depth-separable method to perform convolution. Its structure is as follows Figure 5 shown.

[0101] Furthermore, first According to the number of channels, it is evenly divided into 4 parts, which are recorded as , , , and , , , .

[0102] Depthwise separable convolutions with kernels of different sizes are applied to the first three branches to capture local and global information at different scales: Using 3×3 depth-wise separable convolution, Using 1×11 convolution processing, Using 11×1 convolution, No action is taken.

[0103] ;

[0104] After the convolution operation, the output , , and Cascading along the channel dimension, i.e. concat, forms a richer feature map :

[0105] ;

[0106] Will Through batch normalization, then through multi-layer perceptron (MLP) for nonlinear transformation, and finally combined with the original features Enter the module and perform the following operations:

[0107] ;

[0108] in To decompose the final output of the large kernel convolution module.

[0109] In one embodiment, the loss function is calculated based on the decoding result of the model on the image, and the binary loss function is used to calculate the loss function. Optimize model parameters.

[0110] On the other hand, the present embodiment discloses a lightweight diabetic foot ulcer image segmentation system, applying the above method, including an image acquisition module, a model building module and a recognition module;

[0111] An image acquisition module, used to collect diabetic foot ulcer images and construct a training data set;

[0112] A model building module is used to build an initial diabetic foot ulcer image segmentation model based on the MC-UNet network architecture, and iteratively train the initial diabetic foot ulcer image segmentation model using a training data set to obtain the diabetic foot ulcer image segmentation model; the initial diabetic foot ulcer image segmentation model includes an encoder, an LSDE module, and a decoder connected in sequence, and the LSDE module introduces a channel and spatial attention mechanism and a multi-head cross-axis attention mechanism to capture long-distance dependencies in the training data and enhance feature expression, and the training data comes from the training data set;

[0113] The recognition module is used to obtain the image to be recognized, input the image to be recognized into the diabetic foot ulcer image segmentation recognition model, and obtain a binary segmentation result.

[0114] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0115] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A lightweight diabetic foot ulcer image segmentation method, characterized in that: The specific steps are: Acquire an image to be segmented, and preprocess the image to be segmented to obtain a preprocessed image; Inputting the preprocessed image into a pre-trained diabetic foot ulcer image segmentation model to obtain a segmented image; wherein the diabetic foot ulcer image segmentation model adopts an MC-UNet network architecture, including an encoder, an LSDE module and a decoder connected in sequence, and the LSDE module introduces a channel and spatial attention mechanism and a multi-head cross-axis attention mechanism to capture long-distance dependencies in the preprocessed image and enhance feature expression; The encoder includes a plurality of visual state space module groups and a plurality of block merging operations, and the processing steps are as follows: Performing multi-scale feature extraction on the pre-processed image based on the visual state space module group; Down-sampling the extracted multi-scale features according to the block merging operation to generate multi-scale extracted features; The data processing steps in the LSDE module are: The multi-scale extracted features are weighted combined through the channel and spatial attention mechanism to further capture long-distance dependencies and generate a feature map M1; Extracting multi-scale context information based on the feature map M1 by 1D convolution acting on the horizontal axis and the vertical axis; According to the multi-scale context information, the multi-head cross-axis attention mechanism is used to calculate the cross-axis attention along the x-axis and the y-axis to generate the multi-scale feature M T and multi-scale features M B ; For the multi-scale feature M T and the multi-scale feature M B After the convolution process, it is added to the feature map M1 to generate an enhanced feature map.

2. A lightweight diabetic foot ulcer image segmentation method according to claim 1, characterized in that: The step of acquiring the pre-processed image is: using a bilinear interpolation method to process the image to be segmented into the pre-processed image of a preset resolution.

3. A lightweight diabetic foot ulcer image segmentation method according to claim 1, characterized in that: The visual state space module group includes at least one visual state space module, and the visual state space module includes layer normalization, two parallel branches and a linear layer connected in sequence, and the pre-processed image enters the parallel branch after being processed by the layer normalization, and is processed by the first linear layer and the first activation function in the first branch in sequence to generate a first extracted feature; In the second branch, the second extracted features are generated after the second linear layer, depthwise separable convolution, SS2D module, and layer normalization processing; the first extracted features and the second extracted features are multiplied element by element, and then added to the preprocessed image after being processed by the linear layer to obtain the extracted features.

4. A lightweight diabetic foot ulcer image segmentation method according to claim 1, characterized in that: The steps for obtaining the feature map M1 are: The multi-scale extracted features are processed by the channel spatial attention module and the spatial attention module in sequence, and then normalized and 1×1 convolution is performed to unify the dimensions of the multi-scale extracted features to generate the feature map M1.

5. The lightweight diabetic foot ulcer image segmentation method according to claim 1, characterized in that: The expression of the multi-scale context information is: In the formula, Norm means normalization processing; The x in the figure represents the convolution along the water direction, and i represents the size of the convolution kernel; The y in the figure represents the convolution along the vertical direction, and i represents the size of the convolution kernel; Conv 1×1 Indicates a 1×1 convolution.

6. A lightweight diabetic foot ulcer image segmentation method according to claim 5, characterized in that: The expression of the multi-head cross-axis attention mechanism is: M T =MHCA y (M y ,M x ,M x ); M B =MHCA x (M x ,M y ,M y ); In the formula, MHCA y and MHCA x Respectively represent the multi-head cross-axis attention calculation in the y-axis and x-axis directions; M T is the multi-scale context information in the vertical direction; M B It is the multi-scale context information in the horizontal direction.

7. A lightweight diabetic foot ulcer image segmentation method according to claim 1, characterized in that: The data processing steps in the decoder are: Splitting the enhanced feature graph into a plurality of enhanced feature sub-graphs; Inputting the plurality of enhanced feature sub-graphs into different depth-separable convolution modules respectively to obtain local information and global information at different scales; The local information and global information at different scales are cascaded along the channel dimension to form a stitched image; The spliced ​​image is fused with the enhanced feature map to obtain a fused image.

8. A lightweight diabetic foot ulcer image segmentation system, using a lightweight diabetic foot ulcer image segmentation method as claimed in any one of claims 1 to 7, characterized in that: It includes image acquisition module, model building module and recognition module; The image acquisition module is used to acquire diabetic foot ulcer images and construct a training data set; The model building module is used to build an initial diabetic foot ulcer image segmentation model based on the MC-UNet network architecture, and iteratively train the initial diabetic foot ulcer image segmentation model using the training data set to obtain the diabetic foot ulcer image segmentation model; the initial diabetic foot ulcer image segmentation model includes an encoder, an LSDE module and a decoder connected in sequence, the LSDE module introduces a channel and spatial attention mechanism and a multi-head cross-axis attention mechanism to capture long-distance dependencies in the training data and enhance feature expression, and the training data comes from the training data set; The recognition module is used to obtain the image to be recognized, input the image to be recognized into the diabetic foot ulcer image segmentation and recognition model, and obtain a binary segmentation result.

Images