Ultrasound image segmentation method based on edge perception and spatial channel mamba

Abstract

The present application relates to the field of image segmentation, and more particularly to an ultrasound image segmentation method based on edge perception and spatial channel Mamba, an image segmentation model is constructed, and the image segmentation model is used for segmentation of an ultrasound image, the image segmentation model comprises an encoder and a decoder, and a fusion network arranged between the encoder and the decoder; the encoder comprises four sequentially connected down-sampling modules, the first down-sampling module comprises a Patch Embedding and a spatial channel Mamba module, the second to fourth down-sampling modules each comprise a Patch Merging and a spatial channel Mamba module, and the down-sampling module is used for extracting spatial channel features; the fusion network is used for fusing spatial channel features and edge features of the extracted spatial channel features, and a feature map output by the fusion network is input into the decoder to generate a segmentation result by the decoder; the spatial channel Mamba module is introduced, spatial and channel feature extraction and fusion are performed, and the ultrasound image segmentation precision is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image segmentation, and in particular to an ultrasound image segmentation method based on edge perception and spatial channel Mamba. BACKGROUND

[0002] The fusion of spatial and channel features is crucial for accurate lesion segmentation, however, existing methods are generally difficult to achieve a balance and effective integration of spatial and channel representations; existing edge perception methods rely on prior knowledge and focus on predefined regions of interest, due to the inherent variability and complexity of lesion regions, making the existing methods perform poorly; existing methods attempt to use fixed weights or simple addition to fuse multiple types, without dynamically adjusting the fusion ratio according to the local features of the ultrasound image, resulting in the synergistic effect of deep semantic features and edge features being unable to fully play, and the segmentation result is prone to the problem of region integrity but boundary shift. SUMMARY

[0003] The purpose of the present application is to provide an ultrasound image segmentation method based on edge perception and spatial channel Mamba to improve the accuracy of image segmentation.

[0004] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0005] The ultrasound image segmentation method based on edge perception and spatial channel Mamba constructs an image segmentation model, uses the image segmentation model to segment the ultrasound image, and the image segmentation model includes an encoder and a decoder, and a fusion network arranged between the encoder and the decoder;

[0006] The encoder includes four sequentially connected down-sampling modules, the first down-sampling module includes a PatchEmbedding and a spatial channel Mamba module, and the second to fourth down-sampling modules each include a Patch Merging and a spatial channel Mamba module;

[0007] The down-sampling module first divides the input ultrasound image into multiple image blocks, extracts spatial features and channel features for each image block, multiplies the extracted spatial features and channel features element by element, and outputs spatial channel features;

[0008] The fusion network includes four branches, the outputs of the four down-sampling modules are respectively taken as the inputs of the four branches, and the first branch includes a Scharr feature extraction module;

[0009] The second to fourth branches each comprise two Scharr feature extraction modules for extracting edge features of an input feature map, and a matrix addition module arranged between the two Scharr feature extraction modules for splicing the edge features output by the first Scharr feature extraction module of the branch with the edge features output by the previous branch.

[0010] The fourth branch is further connected with an adaptive kernel fusion module after the last Scharr feature extraction module, which is used for fusing the edge features output by the fourth branch with the feature map output by the fourth downsampling module.

[0011] The feature maps output by the four branches of the fusion network are input into the decoder to generate a segmentation result.

[0012] Preferably, the spatial channel Mamba module comprises two branches, one of which divides the input ultrasound image into a plurality of image blocks, inputs each image block into a first Mamba module for spatial feature extraction, splices the feature maps output by the first Mamba module to obtain a first feature map, and element-wise adds the first feature map with the ultrasound image to obtain a second feature map.

[0013] The other branch divides the channel features of the input ultrasound image into a plurality of separate groups, inputs the channel features of each group into a third Mamba module for channel feature extraction, inputs the output channel features into a linear layer for linear processing, and outputs after layer normalization processing and activation function processing.

[0014] The features output by the two branches are element-wise multiplied to obtain spatial channel features.

[0015] Preferably, the spatial channel features are input into the Scharr feature extraction module to generate an initial edge feature map, the initial edge feature map is input into a multi-scale convolution structure for feature extraction to obtain a multi-scale edge feature map, and a convolution block is used to fuse the multi-scale edge feature maps to output an edge feature map.

[0016] Preferably, the multi-scale convolution structure comprises four branches, the first branch is a 1x1 convolution module, the second branch is a 1x3 convolution module and a 3x1 convolution module connected in sequence, the third branch is a 1x5 convolution module and a 5x1 convolution module connected in sequence, and the fourth branch is a 1x7 convolution module and a 7x1 convolution module connected in sequence.

[0017] Preferably, the adaptive kernel fusion module element-wise adds the input spatial channel feature and the edge feature map to obtain a first fusion feature, and sequentially performs global average pooling, point-by-point convolution operation and softmax function processing on the first fusion feature to generate weights of the spatial channel feature and the edge feature map respectively, and element-wise adds the spatial channel feature and the edge feature map and the corresponding weights after multiplication to output a second fusion feature map.

[0018] By adopting the foregoing design scheme, the application has the beneficial effects that the application introduces the spatial channel Mamba module, effectively extracts and fuses spatial and channel features, and thus can effectively improve the ultrasound image segmentation precision. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 The figure is a schematic diagram of the image segmentation model of the application;

[0020] Figure 2 The figure is a processing flowchart of the spatial channel Mamba module of the application;

[0021] Figure 3 The figure is a schematic diagram of the Scharr feature extraction module of the application;

[0022] Figure 4 The figure is a processing flowchart of the adaptive kernel fusion module of the application;

[0023] Figure 5 The figure is a visual comparison diagram of the image segmentation method and different models on the BUSI dataset of the application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical scheme and advantages of the application more clear, the application will be further described in detail below in combination with the drawings. Obviously, the described embodiments are only some of the embodiments of the application, but not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0025] The terms "first", "second", "third" and the like in the specification and claims of the application and the above drawings are used to distinguish different objects, and are not used to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0026] An ultrasound image segmentation method based on edge perception and spatial channel Mamba is constructed, such as... Figure 1 The image segmentation model shown is used to segment ultrasound images. The image segmentation model includes an encoder and a decoder, as well as a fusion network set between the encoder and the decoder.

[0027] The encoder comprises four sequentially connected downsampling modules. The first downsampling module includes Patch Embedding and a spatial channel Mamba module, while the second through fourth downsampling modules each include Patch Merging and a spatial channel Mamba module. Within the downsampling modules, Patch Embedding captures local features by dividing the input image into multiple small blocks and mapping them to high-dimensional vectors; while Patch Merging reduces the resolution of the feature map by merging adjacent blocks, achieving information fusion, reducing computational complexity, and enhancing global semantic representation.

[0028] The fusion network includes four branches, and the outputs of the four downsampling modules of the encoder are used as the inputs of the four branches respectively. The first branch includes a Scharr Feature Extraction (SFE) module.

[0029] The second to fourth branches each include two Scharr feature extraction modules, and a matrix addition module is set between the two Scharr feature extraction modules. The Scharr feature extraction module is used to extract the edge features of the input feature map, and the matrix addition module is used to concatenate the edge features output by the previous branch with the edge features output by the first Scharr feature extraction module of the branch.

[0030] The fourth branch is connected to the last Scharr feature extraction module and then to an Adaptive Kernel Fusion (AKF) module. This module is used to fuse the edge features output from the fourth branch with the feature map output from the fourth downsampling module to perform depth and edge feature fusion.

[0031] The feature maps output from the four branches of the fusion network are input into the decoder, which generates the segmentation result.

[0032] The downsampling module first divides the input ultrasound image into multiple image blocks, extracts spatial and channel features for each image block, and multiplies the extracted spatial and channel features element by element to output the spatial and channel features.

[0033] like Figure 2As shown, the spatial channel Mamba module includes two branches. One branch divides the input ultrasound image into multiple image blocks, inputs each image block into the first Mamba module for spatial feature extraction, and stitches the feature maps output by the first Mamba module to obtain a first feature map. The first feature map is then added element-wise to the ultrasound image to obtain a second feature map. The second feature map is then pixel-segmented and input into the second Mamba module for spatial feature extraction, outputting a third feature map.

[0034] Another branch divides the channel features of the input ultrasound image into multiple separate groups, inputs the channel features of each group into the third Mamba module for channel feature extraction, inputs the output channel features into a linear layer for linear processing, performs layer normalization on the linearly processed features, and outputs them after processing with an activation function.

[0035] The spatial channel features are obtained by element-wise multiplication of the features output from the two branches. This spatial channel Mamba module extracts spatial and channel features through two branches and achieves efficient interaction between the two, comprehensively capturing the spatial details and channel semantic associations of ultrasound images, enhancing the model's ability to represent lesion areas, effectively adapting to the characteristics of high noise and blurred lesion boundaries in ultrasound images, and improving the accuracy and reliability of ultrasound image segmentation.

[0036] In this embodiment, the encoder employs a cascaded design, first focusing on local spatial structure and then performing global context modeling. To handle segmentation from different spatial dimensions, the input image is first divided into fixed-size image patches, and the Mamba module is used to model local spatial dependencies. The size of the image patch. If the original image size is given, then the number of image patches is... The segmented image is then represented as Therefore, the processing formula for this part can be expressed as:

[0037] ;

[0038] ;

[0039] ;

[0040] in, Represents one-dimensional convolution. This indicates LayerNorm. , and It is an intermediate variable. , This indicates that it can be expanded into a 1-D sequence. Indicates a linear layer. This indicates that the output will be divided into multiple image blocks. This represents the activation function. This represents a selective state-space model. This means changing the shape of the tensor (without changing the element content and total number), only adjusting the dimensional structure to fit the input requirements of the network layer;

[0041] Within each Mamba module, modules are shared to effectively suppress noise and capture locally relevant features guided by the global framework. They are then merged and reconstructed sequentially.

[0042] Subsequently, the reconstructed features are segmented pixel-by-pixel to generate a 1D sequence for capturing global spatial features, which is represented as follows: This sequence is then fed into the Mamba module to capture global space dependencies. This process is expressed by the following formula:

[0043] ;

[0044] ;

[0045] ;

[0046] in, , and It is an intermediate variable. This is the output result.

[0047] Channel Feature Extraction: This section first divides the channel features into G separate groups. Then, it models the relationships between different channels. Guided by these inter-group relationships, the channel features are updated. This process can be represented as:

[0048] ;

[0049] ;

[0050] ;

[0051] in, , and It is an intermediate variable. This is the output result.

[0052] Assumption The output is , , right The processing is represented as:

[0053] ;

[0054] ;

[0055] in, This represents the control state mapping parameters after discretization. and This indicates the parameters corresponding to different positions. This represents the input weight matrix. Represents the state-output weight matrix. This represents the input-output weight matrix.

[0056] Finally Restored to a one-dimensional sequence And a linear mapping is used to obtain the weights at the corresponding positions. This process can be represented as:

[0057] ;

[0058] Where GELU represents the activation function, LN represents LayerNorm, and Linear represents a linear layer.

[0059] In this embodiment, spatial channel features are input to, for example... Figure 3 The Scharr feature extraction module shown generates an initial edge feature map to capture the gradient information of the lesion boundary. The initial edge feature map is then input into a multi-scale convolutional structure for feature extraction to obtain a multi-scale edge feature map that filters out false edges. A 1×1 convolutional block is used to fuse the multi-scale edge feature maps and output the edge feature map.

[0060] The Scharr feature extraction module first uses the Scharr operator to obtain gradient information of the lesion boundary and generate an edge feature map, providing a basis for edge detection. Then, through a multi-scale convolution structure, it adapts to the morphological changes of the lesion boundary at different scales and effectively filters image noise. Finally, it uses a 1×1 convolution block to achieve efficient fusion of multi-scale edge features and outputs an edge feature map that takes into account both details and global consistency. This provides strong constraints for the accurate division of lesion boundaries in ultrasound image segmentation and can significantly improve the edge accuracy and completeness of the segmentation results.

[0061] This multi-scale convolutional structure comprises four branches: the first branch is a 1×1 convolutional module; the second branch consists of a 1×3 convolutional module and a 3×1 convolutional module connected in sequence; the third branch consists of a 1×5 convolutional module and a 5×1 convolutional module connected in sequence; and the fourth branch consists of a 1×7 convolutional module and a 7×1 convolutional module connected in sequence. In this embodiment, the second branch divides the 3×3 convolution into 1×3 and 3×1 convolutional modules to extract width and height spatial features, respectively. The third and fourth branches also extract width and height spatial features, respectively, which will not be elaborated further here.

[0062] Since the Scharr operator may generate false edges during edge feature extraction, this application employs multi-scale convolution operations to refine the edge representation to mitigate this issue. Because the feature resolution differs at different scales, direct summation is not ideal. Therefore, this application adopts a progressive fusion strategy in the overall model architecture, adding the output of each scale to subsequent layers. This ensures that edge information from the early stages effectively propagates to deeper layers, improving the quality of edge-aware segmentation.

[0063] like Figure 4 As shown, the adaptive kernel fusion module adds the input spatial channel feature and the edge feature map element by element to obtain the first fused feature. The first fused feature is then processed by global average pooling, pointwise convolution, and softmax function to generate the weights of the spatial channel feature and the edge feature map, respectively. The spatial channel feature and the edge feature map are multiplied by the corresponding weights and then added element by element to output the second fused feature map.

[0064] Deep features provide rich semantic representations but often lack precise boundary information. Conversely, edge features preserve boundary information but may lose crucial deep semantic information. Therefore, the key to improving segmentation performance lies in the effective fusion of deep and edge features. To effectively fuse depth and edge information, an adaptive kernel module improves segmentation accuracy by coordinating edge and depth features. This process is represented by the following formula:

[0065] ;

[0066] ;

[0067] in, Indicates spatial channel characteristics, Represents the edge feature map, Indicates global average pooling. This represents a multilayer perceptron. This represents the softmax activation function. Weights representing spatial channel characteristics. The weights represent the edge feature maps. This represents the second fused feature map.

[0068] Table 1. Comparison of the image segmentation method of this application with other methods on the BUSI dataset:

[0069]

[0070] Table 2. Generalization experiments of the image segmentation method in this application (STU as an external test dataset):

[0071]

[0072] Table 3 Ablation experiments of the segmentation method in this application (BUSI dataset):

[0073]

[0074] As shown in Tables 1-3, the experimental results demonstrate that the segmentation method proposed in this application achieves a significant performance improvement over existing models. Specifically, as... Figure 5 As shown, the mIOU reached 74.93% on the BUSI dataset. In addition, it also performed well on the external validation dataset STU.

[0075] In summary, this application introduces the spatial channel Mamba module, which effectively extracts and fuses spatial and channel features, thereby significantly improving the segmentation accuracy of ultrasound images.

[0076] This application designs a Scharr feature extraction module that integrates the Scharr operator and multi-scale feature extraction, guiding the network to pay more attention to the boundary region;

[0077] This application designs an adaptive kernel fusion module, which effectively integrates deep semantics and edge features, making the segmentation of regions more accurate.

[0078] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An ultrasound image segmentation method based on edge perception and spatial channel Mamba, characterized in that: An image segmentation model is constructed and used to segment ultrasound images. The image segmentation model includes an encoder and a decoder, as well as a fusion network set between the encoder and the decoder. The encoder includes four downsampling modules connected in sequence. The first downsampling module includes a PatchEmbedding module and a spatial channel Mamba module. The second to fourth downsampling modules each include a Patch Merging module and a spatial channel Mamba module. The downsampling module first divides the input ultrasound image into multiple image blocks, extracts spatial and channel features for each image block, multiplies the extracted spatial and channel features element by element, and outputs the spatial and channel features. The fusion network includes four branches, and the outputs of the four downsampling modules are used as the inputs of the four branches respectively. The first branch includes a Scharr feature extraction module. The second to fourth branches each include two Scharr feature extraction modules, and a matrix addition module is set between the two Scharr feature extraction modules. The Scharr feature extraction module is used to extract the edge features of the input feature map and output the edge feature map. The matrix addition module is used to concatenate the edge features output by the previous branch with the edge features output by the first Scharr feature extraction module of the branch. The fourth branch is connected to the last Scharr feature extraction module, which is an adaptive kernel fusion module. This adaptive kernel fusion module is used to fuse the edge features output by the fourth branch with the feature map output by the fourth downsampling module to perform depth features and edge features. The feature maps output from the four branches of the fusion network are input into the decoder, which generates the segmentation result. The spatial channel Mamba module includes two branches. One branch divides the input ultrasound image into multiple image blocks, inputs each image block into the first Mamba module for spatial feature extraction, and stitches the feature maps output by the first Mamba module to obtain a first feature map. The first feature map is then added element-wise to the ultrasound image to obtain a second feature map. The second feature map is then pixel-segmented and input into the second Mamba module for spatial feature extraction, outputting a third feature map. Another branch divides the channel features of the input ultrasound image into multiple separate groups, inputs the channel features of each group into the third Mamba module for channel feature extraction, inputs the output channel features into a linear layer for linear processing, performs layer normalization on the linearly processed features, and outputs them after processing with an activation function. The spatial channel features are obtained by multiplying the features output from the two branches element by element. The adaptive kernel fusion module adds the input spatial channel feature and the edge feature map element by element to obtain the first fused feature. The first fused feature is then processed by global average pooling, pointwise convolution, and softmax function to generate the weights of the spatial channel feature and the edge feature map, respectively. The spatial channel feature and the edge feature map are multiplied by the corresponding weights and then added element by element to output the second fused feature map.

2. The ultrasound image segmentation method based on edge perception and spatial channel Mamba as described in claim 1, characterized in that: The spatial channel features are input into the Scharr feature extraction module to generate an initial edge feature map. The initial edge feature map is then input into a multi-scale convolutional structure for feature extraction to obtain a multi-scale edge feature map. Finally, a convolutional block is used to fuse the multi-scale edge feature maps and output the edge feature map.

3. The ultrasound image segmentation method based on edge perception and spatial channel Mamba as described in claim 2, characterized in that: The multi-scale convolutional structure includes four branches: the first branch is a 1×1 convolutional module, the second branch is a 1×3 convolutional module and a 3×1 convolutional module connected in sequence, the third branch is a 1×5 convolutional module and a 5×1 convolutional module connected in sequence, and the fourth branch is a 1×7 convolutional module and a 7×1 convolutional module connected in sequence.

Images