Medical image enhancement processing and intelligent tissue boundary recognition method

Abstract

The present invention relates to the field of medical image processing and discloses a method for medical image enhancement processing and intelligent tissue boundary recognition. A medical image segmentation model is constructed, wherein a learnable filter module includes a parallel Gabor filter group and a LoG filter group; a multimodal feature fusion module fuses texture features output by multiple Gabor filters, edge features output by multiple LoG filters, and an original input image; a Swin Transformer is used to divide the fused feature map into a sequence of image blocks and extract a multi-scale feature map from the image block sequence; a segmentation head is used to restore the multi-scale feature map to its original resolution and output a segmentation mask; the constructed medical image segmentation model is trained; and image segmentation processing is performed on clinical medical images using the trained medical image segmentation model, outputting a pixel-level segmentation mask that can clearly display different tissue structures and tissue boundaries. The present invention can effectively improve the recognition accuracy of tissue boundaries in medical images.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and more particularly to a method for medical image enhancement processing and tissue boundary intelligent recognition. Background Art

[0002] Semantic segmentation of medical images is crucial for precision medicine. For example, in radiotherapy and surgical planning, its accuracy directly affects patient treatment outcomes. Although traditional convolutional neural networks (CNNs) have been used in this field, they struggle to effectively model global contextual information due to the local nature of convolution operations. In recent years, the Vision Transformer (ViT) has demonstrated great potential in medical image segmentation tasks due to its ability to capture long-range dependencies. However, existing ViT models lack the ability to model local spatial information when processing small datasets and without large-scale pre-training, making it difficult to accurately segment fine anatomical structure boundaries, such as those of small organs, small tumors, or blood vessels. Summary of the Invention

[0003] The purpose of this invention is to propose a method for medical image enhancement processing and tissue boundary intelligent recognition, so as to effectively improve the recognition accuracy of tissue boundaries in medical images.

[0004] To achieve the above objectives, the present invention proposes a method for medical image enhancement processing and tissue boundary intelligent recognition, comprising:

[0005] A medical image segmentation model is constructed, which includes a learnable filter module, a multimodal feature fusion module, a Swin Transformer backbone network and a segmentation head connected in sequence; the learnable filter module includes a parallel Gabor filter group and a LoG filter group, the Gabor filter group is used to extract multi-scale and multi-directional texture features of different tissues in the input image, and the LoG filter group is used to extract multi-scale edge features of different tissues in the input image; the multimodal feature fusion module is used to fuse the texture features output by multiple Gabor filters, the edge features output by multiple LoG filters, and the original input image to generate a fused feature map with enhanced texture and edge features; the Swin Transformer is used to divide the fused feature map into an image block sequence and extract a multi-scale feature map from the image block sequence; the segmentation head is used to restore the multi-scale feature map to the original resolution and output a segmentation mask;

[0006] The constructed medical image segmentation model is trained using training data; the training data includes medical images and corresponding pixel-by-pixel annotated tissue category segmentation masks;

[0007] The trained medical image segmentation model is used to perform image segmentation processing on clinical medical images, and the output is a pixel-level segmentation mask that can clearly show different tissue structures and tissue boundaries.

[0008] Optionally, the Gabor filter bank includes multiple parallel Gabor filters, each Gabor filter has independent trainable parameters, and the trainable parameters of the multiple Gabor filters are different;

[0009] During training, multiple learnable Gabor filters are used in parallel to dynamically learn the multi-scale and multi-directional texture features of different tissues in the input image.

[0010] Optionally, the function of the Gabor filter is defined as:

[0011] ,

[0012] in: l、 i 、 ψ、σ and c is a trainable parameter; l is a sinusoidal wavelength, used to control the texture scale; i is the direction angle, used to set the texture detection direction; ψ is the phase offset, used to adjust the sine wave phase; s is the standard deviation of the Gaussian envelope, which is used to control the size of the receptive field; c is the spatial aspect ratio, which is used to control the ellipticity of the filter; x θ = x cos i + y sin i , y θ =− x sin i + y cos i , x , y are the pixel coordinates in the original image, x θ 、 y θ are the x and y coordinates after rotation; Gabor( x θ , y θ ) represents the local coordinate system after rotation ( x θ , y θ) is used to quantify the texture intensity of the image at a specific direction θ and scale λ.

[0013] Optionally, the LoG filter bank includes multiple LoG filters in parallel, each LoG filter has independent trainable parameters, and the trainable parameters of the multiple LoG filters are different;

[0014] During training, multiple learnable LoG filters are run in parallel to dynamically learn multi-scale edge features of different tissues in the input image.

[0015] Optionally, the function of the LoG filter is defined as:

[0016] ,

[0017] in: s It is a trainable parameter that represents the Gaussian smoothing scale and is used to control the sensitivity of the filter to edge detection at different scales; x , y is the pixel coordinate in the original image; Log ( x , y ) represents the original coordinate system ( x , y ) is used to quantify the image at a specific scale. s The edge strength below.

[0018] Optionally, the Swin Transformer backbone network includes a convolutional embedding module and an encoder and decoder with a symmetrical structure;

[0019] The convolution embedding module is used to generate a feature map by dividing the fused feature map into image blocks and converting each image block into an embedding vector;

[0020] The encoder is used to perform multi-stage downsampling and multi-scale global feature extraction on the feature map;

[0021] The decoder is used to restore spatial resolution by upsampling feature maps in multiple stages and fuse multi-scale features of the encoder via skip connections to accurately locate anatomical structures.

[0022] Optionally, the operation of the convolution embedding module specifically includes:

[0023] Divide the fused feature map into 4×4 non-overlapping image blocks through a 4×4 convolutional layer with a stride of 4;

[0024] Convert each image patch into an embedding vector through linear projection;

[0025] The embedding vectors of all image blocks are arranged into a 2D grid according to the spatial position of the original image to form a feature map.

[0026] Optionally, the operation of the encoder specifically includes:

[0027] The first stage: The feature map is first processed by multiple Swin Transformer blocks, and then the image blocks are merged. The resolution is halved and the number of channels is doubled by 2×2 convolution with a stride of 2.

[0028] The second stage: First, the feature map output by the first stage is processed by multiple Swin Transformer blocks, and then the image blocks are merged. The resolution is halved and the number of channels is doubled by 2×2 convolution with a stride of 2.

[0029] The third stage: First, the feature map output from the second stage is processed by multiple Swin Transformer blocks, and then the image blocks are merged. The resolution is halved and the number of channels is doubled by a 2×2 convolution with a stride of 2.

[0030] At the end of the encoder, multiple Swin Transformer blocks are used to further extract deep features from the third stage output feature map;

[0031] Among them, the processing of each Swin Transformer block includes: dividing the feature map into windows including multiple image blocks, calculating multi-head self-attention within the window, capturing local context, using shifted window multi-head self-attention to enhance inter-window communication; generating context-aware features through feature transformation.

[0032] Optionally, the operation of the decoder specifically includes:

[0033] The third stage: First, the feature map output by the encoder end is upsampled to double the resolution. The feature map output by the third stage of the encoder is concatenated with the upsampled feature map through a skip connection, and then the concatenated feature map is processed through multiple SwinTransformer blocks;

[0034] The second stage: First, the feature map output by the third stage of the decoder is upsampled to double the resolution. The feature map output by the second stage of the encoder is concatenated with the upsampled feature map through a skip connection, and then the concatenated feature map is processed through multiple Swin Transformer blocks;

[0035] First stage: First, the feature map output by the second stage of the decoder is upsampled to double the resolution, the feature map output by the first stage of the encoder is concatenated with the upsampled feature map through a skip connection, and then the concatenated feature map is processed through multiple Swin Transformer blocks.

[0036] Optionally, the operations of the split header include:

[0037] Restore the feature map to its original size by upsampling;

[0038] Map the number of channels to the number of categories through linear projection to generate a segmentation probability map;

[0039] The Softmax activation function is used to calculate the category probability for each pixel;

[0040] Output segmentation mask.

[0041] The beneficial effects of the present invention are:

[0042] The medical image segmentation model constructed by the present invention innovatively introduces learnable Gabor filters and LoG filters into the Transformer architecture. The two work together, where the Gabor filter is responsible for extracting multi-scale and multi-directional texture information of different tissues in the image, and the LoG filter focuses on edge detection and boundary delineation, and is responsible for extracting multi-scale edge features of different tissues in the image, so that the model performs well in capturing local features. The filter is responsible for finely depicting local texture and boundary details, and the Swin Transformer captures the global dependencies between image blocks through the self-attention mechanism, generates feature representations containing global context information, and understands the image structure from a global perspective. The two complement each other, enabling the model to take into account both the whole and the local when processing complex medical image segmentation tasks, thereby accurately identifying the boundaries and structures of various tissues and organs, and making up for the shortcomings of the traditional Transformer model in this regard. At the same time, the parameters of the learnable Gabor filter and LoG filter filters are dynamically adjusted during the model training process, so that the model can automatically optimize the filter parameters according to the characteristics of different medical images to better adapt to various types of image data, greatly improving the versatility and adaptability of the model to different medical image data and enhancing its practical value.

[0043] The system of the present invention has other features and advantages that will be apparent from or will be described in detail in the accompanying drawings and subsequent detailed description incorporated herein, which together serve to explain the specific principles of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and other objects, features and advantages of the present invention will become more apparent through a more detailed description of exemplary embodiments of the present invention with reference to the accompanying drawings, in which like reference numerals generally represent like components.

[0045] Figure 1 The figure shows a schematic diagram of the structure of a medical image segmentation model in a method for medical image enhancement processing and intelligent identification of tissue boundaries of the present invention. DETAILED DESCRIPTION

[0046] The present invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention may be implemented in various forms and is not limited to the embodiments set forth herein. Rather, these embodiments are provided to make the present invention more thorough and complete and to fully convey the scope of the present invention to those skilled in the art.

[0047] like Figure 1 As shown, this embodiment provides a method for medical image enhancement processing and tissue boundary intelligent recognition, including:

[0048] S1: Construct a medical image segmentation model, which includes a learnable filter module, a multimodal feature fusion module, a Swin Transformer backbone network and a segmentation head connected in sequence; the learnable filter module includes a parallel Gabor filter group and a LoG (Laplacian of Gaussian) filter group, the Gabor filter group is used to extract multi-scale and multi-directional texture features of different tissues in the input image, and the LoG filter group is used to extract multi-scale edge features of different tissues in the input image; the multimodal feature fusion module is used to fuse the texture features output by multiple Gabor filters, the edge features output by multiple LoG filters, and the original input image to generate a fused feature map with enhanced texture and edge features; the Swin Transformer is used to divide the fused feature map into an image block (Patch) sequence and extract a multi-scale feature map from the image block (Patch) sequence; the segmentation head is used to restore the multi-scale feature map to the original resolution and output a segmentation mask;

[0049] In this embodiment, the Gabor filter bank includes multiple parallel Gabor filters, each Gabor filter has independent trainable parameters, and the trainable parameters of the multiple Gabor filters are different;

[0050] During the training process, multiple learnable Gabor filters are used in parallel to dynamically learn the multi-scale and multi-directional texture features of different tissues in the input image, and the parameters are dynamically optimized through back-propagation.

[0051] The Gabor filter function is defined as:

[0052] ,

[0053] in: l、 i 、 ψ、σ and c is a trainable parameter; l is a sinusoidal wavelength, used to control the texture scale; i is the direction angle, used to set the texture detection direction; ψ is the phase offset, used to adjust the sine wave phase; s is the standard deviation of the Gaussian envelope, which is used to control the size of the receptive field; c is the spatial aspect ratio, which is used to control the ellipticity of the filter; x θ = x cos i + y sin i , y θ =− x sin i + y cos i , x , y are the pixel coordinates in the original image, x θ 、 y θ are the x and y coordinates after rotation; Gabor( x θ , y θ ) represents the local coordinate system after rotation ( x θ , y θ ) is used to quantify the texture intensity of the image at a specific direction θ and scale λ.

[0054] In this embodiment, the LoG filter bank includes multiple parallel LoG filters, each LoG filter has independent trainable parameters, and the trainable parameters of the multiple LoG filters are different;

[0055] During the training process, multiple learnable LoG filters in parallel dynamically learn the multi-scale edge features of different tissues in the input image and dynamically optimize the parameters through back-propagation.

[0056] The function of the LoG filter is defined as:

[0057] ,

[0058] in: s It is a trainable parameter that represents the Gaussian smoothing scale and is used to control the sensitivity of the filter to edge detection at different scales; x , y is the pixel coordinate in the original image; Log ( x , y ) represents the original coordinate system ( x , y ) is used to quantify the image at a specific scale. s The edge strength below.

[0059] In this embodiment, the Swin Transformer backbone network includes a convolutional embedding module and a symmetrically structured encoder and decoder;

[0060] The convolution embedding module is used to divide the fused feature map into image patches and convert each image patch into an embedding vector to generate a feature map;

[0061] The encoder is used to perform multi-stage downsampling and multi-scale global feature extraction on the feature map;

[0062] The decoder is used to restore spatial resolution by upsampling feature maps in multiple stages and fuse multi-scale features of the encoder via skip connections to accurately locate anatomical structures.

[0063] The operations of the convolutional embedding module specifically include:

[0064] Divide the fused feature map into 4×4 non-overlapping image patches through a 4×4 convolutional layer with a stride of 4;

[0065] Convert each image patch into an embedding vector through linear projection;

[0066] The embedding vectors of all image patches are arranged into a 2D grid according to the spatial position of the original image to form a feature map.

[0067] The operation of the encoder specifically includes:

[0068] The first stage: First, the feature map is processed by multiple Swin Transformer blocks (CST Block), and then the image block merging operation (Patch merging) is performed. The resolution is halved and the number of channels is doubled through 2×2 convolution with a stride of 2;

[0069] The second stage: First, the feature map output by the first stage is processed by multiple Swin Transformer blocks (CST Blocks), and then the image block merging operation (Patch merging) is performed. The resolution is halved and the number of channels is doubled through a 2×2 convolution with a stride of 2;

[0070] The third stage: First, the feature map output from the second stage is processed by multiple Swin Transformer blocks (CST Blocks), and then the image block merging operation (Patch merging) is performed. The resolution is halved and the number of channels is doubled through a 2×2 convolution with a stride of 2;

[0071] At the end of the encoder, multiple Swin Transformer blocks (CST Block) are used to further extract deep features from the third stage output feature map;

[0072] The processing of each Swin Transformer block (CST Block) includes:

[0073] The feature map is divided into windows consisting of multiple image blocks, multi-head self-attention is calculated within the window to capture local context, and shifted window multi-head self-attention is used to enhance communication between windows;

[0074] Generate context-aware features through feature transformation (Layer Norm, MLP, etc.).

[0075] Specifically, the Swin Transformer Block (CST Block) processes data in the following ways:

[0076] (1) Window partitioning: The feature map is divided into non-overlapping windows of fixed size. For example, the feature map is usually divided into multiple windows of size 7×7 or 14×14. Self-attention calculation is performed between the feature points in each window. This window partitioning makes the calculation more efficient while preserving local spatial information.

[0077] (2) Shifting Window Strategy (SW-MSA): In certain Transformer blocks, a shifting window approach is used. Compared to standard window division, the shifting window first shifts the feature map by half the window size before dividing the window. This allows feature points between adjacent windows to interact in different Transformer blocks, increasing the receptive field and helping to capture richer contextual information.

[0078] (3) Window-wide Multi-Head Self-Attention Mechanism (W-MSA): Within each window, a multi-head self-attention mechanism is used to model the relationship between feature points. The feature vector within each window is divided into multiple heads, and each head performs self-attention calculations independently. By extracting and aggregating features from different heads, the interdependencies between different feature subspaces can be captured, enhancing the model’s feature expression capabilities.

[0079] The feature vector of each window undergoes a nonlinear transformation through a multi-layer perceptron (MLP). An MLP typically consists of two fully connected layers with a GELU activation function in between, introducing nonlinearity and enhancing the model's expressiveness and fitting capabilities. In the Swin Transformer Block, residual connections are used to sum the input and output features. This residual connection helps alleviate the vanishing gradient problem in deep networks, facilitates information flow, and improves training efficiency and model performance. Layer normalization is used within each Swin Transformer Block to normalize features. This helps stabilize the model training process, accelerates convergence, and improves model performance. Through the aforementioned W-MSA / SW-MSA and MLP operations, the Swin Transformer Block gradually extracts and models local and global image features, enabling efficient feature extraction and pattern recognition in image data. This architecture effectively captures long-range dependencies and rich contextual information in medical image segmentation tasks, laying the foundation for the model's high-precision segmentation performance.

[0080] The operation of the decoder specifically includes:

[0081] The third stage: First, the feature map output by the encoder end is upsampled (Up sampling) to double the resolution. The feature map output by the third stage of the encoder is spliced ​​with the upsampled feature map through skip connections, and then the spliced ​​feature map is processed by multiple Swin Transformer blocks (CST Blocks);

[0082] The second stage: First, the feature map output by the third stage of the decoder is upsampled to double the resolution. The feature map output by the second stage of the encoder is concatenated with the upsampled feature map through a skip connection. The concatenated feature map is then processed through multiple Swin Transformer blocks (CST blocks).

[0083] First stage: First, the feature map output by the second stage of the decoder is upsampled (Up sampling) to double the resolution, the feature map output by the first stage of the encoder is concatenated with the upsampled feature map through skip connections, and then the concatenated feature map is processed through multiple Swin Transformer blocks (CST Blocks).

[0084] The operations of the split header include:

[0085] Restore the feature map to its original size by upsampling;

[0086] The number of channels is mapped to the number of categories through 1×1 convolutional layer linear projection to generate a segmentation probability map;

[0087] The Softmax activation function is used to calculate the category probability for each pixel;

[0088] Output segmentation mask.

[0089] In an example, the data processing flow of a medical image segmentation model is as follows:

[0090] 1. Input layer

[0091] Input data:

[0092] Original medical image (such as CT / MRI), size H (height) × W (width) × C (channel number) (Default is 224×224×3, which is a single-channel grayscale image);

[0093] Preprocessing:

[0094] Standardization: adjustment based on modality (e.g., HU value cutoff for CT, z-score normalization for MRI);

[0095] Size alignment: ensure that the image and label dimensions are consistent (e.g. 224×224);

[0096] Output: Normalized image tensor I ∈R 224×224×3 .

[0097] 2. Learnable filters (including parallel Gabor filter banks and LoG filter banks)

[0098] (1) Gabor filter path (N parallel Gabor filters)

[0099] operate:

[0100] 1) Dynamic parameter generation: The parameters of each Gabor filter (θ, σ, λ, ψ, γ) are learned via backpropagation, with initial values ​​based on the traditional Gabor settings;

[0101] 2) Convolution operation: convolve the input image with multiple Gabor filters to generate multi-scale and multi-directional texture features;

[0102] 3) Output: N Gabor feature map F Gabor ∈R 224×224×N .

[0103] (2) LoG filter path (M LoG filters)

[0104] operate:

[0105] 1) Dynamic σ learning: The σ parameter of each LoG filter is optimized through training, and the initial value is based on experience (e.g., σ = 1.0);

[0106] 2) Convolution operation: convolve the input image with the LoG filter to extract edge and boundary features;

[0107] Output: M LoG feature map F LoG ∈R 224×224×M .

[0108] 3. Multimodal Feature Fusion (Concat)

[0109] enter:

[0110] Original image I ∈R 224×224×3 ;

[0111] Gabor features F Gabor ∈R 224×224×N ;

[0112] LoG Features F LoG ∈R 224×224×M ;

[0113] operate:

[0114] Channel splicing: Splice three sets of features along the channel dimension to obtain F fused ∈R 224×224×(3+N+M) , for example, using 5Gabor + 5 LoG filters, the number of channels after fusion is 13;

[0115] Objective: To combine original images with radiomics features to enhance the ability to express local details.

[0116] 4. Convolutional Patch Embedding

[0117] Input: fusion features F fused ∈R 224×224×(3+N+M) ;

[0118] operate:

[0119] (1) Convolutional block: Use a 4×4 convolution kernel with a stride of 4 to divide the image into 56×56 non-overlapping patches, each of size 4×4. Each convolution kernel uses the GELU activation function, and a normalization layer (LN) is added to the second convolution kernel.

[0120] (2) Channel mapping: The number of output channels is adjusted to a fixed dimension D embed =96, get the embedding vector E patch ∈R 56 ×56×96 ;

[0121] Purpose: Preserve spatial locality and provide structured input for Transformer.

[0122] 5. Swin Transformer Encoder

[0123] The encoder consists of four stages, which progressively downsample and extract multi-scale global features:

[0124] (1) Stage 1

[0125] enter: E patch ∈R 56×56×96 ;

[0126] operate:

[0127] Swin Transformer Block (CST Block) ×2:

[0128] Window partitioning: Divide the 56×56 features into 7×7 windows (each window is 8×8 patch);

[0129] Windowed Self-Attention (W-MSA): computes multi-head self-attention within a window to capture local context;

[0130] Shifting windows (SW-MSA): Shifting the window position in the next layer to promote cross-window information interaction;

[0131] Output: Maintain 56×56×96;

[0132] Patch Merging: halve the resolution and double the number of channels through 2×2 convolution (stride 2);

[0133] For example, input 56×56×96 → output 28×28×192.

[0134] (2) Stage 2-3

[0135] Input: output of the previous stage;

[0136] operate:

[0137] Swin Transformer Block ×2: Same as Stage 1, but with a smaller window size (e.g., the Stage 2 window is 4×4);

[0138] Patch Merging: halve the resolution and double the number of channels through 2×2 convolution (stride 2);

[0139] Output:

[0140] Stage 2: 14×14×384;

[0141] Stage 3: 7×7×768.

[0142] 6. Swin Transformer Decoder

[0143] The decoder recovers details through upsampling and skip connections:

[0144] (1) Stage 3

[0145] Input: Encoder Stage 3 output 7×7×768;

[0146] operate:

[0147] Up sampling: Use transposed convolution (kernel 2×2, stride 2) to upsample by a factor of 2; the output is 14×14×384;

[0148] Skip connection: concatenate with the 14×14×384 output of encoder Stage 3, compressing the number of channels to 384;

[0149] Swin Transformer Block ×2: Processing fusion features;

[0150] Output: 14×14×384.

[0151] (2) Stage 3 → Stage 2

[0152] Input: Decoder Stage 3 output 14×14×384;

[0153] operate:

[0154] Upsample to 28×28×192 and concatenate with the encoder Stage 2 features, and process as above;

[0155] Output: 28×28×192;

[0156] (3) Stage 2 → Stage 1

[0157] Input: Decoder Stage 2 output 28×28×192;

[0158] operate:

[0159] Upsample to 56×56×96 and concatenate with the encoder Stage 1 features, and process as above;

[0160] Output: 56×56×96.

[0161] 7. Output layer (segmentation head)

[0162] Input: The decoder outputs 56×56×96.

[0163] operate:

[0164] Final upsampling (Patch expanding): restore the decoder output feature map from 56×56×96 to the original size of 224×224×96 through bilinear interpolation or transposed convolution;

[0165] Convolution prediction: 1×1 convolution: mapping the number of channels to the number of categories K , generate segmentation probability map P ∈R 224×224×K ;

[0166] Softmax activation: calculate the category probability for each pixel;

[0167] Output: Segmentation maskY ∈R 224×224×K , take argmax to get the final category label.

[0168] S2: training the constructed medical image segmentation model using training data; the training data includes medical images and corresponding pixel-by-pixel annotated tissue category segmentation masks;

[0169] Specifically, the model can be trained using existing datasets, such as the Synapse multi-organ segmentation dataset and the ACDC heart segmentation dataset. The model begins training with randomly initialized weights, for example, for 300 epochs. Model optimization is performed using the AdamW optimizer, which combines Adam's adaptive learning rate adjustment and weight decay techniques to help prevent model overfitting. During training, different learning rates can be set for different datasets. Different numbers of filters can be used depending on the dataset. For example, for the ACDC dataset, five Gabor filters and five LoG filters can be trained, while for the Synapse dataset, two Gabor filters and five LoG filters are used. The parameters of these filters are learnable during training and can be dynamically adjusted based on the data characteristics to optimize feature extraction. Through training, the model learns the characteristic representations of tissues and organs in medical images, including local texture, edge, and boundary information, as well as global contextual relationships. This enables accurate segmentation of medical images and precise identification of the boundaries of different tissues and organs. During the training process, the performance of the model is evaluated by monitoring the performance of the model on the validation set, such as evaluation indicators such as the Dice similarity coefficient (DSC) and the 95th percentile Hausdorff distance (HD95), and the model parameters and training strategy are adjusted as needed to optimize the segmentation effect of the model.

[0170] S3: Use the trained medical image segmentation model to perform image segmentation processing on clinical medical images, and output pixel-level segmentation masks that can clearly show different tissue structures and tissue boundaries.

[0171] In summary, this method combines learnable Gabor and LoG filters with the Transformer architecture, effectively solving the problem of insufficient local feature capture of traditional Vision Transformer in medical image segmentation. By introducing Gabor filters to extract multi-scale and multi-directional texture information, and LoG filters to enhance edge detection and boundary delineation accuracy, the model can capture rich local detail features in medical images, while leveraging the powerful global context modeling capabilities of the Transformer to achieve integration and understanding of the global information of the image. This organic combination of local and global feature extraction enables the model to accurately identify the boundaries and structures of various tissues and organs when dealing with complex medical image segmentation tasks. Even in small data set scenarios, it can achieve high-precision segmentation effects with low computing resource consumption due to its lightweight design and dynamically adjustable filter parameters, showing strong adaptability and practicality, and providing an efficient, accurate and applicable solution for a variety of segmentation tasks in the field of medical image segmentation.

[0172] While various embodiments of the present invention have been described above, the above description is intended to be illustrative, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A method for medical image enhancement processing and tissue boundary intelligent recognition, characterized in that: include: A medical image segmentation model is constructed, which includes a learnable filter module, a multimodal feature fusion module, a Swin Transformer backbone network and a segmentation head connected in sequence; the learnable filter module includes a parallel Gabor filter group and a LoG filter group, the Gabor filter group is used to extract multi-scale and multi-directional texture features of different tissues in the input image, and the LoG filter group is used to extract multi-scale edge features of different tissues in the input image; the multimodal feature fusion module is used to fuse the texture features output by multiple Gabor filters, the edge features output by multiple LoG filters, and the original input image to generate a fused feature map with enhanced texture and edge features; the Swin Transformer is used to divide the fused feature map into an image block sequence and extract a multi-scale feature map from the image block sequence; the segmentation head is used to restore the multi-scale feature map to the original resolution and output a segmentation mask; The constructed medical image segmentation model is trained using training data; the training data includes medical images and corresponding pixel-by-pixel annotated tissue category segmentation masks; Use the trained medical image segmentation model to perform image segmentation processing on clinical medical images, and output pixel-level segmentation masks that can clearly show different tissue structures and tissue boundaries; The Gabor filter bank includes multiple parallel Gabor filters, each Gabor filter having independent trainable parameters, and the trainable parameters of the multiple Gabor filters are different; during the training process, the multiple parallel learnable Gabor filters dynamically learn the multi-scale and multi-directional texture features of different tissues in the input image, and dynamically optimize the parameters through back propagation; The LoG filter group includes multiple parallel LoG filters, each LoG filter has independent trainable parameters, and the trainable parameters of the multiple LoG filters are different; during the training process, the multiple parallel learnable LoG filters dynamically learn the multi-scale edge features of different tissues in the input image and dynamically optimize the parameters through back propagation.

2. The method according to claim 1, characterized in that The Gabor filter function is defined as: ， in: λ, θ 、 ψ、σ and γ is a trainable parameter; λ is a sinusoidal wavelength, used to control the texture scale; θ is the direction angle, used to set the texture detection direction; ψ is the phase offset, used to adjust the sine wave phase; σ is the standard deviation of the Gaussian envelope, which is used to control the size of the receptive field; γ is the spatial aspect ratio, which is used to control the ellipticity of the filter; x θ = x cos θ + y sin θ , y θ = x sin θ + y cos θ , x , y are the pixel coordinates in the original image, x θ 、 y θ are the x and y coordinates after rotation; Gabor( x θ , y θ ) represents the local coordinate system after rotation ( x θ , y θ ) is used to quantify the texture intensity of the image at a specific direction θ and scale λ.

3. The method according to claim 1, characterized in that The function of the LoG filter is defined as: ， in: σ It is a trainable parameter that represents the Gaussian smoothing scale and is used to control the sensitivity of the filter to edge detection at different scales; x , y is the pixel coordinate in the original image; LoG ( x , y ) represents the original coordinate system ( x , y ) is used to quantify the image at a specific scale. σ The edge strength below.

4. The method according to claim 1, wherein The SwinTransformer backbone network includes a convolutional embedding module and a symmetrically structured encoder and decoder; The convolution embedding module is used to generate a feature map by dividing the fused feature map into image blocks and converting each image block into an embedding vector; The encoder is used to perform multi-stage downsampling and multi-scale global feature extraction on the feature map; The decoder is used to restore spatial resolution by upsampling feature maps in multiple stages and fuse multi-scale features of the encoder via skip connections to accurately locate anatomical structures.

5. The method according to claim 4, characterized in that The operations of the convolutional embedding module specifically include: Divide the fused feature map into 4×4 non-overlapping image blocks through a 4×4 convolutional layer with a stride of 4; Convert each image patch into an embedding vector through linear projection; The embedding vectors of all image blocks are arranged into a 2D grid according to the spatial position of the original image to form a feature map.

6. The method according to claim 4, characterized in that The operation of the encoder specifically includes: The first stage: The feature map is first processed by multiple Swin Transformer blocks, and then the image blocks are merged. The resolution is halved and the number of channels is doubled by 2×2 convolution with a stride of 2. The second stage: First, the feature map output by the first stage is processed by multiple Swin Transformer blocks, and then the image blocks are merged. The resolution is halved and the number of channels is doubled by 2×2 convolution with a stride of 2. The third stage: First, the feature map output from the second stage is processed by multiple Swin Transformer blocks, and then the image blocks are merged. The resolution is halved and the number of channels is doubled by a 2×2 convolution with a stride of 2. At the end of the encoder, multiple Swin Transformer blocks are used to further extract deep features from the third stage output feature map; Among them, the processing of each Swin Transformer block includes: dividing the feature map into windows including multiple image blocks, calculating multi-head self-attention within the window, capturing local context, using shifted window multi-head self-attention to enhance inter-window communication; generating context-aware features through feature transformation.

7. The method according to claim 6, characterized in that The operation of the decoder specifically includes: The third stage: First, the feature map output by the encoder end is upsampled to double the resolution. The feature map output by the third stage of the encoder is concatenated with the upsampled feature map through a skip connection, and then the concatenated feature map is processed through multiple SwinTransformer blocks; The second stage: First, the feature map output by the third stage of the decoder is upsampled to double the resolution. The feature map output by the second stage of the encoder is concatenated with the upsampled feature map through a skip connection, and then the concatenated feature map is processed through multiple SwinTransformer blocks; First stage: First, the feature map output by the second stage of the decoder is upsampled to double the resolution, the feature map output by the first stage of the encoder is concatenated with the upsampled feature map through skip connections, and then the concatenated feature map is processed by multiple SwinTransformer blocks.

8. The method according to claim 7, characterized in that The operations of the split header include: Restore the feature map to its original size by upsampling; Map the number of channels to the number of categories through linear projection to generate a segmentation probability map; The Softmax activation function is used to calculate the category probability for each pixel; Output segmentation mask.

Images