Brain tumor medical auxiliary segmentation method and device and medium

Abstract

The invention relates to a brain tumor medical auxiliary segmentation method and device, and a medium. The method comprises the steps: carrying out the data preprocessing of an input image; inputting the preprocessed image into a lightweight pyramid vision Transform feature encoder to extract multi-scale features, wherein the multi-scale features comprise a trunk feature and bypass features of multiple scales; the trunk features are input into AFE-HiLo for feature refining; the bypass features of the multiple scales are input into corresponding LCoT for feature enhancement; and in a feature decoder, in combination with a multi-scale depth supervision strategy, carrying out step-by-step up-sampling and feature fusion on the refined trunk feature and the enhanced bypass feature map to generate a final segmentation result. According to the method, the image segmentation precision is remarkably improved, meanwhile, the model parameter quantity and the calculation complexity are greatly reduced through a lightweight network structure and efficient attention calculation, and effective balance between high precision and low calculation cost is achieved.

Description

A method, device and medium for medical-aided segmentation of brain tumors Technical Field

[0001] This invention relates to the field of image segmentation technology, specifically to a method, device, and medium for medical-aided segmentation of brain tumors. Background Technology

[0002] With the rapid growth of medical imaging data, AI-based medical image segmentation technology has become an important tool for assisting medical analysis. Particularly in brain magnetic resonance imaging (MRI) analysis, automated and accurate image segmentation methods provide crucial technical support for subsequent quantitative analysis and visualization. However, due to the often complex morphology and blurred boundaries of target structures in medical images, as well as the inherent noise and inconsistencies in image intensity, designing efficient and accurate segmentation models remains a challenging task.

[0003] In recent years, deep learning methods, represented by Convolutional Neural Networks (CNNs) and Visual Transformers, have made significant progress in image segmentation. However, these methods still have the following technical limitations when applied to medical image segmentation with complex structures and subtle boundaries: On the one hand, existing methods struggle to effectively balance global semantic information and local detail features in a single model. Traditional CNN models are limited by their local receptive fields, making it difficult to model long-range dependencies in images, resulting in insufficient ability to capture the overall contour of the target; while Transformer-based methods, although able to capture global context through self-attention mechanisms, often perform poorly in preserving local spatial details, easily leading to blurred or broken segmentation boundaries.

[0004] On the other hand, in order to improve the segmentation accuracy of the model, existing advanced methods generally adopt network designs with a large number of parameters and complex structures, resulting in high computational complexity, large memory consumption, and slow inference speed. This makes it difficult to deploy such models in real-world application scenarios with limited computing resources, and makes it difficult to meet the application requirements of real-time performance or low resource consumption.

[0005] Therefore, existing technologies lack an image segmentation method that can maintain high segmentation accuracy while possessing low computational complexity and high operating efficiency, especially when processing complex medical images with blurred boundaries, where this problem is even more pronounced. This invention aims to solve the aforementioned technical problems from the perspective of model architecture design. Summary of the Invention

[0006] This invention provides a method, device, and medium for medical-aided segmentation of brain tumors, aiming to solve the technical problem that existing medical image segmentation methods cannot simultaneously achieve high segmentation accuracy and low computational cost.

[0007] To achieve the above objectives, the first aspect of the present invention provides a medical-aided segmentation method for brain tumors, comprising the following steps: preprocessing an input brain tumor MRI image to obtain a preprocessed image; inputting the preprocessed image into a lightweight pyramid visual Transformer feature encoder to extract multi-scale features, the multi-scale features including backbone features and bypass features at multiple scales; inputting the backbone features into an adaptive feature enhancement high / low frequency attention module for feature refinement to obtain refined backbone features; inputting the bypass features at multiple scales into corresponding lightweight context Transformer blocks for feature enhancement to obtain enhanced bypass feature maps; and in the feature decoder, combining a multi-scale deep supervision strategy, performing stepwise upsampling and feature fusion of the refined backbone features and the enhanced bypass feature maps to generate the final segmentation result.

[0008] Furthermore, the data preprocessing includes: normalizing the input brain tumor MRI image; performing data augmentation on the normalized brain tumor MRI image; and scaling the augmented brain tumor MRI image to a preset target resolution using cubic interpolation to obtain the preprocessed image. The normalization process includes: calculating the 1st and 99th percentiles based on the pixel intensity of the input brain tumor MRI image; using the 1st and 99th percentiles to perform intensity cropping on the input image; calculating the mean and standard deviation of the cropped image; and standardizing the cropped image based on the mean and standard deviation. The data augmentation includes: performing geometric transformations on the brain tumor MRI image, including horizontal flipping, vertical flipping, and rotation; performing affine transformations and elastic deformations on the brain tumor MRI image; and performing intensity enhancement on the brain tumor MRI image, including adjusting brightness and contrast, adding random noise, and applying random region occlusion.

[0009] Furthermore, the method for extracting multi-scale features by inputting the preprocessed image into the lightweight pyramid visual Transformer feature encoder includes: inputting the preprocessed image into the lightweight pyramid visual Transformer feature encoder; performing hierarchical feature extraction on the preprocessed image through four sequentially connected stages of the lightweight pyramid visual Transformer feature encoder, wherein each stage performs an overlapping block embedding operation to downsample and transform the input, and then performs feature transformation through multiple cascaded Transformer blocks containing a spatial reduction attention mechanism; the feature maps output by the four stages constitute multi-scale features, wherein the feature maps output by the first three stages serve as bypass features at different scales, and the feature map output by the last stage serves as the backbone features.

[0010] Furthermore, the method for refining features by inputting the backbone features into the adaptive feature enhancement high / low frequency attention module includes: using the backbone features as module input; adding adaptive positional encoding to the module input to generate features containing spatial location information; inputting the features containing spatial location information in parallel to the high-frequency branch and the low-frequency branch; in the high-frequency branch, using a dynamic filling mechanism to adapt the size of the input features containing spatial location information to a preset local window size, and performing multi-head self-attention calculation within each local window to extract high-frequency local detail features; in the low-frequency branch, performing average pooling downsampling on the features containing spatial location information to reduce spatial resolution, using the downsampled features as keys and values, and the original undownsampled features as queries, and performing multi-head self-attention calculation to extract low-frequency global context features; concatenating the high-frequency local detail features and the low-frequency global context features along the channel dimension; and using a lightweight feedforward network to perform nonlinear transformation and feature enhancement on the concatenated fused features, outputting the refined backbone features.

[0011] Furthermore, the method for inputting the bypass features of the multiple scales into the corresponding lightweight contextual Transformer blocks for feature enhancement includes: using the bypass features of each scale as the input of its corresponding block; for each input, performing the following operations in parallel: generating query features through a first 1x1 convolutional layer and generating value features through a third convolutional layer; simultaneously, processing the input through a 3x3 grouped convolutional layer to generate context-aware key features; concatenating the query features, the context-aware key features, and the original input along the channel dimension to form an enhanced feature representation; inputting the enhanced feature representation into an attention head, the attention head including a second 1x1 convolutional layer, a batch normalization layer, a ReLU activation function, and a third 1x1 convolutional layer connected in sequence to generate a spatial attention map; normalizing the spatial attention map using the Sigmoid function; multiplying the normalized spatial attention map element-wise with the value features, and adding the multiplication result back to the original input through a residual connection to obtain the enhanced bypass features corresponding to that scale.

[0012] Furthermore, the method for generating the final segmentation result includes: using the refined backbone features as the initial input to the feature decoder; in each decoding stage, performing the following operations: first, upsampling the fused features from the previous stage using bilinear interpolation; then concatenating the upsampled features with the enhanced bypass features corresponding to the current stage in the channel dimension; finally, performing a convolution operation on the concatenated features to reduce the number of channels and refine the features, obtaining the fused features for this stage; repeating the above steps until a final fused feature with the same spatial resolution as the original input image is obtained; inputting the final fused feature into a segmentation head, the segmentation head including convolutional layers and upsampling operations, to generate the pixel-level segmentation result with the same size as the input image.

[0013] Furthermore, the multi-scale deep supervision strategy includes: setting independent segmentation heads in at least one intermediate stage and the final stage of the feature decoder; performing a 1x1 convolution on the fused features output by each segmentation head and restoring the spatial resolution of the feature map to be consistent with the original input image through an upsampling operation; and calculating an auxiliary loss on the output of each segmentation head using real segmentation labels to provide direct supervision signals for feature learning in that stage during training.

[0014] Furthermore, the training process of the method is optimized through a composite loss function, which includes: the composite loss function is composed of a weighted sum of the main output loss and multiple auxiliary output losses; the main output loss is a weighted combination of cross-entropy loss, Dice loss and boundary loss, which are used to optimize pixel classification accuracy, target region overlap and segmentation boundary accuracy, respectively; the auxiliary output loss is calculated from the output of each segmentation head in the multi-scale deep supervision strategy and the real label.

[0015] To achieve the above objectives, a second aspect of the present invention provides an electronic device including a memory and a processor, the memory being used to store a program that supports the processor in executing the brain tumor medical-assisted segmentation method, and the processor being configured to execute the program stored in the memory.

[0016] To achieve the above objectives, a third aspect of the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, performs the steps of the brain tumor medical-assisted segmentation method.

[0017] The beneficial effects of this invention are as follows: Compared with the prior art, the present invention provides a brain tumor medical-aided segmentation method that efficiently extracts multi-scale features by employing a lightweight pyramid visual Transformer feature encoder and designs an attention bridging fusion mechanism. The adaptive feature enhancement high / low frequency attention module refines the backbone features to simultaneously capture global contours and local details, while the lightweight context Transformer block enhances multi-scale bypass features to improve feature representation capabilities. Combined with a multi-scale deep supervision strategy in the decoder, the optimization and learning of features at different scales are strengthened during the progressive upsampling fusion process. This significantly improves the segmentation accuracy of brain tumor MRI images while greatly reducing the number of model parameters and computational complexity through a lightweight network structure and efficient attention computation, achieving an effective balance between high accuracy and low computational cost. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0019] Figure 1 is a diagram of the overall architecture of a brain tumor medical-aided segmentation method disclosed in an embodiment of the present invention.

[0020] Figure 2 is a data preprocessing flowchart disclosed in an embodiment of the present invention.

[0021] Figure 3 is a semantic segmentation network structure diagram disclosed in an embodiment of the present invention.

[0022] Figure 4 is a structural block diagram of an adaptive feature enhancement high / low frequency attention module disclosed in an embodiment of the present invention.

[0023] Figure 5 is a structural block diagram of a lightweight context converter block disclosed in an embodiment of the present invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] According to embodiments of the present invention, it should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the following methods, in some cases the steps shown or described may be executed in a different order than that shown here.

[0026] With the leap in computing power, AI-assisted solutions are profoundly transforming medical clinical diagnostic processes, showing great potential, especially in areas with limited medical resources. However, in MRI image segmentation tasks for complex tumors such as brain tumors, existing methods generally suffer from large parameter counts and high computational costs when processing images with blurred lesion boundaries and complex morphologies. To address these issues, a medical-assisted segmentation method for brain tumors is proposed, comprising a segmentation framework (LATransNet), the overall structure of which is shown in Figure 1. The method includes: step S100, preprocessing the input brain tumor MRI image to obtain a preprocessed image; step S200, inputting the preprocessed image into a lightweight pyramid visual Transformer feature encoder to extract multi-scale features, the multi-scale features including backbone features and bypass features at multiple scales; step S300, inputting the backbone features into an adaptive feature enhancement high / low frequency attention module for feature refinement to obtain refined backbone features; step S400, inputting the bypass features at multiple scales into corresponding lightweight context Transformer blocks for feature enhancement to obtain enhanced bypass feature maps; step S500, in the feature decoder, combining a multi-scale deep supervision strategy, performing stepwise upsampling and feature fusion of the refined backbone features and the enhanced bypass feature maps to generate the final segmentation result.

[0027] Understandably, the collected MRI image data first undergoes data preprocessing, including normalization and data augmentation, to improve data quality and expand sample diversity. Subsequently, the preprocessed images are fed into a semantic segmentation network. This network first extracts multi-level image features using a lightweight pyramid visual Transformer (PVTv2-B1) encoder. Then, it refines and enhances the backbone features and multi-scale bypass features using an attention bridging fusion method (including an adaptive feature enhancement high / low frequency attention module (AFE-HiLo) and a lightweight context converter block (LCoT)) to improve their feature representation and discriminative power. Finally, the decoder progressively upsamples to restore spatial resolution and generates segmentation results under the influence of a multi-scale depth supervision strategy (MSDS).

[0028] In this embodiment, as shown in step S100 above, data preprocessing is a crucial step before inputting brain tumor MRI images into the segmentation model. Raw medical image data typically cannot be directly used for model training due to several issues: Intensity inconsistency: MRI images from different scanners, different patients, or different acquisition times exhibit vastly different pixel (or voxel) intensity ranges. This inconsistency can mislead the model, causing it to learn pseudo-features unrelated to pathology rather than the true tumor morphology. Without normalization, model convergence becomes difficult, and generalization ability significantly decreases.

[0029] Extreme values ​​and noise interference: The original image may contain extreme bright or dark spots caused by the acquisition equipment or artifacts. These outliers can severely affect the calculation of standard statistics (such as mean and standard deviation), thus interfering with the normalization process and potentially generating excessive gradients during training, thereby compromising the stability of the model.

[0030] Limited data volume and insufficient diversity: Acquiring pixel-level precisely annotated MRI images is costly and time-consuming, resulting in a relatively sparse number of available training samples. This data limitation exposes deep learning models to the risk of overfitting during training, meaning the model may learn the "bias" of the dataset rather than general segmentation knowledge, leading to insufficient generalization ability on unseen data.

[0031] To address the aforementioned problems, this invention designs a targeted data preprocessing workflow, as shown in Figure 2. To address the inconsistencies in intensity and noise artifacts present in the original MRI images, a robust normalization method, ClipZScoreNormalization, is employed. This method systematically improves data quality, laying the foundation for subsequent model training. The steps and formulas are as follows: Percentile Threshold Calculation: For each input MRI image... First, calculate the cumulative distribution of all pixel intensity values ​​and determine the 1st percentile ( ) and the 99th percentile ( These two values ​​will serve as dynamic thresholds to define the boundaries of subsequent cropping operations.

[0032] Intensity Clipping: Next, a clipping operation is applied. Any element in the image with an intensity lower than [a certain value] is clipped. The pixel value was set to Anything higher The pixel value was set to This step aims to effectively remove extreme outliers caused by imaging artifacts or random noise, thereby generating an intermediate image with a more concentrated intensity distribution. .

[0033] Z-Score Standardization: Finally, the mean of the cropped, stable data is calculated. ) and standard deviation ( The image intensity is then cropped first and then normalized using a Z-score transformation, ultimately standardizing the image intensity to a distribution with a mean of 0 and a standard deviation of 1. This strategy of cropping before normalization ensures the robustness of the normalization process and provides high-quality input for model training.

[0034]

[0035] in, This indicates that the output image data after "cropping Z-Score normalization" processing has pixel intensity that conforms to a distribution with a mean of 0 and a standard deviation of 1. Indicates the original image Image data after intensity cropping. This represents the input raw brain tumor MRI image data, i.e., the medical image to be preprocessed. and These are the 1st and 99th percentiles of their pixel intensity, respectively. and These are the mean and standard deviation of the cropped image. It is a very small constant set to prevent the denominator from being zero.

[0036] Understandably, in this formula, It is input. This is an intermediate result after cropping. It is the final standardized image output.

[0037] To overcome the limitation of data volume and improve the model's generalization ability, this method employs a series of data augmentation techniques. This strategy performs real-time image transformations during training, covering three main categories: first, enhancing the model's orientation invariance through various geometric transformations (such as flipping and rotation); second, applying affine and elastic deformations to simulate subtle differences in anatomical structures; and finally, making the model more robust to different imaging qualities and conditions through intensity enhancements (such as adjusting brightness and contrast, adding noise, and random occlusion). All augmented images are uniformly adjusted to the target resolution using cubic interpolation before being input into the model.

[0038] After completing the data preprocessing in step S100 and obtaining a high-quality preprocessed image, the image is sent to the brain tumor MRI semantic segmentation network for feature extraction and segmentation calculation.

[0039] The overall architecture of the semantic segmentation network follows the encoder-decoder paradigm and presents an improved U-shaped structure, as shown in Figure 3. The network consists of three main parts: a lightweight PVTv2-B1 encoder for feature extraction, a feature decoder for restoring resolution and generating results, and an attention bridging module connecting the two, responsible for feature refinement and fusion. In this architecture, the encoder is responsible for extracting multi-level, multi-scale features from the input image; the decoder is responsible for progressively upsampling, fusing features from different levels to restore spatial details, and finally outputting a segmentation map. Unlike the direct skip connections used in classic segmentation networks such as U-Net, this model introduces two specialized attention modules as "bridges" between the feature stream output by the encoder and the input by the decoder: the Adaptive Feature Enhancement High / Low Frequency Attention Module (AFE-HiLo) is responsible for refining and semantically enhancing the "mainstream feature stream" from the deepest layer of the encoder; while the Lightweight Context Transformer (LCoT) is responsible for refining and detail enhancement of the "multi-scale bypass feature stream" from the shallower layers of the encoder. This design aims to achieve more efficient and intelligent feature fusion, thereby improving segmentation accuracy. Furthermore, to further optimize the model training process and performance, this network integrates a multi-scale deep supervision strategy (MSDS) and a composite loss function.

[0040] The specific implementation of step S200 focuses on the encoder part of this architecture. This embodiment uses a lightweight pyramid visual transformer (PVTv2-B1) as the feature encoder. Compared to the limited local receptive field of traditional convolutional networks, and the high complexity and single feature representation caused by the standard ViT's reliance on self-attention mechanisms and MLPs, PVTv2 can efficiently model long-distance dependencies through its self-attention mechanism. More importantly, its unique pyramid structure can generate hierarchical multi-scale feature maps, which is crucial for identifying lesions of different sizes and shapes. Simultaneously, the Spatial Reduction Attention (SRA) mechanism significantly reduces computational complexity while ensuring global information capture capabilities. Therefore, PVTv2 can provide the model with rich, local-to-global contextual information, thereby more accurately understanding the complex relationship between lesions and surrounding tissues and improving segmentation performance.

[0041] Table 1 shows the pseudocode algorithm for using PVTv2-B1 as the encoder. This algorithm constructs a feature pyramid through four stages, with the core being the Spatial Reduction Attention (SRA) mechanism, which introduces a reduction ratio. Downsampling key-value pairs reduces the computational complexity of traditional self-attention from O(n log n). ) dropped significantly to O ( This design significantly reduces computational cost while effectively aggregating global contextual information, enabling the model to handle high-resolution, dense prediction tasks. Although PVTv2 also proposes a more computationally efficient linear SRA, this method employs a standard SRA configuration to fully utilize the pre-trained weights.

[0042] Table 1. Pseudocode of PVTv2-B1 as an encoder

[0043] Understandably, the PVTv2-B1 encoder consists of four sequentially connected stages, forming a feature pyramid. Each stage first performs an "overlapping block embedding" operation, which uses convolutional kernels with overlapping regions to downsample the input and upscale the channel dimensions, transforming it from image patches into feature sequences. Subsequently, each stage stacks multiple Transformer blocks containing a "spatial reduction attention" mechanism. This mechanism downsamples the feature maps used to generate keys and values ​​in space before computing self-attention, significantly reducing the computational complexity of attention from the quadratic order of the feature sequence length to near linear. This allows the model to efficiently capture long-range global dependencies at an affordable computational cost, making it well-suited for processing high-resolution medical images that require global contextual information. Each Transformer block also includes a convolutional feedforward network to further enhance local feature modeling capabilities.

[0044] After four stages of hierarchical processing, the encoder outputs four feature maps at different scales (commonly referred to as F1 to F4), with spatial resolution decreasing sequentially and the number of channels and semantic abstraction increasing sequentially. In the architecture of this invention, the feature maps output by the first three stages (F1, F2, F3) are defined as "multi-scale bypass features," which retain rich spatial details and boundary information; the feature map output by the last stage (F4) is defined as "backbone features," which contains the deepest and most abstract semantic context information.

[0045] The lightweight PVTv2-B1 encoder has successfully extracted multi-level features, namely a backbone feature F4 containing global semantic information and three multi-scale bypass features F1, F2, and F3 containing rich spatial details. These features are fed into the attention bridging fusion mechanism of this method for targeted refinement and enhancement, thereby bridging the semantic gap between the encoder and decoder and preparing for the final high-precision segmentation.

[0046] This attention bridging fusion mechanism consists of two parallel attention paths, which differentiate the different feature streams output by the encoder. The first path is the backbone feature path, which feeds the semantically highest-level backbone feature stream F4 from the encoder's end into the Adaptive Feature Enhancement High / Low Frequency Attention Module (AFE-HiLo). The core idea of ​​this module is to efficiently and evenly capture the long-distance dependencies and fine boundary textures of the tumor region by processing high-frequency (local details) and low-frequency (global context) information in the features in parallel. After processing, the backbone features are transformed into refined features with richer semantic information and stronger context awareness, providing high-quality initial input for the decoder's backbone path.

[0047] The second path is the bypass feature path. It takes the three high-spatial-resolution bypass feature streams F1, F2, and F3 output from the encoder front-end and feeds them into independent Lightweight Context Transformer (LCoT) blocks before passing them as skip connections to the corresponding layers of the decoder. This block uses the contextual information of the input features themselves to dynamically guide the learning of attention weights, thereby proactively enhancing the expressive power and discriminative power of the bypass features at each scale. This transforms the skip connection from a passive, direct feature transfer process into an active, intelligent feature refinement process, ensuring that the decoder obtains significantly improved low-level spatial detail information.

[0048] The following section will first elaborate on the specific implementation of the Adaptive Feature Enhancement High / Low Frequency Attention Module (AFE-HiLo).

[0049] In step S300, the refinement of the backbone features is achieved through an Adaptive Feature Enhancement High / Low Frequency Attention Module (AFE-HiLo). Although the pre-processor PVTv2 encoder has efficiently captured multi-scale features through mechanisms such as Spatial Reduction Attention (SRA), a more refined attention computation paradigm is still needed when processing brain tumor images with low signal-to-noise ratios that contain both subtle lesion boundaries and macroscopic tissue structures. Specifically, mainstream mechanisms such as SRA do not explicitly distinguish between high-frequency components (such as texture and edge details) and low-frequency components (such as overall contours and contextual information) in the feature map when performing feature downsampling and attention computation. This "one-size-fits-all" approach may indiscriminately weaken high-frequency detail information that is crucial to segmentation accuracy during feature compression.

[0050] To address this, our method proposes an Adaptive Feature Enhancement High / Low Frequency Attention Module (AFE-HiLo) to achieve differentiated and refined processing of feature information at different frequencies. Its structure is shown in Figure 4.

[0051] AFE-HiLo processes the input features in parallel into two independent branches. During module initialization, the header allocation ratio is determined according to a preset ratio. The total number of attention heads Assigned to two branches. The Lo-Fi (low frequency) branch receives... The larger one is responsible for capturing the low-frequency global context; while the Hi-Fi (high-frequency) branch obtains... The AFE-HiLo module is responsible for processing high-frequency local details. Furthermore, to address the characteristics of medical image segmentation tasks, such as variable input sizes and critical boundary information, three key optimizations are introduced: a robust dynamic filling mechanism, adaptive positional encoding, and post-attention feature enhancement. These enable it to better preserve and utilize multi-level details crucial to tumor MRI images. The overall processing flow of AFE-HiLo is shown in Figure 4, and its detailed processing flow is as follows: Step S301, Input and Adaptive Positional Encoding: The AFE-HiLo module receives the feature map sequence from the fourth stage of PVTv2. As input, its dimension is ,in It refers to the batch size. , It refers to the number of channels. To enable the model to perceive the spatial relationships of features, a learnable adaptive positional encoding is first introduced. This encoding method can adjust itself according to the data, thus adapting more flexibly to input features of different sizes and structures, making it more flexible than fixed encoding.

[0052]

[0053] in, It is the output feature, which is the feature representation after adding adaptive positional encoding. These are input features, specifically the backbone feature sequence from the fourth stage of the PVTv2-B1 encoder. These are learnable positional encoding parameters that are broadcast to the input features. Add, Represents the set of real numbers. The number of channels representing the feature.

[0054] Step S302, High-Frequency Branch: Local Attention with Dynamic Padding: The high-frequency branch captures fine local patterns by performing self-attention within a local window. However, standard window attention requires the size of the input feature map to be divisible by the window size, which presents a problem when processing medical images of varying sizes. To address this limitation, this method introduces a dynamic padding mechanism to automatically calculate and pad the required boundaries, allowing attention calculations to proceed smoothly. The padding is then removed after calculation, restoring the original size. This eliminates the dependence on a specific input size or window size, enhancing the model's compatibility and robustness. After padding and window partitioning, multi-head self-attention (scaled dot product attention) calculations are performed within each local window:

[0055] in, For the output of the high-frequency branch; Features of the input window; Represents the attention function; Features from the input window are obtained through linear transformation The query, key, and value of the high-frequency branch obtained from the process; This represents the normalized exponential function; This is the transpose of the key matrix of the high-frequency branch. This represents the matrix transpose operation. It is the dimension of a single attention head.

[0056] Step S303, Low-Frequency Branch: Spatial Reduction and Global Context: The goal of the low-frequency (Lo-Fi) branch is to efficiently capture global context information. Unlike the high cost of directly computing global attention on the full-size feature map, the Lo-Fi branch employs a clever strategy to reduce computational complexity. The core idea of ​​this strategy is to reduce the space only for the keys and values. Specifically, an average pooling layer with a stride and kernel size equal to the window size S is used to reduce the resolution of the feature map, thereby obtaining a compact low-frequency representation.

[0057]

[0058] in, This is the output characteristic, i.e., a compact low-frequency representation; This is the average pooling function.

[0059] Subsequently, the spatially reduced features are linearly projected to obtain the key. Sum At the same time, for the original, unsampled features Perform linear projection to obtain the query. Based on this, standard scaled dot product attention calculations are then performed. Because... and The sequence length is greatly reduced, and the complexity of the entire attention computation is reduced from... Reduce to It can efficiently capture long-distance dependencies with relatively low computational cost.

[0060]

[0061] in, This is the output of the low-frequency branch; These are the query matrix, key matrix, and value matrix for the low-frequency branch, respectively. The bond matrix for the low-frequency branch The transpose of .

[0062] In this way, the Lo-Fi branch retains the accuracy of the query while enjoying the computational efficiency brought about by the compression of keys and values, achieving a balance between performance and efficiency in global context modeling.

[0063] Step S304, Feature Fusion and Post-Attention Enhancement: After calculating high-frequency and low-frequency features separately, they are concatenated along the channel dimension to fuse local details and global context. Finally, to promote effective interaction and integration of high- and low-frequency information, a post-attention feature enhancement module is introduced. This module is a lightweight feedforward network (FFN) consisting of two linear layers, a GELU activation function, and a Dropout layer. It performs a non-linear transformation on the fused features to further refine and enhance feature representation, providing the subsequent decoder with richer and more expressive feature maps. This process can be represented as:

[0064] in, This indicates the final output characteristics of the AFE-HiLo module; Indicates a feedforward network; This represents the concatenation function.

[0065] Through the above process, the AFE-HiLo module can not only efficiently process multi-frequency information in parallel, but also significantly enhance the model's ability to capture and extract key semantic features through the dynamic filling, adaptive position encoding and feature enhancement mechanisms introduced in this method, thereby improving its performance and generalization ability in brain tumor segmentation tasks.

[0066] The following section will elaborate on the specific implementation of the Lightweight Context Transformer Block (LCoT).

[0067] In step S400, the enhancement of bypass features is achieved through a Lightweight Context Transformer block (LCoT). Although skip connections between the encoder and decoder in classic U-shaped segmentation networks can effectively fuse multi-scale features, direct feature concatenation can lead to feature mismatch due to the semantic gap between them, thus affecting the final segmentation accuracy. To address this issue and more effectively utilize the contextual information provided by the encoder, this method proposes a Lightweight Context Transformer block (LCoT). This module significantly enhances the fusion quality of multi-scale features by dynamically mining contextual information and reweighting features on skip connections, while also greatly simplifying and optimizing for medical image segmentation tasks and maintaining low computational overhead. Figure 5 shows the detailed structure of the LCoT module.

[0068] Specifically, for a given skip connection feature map First, the query (Q), key (K), and value (Q) are generated through three parallel convolutional layers. In this model, Q and V are generated via 1x1 convolutions. To enable the attention mechanism to perceive local spatial context, K is generated using a 3x3 grouped convolution, thus encoding neighborhood information into the key representation before attention computation. Subsequently, the generated query Q, the context-aware key K, and the original input feature X are concatenated along the channel dimension to form an enhanced feature representation. The concatenated triplet features are fed into an attention head consisting of two 1x1 convolutional layers, batch normalization, and a ReLU activation function to efficiently learn the complex relationships between them and generate a spatial attention map. Unlike the standard Transformer, this method chooses to use the Sigmoid function instead of Softmax to normalize the attention map. This is because in segmentation tasks, different spatial locations of the feature map may contain multiple equally important targets or details, and the Sigmoid function allows multiple locations to simultaneously receive high attention weights. This allows for better preservation of rich spatial information and avoids internal competitive suppression by Softmax.

[0069]

[0070] in, This represents the generated spatial attention map; Represents the Sigmoid function; This is a 1x1 convolution operation; It is a linear rectification activation function; For batch normalization operations; These are the original input features of the LCoT block.

[0071] Finally, the normalized attention map and the value ( Perform element-wise multiplication and add the results back to the original input via residual concatenation. The final enhanced features are obtained from this process.

[0072]

[0073] in, This is the final enhancement feature.

[0074] In this way, the LCoT module can dynamically filter and enhance the features passed from the encoder to the decoder with extremely high computational efficiency, effectively bridging the semantic gap. This allows the decoder to better recover the fine boundaries of the target when performing feature fusion, thereby significantly improving the segmentation performance of the model.

[0075] The specific implementation of the feature decoder will be described in detail below.

[0076] In step S500, the feature decoder aims to efficiently fuse multi-level features from the encoder output, combining deep, high-level semantic information with shallow, low-level spatial details to generate an accurate segmentation map. The decoder follows a progressive upsampling fusion strategy, and its core workflow is as follows: Step S501, Initialization: The decoder uses the backbone features refined by the AFE-HiLo module as its initial input, starting from the highest-level, semantically richest feature map output by the encoder.

[0077] Step S502, stage-by-stage fusion: In each upsampling stage, the decoder first amplifies the fusion features of the previous stage (or encoder backbone) by a factor of two through bilinear interpolation, then concatenates them with the bypass features of the same scale that have been enhanced by the LCoT module, and then performs convolutional dimensionality reduction and feature refinement to form new fusion features, thereby achieving cross-level information fusion.

[0078] Step S503, Segmentation map generation: Finally, the highest resolution fused features are fed into a segmentation head, which consists of a convolutional layer and an upsampling operation, and is responsible for mapping the final feature representation to a pixel-level segmentation prediction result with the same size as the original input image.

[0079] In this way, the decoder can effectively and intelligently fuse high-level semantic information (from backbone features) with optimized low-level spatial details (from side-channel features) at each step of restoring the spatial resolution of the image. This "attention bridging" mechanism can further improve the model's segmentation accuracy for object boundaries and small targets.

[0080] To effectively train the tumor segmentation network and address the core challenges of blurred target boundaries, varied shapes and sizes in medical images, this method introduces a multi-scale deep supervision strategy (MSDS) during the decoding process, along with a designed composite loss function, to jointly guide and optimize the model training.

[0081] The multi-scale deep supervision strategy aims to fully utilize the multi-scale feature information contained in different layers of the decoder. This strategy sets independent segmentation heads in multiple intermediate stages of the decoder and in the final output layer. Layer output Define the corresponding segmentation head :

[0082] in, For the segmentation head, For the decoder's first Layer output, Indicates the total number of layers in the decoder. express Convolution operation, This indicates an upsampling operation, with an upsampling factor of 1. This ensures that all auxiliary outputs have the same size as the original input image. Let the outputs of each layer of the decoder be... The corresponding number of channels are respectively The output of each segmentation head can be represented as:

[0083] in, This represents the Sigmoid activation function. For the first The prediction results for the layer; express Convolutional operations. By introducing supervisory signals at multiple layers, MSDS effectively alleviates the vanishing gradient problem in deep networks, enabling lower-level features to receive more direct supervisory signals. At the same time, different decoding layers capture semantic information at different scales, enabling the network to learn effective feature representations at various scales.

[0084] In image segmentation tasks, a single loss function often struggles to balance region matching and boundary details. To achieve a robust and accurate segmentation model, a composite loss function is constructed, combining the main output loss and multiple auxiliary output losses:

[0085] in, For composite loss function, Main output loss, For the first One auxiliary output loss, For deep supervision weighting coefficients, To supplement the number of auxiliary outputs, the main output loss function uses a weighted combination of multiple losses:

[0086] in, , , Here are the weight coefficients, representing the weights of the cross-entropy loss, Dice loss, and boundary loss, respectively. Cross-entropy loss; for loss; This is the boundary loss.

[0087] Cross-entropy loss ( As a standard loss function for pixel-level classification, it independently estimates the classification correctness of each pixel, ensuring that the model learns basic pixel feature representations; loss( This can directly optimize the overlap (DSC) between the segmentation result and the true label, effectively handling class imbalance problems (such as the tumor region being much smaller than the background region); boundary loss ( By using a fixed Laplacian convolution kernel to extract the boundaries of the predicted probability map and the true label respectively, and then calculating the binary cross-entropy between the two, the network's learning of boundary details is enhanced, thereby improving the clarity and accuracy of the segmentation boundary.

[0088] In addition, for each auxiliary output, the following is adopted: Monitoring losses:

[0089] in, For the first The loss value corresponding to each auxiliary output. This is a hierarchical index, representing the [number]th [level] in the decoder. A level under which deep supervision is applied. Indicates the true label, Indicates the first One auxiliary output.

[0090] The total depth supervision loss is:

[0091] in, For the total depth of supervision loss; This represents the total number of auxiliary outputs, i.e., the total number of decoder layers to which deep supervision is applied.

[0092] Based on training experiments, the final loss function weight configuration is as follows: , , , The introduction of a composite loss function can enhance the model's ability to characterize details such as tumor boundaries, making it suitable for challenging scenarios where tumor boundaries are blurred in medical images. The combination of the composite loss function and multi-scale deep supervision strategies not only accelerates model convergence but, more importantly, enhances the model's ability to identify tumors of different sizes and shapes, improving the robustness and generalization of segmentation.

[0093] According to another aspect of the embodiments of this application, an electronic device is also provided, including a processor and a memory, wherein the processor is configured to implement the steps of the method when executing a computer program stored in the memory.

[0094] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0095] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0096] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0097] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0098] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A medically-assisted segmentation method for brain tumors, characterized in that, The process includes the following steps: preprocessing the input brain tumor MRI image to obtain a preprocessed image; inputting the preprocessed image into a lightweight pyramid visual Transformer feature encoder to extract multi-scale features, which include backbone features and bypass features at multiple scales; and inputting the backbone features into an adaptive feature enhancement high / low frequency attention module for feature refinement to obtain refined backbone features. The bypass features at the multiple scales are respectively input into the corresponding lightweight context Transformer block for feature enhancement, resulting in an enhanced bypass feature map. In the feature decoder, a multi-scale deep supervision strategy is combined to perform stepwise upsampling and feature fusion of the refined backbone features and the enhanced bypass feature map to generate the final segmentation result.

2. The brain tumor medical-aided segmentation method as described in claim 1, characterized in that, The data preprocessing includes: normalizing the input brain tumor MRI image; performing data augmentation on the normalized brain tumor MRI image; and scaling the augmented brain tumor MRI image to a preset target resolution using cubic interpolation to obtain the preprocessed image. The normalization process includes: calculating the 1st and 99th percentiles based on the pixel intensity of the input brain tumor MRI image; using the 1st and 99th percentiles to perform intensity cropping on the input image; calculating the mean and standard deviation of the cropped image; and standardizing the cropped image based on the mean and standard deviation. The data augmentation includes: performing geometric transformations on the brain tumor MRI image, including horizontal flipping, vertical flipping, and rotation; performing affine transformations and elastic deformations on the brain tumor MRI image; and performing intensity enhancement on the brain tumor MRI image, including adjusting brightness and contrast, adding random noise, and applying random region occlusion.

3. The brain tumor medical-aided segmentation method as described in claim 1, characterized in that, The method for extracting multi-scale features by inputting the preprocessed image into the lightweight pyramid visual Transformer feature encoder includes: inputting the preprocessed image into the lightweight pyramid visual Transformer feature encoder; performing hierarchical feature extraction on the preprocessed image through four sequentially connected stages of the lightweight pyramid visual Transformer feature encoder, wherein each stage performs an overlapping block embedding operation to downsample and transform the input, and then performs feature transformation through multiple cascaded Transformer blocks including a spatial reduction attention mechanism; the feature maps output by the four stages constitute multi-scale features, wherein the feature maps output by the first three stages serve as bypass features at different scales, and the feature map output by the last stage serves as the backbone features.

4. The brain tumor medical-aided segmentation method as described in claim 1, characterized in that, The method for refining features by inputting the backbone features into an adaptive feature enhancement high / low frequency attention module includes: using the backbone features as module input; adding adaptive positional encoding to the module input to generate features containing spatial location information; inputting the features containing spatial location information in parallel to the high-frequency branch and the low-frequency branch; in the high-frequency branch, using a dynamic filling mechanism to adapt the size of the input features containing spatial location information to a preset local window size, and performing multi-head self-attention computation within each local window to extract high-frequency local detail features; in the low-frequency branch, performing average pooling downsampling on the features containing spatial location information to reduce spatial resolution, using the downsampled features as keys and values, and the original undownsampled features as queries, and performing multi-head self-attention computation to extract low-frequency global context features; concatenating the high-frequency local detail features and the low-frequency global context features along the channel dimension; and using a lightweight feedforward network to perform nonlinear transformation and feature enhancement on the concatenated fused features, outputting the refined backbone features.

5. The brain tumor medical-aided segmentation method as described in claim 1, characterized in that, The method for feature enhancement by inputting the bypass features of the multiple scales into the corresponding lightweight contextual Transformer blocks includes: using the bypass features of each scale as the input of its corresponding block; for each input, performing the following operations in parallel: generating query features through a first 1x1 convolutional layer and value features through a third convolutional layer; simultaneously, processing the input through a 3x3 grouped convolutional layer to generate context-aware key features; concatenating the query features, the context-aware key features, and the original input along the channel dimension to form an enhanced feature representation; inputting the enhanced feature representation into an attention head, the attention head including a second 1x1 convolutional layer, a batch normalization layer, a ReLU activation function, and a third 1x1 convolutional layer connected in sequence to generate a spatial attention map; normalizing the spatial attention map using the Sigmoid function; multiplying the normalized spatial attention map element-wise with the value features, and adding the multiplication result back to the original input through a residual connection to obtain the enhanced bypass features corresponding to that scale.

6. The brain tumor medical-aided segmentation method as described in claim 1, characterized in that, The method for generating the final segmentation result includes: using the refined backbone features as the initial input to the feature decoder; in each decoding stage, performing the following operations: first, upsampling the fused features from the previous stage using bilinear interpolation; then concatenating the upsampled features with the enhanced bypass features corresponding to the current stage in the channel dimension; finally, performing a convolution operation on the concatenated features to reduce the number of channels and refine the features, obtaining the fused features for this stage; repeating the above steps until a final fused feature with the same spatial resolution as the original input image is obtained; inputting the final fused feature into a segmentation head, the segmentation head including convolutional layers and upsampling operations, to generate the pixel-level segmentation result with the same size as the input image.

7. The brain tumor medical-aided segmentation method as described in claim 1, characterized in that, The multi-scale deep supervision strategy includes: setting independent segmentation heads in at least one intermediate stage and the final stage of the feature decoder; performing a 1x1 convolution on the fused features output by each segmentation head and restoring the spatial resolution of the feature map to be consistent with the original input image through an upsampling operation; and calculating an auxiliary loss on the output of each segmentation head using real segmentation labels to provide direct supervision signals for feature learning in that stage during training.

8. The brain tumor medical-aided segmentation method as described in claim 7, characterized in that, The training process of the method is optimized through a composite loss function, which includes: the composite loss function is composed of a weighted sum of the main output loss and multiple auxiliary output losses; the main output loss is a weighted combination of cross-entropy loss, Dice loss and boundary loss, which are used to optimize pixel classification accuracy, target region overlap and segmentation boundary accuracy, respectively; the auxiliary output loss is calculated from the output of each segmentation head in the multi-scale deep supervision strategy and the real label.

9. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store programs that support the processor in executing any of the brain tumor medical-assisted segmentation methods according to claims 1-8, and the processor is configured to execute the programs stored in the memory.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is run by the processor, it performs the steps of any one of the brain tumor medical-assisted segmentation methods according to claims 1-8.

Images