Brain tumor image segmentation method based on dynamic token processing and multi-scale pyramid attention and related equipment

Abstract

The application discloses a brain tumor image segmentation method based on dynamic token processing and multi-scale pyramid attention and related equipment, and relates to the image segmentation technical field. The method comprises the following steps: preprocessing original multi-modal images in a training data set; performing image blocking operation on the preprocessed images; performing image enhancement operation on the target blocked images; inputting the target enhanced images into an initial brain tumor image segmentation model, obtaining initial features through a double-path convolution feature extraction module; obtaining brain tumor region prediction values through a multi-layer encoder, a bottleneck layer, a multi-layer decoder and a segmentation head, combining real labels to construct a loss function, updating hyperparameters of the initial brain tumor image segmentation model to obtain a target brain tumor image segmentation model; and inputting to-be-tested images into the target brain tumor image segmentation model to obtain brain tumor image segmentation results. The application can improve the segmentation precision of brain tumor images and can be widely applied to the image segmentation technical field.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and in particular to a brain tumor image segmentation method and related equipment based on dynamic token processing and multi-scale pyramid attention. Background Technology

[0002] Traditional brain tumor image segmentation relied primarily on manual delineation and annotation of medical images by professional physicians. However, manual segmentation is not only time-consuming and labor-intensive but also susceptible to subjective influences, leading to inconsistencies and uncertainties in the segmentation results. With the rapid development of medical image processing and artificial intelligence technologies, an increasing number of automated brain tumor segmentation methods have been proposed. Currently, deep learning-based brain tumor segmentation methods mainly employ Convolutional Neural Networks (CNNs) as their foundational framework, with U-Net and its derivative structures (such as 3D U-Net, Attention U-Net, and UNet++) being representative examples. However, due to the limited receptive field of convolutional kernels, traditional CNNs still have inherent limitations in global context modeling, resulting in unsatisfactory segmentation performance for complex and morphologically variable lesions like brain tumors.

[0003] Brain tumors often exhibit characteristics such as blurred boundaries, uneven grayscale distribution, and significant scale differences in medical imaging. There are clear histological differences between different subregions (e.g., enhanced tumor areas, necrotic tumor core areas, and surrounding edematous / invasive tissue areas). Existing convolutional networks tend to extract local texture and edge information, lacking sufficient modeling of global semantic relationships, making it difficult to accurately segment multi-scale lesions in complex scenes. To expand the receptive field, traditional models typically achieve multi-scale feature extraction through multiple simple downsampling operations. However, this process inevitably leads to the loss of spatial information, making the reconstruction of fine-grained structures difficult, especially when identifying small-volume or blurred-boundary lesion regions, where performance significantly degrades. Summary of the Invention

[0004] In view of this, the main objective of the embodiments of the present invention is to provide a brain tumor image segmentation method and related equipment based on dynamic token processing and multi-scale pyramid attention, in order to solve at least one of the problems of the prior art. The present invention can improve the segmentation accuracy of brain tumor images.

[0005] To achieve the above objectives, one aspect of the present invention provides a brain tumor image segmentation method based on dynamic token processing and multi-scale pyramid attention:

[0006] The original multimodal images in the training dataset are preprocessed to obtain preprocessed images;

[0007] The preprocessed image is then divided into blocks to obtain the target block image.

[0008] The target block image is subjected to image enhancement operation to obtain the target enhanced image;

[0009] The target enhanced image is input into the initial brain tumor image segmentation model, and the initial features are obtained by performing feature extraction on the target enhanced image through the dual-path convolution feature extraction module.

[0010] The initial features are processed through a multi-layer encoder, a bottleneck layer, a multi-layer decoder, and a segmentation head to obtain the predicted value of the brain tumor region;

[0011] Based on the predicted values ​​of the brain tumor region and the true labels corresponding to the original multimodal images, a loss function is constructed, and the hyperparameters of the initial brain tumor image segmentation model are updated according to the loss function to obtain the target brain tumor image segmentation model.

[0012] The image to be tested is input into the target brain tumor image segmentation model to obtain the brain tumor image segmentation result.

[0013] In some embodiments, the brain tumor image segmentation method based on dynamic token processing and multi-scale pyramid attention further includes:

[0014] A PyraBlock module is introduced into the encoder, the decoder, and the bottleneck layer;

[0015] The PyraBlock module includes a dynamic token processor submodule, a multi-scale pyramid attention submodule, a gated feedforward submodule, and a spatial attention submodule.

[0016] In some embodiments, preprocessing the original multimodal images in the training dataset to obtain preprocessed images includes the following steps:

[0017] The original multimodal image is standardized to obtain a first multimodal image;

[0018] The first multimodal image is resampled to obtain the second multimodal image;

[0019] The second multimodal image is cropped to obtain the third multimodal image;

[0020] The third multimodal image is subjected to gamma correction to obtain the fourth multimodal image;

[0021] The fourth multimodal image is normalized to obtain the preprocessed image.

[0022] In some embodiments, performing image segmentation on the preprocessed image to obtain the target segmented image includes the following steps:

[0023] Set the preset block size;

[0024] Based on the preset block size, an overlapping block strategy is adopted to obtain the number of overlapping voxels of adjacent image blocks;

[0025] Compare the image size of the preprocessed image with the preset block size;

[0026] When the image size is larger than the preset block size, based on the number of overlapping voxels and the preset block size, the preprocessed image and the corresponding real label of the preprocessed image are divided into blocks using a sliding window method to obtain the target block image and the target block label.

[0027] When the image size is smaller than the preset block size, based on the number of overlapping voxels and the image size, a second block operation is performed on the preprocessed image and the corresponding real label using a zero-padding method to obtain the target block image and the target block label.

[0028] In some embodiments, performing image enhancement operations on the target segmented image to obtain a target enhanced image includes the following steps:

[0029] The target block image is subjected to spatial geometric transformation to obtain a first enhanced image;

[0030] The first enhanced image is subjected to an image intensity domain enhancement operation to obtain a second enhanced image;

[0031] The target enhanced image is obtained by randomly zeroing out the voxel regions of the second enhanced image.

[0032] In some embodiments, the step of performing feature extraction on the target enhanced image using a dual-path convolution feature extraction module to obtain initial features includes the following steps:

[0033] The target enhanced image is input into the dual-path convolution feature extraction module, and local feature extraction is performed on the initial features to obtain local feature information;

[0034] Perform global feature extraction on the initial features to obtain global context information;

[0035] The local feature information and the global context information are fused to obtain the initial feature.

[0036] In some embodiments, the formula used to construct the loss function based on the predicted value of the brain tumor region and the ground truth label corresponding to the original multimodal image includes:

[0037] ;

[0038] In the formula, The total loss represents the loss function; Represents Dice's loss; Represents cross-entropy loss; Represents the weighting coefficient; Represents the number of tumor region categories; This represents the tumor region category index.

[0039] To achieve the above objectives, another aspect of the present invention proposes a brain tumor image segmentation device based on dynamic token processing and multi-scale pyramid attention, the device comprising:

[0040] The preprocessing module is used to preprocess the original multimodal images in the training dataset to obtain preprocessed images;

[0041] The image segmentation module is used to perform image segmentation on the preprocessed image to obtain the target segmented image.

[0042] The image enhancement module is used to perform image enhancement operations on the target block image to obtain the target enhanced image;

[0043] The first segmentation module is used to input the target enhanced image into the initial brain tumor image segmentation model, and to perform feature extraction on the target enhanced image through the dual-path convolution feature extraction module to obtain initial features;

[0044] The second segmentation module is used to process the initial features through a multi-layer encoder, a bottleneck layer, a multi-layer decoder, and a segmentation head to obtain the predicted value of the brain tumor region.

[0045] The model training module is used to construct a loss function based on the predicted value of the brain tumor region and the real label corresponding to the original multimodal image, and update the hyperparameters of the initial brain tumor image segmentation model according to the loss function to obtain the target brain tumor image segmentation model.

[0046] The model testing module is used to input the image to be tested into the target brain tumor image segmentation model to obtain the brain tumor image segmentation result.

[0047] To achieve the above objectives, another aspect of the present invention provides an electronic device, the electronic device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method described above.

[0048] To achieve the above objectives, another aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0049] To achieve the above objectives, another aspect of the present invention provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions to cause the computer device to perform the aforementioned method.

[0050] The embodiments of the present invention include at least the following beneficial effects: The present invention provides a brain tumor image segmentation method and related equipment based on dynamic token processing and multi-scale pyramid attention. This scheme preprocesses the original multimodal images in the training dataset to obtain preprocessed images, reducing image noise; performs image block operations on the preprocessed images to obtain target block images, adapting to the input requirements of the model and improving training efficiency; performs image enhancement operations on the target block images to obtain target enhanced images, expanding the diversity of training data; and inputs the target enhanced images into the initial brain tumor image segmentation model, through a dual-path convolution feature extraction module... Feature extraction is performed on the target enhanced image to obtain richer feature representations, resulting in initial features. These features are then adaptively processed and enhanced through a multi-layer encoder, bottleneck layer, multi-layer decoder, and segmentation head to obtain predicted brain tumor regions. Based on the predicted brain tumor regions and the ground truth labels corresponding to the original multimodal images, a loss function is constructed, and the hyperparameters of the initial brain tumor image segmentation model are updated according to the loss function to optimize model performance, resulting in the target brain tumor image segmentation model. The test image is then input into the target brain tumor image segmentation model to obtain the brain tumor image segmentation results, improving the segmentation accuracy of brain tumor images. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart of a brain tumor image segmentation method based on dynamic token processing and multi-scale pyramid attention provided in an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the overall architecture of the brain tumor segmentation network model based on dynamic token processing and multi-scale pyramid attention provided in this embodiment of the invention.

[0054] Figure 3 This is a schematic diagram of the architecture of the dual-path convolutional feature extraction module provided in an embodiment of the present invention;

[0055] Figure 4 This is a schematic diagram of the architecture of the PyraBlock module provided in an embodiment of the present invention;

[0056] Figure 5 This is a schematic diagram of the architecture of the PyramidAttention module provided in an embodiment of the present invention;

[0057] Figure 6 This is a schematic diagram of the architecture of the SpatialAttention module provided in an embodiment of the present invention;

[0058] Figure 7 This is a flowchart of the training and testing of a brain tumor image segmentation model based on dynamic token processing and multi-scale pyramid attention provided in an embodiment of the present invention.

[0059] Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this invention as detailed in the appended claims.

[0061] It should be noted that although functional modules are divided in the system diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first / S100" and "second / S200" in the specification, claims, and the foregoing drawings may be used herein to describe various concepts, but unless specifically stated otherwise, these concepts are not limited by these terms. These terms are used only to distinguish one concept from another. For example, first information may also be referred to as second information without departing from the scope of the embodiments of the invention, and similarly, second information may also be referred to as first information. Depending on the context, the words "if" or "when" as used herein may be interpreted as "when," "in response to a determination," or "in the event of a determination."

[0062] The terms “at least one,” “multiple,” “each,” “any,” etc., used in this invention, “at least one” includes one, two, or more than two; “multiple” includes two or more than two; “each” refers to each of the corresponding multiple; and “any” refers to any one of the multiple.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0064] In brain tumor image segmentation techniques, automated brain tumor segmentation methods exist, mainly falling into two categories: unsupervised and supervised methods. Unsupervised segmentation methods primarily include techniques based on image morphology, mathematical models, cluster analysis, and region growing. These methods do not require extensive manual annotation, but their segmentation accuracy is limited in complex scenarios. In contrast, supervised brain tumor segmentation methods establish algorithmic models and train them using a large amount of labeled data, enabling them to automatically learn tumor feature information and achieve higher segmentation accuracy. Currently, deep learning-based brain tumor segmentation methods mainly use convolutional neural networks (CNNs) as their basic framework. However, due to the limited receptive field of convolutional kernels, traditional CNNs still have inherent limitations in global context modeling, resulting in unsatisfactory segmentation results for complex and morphologically variable lesions like brain tumors. Some methods introduce attention mechanisms and feature pyramid structures to improve the network's feature selectivity and multi-scale fusion capabilities. However, most existing attention mechanisms are limited to static modeling of channel or spatial dimensions, failing to fully reflect the dynamic interactions and importance differences between features. Furthermore, traditional pyramid feature fusion models still suffer from semantic inconsistencies in information transmission across multiple scale levels. This makes it difficult for models to achieve a balance between global representation and preservation of local details in complex organizational structures.

[0065] Overall, when faced with the complex internal structure, uneven distribution of lesions, and large scale span of brain tumor tissue, traditional convolutional networks struggle to balance global semantic understanding with preservation of local spatial details, resulting in low segmentation accuracy and robustness.

[0066] In view of this, this invention provides a brain tumor image segmentation method and related equipment based on dynamic token processing and multi-scale pyramid attention. This scheme proposes a dynamic token processing module, which dynamically generates channel weights based on the global statistical information of input features through a dynamic scaling mechanism, achieving adaptive adjustment between feature channels and improving model segmentation accuracy and generalization performance. It also constructs a multi-level adaptive pooling pyramid structure to enhance the model's multi-scale perception capability while maintaining spatial resolution. Furthermore, by introducing a gating mechanism and joint modeling with spatial attention, it can adaptively filter key region information during feature flow, strengthening the model's response to important spatial locations, thereby reducing segmentation errors caused by tumor boundary blurring and improving edge segmentation accuracy. In addition, a unified PyraBlock module is constructed to achieve multi-mechanism collaborative feature enhancement.

[0067] Figure 1 This is an optional flowchart of a brain tumor image segmentation method based on dynamic token processing and multi-scale pyramid attention provided in an embodiment of the present invention. Figure 1 The method may include, but is not limited to, steps S100 to S700:

[0068] Step S100: Preprocess the original multimodal images in the training dataset to obtain preprocessed images;

[0069] Step S200: Perform image segmentation on the preprocessed image to obtain the target segmented image;

[0070] Step S300: Perform image enhancement operation on the target block image to obtain the target enhanced image;

[0071] Step S400: Input the target enhanced image into the initial brain tumor image segmentation model, and perform feature extraction on the target enhanced image through the dual-path convolution feature extraction module to obtain the initial features;

[0072] Step S500: The initial features are processed through a multi-layer encoder, a bottleneck layer, a multi-layer decoder, and a segmentation head to obtain the predicted value of the brain tumor region.

[0073] Step S600: Based on the predicted value of the brain tumor region and the real labels corresponding to the original multimodal images, a loss function is constructed, and the hyperparameters of the initial brain tumor image segmentation model are updated according to the loss function to obtain the target brain tumor image segmentation model.

[0074] Step S700: Input the image to be tested into the target brain tumor image segmentation model to obtain the brain tumor image segmentation result.

[0075] In step S100 of some embodiments, a brain tumor segmentation dataset is acquired. A portion of the data is taken from this dataset as a test dataset, and the remaining data is divided into 5 folds. In each training round, one fold is randomly selected as the validation dataset, and the remaining four folds are used as the training dataset. The training dataset is then preprocessed to reduce image noise, improve the contrast between tumor region pixels and background pixels, and thus enhance the segmentation accuracy of the deep learning-based brain tumor image segmentation model for brain tumor images.

[0076] In some embodiments, step S100 may include, but is not limited to, steps S110 to S150:

[0077] Step S110: Standardize the original multimodal image to obtain the first multimodal image;

[0078] Step S120: Resample the first multimodal image to obtain the second multimodal image;

[0079] Step S130: Crop the second multimodal image to obtain the third multimodal image;

[0080] Step S140: Perform gamma correction on the third multimodal image to obtain the fourth multimodal image;

[0081] Step S150: Normalize the fourth multimodal image to obtain the preprocessed image.

[0082] In step S110 of some embodiments, since images of different modalities have different intensity distribution characteristics, the multimodal MRI images (T1, T1ce, T2, Flair) in the training dataset are independently standardized through a standardization operation. For example, the formula used for the standardization operation is as follows:

[0083] (1)

[0084] In the formula, (2)

[0085] (3)

[0086] in, Represents the standardized pixel value of the first multimodal image; Represents the pixel values ​​of the original multimodal image; The mean value of the pixel values ​​in the original multimodal image; The standard deviation of the pixel values ​​in the original multimodal image; The width representing the pixel values ​​of the original multimodal image; The height representing the pixel value of the original multimodal image; Depth representing the pixel values ​​of the original multimodal image; The x-coordinate represents the pixel value of the original multimodal image; The ordinate represents the pixel value of the original multimodal image; Layer coordinates representing the pixel values ​​of the original multimodal image.

[0087] In step S120 of some embodiments, the standardized first multimodal image is resampled to eliminate scale differences caused by different scanning parameters by unifying all first multimodal images to the same voxel spacing. Optionally, the resampling process employs a trilinear interpolation method to ensure the integrity of image details.

[0088] In step S130 of some embodiments, the second multimodal image obtained after resampling is cropped to remove useless background regions in the image, reduce computational burden and improve model training efficiency. The cropping range is automatically determined based on the foreground region.

[0089] In step S140 of some embodiments, gamma correction is performed on the cropped third multimodal image to enhance the contrast between the tumor region and surrounding tissue. Exemplarily, the formula used for gamma correction includes:

[0090] (4)

[0091] In the formula, This represents the pixel value of the fourth multimodal image, i.e., the pixel value of the image after gamma correction; This represents an intermediate variable, and its value is 1. Represents the pixel value of the third multimodal image; This represents an intermediate variable with a value of 0.7.

[0092] In step S150 of some embodiments, the fourth multimodal MRI image obtained after gamma correction is normalized to normalize the image intensity values ​​to between 0 and 1, so as to facilitate subsequent model calculations on the image data. Exemplarily, the formula used for normalization includes:

[0093] (5)

[0094] In the formula, This represents the preprocessed image pixel values, i.e., the normalized image pixel values. This represents taking the minimum value; This represents taking the maximum value.

[0095] In step S200 of some embodiments, an adaptive segmentation strategy is adopted to segment the preprocessed image into blocks. Based on the characteristics of brain tumor MRI images and the limitations of computational resources, the three-dimensional image data is divided into image blocks of appropriate sizes to adapt to the input requirements of the deep learning model and improve training efficiency. During the segmentation process, the corresponding labels in the image are simultaneously subjected to the same image segmentation operation, thereby ensuring that the image and labels are spatially consistent and providing accurate annotation information for subsequent supervised learning.

[0096] In some embodiments, step S200 may include, but is not limited to, steps S210 to S250:

[0097] Step S210: Set the preset block size;

[0098] Step S220: Based on the preset block size, an overlapping block strategy is adopted to obtain the number of overlapping voxels of adjacent image blocks;

[0099] Step S230: Compare the image size of the preprocessed image with the preset block size;

[0100] Step S240: When the image size is larger than the preset block size, according to the number of overlapping voxels and the preset block size, the first block operation is performed on the preprocessed image and the corresponding real label of the preprocessed image by means of a sliding window to obtain the target block image and the target block label.

[0101] Step S250: When the image size is smaller than the preset block size, based on the number of overlapping voxels and the image size, a second block operation is performed on the preprocessed image and the corresponding real label using a zero-padding method to obtain the target block image and the target block label.

[0102] In step S210 of some embodiments, the size of the blocks is preset, thereby dividing the 3D image data into image blocks of appropriate size. Optionally, the image blocks are fixed-size cubes, with a preset block size of 128×128×128 voxels, as shown in the following formula:

[0103] (6)

[0104] in, This represents the preset block size; 128×128×128 indicates the number of voxels in the three dimensions of the image block.

[0105] In step S220 of some embodiments, during image segmentation, an overlapping segmentation strategy is used to obtain the number of overlapping voxels between adjacent image blocks, ensuring a certain overlap between adjacent image blocks to reduce segmentation boundary effects and ensure that the tumor region is not truncated due to segmentation. For example, the formula used with the overlapping segmentation strategy includes:

[0106] (7)

[0107] in, This represents the number of overlapping voxels of adjacent image patches in each dimension.

[0108] In steps S230 to S250 of some embodiments, the image size of the preprocessed image is compared with a preset block size. For preprocessed images with a size larger than the preset block size, a sliding window method is used for block division. For preprocessed images with a size smaller than the preset block size, zero-padding is used to expand the image to the standard block size, ensuring that the image blocks of all input models have a uniform size, which facilitates batch processing.

[0109] For example, for a preprocessed image larger than a preset block size, a window step size is obtained through an overlapping block strategy. Based on this window step size, a sliding window approach is used to perform a first block operation on the preprocessed image and the ground truth labels to obtain the target block image and its corresponding target block label. The formula used in the first block operation includes:

[0110] (8)

[0111] In the formula, This represents the step size of the sliding window in each dimension.

[0112] For example, for a preprocessed image smaller than the preset block size, the size of the padded image in each dimension is obtained based on the size of the preprocessed image in each dimension and the number of overlapping voxels. The preprocessed image is then expanded to the standard block size using zero-padding. The formulas used in the second block operation include:

[0113] (9)

[0114] In the formula, This represents the dimensions of the filled image in each dimension. This represents the size of the preprocessed image in each dimension.

[0115] In step S300 of some embodiments, the segmented brain tumor MRI image, i.e., the target segmented image, undergoes data augmentation processing in a specific order. First, spatial geometric transformation processing is performed, followed by image intensity domain enhancement. In addition, during training, the model randomly zeros out some voxel regions with a low probability. Through data augmentation processing, the diversity of training data is expanded, thereby improving the model's generalization ability and robustness.

[0116] In some embodiments, step S300 may include, but is not limited to, steps S310 to S330:

[0117] Step S310: Perform spatial geometric transformation on the target block image to obtain the first enhanced image;

[0118] Step S320: Perform image intensity domain enhancement operation on the first enhanced image to obtain the second enhanced image;

[0119] Step S330: Randomly zero out the voxel regions of the second enhanced image to obtain the target enhanced image.

[0120] In step S310 of some embodiments, the target image block undergoes spatial geometric transformation processing, including random rotation, scaling, and elastic deformation. Random rotation (angle range ±30°) and random scaling (scale range 0.7–1.4) are performed on the target image block in three-dimensional space, while elastic deformation is introduced to simulate the natural changes in tissue morphology. The deformation intensity and smoothness parameters are a∈[0,1000] and b∈[10,13], respectively. After the geometric transformation is completed, the image is randomly mirrored along the x, y, and z axes to increase the directional diversity of the samples, thus obtaining the first enhanced image.

[0121] In step S320 of some embodiments, image intensity domain enhancement is performed on the first enhanced image, including random Gaussian noise, Gaussian blur, brightness and contrast adjustment, and Gamma correction. For example, Gaussian noise is added to the first enhanced image with approximately 10% probability, Gaussian blur (kernel radius 0.5–1.0) is applied with 20% probability, and brightness multiplicative adjustment (coefficient range 0.75–1.25) is performed on the first enhanced image with 15% probability, and contrast enhancement is performed with 15% probability. Then, Gamma correction (Gamma value range 0.7–1.5) is performed with 15% probability to obtain the second enhanced image. This image intensity domain enhancement process can adjust local brightness and contrast while maintaining the overall intensity distribution.

[0122] In step S330 of some embodiments, during model training, some voxel regions of the second enhanced image are randomly zeroed with a low probability to simulate the situation of local information loss, thereby obtaining the target enhanced image and enhancing the robustness of the model.

[0123] All augmentation operations are executed sequentially, forming a complete 3D data augmentation pipeline. This augmentation strategy effectively enhances the diversity of training data and improves the model's generalization performance under different patient and scanning conditions.

[0124] In this embodiment of the invention, a brain tumor segmentation network (PyraSegNet) based on dynamic token processing and multi-scale pyramid attention was built using the PyTorch deep learning framework. Figure 2 As shown, the network model in this embodiment of the invention is an improvement on the classic image segmentation network U-Net. Its overall structure is a U-shape, consisting of an encoder on the left, a decoder on the right, and a bottleneck layer at the bottom. For example, the network model employs a deep structure design with 5 encoder layers and 5 decoder layers, providing richer feature extraction capabilities. The initial layer of the network uses a dual-path convolution feature extraction module (DualPath Convolution), referencing... Figure 3 The DualPathConv convolutional block enhances the expressive power of initial features by combining standard convolutions of the main path with depthwise separable convolutions of the auxiliary path.

[0125] In step S400 of some embodiments, a dual-path convolution feature extraction module is proposed for extracting the output feature map of a brain tumor image segmentation network. This dual-path convolution feature extraction module, through a parallel main path and auxiliary path structure, can simultaneously extract local detail information and global contextual information, thereby obtaining richer feature representations.

[0126] like Figure 3 As shown, the dual-path convolutional feature extraction module consists of a main path, an auxiliary path, and a feature fusion layer. The main path uses the standard convolutional module of the MONAI framework, including 3×3 convolution, instance normalization (InstanceNorm3d), and the GELU activation function, to extract basic local feature information. The auxiliary path uses a depthwise separable convolutional structure, first extracting spatial correlation through 3×3 depthwise convolution, then performing channel transformation through 1×1 pointwise convolution, and combining instance normalization and the GELU activation function to capture broader global contextual information.

[0127] In some embodiments, step S400 may include, but is not limited to, steps S410 to S430:

[0128] Step S410: Input the target enhancement image into the dual-path convolution feature extraction module to perform local feature extraction on the initial features to obtain local feature information;

[0129] Step S420: Perform global feature extraction on the initial features to obtain global context information;

[0130] Step S430: Fuse local feature information and global context information to obtain initial features.

[0131] In step S410 of some embodiments, the target enhancement image is input into the main path of the dual-path convolutional feature extraction module, and local feature extraction is performed through 3×3 convolution, instance normalization (InstanceNorm3d) and GELU activation function to obtain basic local feature information.

[0132] In step S420 of some embodiments, the target enhancement image is input into the auxiliary path of the dual-path convolutional feature extraction module. A depth-separable convolutional structure is adopted. First, spatial correlation is extracted through 3×3 depth convolution, and then channel transformation is performed through 1×1 pointwise convolution. Global feature extraction is performed by combining instance normalization and GELU activation function, which can obtain extensive global context information.

[0133] In step S430 of some embodiments, the local feature information output by the main path and the global context information output by the auxiliary path are concatenated and then fused by 1×1 convolution to obtain the initial features.

[0134] In some embodiments, such as Figure 3 As shown, let the target enhanced image input to the dual-path convolutional feature extraction module be... ,in For batch size, Input the number of channels. For spatial dimensions. In the dual-path convolution feature extraction module, the feature extraction process for the main path is formulated as follows:

[0135] (10)

[0136] The formula for feature extraction of auxiliary paths is as follows:

[0137] (11)

[0138] The outputs of the two paths are concatenated along the channel dimension, and then feature fusion is performed using a 1×1 convolution to generate initial features. The formulas used include:

[0139] (12)

[0140] In the formula, This represents the main path features, i.e., local feature information; This represents auxiliary path features, i.e., global context information; Representative instance normalization; This represents a concatenation operation along the channel dimension. Represents initial characteristics.

[0141] The dual-path design in the dual-path convolutional feature extraction module enables the network model to obtain richer feature representations at an early stage. The main path focuses on extracting precise local features, while the auxiliary path captures a wider range of contextual information through depthwise separable convolutions. The combination of these two paths generates richer and more comprehensive feature representations. Simultaneously, the auxiliary path employs a depthwise separable convolutional structure, reducing computational complexity while maintaining performance. Feature fusion through concatenation and 1×1 convolutions effectively integrates information from different paths, enhancing feature representation capabilities. The dual-path structure itself has a certain regularization effect, which can alleviate overfitting and improve generalization ability.

[0142] In some embodiments, such as Figure 2 As shown, PyraBlock modules are introduced into the encoder, decoder, and bottleneck layer of the network model. The PyraBlock module integrates a dynamic token processor, a multi-scale pyramid attention submodule, a gated feedforward network, and a spatial attention submodule. Through the collaborative work of these modules, adaptive feature processing and enhancement are achieved. Furthermore, residual connections combine the inputs and outputs of each module, enabling efficient information transfer and stable gradient propagation, ultimately achieving high-precision brain tumor segmentation.

[0143] In step S500 of some embodiments, a PyraBlock module is designed as an enhanced feature extraction unit for the encoder, decoder, and bottleneck layer in the brain tumor image segmentation network model. The PyraBlock module effectively extracts and enhances feature representations by integrating dynamic token processing, multi-scale pyramid attention, gated feedforward networks, and spatial attention mechanisms. This solves the problem that traditional brain tumor segmentation networks struggle to simultaneously capture and fuse local details and global contextual information during feature extraction. Figure 4As shown, the PyraBlock module consists of convolutional layers, normalization layers, activation functions, a Dynamic Token Processor, a Pyramid Attention Module, a Gated Feedforward network, and a Spatial Attention Module. The Dynamic Token Processor enhances channel representation through dynamic scaling; the Pyramid Attention Module captures long-distance dependencies through multi-scale attention; the Gated Feedforward selectively enhances features through gating; and Spatial Attention highlights important spatial locations through spatial attention. The synergistic effect of these mechanisms enables the PyraBlock module to extract richer and more accurate feature representations, enhancing feature representations from multiple dimensions, thereby improving the accuracy of brain tumor segmentation.

[0144] For example, suppose the feature map of the input PyraBlock module is... , For batch size, Input the number of channels. This refers to the spatial dimension. Therefore, the data processing in the PyraBlock module includes the following steps:

[0145] Step 1: Feature Map First, feature extraction is performed using 3×3 convolution, GroupNorm normalization, and GELU activation function. The formulas used include:

[0146] (13)

[0147] in, Represents the first intermediate processing feature; Represents the GELU activation function; Represents GroupNorm normalization; Represents a 3×3 convolution; The feature map representing the input.

[0148] Step 2: Next, depthwise separable convolutions and dynamic scaling mechanisms are employed in the dynamic token processor submodule to enhance feature representation. This submodule first generates channel scaling factors through adaptive average pooling and convolutional networks, using the following formulas:

[0149] (14)

[0150] in, This represents the Sigmoid activation function; Represents the channel scaling factor. ; Represents a 1×1 convolution; Represents the ReLU function; This represents adaptive average pooling.

[0151] The features are then processed using depthwise convolution and pointwise convolution, using the following formulas:

[0152] (15)

[0153] (16)

[0154] in, Represents the features after depthwise convolution; Represents depthwise convolution; Represents the features after pointwise convolution; This represents pointwise convolution.

[0155] The scaling factor is applied to the features after pointwise convolution, and the output features of the dynamic token processor submodule are obtained through residual connections. The formulas used include:

[0156] (17)

[0157] in, This represents the output characteristics of the dynamic token processor submodule; Represents the learnable residual weight parameters.

[0158] Step 3: Input the output features of the dynamic token processor submodule into the pyramid attention submodule, and extract attention features at different resolutions using a multi-scale pyramid structure. For example, such as... Figure 5 As shown, the pyramid attention submodule extracts multi-scale features through different levels of adaptive average pooling, and the formulas used include:

[0159] (18)

[0160] in, Represents a pyramid level; Representing the Features after hierarchical pooling.

[0161] Optionally, the pyramid level of PyraBlock is different for each layer. The PyraBlock of the encoder from the second to the fifth layer is [3,3,2,1], the PyraBlock of the bottleneck layer (the sixth layer) is 1, and the PyraBlock of the decoder from the seventh to the tenth layer is [1,2,3,3].

[0162] Next, the pooled features at each scale are queried. ,key Sum The projection is performed, and attention features are calculated through a multi-head self-attention mechanism. Then, the attention features are upsampled to the original size through trilinear interpolation and weighted and fused to obtain the output features of the pyramid attention submodule. The formula used is as follows:

[0163] (19)

[0164] (20)

[0165] (twenty one)

[0166] (twenty two)

[0167] (twenty three)

[0168] in, Represents attentional characteristics; This represents the output features of the pyramid attention submodule; Represents the number of pyramid levels; Representing the Learnable weights at each level; This represents trilinear interpolation upsampling; This represents a multi-head self-attention mechanism.

[0169] Step 4: Input the output features of the pyramid attention submodule into the gated feedforward submodule, using a gating mechanism to enhance the feature representation capability of the feedforward network. This module processes the output features of the pyramid attention submodule through two adaptive convolutional layers, using the following formulas:

[0170] (twenty four)

[0171] in, This represents a 1×1 convolution with a learnable scaling factor and normalization; This indicates the second intermediate processing feature.

[0172] The gating signal is generated using global information, and the formulas used include:

[0173] (25)

[0174] in, This represents a gating signal.

[0175] Next, the gated signal is applied to the second intermediate processing feature and residually connected to the output feature of the pyramid attention submodule. The formulas used include:

[0176] (26)

[0177] in, This represents the output characteristics of the gated feedforward submodule.

[0178] Step 5: Input the output features of the gated feedforward submodule into the spatial attention submodule, such as... Figure 6 As shown, spatial attention weights are generated through channel-wide pooling and convolution operations. For example, firstly, channel-wide average pooling is performed on the output features of the input gated feedforward submodule. Then, spatial attention weights are generated through convolution and activation functions. These spatial attention weights are then applied to the output features of the gated feedforward submodule. Finally, residual concatenation is performed between the first and fourth intermediate processing features to obtain the output features of the spatial attention submodule. The formulas used include:

[0179] (27)

[0180] (28)

[0181] (29)

[0182] (30)

[0183] in, Represents the third intermediate processing feature; Represents spatial attention weights; Represents the fourth intermediate processing feature; This represents the output features of the spatial attention submodule.

[0184] In some embodiments, such as Figure 2 As shown, the spatial attention submodule output features of the last PyraBlock module (hereinafter referred to as the final-level output features) are converted into a probability map by the segmentation head, thus obtaining the predicted value of the brain tumor region. Optionally, the final-level output features input to the segmentation head are first processed using a convolutional layer with a kernel size of 1x1, adjusting the number of channels of the final-level output features to the number of classes, resulting in an intermediate feature map with the number of channels equal to the number of classes. For example, for a binary classification problem (such as foreground / background), the number of classes is 2. Next, the segmentation head upsamples the intermediate feature map to the size of the original image. Then, the Softmax function is applied to all channel values ​​at each pixel position along the channel dimension, normalizing the value of each channel to a probability between [0,1], and the sum of the probabilities of all channels is 1, finally obtaining the image segmentation probability map, which can then be further processed by the Argmax function to generate the predicted value of the brain tumor region.

[0185] In step S600 of some embodiments, during the training of the initial brain tumor image segmentation model using the original multimodal images of the training dataset, a loss function is constructed using the ground truth labels and predicted values ​​of the brain tumor region. For example, the loss function used during training is DiceCELoss(Dice + Cross Entropy Loss), which combines Dice loss and cross-entropy loss, and the formula used includes:

[0186] (31)

[0187] In the formula, (32)

[0188] (33)

[0189] in, The total loss represents the loss function; Represents Dice's loss; Represents cross-entropy loss; Represents the weighting coefficient; Represents the number of tumor region categories; Represents a tumor region category index; Represents the total number of pixels in the image; Represents input pixels Belongs to the Predicted probability of tumor-like regions; Representing pixels In the True labeling of tumor-like regions; This represents a smoothing factor to avoid division by zero errors.

[0190] In the BraTS21 brain tumor image segmentation task, the output includes three main tumor regions:

[0191] (1) Channel 0: ET (Enhancing Tumor) enhances the tumor region;

[0192] (2) Channel 1: NCR (Non-Enhancing Tumor Core) Necrotic tumor core area;

[0193] (3) Channel 2: ED (Peritumoral Edema) Peritumoral edema / invasive tissue area around the tumor.

[0194] The true label for each pixel The value is 1 (belonging to the tumor area) or 0 (not belonging to the tumor area).

[0195] The trained model is validated using validation set images. Based on the current model's segmentation performance on the validation set, the model's hyperparameters are tuned. Through multiple adjustments to the hyperparameters, the model's performance is optimized, thus completing the model training and obtaining the target brain tumor image segmentation model.

[0196] In step S700 of some embodiments, after training the brain tumor segmentation model is completed, the segmentation model is tested using the images to be tested in the test dataset. The tumor segmentation effect is evaluated according to the evaluation index, which is the Dice coefficient. The formula used is as follows:

[0197] (34)

[0198] In the formula, A set of pixels representing the actual tumor region; The set of pixels representing the tumor region predicted by the target brain tumor image segmentation model; This represents the number of pixels where the predicted region intersects with the actual region. The total number of pixels representing the real area; This represents the total number of pixels in the predicted region.

[0199] The Dice coefficient ranges from 0 to 1; a higher value indicates better segmentation performance. The Dice coefficient comprehensively considers both precision and recall, effectively reflecting the model's accuracy in segmenting different tumor regions. For the three tumor regions (ET, NET / NCR, TC) in the BraTS21 dataset, the Dice coefficient for each category was calculated to evaluate the model's segmentation performance for different tumor sub-regions. The overall segmentation performance evaluation metric was obtained by averaging the Dice coefficients of the three regions.

[0200] The method of this invention was experimentally verified on the BraTS21 public dataset. The brain tumor image segmentation model based on dynamic token processing and multi-scale pyramid attention showed excellent overall segmentation performance. The model achieved an average Dice coefficient of 0.9240 on the validation set, which is significantly better than the segmentation effect of traditional U-Net and its improved structures (such as 3D U-Net, Attention U-Net, UNet++). This fully demonstrates the high accuracy and high stability of the embodiments of this invention in the segmentation task of complex brain tumor image structures.

[0201] In some embodiments, reference Figure 7 The training and testing steps of the brain tumor image segmentation model based on dynamic token processing and multi-scale pyramid attention in this embodiment of the invention are as follows:

[0202] Step 1: Obtain the brain tumor segmentation dataset. Take a portion of the data from the dataset as the test set, and divide the remaining data into 5 folds. Set the training to 1000 rounds, and in each round, randomly select 1 fold as the validation set, and the remaining 4 folds as the training set.

[0203] Step 2: Preprocess the training dataset to reduce image noise, improve the contrast between pixels in the tumor region and background pixels, and thus improve the segmentation accuracy of the deep learning model for brain tumors.

[0204] Step 3: Perform image block processing on the training dataset to adapt to the input requirements of the deep learning model and improve training efficiency.

[0205] Step 4: Perform data augmentation on the segmented brain tumor MRI images in a specific order to expand the diversity of training data and improve the model's generalization ability and robustness.

[0206] Step 5: Use the augmented images from the training set to train the PyraSegNet model, employing the DiceCELoss loss function. The trained model is then validated using images from the validation set. Based on the model's segmentation performance on the validation set, hyperparameters are tuned. Through multiple adjustments to the hyperparameters, the model's performance is optimized, thus completing the training process.

[0207] Step 6: After completing the training of the brain tumor image segmentation model, the segmentation model is tested using preprocessed test set images, and the tumor segmentation effect is evaluated according to the evaluation index.

[0208] This invention also provides a brain tumor image segmentation device based on dynamic token processing and multi-scale pyramid attention, which can implement the above-mentioned brain tumor image segmentation method based on dynamic token processing and multi-scale pyramid attention. The device includes:

[0209] The preprocessing module is used to preprocess the original multimodal images in the training dataset to obtain preprocessed images;

[0210] The image segmentation module is used to perform image segmentation on the preprocessed image to obtain the target segmented image;

[0211] The image enhancement module is used to perform image enhancement operations on the target block image to obtain the target enhanced image;

[0212] The first segmentation module is used to input the target enhanced image into the initial brain tumor image segmentation model, and to perform feature extraction on the target enhanced image through the dual-path convolution feature extraction module to obtain the initial features;

[0213] The second segmentation module is used to process the initial features through a multi-layer encoder, a bottleneck layer, a multi-layer decoder, and a segmentation head to obtain the predicted value of the brain tumor region.

[0214] The model training module is used to construct a loss function based on the predicted values ​​of the brain tumor region and the real labels corresponding to the original multimodal images, and to update the hyperparameters of the initial brain tumor image segmentation model according to the loss function to obtain the target brain tumor image segmentation model.

[0215] The model testing module is used to input the image to be tested into the target brain tumor image segmentation model to obtain the brain tumor image segmentation result.

[0216] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0217] This invention also provides an electronic device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including a tablet computer, an in-vehicle computer, or similar device.

[0218] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0219] refer to Figure 8 , Figure 8 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0220] The processor 801 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0221] The memory 802 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 802 and is called and executed by the processor 801.

[0222] The 803 input / output interface is used to implement information input and output.

[0223] The communication interface 804 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0224] Bus 805 transmits information between various components of the device (e.g., processor 801, memory 802, input / output interface 803, and communication interface 804);

[0225] The processor 801, memory 802, input / output interface 803, and communication interface 804 are connected to each other within the device via bus 805.

[0226] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0227] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0228] This invention also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions to cause the computer device to perform the aforementioned method.

[0229] In summary, the brain tumor image segmentation method and related equipment based on dynamic token processing and multi-scale pyramid attention according to embodiments of the present invention have the following advantages:

[0230] 1. Traditional convolutional structures cannot adaptively adjust the importance of different channels during feature extraction, making them susceptible to interference from invalid features. This invention proposes a Dynamic Token Processor module, which uses a dynamic scaling mechanism to dynamically generate channel weights based on global statistical information of the input features. This enables adaptive adjustment between feature channels, allowing the model to focus on high-value features and suppress redundant features, thereby enhancing feature representation capabilities and improving segmentation accuracy and generalization performance.

[0231] 2. Existing brain tumor segmentation networks often employ multiple downsampling operations to acquire multi-scale semantics, but this results in the loss of spatial details. This invention proposes a multi-scale pyramid attention module, which extracts contextual information from different scales by constructing a multi-level adaptive pooling pyramid structure. Furthermore, it achieves cross-scale feature fusion through a multi-head self-attention mechanism, thereby enhancing the model's multi-scale perception capability while maintaining spatial resolution.

[0232] 3. Traditional feedforward networks suffer from information redundancy issues during feature nonlinear transformation. This invention proposes a fusion design combining a gated feedforward network and spatial attention. By introducing joint modeling with the gated mechanism and spatial attention, key region information can be adaptively filtered during feature flow, enhancing the model's response to important spatial locations. This reduces segmentation errors caused by blurred tumor boundaries and improves edge segmentation accuracy.

[0233] 4. This invention integrates dynamic token processing, multi-scale pyramid attention, gated feedforward networks, and spatial attention into a single module to achieve multi-dimensional and multi-layered feature enhancement. A unified PyraBlock module is also constructed to achieve multi-mechanism collaborative feature enhancement. The PyraBlock module possesses powerful global semantic capture and local detail preservation capabilities, effectively alleviating the tumor fragmentation and misclassification problems caused by the limited receptive field of traditional CNNs.

[0234] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0235] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0236] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0237] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0238] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0239] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0240] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0241] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0242] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A brain tumor image segmentation method based on dynamic token processing and multi-scale pyramid attention, characterized in that, Includes the following steps: The original multimodal images in the training dataset are preprocessed to obtain preprocessed images; The preprocessed image is then divided into blocks to obtain the target block image. The target block image is subjected to image enhancement operation to obtain the target enhanced image; The target enhanced image is input into the initial brain tumor image segmentation model, and the initial features are obtained by performing feature extraction on the target enhanced image through the dual-path convolution feature extraction module. The initial features are processed through a multi-layer encoder, a bottleneck layer, a multi-layer decoder, and a segmentation head to obtain the predicted value of the brain tumor region; Based on the predicted values ​​of the brain tumor region and the true labels corresponding to the original multimodal images, a loss function is constructed, and the hyperparameters of the initial brain tumor image segmentation model are updated according to the loss function to obtain the target brain tumor image segmentation model. The image to be tested is input into the target brain tumor image segmentation model to obtain the brain tumor image segmentation result; The PyraBlock module is introduced into the encoder, the decoder and the bottleneck layer; the PyraBlock module includes a dynamic token processor submodule, a multi-scale pyramid attention submodule, a gated feedforward submodule and a spatial attention submodule. The data processing in the PyraBlock module includes: Feature extraction is performed on the input feature map to obtain the first intermediate processing features; The first intermediate processing feature is input into the dynamic token processor submodule, and depthwise separable convolution and dynamic scaling mechanism are used to enhance the feature representation, so as to obtain the output feature of the dynamic token processor submodule. The output features of the dynamic token processor submodule are input into the pyramid attention submodule, and attention features at different resolutions are extracted using a multi-scale pyramid structure to obtain the output features of the pyramid attention submodule. The output features of the pyramid attention submodule are input into the gated feedforward submodule. The gating mechanism is used to enhance the feature representation capability of the feedforward network, and the output features of the gated feedforward submodule are obtained. The output features of the gated feedforward submodule are input into the spatial attention submodule. First, channel global average pooling is performed on the output features of the gated feedforward submodule. Then, spatial attention weights are generated through convolution and activation functions. The spatial attention weights are then applied to the output features of the gated feedforward submodule to obtain the fourth intermediate processing feature. Finally, the output features of the spatial attention submodule are obtained by performing residual connection between the first intermediate processing feature and the fourth intermediate processing feature. The spatial attention submodule output by the last PyraBlock module outputs features, which are then converted into a probability map by the segmentation head to obtain the predicted value of the brain tumor region.

2. The method according to claim 1, characterized in that, The preprocessing of the original multimodal images in the training dataset to obtain preprocessed images includes the following steps: The original multimodal image is standardized to obtain a first multimodal image; The first multimodal image is resampled to obtain the second multimodal image; The second multimodal image is cropped to obtain the third multimodal image; The third multimodal image is subjected to gamma correction to obtain the fourth multimodal image; The fourth multimodal image is normalized to obtain the preprocessed image.

3. The method according to claim 1, characterized in that, The step of performing image block segmentation on the preprocessed image to obtain the target block image includes the following steps: Set the preset block size; Based on the preset block size, an overlapping block strategy is adopted to obtain the number of overlapping voxels of adjacent image blocks; Compare the image size of the preprocessed image with the preset block size; When the image size is larger than the preset block size, based on the number of overlapping voxels and the preset block size, the preprocessed image and the corresponding real label of the preprocessed image are divided into blocks using a sliding window method to obtain the target block image and the target block label. When the image size is smaller than the preset block size, based on the number of overlapping voxels and the image size, a second block operation is performed on the preprocessed image and the corresponding real label using a zero-padding method to obtain the target block image and the target block label.

4. The method according to claim 1, characterized in that, The process of performing image enhancement on the target segmented image to obtain the target enhanced image includes the following steps: The target block image is subjected to spatial geometric transformation to obtain a first enhanced image; The first enhanced image is subjected to an image intensity domain enhancement operation to obtain a second enhanced image; The target enhanced image is obtained by randomly zeroing out the voxel regions of the second enhanced image.

5. The method according to claim 1, characterized in that, The step of performing feature extraction on the target enhanced image using a dual-path convolution feature extraction module to obtain initial features includes the following steps: The target enhanced image is input into the dual-path convolution feature extraction module, and local feature extraction is performed on the initial features to obtain local feature information; Perform global feature extraction on the initial features to obtain global context information; The local feature information and the global context information are fused to obtain the initial feature.

6. The method according to claim 1, characterized in that, The loss function is constructed based on the predicted value of the brain tumor region and the ground truth label corresponding to the original multimodal image. The formula used includes: ； In the formula, The total loss represents the loss function; Represents Dice's loss; Represents cross-entropy loss; Represents the weighting coefficient; Represents the number of tumor region categories; This represents the tumor region category index.

7. A brain tumor image segmentation device based on dynamic token processing and multi-scale pyramid attention, characterized in that, include: The preprocessing module is used to preprocess the original multimodal images in the training dataset to obtain preprocessed images; The image segmentation module is used to perform image segmentation on the preprocessed image to obtain the target segmented image. The image enhancement module is used to perform image enhancement operations on the target block image to obtain the target enhanced image; The first segmentation module is used to input the target enhanced image into the initial brain tumor image segmentation model, and to perform feature extraction on the target enhanced image through the dual-path convolution feature extraction module to obtain initial features; The second segmentation module is used to process the initial features through a multi-layer encoder, a bottleneck layer, a multi-layer decoder, and a segmentation head to obtain the predicted value of the brain tumor region. The model training module is used to construct a loss function based on the predicted value of the brain tumor region and the real label corresponding to the original multimodal image, and update the hyperparameters of the initial brain tumor image segmentation model according to the loss function to obtain the target brain tumor image segmentation model. The model testing module is used to input the image to be tested into the target brain tumor image segmentation model to obtain the brain tumor image segmentation result. The PyraBlock module is introduced into the encoder, the decoder and the bottleneck layer; the PyraBlock module includes a dynamic token processor submodule, a multi-scale pyramid attention submodule, a gated feedforward submodule and a spatial attention submodule. The data processing in the PyraBlock module includes: Feature extraction is performed on the input feature map to obtain the first intermediate processing features; The first intermediate processing feature is input into the dynamic token processor submodule, and depthwise separable convolution and dynamic scaling mechanism are used to enhance the feature representation, so as to obtain the output feature of the dynamic token processor submodule. The output features of the dynamic token processor submodule are input into the pyramid attention submodule, and attention features at different resolutions are extracted using a multi-scale pyramid structure to obtain the output features of the pyramid attention submodule. The output features of the pyramid attention submodule are input into the gated feedforward submodule. The gating mechanism is used to enhance the feature representation capability of the feedforward network, and the output features of the gated feedforward submodule are obtained. The output features of the gated feedforward submodule are input into the spatial attention submodule. First, channel global average pooling is performed on the output features of the gated feedforward submodule. Then, spatial attention weights are generated through convolution and activation functions. The spatial attention weights are then applied to the output features of the gated feedforward submodule to obtain the fourth intermediate processing feature. Finally, the output features of the spatial attention submodule are obtained by performing residual connection between the first intermediate processing feature and the fourth intermediate processing feature. The spatial attention submodule output by the last PyraBlock module outputs features, which are then converted into a probability map by the segmentation head to obtain the predicted value of the brain tumor region.

8. An electronic device, characterized in that, Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 6.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images