Brain glioma segmentation method and system based on region of interest combined with artificial intelligence

Abstract

This invention provides a glioma segmentation method and system based on regions of interest (ROIs) combined with artificial intelligence (AI). This method relates to the field of image segmentation technology and involves preprocessing brain magnetic resonance images; extracting image features using a multiscale convolutional neural network; calculating attention weights to generate a heat map and binarizing it to obtain candidate regions; adaptively adjusting convolution kernel parameters based on the size of the candidate regions, extracting and fusing multiscale features using spatial pyramid pooling; performing semantic segmentation using a segmentation module; and labeling glioma regions. This invention improves the accuracy and efficiency of glioma segmentation and reduces computing resource consumption.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to a method and system for brain glioma segmentation based on regions of interest combined with artificial intelligence. Background Art

[0002] Traditional glioma segmentation methods rely primarily on manual or semi-automatic segmentation techniques, which typically require significant time and effort from professional physicians and suffer from subjectivity and poor reproducibility. With the advancement of computer vision and artificial intelligence technologies, automatic segmentation methods based on deep learning have become a research hotspot. Convolutional neural networks (CNNs), in particular, have demonstrated superior performance in the field of medical image segmentation.

[0003] However, existing deep learning-based glioma segmentation technologies still have the following defects and deficiencies:

[0004] Existing methods lack adaptability when dealing with brain gliomas of different sizes and shapes. The segmentation accuracy of tumor areas with fuzzy boundaries and different sizes is not high, and it is difficult to accurately identify the fine structure and boundary information of the tumor.

[0005] Traditional convolutional neural networks have high computational complexity when processing entire brain MRI images, making it difficult to effectively focus on the tumor area. This results in insufficient feature extraction of key areas by the network, affecting the accuracy and efficiency of segmentation. Summary of the Invention

[0006] The embodiments of the present invention provide a method and system for brain glioma segmentation based on regions of interest combined with artificial intelligence, which can solve the problems in the prior art.

[0007] A first aspect of an embodiment of the present invention provides a method for segmenting gliomas based on regions of interest combined with artificial intelligence, comprising:

[0008] receiving a brain magnetic resonance image acquired by a medical imaging device, and preprocessing the brain magnetic resonance image to obtain a preprocessed brain magnetic resonance image;

[0009] Based on a multi-scale convolutional neural network, a hierarchical feature processing mechanism is used to extract features from the preprocessed brain magnetic resonance image to obtain image features;

[0010] Calculating the attention weight of each pixel in the image feature, generating a heat map according to the attention weight, and binarizing the heat map based on a preset threshold to obtain a candidate region containing the region of interest;

[0011] Adaptively adjust the convolution kernel parameters according to the size of the candidate region, use spatial pyramid pooling structures of different scales to extract multi-scale features of the candidate region, and fuse the multi-scale features through a feature fusion network to obtain enhanced candidate region features;

[0012] Inputting the enhanced candidate region features into a segmentation module, performing semantic segmentation on the region of interest through the segmentation module to obtain a segmentation result of the glioma;

[0013] The glioma region in the preprocessed brain magnetic resonance image is marked based on the segmentation result to obtain a glioma segmentation image.

[0014] Based on a multi-scale convolutional neural network, a hierarchical feature processing mechanism is used to extract features from the pre-processed brain magnetic resonance image to obtain image features, including:

[0015] Constructing a multi-scale convolutional neural network model, inputting the preprocessed brain magnetic resonance image into a feature extraction layer of the multi-scale convolutional neural network model, and extracting initial features of the preprocessed brain magnetic resonance image through the feature extraction layer;

[0016] Inputting the initial features into the feature enhancement layer of the multi-scale convolutional neural network model, performing feature decomposition on the initial features using convolution kernels of different scales to obtain feature maps of multiple scales;

[0017] Calculating a spatial attention coefficient and a channel attention coefficient for each scale feature map based on an attention mechanism, and performing weighted fusion on the feature maps of the multiple scales according to the spatial attention coefficient and the channel attention coefficient to obtain an enhanced feature map;

[0018] Inputting the enhanced feature map into the feature optimization layer of the multi-scale convolutional neural network model, constructing a residual connection module, and inputting the enhanced feature map into multiple parallel residual branches respectively, each of the residual branches is provided with a different number of convolutional layers;

[0019] A feature importance score is calculated for the output feature of each residual branch, and the output features of each residual branch are dynamically fused based on the feature importance score to obtain an optimized feature map, and image features are output based on the optimized feature map.

[0020] Calculating the attention weight of each pixel in the image feature, generating a heat map according to the attention weight, and binarizing the heat map based on a preset threshold to obtain a candidate region containing the region of interest, including:

[0021] Constructing a spatial attention module and a channel attention module, and inputting the image features into the spatial attention module and the channel attention module respectively;

[0022] A multi-scale receptive field unit is constructed through the spatial attention module to extract multi-level spatial context information of the image features, and a non-local correlation calculation unit is used to obtain the spatial dependency of pixels in the multi-level spatial context information to obtain spatial attention weights;

[0023] A channel interaction learning unit is constructed through the channel attention module to establish a correlation mapping between the channels of the image features, and a channel recalibration unit is used to adaptively calibrate the channel features to obtain a channel attention weight;

[0024] Fusing the spatial attention weight with the channel attention weight to obtain a mixed attention weight for each pixel in the image feature;

[0025] An attention weight enhancement module is constructed based on a deep separable convolutional network, and the hybrid attention weight is enhanced and progressively optimized layer by layer by the attention weight enhancement module to generate a high-resolution heat map;

[0026] An adaptive segmentation threshold of the high-resolution heat map is calculated based on a region growing algorithm, and the high-resolution heat map is binarized to obtain a candidate region containing the region of interest.

[0027] An attention weight enhancement module is constructed based on a deep separable convolutional network. The attention weight enhancement module performs layer-by-layer feature enhancement and progressive optimization on the hybrid attention weight to generate a high-resolution heat map, including:

[0028] Inputting the hybrid attention weight into the feature extraction unit of the depthwise separable convolutional network, extracting the channel features of the hybrid attention weight through the point-by-point convolutional layer in the feature extraction unit, and extracting the spatial features of the channel features through the depthwise convolutional layer in the feature extraction unit to obtain multidimensional basic features;

[0029] Inputting the multidimensional basic features into the feature optimization unit of the depthwise separable convolutional network, constructing a multi-scale residual connection module through the feature optimization unit to perform feature decomposition on the multidimensional basic features, decomposing the multidimensional basic features into feature subgraphs of different scales, and performing residual connection optimization on the feature subgraphs to obtain hierarchical enhanced features;

[0030] Inputting the hierarchical enhanced features into the first feature enhancement unit of the depthwise separable convolutional network, constructing a hierarchical feature enhancement network through the first feature enhancement unit, performing progressive feature extraction on the hierarchical enhanced features using a dense connection structure, and introducing an attention feedback mechanism in each progressive layer to adaptively calibrate the features to obtain multiple levels of calibrated features;

[0031] A pyramid feature fusion network is constructed through the first feature enhancement unit, cross-level feature alignment and dynamic weight allocation are performed on the calibration features of the multiple levels, and the aligned features are adaptively fused to generate a high-resolution heat map.

[0032] Adaptively adjust the convolution kernel parameters according to the size of the candidate region, use spatial pyramid pooling structures of different scales to extract multi-scale features of the candidate region, and fuse the multi-scale features through a feature fusion network to obtain enhanced candidate region features, including:

[0033] Perform multi-dimensional feature analysis on the candidate area through a deep attention network to generate a spatial dimension feature vector and a channel dimension feature vector, perform dynamic feature fusion on the spatial dimension feature vector and the channel dimension feature vector to generate geometric feature parameters;

[0034] Inputting the geometric feature parameters into a feature mapping network, constructing an adaptive parameter generation module through the feature mapping network, performing nonlinear mapping and progressive optimization on the geometric feature parameters, and generating adaptive convolution parameters that match the size of the candidate region;

[0035] Constructing a multi-level spatial pyramid pooling network, wherein the multi-level spatial pyramid pooling network adopts a cascade structure to set multiple scale-variable pooling branches, and inputting the candidate region and the adaptive convolution parameters into the multi-level spatial pyramid pooling network;

[0036] Performing hierarchical feature extraction on the candidate region through the pooling branches of the multi-level spatial pyramid pooling network, constructing a second feature enhancement unit in each pooling branch in combination with the adaptive convolution parameters, and adaptively recalibrating the extracted features through the second feature enhancement unit to generate a multi-scale feature representation;

[0037] A cross-scale feature fusion network is constructed, and feature alignment and hierarchical adaptive fusion are performed on the multi-scale feature representations through the cross-scale feature fusion network to obtain enhanced candidate region features.

[0038] The method further comprises: extracting hierarchical features of the candidate region by using the pooling branches of the multi-level spatial pyramid pooling network; constructing a second feature enhancement unit in each pooling branch in combination with the adaptive convolution parameters; and adaptively recalibrating the extracted features by using the second feature enhancement unit to generate a multi-scale feature representation, including:

[0039] Perform multi-scale feature analysis on the candidate region through a deep feature extraction network, wherein the deep feature extraction network uses a residual connection structure to perform feature decomposition on the candidate region to generate a multi-level feature vector;

[0040] Inputting the multi-level feature vector into an attention parameter network, performing channel feature response calculation and spatial dependency calculation on the multi-level feature vector through the attention parameter network to generate a channel enhancement factor and a spatial enhancement factor;

[0041] Constructing a dynamic parameter generation module based on the channel enhancement factor and the spatial enhancement factor, and generating adaptive convolution parameters through the dynamic parameter generation module;

[0042] The multi-level feature vector is subjected to hierarchical feature extraction by the pooling branch of the multi-level spatial pyramid pooling network, wherein the pooling branch extracts features layer by layer from fine granularity to coarse granularity to generate multi-level initial features;

[0043] Constructing a second feature enhancement unit in each pooling branch of the multi-level spatial pyramid pooling network, wherein the second feature enhancement unit uses the adaptive convolution parameters to perform a nonlinear transformation on the multi-level initial features to obtain transformed features;

[0044] The transformed features are adaptively recalibrated using the channel enhancement factor and the spatial enhancement factor to generate a multi-scale feature representation.

[0045] Inputting the enhanced candidate region features into a segmentation module, performing semantic segmentation on the region of interest by the segmentation module, and obtaining a segmentation result of the glioma, including:

[0046] Inputting the enhanced candidate region features into a segmentation module, performing multi-scale feature decomposition on the enhanced candidate region features through a feature extraction unit in the segmentation module to generate hierarchical semantic features;

[0047] Inputting the hierarchical semantic features into a regional correlation calculation module, calculating the cross-region feature dependency of the hierarchical semantic features by the regional correlation calculation module, and generating a spatial enhancement factor;

[0048] Performing region of interest detection on the hierarchical semantic features by a region positioning unit in the segmentation module to generate a region feature map;

[0049] Building a boundary perception module based on the region feature map, extracting boundary information of the region of interest through the boundary perception module, and generating boundary constraint features;

[0050] Performing initial semantic segmentation on the region of interest by a segmentation processing unit in the segmentation module to generate an initial segmentation boundary;

[0051] A boundary mapping relationship is established based on the initial segmentation boundary, and the initial segmentation boundary is feature remapped and the boundary is reconstructed in combination with the boundary constraint feature and the spatial enhancement factor; the reconstructed boundary is feature remapped and the boundary is reconstructed again, and the segmentation result of the glioma is generated through multiple rounds of iterative boundary optimization.

[0052] A second aspect of an embodiment of the present invention provides a brain glioma segmentation system based on regions of interest combined with artificial intelligence, comprising:

[0053] The first unit is configured to receive a brain magnetic resonance image acquired by a medical imaging device and preprocess the brain magnetic resonance image to obtain a preprocessed brain magnetic resonance image;

[0054] The second unit is configured to extract features from the preprocessed brain magnetic resonance image using a hierarchical feature processing mechanism based on a multi-scale convolutional neural network to obtain image features;

[0055] a third unit, configured to calculate an attention weight of each pixel in the image feature, generate a heat map according to the attention weight, and binarize the heat map based on a preset threshold to obtain a candidate region containing the region of interest;

[0056] The fourth unit is configured to adaptively adjust the convolution kernel parameters according to the size of the candidate region, extract multi-scale features of the candidate region using spatial pyramid pooling structures of different scales, and fuse the multi-scale features through a feature fusion network to obtain enhanced candidate region features;

[0057] A fifth unit is configured to input the enhanced candidate region features into a segmentation module, and perform semantic segmentation on the region of interest through the segmentation module to obtain a segmentation result of the glioma;

[0058] The sixth unit is configured to mark the glioma region in the preprocessed brain magnetic resonance image based on the segmentation result to obtain a glioma segmentation image.

[0059] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including:

[0060] processor;

[0061] a memory for storing processor-executable instructions;

[0062] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0063] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0064] The beneficial effects of this application are as follows:

[0065] The present invention adopts a multi-scale convolutional neural network and a hierarchical feature processing mechanism to extract image features, combines it with an attention mechanism to generate a heat map and perform candidate region screening, effectively improving the positioning accuracy of the brain glioma area, reducing background interference, and improving segmentation efficiency.

[0066] The present invention adaptively adjusts the convolution kernel parameters according to the size of the candidate region and extracts multi-scale features through a spatial pyramid pooling structure, which solves the problem that traditional methods are difficult to handle gliomas of different sizes and morphologies, and enhances the adaptability of the algorithm to complex lesion areas.

[0067] The present invention effectively integrates multi-scale features through a feature fusion network and uses a semantic segmentation module to achieve refined segmentation, significantly improving the segmentation accuracy of brain glioma boundaries and providing a more reliable imaging basis for clinical diagnosis and treatment planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 Schematic diagram of the flow of a method for brain glioma segmentation based on regions of interest combined with artificial intelligence according to an embodiment of the present invention;

[0069] Figure 2 Generate heatmaps and candidate region extraction processes for the hybrid attention mechanism of this application;

[0070] Figure 3 Schematic diagram of the semantic segmentation framework for glioma based on multi-layer feature optimization and boundary reconstruction. DETAILED DESCRIPTION

[0071] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0072] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0073] Figure 1 FIG. 1 is a flow chart of a method for segmenting glioma based on regions of interest combined with artificial intelligence according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0074] receiving a brain magnetic resonance image acquired by a medical imaging device, and preprocessing the brain magnetic resonance image to obtain a preprocessed brain magnetic resonance image;

[0075] Based on a multi-scale convolutional neural network, a hierarchical feature processing mechanism is used to extract features from the preprocessed brain magnetic resonance image to obtain image features;

[0076] Calculating the attention weight of each pixel in the image feature, generating a heat map according to the attention weight, and binarizing the heat map based on a preset threshold to obtain a candidate region containing the region of interest;

[0077] Adaptively adjust the convolution kernel parameters according to the size of the candidate region, use spatial pyramid pooling structures of different scales to extract multi-scale features of the candidate region, and fuse the multi-scale features through a feature fusion network to obtain enhanced candidate region features;

[0078] Inputting the enhanced candidate region features into a segmentation module, performing semantic segmentation on the region of interest through the segmentation module to obtain a segmentation result of the glioma;

[0079] The glioma region in the preprocessed brain magnetic resonance image is marked based on the segmentation result to obtain a glioma segmentation image.

[0080] In an optional embodiment, based on a multi-scale convolutional neural network, a hierarchical feature processing mechanism is used to extract features from the preprocessed brain magnetic resonance image to obtain image features, including:

[0081] Constructing a multi-scale convolutional neural network model, inputting the preprocessed brain magnetic resonance image into a feature extraction layer of the multi-scale convolutional neural network model, and extracting initial features of the preprocessed brain magnetic resonance image through the feature extraction layer;

[0082] Inputting the initial features into the feature enhancement layer of the multi-scale convolutional neural network model, performing feature decomposition on the initial features using convolution kernels of different scales to obtain feature maps of multiple scales;

[0083] Calculating a spatial attention coefficient and a channel attention coefficient for each scale feature map based on an attention mechanism, and performing weighted fusion on the feature maps of the multiple scales according to the spatial attention coefficient and the channel attention coefficient to obtain an enhanced feature map;

[0084] Inputting the enhanced feature map into the feature optimization layer of the multi-scale convolutional neural network model, constructing a residual connection module, and inputting the enhanced feature map into multiple parallel residual branches respectively, each of the residual branches is provided with a different number of convolutional layers;

[0085] A feature importance score is calculated for the output feature of each residual branch, and the output features of each residual branch are dynamically fused based on the feature importance score to obtain an optimized feature map, and image features are output based on the optimized feature map.

[0086] In this example, the brain MRI images are first preprocessed, including image normalization, denoising, and enhancement. After preprocessing, the image pixel values are scaled to the range [0, 1], noise is removed using Gaussian filtering, and image contrast is enhanced using histogram equalization, making the image more suitable for subsequent feature extraction.

[0087] A multi-scale convolutional neural network model was constructed, consisting of a feature extraction layer, a feature enhancement layer, and a feature optimization layer. The feature extraction layer consisted of three convolutional blocks, each of which contained two 3×3 convolutional layers and a maximum pooling layer. The first convolutional block had 32 kernels, the second 64, and the third 128 kernels. Reinforced linear unit (ReLU) activation function was used in all cases. Preprocessed brain magnetic resonance images (256×256×1) were input to the feature extraction layer. Initial image features were extracted through layer-by-layer convolution and pooling operations, resulting in an output feature map of 32×32×128 size.

[0088] The initial features are fed into the feature enhancement layer, which uses a multi-scale convolutional architecture to decompose the initial features using convolution kernels of different scales. Specifically, four convolution kernels of different sizes (1×1, 3×3, 5×5, and 7×7) are used, with 64 kernels of each type. Convolution operations are performed on the initial features to produce four feature maps of different scales, each with a size of 32×32×64.

[0089] The spatial attention coefficient and channel attention coefficient are calculated for each scale feature map based on the attention mechanism. For spatial attention, each feature map is average pooled and max pooled in the channel dimension to obtain two 32×32×1 feature maps. These two feature maps are concatenated and passed through a 7×7 convolutional layer to output a 32×32×1 spatial attention map. This is then normalized using the sigmoid function to obtain the spatial attention coefficient. For channel attention, each feature map is average pooled and max pooled in the spatial dimension to obtain two 1×1×64 feature vectors. These two feature vectors are concatenated and passed through two fully connected layers (the first fully connected layer reduces the number of channels to 1 / 16 of the original number, and the second fully connected layer restores the original number of channels). This outputs a 1×1×64 channel attention vector. This is then normalized using the sigmoid function to obtain the channel attention coefficient.

[0090] The feature maps of multiple scales are weighted and fused according to the spatial attention coefficient and the channel attention coefficient. Specifically, each scale feature map is multiplied by the corresponding spatial attention coefficient and the channel attention coefficient to obtain a weighted feature map. Then, the four weighted feature maps are element-wise summed to obtain an enhanced feature map with a size of 32×32×64.

[0091] The enhanced feature maps are fed into the feature optimization layer to construct a residual connection module. The feature optimization layer consists of three parallel residual branches, each with a different number of convolutional layers. The first residual branch contains one 3×3 convolutional layer, the second residual branch contains two 3×3 convolutional layers, and the third residual branch contains three 3×3 convolutional layers. Each convolutional layer has 64 convolution kernels and uses the ReLU activation function. Each residual branch also contains a skip connection that adds the input features to the output features of the convolutional layer, forming a residual learning structure.

[0092] The feature importance score is calculated for the output features of each residual branch. The output features of each residual branch (size 32×32×64) are converted into a 1×1×64 feature vector through global average pooling. The feature vector is then mapped to a scalar value through a fully connected layer and normalized using the Softmax function to obtain the feature importance score of each residual branch.

[0093] The output features of each residual branch are dynamically fused based on the feature importance score. The output features of each residual branch are multiplied by the corresponding feature importance score, and then the weighted features are element-wise summed to obtain the optimized feature map with a size of 32×32×64.

[0094] Finally, the optimized feature map is converted into a 1×1×64 feature vector through a global average pooling layer, and then the feature vector is mapped into the final image feature through a fully connected layer with a feature dimension of 256. These features can be used for subsequent brain disease classification or other tasks.

[0095] In practical applications, this method has demonstrated promising results in diagnosing Alzheimer's disease. Experiments were conducted using a dataset of 1,000 brain magnetic resonance images, 500 of which were from Alzheimer's patients and 500 from a healthy control group. Using five-fold cross-validation, this method achieved a classification accuracy of 93.5% on the test set, a 5.2 percentage point improvement over traditional single-scale convolutional neural networks, demonstrating the effectiveness of its multi-scale feature extraction and hierarchical feature processing mechanisms.

[0096] In an optional embodiment, the attention weight of each pixel in the image feature is calculated, a heat map is generated according to the attention weight, and the heat map is binarized based on a preset threshold to obtain a candidate region containing the region of interest, including:

[0097] Constructing a spatial attention module and a channel attention module, and inputting the image features into the spatial attention module and the channel attention module respectively;

[0098] A multi-scale receptive field unit is constructed through the spatial attention module to extract multi-level spatial context information of the image features, and a non-local correlation calculation unit is used to obtain the spatial dependency of pixels in the multi-level spatial context information to obtain spatial attention weights;

[0099] A channel interaction learning unit is constructed through the channel attention module to establish a correlation mapping between the channels of the image features, and a channel recalibration unit is used to adaptively calibrate the channel features to obtain a channel attention weight;

[0100] Fusing the spatial attention weight with the channel attention weight to obtain a mixed attention weight for each pixel in the image feature;

[0101] An attention weight enhancement module is constructed based on a deep separable convolutional network, and the hybrid attention weight is enhanced and progressively optimized layer by layer by the attention weight enhancement module to generate a high-resolution heat map;

[0102] An adaptive segmentation threshold of the high-resolution heat map is calculated based on a region growing algorithm, and the high-resolution heat map is binarized to obtain a candidate region containing the region of interest.

[0103] Figure 2For this application, the hybrid attention mechanism generates heat maps and candidate region extraction processes, such as Figure 2 As shown, in this embodiment, a method for calculating the attention weight of each pixel in the image features and generating a heat map is provided. The method first constructs a spatial attention module and a channel attention module, and inputs the image features into these two modules for processing.

[0104] The spatial attention module extracts multi-level spatial contextual information from image features by constructing multi-scale receptive field units. Specifically, the module convolves the input image features with three convolution kernels of different sizes: 3×3, 5×5, and 7×7. Each convolution kernel has a stride of 1 and padding of "same" to maintain the size of the feature map. These three convolution kernels of different sizes capture spatial contextual information within different receptive fields. The three convolution results are then element-wise weighted summed with weights of 0.3, 0.3, and 0.4, respectively, to produce the fused multi-scale spatial features.

[0105] The non-local correlation calculation unit is used to obtain the spatial dependencies of pixels in multi-level spatial context information. This unit first reduces the fused multi-scale spatial features to 1 / 8 of the original number of channels through a 1×1 convolution, obtaining the query feature Q and the key feature K. The similarity matrix between Q and K is then calculated using a dot product operation followed by softmax normalization. Simultaneously, the original multi-scale spatial features are subjected to another 1×1 convolution to obtain the value feature V. Finally, the normalized similarity matrix is matrix multiplied with V and added to the original features through a residual connection to obtain the spatial attention weights that incorporate global spatial dependencies.

[0106] The channel attention module first constructs a channel interaction learning unit to establish a correlation map between image feature channels. This unit uses global average pooling to compress the spatial dimensions, resulting in a global descriptor for each channel. It then transforms features using a bottleneck structure consisting of two fully connected layers. The first fully connected layer reduces the number of channels to 1 / 16 of the original number using the ReLU activation function; the second fully connected layer restores the number of channels to the original number. This design effectively captures nonlinear relationships between channels while reducing the number of parameters.

[0107] The channel recalibration unit adaptively calibrates channel features. This unit first applies a sigmoid activation to the features output by the channel interaction learning unit, mapping the values to a range between 0 and 1 as the importance weights of each channel. These weights are then multiplied element-wise by the original input features along the channel dimension to achieve adaptive adjustment of each channel and obtain the channel attention weights.

[0108] The spatial attention weights are fused with the channel attention weights to obtain the hybrid attention weights. The specific fusion method is to perform element-wise multiplication of the spatial attention weights and the channel attention weights, and then perform feature integration through 1×1 convolution to obtain the final hybrid attention weights.

[0109] An attention weight enhancement module is constructed based on a depthwise separable convolutional network, performing layer-by-layer feature enhancement and progressive optimization of the hybrid attention weights. This module contains three depthwise separable convolutional blocks, each consisting of a depthwise convolution (kernel_size=3×3, stride=1, padding="same") and a pointwise convolution (kernel_size=1×1). The first block has 64 output channels, the second has 32, and the third has 16. Each depthwise separable convolutional block is followed by a BatchNormalization layer and a ReLU activation function. Finally, a 1×1 convolution is performed to reduce the number of channels to 1, and the feature map is upsampled to the resolution of the original input image using bilinear interpolation to generate a high-resolution heatmap.

[0110] The adaptive segmentation threshold of the high-resolution heat map is calculated based on the region growing algorithm. First, the histogram of the heat map is calculated to count the pixel value distribution. The histogram is divided into 256 bins, and the width of each bin is 1 / 256. Then the mean μ and standard deviation σ of the heat map are calculated, and the initial threshold T0 is set to μ+0.5σ. Next, iterative optimization is performed: the heat map is divided into the foreground area F and the background area B according to the current threshold Tk, and the mean values μF and μB of the foreground area and the background area are calculated respectively, and the threshold Tk+1=(μF+μB) / 2 is updated. The iteration is stopped when |Tk+1-Tk|<0.001 or the number of iterations reaches 50, and the final adaptive segmentation threshold T is obtained.

[0111] The high-resolution heatmap is binarized using the resulting adaptive segmentation threshold T. For each pixel in the heatmap, if its value is greater than or equal to the threshold T, it is set to 1 (foreground), otherwise it is set to 0 (background). The resulting binary image contains candidate regions of interest.

[0112] In practical applications, using a 512×512 pixel medical image as an example, the heat map generated by the above method clearly highlights the lesion area. Using an adaptive threshold T=0.427 for binarization, the method successfully segmented a candidate lesion region of 1243 pixels, achieving 91.5% overlap with the area manually annotated by professional doctors, demonstrating the effectiveness of this method in detecting regions of interest in medical images.

[0113] Table 1: Candidate region generation results for different types of gliomas

[0114]

[0115] As shown in Table 1, five different types of glioma test cases were recorded. Case 1 is a low-grade astrocytoma with a tumor size of 876 pixels. The calculated threshold is 0.382, achieving an 89.3% overlap and a false positive rate of 3.5%. Case 2 is a glioblastoma multiforme with the largest tumor area, 1243 pixels. The calculated threshold is 0.427, achieving the highest overlap of 91.5% and a false positive rate of only 2.8%. Case 3 is an oligodendroglioma with an area of 652 pixels. The calculated threshold is 0.415, achieving an overlap of 90.7%. Case 4 is an ependymoma with an area of 945 pixels and a lower overlap of 88.6%. Case 5 is an anaplastic astrocytoma with an area of 1087 pixels. The calculated threshold is 0.431, achieving an overlap of 92.3% and the lowest false positive rate of only 2.5%. The average tumor size was 960.6 pixels, the average calculation threshold was 0.411, the average overlap reached 90.5%, and the average false positive rate was 3.2%. These data demonstrate that the method of this application has good adaptability and stability for brain gliomas of different types and sizes.

[0116] In an optional embodiment, an attention weight enhancement module is constructed based on a deep separable convolutional network, and the hybrid attention weight is subjected to layer-by-layer feature enhancement and progressive optimization by the attention weight enhancement module to generate a high-resolution heat map, including:

[0117] Inputting the hybrid attention weight into the feature extraction unit of the depthwise separable convolutional network, extracting the channel features of the hybrid attention weight through the point-by-point convolutional layer in the feature extraction unit, and extracting the spatial features of the channel features through the depthwise convolutional layer in the feature extraction unit to obtain multidimensional basic features;

[0118] Inputting the multidimensional basic features into the feature optimization unit of the depthwise separable convolutional network, constructing a multi-scale residual connection module through the feature optimization unit to perform feature decomposition on the multidimensional basic features, decomposing the multidimensional basic features into feature subgraphs of different scales, and performing residual connection optimization on the feature subgraphs to obtain hierarchical enhanced features;

[0119] Inputting the hierarchical enhanced features into the first feature enhancement unit of the depthwise separable convolutional network, constructing a hierarchical feature enhancement network through the first feature enhancement unit, performing progressive feature extraction on the hierarchical enhanced features using a dense connection structure, and introducing an attention feedback mechanism in each progressive layer to adaptively calibrate the features to obtain multiple levels of calibrated features;

[0120] A pyramid feature fusion network is constructed through the first feature enhancement unit, cross-level feature alignment and dynamic weight allocation are performed on the calibration features of the multiple levels, and the aligned features are adaptively fused to generate a high-resolution heat map.

[0121] The hybrid attention weights are input to the feature extraction unit of the depthwise separable convolutional network for processing. The feature extraction unit consists of multiple depthwise separable convolutional blocks, each of which contains a pointwise convolution layer and a depthwise convolution layer. The pointwise convolution layer uses 1×1 convolution kernels to extract channel-dimensional features from the hybrid attention weights. The number of convolution kernels is set to 64, with a stride of 1 and no padding. After the pointwise convolution, the channel feature map is obtained, with a size of H×W×64, where H and W are the height and width of the input feature map, respectively. The depthwise convolution layer uses 3×3 convolution kernels to extract spatial features from the channel features. Each channel uses a separate convolution kernel with a stride of 1 and padding of 1, maintaining the feature map size. After the depthwise convolution, the spatial feature map is obtained, with a size of H×W×64. Three consecutive depthwise separable convolution blocks are set in the feature extraction unit, with batch normalization layers and ReLU activation functions added between each block. The final output is a multidimensional basic feature map with a size of H×W×128.

[0122] The multidimensional basic features are input into the feature optimization unit of the depthwise separable convolutional network for processing. The feature optimization unit constructs a multi-scale residual connection module to perform feature decomposition on the multidimensional basic features. First, three parallel depthwise separable convolution branches are used to extract features at different scales from the input features. The first branch uses a 3×3 convolution kernel with a stride of 1 and padding of 1; the second branch uses a 5×5 convolution kernel with a stride of 1 and padding of 2; and the third branch uses a 7×7 convolution kernel with a stride of 1 and padding of 3. Each of the three branches generates a feature subgraph of size H×W×64. For each feature subgraph, a residual connection structure is constructed. The original input features are reduced in dimension through 1×1 convolution and then element-wise fused with the corresponding feature subgraph by element-wise addition. The three feature subgraphs optimized by the residual connection are then merged through channel concatenation to obtain a hierarchical enhanced feature of size H×W×192.

[0123] The hierarchical enhanced features are input to the first feature enhancement unit of a depthwise separable convolutional network for processing. The first feature enhancement unit constructs a hierarchical feature enhancement network, which uses a densely connected structure to progressively extract features from the hierarchical enhanced features. The network consists of four progressive layers, each consisting of a depthwise separable convolutional block and an attention feedback module. In each progressive layer, the depthwise separable convolutional block extracts features using a 3×3 convolution kernel with a stride of 1, padding of 1, and 64 output channels. The attention feedback mechanism is implemented by combining spatial attention and channel attention. Spatial attention generates two spatial descriptors through global average pooling and max pooling. These two descriptors are concatenated and passed through a 7×7 convolutional layer with a sigmoid activation function to generate spatial attention weights. Channel attention generates two channel descriptors through global average pooling and max pooling. These two descriptors are processed separately through a shared multi-layer perceptron, added together, and then activated with a sigmoid activation function to generate channel attention weights. The spatial attention weights are multiplied with the channel attention weights to generate a combined attention weight, which is used for adaptive feature calibration. After processing through four progressive layers, four calibration features at different levels are obtained, with sizes of H×W×64, H×W×64, H×W×64, and H×W×64 respectively.

[0124] The first feature enhancement unit also constructs a pyramid feature fusion network to perform cross-level feature alignment and dynamic weight assignment for the calibrated features at multiple levels. First, the calibrated features at the four levels are upsampled or downsampled to a consistent size of H×W×64. For lower-resolution features, bilinear interpolation is used for upsampling; for higher-resolution features, 2×2 max pooling with a stride of 2 is used for downsampling. After feature alignment, a channel attention module is constructed to assign dynamic weights to each level of features. This module compresses features into 1×1×64 vectors using global average pooling. Channel weight vectors are then generated through two fully connected layers and a sigmoid activation function. Each level feature is multiplied by the corresponding channel weight vector to achieve adaptive feature weighting. Finally, the weighted features from the four levels are fused element-wise by addition, resulting in a fused feature of size H×W×64. The number of channels of the fused features is adjusted to 1 through a 1×1 convolution layer, and then normalized to the range of 0-1 through the Sigmoid activation function to generate the final high-resolution heat map with a size of H×W×1.

[0125] In practice, the input image size is 224×224×3. After the above processing, the resulting high-resolution heatmap is 56×56×1. This can be upsampled to the same size as the original image, 224×224×1, using bilinear interpolation to visualize the importance distribution of target regions. Experimental results show that this method can improve the average precision (AP) by 2.7 percentage points in object detection tasks and by 1.8 percentage points in image classification tasks.

[0126] In an optional embodiment, the convolution kernel parameters are adaptively adjusted according to the size of the candidate region, a spatial pyramid pooling structure of different scales is used to extract multi-scale features of the candidate region, and the multi-scale features are fused through a feature fusion network to obtain enhanced candidate region features, including:

[0127] Perform multi-dimensional feature analysis on the candidate area through a deep attention network to generate a spatial dimension feature vector and a channel dimension feature vector, perform dynamic feature fusion on the spatial dimension feature vector and the channel dimension feature vector to generate geometric feature parameters;

[0128] Inputting the geometric feature parameters into a feature mapping network, constructing an adaptive parameter generation module through the feature mapping network, performing nonlinear mapping and progressive optimization on the geometric feature parameters, and generating adaptive convolution parameters that match the size of the candidate region;

[0129] Constructing a multi-level spatial pyramid pooling network, wherein the multi-level spatial pyramid pooling network adopts a cascade structure to set multiple scale-variable pooling branches, and inputting the candidate region and the adaptive convolution parameters into the multi-level spatial pyramid pooling network;

[0130] Performing hierarchical feature extraction on the candidate region through the pooling branches of the multi-level spatial pyramid pooling network, constructing a second feature enhancement unit in each pooling branch in combination with the adaptive convolution parameters, and adaptively recalibrating the extracted features through the second feature enhancement unit to generate a multi-scale feature representation;

[0131] A cross-scale feature fusion network is constructed, and feature alignment and hierarchical adaptive fusion are performed on the multi-scale feature representations through the cross-scale feature fusion network to obtain enhanced candidate region features.

[0132] In this embodiment, the convolution kernel parameters are adaptively adjusted according to the size of the candidate region, and the multi-scale features of the candidate region are extracted using spatial pyramid pooling structures of different scales. The multi-scale features are fused through a feature fusion network to obtain the enhanced candidate region features. The specific implementation process is as follows:

[0133] The system first performs multi-dimensional feature analysis on candidate regions using a deep attention network. This deep attention network consists of a spatial attention module and a channel attention module. The spatial attention module processes the input candidate region feature map using a 3×3 convolutional layer to generate a spatial weight map, which reflects the importance of different locations in the feature map. Specifically, it performs max pooling and average pooling on the input feature map F to obtain two feature maps, F_max and F_avg. These two feature maps are then concatenated along the channel dimension and passed through a 3×3 convolutional layer and a sigmoid activation function to generate a spatial feature vector S. The channel attention module then generates two channel descriptors using global average pooling and global max pooling, respectively. These two descriptors are processed by a shared multi-layer perceptron and fused through element-wise addition. Finally, a sigmoid function is used to generate a channel feature vector C. The spatial feature vector S has dimensions H×W×1, and the channel feature vector C has dimensions 1×1×C, where H and W represent the height and width of the feature map, respectively, and C represents the number of channels.

[0134] Dynamic feature fusion is performed on the spatial dimension feature vector S and the channel dimension feature vector C to generate the geometric feature parameter G. Specifically, S is globally averaged pooled to obtain the spatial global descriptor S_g, and C is spatially expanded to obtain C_e so that its size is consistent with the input feature Figure 1 The tensor multiplication of S_g and C is performed to obtain the initial fused features F_i. F_i is then residually connected to the input feature map and compressed through a 1×1 convolution layer to obtain the geometric feature parameters G. In practical applications, for an input feature map of size 224×224×64, the dimension of the generated geometric feature parameters G is 128.

[0135] The geometric feature parameters G are input into the feature mapping network. The adaptive parameter generation module is constructed within the feature mapping network. This performs nonlinear mapping and progressive optimization on the geometric feature parameters to generate adaptive convolution parameters that match the size of the candidate region. The feature mapping network adopts a multi-layer perceptron architecture consisting of three fully connected layers, with ReLU activation functions and dropout layers (with a dropout rate of 0.2) between each layer. The first layer maps the 128-dimensional geometric feature parameters G to 256-dimensional intermediate features. The second layer maps the intermediate features to 512-dimensional high-level features. The third layer maps the high-level features to the final adaptive convolution parameters. The parameter dimensions are dynamically adjusted based on the target convolution kernel size. For example, for a 3×3 convolution kernel, the output dimensions are 3×3×C_in×C_out, where C_in and C_out are the number of input and output channels, respectively. To enhance the model's generalization, L2 regularization is applied to the output of the second layer during training, with a regularization coefficient of 0.001.

[0136] A multi-level spatial pyramid pooling network is constructed. This network uses a cascaded structure with multiple scale-adjustable pooling branches. Candidate regions and adaptive convolution parameters are fed into the multi-level spatial pyramid pooling network. The network consists of four pooling branches, corresponding to four different pooling scales: 1×1, 2×2, 3×3, and 6×6. Each pooling branch first performs adaptive average pooling of the input feature map at the corresponding scale, and then uniformly adjusts the number of feature channels to 256 through a 1×1 convolutional layer.

[0137] Hierarchical features are extracted from candidate regions using the pooling branches of a multi-level spatial pyramid pooling network. A second feature enhancement unit is constructed in each pooling branch using adaptive convolution parameters. The extracted features are adaptively recalibrated using the second feature enhancement unit to generate a multi-scale feature representation. The second feature enhancement unit employs an adaptive convolution module, which convolves the pooled features using the previously generated adaptive convolution parameters. Specifically, for the feature F_i extracted by the i-th pooling branch, the corresponding adaptive convolution parameters W_i are used for convolution to generate the enhanced feature E_i. E_i is then recalibrated using a channel attention mechanism to generate the final multi-scale feature representation M_i. In practical applications, for pooled features of size 7×7×256, after adaptive convolution and channel attention recalibration, the feature size remains unchanged, but the discriminability of the features is enhanced.

[0138] A cross-scale feature fusion network is constructed to align and adaptively fuse multi-scale feature representations to obtain enhanced candidate region features. The cross-scale feature fusion network adopts a bidirectional feature fusion strategy, combining top-down and bottom-up approaches. The top-down pathway transfers high-level semantic features to low-level features through upsampling, while the bottom-up pathway fuses low-level detail features into high-level features through skip connections. Specifically, the multi-scale feature representations M_1, M_2, M_3, and M_4 of the four pooling branches are upsampled to the same spatial size (typically 1 / 4 the size of the input feature map) and then channel-aligned through a 1×1 convolutional layer. The aligned features are fused via a weighted summation, with the weight coefficients adaptively determined by learnable parameters. Finally, a 3×3 convolutional layer and a global average pooling layer are used to compress the fused features into a fixed-dimensional vector representation (e.g., 2048 dimensions) to serve as the enhanced candidate region features. On the test dataset, this method improves the average accuracy of target detection by 3.2 percentage points compared to the traditional ROI pooling method, especially for small targets and targets with complex shapes.

[0139] In an optional embodiment, hierarchical feature extraction is performed on the candidate region using the pooling branches of the multi-level spatial pyramid pooling network, a second feature enhancement unit is constructed in each pooling branch in combination with the adaptive convolution parameters, and the extracted features are adaptively recalibrated by the second feature enhancement unit to generate a multi-scale feature representation, including:

[0140] Perform multi-scale feature analysis on the candidate region through a deep feature extraction network, wherein the deep feature extraction network uses a residual connection structure to perform feature decomposition on the candidate region to generate a multi-level feature vector;

[0141] Inputting the multi-level feature vector into an attention parameter network, performing channel feature response calculation and spatial dependency calculation on the multi-level feature vector through the attention parameter network to generate a channel enhancement factor and a spatial enhancement factor;

[0142] Constructing a dynamic parameter generation module based on the channel enhancement factor and the spatial enhancement factor, and generating adaptive convolution parameters through the dynamic parameter generation module;

[0143] The multi-level feature vector is subjected to hierarchical feature extraction by the pooling branch of the multi-level spatial pyramid pooling network, wherein the pooling branch extracts features layer by layer from fine granularity to coarse granularity to generate multi-level initial features;

[0144] Constructing a second feature enhancement unit in each pooling branch of the multi-level spatial pyramid pooling network, wherein the second feature enhancement unit uses the adaptive convolution parameters to perform a nonlinear transformation on the multi-level initial features to obtain transformed features;

[0145] The transformed features are adaptively recalibrated using the channel enhancement factor and the spatial enhancement factor to generate a multi-scale feature representation.

[0146] In this embodiment, hierarchical feature extraction is performed on the candidate region through the pooling branch of the multi-level spatial pyramid pooling network, and the second feature enhancement unit is constructed in combination with the adaptive convolution parameters. The extracted features are adaptively recalibrated to generate a multi-scale feature representation. The specific implementation process is as follows:

[0147] The candidate region is first subjected to multi-scale feature analysis through a deep feature extraction network. The deep feature extraction network uses a residual connection structure to decompose the features of the candidate region and generate a multi-level feature vector. Specifically, the deep feature extraction network contains 5 convolution blocks, each of which contains 3 convolution layers, with a convolution kernel size of 3×3, a stride of 1, and a padding of 1. The number of output channels of the first convolution block is 64, the second convolution block is 128, the third convolution block is 256, the fourth convolution block is 512, and the fifth convolution block is 512. Each convolution block is connected by a maximum pooling layer with a pooling kernel size of 2×2 and a stride of 2. Within each convolution block, a residual connection structure is used, that is, the input features and the convolved features are added element-wise to alleviate the gradient vanishing problem and improve feature extraction capabilities. For example, for a candidate region image with an input size of 224×224×3, the feature map size obtained after the first convolution block is 112×112×64, and the feature map size obtained after the second convolution block is 56×56×128, and so on, finally obtaining a multi-level feature vector.

[0148] The multi-level feature vectors are input into the attention parameter network, which calculates channel feature responses and spatial dependencies on the multi-level feature vectors to generate channel enhancement factors and spatial enhancement factors. Specifically, the attention parameter network consists of a channel attention branch and a spatial attention branch. The channel attention branch first performs global average pooling and global maximum pooling on the input features to obtain two channel descriptors: the average pooled channel descriptor and the maximum pooled channel descriptor. These two descriptors are processed by a shared-weight multilayer perceptron (MLP). The MLP consists of two fully connected layers with a ReLU activation function in between. The first fully connected layer reduces the number of channels to 1 / 16 of the original size, while the second fully connected layer restores the number of channels to their original size. The two processed channel descriptors are element-wise added and the channel enhancement factor is obtained using a sigmoid function. For example, for an input feature size of 14×14×512, global pooling results in a 1×1×512 channel descriptor. After processing by the MLP, the channel enhancement factor is 1×1×512.

[0149] The spatial attention branch performs max pooling and average pooling on the input features in the channel dimension, obtaining two spatial feature maps. These two feature maps are then concatenated in the channel dimension and processed through a 7×7 convolution layer and a sigmoid function to obtain a spatial enhancement factor. For example, for an input feature size of 14×14×512, channel pooling yields two 14×14×1 spatial feature maps. These concatenation yields a 14×14×2 feature map. After convolution and sigmoid function processing, the spatial enhancement factor is 14×14×1.

[0150] A dynamic parameter generation module is constructed based on the channel enhancement factor and the spatial enhancement factor, and adaptive convolution parameters are generated through the dynamic parameter generation module. Specifically, the dynamic parameter generation module first performs feature fusion on the channel enhancement factor and the spatial enhancement factor, and combines the two using element-level multiplication operations to obtain comprehensive enhanced features. Then, a 1×1 convolution layer is used to map the comprehensive enhanced features to the parameter space to generate adaptive convolution parameters. The adaptive convolution parameters include convolution kernel weights and bias terms, which are used in subsequent feature enhancement units. For example, for a channel enhancement factor of size 1×1×512 and a spatial enhancement factor of size 14×14×1, a comprehensive enhanced feature of size 14×14×512 is obtained after feature fusion, and adaptive convolution parameters are generated through a 1×1 convolution layer.

[0151] The multi-level feature vectors are passed through the pooling branches of the multi-level spatial pyramid pooling network for hierarchical feature extraction. The pooling branches extract features layer by layer from fine-grained to coarse-grained, generating multi-level initial features. Specifically, the multi-level spatial pyramid pooling network contains four pooling branches, corresponding to different pooling scales: 1×1, 2×2, 3×3, and 6×6. Each pooling branch first performs adaptive average pooling of the input features at the corresponding scale, and then unifies the number of channels to 256 through a 1×1 convolutional layer. For example, for an input feature size of 14×14×512, the 1×1 pooling branch produces 1×1×256 features, the 2×2 pooling branch produces 2×2×256 features, and so on.

[0152] A second feature enhancement unit is constructed in each pooling branch of the multi-level spatial pyramid pooling network. The second feature enhancement unit uses adaptive convolution parameters to perform nonlinear transformation on the multi-level initial features to obtain the transformed features. Specifically, the second feature enhancement unit first processes the multi-level initial features through an adaptive convolution layer, which uses the adaptive convolution parameters generated previously. Then, a nonlinear transformation is performed through a batch normalization layer and a ReLU activation function. For example, for the 2×2×256 features obtained by the 2×2 pooling branch, after processing through the adaptive convolution layer, batch normalization layer, and ReLU activation function, the transformed features are obtained, and the size is still 2×2×256.

[0153] The transformed features are adaptively recalibrated using channel enhancement factors and spatial enhancement factors to generate multi-scale feature representations. Specifically, the transformed features are first multiplied element-wise by the channel enhancement factors to achieve feature recalibration in the channel dimension. Then, the result is multiplied element-wise by the spatial enhancement factors to achieve feature recalibration in the spatial dimension. Finally, the recalibrated features of each pooling branch are upsampled to the same size as the original input features and concatenated in the channel dimension to obtain the final multi-scale feature representation. For example, for a transformed feature size of 2×2×256, after channel and spatial recalibration, it is upsampled to 14×14×256 and concatenated with the features of other pooling branches to obtain a multi-scale feature representation of size 14×14×1024.

[0154] In an optional embodiment, the enhanced candidate region features are input into a segmentation module, and the segmentation module performs semantic segmentation on the region of interest to obtain a segmentation result of the glioma, including:

[0155] Inputting the enhanced candidate region features into a segmentation module, performing multi-scale feature decomposition on the enhanced candidate region features through a feature extraction unit in the segmentation module to generate hierarchical semantic features;

[0156] Inputting the hierarchical semantic features into a regional correlation calculation module, calculating the cross-region feature dependency of the hierarchical semantic features by the regional correlation calculation module, and generating a spatial enhancement factor;

[0157] Performing region of interest detection on the hierarchical semantic features by a region positioning unit in the segmentation module to generate a region feature map;

[0158] Building a boundary perception module based on the region feature map, extracting boundary information of the region of interest through the boundary perception module, and generating boundary constraint features;

[0159] Performing initial semantic segmentation on the region of interest by a segmentation processing unit in the segmentation module to generate an initial segmentation boundary;

[0160] A boundary mapping relationship is established based on the initial segmentation boundary, and the initial segmentation boundary is feature remapped and the boundary is reconstructed in combination with the boundary constraint feature and the spatial enhancement factor; the reconstructed boundary is feature remapped and the boundary is reconstructed again, and the segmentation result of the glioma is generated through multiple rounds of iterative boundary optimization.

[0161] Figure 3 Schematic diagram of the semantic segmentation framework of glioma based on multi-layer feature optimization and boundary reconstruction, as shown in Figure 3 As shown in the figure, the enhanced candidate region features are input into the feature extraction unit in the segmentation module. This feature extraction unit uses a multi-layer convolutional neural network structure consisting of five convolutional layers, each with a 3×3 kernel size, a stride of 1, and a padding of 1. The first convolutional layer has the same number of input channels as the enhanced candidate region features, and outputs 64 channels. The second convolutional layer has 64 input channels and 128 output channels. The third convolutional layer has 128 input channels and 256 output channels. The fourth convolutional layer has 256 input channels and 512 output channels. The fifth convolutional layer has 512 input channels and 512 output channels. Each convolutional layer is followed by a batch normalization layer and a Reluctant Unified Unit (ReLU) activation function, and downsampling is performed using a max pooling operation with a 2×2 kernel size and a stride of 2. Through this multi-scale feature decomposition method, hierarchical semantic features containing information of different resolutions are generated, which are denoted as F1, F2, F3, F4 and F5, respectively, where F1 represents the lowest level feature and F5 represents the highest level feature.

[0162] The hierarchical semantic features F1 to F5 are input into the regional correlation calculation module, which uses an attention mechanism to calculate cross-region feature dependencies. Specifically, for each feature layer Fi, the number of channels is first reduced to 1 / 8 of the original through a 1×1 convolution, resulting in query features Qi, key features Ki, and value features Vi. The query features Qi and key features Ki are reshaped into matrices of shape (H×W)×C, where H and W are the height and width of the feature map, respectively, and C is the number of channels. The matrix product of the transposed values of Qi and Ki is calculated to obtain a correlation matrix Ai of size (H×W)×(H×W). Ai is softmax-normalized and then matrix-multiplied with the value features Vi, reshaped to C×(H×W), to obtain weighted features. The weighted features are reshaped back to their original shape (H×W×C) and restored to their original number of channels through a 1×1 convolution. Finally, a residual connection is performed with the original features Fi to obtain the spatial enhancement factors Si. For a sample in the glioma dataset, the spatial enhancement factor Si can effectively capture the correlation between the tumor area and the surrounding tissue and enhance the feature representation of the tumor boundary.

[0163] Regions of interest (ROIs) are detected using hierarchical semantic features using the region localization unit in the segmentation module. This unit uses a region proposal network architecture to process the feature map F5. F5 is first extracted using a 3×3 convolutional layer, followed by two 1×1 convolutional layers to predict region scores and bounding box regression parameters. For the glioma data, nine anchor boxes of different scales and ratios are used, including square anchor boxes of 8×8, 16×16, and 32×32, and rectangular anchor boxes with aspect ratios of 1:2 and 2:1. The top 100 candidate regions are selected based on the predicted scores, and a non-maximum suppression algorithm is applied with an intersection-over-union threshold of 0.7. The 10 regions with the highest scores are retained. These regions are then mapped back to the original image space to generate the region feature map R.

[0164] A boundary-aware module is constructed based on the regional feature map R. This module uses an edge detection network to extract boundary information of the region of interest. In specific implementation, the Sobel operator is used to calculate the horizontal and vertical gradients of the regional feature map R, resulting in gradient magnitude and direction maps. A 3×3 convolutional layer is then used to extract features from the gradient information, using 64 convolution kernels, a stride of 1, and padding of 1. The extracted boundary features are then upsampled to the same resolution as the original input using bilinear interpolation and fused with the low-level features F1 to generate boundary-constrained features B. For glioma data, boundary-constrained features B effectively represent the fine structure of the tumor edge, helping to improve segmentation accuracy.

[0165] The segmentation processing unit in the segmentation module performs initial semantic segmentation on the region of interest. This unit adopts a U-shaped network structure, consisting of an encoder and a decoder. The encoder uses the previously generated hierarchical semantic features F1 to F5; the decoder performs upsampling through transposed convolutions and makes jump connections with the features of the corresponding encoder layer. Specifically, the decoder contains four transposed convolutional layers, each with a 4×4 kernel size, a stride of 2, and a padding of 1. The first transposed convolution layer has 512 input channels and 512 output channels; the second transposed convolution layer has 1024 input channels (512 + 512) and 256 output channels; the third transposed convolution layer has 512 input channels (256 + 256) and 128 output channels; and the fourth transposed convolution layer has 256 input channels (128 + 128) and 64 output channels. Finally, a 1×1 convolutional layer is used to map the number of channels to the number of categories (for the glioma segmentation task, the number of categories is 4, representing background, edema area, enhanced tumor area and necrotic area respectively) to generate the initial segmentation boundary M.

[0166] Based on the initial segmentation boundary M, a boundary mapping relationship is established. The initial segmentation boundary is then remapped and reconstructed by combining the boundary constraint feature B and the spatial enhancement factor S. Specifically, the initial segmentation boundary M is first element-wise multiplied with the boundary constraint feature B to obtain the boundary enhancement feature EB. EB is then weightedly fused with the spatial enhancement factor S, with weights of 0.7 and 0.3, to obtain the remapped feature RM. RM is then processed using a 3×3 convolutional layer with 64 kernels, a stride of 1, and padding of 1 to generate the reconstructed boundary RC.

[0167] The reconstructed boundary RC is re-featured and reconstructed, and the final brain glioma segmentation result is generated through multiple rounds of iterative boundary optimization. In this embodiment, the number of iterations is set to 3. In each iteration, the reconstructed boundary RC of the previous round is used as input, and the aforementioned feature remapping and boundary reconstruction process is repeated. After 3 rounds of iterations, the final feature map is normalized using the softmax function to obtain the probability distribution of each pixel belonging to each category, and the category with the highest probability is taken as the segmentation label of the pixel, thereby generating the final segmentation result of the brain glioma. In practical applications, the test results of this method on the BraTS2020 dataset show that the Dice similarity coefficient reaches 0.91, the sensitivity is 0.89, and the specificity is 0.94, which is about 15% higher than the traditional method.

[0168] The embodiment of the present invention provides a glioma segmentation system based on regions of interest combined with artificial intelligence, including:

[0169] The first unit is configured to receive a brain magnetic resonance image acquired by a medical imaging device and preprocess the brain magnetic resonance image to obtain a preprocessed brain magnetic resonance image;

[0170] The second unit is configured to extract features from the preprocessed brain magnetic resonance image using a hierarchical feature processing mechanism based on a multi-scale convolutional neural network to obtain image features;

[0171] a third unit, configured to calculate an attention weight of each pixel in the image feature, generate a heat map according to the attention weight, and binarize the heat map based on a preset threshold to obtain a candidate region containing the region of interest;

[0172] The fourth unit is configured to adaptively adjust the convolution kernel parameters according to the size of the candidate region, extract multi-scale features of the candidate region using spatial pyramid pooling structures of different scales, and fuse the multi-scale features through a feature fusion network to obtain enhanced candidate region features;

[0173] A fifth unit is configured to input the enhanced candidate region features into a segmentation module, and perform semantic segmentation on the region of interest through the segmentation module to obtain a segmentation result of the glioma;

[0174] The sixth unit is configured to mark the glioma region in the preprocessed brain magnetic resonance image based on the segmentation result to obtain a glioma segmentation image.

[0175] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including:

[0176] processor;

[0177] a memory for storing processor-executable instructions;

[0178] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0179] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0180] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0181] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A glioma segmentation method based on regions of interest combined with artificial intelligence, characterized by: include: receiving a brain magnetic resonance image acquired by a medical imaging device, and preprocessing the brain magnetic resonance image to obtain a preprocessed brain magnetic resonance image; Based on a multi-scale convolutional neural network, a hierarchical feature processing mechanism is used to extract features from the preprocessed brain magnetic resonance image to obtain image features; Calculating the attention weight of each pixel in the image feature, generating a heat map according to the attention weight, and binarizing the heat map based on a preset threshold to obtain a candidate region containing the region of interest, including: Constructing a spatial attention module and a channel attention module, inputting the image features into the spatial attention module and the channel attention module respectively; constructing a multi-scale receptive field unit through the spatial attention module to extract multi-level spatial context information of the image features, using a non-local correlation calculation unit to obtain the spatial dependency of pixels in the multi-level spatial context information, and obtaining a spatial attention weight; A channel interaction learning unit is constructed through the channel attention module to establish a correlation mapping between the channels of the image feature, and a channel recalibration unit is used to adaptively calibrate the channel features to obtain channel attention weights; the spatial attention weights are fused with the channel attention weights to obtain a hybrid attention weight for each pixel in the image feature; An attention weight enhancement module is constructed based on a deep separable convolutional network, and the hybrid attention weight is subjected to layer-by-layer feature enhancement and progressive optimization by the attention weight enhancement module to generate a high-resolution heat map. An adaptive segmentation threshold of the high-resolution heat map is calculated based on a region growing algorithm, and the high-resolution heat map is binarized and segmented to obtain a candidate region containing the region of interest. Adaptively adjust the convolution kernel parameters according to the size of the candidate region, use spatial pyramid pooling structures of different scales to extract multi-scale features of the candidate region, and fuse the multi-scale features through a feature fusion network to obtain enhanced candidate region features; Inputting the enhanced candidate region features into a segmentation module, performing semantic segmentation on the region of interest through the segmentation module to obtain a segmentation result of the glioma; The glioma region in the preprocessed brain magnetic resonance image is marked based on the segmentation result to obtain a glioma segmentation image.

2. The method according to claim 1, characterized in that Based on a multi-scale convolutional neural network, a hierarchical feature processing mechanism is used to extract features from the pre-processed brain magnetic resonance image to obtain image features, including: Constructing a multi-scale convolutional neural network model, inputting the preprocessed brain magnetic resonance image into a feature extraction layer of the multi-scale convolutional neural network model, and extracting initial features of the preprocessed brain magnetic resonance image through the feature extraction layer; Inputting the initial features into the feature enhancement layer of the multi-scale convolutional neural network model, performing feature decomposition on the initial features using convolution kernels of different scales to obtain feature maps of multiple scales; Calculating a spatial attention coefficient and a channel attention coefficient for each scale feature map based on an attention mechanism, and performing weighted fusion on the feature maps of the multiple scales according to the spatial attention coefficient and the channel attention coefficient to obtain an enhanced feature map; Inputting the enhanced feature map into the feature optimization layer of the multi-scale convolutional neural network model, constructing a residual connection module, and inputting the enhanced feature map into multiple parallel residual branches respectively, each of the residual branches is provided with a different number of convolutional layers; A feature importance score is calculated for the output feature of each residual branch, and the output features of each residual branch are dynamically fused based on the feature importance score to obtain an optimized feature map, and image features are output based on the optimized feature map.

3. The method according to claim 1, characterized in that An attention weight enhancement module is constructed based on a deep separable convolutional network. The attention weight enhancement module performs layer-by-layer feature enhancement and progressive optimization on the hybrid attention weight to generate a high-resolution heat map, including: Inputting the hybrid attention weight into the feature extraction unit of the depthwise separable convolutional network, extracting the channel features of the hybrid attention weight through the point-by-point convolutional layer in the feature extraction unit, and extracting the spatial features of the channel features through the depthwise convolutional layer in the feature extraction unit to obtain multidimensional basic features; Inputting the multidimensional basic features into the feature optimization unit of the depthwise separable convolutional network, constructing a multi-scale residual connection module through the feature optimization unit to perform feature decomposition on the multidimensional basic features, decomposing the multidimensional basic features into feature subgraphs of different scales, and performing residual connection optimization on the feature subgraphs to obtain hierarchical enhanced features; Inputting the hierarchical enhanced features into the first feature enhancement unit of the depthwise separable convolutional network, constructing a hierarchical feature enhancement network through the first feature enhancement unit, performing progressive feature extraction on the hierarchical enhanced features using a dense connection structure, and introducing an attention feedback mechanism in each progressive layer to adaptively calibrate the features to obtain multiple levels of calibrated features; A pyramid feature fusion network is constructed through the first feature enhancement unit, cross-level feature alignment and dynamic weight allocation are performed on the calibration features of the multiple levels, and the aligned features are adaptively fused to generate a high-resolution heat map.

4. The method according to claim 1, wherein Adaptively adjust the convolution kernel parameters according to the size of the candidate region, use spatial pyramid pooling structures of different scales to extract multi-scale features of the candidate region, and fuse the multi-scale features through a feature fusion network to obtain enhanced candidate region features, including: Perform multi-dimensional feature analysis on the candidate area through a deep attention network to generate a spatial dimension feature vector and a channel dimension feature vector, perform dynamic feature fusion on the spatial dimension feature vector and the channel dimension feature vector to generate geometric feature parameters; Inputting the geometric feature parameters into a feature mapping network, constructing an adaptive parameter generation module through the feature mapping network, performing nonlinear mapping and progressive optimization on the geometric feature parameters, and generating adaptive convolution parameters that match the size of the candidate region; Constructing a multi-level spatial pyramid pooling network, wherein the multi-level spatial pyramid pooling network adopts a cascade structure to set multiple scale-variable pooling branches, and inputting the candidate region and the adaptive convolution parameters into the multi-level spatial pyramid pooling network; Performing hierarchical feature extraction on the candidate region through the pooling branches of the multi-level spatial pyramid pooling network, constructing a second feature enhancement unit in each pooling branch in combination with the adaptive convolution parameters, and adaptively recalibrating the extracted features through the second feature enhancement unit to generate a multi-scale feature representation; A cross-scale feature fusion network is constructed, and feature alignment and hierarchical adaptive fusion are performed on the multi-scale feature representations through the cross-scale feature fusion network to obtain enhanced candidate region features.

5. The method according to claim 4, characterized in that The method further comprises: extracting hierarchical features of the candidate region by using the pooling branches of the multi-level spatial pyramid pooling network; constructing a second feature enhancement unit in each pooling branch in combination with the adaptive convolution parameters; and adaptively recalibrating the extracted features by using the second feature enhancement unit to generate a multi-scale feature representation, including: Perform multi-scale feature analysis on the candidate region through a deep feature extraction network, wherein the deep feature extraction network uses a residual connection structure to perform feature decomposition on the candidate region to generate a multi-level feature vector; Inputting the multi-level feature vector into an attention parameter network, performing channel feature response calculation and spatial dependency calculation on the multi-level feature vector through the attention parameter network to generate a channel enhancement factor and a spatial enhancement factor; Constructing a dynamic parameter generation module based on the channel enhancement factor and the spatial enhancement factor, and generating adaptive convolution parameters through the dynamic parameter generation module; The multi-level feature vector is subjected to hierarchical feature extraction by the pooling branch of the multi-level spatial pyramid pooling network, wherein the pooling branch extracts features layer by layer from fine granularity to coarse granularity to generate multi-level initial features; Constructing a second feature enhancement unit in each pooling branch of the multi-level spatial pyramid pooling network, wherein the second feature enhancement unit uses the adaptive convolution parameters to perform a nonlinear transformation on the multi-level initial features to obtain transformed features; The transformed features are adaptively recalibrated using the channel enhancement factor and the spatial enhancement factor to generate a multi-scale feature representation.

6. The method according to claim 1, wherein Inputting the enhanced candidate region features into a segmentation module, performing semantic segmentation on the region of interest by the segmentation module, and obtaining a segmentation result of the glioma, including: Inputting the enhanced candidate region features into a segmentation module, performing multi-scale feature decomposition on the enhanced candidate region features through a feature extraction unit in the segmentation module to generate hierarchical semantic features; Inputting the hierarchical semantic features into a regional correlation calculation module, calculating the cross-region feature dependency of the hierarchical semantic features by the regional correlation calculation module, and generating a spatial enhancement factor; Performing region of interest detection on the hierarchical semantic features by a region positioning unit in the segmentation module to generate a region feature map; Building a boundary perception module based on the region feature map, extracting boundary information of the region of interest through the boundary perception module, and generating boundary constraint features; Performing initial semantic segmentation on the region of interest by a segmentation processing unit in the segmentation module to generate an initial segmentation boundary; A boundary mapping relationship is established based on the initial segmentation boundary, and the initial segmentation boundary is feature remapped and the boundary is reconstructed in combination with the boundary constraint feature and the spatial enhancement factor; the reconstructed boundary is feature remapped and the boundary is reconstructed again, and the segmentation result of the glioma is generated through multiple rounds of iterative boundary optimization.

7. A glioma segmentation system based on regions of interest combined with artificial intelligence, for implementing the method according to any one of claims 1 to 6, characterized in that: include: The first unit is configured to receive a brain magnetic resonance image acquired by a medical imaging device and preprocess the brain magnetic resonance image to obtain a preprocessed brain magnetic resonance image; The second unit is configured to extract features from the preprocessed brain magnetic resonance image using a hierarchical feature processing mechanism based on a multi-scale convolutional neural network to obtain image features; a third unit, configured to calculate an attention weight of each pixel in the image feature, generate a heat map according to the attention weight, and binarize the heat map based on a preset threshold to obtain a candidate region containing the region of interest; The fourth unit is configured to adaptively adjust the convolution kernel parameters according to the size of the candidate region, extract multi-scale features of the candidate region using spatial pyramid pooling structures of different scales, and fuse the multi-scale features through a feature fusion network to obtain enhanced candidate region features; A fifth unit is configured to input the enhanced candidate region features into a segmentation module, and perform semantic segmentation on the region of interest through the segmentation module to obtain a segmentation result of the glioma; The sixth unit is configured to mark the glioma region in the preprocessed brain magnetic resonance image based on the segmentation result to obtain a glioma segmentation image.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images