A brain tumor image classification and segmentation method and device based on a fusion multi-attention mechanism, a computer device, and a storage medium

Abstract

A brain tumor image classification and segmentation method fusing a multi-attention mechanism comprises: acquiring an original brain tumor image, and obtaining a to-be-processed image after preprocessing; inputting the to-be-processed image into a pre-trained double-task processing model to obtain a classification result and a segmentation result, the double-task processing model being improved based on a U-net network model, and the double-task processing model comprising an encoding network, a bottleneck network, a decoding network and a classification module.The present application provides a brain tumor image classification and segmentation method, device, computer equipment and storage medium fusing a multi-attention mechanism, which can simultaneously complete image classification and image segmentation, and effectively improve the efficiency and precision of image segmentation.

Description

Technical Field

[0001] This invention relates to the field of brain tumor image segmentation technology, specifically to a brain tumor image classification and segmentation method, apparatus, computer equipment, and storage medium that integrates multiple attention mechanisms. Background Technology

[0002] Brain tumors are among the most common diseases of the central nervous system. Accurate tumor classification and precise anatomical segmentation are crucial for clinical diagnosis, treatment planning, and prognostic assessment. In clinical practice, radiologists need to perform two key tasks simultaneously: first, classifying and diagnosing the tumor type to determine its benign or malignant nature and histological type; and second, performing pixel-level segmentation of the tumor region to precisely delineate tumor boundaries and determine tumor volume and spatial relationships. Traditionally, both tasks have been performed manually by physicians, resulting in a heavy workload, high subjectivity, and poor consistency.

[0003] Early brain tumor analysis relied primarily on traditional image processing techniques and machine learning methods. For classification, the main approach combined hand-designed features (such as texture, shape, and intensity features) with classifiers like Support Vector Machines (SVM) and Random Forests. For segmentation, algorithms such as thresholding, region growing, graph cut, and level set were used.

[0004] While these methods have achieved automated analysis to some extent, they have significant limitations: the design of handcrafted features relies on expert experience and is difficult to capture complex tumor features; classification and segmentation tasks are independent of each other and cannot fully utilize the correlation between the two tasks; and they have limited effectiveness for tumors with complex shapes and blurred boundaries.

[0005] With the rapid development of deep learning, convolutional neural network (CNN)-based methods have revolutionized the technological landscape of brain tumor analysis. In classification tasks, CNN models pre-trained on ImageNet have achieved excellent classification performance through transfer learning. In segmentation tasks, the emergence of fully convolutional networks (FCNs) and U-Net architectures has made end-to-end pixel-level prediction possible. However, problems such as class imbalance, blurred boundaries, scale diversity, and morphological complexity still exist, leading to relatively poor accuracy. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a brain tumor image classification and segmentation method, apparatus, computer equipment, and storage medium that integrates multiple attention mechanisms. This method can simultaneously perform image classification and image segmentation, and effectively improves the efficiency and accuracy of image segmentation.

[0007] To achieve the above objectives, the specific solution adopted by the present invention is as follows: a brain tumor image classification and segmentation method integrating multiple attention mechanisms, comprising: The original brain tumor image was acquired and preprocessed to obtain the image to be processed; The image to be processed is input into a pre-trained dual-task processing model to obtain classification and segmentation results. The dual-task processing model is an improvement on the U-net network model and includes an encoder network, a bottleneck network, a decoder network, and a classification module. The methods used by the dual-task processing module to process the image to be processed include: The image to be processed is encoded multiple times using an encoding network to obtain multiple encoded data. Then, multi-scale feature fusion processing, depth-separable convolution processing, and coordinated attention processing are performed on some of the encoded data to obtain backup data. The bottleneck network is used to encode the last encoded data to obtain transition data, and the classification module is used to generate classification results based on the last encoded data. The transition data is upsampled multiple times using a decoding network to obtain multiple upsampled data. Then, based on an improved context anchor attention mechanism, the upsampled data is fused with the backup data and decoded to obtain multiple decoded data. Finally, the segmentation result is generated based on the last decoded data.

[0008] As a further optimization of the brain tumor image classification and segmentation method that integrates multiple attention mechanisms, the decoding network includes N coding blocks of different levels. Each coding block includes a coding convolutional layer and a downsampling layer. The coding convolutional layer is used to encode the first input data to obtain coded data, and the downsampling layer is used to downsample the coded data to obtain downsampled data. The first input data is the image to be processed or the downsampled data generated by the coding block of the previous level.

[0009] As a further optimization of the brain tumor image classification and segmentation method that integrates multiple attention mechanisms, the first to N-1 coding blocks also include an enhancement module. The enhancement module is used to perform multi-scale feature fusion processing, depthwise separable convolution processing, and coordinated attention processing on the coded data to obtain backup data.

[0010] As a further optimization of the brain tumor image classification and segmentation method that integrates multiple attention mechanisms, the enhancement module includes a CI feature fusion unit and a Back enhancement unit. The CI feature fusion unit is used to perform multi-scale feature fusion processing on the encoded data, and the Back enhancement unit is used to perform depthwise separable convolution processing and coordinated attention processing on the encoded data.

[0011] As a further optimization of the brain tumor image classification and segmentation method that integrates multiple attention mechanisms, the decoding network includes N decoding blocks of different levels, and the decoding blocks correspond one-to-one with the encoding blocks. The decoding block includes an upsampling layer, a fusion block, and a decoding convolutional layer. The upsampling layer is used to upsample the second input data to obtain upsampled data. The fusion block is used to fuse the upsampled data and the backup data based on the improved context anchor attention mechanism to obtain fused data. The decoding convolutional layer is used to decode the fused data to obtain decoded data. The second input data is the transition data or the upsampled data generated by the next level of the decoding block.

[0012] As a further optimization of the aforementioned brain tumor image classification and segmentation method incorporating multiple attention mechanisms: the method for fusing upsampled data and backup data based on an improved context anchor attention mechanism to obtain decoded data includes: Calculate the channel attention weights and spatial attention weights of the third input data, which can be either upsampled data or backup data. The channel attention weights and spatial attention weights are weighted and fused, and then the weights are normalized to obtain the fused weights. The upsampled data and the backup data are fused based on the fusion weight to obtain fused data.

[0013] As a further optimization of the aforementioned brain tumor image classification and segmentation method incorporating multiple attention mechanisms, the method for calculating the channel attention weights of the third input data includes: Compressed features are obtained by dimensionality compression of the third input data; The compressed data is subjected to channel dimensionality reduction and channel dimensionality increase sequentially to obtain dimensionality-incremented features; Channel attention weights are obtained by activating and reshaping the dimensions of the upgraded features; Methods for calculating spatial attention weights for the third input data include: Enhanced features are obtained by channel adjustment and nonlinear activation of the third input data; The enhanced features are then subjected to horizontal and vertical splitting convolutions in sequence to obtain the neighborhood enhanced features. Spatial attention weights are obtained by convolution and BN normalization of the neighborhood enhancement features.

[0014] A brain tumor image classification and segmentation system incorporating multiple attention mechanisms, used to implement the aforementioned brain tumor image classification and segmentation method incorporating multiple attention mechanisms, the system comprising: The data acquisition module is used to acquire raw brain tumor images and preprocess them to obtain images to be processed. The data processing module is used to process the image to be processed using a pre-trained dual-task processing model to obtain classification and segmentation results. The dual-task processing model is an improvement on the U-net network model and includes an encoding network, a bottleneck network, a decoding network, and a classification module.

[0015] Computer equipment, including: Memory, used to store computer programs; A processor is used to read and execute the computer program to implement the above-described brain tumor image classification and segmentation method that incorporates multiple attention mechanisms.

[0016] A storage medium for storing a computer program that, when executed, implements the aforementioned brain tumor image classification and segmentation method incorporating a multi-attention mechanism.

[0017] Beneficial effects: This invention improves feature utilization by sharing encoder features to simultaneously complete segmentation and classification tasks; by introducing CI, Back, and CCSA modules through skip connections in the encoder and decoder, the CI module achieves multi-scale feature fusion, the Back module reduces computational complexity through depthwise separable convolution, and the learnable parameters in the CCSA module achieve adaptive fusion of channel attention and spatial attention; this invention has clear logic, is safe, effective, reliable, and easy to operate, effectively improving the efficiency and accuracy of image segmentation. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the brain tumor classification and segmentation research method that integrates multiple attention mechanisms provided by the present invention. Figure 2 This is a schematic diagram of the dual-task processing model in this invention; Figure 3 This is a structural diagram of the CCSA module. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] like Figures 1 to 3 As shown, a brain tumor image classification and segmentation method integrating multiple attention mechanisms includes: S1. Obtain the original brain tumor image and perform preprocessing to obtain the image to be processed.

[0021] S2. Input the image to be processed into the pre-trained dual-task processing model to obtain classification and segmentation results. The dual-task processing model is an improvement on the U-net network model. The dual-task processing model includes an encoding network, a bottleneck network, a decoding network, and a classification module. The dual-task processing module processes the image to be processed in the following ways: S21 to S23.

[0022] S21. The image to be processed is encoded multiple times using an encoding network to obtain multiple encoded data. Then, multi-scale feature fusion processing, depth-separable convolution processing, and coordinated attention processing are performed on some of the encoded data to obtain backup data.

[0023] More specifically, the decoding network comprises N different levels of coding blocks. Each coding block includes a coding convolutional layer and a downsampling layer. The coding convolutional layer encodes the first input data to obtain coded data, and the downsampling layer downsamples the coded data to obtain downsampled data. The first input data is the image to be processed or downsampled data generated by the coding block of the previous level. The first to N-1 coding blocks also include an enhancement module. This enhancement module performs multi-scale feature fusion, depthwise separable convolution, and coordinated attention processing on the coded data to obtain backup data. The enhancement module includes a CI feature fusion unit and a Back enhancement unit. The CI feature fusion unit performs multi-scale feature fusion on the coded data, and the Back enhancement unit performs depthwise separable convolution and coordinated attention processing on the coded data.

[0024] In one embodiment of the present invention, the decoding network includes four coding blocks with different levels, which are respectively designated as the first coding block, the second coding block, the third coding block, and the fourth coding block. The first coding block, the second coding block, and the third coding block include enhancement modules. In each coding block, the coding convolutional layer and the downsampling layer are also numbered first to fourth, and the enhancement modules are numbered first to third. Based on this, the specific principles of the four coding blocks are as follows.

[0025] The first encoding block encodes the image b to be processed through a first encoding convolutional layer to obtain first encoded data c1, which is then passed to a first downsampling layer. The first downsampling layer downsamples the first encoded data c1 to obtain first downsampled data d1, which is then passed to the second encoding block. The first enhancement module incorporates a CI feature fusion module and a Back enhancement unit. The first encoded data c1 is input into the CI module, which fuses multi-scale features through convolution and pooling operations. The result is then input into the Back module. The Back unit integrates depthwise separable convolution and a CoordAttention mechanism to enhance channel-spatial correlation, resulting in backup data e1.

[0026] The second encoding block encodes the first downsampled data d1 through a second encoding convolutional layer to obtain the second encoded data c2, which is then passed to the second downsampling layer. The second downsampling layer downsamples the second encoded data c2 to obtain the second downsampled data d2, which is then passed to the third encoding block. The second enhancement module still incorporates a CI feature fusion module and a Back enhancement unit. The second encoded data c2 is input into the CI module, which fuses multi-scale features through convolution and pooling operations. The result is then input into the Back module. The Back unit integrates depthwise separable convolution and a CoordAttention mechanism to enhance channel-spatial correlation, resulting in backup data e2.

[0027] The third encoding block encodes the second downsampled data d2 through the third encoding convolutional layer to obtain the third encoded data c3, which is then passed to the third downsampling layer. The third downsampling layer downsamples the third encoded data c3 to obtain the third downsampled data d3, which is then passed to the fourth encoding block. The third enhancement module still incorporates a CI feature fusion module and a Back enhancement unit. The third encoded data c3 is input into the CI module, which fuses multi-scale features through convolution and pooling operations. The result is then input into the Back module. The Back unit integrates depthwise separable convolution and a CoordAttention mechanism to enhance channel-spatial correlation, resulting in backup data e3.

[0028] The fourth encoding block is used to encode the third downsampled data d3 through the fourth encoding convolutional layer to obtain the fourth encoded data c4, which is then passed to the fourth downsampling layer. The fourth downsampling layer downsamples the fourth encoded data c4 to obtain the fourth downsampled data d4, which is then passed to the bottleneck module. The classification module then performs pooling and flattening on the fourth downsampled data d4, and outputs the classification probability through a fully connected layer to obtain the classification result.

[0029] S22. The bottleneck network is used to encode the last encoded data to obtain transition data c5, and the classification module is used to generate classification results based on the last encoded data.

[0030] S23. The transition data is upsampled multiple times using a decoding network to obtain multiple upsampled data. Based on the improved context anchor attention mechanism, the upsampled data is fused with the backup data and then decoded to obtain multiple decoded data. Finally, the segmentation result is generated based on the last decoded data.

[0031] More specifically, the decoding network comprises N different levels of decoding blocks, with each decoding block corresponding one-to-one with an encoding block. Each decoding block includes an upsampling layer, a fusion block, and a decoding convolutional layer. The upsampling layer upsamples the second input data to obtain upsampled data. The fusion block fuses the upsampled data and backup data using an improved context anchor attention mechanism to obtain fused data. The decoding convolutional layer decodes the fused data to obtain decoded data. The second input data is either transitional data or upsampled data generated by the next level's decoding block. The method by which the fusion block fuses the upsampled data and backup data using the improved context anchor attention mechanism to obtain decoded data includes: calculating the channel attention weights and spatial attention weights of the third input data (either upsampled data or backup data); weighting and fusing the channel attention weights and spatial attention weights, and then normalizing the weights to obtain fusion weights; and fusing the upsampled data and backup data based on these fusion weights to obtain fused data.

[0032] Furthermore, the method for calculating the channel attention weights of the third input data includes: compressing the third input data to obtain compressed features; sequentially performing channel dimensionality reduction and channel dimensionality increase on the compressed data to obtain dimensionality-increased features; and activating and reshaping the dimensionality-increased features to obtain the channel attention weights.

[0033] The method for calculating the spatial attention weights of the third input data includes: performing channel adjustment and nonlinear activation on the third input data to obtain enhanced features; sequentially performing horizontal and vertical splitting convolutions on the enhanced features to obtain neighborhood enhanced features; and performing convolution and BN normalization on the neighborhood enhanced features to obtain spatial attention weights.

[0034] As an extension of the above embodiment, in this embodiment, four decoding blocks are also provided, respectively labeled as the first decoding block, the second decoding block, the third decoding block, and the fourth decoding block. Correspondingly, the upsampling layer, the fusion block, and the decoding convolutional layer are also labeled as the first to the fourth, respectively. The specific working principle of the decoding network is as follows.

[0035] The fourth decoding block is used to upsample the transition data c5 through the fourth upsampling layer to obtain the fourth upsampled data f4. The fourth fusion block is used to fuse the fourth encoded data c4 and the fourth upsampled data f4. The fusion operation has a built-in CCSA context anchor attention module, which performs channel and spatial weighting on the skip connection features to focus on key tumor regions. Then, it is decoded through the fourth decoding convolutional layer to obtain the fourth decoded data g4, which is then passed to the third decoding block.

[0036] The third decoding block is used to upsample the fourth decoded data g4 through the third upsampling layer to obtain the third upsampled data f3. The third fusion block is used to fuse the data e3 that needs to be fused with the third decoded data and the third upsampled data f3. The fusion operation has a built-in CCSA context anchor attention module, which performs channel and spatial weighting on the skip connection features, focuses on key tumor regions, and then decodes them through the third decoding convolutional layer to obtain the third decoded data g3, which is then passed to the second decoding block.

[0037] The second decoding block is used to upsample the third decoded data g3 through the second upsampling layer to obtain the second upsampled data f2. The second fusion block is used to fuse the data e2 that needs to be fused with the second decoded data and the second upsampled data f2. The fusion operation has a built-in CCSA context anchor attention module, which performs channel and spatial weighting on the skip connection features, focuses on the key tumor region, and then decodes it through the second decoding convolutional layer to obtain the second decoded data g2, which is then passed to the first decoding block.

[0038] The first decoding block is used to upsample the second decoded data g2 through the first upsampling layer to obtain the first upsampled data f1. The first fusion block is used to perform a fusion operation on the data e1 that needs to be fused with the first decoded data and the first upsampled data f1. The fusion operation has a built-in CCSA context anchor attention module, which performs channel and spatial weighting on the skip connection features, focuses on the key tumor region, and then decodes it through the first decoding convolutional layer to finally output the image segmentation result.

[0039] Based on the above-mentioned encoding and decoding network structure design, this invention improves feature utilization by sharing encoder features to simultaneously complete segmentation and classification tasks. By introducing CI, Back, and CCSA modules through skip connections in the encoder and decoder, the CI module achieves multi-scale feature fusion, the Back module reduces computational complexity through depthwise separable convolution, and the learnable parameters in the CCSA module enable adaptive fusion of channel attention and spatial attention. This invention is logically clear, safe, effective, reliable, and easy to operate, effectively improving the efficiency and accuracy of image segmentation.

[0040] In one embodiment of the present invention, the first, second, and third coding convolutional layers have identical structures, each comprising a first 3×3 convolutional layer and a second 3×3 convolutional layer. The fourth coding convolutional layer comprises a first 3×3 convolutional layer and a second 3×3 convolutional layer connected in sequence. The first, second, and third enhancement modules each include a feature fusion module (CI) and a residual enhancement module (Back). The feature fusion module (CI) comprises a third 3×3 convolutional layer, a 2×2 max pooling operation, and a first 1×1 convolutional layer. The residual enhancement module (Back) comprises a second 1×1 convolutional layer, a 3×3 depthwise separable convolutional layer, a CoordAttention module, and a third 1×1 convolutional layer. The first, second, third, and fourth downsampling layers have identical structures, each comprising a 2×2 max pooling operation. The outputs of the first and second 3×3 convolutional layers have built-in data normalization and ReLU activation functions. The outputs of the second and third 1×1 convolutional layers have built-in data normalization and ReLU activation functions.

[0041] The bottleneck mode is configured with a fourth 3×3 convolutional layer and a fifth 3×3 convolutional layer. The outputs of the fourth and fifth 3×3 convolutional layers have built-in data normalization functions, ReLU activation functions, and Dropout operations.

[0042] The first, second, third, and fourth decoding convolutional layers have the same structure, each including a sixth 3×3 convolutional layer and a seventh 3×3 convolutional layer. The first, second, third, and fourth upsampling layers also have the same structure, each including a first 4×4 transposed convolutional layer. The outputs of the sixth and seventh 3×3 convolutional layers have built-in data normalization and ReLU activation functions.

[0043] The first, second, third, and fourth fusion blocks have the same structure and all have our designed CCSA module built in. The CCSA module is an improvement on the CAA module. In this module, the channel attention and spatial attention branches are weighted and fused after scaling, and finally output through residual connection.

[0044] Furthermore, the specific processing procedure for the fusion block is as follows.

[0045] Step 1: For the third input data x, perform adaptive global average pooling on its dimension (b,c,h,w) to compress the dimension (h,w) of each channel into 1 value, obtain the global channel feature y_pool, and change the dimension of the third input data x from (b,c,h,w) to (b,c,1,1).

[0046] .

[0047] Step 2: After flattening y_pool to (b,c) dimensions, channel dimensionality reduction and dimensionality increase are achieved through two fully connected layers, introducing nonlinearity and reducing the number of parameters. The first fully connected layer reduces the number of channels from c to c / / r with the ReLU activation function, and the second fully connected layer increases the number of channels back to c with no activation function, resulting in the dimensionality-upgraded feature y_mlp2.

[0048] Step 3: Apply the Sigmoid activation function to the upgraded feature y_mlp2 to compress the weights to the [0,1] interval, then reshape it to (b,c,1,1) dimensions to obtain the channel attention weights y_channel: Channel attention weights represent the importance of each channel; the larger the value, the more attention is paid to the features of the corresponding channel.

[0049] Step 4: Apply channel adjustment and non-linear activation to the third input data x using a 1×1 convolution spatial_conv1 to reduce the computational cost of subsequent convolutions and enhance feature representation. During this process, the dimension of the third input data x remains unchanged.

[0050] Step 5: Use grouped convolution h_conv, sliding only in the horizontal direction, to capture the horizontal neighborhood information at each position. The convolution kernel of grouped convolution h_conv is (1, h_kernel_size). Because grouped convolution is used, each channel is computed independently, thus avoiding interference between channels. ; Where k_h = h_kernel_size, and padding is set to (0, k_h / / 2) to ensure that the horizontal dimension remains unchanged.

[0051] Step 6: Use grouped convolution v_conv, sliding only in the vertical direction, to capture the vertical neighborhood information at each position, and the convolution kernel of grouped convolution v_conv is (v_kernel_size, 1): ; Where k_v = v_kernel_size, and padding is set to (k_v / / 2,0) to ensure that the vertical dimension remains unchanged.

[0052] Step 7: Perform a 1×1 convolution and BN normalization on y_v using spatial_conv2, and fuse horizontal and vertical contextual features to obtain the spatial attention weight y_spatial. The dimension remains unchanged during this process. .

[0053] Step 8: Weight the two attention types using learnable parameters scale and bias, respectively, where the initial value of scale is 0 and the initial value of bias is 1, and then sum them: ; Where y_channel is of dimension (b,c,1,1), and after being broadcast to (b,c,h,w), it is added to y_spatial.

[0054] Scale and bias are learnable parameters that adaptively adjust the weight ratio of channel attention and spatial attention during model training.

[0055] Step 9: Apply Sigmoid activation to the fused feature y_fusion to compress the attention factor to the [0,1] interval, obtaining the final attention weight attn_factor: ; This weight includes both channel importance and spatial location importance; the larger the value, the more critical the corresponding channel-spatial location feature is.

[0056] Step 10: Perform channel adjustment and BN normalization on the input x using a 1×1 convolution conv_residual to ensure that the residual branch has the same dimension and distribution as the main branch. .

[0057] Step 11: Add the main branch features and the residual branch features, where the main branch features are the element-wise multiplication results of the third input data x and the attention weight attn_factor, to obtain the final output of the module: ; The dimensions of the output remain (b,c,h,w), consistent with the third input data x.

[0058] The present invention further provides a brain tumor image classification and segmentation system that integrates multiple attention mechanisms, for implementing the above-mentioned brain tumor image classification and segmentation method that integrates multiple attention mechanisms. The system includes a data acquisition module and a data processing module.

[0059] The data acquisition module is used to acquire raw brain tumor images and preprocess them to obtain images to be processed.

[0060] The data processing module is used to process the image to be processed using a pre-trained dual-task processing model to obtain classification and segmentation results. The dual-task processing model is an improvement on the U-net network model and includes an encoding network, a bottleneck network, a decoding network, and a classification module.

[0061] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or modules may be electrical, mechanical, or other forms.

[0062] The present invention also provides a computer device, including a memory and a processor.

[0063] Memory is used to store computer programs.

[0064] The processor is used to read and execute computer programs to implement the aforementioned brain tumor image classification and segmentation method that incorporates multiple attention mechanisms.

[0065] Finally, the present invention provides a storage medium for storing a computer program that, when executed, implements the above-described brain tumor image classification and segmentation method incorporating a multi-attention mechanism.

[0066] The memory, as a carrier of resources, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored on it can include the operating system, computer programs, etc., and the storage method can be temporary or permanent storage. The operating system is used to manage and control the various hardware devices and computer programs on the electronic device, and can be Windows Server, Netware, Unix, Linux, etc. In addition to including computer programs capable of performing the adaptive emotion regulation method based on personalized reconfigurable music disclosed in any of the foregoing embodiments, the computer programs may further include computer programs capable of performing other specific tasks. The processor can be a general-purpose processor product based on architectures such as x86, IA64, RISC, MIPS, and ARM.

[0067] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A brain tumor image classification and segmentation method integrating multiple attention mechanisms, characterized in that, include: The original brain tumor image was acquired and preprocessed to obtain the image to be processed; The image to be processed is input into a pre-trained dual-task processing model to obtain classification and segmentation results. The dual-task processing model is an improvement on the U-net network model and includes an encoder network, a bottleneck network, a decoder network, and a classification module. The methods used by the dual-task processing module to process the image to be processed include: The image to be processed is encoded multiple times using an encoding network to obtain multiple encoded data. Then, multi-scale feature fusion processing, depth-separable convolution processing, and coordinated attention processing are performed on some of the encoded data to obtain backup data. The bottleneck network is used to encode the last encoded data to obtain transition data, and the classification module is used to generate classification results based on the last encoded data. The transition data is upsampled multiple times using a decoding network to obtain multiple upsampled data. Then, based on an improved context anchor attention mechanism, the upsampled data is fused with the backup data and decoded to obtain multiple decoded data. Finally, the segmentation result is generated based on the last decoded data.

2. The brain tumor image classification and segmentation method incorporating multiple attention mechanisms as described in claim 1, characterized in that, The coding network includes N coding blocks with different levels. Each coding block includes a coding convolutional layer and a downsampling layer. The coding convolutional layer is used to encode the first input data to obtain coded data, and the downsampling layer is used to downsample the coded data to obtain downsampled data. The first input data is the image to be processed or the downsampled data generated by the coding block of the previous level.

3. The brain tumor image classification and segmentation method incorporating multiple attention mechanisms as described in claim 2, characterized in that, The first to N-1 coding blocks also include an enhancement module, which is used to perform multi-scale feature fusion processing, depthwise separable convolution processing, and coordinated attention processing on the coded data to obtain backup data.

4. The brain tumor image classification and segmentation method incorporating multiple attention mechanisms as described in claim 3, characterized in that, The enhancement module includes a CI feature fusion unit and a Back enhancement unit. The CI feature fusion unit is used to perform multi-scale feature fusion processing on the encoded data, and the Back enhancement unit is used to perform depthwise separable convolution processing and coordinated attention processing on the encoded data.

5. The brain tumor image classification and segmentation method incorporating multiple attention mechanisms as described in claim 2, characterized in that, The decoding network includes N decoding blocks at different levels, and each decoding block corresponds one-to-one with the encoding block. Each decoding block includes an upsampling layer, a fusion block, and a decoding convolutional layer. The upsampling layer is used to upsample the second input data to obtain upsampled data. The fusion block is used to fuse the upsampled data with the backup data based on an improved context anchor attention mechanism to obtain fused data. The decoding convolutional layer is used to decode the fused data to obtain decoded data. The second input data is the transition data or the upsampled data generated by the next level decoding block.

6. The brain tumor image classification and segmentation method incorporating multiple attention mechanisms as described in claim 5, characterized in that, The method for fusing upsampled data and backup data to obtain decoded data based on the improved context anchor attention mechanism includes: Calculate the channel attention weights and spatial attention weights of the third input data, which can be either upsampled data or backup data. The channel attention weights and spatial attention weights are weighted and fused, and then the weights are normalized to obtain the fused weights. The upsampled data and the backup data are fused based on the fusion weight to obtain fused data.

7. The brain tumor image classification and segmentation method incorporating multiple attention mechanisms as described in claim 6, characterized in that, Methods for calculating the channel attention weights of the third input data include: Compressed features are obtained by dimensionality compression of the third input data; The compressed data is subjected to channel dimensionality reduction and channel dimensionality increase sequentially to obtain dimensionality-incremented features; Channel attention weights are obtained by activating and reshaping the dimensions of the upgraded features; Methods for calculating spatial attention weights for the third input data include: Enhanced features are obtained by channel adjustment and nonlinear activation of the third input data; The enhanced features are then subjected to horizontal and vertical splitting convolutions in sequence to obtain the neighborhood enhanced features. Spatial attention weights are obtained by convolution and BN normalization of the neighborhood enhancement features.

8. A brain tumor image classification and segmentation system integrating multiple attention mechanisms, characterized in that, The system is used to implement a brain tumor image classification and segmentation method incorporating multiple attention mechanisms as described in any one of claims 1-7, the system comprising: The data acquisition module is used to acquire raw brain tumor images and preprocess them to obtain images to be processed. The data processing module is used to process the image to be processed using a pre-trained dual-task processing model to obtain classification and segmentation results. The dual-task processing model is an improvement on the U-net network model and includes an encoding network, a bottleneck network, a decoding network, and a classification module.

9. A computer device, characterized in that, include: Memory, used to store computer programs; A processor for reading and executing the computer program to implement a brain tumor image classification and segmentation method incorporating multiple attention mechanisms as described in any one of claims 1-7.

10. A storage medium, characterized in that, Used to store a computer program, which, when executed, implements a brain tumor image classification and segmentation method incorporating a multi-attention mechanism as described in any one of claims 1-7.

Images