A 3D segmentation method for brain tumor MRI images based on RAPNet network

Abstract

The present invention belongs to the field of image processing, and specifically relates to a three-dimensional segmentation method of brain tumor MRI images based on a RAPNet network, comprising constructing a RAPNet network and training the same; inputting a brain MRI image into a trained RAPNet network for image recognition and segmentation to obtain a segmented brain tumor MRI image and its substructure region, wherein the RAPNet network comprises a backbone network, a feature pyramid and auxiliary prediction, and the backbone network is composed of a dilated convolution and a plurality of ISE-R2CU units, and is used to extract shallow features and deep features of an input image; the feature pyramid composed of a 3D dilated convolution and a cross-model attention mechanism adopted by the present invention is combined with the backbone to learn effective features of the entire tumor and its substructures, thereby having the advantage of fitting various tissue boundaries within the tumor.

Description

Technical Field

[0001] The invention belongs to the field of image processing, and in particular relates to a three-dimensional segmentation method of brain tumor MRI images based on a RAPNet network. Background Art

[0002] Brain tumors are abnormal cells that gather in the brain and are an extremely dangerous disease. Tumor cells divide rapidly and proliferate indefinitely, gradually invading the human central nervous system and eventually leading to death. Medical imaging provides technical support for the early diagnosis of brain tumors. Among many imaging methods, magnetic resonance imaging (MRI) technology provides excellent contrast for multi-directional imaging of brain soft tissues, and is non-invasive and radiation-free. Therefore, MRI has become a common technology for diagnosing brain tumors. During the treatment process, in order to kill tumor cells while protecting normal cells to the maximum extent, it is particularly important to segment the tumor before treatment. However, it is a very time-consuming task to annotate and segment a large number of multimodal MRI images using manual methods, and this method is greatly affected by subjective factors, and its segmentation efficiency is low.

[0003] With the rapid development of artificial intelligence technology, especially the breakthrough of deep learning technology and the rise of medical big data, more and more researchers are committed to applying artificial intelligence technology to the automatic segmentation of brain tumors in MRI images. Among them, the method based on deep convolutional neural network (DCNN) constructs complex neural network models through end-to-end learning mechanism in a data-driven manner. These DCNN networks can be divided into two categories, namely multi-stream structure and encoder-decoder structure. Multi-stream architecture uses multiple inputs of different scales to build a network in a shared parameter manner. Such networks consider the spatial relationship between pixels / voxels in MRI data. Its architecture is usually integrated into another cascade structure, and small pixel / voxel blocks and large pixel / voxel blocks are sampled separately in a sliding window manner. Small block sampling is responsible for learning detailed local features of MRI scans, while large block sampling uses contextual information to improve the final segmentation performance. The multi-data stream architecture first builds pyramids for input images of different sizes and then feeds them to learn multi-scale features. This method is also called multi-image input with multi-feature output. The multi-data stream method is relatively simple to implement, but the input images of the pyramid are repeatedly input to the same network, which will lead to computational redundancy.

[0004] Different from the multi-stream architecture, the encoder-decoder structure mainly uses the idea of ​​pixel-by-pixel semantic segmentation learned end-to-end. Chen et al. proposed a deep convolutional symmetric neural network to achieve automatic segmentation of brain tumors by adding symmetric masks in the convolutional layer to effectively utilize prior knowledge. Jiang et al. proposed a new image enhancement algorithm, namely a three-dimensional fully convolutional network for multi-category semantic segmentation, by exploring multimodal image patches for effective data enhancement. Li et al. proposed an upward jump connection structure based on the improved fully convolutional network to improve network connectivity and enhance the information flow between the encoder and decoder parts. At the same time, the initial module was adopted in each basic module of the U-shaped structure to enable the network to learn richer feature representations. Hu et al. proposed a segmentation method based on multi-level convolutional neural network and fully connected conditional random field. This method designs a multi-level network architecture by combining the intermediate results of multiple connected components to consider the local dependency of labels, and uses conditional random fields to consider contextual information and eliminate some false outputs for fine segmentation. Rehman et al. introduced two modules, residual extended jump connection and wide context, into the existing U-Net structure to effectively expand the receptive field and enhance the background feature extraction of MRI. In these networks, the encoder is responsible for converting the input into a low-resolution representation, while the decoder maps the low-resolution representation to pixel predictions. Due to the relocation of fully connected layers, this type of architecture allows for pixel-level dense predictions for arbitrary input sizes.

[0005] Although the above networks have greatly promoted the development of medical image segmentation, since brain tumors appear in different sizes and shapes in MRI images of different patients, single-scale networks still have the following limitations in brain tumor segmentation: overly complex network models are suitable for image processing with large data volumes and require powerful computing resources. When the network parameters are large, the network model will have gradient diffusion problems. Most of the existing DCNNs for brain tumor segmentation have limited multi-scale processing capabilities and often ignore auxiliary information such as the surrounding environment of the lesion tissue that can be used to distinguish different tissues. Therefore, there is an urgent need for a brain tumor MRI image segmentation model for different scales for image segmentation processing. Summary of the invention

[0006] In order to solve the problem that brain tumors present different sizes and shapes in MRI images of different patients, resulting in low segmentation accuracy based on single-scale DCNN, this paper proposes a three-dimensional segmentation method for brain tumor MRI images based on the RAPNet network.

[0007] Build the RAPNet network and train it;

[0008] Input the brain MRI image into the trained RAPNet network for image recognition and segmentation to obtain the segmented brain tumor MRI image and its substructure area;

[0009] The RAPNet network includes a backbone network, a feature pyramid, and auxiliary predictions, among which:

[0010] The backbone network consists of dilated convolutions and multiple improved squeeze-and-excitation recurrent residual convolutional units (ISE-R2CU), which are used to extract shallow and deep features of the input image.

[0011] The feature pyramid consists of a dilated convolution and a cross-model attention mechanism, which is used to extract feature maps of different scales to improve the overall model's ability to recognize tumors of different sizes.

[0012] The auxiliary prediction part is composed of standard convolutions, which is used to comprehensively observe all targets that benefit from the feature fusion module.

[0013] Furthermore, ISE-R2CU consists of three ISE-Res modules, three 1×1×1 standard convolutional layers, one 3×3×3 standard convolutional layer and one residual connection layer, where:

[0014] The 3×3×3 standard convolutional layer obtains a high-resolution image based on the input image;

[0015] Input the high-resolution image into a 1×1×1 standard convolutional layer to compress the number of channels without changing the image size;

[0016] The image obtained by 1×1×1 standard convolution is added to the high-resolution image and then input into the ISE-Res module;

[0017] The features output by the first ISE-Res module are then fed into a 1×1×1 standard convolutional layer to compress the number of channels without changing the image size.

[0018] The image obtained by 1×1×1 standard convolution is added to the high-resolution image and then input into the second ISE-Res module;

[0019] The features output by the second ISE-Res module are then fed into a 1×1×1 standard convolutional layer to compress the number of channels without changing the image size.

[0020] The image obtained by 1×1×1 standard convolution is added to the high-resolution image and then input into the third ISE-Res module;

[0021] The features output by the second ISE-Res module are added to the input image as the output of the ISE-R2CU unit.

[0022] Furthermore, ISE-R2CU extracts shallow features and deep features of the input image, specifically including the following steps:

[0023]

[0024]

[0025]

[0026] in, represents the output of the ISE-R2CU unit network at time step t, and are the inputs of the standard convolution and recursive convolution units of the lth layer in the network, respectively. and The values ​​of are the weights of the standard convolution and recursive convolution units in the k-th layer feature map, b k is the deviation, is the output of the recursive convolutional unit in layer l, x l is the input sample of the lth layer in the network, w l is the pixel on the feature map of layer l.

[0027] Furthermore, the cross-model attention mechanism includes three ISE-Res; each ISE-Res is integrated in each feature extraction branch of the feature pyramid, and an element-wise addition operation is performed at the output position of each attention mechanism module; the activation features of each branch are connected from the previous branch to the feature map of the next branch through a splicing operation.

[0028] Furthermore, the ISE-Res module consists of a global mean pooling, a global maximum pooling, three ReLU layers, a Sigmoid layer and a residual connection layer, where:

[0029] The input data uses global mean pooling and global maximum pooling to sample the global information of each channel in parallel, and the features after parallel sampling are fed to two ReLU layers for activation;

[0030] The outputs of the two ReLU layers are added and input into the Sigmoid layer;

[0031] The residual connection layer multiplies the Sigmoid output by the input data and then linearly superimposes it with the input data as a residual connection;

[0032] The superimposed data is input into a ReLU layer for activation;

[0033] The residual connection layer

[0034] Furthermore, the auxiliary prediction structure includes three 1×1×1 standard convolutional layers and a segmentation prediction layer; each input feature map branch uses a standard convolutional and segmentation prediction layer to obtain the prediction result of the branch.

[0035] Furthermore, the training process of the RAPNet network includes:

[0036] S1: Input the image data in the training set into the RAPNet network for processing;

[0037] S2: The 3D recursive residual convolution unit of the backbone network accumulates features of different depths through multiple operations on the input data;

[0038] S3: The dilated convolutions with different dilation rates in the feature pyramid learn the multi-scale features of the input data, and through the feature recalibration mechanism, learn to use global information to selectively emphasize the information-rich features;

[0039] S4: The auxiliary prediction structure compares the multi-layer prediction results with different scale features with the label image pixel by pixel to obtain the prediction result error of each branch;

[0040] S5: Calculate the loss function of each branch based on the error result, and input the error into the model through the back propagation algorithm, update the convolution kernel weights and parameters in each module when the network extracts features, and complete the model training when the loss function is minimized.

[0041] Furthermore, the loss function of the RAPNet network is:

[0042] L total (G,Y,I)=L aided (G,Y,I)+L fuse (G,Y,I)

[0043]

[0044] α m ×0.1×(1-t / N)→α m

[0045]

[0046] Among them, I is the 3D MRI image input sample, G is the corresponding true segmentation result, Y is the actual segmentation result generated by using RAPNet, and L total (G,Y,I) is the overall loss of the network, L aided (·,·) represents the loss of auxiliary structure for hierarchical prediction, L fuse(·,·) represents the loss of the fusion layer, M represents the total number of layers of the auxiliary structure, and α m The parameter is used to adjust the ratio of the overall loss and the loss of the mth layer in the auxiliary structure. t represents the number of rounds, and N represents the total number of rounds. represents the loss function that measures the similarity between the prediction result of the mth layer in the auxiliary structure and the true segmentation result, σ(·) represents the composite nonlinear mapping of BN and ReLU after the standard convolution function, β m represents the learnable fusion weights, represents the output of the mth layer in the auxiliary structure, Indicates G and The loss value between .

[0047] The RAPNet in the present invention uses an improved recursive residual convolution unit ISE-R2CU to form the network backbone to alleviate the gradient diffusion problem caused by the large number of network parameters. Considering the inherent multi-scale hierarchical characteristics of DCNN, the present invention designs a feature pyramid composed of 3D hole convolution and cross-model attention mechanism combined with the backbone to learn the effective features of the entire tumor and its substructures, thereby having the advantage of fitting various tissue boundaries within the tumor; the present invention combines the designed network with the terminal auxiliary prediction structure whose main idea is to fuse the hierarchical prediction results, so that all targets that benefit from the feature fusion module can be comprehensively observed. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of the RAPNet network structure of the present invention;

[0049] Figure 2 This is a structural diagram of an improved compression-excitation residual module of the present invention;

[0050] Figure 3 The improved recursive residual convolution unit structure diagram of the present invention;

[0051] Figure 4 This is a structural diagram of the cross-model attention mechanism of the present invention;

[0052] Figure 5 Schematic diagram of the overall loss of RAPNet of the present invention;

[0053] Figure 6 It is a detection process diagram of the present invention;

[0054] Figure 7 This is the visual segmentation result diagram of the RAPNet training model used in the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] The present invention provides a method for three-dimensional segmentation of brain tumor MRI images based on a RAPNet network, comprising the following steps:

[0057] Build the RAPNet network and train it;

[0058] Input the brain MRI image into the trained RAPNet network for image recognition and segmentation to obtain the segmented brain tumor MRI image and its substructure area;

[0059] The RAPNet network includes a backbone network, a feature pyramid, and auxiliary predictions, among which:

[0060] The backbone network consists of dilated convolutions and multiple ISE-R2CU units, which are used to extract shallow and deep features of the input image.

[0061] The feature pyramid consists of a dilated convolution and a cross-model attention mechanism, which is used to extract feature maps of different scales to improve the overall model's ability to recognize tumors of different sizes.

[0062] The auxiliary prediction part is composed of standard convolutions, which is used to comprehensively observe all targets that benefit from the feature fusion module.

[0063] In this embodiment, if Figure 6 As shown in the figure, the brain MRI image is input into the trained RAPNet network for image recognition and segmentation to obtain the segmented brain tumor MRI image and its substructure area; the improved fully convolutional neural network RAPNet includes a backbone network, a feature pyramid, and an auxiliary prediction part.

[0064] The backbone network composed of improved recursive residual convolution units used in this embodiment is connected to the feature pyramid of the backbone network and the terminal auxiliary prediction structure whose main idea is to fuse the hierarchical prediction results; the backbone network is composed of dilated convolution and multiple recursive residual convolution units ISE-R2CU fused with improved compressed excitation residual modules, which are used to extract shallow and deep features of the input image and alleviate network degradation and gradient diffusion caused by the overly complex network model; the feature pyramid is composed of dilated convolution and cross-model attention mechanism, which is used to extract feature maps with different scales to improve the overall model's recognition ability for tumors of different sizes; the auxiliary prediction part is composed of standard convolution, which is used to comprehensively observe all targets that benefit from the feature fusion module.

[0065] The structure of the improved compression excitation residual module ISE-Res is as follows: Figure 2 As shown in the figure, the module consists of a global mean pooling, a global maximum pooling, three ReLUs, a Sigmoid and a residual connection layer. It improves the representation ability of the network by explicitly modeling the interdependence and nonlinear interactions between the channels of its convolutional features. First, the global information of each channel is sampled, including global mean pooling and maximum pooling. Parallel sampling can extract richer effective features. Mean pooling can reduce the increase in the variance of the estimated value caused by the limited neighborhood size, thereby retaining more image background information. Maximum pooling can reduce the deviation of the estimated mean caused by the convolution layer parameter error and retain more image texture information. The features after parallel sampling are fed to the activation function ReLU layer for fusion, and then sent to Sigmoid. The result of multiplying the Sigmoid output with the ISE-Res input is fused with the ISE-Res input to generate a scaled input, which is then sent to the ReLU layer.

[0066] In this embodiment, the recursive convolution residual unit ISE-R2CU fused with ISE-Res consists of three ISE-Res modules, three 1×1×1 standard convolution layers, one 3×3×3 standard convolution layer and a residual connection layer, and the compressed excitation residual module is followed by the recursive structure. The recursive convolution unit improves the spatial correlation of feature learning through multiple operations, accumulates features of different depths, ensures the accumulation of low-level features on the same level of codec structure, and provides better feature representation.

[0067] Consider the input sample x of the lth layer in the network l and the pixel at the input sample (i, j) on the k-th feature map in the recursive convolution unit. Assume that the network The output of at time step t can be expressed as:

[0068]

[0069] in, and are the inputs of the standard convolution and recursive convolution units of the lth layer in the network, respectively. and The values ​​of are the weights of the standard convolution and recursive convolution units in the k-th layer feature map, b k is the bias. The output of the recursive convolutional unit is fed into the standard ReLU activation function f and is expressed as:

[0070]

[0071] in, is the output of the RCU in layer l. Figure 3 The final output of the recurrent residual unit shown is calculated as follows:

[0072]

[0073] Among them, x l Represented as the input of the recurrent residual unit, x l+1 The samples are used as input for subsequent subsampling or dilated convolutional layers.

[0074] If the training dataset contains objects of various scales, the deep convolutional neural network (DCNN) can handle multi-scale representations, which enables DCNN to segment lesions of different sizes. Since the feature maps generated by DCNN have multi-scale characteristics, a multi-input branch structure constructed by 3D dilated convolution is added to the end of the backbone network for multi-scale feature learning. The dilated convolution when the dilation rate is 1 is the standard convolution. If the dilation rate is set to 1, its step size is greater than 1 and the padding is set to 0, this convolution operation is equivalent to a downsampling operation, and the size of the output signal will only account for a small fraction of the input. When its padding is close to the dilation rate and the step size is set smaller, DCNN can learn denser features. By adjusting the dilation rate and step size of the dilated convolution, DCNN can extract feature maps of different scales. When the network input size is larger than the DCNN receptive field, the final softmax layer will produce multiple predicted feature maps at the same time. As long as the receptive field of the DCNN can cover the entire input without padding, all predictions are equally reliable. The algorithm avoids repeated convolutions on the same voxel in overlapping blocks, which greatly reduces the computational cost and memory load. The feature maps output from the backbone network will be propagated separately to each branch consisting of 3D dilated convolutions with different expansion rates. In order to deal with the "internal covariate drift" phenomenon that hinders model performance during training, the present invention uses batch normalization (BN) technology for all convolutional layers in RAPNet.

[0075] In order to strengthen the connection between feature maps of different densities, this paper designs a cross-model attention mechanism based on ISE-Res and integrates it into multi-branch feature extraction. The detailed structure of each attention mechanism module on each branch is as follows: Figure 4 As shown. Through the feature recalibration mechanism, this network can learn to use global information to selectively emphasize informative features and suppress less useful features. To improve the local spatial consistency of the predicted feature map, an element-wise addition operation is performed at the output position of each attention mechanism module to activate effective features of the tumor containing position information. The activated features of each branch are connected from the previous branch to the feature map of the next branch through a concatenation operation.

[0076] An auxiliary prediction structure is added at the end of the network to obtain multi-layer prediction results with different scale features, and then the non-maximum suppression NMS idea is used to obtain the final segmentation result. Each input feature map branch in the auxiliary prediction structure uses a standard convolution to obtain the prediction result of the branch. Assume that the prediction result of the lth layer is represented as v l , then v l The Dice similarity coefficient DSC at can be expressed as:

[0077]

[0078] Where TP, FP and FN are the number of predicted voxels of true positive, false positive and false negative, respectively. After selecting the prediction result with the maximum confidence, the remaining prediction results and their DSC values ​​are calculated, and the prediction results greater than the threshold are excluded. The attenuation function of NMS can be expressed as:

[0079]

[0080] like Figure 5 As shown in Figure 1, the loss of the entire network consists of all the loss values ​​of all layers of the auxiliary prediction structure and the fusion layer. The 3D MRI image input sample is represented as I = {v i |i=1,...,n}, the corresponding true segmentation result GT is expressed as G={g i |i=1,...,n}, the actual segmentation result produced by RAPNet is expressed as Y={y i |i=1,...,n}, where sample v i , g i and i They are represented as the i-th input sample and the corresponding GT voxel, as well as the voxel corresponding to the actual segmentation result. The overall loss function of the network is as follows:

[0081] L total (G,Y,I)=L aided (G,Y,I)+L fuse(G,Y,I)

[0082] Among them, L aided (·,·) represents the loss of auxiliary structure for hierarchical prediction, L fuse (·,·) represents the loss of the fusion layer. The loss branch of each layer in the auxiliary structure is associated with the prediction result fusion module used to calculate the loss through the softmax function. The loss of the auxiliary structure can be expressed as:

[0083]

[0084] Where M represents the total number of auxiliary structures, α m The parameter is used to adjust the ratio of the overall loss and the loss of the mth layer in the auxiliary structure. The loss of the auxiliary structure layer is set to zero using the decay function, i.e., α m ×0.1×(1-t / N)→α m , where t represents the number of rounds, N represents the total number of rounds, Represents the loss function that measures the similarity between the prediction results of the mth layer in the auxiliary structure and the GT.

[0085] In order to effectively fuse the prediction results of each layer in the auxiliary structure, the final fusion layer is weighted during the training process, and the loss function of the fusion layer is expressed as:

[0086]

[0087] Among them, σ(·) represents the composite nonlinear mapping of BN and ReLU after the standard convolution function, β m represents the learnable fusion weights, represents the output of the mth layer in the auxiliary structure, l(·,·) is formally equivalent to That is the loss function.

[0088] The process of training the model in this embodiment specifically includes the following steps:

[0089] S1: Input the image data in the training set into the RAPNet network for processing;

[0090] S2: The 3D recursive residual convolution unit of the backbone network accumulates features of different depths through multiple operations on the input data;

[0091] S3: The dilated convolutions with different dilation rates in the feature pyramid learn the multi-scale features of the input data. Through the feature recalibration mechanism, it learns to use global information to selectively emphasize the informative features and suppress the less useful features.

[0092] S4: The auxiliary prediction structure compares the multi-layer prediction results with different scale features with the label image pixel by pixel to obtain the prediction result error of each branch;

[0093] S5: Calculate the loss function of each branch based on the error result, and input the error into the model through the back propagation algorithm, update the convolution kernel weights and parameters in each module when the network extracts features, and complete the model training when the loss function is minimized.

[0094] The proposed method is compared with the current typical methods on the BraTS 2019 dataset, and the comparable results are summarized in Table 1. The evaluation results show that RAPNet has achieved competitive advantages in segmenting TC and ET. The main advantages of RAPNet are that the network fuses multi-scale contextual information after a certain amount of feature accumulation, and uses feature pyramids to learn the characteristics of lesions, which improves the model's ability to distinguish various lesion tissues. The cross-model attention mechanism integrated in the pyramid structure enhances the tumor region features from each branch, that is, the features learned by the network have the natural characteristics of multi-scale and nonlinear abstraction. The model reasonably utilizes this feature, which is conducive to aggregating abstract information at different levels and improving the focus of the target area through operations such as dilated convolution.

[0095] Table 1 Comparison results with current advanced methods

[0096]

[0097]

[0098] Figure 7 Results of visual segmentation of the BraTS 2019 dataset using the trained model are shown. To verify the effectiveness and robustness of the proposed method, a subject named #Brats19_CBICA_ATX_1 is randomly selected from the test set selected from BraTS 2019, and the segmentation results of T1, T2, T1ce, and Flair modes in the coronal, sagittal, and horizontal planes are shown and compared with Ground-Truth. Compared with Ground-Truth, RAPNet not only distinguishes the entire tumor from healthy tissue, but also preserves the hierarchical structure of the tumor well. These examples show that RAPNet has a better effect in learning multi-scale features of lesions.

[0099] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A three-dimensional segmentation method for brain tumor MRI images based on RAPNet network, characterized in that: The following steps are involved: Build the RAPNet network and train it; Input the brain MRI image into the trained RAPNet network for image recognition and segmentation to obtain the segmented brain tumor MRI image and its substructure area; The RAPNet network includes a backbone network, a feature pyramid, and auxiliary predictions, among which: The backbone network consists of dilated convolution and multiple improved recursive residual convolution units, which are used to extract shallow and deep features of the input image; the improved recursive residual convolution unit consists of three ISE-Res modules, three 1×1×1 standard convolution layers, one 3×3×3 standard convolution layer and one residual connection layer, where: The 3×3×3 standard convolutional layer obtains a high-resolution image based on the input image; Input the high-resolution image into a 1×1×1 standard convolutional layer to compress the number of channels without changing the image size; The image obtained by 1×1×1 standard convolution is added to the high-resolution image and then input into the ISE-Res module; The features output by the first ISE-Res module are then fed into a 1×1×1 standard convolutional layer to compress the number of channels without changing the image size. The image obtained by 1×1×1 standard convolution is added to the high-resolution image and then input into the second ISE-Res module; The features output by the second ISE-Res module are then fed into a 1×1×1 standard convolutional layer to compress the number of channels without changing the image size. The image obtained by 1×1×1 standard convolution is added to the high-resolution image and then input into the third ISE-Res module; The features output by the third ISE-Res module are added to the input image as the output of the improved recursive residual convolution unit; The ISE-Res module consists of a global mean pooling, a global maximum pooling, three ReLU layers, a Sigmoid layer and a residual connection layer, where: The input data uses global mean pooling and global maximum pooling to sample the global information of each channel in parallel, and the features after parallel sampling are fed to two ReLU layers for activation; The outputs of the two ReLU layers are added and input into the Sigmoid layer; The residual connection layer multiplies the Sigmoid output by the input data and then linearly superimposes it with the input data as a residual connection; The superimposed data is input into a ReLU layer for activation; The feature pyramid consists of a dilated convolution and a cross-model attention mechanism, which is used to extract feature maps of different scales to improve the overall model's ability to recognize tumors of different sizes. The auxiliary prediction part is composed of standard convolutions, which is used to comprehensively observe all targets that benefit from the feature fusion module.

2. According to claim 1, a method for three-dimensional segmentation of brain tumor MRI images based on RAPNet network is characterized in that: The improved recursive residual convolution unit extracts shallow and deep features of the input image, which specifically includes the following steps: in, represents the output of the ISE-R2CU unit network at time step t, and are the inputs of the standard convolution and recursive convolution units of the lth layer in the network, respectively. and The values ​​of are the weights of the standard convolution and recursive convolution units in the k-th layer feature map, b k is the deviation, is the output of the recursive convolutional unit in layer l, x l is the input sample of the lth layer in the network, w l is the pixel on the feature map of layer l.

3. The method for three-dimensional segmentation of brain tumor MRI images based on RAPNet network according to claim 1, characterized in that: The cross-model attention mechanism includes three ISE-Res; each ISE-Res is integrated in each feature extraction branch of the feature pyramid, and an element addition operation is performed at the output position of each attention mechanism module; the activation features of each branch are connected from the previous branch to the feature map of the next branch through a splicing operation.

4. The method for three-dimensional segmentation of brain tumor MRI images based on RAPNet network according to claim 1, characterized in that: The auxiliary prediction structure contains three 1×1×1 standard convolutional layers and a segmentation prediction layer; each input feature map branch uses a standard convolutional and segmentation prediction layer to obtain the prediction result of the branch.

5. The method for three-dimensional segmentation of brain tumor MRI images based on RAPNet network according to claim 1, characterized in that: The training process of the RAPNet network includes: S1: Input the image data in the training set into the RAPNet network for processing; S2: The 3D recursive residual convolution unit of the backbone network accumulates features of different depths through multiple operations on the input data; S3: The dilated convolutions with different dilation rates in the feature pyramid learn the multi-scale features of the input data, and through the feature recalibration mechanism, learn to use global information to selectively emphasize the information-rich features; S4: The auxiliary prediction structure compares the multi-layer prediction results with different scale features with the label image pixel by pixel to obtain the prediction result error of each branch; S5: Calculate the loss function of each branch based on the error result, and input the error into the model through the back propagation algorithm, update the convolution kernel weights and parameters in each module when the network extracts features, and complete the model training when the loss function is minimized.

6. The method for three-dimensional segmentation of brain tumor MRI images based on RAPNet network according to claim 5, characterized in that: The loss function of the RAPNet network is: L total (G,Y,I)=L aided (G,Y,I)+L fuse (G,Y,I) a m ×0.1×(1-t / N)→α m Among them, I is the 3D MRI image input sample, G is the corresponding true segmentation result, Y is the actual segmentation result generated by using RAPNet, and L total (G,Y,I) is the overall loss of the network, L aided (G,Y,I) represents the loss of auxiliary structure for hierarchical prediction, L fuse (G,Y,I) represents the loss of the fusion layer, M represents the total number of layers of the auxiliary structure, α m The parameter is used to adjust the ratio of the overall loss and the loss of the mth layer in the auxiliary structure. t represents the number of rounds, and N represents the total number of rounds. represents the loss function that measures the similarity between the prediction result of the mth layer in the auxiliary structure and the true segmentation result, σ(·) represents the composite nonlinear mapping of BN and ReLU after the standard convolution function, β m represents the learnable fusion weights, represents the output of the mth layer in the auxiliary structure, Indicates G and The loss value between .

Images