A Gastrointestinal MRI Image Segmentation Method Based on BFTransNet Hybrid Network

Abstract

This invention discloses a gastrointestinal MRI image segmentation method based on a BFTransNet hybrid network. The method involves overlaying and fusing multiple adjacent gastrointestinal MRI slice images to form a target gastrointestinal MRI image, which is then input into a BFTransNet hybrid network pre-trained for gastrointestinal region segmentation to obtain the gastrointestinal region segmentation result of the target gastrointestinal MRI image. The BFTransNet hybrid network includes a hybrid encoding module based on a multi-level CNN convolutional network and a Transformer transformation model, and an attention fusion segmentation module based on a multi-level BiFuse network. During the training process of the BFTransNet hybrid network, this invention constructs a multi-level total loss function to guide the optimization and updating of the BFTransNet hybrid network's recognition performance parameters. This method can quickly and accurately segment and recognize specified gastrointestinal regions in gastrointestinal MRI images, thereby solving the problem of time delays caused by manually outlining the location of the stomach, intestines, and other regions during radiotherapy, resulting in prolonged patient suffering.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing technology and artificial intelligence neural network technology, specifically to a method for segmenting gastrointestinal MRI images based on a BFTransNet hybrid network. Background Technology

[0002] Radiation oncologists attempt to use high-dose X-rays to target tumors and kill cancer cells while avoiding the stomach and intestines, using integrated magnetic resonance imaging (MRI) and linear accelerator systems to visualize the daily location of the tumor and gastrointestinal tract. However, because the tumor's location changes daily due to patient movement and posture, radiation oncologists must manually delineate the stomach and intestines to adjust the direction of X-rays and avoid irradiating the gastrointestinal tract. This process is often very time-consuming, extending daily treatments from 15 minutes to over an hour, causing greater suffering for the patient.

[0003] Using deep learning methods to automatically segment regions such as the stomach and intestines can reduce the time wasted manually outlining their locations, thus accelerating the patient's treatment process, reducing patient suffering, and enabling more effective treatment. In the application of deep learning methods, Convolutional Neural Networks (CNNs) have achieved great success in various visual tasks such as classification, segmentation, and object detection. Fully Convolutional Networks (FCNs) were the first to achieve end-to-end semantic segmentation. U-Net, using a symmetric encoder-decoder structure with skip connections to improve detail preservation, has become the mainstream architecture for medical image segmentation. Many U-Net variants, such as U-Net++ and Res-UNet, have further improved image segmentation performance.

[0004] While CNN-based methods possess excellent feature representation capabilities, the limited receptive field of convolutional kernels makes it difficult to establish explicit long-range dependencies. Specifically, in CNNs, each convolutional kernel only focuses on a local subset of pixels within the entire image. This causes the network to concentrate on local features, failing to accurately capture global spatial context features. This limitation of convolutional operations poses a significant challenge to learning global features, which is crucial for dense prediction tasks such as segmentation.

[0005] Current solutions for automatically segmenting the stomach and intestines in gastrointestinal MRI images using deep learning methods are still insufficient in terms of localization and segmentation accuracy due to the aforementioned limitations, making them difficult to apply effectively in real-world scenarios. Summary of the Invention

[0006] To address the aforementioned shortcomings of existing technologies, the present invention aims to provide a gastrointestinal MRI image segmentation method based on a BFTransNet hybrid network. This method can quickly and accurately segment and identify specified gastrointestinal regions in gastrointestinal MRI images, thereby solving the problem of prolonged patient suffering caused by manually outlining the location of the stomach, intestines, and other regions during radiotherapy.

[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0008] A method for segmenting gastrointestinal MRI images based on a BFTransNet hybrid network is proposed. This method involves overlaying and fusing multiple adjacent gastrointestinal MRI slice images to form a gastrointestinal MRI image to be tested. The image is then input into a BFTransNet hybrid network that has been pre-trained for gastrointestinal region segmentation to obtain the gastrointestinal region segmentation result of the gastrointestinal MRI image to be tested.

[0009] The BFTransNet hybrid network includes a hybrid encoding module based on a CNN convolutional network and a Transformer transformation model, and an attention fusion segmentation module based on a BiFuse network. The hybrid encoding module is used to extract features from the gastrointestinal MRI image under test using CNN convolutional downsampling feature extraction and Transformer encoding feature extraction, respectively. After obtaining the convolutional downsampling feature vector and the encoding downsampling feature vector corresponding to the gastrointestinal MRI image under test, the attention fusion segmentation module performs BiFuse cross-fusion of the convolutional downsampling feature vector and the encoding downsampling feature vector corresponding to the gastrointestinal MRI image under test with spatial attention and channel attention. Then, the obtained fused feature image is upsampled to restore the image, and a segmentation marker image carrying the gastrointestinal region segmentation mask with prediction recognition is obtained, which is used as the gastrointestinal region segmentation result of the gastrointestinal MRI image under test.

[0010] As a preferred approach, the construction method for the gastrointestinal MRI images to be tested is as follows:

[0011] Multiple adjacent gastrointestinal MRI slice images are selected, and each gastrointestinal MRI slice image is used as a pixel channel of the gastrointestinal MRI image to be tested. This allows the corresponding pixels of the selected gastrointestinal MRI slice images to be superimposed to obtain the gastrointestinal MRI image to be tested.

[0012] As a preferred approach, the specific processing method for fusing multiple adjacent gastrointestinal MRI slice images to generate the gastrointestinal MRI image to be tested is as follows:

[0013] One gastrointestinal MRI slice image was selected as the baseline image to be tested. Then, the K adjacent gastrointestinal MRI slice images before and after it were selected, for a total of 2K+1 gastrointestinal MRI slice images.

[0014] Generate a multi-channel image F(2K+1,H,W) where all pixel values ​​are 0. The pixel height and width of this multi-channel image F are equal to the pixel height H and pixel width W of the gastrointestinal MRI slice image, respectively, and the number of pixel channels C of the multi-channel image F is equal to 2K+1. Then, superimpose 2K+1 selected gastrointestinal MRI slice images onto the 2K+1 pixel channels of the multi-channel image F, i.e., let f... Ck =P k f Ck P represents the k-th pixel channel of a multi-channel image F. k This represents the kth selected gastrointestinal MRI slice image, where k = 1, 2, ..., 2K+1;

[0015] The resulting multi-channel image F, which is a superposition of 2K+1 gastrointestinal MRI slices, is used as the gastrointestinal MRI image to be tested.

[0016] As a preferred approach, the specific method for obtaining the convolutional downsampling feature vector corresponding to the gastrointestinal MRI image under test in the hybrid coding module is as follows:

[0017] The gastrointestinal MRI image to be tested is subjected to L-layer convolutional downsampling to obtain L-layer downsampled images of the gastrointestinal MRI image; for each downsampled image, it is divided and reconstructed into a sequence of image blocks. in, This represents the nth image block of the l-th downsampled image of the gastrointestinal MRI image to be tested. Each image block... The pixel size is p×p, and N is the total number of image blocks into which the downsampled image is divided. H and W represent the pixel height and pixel width of the gastrointestinal MRI image to be tested, respectively, and C represents the number of pixel channels in the gastrointestinal MRI image to be tested. Then, the image patch sequence of each downsampled image is mapped to a latent D-dimensional embedding space to obtain the position embedding vector corresponding to each downsampled image, which serves as the L-layer convolutional downsampled feature vector corresponding to the gastrointestinal MRI image to be tested.

[0018]

[0019] Among them, g l This represents the position embedding vector corresponding to the l-th layer downsampled image of the gastrointestinal MRI image under test, which is the l-th layer convolution downsampled feature vector corresponding to the gastrointestinal MRI image under test. Represents block embedding mapping, This indicates positional embedding.

[0020] As a preferred approach, the specific method for obtaining the encoded downsampled feature vector corresponding to the gastrointestinal MRI image under test in the hybrid coding module is as follows:

[0021] Embed the position corresponding to the Lth layer downsampled image into vector g. L The input is fed into the Transformer model for Transformer feature encoding, resulting in the Lth layer encoded feature vector t. L Then encode the feature vector t in the Lth layer. L L-1 layers of convolutional upsampling are performed to obtain a total of L layers of encoded feature vectors, which are used as the L-layer encoded downsampling feature vectors corresponding to the gastrointestinal MRI image to be tested.

[0022] As a preferred embodiment, the convolutional downsampling feature vector and the encoded downsampling feature vector corresponding to each layer of the gastrointestinal MRI image to be tested are respectively subjected to BiFuse cross-fusion through the attention fusion segmentation module; wherein, for any l-th layer convolutional downsampling feature vector and encoded downsampling feature vector, l = 1, 2, ..., L, the specific method of BiFuse cross-fusion through the attention fusion segmentation module is as follows:

[0023] Spatial attention and channel attention operations are performed on the convolutional downsampled feature vector and the encoded downsampled feature vector of the l-th layer, respectively. Then, element-wise dot product dimensionality reduction and fusion operations are performed on the convolutional downsampled feature vector and the encoded downsampled feature vector. Finally, the results of each operation are concatenated and residual fusion is performed to obtain the fused feature image of the l-th layer.

[0024]

[0025]

[0026]

[0027]

[0028] Among them, g l t l These represent the l-th layer convolutional downsampled feature vector and the encoded downsampled feature vector corresponding to the gastrointestinal MRI image under test, respectively. This indicates that the feature vector g is downsampled from the l-th convolutional layer. l The result of spatial attention computation. This indicates that the feature vector t is downsampled from the encoding of the l-th layer. l The result of channel attention calculation. This indicates that the feature vector g is downsampled from the l-th convolutional layer. l and encoding downsampled feature vector t l The result of element-wise dot product dimensionality reduction and fusion operation, f lThis represents the fused feature image of layer l; SpatialAttn() represents spatial attention operation; ChannelAttn() represents channel attention operation; Conv() represents convolution operation. These represent the preset downsampled feature vectors t of the l-th layer encoding, respectively. l and convolution downsampling feature vector g l The training weights, `element-wise()` represents element-wise dot product operation; `Concat()` represents concatenation operation; and `Residual()` represents residual operation.

[0029] As a preferred approach, the specific processing method for upsampling image restoration of the obtained fused feature image is as follows:

[0030] The fused feature image f of layer L L With the encoded feature vector t of the Lth layer L After stitching, convolutional upsampling is performed to obtain the upsampled restored image of the Lth layer.

[0031] Other arbitrary fused feature images f of layer l l Then the upsampled image of layer l+1 is restored. After stitching, convolutional upsampling is performed to obtain the upsampled restored image of the l-th layer.

[0032] Thus, the first layer upsampled restored image is obtained. The dimensions restored to pixel height H and pixel width W are segmentation marker images carrying a segmentation mask for the gastrointestinal region with predicted recognition, which are used as the gastrointestinal region segmentation results of the gastrointestinal MRI image to be tested.

[0033] As a preferred embodiment, the training method for gastrointestinal region segmentation of the BFTransNet hybrid network is as follows:

[0034] The types of gastrointestinal regions to be segmented and identified are pre-determined. Gastrointestinal MRI sample images with pre-marked gastrointestinal region segmentation masks for each type of gastrointestinal region are used as training samples to form a training sample set for the corresponding gastrointestinal region identification type. This set is input into the BFTransNet hybrid network, and a total loss function containing Dice loss and cross-entropy loss for evaluating the recognition performance of each gastrointestinal region identification type is constructed. The recognition performance parameters of the BFTransNet hybrid network for each gastrointestinal region identification type are optimized and updated with the goal of minimizing the total loss function, thereby training the BFTransNet hybrid network.

[0035] As a preferred embodiment, the BFTransNet hybrid network performs L-layer convolutional downsampling feature extraction and upsampling image restoration. The total loss function CL used for training the BFTransNet hybrid network for gastrointestinal region segmentation is... total This includes a loss function for evaluating the recognition performance of the upsampled reconstructed image from the L-layer output:

[0036]

[0037] in, Here, CL represents the weights of the loss function corresponding to the performance evaluation of the l-th layer, where l = 1, 2, ..., L, and L is the number of layers in the BFTransNet hybrid network for upsampling image reconstruction. l Let be the loss function used to evaluate the recognition performance of the upsampled reconstructed image output from layer l, and:

[0038] CL l =WDL l +WCE l ;

[0039] WDL l WCE l denoted as Dice loss and cross-entropy loss, respectively, used to evaluate the performance of upsampled image recognition based on the output of layer l.

[0040] As a preferred approach, the Dice loss WDL is used to evaluate the performance of upsampled image recognition based on the output of layer l. l Represented as:

[0041]

[0042] in, This represents the Dice score of the upsampled restored image output from layer l for the type of gastrointestinal region identified, where m = 1, 2, ..., M, and M is the number of types of gastrointestinal regions identified. The pre-defined Dice loss weights are used to identify the types corresponding to the m-th gastrointestinal region. This represents the segmentation mask value predicted by the upsampled restored image output from layer l for the i-th training sample in the m-th gastrointestinal region identification category. This represents the true segmentation mask value of the i-th training sample in the m-th gastrointestinal region identification category, where i = 1, 2, ..., N. m N m This represents the number of training samples for identifying the species in the m-th gastrointestinal region; The weighted Dice score represents the performance of the upsampled reconstructed image recognition based on the output of layer l.

[0043] The cross-entropy loss WCE is used to evaluate the performance of upsampled image recognition based on the output of layer l.l Represented as:

[0044]

[0045] in, The cross-entropy loss weight is preset for the type of identification corresponding to the m-th gastrointestinal region.

[0046] Compared with the prior art, the present invention has the following beneficial technical effects:

[0047] 1. This invention constructs a gastrointestinal MRI image carrying 2.5D structural data from multiple adjacent gastrointestinal MRI slice images, which can reflect certain spatial contextual features, facilitating better extraction of regional structural features in the image. Furthermore, it uses a BFTransNet hybrid network pre-trained for gastrointestinal region segmentation to predict and identify the segmentation mask for the gastrointestinal region, thereby achieving fast and accurate segmentation of the gastrointestinal region in the gastrointestinal MRI image.

[0048] 2. The BFTransNet hybrid network model used in this invention employs a CNN-Transformer hybrid encoding module for feature extraction. This effectively leverages the representational ability of the CNN convolutional network for local features and the modeling ability of the Transformer model for global contextual features. By continuously upsampling the encoded downsampled feature vectors extracted by the Transformer model to restore the size of the feature map, it is then fused with the corresponding dimension convolutional downsampled feature vectors extracted by the CNN convolutional network through the BiFuse module. By cross-modeling the features extracted by the CNN and Transformer with different attention levels, high and low-level features are utilized more efficiently, thus preventing information redundancy. Subsequently, the fused result is convolutionally upsampled, gradually fusing features from different levels from the bottom layer to finally restore the original image size, resulting in a segmentation marker image carrying a segmentation mask for the gastrointestinal region with predictive recognition, thus predicting the segmentation result of the gastrointestinal region.

[0049] 3. This invention also uses a multi-level total loss function to guide the optimization and updating of the recognition performance parameters of the BFTransNet hybrid network. It predicts segmentation masks for both low-level features and high-level layer outputs, and then calculates the loss function with the real mask. Different weights are then assigned to the loss functions at different levels, which makes fuller use of low-level features and has a positive effect on improving the training effect of the BFTransNet hybrid network and the segmentation accuracy after training.

[0050] 4. The method of the present invention can quickly and accurately segment and identify a specified gastrointestinal region in gastrointestinal MRI images, thereby solving the problem of time delay caused by manually outlining the location of the stomach, intestines and other regions during radiotherapy, resulting in longer periods of patient suffering. Attached Figure Description

[0051] Figure 1 This is a flowchart of the gastrointestinal MRI image segmentation method based on the BFTransNet hybrid network of the present invention.

[0052] Figure 2 This is a schematic diagram of an example structure of the BFTransNet hybrid network used in this invention.

[0053] Figure 3 This is a schematic diagram of the data processing flow of the BiFuse network module in the BFTransNet hybrid network.

[0054] Figure 4 This is a schematic diagram illustrating the process of using a multi-level total loss function to guide the optimization training of the BFTransNet hybrid network's recognition performance parameters in the method of this invention.

[0055] Figure 5 This is a comparison chart of the gastrointestinal region segmentation results using the method of the present invention and other model segmentation methods in the embodiments. Detailed Implementation

[0056] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0057] like Figure 1 As shown, this invention proposes a gastrointestinal MRI image segmentation method based on a BFTransNet hybrid network. This method first overlays and fuses multiple adjacent gastrointestinal MRI slice images to form a gastrointestinal MRI image to be tested, which is then input into a BFTransNet hybrid network that has been pre-trained for gastrointestinal region segmentation to obtain the gastrointestinal region segmentation result of the gastrointestinal MRI image to be tested, thereby achieving image segmentation of the gastrointestinal region.

[0058] The BFTransNet hybrid network used in the method of this invention includes a hybrid encoding module based on a CNN convolutional network and a Transformer transformation model, and an attention fusion segmentation module based on a BiFuse network. The hybrid encoding module performs CNN convolutional downsampling feature extraction and Transformer encoding feature extraction on the gastrointestinal MRI image to be tested, obtaining convolutional downsampling feature vectors (which can be considered as CNN branch feature vectors) and encoding downsampling feature vectors (which can be considered as Transformer branch feature vectors) corresponding to the gastrointestinal MRI image. Then, the attention fusion segmentation module performs BiFuse cross-fusion of the convolutional downsampling feature vectors and encoding downsampling feature vectors corresponding to the gastrointestinal MRI image to be tested, applying spatial attention and channel attention. Finally, the obtained fused feature image is upsampled for image restoration to obtain a segmentation marker image carrying a gastrointestinal region segmentation mask with predicted recognition, which serves as the gastrointestinal region segmentation result of the gastrointestinal MRI image to be tested.

[0059] The following is a more detailed description of the gastrointestinal MRI image segmentation method based on the BFTransNet hybrid network of the present invention.

[0060] 1. Image preprocessing

[0061] To better utilize the relationships between MRI slices, this invention employs a preprocessing method of overlaying and fusing multiple adjacent gastrointestinal MRI slice images to construct a gastrointestinal MRI image with 2.5D structural features. The technical implementation involves selecting multiple adjacent gastrointestinal MRI slice images, using each slice as a pixel channel of the target image, and then overlaying corresponding pixels from each slice to obtain the final gastrointestinal MRI image. Therefore, by constructing a 2.5D structural data image from multiple adjacent gastrointestinal MRI slice images, the constructed image carrying 2.5D structural data can reflect certain spatial contextual features, which is beneficial for subsequent extraction of regional structural features.

[0062] The specific construction method is as follows: Select a gastrointestinal MRI slice image as the baseline image to be tested. Then select the K adjacent gastrointestinal MRI slice images before and after it, for a total of 2K+1 gastrointestinal MRI slice images. Next, generate a multi-channel image F(2K+1,H,W) where all pixel values ​​are 0. The pixel height and pixel width of this multi-channel image F are equal to the pixel height H and pixel width W of the gastrointestinal MRI slice image, respectively, and the number of pixel channels C of the multi-channel image F is equal to 2K+1. Then, superimpose the selected 2K+1 gastrointestinal MRI slice images onto the 2K+1 pixel channels of the multi-channel image F, i.e., let f...Ck =P k f Ck P represents the k-th pixel channel of a multi-channel image F. k Let f represent the selected k-th gastrointestinal MRI slice image, k = 1, 2, ..., 2K+1; the resulting multi-channel image F, which is a superposition of 2K+1 gastrointestinal MRI slice images, is the gastrointestinal MRI image to be tested.

[0063] From a technical implementation perspective, when selecting multiple adjacent gastrointestinal MRI slice images, the value of K can range from 1 to 4, meaning 1, 2, 3, or 4 images can be selected. In other words, the number of gastrointestinal MRI slice images selected can range from 3 to 9. For specific technical applications, the preferred value of K is 1, meaning 3 gastrointestinal MRI slice images are selected. The number of gastrointestinal MRI slice images selected to construct the 2.5D structural data image should not be too large. On the one hand, too many images increase the amount of data processing; on the other hand, too many image channels may adversely affect the subsequent identification of the boundaries of the gastrointestinal region structures.

[0064] 2. BFTransNet Hybrid Network

[0065] After being trained on gastrointestinal region segmentation, the BFTransNet hybrid network is used to predict and identify the gastrointestinal region segmentation mask of the gastrointestinal MRI image to be tested. The input gastrointestinal MRI image to be tested is processed by the BFTransNet hybrid network and can output a segmentation marker image carrying the predicted and identified gastrointestinal region segmentation mask.

[0066] A schematic diagram of an example structure of the BFTransNet hybrid network used in the method of this invention is shown below. Figure 2 As shown, based on the Unet architecture, a multi-layered CNN-Transformer hybrid coding network module structure can be designed to perform multi-layered CNN convolutional downsampling feature extraction and Transformer coding feature extraction on the gastrointestinal MRI images to be tested. Then, a BiFuse network module is used to cross-fuse the feature vectors of each layer's CNN branch and Transformer branch. The outputs of each layer's BiFuse network module are then concatenated and upsampled layer by layer with the output of the bottom Transformer module until the original image size is restored, resulting in a segmented labeled image carrying a gastrointestinal region segmentation mask with predicted recognition. During the training of the BFTransNet hybrid network, a loss function for evaluating recognition performance is introduced for the upsampled restored image output at each layer, constructing a multi-layered total loss function to guide the optimization and updating of the BFTransNet hybrid network's recognition performance parameters.

[0067] 2.1) CNN-Transformer Hybrid Coding Module

[0068] The hybrid encoding module based on CNN convolutional network and Transformer transformation model is used to extract CNN convolutional downsampling features and Transformer encoding features from the gastrointestinal MRI image to be tested, so as to obtain the convolutional downsampling feature vector (which can be regarded as CNN branch feature vector) and the encoding downsampling feature vector (which can be regarded as Transformer branch feature vector) corresponding to the gastrointestinal MRI image to be tested.

[0069] Specifically, such as Figure 2 As shown, the specific method by which the CNN-Transformer hybrid coding module obtains the convolutional downsampling feature vector corresponding to the gastrointestinal MRI image under test is as follows:

[0070] First, L-layer convolutional downsampling is performed on the gastrointestinal MRI images to be tested. Common downsampling models such as ResNet50 can be used to perform convolutional downsampling, downsampling the (H,W) sized (i.e., pixel height H, pixel width W) gastrointestinal MRI images to be tested into (H / 2,W / 2)), (H / 4,W / 4), (H / 8,W / 8), ..., (H / 2) layers. L W / 2 L The image is downsampled into L layers of size L. From a technical implementation perspective, the number of convolutional downsampling layers L can be set from 1 to 6 layers depending on the situation, but the number of layers should not be too many, as too many downsampling layers will increase the amount of data processing without significantly improving the segmentation accuracy. For specific technical applications, it is preferable to set 4 layers, that is, the preferred value of L is 4. Then, for each downsampled layer image, it is divided and reconstructed into a sequence of image blocks. in, This represents the nth image block of the l-th downsampled image of the gastrointestinal MRI image to be tested. Each image block... The pixel size is p×p, and N is the total number of image blocks into which the downsampled image is divided. H and W represent the pixel height and pixel width of the gastrointestinal MRI image to be tested, respectively, and C represents the number of pixel channels in the gastrointestinal MRI image to be tested. Then, the image patch sequences of each downsampled image layer are mapped to a latent D-dimensional embedding space to obtain the position embedding vectors corresponding to each downsampled image layer, which serve as the L-layer convolutional downsampled feature vectors corresponding to the gastrointestinal MRI image to be tested. By learning specific position embeddings, these position embeddings are added to the image patch embeddings, ensuring that the resulting L-layer convolutional downsampled feature vectors are all feature vectors carrying positional information, as specifically represented below:

[0071]

[0072] Among them, g l This represents the position embedding vector corresponding to the l-th layer downsampled image of the gastrointestinal MRI image under test, which is the l-th layer convolution downsampled feature vector corresponding to the gastrointestinal MRI image under test. Represents block embedding mapping, E pos ∈R N×D This indicates positional embedding.

[0073] The specific method used in the CNN-Transformer hybrid coding module to obtain the encoded downsampled feature vector corresponding to the gastrointestinal MRI image under test is as follows:

[0074] After obtaining the L-th layer convolutional downsampled feature vector, the position corresponding to the L-th layer downsampled image is embedded into the vector g. L The input is fed into the Transformer model for Transformer feature encoding, resulting in the Lth layer encoded feature vector t. L The resulting Lth layer encoded feature vector t L It is the reconstructed (H / 2) L W / 2 L The feature image is of size t; then, the feature vector t is encoded in the Lth layer. L By performing L-1 layer convolution upsampling, and then upsampling layer by layer, we obtain (H / 2) L W / 2 L (H / 2) L -1 W / 2 L-1 The L-layer encoded feature vectors of sizes (H / 4, W / 4), (H / 2, W / 2) are used as the L-layer encoded downsampling feature vectors corresponding to the gastrointestinal MRI images to be tested.

[0075] 2.3) BiFuse Attention Fusion Segmentation Module

[0076] The convolutional downsampled feature vectors and encoded downsampled feature vectors corresponding to each layer of the gastrointestinal MRI image to be tested are respectively subjected to BiFuse cross-fusion through an attention fusion segmentation module based on a BiFuse network. Specifically, for any l-th layer convolutional downsampled feature vector and encoded downsampled feature vector, l = 1, 2, ..., L, the process of BiFuse cross-fusion through the attention fusion segmentation module is as follows: Figure 3 As shown, the specific method is as follows:

[0077] Spatial attention and channel attention operations are performed on the convolutional downsampled feature vector and the encoded downsampled feature vector of the l-th layer, respectively. Then, element-wise dot product dimensionality reduction and fusion operations are performed on the convolutional downsampled feature vector and the encoded downsampled feature vector. Finally, the results of each operation are concatenated and residual fusion is performed to obtain the fused feature image of the l-th layer.

[0078]

[0079]

[0080]

[0081]

[0082] Among them, g l t l These represent the l-th layer convolutional downsampled feature vector and the encoded downsampled feature vector corresponding to the gastrointestinal MRI image under test, respectively. This indicates that the feature vector g is downsampled from the l-th convolutional layer. l The result of spatial attention computation. This indicates that the feature vector t is downsampled from the encoding of the l-th layer. l The result of channel attention calculation. This indicates that the feature vector g is downsampled from the l-th convolutional layer. l and encoding downsampled feature vector t l The result of element-wise dot product dimensionality reduction and fusion operation, f l This represents the fused feature image of layer l; SpatialAttn() represents the spatial attention operation (SA operation); ChannelAttn() represents the channel attention operation (CA operation); Conv() represents the convolution operation. These represent the preset downsampled feature vectors t of the l-th layer encoding, respectively. l and convolution downsampling feature vector g l The training weights, `element-wise()` represents element-wise dot product operation; `Concat()` represents concatenation operation; and `Residual()` represents residual operation.

[0083] from Figure 3 As shown in the BiFuse cross-fusion process, when the encoded downsampled feature vector t is... l In the CA operation chain, the ChannelAttn channel attention operation process is as follows: First, the encoded downsampled feature vector t is processed. l Pooling is performed along the spatial dimension to compress it into a single value, thus encoding the downsampled feature vector t.l Just by (H / 2) l W / 2 l The image of size 1) is transformed into a one-dimensional vector, and then the channel dimension is scaled using three 1×1 convolutions and batch normalization (fully connected layer FC). Finally, the scaled result is combined with the encoded downsampled feature vector t. l Perform a Sigmoid calculation to obtain the channel attention operation result.

[0084] Downsampling the feature vector g of the convolution l In the SA operation chain, the SpatialAttn spatial attention operation process is as follows: First, the convolution downsamples the feature vector g. l Max pooling and average pooling are performed separately along the channel dimension, followed by dimensionality reduction using a 3×3 convolutional layer (Conv). Finally, the dimensionality reduction result is combined with the convolutional downsampled feature vector g. l Perform a Sigmoid calculation to obtain the spatial attention operation result.

[0085] And in Figure 3 In the intermediate element-wise dot product dimensionality reduction and fusion operation chain shown, the encoded downsampled feature vector t is then processed. l and convolution downsampling feature vector g l Multiply by their respective training weights The two data points are then fused using element-wise dot product operations (Multiply), followed by dimensionality reduction of the fused feature data using a 3x3 convolutional layer (Conv), yielding the element-wise dot product dimensionality reduction and fusion result.

[0086] Finally, After concatenating the three elements, a residual module is used to obtain the fused feature image f. l .

[0087] The above processing procedure uses SpatialAttn spatial attention operation to downsample the convolutional feature vector g. l Channel attention is used to facilitate the extraction of convolutional downsampling feature vectors g from the Transformer branch. l Global information; and through ChannelAttn channel attention operations, the encoded downsampled feature vector t is enhanced. l Local details are extracted, and irrelevant regions are suppressed; a convolutional downsampling feature vector g is constructed through element-wise dot product dimensionality reduction and fusion operations. l and encoding downsampled feature vector t lThe intersection between the two; finally, the interactive fusion features. and The fused feature image f generated by the residual blocks after concatenation l It effectively captures the global and local context of the current spatial resolution, thereby helping to segment the gastrointestinal region more accurately.

[0088] After obtaining the BiFuse cross-fusion output of each layer, the obtained fusion feature images of each layer are upsampled and restored using a progressive upsampling method. Specifically, the fusion feature image f of the Lth layer is... L With the encoded feature vector t of the Lth layer L After stitching, convolutional upsampling is performed to obtain the upsampled restored image of the Lth layer. And the fused feature image f of the l-th layer of any other non-L layer l Then, the image is recovered by upsampling from the (l+1)th layer. After stitching, convolutional upsampling is performed to obtain the upsampled restored image of the l-th layer. This stitching can utilize fused global and local spatial context features at different scales; thus, progressive upsampling is performed layer by layer, and finally, the fused feature image f from layer 1 is used. 1 Upsampled image from layer 2 The first-layer upsampled restored image obtained by convolutional upsampling after stitching. This is an image that has been restored to its original dimensions of pixel height H and pixel width W; this upsampled restored image It is a segmentation marker image carrying a segmentation mask for the gastrointestinal region with predictive recognition, which is used as the segmentation result of the gastrointestinal region in the MRI image of the gastrointestinal tract to be tested.

[0089] 3. Multi-level total loss function guidance strategy for the BFTransNet hybrid network training and optimization process

[0090] The method of this invention uses a pre-trained BFTransNet hybrid network to classify and predict the gastrointestinal region, thereby achieving image segmentation of the gastrointestinal region in gastrointestinal MRI images. The effect and accuracy of image segmentation largely depend on the training and optimization method of the hybrid network and the selection of the loss function.

[0091] Therefore, during the training of the BFTransNet hybrid network, this invention introduces a loss function for evaluating recognition performance for the upsampled restored image output at each level, thereby constructing a multi-level total loss function to guide the optimization and updating of the recognition performance parameters of the BFTransNet hybrid network.

[0092] Specifically, in the process of training and optimizing the BFTransNet hybrid network, the method of this invention adopts a combination of weighted Dice loss and weighted cross-entropy loss to supervise the segmentation training process. The training method is as follows: the types of gastrointestinal regions to be segmented and identified are determined in advance; gastrointestinal MRI sample images with pre-marked gastrointestinal region segmentation masks for each type of gastrointestinal region are used as training samples to form a training sample set for the corresponding type of gastrointestinal region identification. This set is input into the BFTransNet hybrid network, and a total loss function containing Dice loss and cross-entropy loss for evaluating the recognition performance of each type of gastrointestinal region identification is constructed. The recognition performance parameters of the BFTransNet hybrid network for each type of gastrointestinal region identification are optimized and updated with the goal of minimizing the total loss function, thereby training the BFTransNet hybrid network.

[0093] In particular, since the BFTransNet hybrid network performs L layers of convolutional downsampling feature extraction and upsampling image restoration, the total loss function CL used for training the BFTransNet hybrid network for gastrointestinal region segmentation is... total This includes a loss function for evaluating the recognition performance of the upsampled reconstructed image from the L-layer output:

[0094]

[0095] in, Here, CL represents the weights of the loss function corresponding to the performance evaluation of the l-th layer, where l = 1, 2, ..., L, and L is the number of layers in the BFTransNet hybrid network for upsampling image reconstruction. l Let be the loss function used to evaluate the recognition performance of the upsampled reconstructed image output from layer l, and:

[0096] CL l =WDL l +WCE l ;

[0097] WDL l WCE l denoted as Dice loss and cross-entropy loss, respectively, used to evaluate the performance of upsampled image recognition based on the output of layer l.

[0098] The Dice loss WDL is used to evaluate the performance of upsampled image recognition based on the output of layer l. l Represented as:

[0099]

[0100] in, This represents the Dice score of the upsampled restored image output from layer l for the type of gastrointestinal region identified, where m = 1, 2, ..., M, and M is the number of types of gastrointestinal regions identified. The pre-defined Dice loss weights are used to identify the types corresponding to the m-th gastrointestinal region. This represents the segmentation mask value predicted by the upsampled restored image output from layer l for the i-th training sample in the m-th gastrointestinal region identification category. This represents the true segmentation mask value of the i-th training sample in the m-th gastrointestinal region identification category, where i = 1, 2, ..., N. m N m This represents the number of training samples for identifying the species in the m-th gastrointestinal region; The weighted Dice score represents the performance of the upsampled reconstructed image recognition based on the output of layer l.

[0101] The cross-entropy loss WCE is used to evaluate the performance of upsampled image recognition based on the output of layer l. l Represented as:

[0102]

[0103] in, The cross-entropy loss weight is preset for the type of identification corresponding to the m-th gastrointestinal region.

[0104] For example, if the gastrointestinal regions to be segmented are categorized into four classes: background, large intestine, small intestine, and stomach (background as class 1, large intestine as class 2, small intestine as class 3, and stomach as class 4), the BFTransNet hybrid network will look like... Figure 2 As shown, a four-layer convolutional downsampling feature extraction and upsampling image restoration were performed. Figure 4 As shown, during the training process of the BFTransNet hybrid network using training sample sets of four classes, after obtaining the upsampled restored images of its four-layer output, the Dice score of each class in each layer is... According to the formula The calculations are performed, l = 1, 2, 3, 4; then, different Dice loss weights are assigned to the Dice scores of the four classes in each layer. For example, the Dice loss weights for the background, large intestine, small intestine, and stomach are assigned as 1, 3, 2, and 5, respectively, according to the formula... Calculate the weighted Dice score for each level Then, the Dice loss WDL for each layer for the four classes: background, large intestine, small intestine, and stomach is obtained. l Then through the formula Calculate the cross-entropy loss (WCE) for each layer for the four classes: background, large intestine, small intestine, and stomach. l Cross-entropy loss weights for the four classes They are also assigned numbers 1, 3, 2, and 5 respectively; then, the loss function CL for each layer for the four classes—background, large intestine, small intestine, and stomach—is calculated. l =WDL l +WCE l Finally, determine the total loss function for layer L. These are the weights of the loss function for the four layers in the BFTransNet hybrid network.

[0105] 4. Example

[0106] To demonstrate the segmentation performance of the proposed architecture, comparative experiments on gastrointestinal MRI image segmentation using different methods were conducted using examples on the UWMGI dataset.

[0107] In this embodiment, the gastrointestinal MRI image segmentation method based on the BFTransNet hybrid network of the present invention is compared with segmentation methods using networks such as U-Net, FPN, Deeplabv3, Deeplabv3+, and TransUent. For a fair performance comparison, the same strategy is used to train and test various different segmentation methods, such as the same input image size, hyperparameter settings, learning rate adjustment strategy, and loss function.

[0108] Table 1 below shows the performance comparison of the BFTransNet hybrid network model of this invention with other models on the UWMGI dataset for gastrointestinal region segmentation. Before training with the introduction of a multi-level total loss function guidance strategy, the Dice (Dice loss) and Iou (cross-entropy loss) of the BFTransNet hybrid network model of this invention trained on the test set reached 88.99% and 89.16%, respectively. After introducing the multi-level total loss function guidance strategy, the Dice (Dice loss) and Iou (cross-entropy loss) of the BFTransNet hybrid network model of this invention trained on the test set reached 89.16% and 86.18%, respectively. Furthermore, compared with the segmentation methods using other models, the loss values ​​of the BFTransNet hybrid network model of this invention show superior performance.

[0109] This demonstrates that the gastrointestinal MRI image segmentation method based on the BFTransNet hybrid network of this invention can achieve more accurate gastrointestinal region segmentation results.

[0110] Table 1: Comparison of segmentation performance of different models on the UWMGI dataset

[0111]

[0112] exist Figure 5The image visualization shows the segmentation results of the proposed BFTransNet hybrid network and other network models on the UWMGI dataset. It can be seen that the proposed method produces better segmentation results, more closely resembling the true region segmentation labels compared to other methods. The proposed method not only highlights the correct salient regions and eliminates confusing false positive lesions, but also produces coherent boundaries. This also demonstrates that the gastrointestinal MRI image segmentation method based on the BFTransNet hybrid network of this invention can perform more refined segmentation while preserving detailed region segmentation shape information.

[0113] This experiment also conducted a series of ablation experiments using TransUnet as a baseline to demonstrate the effectiveness of the proposed method. Specific experimental results are shown in Table 2. In TransUnet, BiFusion was used to replace the Concat operation in the decoder to fuse upsampled features and skip connection features. It can be seen that introducing the BiFusion module improved the Dice score by 0.32% compared to TransUnet. Subsequently, the TUP upsampling strategy was combined with the BiFusion module, resulting in improvements of 0.6% and 0.27% in Dice (Dice loss) and Iou (cross-entropy loss) compared to the baseline. After using a multi-level total loss function to guide the policy training process, Dice (Dice loss) and Iou (cross-entropy loss) were further improved by 1.21% and 0.96% compared to the baseline.

[0114] Table 2: Ablation Experiment

[0115]

[0116] To determine the optimal weights for different levels of the loss function, this experiment also tested different weights. A BFTransNet hybrid network was set up for 4 layers of convolutional downsampling feature extraction and upsampling image restoration. The sum of the weights of the loss function in the 4 layers was set to 1, and the weight of the loss function in the 1st layer (the total output layer) was set to... Let the values ​​be 0.5, 0.6, and 0.7 respectively, and set the weights of the loss function for layers 2, 3, and 4. Experiments were conducted with values ​​of 0.16, 0.13, and 0.1, respectively, and the results showed that when... and The segmentation effect is best when the value is 0.1.

[0117] Table 3: Comparison of segmentation performance with different loss function weights

[0118]

[0119] In summary, the method of the present invention has the following technical advantages:

[0120] 1. This invention constructs a gastrointestinal MRI image carrying 2.5D structural data from multiple adjacent gastrointestinal MRI slice images, which can reflect certain spatial contextual features, facilitating better extraction of regional structural features in the image. Furthermore, it uses a BFTransNet hybrid network pre-trained for gastrointestinal region segmentation to predict and identify the segmentation mask for the gastrointestinal region, thereby achieving fast and accurate segmentation of the gastrointestinal region in the gastrointestinal MRI image.

[0121] 2. The BFTransNet hybrid network model used in this invention employs a CNN-Transformer hybrid encoding module for feature extraction. This effectively leverages the representational ability of the CNN convolutional network for local features and the modeling ability of the Transformer model for global contextual features. By continuously upsampling the encoded downsampled feature vectors extracted by the Transformer model to restore the size of the feature map, it is then fused with the corresponding dimension convolutional downsampled feature vectors extracted by the CNN convolutional network through the BiFuse module. By cross-modeling the features extracted by the CNN and Transformer with different attention levels, high and low-level features are utilized more efficiently, thus preventing information redundancy. Subsequently, the fused result is convolutionally upsampled, gradually fusing features from different levels from the bottom layer to finally restore the original image size, resulting in a segmentation marker image carrying a segmentation mask for the gastrointestinal region with predictive recognition, thus predicting the segmentation result of the gastrointestinal region.

[0122] 3. This invention also uses a multi-level total loss function to guide the optimization and updating of the recognition performance parameters of the BFTransNet hybrid network. It predicts segmentation masks for both low-level features and high-level layer outputs, and then calculates the loss function with the real mask. Different weights are then assigned to the loss functions at different levels, which makes fuller use of low-level features and has a positive effect on improving the training effect of the BFTransNet hybrid network and the segmentation accuracy after training.

[0123] 4. The method of the present invention can quickly and accurately segment and identify a specified gastrointestinal region in gastrointestinal MRI images, thereby solving the problem of time delay caused by manually outlining the location of the stomach, intestines and other regions during radiotherapy, resulting in longer periods of patient suffering.

[0124] Finally, it should be noted that although the invention has been described with reference to preferred embodiments thereof, those skilled in the art should understand that various changes in form and detail may be made thereto without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A method for segmenting gastrointestinal MRI images based on a BFTransNet hybrid network, characterized in that, Multiple adjacent gastrointestinal MRI slice images are superimposed and fused to form the gastrointestinal MRI image to be tested. The image is then input into a BFTransNet hybrid network that has been pre-trained for gastrointestinal region segmentation to obtain the gastrointestinal region segmentation result of the gastrointestinal MRI image to be tested. The BFTransNet hybrid network includes a hybrid coding module based on a CNN convolutional network and a Transformer transformation model, and an attention fusion segmentation module based on a BiFuse network. The hybrid coding module is used to extract features from the gastrointestinal MRI image under test using CNN convolutional downsampling feature extraction and Transformer coding feature extraction, respectively. After obtaining the convolutional downsampling feature vector and the coding downsampling feature vector corresponding to the gastrointestinal MRI image under test, the attention fusion segmentation module performs BiFuse cross-fusion of the convolutional downsampling feature vector and the coding downsampling feature vector corresponding to the gastrointestinal MRI image under test with spatial attention and channel attention. Then, the obtained fused feature image is upsampled to restore the image, and a segmentation marker image carrying the gastrointestinal region segmentation mask with prediction recognition is obtained, which is used as the gastrointestinal region segmentation result of the gastrointestinal MRI image under test. The method for constructing the gastrointestinal MRI images to be tested is as follows: Multiple adjacent gastrointestinal MRI slice images are selected, and each gastrointestinal MRI slice image is used as a pixel channel of the gastrointestinal MRI image to be tested. This allows the corresponding pixels of the selected gastrointestinal MRI slice images to be superimposed to obtain the gastrointestinal MRI image to be tested. The specific processing method for generating the gastrointestinal MRI image to be tested by fusing multiple adjacent gastrointestinal MRI slice images is as follows: A gastrointestinal MRI slice image is selected as the baseline image. Then, K adjacent gastrointestinal MRI slice images and K adjacent gastrointestinal MRI slice images are selected, for a total of 2K+1 gastrointestinal MRI slice images. A multi-channel image F(2K+1,H,W) with all pixel values ​​set to 0 is generated. The pixel height and pixel width of this multi-channel image F are equal to the pixel height H and pixel width W of the gastrointestinal MRI slice image, respectively, and the number of pixel channels C of the multi-channel image F is equal to 2K+1. The 2K+1 selected gastrointestinal MRI slice images are then superimposed onto the 2K+1 pixel channels of the multi-channel image F, i.e.,... , This represents the k-th pixel channel of a multi-channel image F. This represents the k-th selected gastrointestinal MRI slice image. The resulting multi-channel image F, which is a superposition of 2K+1 gastrointestinal MRI slices, is used as the gastrointestinal MRI image to be tested; the value of K ranges from 1 to 4. The specific method used in the hybrid coding module to obtain the convolutional downsampling feature vector corresponding to the gastrointestinal MRI image under test is as follows: The gastrointestinal MRI image to be tested is subjected to L-layer convolutional downsampling to obtain L-layer downsampled images of the gastrointestinal MRI image; for each downsampled image, it is divided and reconstructed into a sequence of image blocks. ,in, The first image represents the gastrointestinal MRI image to be tested. The first layer of downsampled image partitioning Each image block The pixel size is , The total number of image blocks into which the downsampled image is divided, and , , These represent the pixel height and pixel width of the gastrointestinal MRI image to be tested. The pixel channel count is the number of the gastrointestinal MRI image to be tested; then, the image patch sequence of each downsampled image is mapped to the potential... In the dimensional embedding space, the position embedding vectors corresponding to the downsampled images of each layer are obtained, which are used as the L-layer convolutional downsampled feature vectors corresponding to the gastrointestinal MRI images to be tested: ， ； in, The first image represents the gastrointestinal MRI image to be tested. The position embedding vector corresponding to the subsampled image of the layer, that is, the first position of the gastrointestinal MRI image to be tested. Convolutional downsampling of feature vectors; Represents block embedding mapping, Indicates positional embedding; The specific method used in the hybrid coding module to obtain the coded downsampled feature vector corresponding to the gastrointestinal MRI image under test is as follows: Embed the position corresponding to the Lth layer downsampled image into the vector. The input is fed into the Transformer model for Transformer feature encoding, resulting in the Lth layer encoded feature vector. Then encode the feature vector of the Lth layer. L-1 layers of convolutional upsampling are performed to obtain a total of L layers of encoded feature vectors, which are used as the L-layer encoded downsampling feature vectors corresponding to the gastrointestinal MRI image to be tested.

2. The gastrointestinal MRI image segmentation method based on BFTransNet hybrid network according to claim 1, characterized in that, The convolutional downsampled feature vector and the encoded downsampled feature vector corresponding to each layer of the gastrointestinal MRI image to be tested are respectively cross-fused using the attention fusion segmentation module using BiFuse; wherein, for any ... Convolutional downsampling feature vectors and encoded downsampling feature vectors The specific method of BiFuse cross-fusion through the attention fusion segmentation module is as follows: For the Spatial attention and channel attention operations are performed on the convolutional downsampled feature vector and the encoded downsampled feature vector, respectively. Then, element-wise dot product dimensionality reduction and fusion are performed on the convolutional downsampled feature vector and the encoded downsampled feature vector. Finally, the results of each operation are concatenated and residual fusion is performed to obtain the first... Layer fusion feature image: ； ； ； ； in, , These represent the first and second images of the gastrointestinal MRI images to be tested. Convolutional downsampling feature vectors and encoded downsampling feature vectors Indicates the first Convolutional downsampling feature vector The result of spatial attention computation. Indicates the first Layer-encoded downsampled feature vector The result of channel attention calculation. Indicates the first Convolutional downsampling feature vector and encoding downsampled feature vectors The result of element-wise dot product dimensionality reduction and fusion operation. For the first Layer fusion feature image; Represents spatial attention operations; This represents channel attention operations; This represents the convolution operation. , These represent the preset values ​​for the first... Layer-encoded downsampled feature vector and convolution downsampling feature vector The training weights, This represents the element-wise dot product operation; This indicates a concatenation operation. This indicates residual operations.

3. The gastrointestinal MRI image segmentation method based on BFTransNet hybrid network according to claim 2, characterized in that, The specific processing method for upsampling image restoration of the obtained fused feature image is as follows: The fused feature image of layer L With the encoded feature vector of the Lth layer After stitching, convolutional upsampling is performed to obtain the upsampled restored image of the Lth layer. ; Other arbitrary numbers Layer fusion feature image Then with the first Upsampling and restoration of the image at +1 layer After concatenation and upsampling, the first convolution is performed to obtain the second convolution. Upsampling to restore the image of the layer ; Thus, the first layer upsampled restored image is obtained. Restored to pixel height and pixel width The size of the image is a segmentation marker image carrying a segmentation mask for the gastrointestinal region of the predicted recognition, which is used as the gastrointestinal region segmentation result of the gastrointestinal MRI image to be tested.

4. The gastrointestinal MRI image segmentation method based on BFTransNet hybrid network according to claim 1, characterized in that, The training method for gastrointestinal region segmentation of the BFTransNet hybrid network is as follows: The types of gastrointestinal regions to be segmented and identified are pre-determined. Gastrointestinal MRI sample images with pre-marked gastrointestinal region segmentation masks for each type of gastrointestinal region are used as training samples to form a training sample set for the corresponding gastrointestinal region identification type. This set is input into the BFTransNet hybrid network, and a total loss function containing Dice loss and cross-entropy loss for evaluating the recognition performance of each gastrointestinal region identification type is constructed. The recognition performance parameters of the BFTransNet hybrid network for each gastrointestinal region identification type are optimized and updated with the goal of minimizing the total loss function, thereby training the BFTransNet hybrid network.

5. The gastrointestinal MRI image segmentation method based on BFTransNet hybrid network according to claim 4, characterized in that, The BFTransNet hybrid network performs L-layer convolutional downsampling feature extraction and upsampling image restoration. The total loss function used in training the BFTransNet hybrid network for gastrointestinal region segmentation is... This includes a loss function for evaluating the recognition performance of the upsampled reconstructed image from the L-layer output: ； in, For the corresponding number The weights of the loss function for layer recognition performance evaluation. L represents the number of layers in the BFTransNet hybrid network used for upsampling image recovery. For the first The loss function used to evaluate the recognition performance of the upsampled reconstructed image output by the layer is: ； , They represent the items used to evaluate the first. The upsampling of the layer output recovers the Dice loss and cross-entropy loss for image recognition performance.

6. The gastrointestinal MRI image segmentation method based on BFTransNet hybrid network according to claim 5, characterized in that, Evaluation of the first Dice loss in image recognition performance of upsampling recovery of layer output Represented as: ， ， ； in, Indicates the first Upsampled restored image of layer output for the first layer Dice scores for identifying different types of gastrointestinal regions , To identify the number of species in the gastrointestinal region, For the corresponding number Preset Dice loss weights for each gastrointestinal region to identify different types. Indicates the first Upsampled restored image of layer output for the first layer Among the categories of gastrointestinal region recognition, the first... The segmentation mask value predicted and identified from each training sample. Indicates the first Among the categories of gastrointestinal region recognition, the first... The true segmentation mask value of each training sample. , Indicates the first The number of training samples for identifying species in each gastrointestinal region; Indicates the assessment of the first The weighted Dice score for the recognition performance of the upsampled restored image output of the layer; Evaluation of the first Cross-entropy loss of upsampled image recognition performance of layer output Represented as: ； in, For the corresponding number Each gastrointestinal region is identified by a pre-defined cross-entropy loss weight.

Images