A brain tumor MRI image segmentation method based on small sample learning of dual-branch network

Abstract

This invention proposes a brain tumor MRI image segmentation method based on dual-branch network small-sample learning. The method segments three specific tumor regions in brain tumor images, including the following steps: Step 1: Determine a small-sample brain tumor dataset, input brain tumor MRI image data, preprocess the medical image, and divide the dataset into a training set, a validation set, and a test set. These three datasets are further divided into a support set and a query set; Step 2: Construct a dual-branch network small-sample learning segmentation model and incorporate a dual-branch concurrent spatial and channel squeezing and excitation interaction module (DSCSE); Step 3: Construct a brain tumor difficult sample optimization module and incorporate it into the dual-branch network small-sample learning segmentation model to optimize network performance; Step 4: Use the obtained dual-branch network small-sample learning segmentation model to predict the test set data to obtain a segmented image. This invention solves the problem of poor segmentation results caused by a scarce dataset and improves segmentation accuracy.

Description

Technical Field

[0001] The present invention relates to a brain tumor MRI image segmentation method based on dual-branch network small sample learning, and belongs to the field of medical image processing. Background Art

[0002] Among existing medical imaging technologies, Magnetic Resonance Imaging (MRI) is the most widely used. MRI technology can provide researchers with detailed images of the brain. Its brain tumor MRI images include four modalities: T1-weighted, contrast-enhanced T1-weighted (T1C), T2-weighted (T2-weighted), and fluid-attenuated inversion recovery (FLAIR). In the brain tumor impact dataset, the areas that need to be segmented include three: the entire tumor area, the tumor core area, and the enhanced tumor area. Therefore, these multimodal MRI brain tumor images can more accurately present intracranial lesions, providing more assistance for tumor treatment and surgical guidance.

[0003] With the continuous advancement of medical image segmentation technology, more and more high-quality annotated datasets are needed to compare segmentation results with, and achieve increasingly precise segmentation results. However, such high-quality annotated datasets are often scarce, and obtaining them from medical experts is both time-consuming and expensive. Therefore, research on small datasets not only allows us to achieve good segmentation results even in the face of dataset scarcity, but also saves costs.

[0004] Few-Shot Learning (FSL) is a machine learning framework that enables pre-trained models to generalize to new categories of data (data that the pre-trained model has not seen during training). Few-Shot Learning consists of two important concepts: the support set and the query set. The support set consists of a small number of labeled examples for each new data category, which the pre-trained model uses to generalize to these new categories. The query set consists of examples from both new and old data categories, which the model needs to generalize to using prior knowledge and information obtained from the support set. The application of the support set and query set in few-shot learning has led to the new concept of N-Way K-Shot learning scheme. "N-Way" indicates that the pre-trained model needs to generalize to "N" new categories. A higher "N" value means a more difficult task. "K-Shot" defines the number of labeled examples available in the support set for each of the "N" categories.

[0005] While challenging, few-shot learning could have a significant impact on the field of medical image segmentation because it addresses the problem of learning from a small number of annotations, a fundamental issue stemming from an over-reliance on manual labeling by medical experts. Few-shot learning models extract discriminative representations of unseen classes from only a few annotated instances (often denoted as the support set) to make predictions about unannotated examples (often denoted as the query set). This suggests that few-shot learning holds great potential in the field of medical image segmentation. Summary of the Invention

[0006] The purpose of this invention is to provide a brain tumor MRI image segmentation method based on small sample learning using a dual-branch network. This invention effectively solves the problem of small sample segmentation, solves the problem of poor segmentation results caused by scarce datasets, and improves segmentation accuracy.

[0007] To achieve the above objectives, the present invention provides a brain tumor MRI image segmentation method based on dual-branch network small sample learning, comprising the following steps:

[0008] Step 1: Input medical image data. The datasets include Kaggle datasets and BraTs datasets. The Kaggle dataset is used as pre-trained network data input. The Kaggle dataset includes a training set and a validation set. The BraTs dataset is used as a test set to segment the target image. The training set, validation set, and test set are all divided into a support set and a query set.

[0009] Step 2: Construct a dual-branch network small-sample learning segmentation model. The dual-branch network is constructed based on the U-Net network, and then the small-sample learning network is constructed. The small-sample learning network is divided into a conditional branch, an interaction module, and a segmentation branch. The small-sample learning network is provided with image features by the conditional branch, and the segmentation features of the conditional branch are provided to the segmentation branch for segmenting new images through the dual-branch concurrent space and channel squeezing and excitation interaction module.

[0010] Step 3: Constructing a brain tumor difficult sample optimization module and adding it to the dual-branch network small sample learning segmentation model in step 2, and designing a difficult sample mixed loss function. First, a contrastive learning method is used to generate target and non-target classes from the images and labels of the support set to optimize the loss function. Second, the feature information generated by the support set is used to optimize the loss function of the difficult sample portion of the brain tumor image.

[0011] Step 4: Use the dual-branch network small sample learning segmentation model obtained in step 3 to predict the data of the test set to obtain the final segmentation prediction image.

[0012] As a further improvement of the present invention, the step 1 includes the following steps:

[0013] Step 11: Input and read data set;

[0014] Step 12: In the training network, the Kaggle dataset is used as a training set and is divided into a support set and a query set. The support set and the query set are input into the small sample model network for training. The BraTs dataset is used as a test set for the testing process and is divided into a support set and a query set. The support set inputs 5 randomly selected brain tumor images, and the query set inputs randomly extracted sample slices.

[0015] As a further improvement of the present invention, a small number of brain tumor MRI images in the Kaggle dataset are used as a training set. The training set is further divided into a support set and a query set, and the support set provides features for the query set. The BraTs dataset takes 5 sample slices. Since brain tumor segmentation is a multi-classification task, a total of 4 categories need to be segmented, which overall constitutes the small sample learning concept: 4-Way 5-Shot.

[0016] As a further improvement of the present invention, the step 2 comprises the following steps:

[0017] Step 21: Build a preliminary two-branch training network using U-Net as the baseline network.

[0018] Step 22: Construct a small sample learning framework based on the dual-branch training network architecture; the selected framework model is divided into three parts: a conditional branch, an interaction module and a segmentation branch, the conditional branch processes the support set image to simulate the appearance of the new semantic class in the image, and the conditional branch passes information to the segmentation branch; the segmentation branch segments the new semantic class in the new query image by utilizing the information of the conditional branch; the interaction module is used to interact information between the conditional branch and the segmentation branch so that the segmentation branch can better receive the image features of the conditional branch and segment the image more accurately.

[0019] The interaction module is a dual-branch concurrent space and channel squeezing and excitation interaction module, and its input is the feature vector output by the conditional branch convolution module and the feature vector output by the segmentation branch convolution module. The calculation formula of the interaction module is as follows:

[0020]

[0021] where u con is the feature vector output by the conditional branch convolution block, u seg To segment the feature vector output by the branch convolution block, the Q(·) function operation is divided into two steps. The first step is global average pooling, the second step is two 1*1*1 convolution operations, the P(·) function operation is a 1*1*1 convolution operation, and the σ(·) function is a sigmoid function. Finally, Represents the output of the module.

[0022] As a further improvement of the present invention, in step 2, the small sample learning framework is divided into the conditional branch and the segmentation branch, and the conditional branch and the segmentation branch both use the U-Net network as the basis to obtain feature vectors, and the feature vectors provided by the conditional branch assist the segmentation branch in segmentation through the interaction module; the conditional branch and the segmentation branch networks are symmetrically arranged, and both contain four encoders, decoders and a bottleneck block; and each decoding block and encoding block is composed of a convolution block, and the convolution blocks of the conditional branch and the segmentation branch are linked by the dual-branch concurrent space and channel squeezing and excitation interaction module; each layer of convolution block of downsampling and upsampling is mainly composed of two repeated 3*3 convolution blocks, followed by using the Relu activation function and a 2*2 maximum pooling operation with a step size of 2 between the downsampling convolution blocks, and this is repeated many times, and finally a 1*1 convolution operation is performed and the Sigmoid activation function is used to output the segmentation probability map of the network.

[0023] As a further improvement of the present invention, step 3 includes the following steps:

[0024] Step 31: Mark the enhanced tumor area in the brain tumor image dataset as a difficult sample segmentation area;

[0025] Step 32: Using the contrastive learning module, learning based on the difficult sample feature information can distinguish foreground and background features from the perspective of the overall semantic class. According to the characteristics of the brain tumor dataset, the enhanced tumor region is positioned as the foreground, because its segmentation data is poor compared to the other two categories, it is the difficult sample part. The other regions except the enhanced tumor region are identified as the background class. The calculation formula is as follows:

[0026]

[0027] The support set feature map and support set label in the conditional branch are respectively generated by label average pooling operation to generate feature information vectors, and the query set features are used to obtain the mean vector v through global average pooling. q ; In addition, the mean vector v q is regarded as a query vector, and the hard sample region in the support set is regarded as a positive key, with p s Indicated by; while the out-of-target prototype features are regarded as negative keys, represented by p nt Indicates that τ is a temperature hyperparameter, τ is a scalar, and K refers to the number of negative samples;

[0028] Step 33: Use the difficult sample area optimization module to extract useful information about enhanced tumors from the results of the conditional branch to optimize the enhanced tumor loss function of the query set in the segmentation branch. The cross entropy loss function is used for calculation, and its formula is as follows:

[0029]

[0030] where q i (x) is the label map of the difficult sample area of ​​brain tumor, p i (x) is the enhanced tumor feature extraction prediction map in the brain tumor image features, and N is the number of labels;

[0031] Step 34: The result L H Compared with the learning function result L cl Combined with the final query set segmentation result L to obtain the final difficult sample mixed loss function L CH , the formula is as follows:

[0032] L CH =L+λ1L cl +λ2L H

[0033] Where λ1 and λ2 are two weight coefficients, the range of λ1 and λ2 is [0-1], and L is the cross entropy loss function.

[0034] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0035] First, the present invention is based on a brain tumor MRI image segmentation method based on small sample learning of a dual-branch network. It uses a small sample learning model and proposes a new interaction module - a dual-branch concurrent spatial and channel squeezing and excitation interaction module. This module makes full use of the conditional branch to interact with the feature vectors obtained by each convolution block of the segmentation branch, so that the output image of the segmentation branch is optimized.

[0036] Secondly, the enhanced tumor area in the brain tumor image dataset of this invention is marked as a difficult sample segmentation area, and a difficult sample mixed loss function is designed. The contrastive learning technique is used to generate foreground and background classes from the labels of the support set, and the feature vectors generated by the support set are optimized. The loss function L is obtained using the information noise contrast estimation function. cl Then, we focus on the characteristics of brain tumor MRI datasets and make full use of the interaction between the image feature characteristics of the conditional branch and the feature of the segmentation branch in the small sample learning framework. On this basis, we focus on the optimization of the difficult sample area of ​​brain tumor images to enhance the tumor area, and obtain the loss function L H Finally, these two loss functions are optimized to obtain the final loss function L CH , to achieve the purpose of optimizing the output feature graph of the split branch query set. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flowchart of a brain tumor MRI image segmentation method based on small sample learning using a dual-branch network in an embodiment of the present invention.

[0038] Figure 2 2 is a schematic diagram illustrating a small sample learning network model based on a dual-branch network in an embodiment.

[0039] Figure 3 It is a schematic diagram of the dual-branch concurrent space and channel extrusion and excitation interaction module in the present invention. DETAILED DESCRIPTION

[0040] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] like Figure 1 and Figure 2 As shown, the present invention proposes a brain tumor MRI image segmentation method based on dual-branch network small sample learning, which is mainly aimed at brain tumor MRI image segmentation in medical images, and specifically includes the following steps:

[0042] Step 1: Input medical image data. Two datasets are available: a Kaggle dataset and a BraTs dataset. The Kaggle dataset serves as the pre-trained network input. The Kaggle dataset includes both a training set and a validation set, while the BraTs dataset serves as the test set for segmenting the target image. The training, validation, and test sets are all divided into support and query sets. A small amount of the Kaggle dataset is selected from the training network, divided into support and query sets, and then input into the small-shot model network for training. The BraTs dataset is used as the testing dataset, also divided into support and query sets. The BraTs dataset requires five sample slices. Since brain tumor segmentation is a multi-classification task requiring four classes, the overall small-shot learning concept is: 4-Way 5-Shot.

[0043] Step 2: If Figure 2 As shown, the network architecture of the present invention is a small-sample network model based on a two-branch network. The network structure consists of three building blocks: a conditional branch, an interaction module, and a segmentation branch. The conditional branch processes the support set image to simulate the appearance of the new semantic class in the image and passes this information to the interaction module. The segmentation branch uses information from the interaction module to segment the new semantic class in the new query image. The segmentation branch and the conditional branch have a symmetrical layout, namely, they both have four encoder and decoder blocks separated by a bottleneck block. This symmetrical layout facilitates strong interaction between the matching blocks through the interaction module. Each decoding block and encoding block is composed of a convolution block. Each convolution block in the downsampling layer is primarily composed of two repeated 3*3 convolution blocks, followed by a ReLU activation function and a 2*2 max pooling operation with a stride of 2 between the downsampling convolution blocks. With each downsampling layer in the encoding path, the number of feature channels doubles. The image size is halved and concatenated with the feature map after the previous layer's max pooling to form a new input, reducing the loss of feature information caused by the pooling operation. The decoding path's common block is preceded by an unpooling layer, followed by a corresponding unpooling layer for each subsequent encoding path. The pooling indices from the max pooling operation are stored and used in the corresponding unpooling stage of the decoder block to upsample the feature maps. The unpooling operation requires no parameters, which reduces model complexity and helps maintain spatial consistency for fine-grained segmentation. Finally, a 1x1 convolution followed by a sigmoid activation function is used to output the network's segmentation probability map. To obtain a more accurate segmentation map, the network model uses skip links to learn multi-scale information about the target.

[0044] Step 3: Figure 3This is the model diagram of the dual-branch concurrent spatial and channel squeeze and excitation interaction module (Dual-Branch Concurrent Spatial and Channel Squeeze and Channel Excitation, DSCSE). The interaction module consists of two parts: one is the spatial compression and channel excitation module, and the other is the compressed channel mode and excitation space. con is the feature map obtained by each convolution module of the conditional branch, u seg This is the feature map obtained by the convolution module of the segmentation branch. The spatial compression and channel excitation module first performs a global average pooling operation, then processes the information through two 1*1*1 convolutions to obtain the feature vector, which is then subjected to sigmoid normalization. The other part is the compression channel mode and excitation space module. This module directly performs a 1*1*1 convolution operation on the feature vector obtained by the conditional branch, and then performs a sigmoid normalization operation on it to obtain a spatial attention image. This is then applied to the feature vector obtained by the segmentation branch for information calibration. The calculation formula of this module is as follows:

[0045]

[0046] Step 4: Construct a brain tumor difficult sample optimization module and add it to the dual-branch network small sample learning segmentation model in step 2 to optimize the parts with poor segmentation results in brain tumor image segmentation. In the brain tumor dataset, three foreground areas and one background area need to be segmented. The three foreground areas are: the whole tumor area (WT), the tumor core area (TC), and the enhanced tumor area (ET). In the original image, the area where the enhanced tumor is located is also the smallest part of the entire image. Compared with the whole tumor area and the core tumor area, the enhanced tumor area can be regarded as the difficult sample part in the brain tumor image dataset. The segmentation of this area has been studied and improved. There are two improvement methods: one is the small sample contrast learning difficult sample optimization module. The other is the small sample support set feature information brain tumor image difficult sample optimization module.

[0047] Explore the use of feature information generated by support set images and their labels for guidance, which can constitute contrastive learning with query and support features. On the one hand, this will help enhance the discriminability of model learning features and improve segmentation performance; on the other hand, it will also make better use of medical image datasets in small sample segmentation. This module uses a difficult sample contrastive learning module to optimize the difficult sample loss function. Learning based on difficult sample feature information can distinguish foreground and background features from the perspective of the overall semantic class. According to the characteristics of the brain tumor dataset, the foreground of tumor region positioning will be enhanced, because its segmentation data is poorer than the other two categories, that is, the difficult sample part, and the other classes are identified as background classes. Specifically, the support set feature map and support set label in the conditional branch generate their feature information vectors through label average pooling operations respectively. The query feature is used to obtain the mean vector v through global average pooling. q In addition, this vector is regarded as the query vector, and the hard sample region in the support set is regarded as a positive key, with p s Indicated by; while the out-of-target prototype features are regarded as negative keys, represented by p nt Represents. Using contrastive learning, the commonly used information noise contrast estimation loss function is as follows:

[0048]

[0049] Combining the characteristics of brain tumor datasets and small sample learning, an optimization method for the loss function of enhanced tumor areas with poor segmentation is proposed. Due to the use of a small sample learning framework with a dual-branch structure, the query set is mainly segmented by the segmentation branch. At the same time, useful information can be extracted from the data in the support set to the segmentation branch through the interactive module to help segment it. Therefore, useful information about enhanced tumors can be extracted from the results of the conditional branch to optimize the enhanced tumor loss function of the query set in the segmentation branch. The main loss function used in this article is still the cross-entropy loss function. In the conditional branch, the input is a brain tumor image and its corresponding label, and the labels clearly identify each area of ​​the brain tumor. In this section, the feature map results of the support set are sampled through the corresponding areas marked with the labels to extract the required enhanced tumor areas, and then the cross-entropy loss function is calculated with the corresponding areas of the labels. The result is L H , whose formula is as follows:

[0050]

[0051] where q i (x) is the label map of the difficult sample area of ​​brain tumor, p i (x) is the enhanced tumor feature extraction prediction map in the brain tumor image feature, and N is the number of labels. Then this result L H Compared with the learning function result L clCombined with the final query set segmentation result L to obtain the final loss function L CH , the formula is as follows:

[0052] L CH =L+λ1L cl +λ2L H

[0053] Among them, λ1 and λ2 are two weight coefficients. Through experiments, it can be known that the range of λ1 and λ2 is [0-1].

[0054] Finally, the dual-branch network small sample learning segmentation model obtained in step 3 is used to predict the test set data to obtain the final segmentation prediction image.

[0055] In summary, the present invention discloses a brain tumor MRI image segmentation method based on small sample learning of a dual-branch network. It uses a small sample learning model and proposes a new interaction module - a dual-branch concurrent space and channel squeezing and excitation interaction module. The concurrent space and channel squeezing and excitation interaction module makes full use of the conditional branch and the feature vectors obtained by each convolution block of the segmentation branch to interact, so that the output image of the segmentation branch is optimized. At the same time, the enhanced tumor area with poor segmentation accuracy in the brain tumor image dataset is marked as a difficult sample segmentation area, and a difficult sample mixed loss function is proposed. The contrast learning technology is used to generate the foreground class and background class from the label of the support set, and the feature vector generated by the support set is optimized. The information noise contrast estimation function is used to obtain the loss function L cl Then, we focus on the characteristics of brain tumor MRI datasets and make full use of the interaction between the image feature characteristics of the conditional branch and the feature of the segmentation branch in the small sample learning framework. On this basis, we focus on the optimization of the difficult sample area of ​​brain tumor images to enhance the tumor area, and obtain the loss function L H Finally, these two loss functions are optimized to obtain the final loss function L CH The output feature map of the query set of the optimized segmentation branch is achieved, which improves the accuracy of the output feature image.

Claims

1. A brain tumor MRI image segmentation method based on small sample learning of a dual-branch network, characterized in that: The steps include: Step 1: Input medical image data. The datasets include Kaggle datasets and BraTs datasets. The Kaggle dataset is used as pre-trained network data input. The Kaggle dataset includes a training set and a validation set. The BraTs dataset is used as a test set to segment the target image. The training set, validation set, and test set are all divided into a support set and a query set. Step 2: Construct a dual-branch network small-sample learning segmentation model. The dual-branch network is constructed based on the U-Net network, and then the small-sample learning network is constructed. The small-sample learning network is divided into a conditional branch, an interaction module, and a segmentation branch. The small-sample learning network is provided with image features by the conditional branch, and the segmentation features of the conditional branch are provided to the segmentation branch for segmenting new images through the dual-branch concurrent space and channel squeezing and excitation interaction module. Step 3: Constructing a brain tumor difficult sample optimization module and adding it to the dual-branch network small sample learning segmentation model in step 2, and designing a difficult sample mixed loss function. First, a contrastive learning method is used to generate target and non-target classes from the images and labels of the support set to optimize the loss function. Second, the feature information generated by the support set is used to optimize the loss function of the difficult sample portion of the brain tumor image. Step 4: Use the dual-branch network small sample learning segmentation model obtained in step 3 to predict the data of the test set to obtain the final segmentation prediction image.

2. The brain tumor MRI image segmentation method based on dual-branch network small sample learning according to claim 1 is characterized in that: The step 1 comprises: Step 11: Input and read data set; Step 12: In the training network, the Kaggle dataset is used as a training set and is divided into a support set and a query set. The support set and the query set are input into the small sample model network for training. The BraTs dataset is used as a test set for the testing process and is divided into a support set and a query set. The support set inputs 5 randomly selected brain tumor images, and the query set inputs randomly extracted sample slices.

3. The brain tumor MRI image segmentation method based on dual-branch network small sample learning according to claim 2 is characterized by: A small number of brain tumor MRI images in the Kaggle dataset are used as a training set. The training set is further divided into a support set and a query set. The support set provides features for the query set. The BraTs dataset takes 5 sample slices. Since brain tumor segmentation is a multi-classification task, a total of 4 categories need to be segmented, which overall constitutes the small sample learning concept: 4-Way 5-Shot.

4. The brain tumor MRI image segmentation method based on dual-branch network small sample learning according to claim 1, characterized in that: The step 2 includes: Step 21: Use U-Net as the baseline network to build a preliminary two-branch training network; Step 22: Construct a small sample learning framework based on the dual-branch training network architecture; the selected framework model is divided into three parts: a conditional branch, an interaction module, and a segmentation branch; the conditional branch processes the support set image to simulate the appearance of the new semantic class in the image, and the conditional branch transmits information to the segmentation branch; the segmentation branch segments the new semantic class in the new query image by utilizing the information from the conditional branch; the interaction module is used to exchange information between the conditional branch and the segmentation branch so that the segmentation branch can better receive the image features of the conditional branch and segment the image more accurately; The interaction module is a dual-branch concurrent space and channel squeezing and excitation interaction module, and its input is the feature vector output by the conditional branch convolution module and the feature vector output by the segmentation branch convolution module. The calculation formula of the interaction module is as follows: where u con is the feature vector output by the conditional branch convolution block, u seg To segment the feature vector output by the branch convolution block, the Q(·) function operation is divided into two steps. The first step is global average pooling, the second step is two 1*1*1 convolution operations, the P(·) function operation is a 1*1*1 convolution operation, and the σ(·) function is a sigmoid function. Finally, Represents the output of the module.

5. The brain tumor MRI image segmentation method based on dual-branch network small sample learning according to claim 4 is characterized by: In step 2, the small sample learning framework is divided into the conditional branch and the segmentation branch. Both the conditional branch and the segmentation branch use the U-Net network as the basis to obtain feature vectors. The feature vectors provided by the conditional branch assist the segmentation branch in segmentation through the interaction module. The conditional branch and the segmentation branch network are symmetrically arranged, and both contain four encoders, decoders and a bottleneck block. Each decoding block and encoding block is composed of a convolution block, and the convolution blocks of the conditional branch and the segmentation branch are linked by the dual-branch concurrent space and channel squeezing and excitation interaction module. Each layer of convolution block of downsampling and upsampling is mainly composed of two repeated 3*3 convolution blocks, followed by a Relu activation function and a 2*2 maximum pooling operation with a step size of 2 between the downsampling convolution blocks. This is repeated many times, and finally a 1*1 convolution operation is performed and a Sigmoid activation function is used to output the segmentation probability map of the network.

6. The brain tumor MRI image segmentation method based on dual-branch network small sample learning according to claim 1, characterized in that: The step 3 specifically includes the following steps: Step 31: Mark the enhanced tumor area in the brain tumor image dataset as a difficult sample segmentation area; Step 32: Using the contrastive learning module, learning based on the difficult sample feature information can distinguish foreground and background features from the perspective of the overall semantic class. According to the characteristics of the brain tumor dataset, the enhanced tumor region is positioned as the foreground, because its segmentation data is poor compared to the other two categories, it is the difficult sample part. The other regions except the enhanced tumor region are identified as the background class. The calculation formula is as follows: The support set feature map and support set label in the conditional branch are respectively generated by label average pooling operation to generate feature information vectors, and the query set features are used to obtain the mean vector v through global average pooling. q ; In addition, the mean vector v q is regarded as a query vector, and the hard sample region in the support set is regarded as a positive key, with p s Indicated by; while the out-of-target prototype features are regarded as negative keys, represented by p nt Indicates that τ is a temperature hyperparameter, τ is a scalar, and K refers to the number of negative samples; Step 33: Use the difficult sample area optimization module to extract useful information about enhanced tumors from the results of the conditional branch to optimize the enhanced tumor loss function of the query set in the segmentation branch. The cross entropy loss function is used for calculation, and its formula is as follows: where q i (x) is the label map of the difficult sample area of ​​brain tumor, p i (x) is the enhanced tumor feature extraction prediction map in the brain tumor image features, and N is the number of labels; Step 34: The result L H Compared with the learning function result L cl Combined with the final query set segmentation result L to obtain the final difficult sample mixed loss function L CH , the formula is as follows: L CH =L+λ1L cl +λ2L H Where λ1 and λ2 are two weight coefficients, the range of λ1 and λ2 is [0-1], and L is the cross entropy loss function.

Images