Multi-task learning improves small sample medical image segmentation method

Abstract

The present application relates to a multi-task learning method for improving small sample medical image segmentation, comprising: obtaining a medical image to be segmented, inputting the segmented medical image into a neural network model, and obtaining the segmented medical image; wherein the neural network model is obtained by training a training set, and the training set includes a support set and a query set; a 1-class 1-sample strategy is added as an additional classification task in the training process of the neural network model, the neural network model classifies the target of the image based on an FS-C module, introduces a classification loss, reduces the weight of easily classified samples, and assigns a segmentation label to the pixels in the image through a masking average pooling module to obtain a segmentation loss. The present application has high performance of deep neural networks under the condition of scarce training data.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and in particular to a method for improving small-sample medical image segmentation through multi-task learning. Background Technology

[0002] The development and widespread adoption of medical imaging equipment have led to the extensive use of medical images in clinical practice. Imaging technology plays a crucial role in disease diagnosis, surgical planning, prognostic assessment, and follow-up. Medical images are one of the most important sources of evidence for clinical analysis and medical intervention. Medical image segmentation, which extracts key information from specific tissue images, is a critical step in achieving medical image visualization. Segmented images are then provided to physicians for various tasks, including quantitative analysis of tissue volume, diagnosis, localization of pathological changes, delineation of anatomical structures, and treatment planning. Given the vast amount of information in medical images, manually delineating target regions is a time-consuming and laborious task, significantly increasing the burden on clinicians' daily work.

[0003] Medical image segmentation is one of the most important tasks in medical image processing. Medical image data is complex, high-dimensional, and often suffers from noise and artifacts. Therefore, automated medical image segmentation methods have become an urgent need. Traditional medical image segmentation methods require experts to manually label training data, which consumes a lot of time and manpower. In recent years, with the continuous development of deep learning technology, deep learning methods have gradually become the mainstream methods in the field of medical image segmentation.

[0004] However, deep learning methods also face challenges. Medical images often suffer from problems such as small data volume, class imbalance, and poor model interpretability. Furthermore, the model's generalization ability on unknown datasets is also a significant issue.

[0005] In summary, medical image segmentation is one of the most important tasks in medical image processing. The application of deep learning methods in medical image segmentation has greatly advanced the development of medical diagnosis and treatment. In the future, we can expect the emergence of more stable, efficient, and accurate medical image segmentation algorithms, which will further improve the level of medical image processing and make a greater contribution to human health. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides a multi-task learning method to improve the segmentation of small-sample medical images, aiming to enhance the performance of deep neural networks under conditions of scarce training data.

[0007] To achieve the above objectives, the present invention provides the following solution: Multi-task learning improves methods for segmenting small-sample medical images, including: A medical image to be segmented is obtained, and the medical image to be segmented is input into a neural network model to obtain a segmented medical image. The neural network model is trained using a training set, which includes a support set and a query set. During training, a 1-class-1-example strategy is added as an additional classification task. The neural network model performs target classification of the image based on the FS-C module and introduces a classification loss to reduce the weight of easily classified samples. The neural network model assigns segmentation labels to pixels in the image through a masked average pooling module and obtains the segmentation loss.

[0008] Optionally, training the neural network model includes: The support set and the support images in the query set are input into the BackBone module to obtain multi-scale features. The multi-scale features are then input into the FS-C module for target classification of the image to obtain the output vector. The output vector is used as a prefix feature for auxiliary segmentation, and the multi-scale feature is input into the occlusion average pooling module to assign segmentation labels to the pixels in the image, thereby obtaining the segmented medical image.

[0009] Optionally, the BackBone module includes: The first BackBone submodule is used to extract features from the support images in the support set to obtain the first multi-scale features. The second BackBone submodule is used to extract features from the query images in the query set to obtain second multi-scale features.

[0010] Optionally, the FS-C module includes: The decoder is used to take the multi-scale features as keys and values, preset the number of targets to be queried, interact the number of targets to be queried with the keys and values, obtain the features of the corresponding positions in the image, and identify and segment the corresponding targets. The FFN submodule is used to generate the class probability for each target.

[0011] Optionally, the decoder embeds a prefix hint: in, For the first converter encoder layer The output embedding, For the i-th converter encoder layer The output embedding, This is the prefix information output by the first layer. This is the first Transformer encoder layer. For input data, The initial prefix features are input to the first Transformer encoder layer. This is the prefix information for the output of the i-th layer. For the first A Transformer encoder layer, For the (i-1)th converter encoder layer The output embedding, This is the prefix information for the output of the (i-1)th layer. For the final output, This represents the output state of the last encoder layer.

[0012] Alternatively, the method for introducing classification loss is as follows: in, For classifying losses, The predicted probability for each category, For multi-hot encoding, For the number of categories, and These are adjustable hyperparameters used to adjust the weights of positive and negative classes, thereby handling imbalanced data.

[0013] Optionally, the occlusion average pooling module includes: The occlusion average pooling submodule is used to perform average pooling on pixel features of the same category in the multi-scale features to obtain prototype features. Based on the prototype features, the first multi-scale features and the first multi-scale features are used to calculate cosine similarity to measure the similarity of the two vectors in the direction and obtain the cosine similarity value. Based on the cosine similarity value, segmentation labels are assigned to pixels in the image and the segmentation loss is obtained.

[0014] Optionally, the method for obtaining the segmentation loss is as follows: in, To divide the loss, The height of the image. The width of the image. and To determine the true label for each pixel position in the query image, and The background and foreground probabilities are predicted.

[0015] Optionally, after obtaining the medical image to be segmented, the process includes: standardizing and cropping the medical image to be segmented to obtain a preprocessed segmented medical image.

[0016] The beneficial effects of this invention are as follows: Traditional few-shot segmentation methods primarily employ a single-class, single-sample training strategy, assuming that the classes in the query set and support set are consistent. However, single-class segmentation tasks can only distinguish between foreground and background, leading to prediction errors. For example, when the support set provides a mask for the spleen class, conventional models often predict other class masks, such as stomach or liver, which is inconsistent with the expected result. A completely accurate result should only predict the target class appearing in the provided support set. Therefore, in practical applications, the model is required to distinguish the class differences between the support set and the query set, and to make correct judgments when the support set and query set contain inconsistent class samples. To address this issue, this invention proposes a multi-task learning method to enhance few-shot medical image segmentation. By constructing a multi-label classification system within the few-shot learning process, it learns unseen classes in both the query set and support set. This ensemble approach not only improves the model's ability to identify novel classes but also maintains the practicality and efficiency of training. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram illustrating the multi-task learning method for improving small-sample medical image segmentation according to an embodiment of the present invention; Figure 2 This is a visual diagram illustrating the classification and prototype extraction operations in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] This invention focuses on few-shot learning in the field of medical image analysis, which has been widely applied in various tasks such as classification and segmentation. Current few-shot segmentation methods typically employ a strategy of training with one cue image per class, assuming that the class matches the query set and support set.

[0022] Considering the real-world situation, the trained model can segment any target that appears during the training phase. However, during the inference phase, when the image classes in the support set are inconsistent with the query set, the model can still distinguish the classes that appear in the query set. The desired outcome of this embodiment is that the model should be able to distinguish, and only be able to distinguish, the consistent categories that appear in the query set and the support set.

[0023] The classification task is integrated into the original few-shot segmentation, where the classification task can extract the desired category from the support set. The few-shot segmentation training paradigm then determines whether the category exists in the support set.

[0024] In addition, this model can not only distinguish which categories are in the support set, but also maintain a feasible few-shot segmentation training paradigm. In this case, this embodiment uses multi-label classification instead of multi-class classification. To achieve this, this embodiment proposes a novel learning scheme that learns categories not seen in the query and support by applying multi-label classification to few-shot learning. Multi-label classification can distinguish all categories initially contained in a support image.

[0025] like Figure 1 As shown, this invention discloses a multi-task learning method to improve small-sample medical image segmentation, including: acquiring a medical image to be segmented; inputting the segmented medical image into a neural network model to obtain the segmented medical image; wherein, the neural network model is trained using a training set, which includes a support set and a query set; during the training process, a one-class-one-example strategy is added to the neural network model as an additional classification task; the neural network model performs target classification and segmentation of the image based on the FS-C module, and introduces a classification loss to reduce the weight of easily classified samples; the neural network model assigns segmentation labels to pixels in the image through a masked average pooling module and obtains the segmentation loss, specifically: This embodiment begins with a setup for small-sample learning. Given a small set of task-specific resources... Dataset of supervised information and a with irrelevant auxiliary datasets The goal of few-shot learning is to build a dataset for the task. function The process of completing this task utilized... A small amount of supervisory information and The knowledge in the above definition is used to successfully map the input to the target. Irrelevant terms and The categories in the data are orthogonal, that is... In the context of few-shot learning, the function A mathematical representation of a model (usually a deep learning network) whose main purpose is to solve a specific task by learning a mapping from input to output. The function The input parameters typically include data from the dataset. The input parameters are images or other types of data samples. These input parameters are preprocessed (e.g., normalized, cropped) data to suit the model's processing. The output target, in medical image segmentation tasks, is typically a segmentation mask for the image, defining the category (e.g., background, specific organ, etc.) for each pixel in the image. Therefore, the function... The goal is to map the input image to its corresponding segmentation mask.

[0026] The process requires adding an additional classification task to the original few-shot learning task. Specifically, one-class-one-sample means that the support set for each training iteration contains only one sample of one class. The classification task is the function of the FS-C module. This embodiment assumes there is one support set. and a query set ,in Represents the original image. This represents a binary mask corresponding to a specific category, while q is a multi-hot vector containing the categories from the original image. Existing models only query the image during each training iteration. Optimization is performed under the constraints of the N-way K-shot scenario. This embodiment adopts the 1-class-1-example strategy used in all benchmarks. N-way K-shot learning is a typical few-shot learning setting, where: N classes represent the total number of classes the model needs to learn to distinguish during training; K examples represent the K training samples available for each class. In the specific case of 1-class-1-example, this means that only one class is considered in each training iteration, and there is only one example for each class. This extreme few-shot learning scenario particularly challenges the model's fast adaptation and generalization capabilities. Specifically: a set of classes is selected from a large dataset, and N classes are randomly selected in each training iteration. For each class, K samples are randomly selected as the support set. For the actual 1-class-1-example strategy, this means randomly selecting one class and one sample from the dataset.

[0027] Training the neural network model includes: inputting support images from the support set and query set into the BackBone module to obtain multi-scale features; inputting the multi-scale features into the FS-C module to classify the target in the image and obtain the output vector; using the output vector as prefix features for auxiliary segmentation; and inputting the multi-scale features into the occlusion average pooling module to assign segmentation labels to the pixels in the image and obtain the segmented medical image.

[0028] The BackBone module includes: a first BackBone submodule, used to extract features from the support images in the support set to obtain first multi-scale features; and a second BackBone submodule, used to extract features from the query images in the query set to obtain second multi-scale features.

[0029] like Figure 2 As shown, the FS-C module includes: a decoder, used to use multi-scale features as keys and values, preset the number of targets to be queried, interact with the number of targets to be queried with the keys and values ​​to obtain features at the corresponding locations in the image, and identify and segment the corresponding targets; and an FFN submodule, used to generate the class probability corresponding to each target, specifically: The entire FS-C module consists of two parts: a multi-level transformer decoder and a shared feedforward network (FFN). The multi-level transformer decoder uses flattened features F as keys and values, along with learnable parameters, and Q as the number of classes. Furthermore, the feedforward network (FFN) is used to compute the predicted logistic value for each class, which can be expressed as follows: FFN (attention) Q , F , F ) in This represents the confidence level for each category. In this study, the attention mechanism is implemented in accordance with the standard transformer decoder, while the FFN contains only a simple linear layer. The process can be roughly divided into the following steps: 1. Input Image Processing: The input image is first processed by a convolutional neural network (backbone), specifically a pre-trained ResNet. This backbone network transforms the image into a spatial feature map. 2. Spatial Feature Extraction: Spatial features are extracted by the backbone and represented at multiple scales. These features serve as input to the Transformer decoder. 3. Transformer Decoder: The decoder's input includes flattened features F, acting as both key and value inputs. Additionally, there is a predefined number of queries corresponding to the number of targets to be detected and classified (in this example, liver, right kidney, left kidney, and spleen). The decoder's role is to find features at corresponding locations in the image through the interaction of the queries with the key and value, thereby identifying and segmenting the corresponding organs.

[0030] The Transformer Encoder is the basic network structure used for feature extraction. 4. FFN (Feedforward Network) Output: The decoder's output is fed into a feedforward network (FFN), which generates the class probability (Prob) for each query. The FFN's confidence level reflects the likelihood that the target corresponding to each query exists in the image.

[0031] Prefix prompts are embedded into the encoder of the transformer, and each prompt is a d-dimensional vector learned from multi-task classification. The prefix prompt is represented as... Therefore, the encoder output is as follows: in, For the first converter encoder layer The output embedding, For the i-th converter encoder layer The output embedding, This is the prefix information output by the first layer. This is the first Transformer encoder layer. For input data, The initial prefix features are input to the first Transformer encoder layer. This is the prefix information for the output of the i-th layer. This is the first Transformer encoder layer. For the (i-1)th converter encoder layer The output embedding, This is the prefix information for the output of the (i-1)th layer. For the final output, This is the output state of the last encoder layer. The length of the input is the number of categories plus a fixed number of patches. In this implementation, the transformer encoder is consistent with the standard transformer encoder. Prefix prompting is an effective optimization method to improve the performance of deep learning models in specific tasks, especially in transfer learning and few-shot learning environments. This technique is achieved by adding extra information to the model's input, typically by embedding it into the model's encoder. In this specific scenario, prefix prompting is used to enhance the capabilities of transformer models (such as Transformers), especially when handling multi-task classification tasks. The benefits of prefix prompting include: adding pre-learned contextual information, helping the model quickly understand and adapt to new tasks; changing the internal information flow and feature extraction process of the model, helping the model more accurately locate key information in the input data that is relevant to the current task; reducing the amount of data and iterations required to train the model for a specific task; and improving the model's generalization ability on unseen data or slightly different task settings.

[0032] This embodiment of multi-label classification does not require the introduction of a new loss function due to the cross-attention mechanism built into the transformer decoder. This embodiment uses an asymmetric focal loss designed to adjust the loss function, reducing the weight of easily classified samples, thereby allowing the model to focus more on difficult-to-classify or misclassified samples. It introduces different weights or different adjustment factors for positive and negative samples. Different γ values ​​are used for positive and negative samples. The loss can be defined by the following formula: in, For classifying losses, The predicted probability for each category, For multi-hot encoding, For the number of categories, and These are adjustable hyperparameters used to adjust the weights of positive and negative classes, thereby handling imbalanced data. It is the predicted probability for each category. It is a multi-hot encoding. The final loss is the average of the losses of all samples in the support set. In the experiments of this embodiment, γ+=0 and γ-=-1 are the default values, and the segmentation loss of this embodiment does not require an additional new loss function.

[0033] The occlusion average pooling module includes: an occlusion average pooling submodule, used to perform average pooling on pixel features of the same category in multi-scale features to obtain prototype features; based on the prototype features, cosine similarity is calculated between the first multi-scale features and the first multi-scale features to measure the similarity of the two vectors in direction, obtaining the cosine similarity value; based on the cosine similarity value, segmentation labels are assigned to pixels in the image, and segmentation loss is obtained, specifically: Within the framework of few-shot segmentation learning, pixel similarity computation is a crucial step in guiding the model to segment the query image based on given support images (or labeled images). This is achieved by effectively incorporating pixel similarities between features from the support and query images. Specifically, the model uses Masked Average Pooling (MAP) to transform features from the support images into prototypes, then computes cosine similarity to predict the mask, thus determining the loss for segmentation. The predicted mask is the segmentation result, which is the segmented image. The predicted mask is the predicted segmentation result. The loss can be calculated by comparing it with the real mask.

[0034] First, the model extracts features from both the support and query images using a series of convolutional layers. These features contain key visual information about the images, such as edges and textures. For the support image, MAP (Mapping Assignment) averages all pixel features belonging to the same category (e.g., organ or lesion) to obtain a prototype. This prototype represents the average features of that category, i.e., the prototype features, and will be used as a benchmark for subsequent steps. Cosine similarity is calculated between each pixel feature of the query image and the prototype features of the support image. Cosine similarity is a measure of the similarity between two vectors in a direction, ranging from -1 to 1, where 1 represents exactly the same direction and -1 represents exactly opposite directions. For similarity mapping, the model assigns a segmentation label to each pixel in the query image based on the calculated cosine similarity. Specifically, if a pixel's features have a high cosine similarity to the prototype of a certain category, the pixel will be labeled as belonging to that category. For loss calculation, during training, the segmentation results are compared with the actual ground truth to calculate the segmentation loss. Commonly used loss functions include cross-entropy loss or Dice loss. This loss is used to guide model training and optimize its segmentation performance.

[0035] Here, H and W are the height and width of the image, and the loss function is calculated by summing all the pixels and then dividing by the total number of pixels. To obtain the average loss, and These are the true labels for each pixel position (x, y) for the query image q. and These are the predicted background and foreground probabilities.

[0036] Therefore, this embodiment only needs to calculate the binary cross-entropy loss between the ground truth and the prediction mask. In summary, the final loss function of this embodiment is: Indicates the loss from partitioning. Represents classification loss, Represents the matching regularization loss of the prototype: The prototype matching regularization loss is a segmentation loss that uses the query image as the support set and the support images as the labels.

[0037] Matching regularization, PAR or reg, can be intuitively explained as follows: if the model can predict a good segmentation mask for the query image using prototypes extracted from the support set, then the prototypes learned from the query set based on the predicted mask should be able to segment the support images well. Therefore, PAR / reg encourages the resulting segmentation model to learn segmentation of the support images in the opposite direction, using the query image and the predicted mask as a new support set.

[0038] This embodiment discloses a multi-task learning method for improving small-sample medical image segmentation, including the following steps: Dataset and Evaluation Metrics: This embodiment evaluates its architecture on the CHAOST dataset, publicly available at the ISBI 2019 Joint Healthy Abdominal Organ Segmentation Challenge. This dataset consists of 20 3D T2-SPIR MRI scans, with an average of 36 slices per scan. The publicly available dataset was used to test and compare different segmentation methods under the same benchmark conditions. The dataset includes four categories: liver, left kidney, right kidney, and spleen. Evaluation metrics not only quantify the model's performance, ensuring the reliability and comparability of the results, but also help in understanding the model's effectiveness and weaknesses on specific tasks, thereby guiding future optimization and improvement.

[0039] This embodiment uses the widely used Dice similarity coefficient (DSC) as the segmentation evaluation index, which is defined as follows: , in, This indicates accurate labeling. This represents the predicted mask. This metric evaluates the predicted mask. and real labeling The overlap ratio between them.

[0040] Experimental parameter settings and details: In the experiment of this embodiment, the pseudo-label generation method of superpixels and supervoxels was adopted.

[0041] The model was trained using a self-supervised method and evaluated using a five-fold cross-validation method, where the support image set for each subset came from one patient, while the remaining patients were treated as objects in the query object set.

[0042] This embodiment implements the proposed method on each subset using PyTorch on an NVIDIA V100 GPU.

[0043] Specifically, for data preprocessing, this embodiment truncates the brightness endpoints (first 0.5%) of the histogram to improve the misalignment problem, resamples image slices to produce the same resolution as before, and crops each slice to a size of 256x256.

[0044] Furthermore, to accommodate the backbone network, each slice is copied three times along the channel dimension. As the image passes through the backbone network, the primary process is feature extraction. The backbone network is typically a pre-trained convolutional neural network, such as ResNet, VGG, or DenseNet.

[0045] This embodiment uses stochastic gradient descent (SGD) with 0.9 momentum to optimize the loss function, sets the learning rate to 1e-3, and employs the MultiStepLR scheduler with a decay rate of 0.98 every k epochs.

[0046] The model can be trained up to 70k iterations. Since a self-supervised method is used for training, this embodiment will not discard image slices containing the target category.

[0047] This embodiment provides an overview of its architecture and process. The method in this embodiment first uses a ResNet-100 network as the backbone to extract features from the original image pairs, then flattens them, and passes the features to the FS-C model and the Transformer encoder, respectively.

[0048] Each output feature of the FS-C model is used to predict the category and calculate the classification loss, while each output feature of the Transformer encoder is used to predict the segmentation mask and calculate the segmentation loss.

[0049] The model employs a supervoxel image preprocessing method and is trained using self-supervised, five-fold cross-validation, where support images are obtained from one patient per fold, while the remaining patients are used as query objects. The proposed method is implemented using PyTorch on an NVIDIA V100 GPU. Specifically, for data preprocessing, this embodiment removes the brightest parts of the histogram (up to 0.5%) to improve the detuning problem, resamples image slices to maintain the same resolution as before, and crops each slice to 256x256.

[0050] Furthermore, to accommodate the backbone network, each slice is copied three times along the channel dimension. This embodiment uses stochastic gradient descent (SGD) for optimization, with a momentum of 0.9, a learning rate of 1e-3, and a MultiStepLR scheduler with a decay rate of 0.98 every k epochs. The model is trained for a maximum of 70k iterations. Due to the use of self-supervised training, this embodiment does not discard image slices containing the target category.

[0051] This embodiment attempts two widely used inference protocols in this study. Protocol 1 is the standard testing paradigm. It filters out slices that do not match the corresponding ground truth labels and does not require segmentation before extracting the query image, meaning that weak query labels are not needed during inference. Protocol 2 relaxes the assumption that query image volume slices must be filtered to avoid the need for weak query labels. It extracts a slice from the supporting foreground volume and segments the entire query volume with the help of the slice. This protocol is more general and applicable to real-world situations.

[0052] Regarding the ablation experiment: This embodiment further conducted an ablation study to verify the effectiveness of different components of the model. The results of replacing the tokens in the classification with random learnable variables are referred to as "learnable tokens". Furthermore, the classification loss is 0, labeled "w / o prompts".

[0053] The above embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A multi-task learning method to improve small-sample medical image segmentation, characterized in that, include: A medical image to be segmented is obtained, and the medical image to be segmented is input into a neural network model to obtain a segmented medical image. The neural network model is trained using a training set, which includes a support set and a query set. During training, a 1-class-1-example strategy is added as an additional classification task. The neural network model performs target classification of the image based on the FS-C module and introduces a classification loss to reduce the weight of easily classified samples. The neural network model assigns segmentation labels to pixels in the image through a masked average pooling module and obtains the segmentation loss.

2. The multi-task learning method for improving small-sample medical image segmentation according to claim 1, characterized in that, Training the neural network model includes: The support set and the support images in the query set are input into the BackBone module to obtain multi-scale features. The multi-scale features are then input into the FS-C module for target classification of the image to obtain the output vector. The output vector is used as a prefix feature for auxiliary segmentation, and the multi-scale feature is input into the occlusion average pooling module to assign segmentation labels to the pixels in the image, thereby obtaining the segmented medical image.

3. The multi-task learning method for improving small-sample medical image segmentation according to claim 2, characterized in that, The BackBone module includes: The first BackBone submodule is used to extract features from the support images in the support set to obtain the first multi-scale features. The second BackBone submodule is used to extract features from the query images in the query set to obtain second multi-scale features.

4. The multi-task learning method for improving small-sample medical image segmentation according to claim 2, characterized in that, The FS-C module includes: The decoder is used to take the multi-scale features as keys and values, preset the number of targets to be queried, interact the number of targets to be queried with the keys and values, obtain the features of the corresponding positions in the image, and identify and segment the corresponding targets. The FFN submodule is used to generate the class probability for each target.

5. The multi-task learning method for improving small-sample medical image segmentation according to claim 4, characterized in that, The decoder embedding prefix hint: in, For the first converter encoder layer The output embedding, For the i-th converter encoder layer The output embedding, This is the prefix information output by the first layer. This is the first Transformer encoder layer. For input data, The initial prefix features are input to the first Transformer encoder layer. This is the prefix information for the output of the i-th layer. For the first A Transformer encoder layer, For the (i-1)th converter encoder layer The output embedding, This is the prefix information for the output of the (i-1)th layer. For the final output, This represents the output state of the last encoder layer.

6. The multi-task learning method for improving small-sample medical image segmentation according to claim 1, characterized in that, The method for introducing classification loss is as follows: in, For classifying losses, The predicted probability for each category, For multi-hot encoding, For the number of categories, and These are adjustable hyperparameters used to adjust the weights of positive and negative classes, thereby handling imbalanced data.

7. The multi-task learning method for improving small-sample medical image segmentation according to claim 3, characterized in that, The masking average pooling module includes: The occlusion average pooling submodule is used to perform average pooling on pixel features of the same category in the multi-scale features to obtain prototype features. Based on the prototype features, the first multi-scale features and the first multi-scale features are used to calculate cosine similarity to measure the similarity of the two vectors in the direction and obtain the cosine similarity value. Based on the cosine similarity value, segmentation labels are assigned to pixels in the image and the segmentation loss is obtained.

8. The multi-task learning method for improving small-sample medical image segmentation according to claim 1, characterized in that, The method for obtaining the segmentation loss is as follows: in, To divide the loss, The height of the image. The width of the image. and To determine the true label for each pixel position in the query image, and The background and foreground probabilities are predicted.

9. The multi-task learning method for improving small-sample medical image segmentation according to claim 1, characterized in that, After obtaining the medical image to be segmented, the process includes: standardizing and cropping the medical image to be segmented to obtain a preprocessed segmented medical image.

Images