Partial Supervision Abdominal CT Sequential Image Multi-Organ Automatic Segmentation Method and Device

Abstract

Partial supervised multi-organ automatic segmentation method and device for abdominal CT sequence images. The method includes: (1) obtaining a dataset of abdominal multi-organ CT sequence images to be segmented; (2) preprocessing the obtained dataset of abdominal multi-organ CT sequence images; (3) dividing the preprocessed dataset of abdominal multi-organ CT sequence images, which includes a training set, a validation set, and a test set; (4) constructing a deep learning model for abdominal multi-organ CT sequence image segmentation, and the deep learning model is a U-shaped network based on Swin Transformer technology; (5) training the deep learning model using the above training set and partial supervised loss, and saving the best model according to the above validation set, where the partial supervised loss is a linear combination of margin loss and exclusive loss; (6) using the above saved best model to predict the above test set to obtain segmentation results of multiple organs in the abdominal CT sequence images.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method for automatically segmenting multiple organs in abdominal CT sequence images with partial supervision, and an apparatus for automatically segmenting multiple organs in abdominal CT sequence images with partial supervision. Background Art

[0002] Multi-organ segmentation has been widely applied in clinical practice, including diagnostic intervention, treatment planning, and treatment implementation. In radiotherapy planning, this is a time-consuming task, and manual or semi-automatic tools are often used to delineate the organs at risk. Therefore, in order to improve the efficiency of organ segmentation, automatic segmentation methods such as statistical models, multi-atlas label fusion, and registration-free methods have been developed. Unfortunately, these methods may be affected by image deformation and inter-subject variability, and their success in clinical applications is limited.

[0003] Medical image segmentation methods based on deep learning have been widely used in the literature to classify each pixel / voxel of a given 2D / 3D medical image, and have significantly improved the performance of multi-organ automatic segmentation. A prominent convolutional neural network (CNN) model is U-Net, and its latest variant nnUNet, which learns multi-scale features through skip connections. Other frameworks for multi-organ segmentation include DenseVNet, OBELISK-Net, and organ attention networks. A large amount of subsequent work has focused on improving existing frameworks by discovering and representing mutual relationships based on canonical correlation analysis, especially by constructing and utilizing statistical atlases.

[0004] However, almost all current segmentation models rely on fully annotated data with strong supervision. Managing a large-scale fully annotated dataset is a challenging task, which is both expensive and time-consuming. The sample size of the current labeled datasets is usually low and mostly partially labeled, which is also a bottleneck in the field of multi-organ segmentation research. That is, a dataset labels some organs, but not all organs. This partially annotated dataset avoids the use of segmentation methods that require full supervision.

[0005] How to make full use of these partially annotated data to improve the accuracy and robustness of segmentation has become a practical problem to be solved. When the network model is powerful enough, a large amount of data usually means that it is more likely to represent the actual distribution of data in reality, thus bringing better overall performance. Inspired by this, the present invention jointly learns a multi-organ segmentation network for a single abdominal CT sequence image from these partially labeled datasets, and this learning does not introduce any additional computational overhead. Summary of the Invention

[0006] To overcome the deficiencies of the prior art, the present invention provides a method for automatically segmenting multiple organs in abdominal CT sequence images with partial supervision, which can solve the bottleneck problem of the prior art that requires a large number of fully labeled data sets for abdominal multi-organ segmentation, and improves the accuracy and robustness of abdominal multi-organ CT sequence image segmentation without introducing any additional computational overhead.

[0007] The technical solution of the present invention is as follows: This method for automatically segmenting multiple organs in abdominal CT sequence images with partial supervision includes the following steps:

[0008] (1) Obtain a data set of abdominal multi-organ CT sequence images to be segmented;

[0009] (2) Preprocess the obtained data set of abdominal multi-organ CT sequence images;

[0010] (3) Divide the preprocessed data set of abdominal multi-organ CT sequence images, and this data set includes a training set, a validation set, and a test set;

[0011] (4) Construct a deep learning model for segmenting abdominal multi-organ CT sequence images, and this deep learning model is a U-shaped network based on Swin Transformer technology;

[0012] (5) Use the above training set and partial supervision loss to train the above deep learning model, and save the best model according to the above validation set, and the partial supervision loss is a linear combination of margin loss and exclusive loss;

[0013] (6) Use the above saved best model to predict the above test set to obtain the segmentation results of multiple organs in the abdominal CT sequence images.

[0014] The present invention overcomes the bottleneck of requiring full annotation for segmenting abdominal multi-organ CT sequence images by jointly using a partially labeled data set, training a U-shaped network based on Swin Transformer technology using six benchmark data sets including the liver, left kidney, right kidney, spleen, and pancreas and partial supervision loss, and improves the accuracy and robustness of abdominal multi-organ CT sequence image segmentation without introducing any additional computational overhead.

[0015] The present invention also provides a device for automatically segmenting multiple organs in abdominal CT sequence images with partial supervision, which includes:

[0016] An acquisition module configured to acquire a data set of abdominal multi-organ CT sequence images to be segmented;

[0017] A preprocessing module configured to preprocess the acquired data set of abdominal multi-organ CT sequence images;

[0018] A partitioning module, configured to partition a preprocessed abdominal multi-organ CT sequence image dataset, which dataset includes a training set, a validation set, and a test set;

[0019] A construction module, configured to construct a deep learning model for abdominal multi-organ CT sequence image segmentation, which deep learning model is a U-shaped network based on Swin Transformer technology;

[0020] A training module, configured to train the above deep learning model using the above training set and a partial supervision loss, and save the best model according to the above validation set, where the partial supervision loss is a linear combination of a margin loss and an exclusive loss;

[0021] A prediction module, configured to use the best model saved above to predict the above test set, and obtain segmentation results of multiple organs in the abdominal CT sequence image. Description of the Drawings

[0022] Figure 1 is a flowchart of a multi-organ automatic segmentation method for abdominal CT sequence images with partial supervision according to the present invention.

[0023] Figure 2 is an architecture diagram of a deep learning model according to the present invention.

[0024] Figure 3 is a structural diagram of a Swin Transformer block according to the present invention.

[0025] Figure 4 is a structural diagram of a residual block according to the present invention. Detailed Embodiments

[0026] As Figure 1 shown, this multi-organ automatic segmentation method for abdominal CT sequence images with partial supervision includes the following steps:

[0027] (1) Obtain an abdominal multi-organ CT sequence image dataset to be segmented;

[0028] (2) Preprocess the obtained abdominal multi-organ CT sequence image dataset;

[0029] (3) Partition the preprocessed abdominal multi-organ CT sequence image dataset, which dataset includes a training set, a validation set, and a test set;

[0030] (4) Construct a deep learning model for abdominal multi-organ CT sequence image segmentation, which deep learning model is a U-shaped network based on Swin Transformer technology;

[0031] (5) Train the above deep learning model using the above training set and partial supervision loss, and save the best model according to the above validation set. The partial supervision loss is a linear combination of margin loss and exclusive loss;

[0032] (6) Use the best model saved above to predict the above test set to obtain the segmentation results of multiple organs in the abdominal CT sequence images.

[0033] By jointly using a partially labeled dataset, the present invention trains a U-shaped network based on the Swin Transformer technology using six benchmark datasets including the liver, left kidney, right kidney, spleen, and pancreas and partial supervision loss, overcomes the bottleneck that requires full annotation for segmenting abdominal multi-organ CT sequence images, and improves the accuracy and robustness of abdominal multi-organ CT sequence image segmentation without introducing any additional computational overhead.

[0034] Preferably, in the step (1), for the fully labeled dataset D: each abdominal CT image is labeled with the liver, left kidney, right kidney, spleen, and pancreas; the liver dataset D Liver : each abdominal CT image is labeled with the liver; the left kidney dataset DLKidney: each abdominal CT image is labeled with the left kidney; the right kidney dataset D RKidney : each abdominal CT image is labeled with the right kidney; the spleen dataset D Spleen : each abdominal CT image is labeled with the spleen; the pancreas dataset D Pancreas : each abdominal CT image is labeled with the pancreas.

[0035] Preferably, in the step (2), for D, all CT images are interpolated into isotropic voxel spacings of 1.5×1.5×2.0 mm 3 and the CT values are limited to the range of [-125, 275] HU using a soft tissue window; for D Liver , all CT images are interpolated into isotropic voxel spacings of 1.0×1.0×1.0 mm 3 and the CT values are limited to the range of [-21, 189] HU using a soft tissue window; for D LKidney , all CT images are interpolated into isotropic voxel spacings of 1.5×1.5×3.0 mm 3 and the CT values are limited to the range of [-200, 200] HU using a soft tissue window; for D RKidney , all CT images are interpolated into isotropic voxel spacings of 1.5×1.5×3.0 mm 3 and the CT values are limited to the range of [-200, 200] HU using a soft tissue window; for DS pleen, interpolate all CT images to an isotropic voxel spacing of 1.0×1.0×1.0mm 3 and limit the CT values within the range of [-125, 275] HU using a soft tissue window; for D Pancreas , interpolate all CT images to 1.0×1.0×1.0 mm 3 isotropic voxel spacing, and limit the CT values within the range of [-87, 199] HU using a soft tissue window.

[0036] Preferably, in step (3), the datasets D, D Liver , D LKidney , D RKidney , DS pleen and D Pancreas are randomly divided into two parts in a ratio of 4:1. The first part is further divided into a training set and a validation set using five-fold cross-validation, and the second part is directly used as a test set.

[0037] Preferably, in step (4), the deep learning model consists of an encoder based on Swin Transformer and a decoder based on CNN. The patch size used by the encoder is 2×2×2, the feature dimension is 2×2×2×1 = 8, and the embedding space P = 48 dimensions; the overall architecture of the encoder consists of four stages, each stage includes two Swin Transformer blocks. Between each stage, the resolution is reduced by a factor of two using a patch merging layer. Stage 1 includes a linear embedding layer and a Swin Transformer block with a number of tokens of . The patch merging layer groups patches with a resolution of 2×2×2 and concatenates them to form a 4P-dimensional feature embedding, and then uses a linear embedding layer to reduce the feature dimension to 2P. The same process continues in stages 2, 3, and 4, and the number of tokens in their Swin Transformer blocks are and respectively. The hierarchical representations of the encoder at different stages are used for multi-scale feature extraction of abdominal multi-organ CT sequence image segmentation.

[0038] Preferably, in step (4), the decoder consists of five consecutive transposed convolutional layers. The encoder is connected to the decoder at each resolution to create a U-shaped network for abdominal multi-organ CT sequence image segmentation; the model extracts the output sequence features of each stage i and reshapes them to a size of The features are that in the encoder, i ∈ {0, 1, 2, 3, 4} and in the bottleneck layer i = 5; then the features extracted in each stage are input into a residual block which consists of two post-instance normalization 3×3×3 convolutional layers. The features processed in each stage are upsampled using a transposed convolutional layer and concatenated with the features processed in the previous stage, and the concatenated features are input into the above residual block; the output of the decoder is concatenated with the features processed from the input volume data, and they are input into the above residual block, and then a 1×1×1 convolutional layer with a softmax function is used to calculate the final segmentation probability.

[0039] Preferably, in the step (5), for training a deep learning model for abdominal multi-organ CT sequence image segmentation, when dealing with partial labels, use margin cross-entropy loss, margin Dice loss, and their combination; also use exclusive cross-entropy loss and exclusive Dice loss; extract patches with a resolution of (96, 96, 96) from the preprocessed abdominal multi-organ CT sequence images as inputs, the maximum number of training epochs is 200. During training, each epoch iterates 250 batches, and each batch uses two patches. To ensure the stability of model training, at least one patch in each batch during training contains foreground voxels. The deep learning model is implemented using PyTorch, and the initial learning rate is 1×e -1 , when the loss reduction amount is less than 1×e in 10 consecutive epochs -3 , the learning rate decays by 20%.

[0040] Preferably, in the step (5), the deep learning model for abdominal multi-organ CT sequence image segmentation is trained in two stages to prevent instability caused by a large loss value at the beginning of training; in the first stage, only the fully annotated dataset D is used, and the goal is to minimize the conventional loss function using the Adam optimizer. The purpose of the first stage is to give the model an initial weight in multi-class segmentation to prevent a large loss value when applying the margin loss function; in the second stage, six datasets are used for joint training in each epoch. In each epoch, 500 patches are randomly selected from each training dataset, and the batch size is 2. According to the source of the CT sequence images, if the image comes from the fully annotated dataset D, the conventional loss is used; if the image comes from the partially annotated dataset DL iver , DL Kidney , DR Kidney、 DS pleen or DP ancreas, marginal loss and exclusive loss are used; in actual training, the first stage contains 50 epochs, and the second stage contains 150 epochs. Online evaluation is performed during the training process, that is, samples are randomly selected from the validation set for evaluation, and the checkpoint with the best online evaluation result is selected as the best model and saved to disk.

[0041] Preferably, in step (6), the CT sequence images in the test sets of the datasets D, DL iver , D LKidney , D RKidney、 D Spleen and D Pancreas are input into the saved best model for prediction; for D, the segmentation results of the liver, left kidney, right kidney, spleen, and pancreas in the abdominal CT sequence images are obtained simultaneously; for D Liver , the segmentation result of the liver in the abdominal CT sequence images is obtained; for D LKidney , the segmentation result of the left kidney in the abdominal CT sequence images is obtained; for D RKidney , the segmentation result of the right kidney in the abdominal CT sequence images is obtained; for D Spleen , the segmentation result of the spleen in the abdominal CT sequence images is obtained; for D Pancreas , the segmentation result of the pancreas in the abdominal CT sequence images is obtained.

[0042] Those of ordinary skill in the art can understand that all or part of the steps in implementing the above method embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. When the program is executed, it includes the steps of the above method embodiments, and the storage medium can be: ROM / RAM, magnetic disk, optical disk, memory card, etc. Therefore, corresponding to the method of the present invention, the present invention also simultaneously includes a multi-organ automatic segmentation device for abdominal CT sequence images with partial supervision, and this device is usually represented in the form of functional modules corresponding to the steps of the method. This device includes:

[0043] An acquisition module configured to acquire a dataset of abdominal multi-organ CT sequence images to be segmented;

[0044] A preprocessing module configured to preprocess the acquired dataset of abdominal multi-organ CT sequence images;

[0045] A partitioning module configured to partition the preprocessed dataset of abdominal multi-organ CT sequence images, and this dataset includes a training set, a validation set, and a test set;

[0046] A construction module configured to construct a deep learning model for abdominal multi-organ CT sequence image segmentation, and this deep learning model is a U-shaped network based on Swin Transformer technology;

[0047] A training module configured to train the above deep learning model using the above training set and partial supervision loss, and save the best model according to the above validation set, where the partial supervision loss is a linear combination of margin loss and exclusive loss;

[0048] A prediction module configured to use the above saved best model to predict the above test set and obtain segmentation results of multiple organs in the abdominal CT sequence images.

[0049] The content of the present invention will be described in more detail below.

[0050] The deep learning model consists of a Swin Transformer encoder that directly uses 3D patches and a CNN-based decoder. The input of the encoder is a volume data The dimension of a 3D token with a patch resolution of (H′, W′, D′) is H′×W′×D′×C. H, W, D and H′, W′, D′ are the heights, widths, and depths of the sub-volume data and patches respectively; C is the number of channels. The patch partition layer creates a 3D token sequence of size and projects it into a P-dimensional space through an embedding layer. To perform token interaction efficiently, the input volume data is divided into non-overlapping windows, and local self-attention is calculated within each region. Specifically, at the l-th layer, a window of size M×M×M is used to evenly divide a 3D token into windows. At the l+1-th layer, the partition window is shifted by voxels. The outputs of the encoder blocks at the l-th layer and the l+1-th layer are calculated as follows:

[0051]

[0052]

[0053]

[0054]

[0055] where W-MSA and SW-MSA represent regular partition and window partition multi-head self-attention modules respectively, and are the outputs of W-MSA and SW-MSA; LN and MLP represent layer normalization and multi-layer perceptron respectively. 3D cyclic shift is adopted for efficient batch calculation of the shifted windows. Further, the self-attention is calculated according to the following formula:

[0056]

[0057] Among them, Q, K, and V represent query, key, and value respectively, d is the size of the query and the key, and B represents the relative position deviation.

[0058] The partial supervision loss is calculated by the following formula:

[0059] L = λ1L m + λ2L e ,

[0060] where L m is the margin loss, and L e is the exclusive loss. λ1 and λ2 are hyperparameters, and λ1 = 0.3 and λ2 = 0.7 are set.

[0061] The above margin loss is calculated by the following formula:

[0062] L m = L mCE + L mDice ,

[0063] where L mCE is the margin cross-entropy loss, and L mDice is the margin Dice loss.

[0064] The above exclusive loss is calculated by the following formula:

[0065] L e = L ecE + L eDice ,

[0066] where L eCE is the exclusive cross-entropy loss, and L eDice is the exclusive Dice loss.

[0067] For a multi-class classification task with N labels, its label indices are set to Ω N = {C1, C2,..., C N}, and its data sample x (i.e., a voxel in the CT image) belongs to one of the N classes, called class C n , and this class is encoded as an N-dimensional one-hot encoding vector where y n = 1, and the others are 0. The multi-class classifier consists of a set of response functions {a n (x): n ∈ Ω N} that constitute the output of the segmentation network. From these response functions, the posterior classification probability is calculated through a softmax activation function:

[0068]

[0069] Suppose there are M classes (M < N) in a partially labeled dataset, and its corresponding label indices are set to Ω'M = {C′1, C′2,..., C′ M}. For each merged class label m ∈ Ω′ M , there is a corresponding subset which consists of the label indices in Ω N that can be merged into the same class m.

[0070] The above marginal cross - entropy loss and marginal Dice loss are calculated as follows:

[0071]

[0072]

[0073] where the class m ∈ Ω′ M is denoted as a one - hot encoded vector of M dimensions, where z m = 1 and the others are 0. q m is the marginal probability of class m:

[0074]

[0075] In multi - organ segmentation, the organ classes are mutually exclusive. The exclusive subset of class n is defined as E n , which contains all the label indices that are mutually exclusive with class n. The exclusive label information is encoded as an N - dimensional vector and

[0076] The above exclusive cross - entropy loss and exclusive Dice loss are calculated as follows:

[0077]

[0078]

[0079] where ∈ is introduced to avoid falling into - ∞, and ∈ = 1 is set.

[0080] The above are only the preferred embodiments of the present invention, and do not impose any formal limitations on the present invention. Any simple modifications, equivalent changes, and decorations made to the above embodiments based on the technical essence of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A method for automatic multi-organ segmentation of partially supervised abdominal CT sequence images, characterized in that: It includes the following steps: (1) Obtain the abdominal multi-organ CT sequence image dataset to be segmented; (2) Preprocess the obtained abdominal multi-organ CT sequence image dataset; (3) Divide the preprocessed abdominal multi-organ CT sequence image dataset, which includes a training set, a validation set, and a test set; (4) Construct a deep learning model for abdominal multi-organ CT sequence image segmentation, and this deep learning model is a U-shaped network based on the Swin Transformer technology; (5) Use the above training set and partial supervision loss to train the above deep learning model, and save the best model according to the above validation set, and the partial supervision loss is a linear combination of the margin loss and the exclusive loss; (6) Use the best model saved above to predict the above test set to obtain the segmentation results of multiple organs in the abdominal CT sequence images; In the step (4), the deep learning model consists of an encoder based on Swin Transformer and a decoder based on CNN. The patch size used by the encoder is 2×2×2, the feature dimension is 2×2×2×1 = 8, and the embedding space P = 48 dimensions. The overall architecture of the encoder consists of four stages, each stage includes two Swin Transformer blocks. Between each stage, the patch merging layer is used to reduce the resolution by two times. Stage 1 includes a linear embedding layer and a Swin Transformer block with the number of tokens being The patch merging layer groups the patches with a resolution of 2×2×2 and stitches them together to form a 4P-dimensional feature embedding, and then the linear embedding layer is used to reduce the feature dimension to 2P. The same process continues to be executed in Stages 2, 3, and 4, and the number of tokens in their Swin Transformer blocks are and The hierarchical representations of the encoder at different stages are used for multi-scale feature extraction of abdominal multi-organ CT sequence image segmentation. The decoder consists of five consecutive transposed convolution layers, and the encoder jumps to connect to the decoder at each resolution to create a U-shaped network for abdominal multi-organ CT sequence image segmentation; The model extracts the output sequence features of each stage i and reshapes them into features of size where i ∈ {0, 1, 2, 3, 4} in the encoder and i = 5 in the bottleneck layer; then the features extracted at each stage are input into a residual block consisting of two post-instance normalization 3×3×3 convolutional layers. The features processed at each stage are upsampled using a deconvolution layer and concatenated with the features processed at the previous stage, and the concatenated features are input into the above residual block; the output of the decoder is concatenated with the features processed from the input volume data and input into the above residual block, and then a 1×1×1 convolutional layer with a softmax function is used to calculate the final segmentation probability.

2. The method for automatically segmenting multiple organs from partially supervised abdominal CT sequence images according to claim 1, wherein: In the step (1), the fully labeled dataset D: for each abdominal CT image, the liver, left kidney, right kidney, spleen, and pancreas are labeled; the liver dataset D Liver : for each abdominal CT image, the liver is labeled; the left kidney dataset D LKidney : for each abdominal CT image, the left kidney is labeled; the right kidney dataset D RKidney : for each abdominal CT image, the right kidney is labeled; the spleen dataset D Spleen : for each abdominal CT image, the spleen is labeled; the pancreas dataset D Pancreas : for each abdominal CT image, the pancreas is labeled.

3. The method for automatically segmenting multiple organs from partially supervised abdominal CT sequence images according to claim 2, wherein: In step (2), for D, all CT images are interpolated into an isotropic voxel spacing of 1.5×1.5×2.0 mm 3 , and the CT values are limited to the range of [-125, 275] HU using a soft tissue window; for D Liver , all CT images are interpolated into an isotropic voxel spacing of 1.0×1.0×1.0 mm 3 , and the CT values are limited to the range of [-21, 189] HU using a soft tissue window; for D LKidney , all CT images are interpolated into an isotropic voxel spacing of 1.5×1.5×3.0 mm 3 , and the CT values are limited to the range of [-200, 200] HU using a soft tissue window; for D RKidney , all CT images are interpolated into an isotropic voxel spacing of 1.5×1.5×3.0 mm 3 , and the CT values are limited to the range of [-200, 200] HU using a soft tissue window; for D Spleen , all CT images are interpolated into an isotropic voxel spacing of 1.0×1.0×1.0 mm 3 , and the CT values are limited to the range of [-125, 275] HU using a soft tissue window; for D Pancreas , all CT images are interpolated into an isotropic voxel spacing of 1.0×1.0×1.0 mm 3 , and the CT values are limited to the range of [-87, 199] HU using a soft tissue window.

4. The method for automatically segmenting multiple organs from partially supervised abdominal CT sequence images according to claim 3, wherein: In the step (3), the data sets D, D Liver , D LKidney , D RKidney , D Spleen and D Pancreas are randomly divided into two parts in a ratio of 4:

1. The first part is further divided into a training set and a validation set using the five-fold cross-validation method, and the second part is directly used as a test set.

5. The method for automatically segmenting multiple organs in a partially supervised abdominal CT sequence image according to claim 1, wherein: In step (5), for training a deep learning model for abdominal multi-organ CT sequence image segmentation, when dealing with partial labels, use marginal cross-entropy loss, marginal Dice loss, and their combinations; also use exclusive cross-entropy loss and exclusive Dice loss; extract patches with a resolution of (96, 96, 96) from the preprocessed abdominal multi-organ CT sequence images as inputs, the maximum number of training epochs is 200, during training, each epoch iterates 250 batches, each batch uses two patches, in order to ensure the stability of model training, at least one patch in each batch during training contains foreground voxels, the deep learning model is implemented using PyTorch, and the initial learning rate is 1×e -1 , when the loss reduction amount is less than 1×e in 10 consecutive epochs -3 , the learning rate decays by 20%.

6. The method for automatically segmenting multiple organs in a partially supervised abdominal CT sequence image according to claim 1, wherein: In the step (5), the deep learning model for abdominal multi-organ CT sequence image segmentation is trained in two stages to prevent instability caused by large loss values at the beginning of training; In the first stage, only the fully annotated dataset D is used, and the goal is to minimize the conventional loss function using the Adam optimizer. The purpose of the first stage is to give the model an initial weight in multi-class segmentation to prevent large loss values when applying the margin loss function; in the second stage, six datasets are used for joint training in each epoch. In each epoch, 500 patches are randomly selected from each training dataset, and the batch size is 2. According to the source of the CT sequence images, if the image comes from the fully annotated dataset D, the conventional loss is used; if the image comes from the partially annotated datasets D Liver 、D LKidney 、D RKidney 、D Spleen or D Pancreas , the margin loss and the exclusive loss are used; In actual training, the first stage contains 50 epochs, the second stage contains 150 epochs, and online evaluation is performed during training, that is, samples are randomly selected from the validation set for evaluation, and the checkpoint with the best online evaluation result is selected as the best model and saved to disk.

7. The method for automatically segmenting multiple organs from partially supervised abdominal CT sequence images according to claim 1, wherein: In step (6), input the CT sequence images in the test sets of datasets D, D Liver , D LKidney , D RKidney , D Spleen and D Pancreas into the saved optimal model for prediction; for D, simultaneously obtain the segmentation results of the liver, left kidney, right kidney, spleen, and pancreas in the abdominal CT sequence images; for D Liver , obtain the segmentation result of the liver in the abdominal CT sequence images; for D LKidney , obtain the segmentation result of the left kidney in the abdominal CT sequence images; for D RKidney , obtain the segmentation result of the right kidney in the abdominal CT sequence images; for D Spleen , obtain the segmentation result of the spleen in the abdominal CT sequence images; for D Pancreas , obtain the segmentation result of the pancreas in the abdominal CT sequence images.

8. The multi-organ automatic segmentation device for abdominal CT sequence images with partial supervision, which executes the method according to claim 1, is characterized in that: It includes: An acquisition module configured to obtain the abdominal multi-organ CT sequence image dataset to be segmented; A preprocessing module configured to preprocess the obtained abdominal multi-organ CT sequence image dataset; A division module configured to divide the preprocessed abdominal multi-organ CT sequence image dataset, which includes a training set, a validation set, and a test set; A construction module configured to construct a deep learning model for abdominal multi-organ CT sequence image segmentation, and this deep learning model is a U-shaped network based on the Swin Transformer technology; A training module configured to use the above training set and partial supervision loss to train the above deep learning model, and save the best model according to the above validation set, and the partial supervision loss is a linear combination of the margin loss and the exclusive loss; A prediction module configured to use the best model saved above to predict the above test set to obtain the segmentation results of multiple organs in the abdominal CT sequence images.

Images