A multi-organ segmentation method based on parallel deep U-shaped network and probability density map

Abstract

The application is a kind of multi-organ segmentation method based on parallel deep U-shaped network and probability density map, and the steps are as follows: step one: self-supervised pre-training of deep convolutional instance segmentation network encoder based on contrast learning on large-scale unlabeled image dataset; step two: based on organ probability density map and pre-training network weight, multi-organ preliminary segmentation of input image is carried out by using multi-scale parallel deep U-shaped instance segmentation network; step three: the preliminary segmentation result is improved by using multi-scale adaptive fusion module, and the fine multi-organ segmentation of the input image is obtained. The application can make full use of large-scale unlabeled data which is difficult to use in traditional supervised learning, and at the same time, the probability density map guide and multi-scale adaptive fusion module are introduced for multi-organ segmentation task, which ensures the accuracy and reliability of multi-organ segmentation result. The application can be closely combined with clinical diagnosis, radiotherapy and chemotherapy plan making and other related fields, and has broad market prospect and application value.

Description

Technical Field

[0001] This invention relates to a multi-organ segmentation method based on parallel deep U-shaped networks and probability density maps, belonging to the fields of medical image segmentation, pattern recognition, and computer vision. It mainly involves self-supervised learning, deep convolutional networks, and instance segmentation. It has broad application prospects in clinical diagnosis and treatment. Background Technology

[0002] Medical image segmentation refers to dividing an image into several regions based on the similarity or difference between regions within the image. The segmentation targets are typically organs, tissues, or cells. Compared to manual segmentation by doctors, automatic segmentation using computer vision algorithms offers advantages such as reduced time consumption and better consistency. Therefore, using computer vision algorithms for medical image segmentation can significantly reduce the workload of doctors and lower the human and time costs in clinical diagnosis and treatment.

[0003] In recent years, deep learning has developed rapidly and made remarkable progress in many fields. Numerous methods for image segmentation using deep networks have also emerged, quickly demonstrating performance advantages over traditional segmentation methods (thresholding, region growing algorithms, etc.). The fully convolutional network proposed by Long et al. pioneered the application of deep convolutional networks to image segmentation tasks. It uses convolution operations to obtain image depth features and then uses deconvolution to upsample the features, obtaining a class prediction for each pixel (see reference: J. Long, E. Shelhamer, T. Darrell, “Fully convolutional networks for semantic segmentation”, Proceedings of the IEEE conference on computer vision and pattern recognition, 3431-3440, 2015). Ronneberger et al. proposed U-Net based on fully convolutional networks, applying an encoder-decoder structure and using skip connections to improve the segmentation accuracy of detailed parts (see reference, Ronneberger, P. Fischer, T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation”, International Conference on Medical Image Computing and Computer-Assisted Intervention, 234-241, 2015). Due to its symmetrical structure and ingenious skip connection mechanism, U-Net has performed excellently in medical image segmentation tasks and has been widely used.Isensee et al. designed and proposed an adaptive medical image segmentation framework, nnU-Net, based on the U-Net network architecture. This framework can adaptively determine the network depth, data augmentation strategy, and data preprocessing and postprocessing methods according to the data to be segmented, and can achieve advanced segmentation accuracy in various medical image segmentation tasks (see reference, Isensee, Jaeger, Kohl, et al., nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods, 18(2):203-211, 2021). Luc et al. were the first to apply generative adversarial networks to image segmentation tasks, using segmentation networks such as fully convolutional networks as generators, and then constructing a discriminator for adversarial training to optimize the parameters of the segmentation network. (See reference, P. Luc, C. Couprie, S. Chintala, et al., “Semantic Segmentation using Adversarial Networks”, Preprint at https: / / arxiv.org / abs / 1611.08408, 2016). Xue et al. proposed SegAN, a generative adversarial network for medical image segmentation, and proposed a multi-scale L1 loss function to optimize the segmentation network, which improves the class imbalance problem in medical image segmentation (see reference, Yuan Xue, Tao Xu, Han Zhang, et al., “SegAN: Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentation”, Neuroinformatics, 16(3):383-392, 2018).

[0004] However, when applying the above algorithms to multi-organ segmentation tasks, the following problems still exist: Due to the high cost of obtaining expert annotations, labeled samples are relatively scarce in medical image segmentation, and supervised learning methods cannot utilize the large number of unlabeled samples. The limited amount of training data restricts the generalization performance of deep networks. The distribution location and range of the same organ in different human bodies have a certain degree of consistency and stability, but in 3D medical image segmentation, due to computational resource limitations, the entire image is usually cropped into independent image blocks for segmentation. In addition, the feature extraction of convolution operations has translation invariance, making it difficult to utilize the prior information of the organ's position in the overall image. Different organs in the human body vary in size and scale. Although some segmentation networks use feature map cross-connection to improve the segmentation effect of organs of different sizes, it is still difficult to guarantee accurate segmentation of organs at all scales. This invention proposes a multi-organ segmentation method based on parallel deep U-shaped networks and probability density maps. It uses the statistical probability density maps of corresponding organs to assist and guide the segmentation. Through contrastive learning and self-supervised pre-training, it makes full use of large-scale unlabeled data to improve the feature extraction capability of the encoder in the segmentation network. Furthermore, it employs a multi-scale adaptive fusion module to further improve the segmentation accuracy of organs of different sizes. Summary of the Invention

[0005] To address the aforementioned problems, the present invention aims to provide a multi-organ segmentation method based on a parallel deep U-shaped network and probability density maps. This method utilizes the statistical probability density maps of corresponding organs to assist and guide segmentation. Through contrastive learning and self-supervised pre-training, it fully leverages large-scale unlabeled data to enhance the feature extraction capability of the encoder in the segmentation network. Furthermore, a multi-scale adaptive fusion module is employed to further improve the segmentation accuracy for organs of different sizes. The overall technical solution of this invention is mainly reflected in the following three aspects:

[0006] 1) Construct a contrastive learning self-supervised pre-training framework and use a large-scale unlabeled dataset to pre-train the deep segmentation network encoder to improve its feature extraction capability;

[0007] 2) Obtain multi-organ probability density maps using expert annotations from the training set, and use them together with the image to be segmented as input to the U-shaped instance segmentation network. The prior information on organ location is used to assist and guide the segmentation process.

[0008] 3) Design and construct a multi-scale adaptive fusion module. During fusion, learn adaptive weights for different organs at different scales to achieve accurate segmentation of organs at different scales.

[0009] This invention relates to a multi-organ segmentation method based on parallel deep U-shaped networks and probability density maps. The specific steps of this method are as follows:

[0010] Step 1: Perform self-supervised pre-training of the deep convolutional instance segmentation network encoder based on contrastive learning on a large-scale unlabeled image dataset;

[0011] A deep convolutional encoder network is constructed based on a U-shaped instance segmentation network. Using a large-scale unlabeled image dataset, the deep convolutional encoder network is pre-trained within a pixel-propagation contrastive learning framework. The encoder network parameters are optimized based on the pixel-propagation loss function, and the pre-trained encoder network parameters are finally output. The specific process is as follows:

[0012] S11. Construct a pixel propagation contrastive learning framework. This framework contains two forward paths, each containing an independent segmentation network encoder and a feature projection layer. Path A also includes an additional pixel propagation module. The input to this framework is two image patches of the same image from different viewpoints. The two image patches are processed through path A and path B to obtain two feature maps, respectively. The output of the framework is the pixel propagation loss function value between the two feature maps.

[0013] S12. The unlabeled image used for self-supervised pre-training is randomly cropped twice to obtain two image patches from different viewpoints. After random data augmentation, these patches are used as input sample pairs. The segmentation network encoder is pre-trained according to the contrastive learning framework described in S11. Specifically, the encoder network parameters in the contrastive learning framework are optimized based on the pixel propagation loss between the two feature maps obtained from the two image patches through path A and path B. The encoder network parameters in path A are updated using the backpropagation gradient of the pixel propagation loss function, while the encoder network parameters in path B are updated using the exponential moving average. The pre-trained encoder network parameters are obtained after the pixel propagation loss function converges.

[0014] Step 2: Based on the organ probability density map and the pre-trained encoder network parameters, a multi-scale parallel U-shaped instance segmentation network is used to perform multi-scale preliminary segmentation of multiple organs on the input image.

[0015] Based on the multi-organ annotations of the labeled dataset, a 3D multi-organ probability density map is statistically constructed. A U-shaped instance segmentation network is constructed, using the probability density map as prior guiding information, along with the image to be segmented, as input to the U-shaped instance segmentation network. Manual annotations of the corresponding images by relevant domain experts are used as labels to train the U-shaped instance segmentation network. The U-shaped instance segmentation network encodes depth features into the network input through an encoder, and then decodes the depth features to obtain a preliminary segmentation of the input image. The parameters of the U-shaped instance segmentation network are optimized based on the Dice loss function and cross-entropy loss function between the segmentation results of the U-shaped instance segmentation network and the expert annotations. The input and annotations of the U-shaped instance segmentation network are resampled, and the above optimization process is performed at multiple scales. The final output is a multi-scale preliminary segmentation result of the image by a multi-scale parallel U-shaped instance segmentation network. The specific process is as follows:

[0016] S21. Based on the multi-organ annotations in the labeled dataset, construct a three-dimensional multi-organ probability density map. Specifically, the expert annotations in the dataset are registered and aligned according to the boundaries of the labeled organs. Then, each image is resampled to the standard size. Subsequently, the probability of all organs to be segmented appearing in the dataset is calculated on a voxel-by-voxel basis, and the calculated probabilities are normalized to obtain the probability density map of each organ.

[0017] S22. Construct a U-shaped instance segmentation network. This network comprises a deep convolutional encoder and a deep convolutional decoder. Its input consists of the image to be segmented and a multi-organ probability density map, and its output is a preliminary segmentation result of the input image. The deep convolutional encoder in the U-shaped instance segmentation network extracts multi-scale convolutional features from the input. As the number of layers in the U-shaped instance segmentation network increases, the spatial size of the output feature map decreases while the number of channels increases. Shallow features contain more local detail information, while deep features contain more global semantic information. The deep convolutional decoder in the U-shaped instance segmentation network fuses and decodes the multi-scale convolutional features extracted by the encoder to obtain the preliminary segmentation result of the input image.

[0018] S23. Using the multi-organ probability density map obtained in S21 and the images and expert annotation data in the training set, the U-shaped instance segmentation network constructed in S22 is trained and optimized. First, the parameters of the U-shaped instance segmentation network are initialized. The initialization parameters of the deep convolutional encoder are the pre-trained encoder network parameters obtained in step one, while the initialization parameters of the deep convolutional decoder are obtained through random initialization. Then, the segmentation loss function L between the segmentation result of the U-shaped instance segmentation network on the input image and the corresponding expert annotation is calculated. The segmentation loss function L is the sum of the Dice loss function and the cross-entropy loss function. The expression for the Dice loss function is: Where |X| and |Y| represent the number of elements in the expert annotation and segmentation results, and |X∩Y| represents the number of elements in their intersection. The expression for the cross-entropy loss function is: Where p(x) i ) and q(x i The symbols () represent expert annotations and corresponding segmentation results at a given location. A stochastic gradient descent optimizer is used to optimize the U-shaped instance segmentation network, with an initial learning rate of 10. -2 Furthermore, a momentum update and weight decay mechanism is introduced, and the segmentation loss function L is optimized by adjusting the weight values ​​of the U-shaped instance segmentation network through gradient backpropagation.

[0019] S24. Perform multi-scale resampling on the training set images, expert annotations, and multi-organ probability density maps. Repeat the above optimization process on the U-shaped instance segmentation network at multiple scales to finally obtain the multi-scale preliminary segmentation results of the input image by the multi-scale parallel U-shaped instance segmentation network.

[0020] Step 3: Improve the initial segmentation results using a multi-scale adaptive fusion module to obtain fine multi-organ segmentation of the input image.

[0021] A multi-scale adaptive fusion module is constructed, with its weight parameters being adaptive weights for each organ at different scales. The module takes the preliminary multi-scale segmentation results as input and uses the module's weight parameters to perform weighted fusion of the multi-scale inputs, obtaining the multi-scale fused segmentation result of the input image. The adaptive weight parameters for different scales are optimized and updated based on the segmentation loss function between multi-scale fused segmentation and expert annotation. Specifically, the expression for the multi-scale fusion operation is as follows: Where P i P represents the probability that a certain location in the image belongs to the i-th organ after fusion. ij w represents the probability that this position belongs to the i-th organ in the preliminary segmentation results at the j-th scale. ij Let w represent the adaptive weight parameters for the i-th organ at the j-th scale, and n represent all scales used in step two. The adaptive weight parameters w for different organs at multiple scales are based on the segmentation loss function L between multi-scale fusion segmentation and expert annotation. ij For optimization and updates, the segmentation loss function L is the sum of the Dice loss function and the cross-entropy loss function. The expression for the Dice loss function is: Where |X| and |Y| represent the number of elements in expert annotation and network segmentation, and |X∩Y| represents the number of elements in their intersection. The expression for the cross-entropy loss function is: Where p(x) i ) and q(x iThe symbols ) represent expert annotations at a given location and the corresponding network segmentation result. The initial learning rate is set to 10. -3 Furthermore, a momentum update and weight decay mechanism is introduced, and the segmentation loss function L is optimized by adjusting the multi-scale adaptive weights through gradient backpropagation. The final output is the fine segmentation result of the input image obtained after multi-scale preliminary segmentation and multi-scale adaptive fusion module.

[0022] The multi-organ segmentation method based on parallel deep U-shaped networks and probability density maps is as follows: Figure 1 As shown, the statistical probability density map of the corresponding organ is used to assist and guide the segmentation. By contrastive learning and self-supervised pre-training, the feature extraction capability of the encoder in the segmentation network is fully utilized to improve the feature extraction capability of the encoder. Furthermore, a multi-scale adaptive fusion module is used to further improve the segmentation accuracy of organs of different sizes, and finally, accurate multi-organ segmentation results are obtained.

[0023] The advantages and effects of this invention are as follows: This invention proposes a multi-organ segmentation method based on a parallel deep U-shaped network and probability density maps. Addressing the difficulty in utilizing prior information about organ location in medical image segmentation, it statistically constructs multi-organ probability density maps to assist and guide the U-shaped instance segmentation network. A pixel-propagation contrastive learning framework is applied to pre-train the network encoder, fully utilizing large-scale unlabeled image data. Simultaneously, a multi-scale adaptive fusion module is designed to effectively improve the detection accuracy of organs at different scales, achieving accurate segmentation of multiple organs. Compared to manual segmentation by doctors, this algorithm significantly reduces labor and time costs, possesses high segmentation accuracy, and can serve as an effective aid in clinical diagnosis and treatment, with broad market prospects and application value. Attached Figure Description

[0024] Figure 1 This is a flowchart of the overall process for a multi-organ segmentation method based on parallel deep U-shaped networks and probability density maps.

[0025] Figure 2 This is a diagram of the pixel propagation contrastive learning framework.

[0026] Figure 3 A schematic diagram of a U-shaped instance segmentation network.

[0027] Figure 4 This is a schematic diagram of a multi-scale adaptive fusion module.

[0028] Figures 5a-5d This is a two-dimensional visualization illustration of multi-organ segmentation in chest and abdominal CT images using the method of the present invention.

[0029] Figures 6a-6b This is a schematic diagram of the three-dimensional visualization of multi-organ segmentation in chest and abdominal CT images using the method of the present invention. Detailed Implementation

[0030] To better understand the technical solution of the present invention, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0031] This invention is a multi-organ segmentation method based on parallel deep U-shaped networks and probability density maps. Its overall process is as follows: Figure 1 As shown, the specific implementation details for each part are as follows:

[0032] Step 1: Perform self-supervised pre-training of the deep convolutional instance segmentation network encoder based on contrastive learning on a large-scale unlabeled image dataset;

[0033] S11. Construct a pixel propagation contrastive learning framework, such as Figure 2 The framework shown contains two forward paths, each containing a segmentation network encoder and a feature projection layer. The segmentation network encoder and feature projection layer have the same structure in both paths, but their parameters are independent. The segmentation network encoder has the same structure as the encoder part of the U-shaped instance segmentation network used in step two, while the feature projection layer consists of convolutional layers, normalization layers, and nonlinear layers. Path A also includes an additional pixel propagation module. The input to this framework is two image patches from the same image at different viewpoints. The two image patches are processed through path A and path B respectively to obtain two feature maps. The output of the framework is the pixel propagation loss function value between the two feature maps.

[0034] S12. The unlabeled image used for self-supervised pre-training is randomly cropped twice to obtain two image patches from different viewpoints, ensuring a certain overlap between the two image patches. The corresponding pixels in the overlapping area become positive pixel pairs. The two image patches are then used as input sample pairs after random data augmentation, including random Gaussian noise, random Gaussian blur, random brightness enhancement, and random contrast enhancement. The segmentation network encoder is pre-trained according to the contrastive learning framework described in S11. Specifically, the parameters of the segmentation network encoder in the contrastive learning framework are optimized based on the pixel propagation loss between the two feature maps obtained from the two image patches in the input sample pair through path A and path B, respectively. The expression for the pixel propagation loss is L. PixPro =-cos(y p1 x p2 ′)-cos(y p2 x p1y'), where p1 and p2 represent two pixels in a pair of positive pixels, and y and x' represent the feature maps output by pathway A and pathway B, respectively. During parameter optimization, the encoder network parameters in pathway A are updated using backpropagation of the loss function gradient, while those in pathway B are updated using exponential moving average. The pre-trained encoder network parameters are obtained after the pixel propagation loss function converges. The specific framework is as follows: Figure 2 As shown.

[0035] Step 2: Based on the organ probability density map and the pre-trained encoder network parameters, use the U-shaped instance segmentation network to perform preliminary multi-organ segmentation on the input image.

[0036] S21. Based on the multi-organ annotations in the labeled dataset, construct a 3D multi-organ probability density map. Specifically, register and align the expert annotations in the dataset according to the labeled organ boundaries. Then, resample each image to a standard size. Subsequently, calculate the probability of all organs to be segmented appearing in the dataset on a voxel-by-voxel basis, and normalize the obtained probabilities to obtain the probability density map of each organ. The expression for calculating the probability that a certain position m belongs to category n is: Where N represents the number of images in the training set, C km This indicates the category to which position m belongs in the expert annotation of the k-th image. if(·) is a logical judgment function. The function value is 1 when the proposition in the parentheses is true, and the function value is 0 otherwise.

[0037] S22. Construct a U-shaped instance segmentation network, the structure of which is shown in the diagram below. Figure 3 As shown, the U-shaped instance segmentation network comprises a deep convolutional encoder and a deep convolutional decoder. Both the encoder and decoder consist of the same number of sub-encoders and sub-decoders, which mainly include convolutional layers, normalization layers, and nonlinear layers. The input to the U-shaped instance segmentation network is the image to be segmented and a multi-organ probability density map; the output is the preliminary segmentation result of the input image. The deep convolutional encoder in the U-shaped instance segmentation network extracts multi-scale convolutional features from the input. As the number of layers in the U-shaped instance segmentation network increases, the spatial size of the output feature map decreases while the number of channels increases. Shallow features contain more local detail information, while deep features contain more global semantic information. The deep convolutional decoder in the U-shaped instance segmentation network fuses and decodes the convolutional features extracted by the encoder at different depths to obtain the preliminary segmentation result of the input image.

[0038] S23. Using the multi-organ probability density map obtained in S21 and the images and expert annotation data in the training set, the U-shaped instance segmentation network constructed in S22 is trained and optimized. First, the network parameters of the U-shaped segmentation network are initialized. The initialization parameters of the deep convolutional encoder are the pre-trained encoder network parameters obtained in step one, while the initialization parameters of the deep convolutional decoder are obtained through random initialization. Then, the segmentation loss function L between the segmentation result of the U-shaped instance segmentation network on the input image and the corresponding expert annotation is calculated. The segmentation loss function L is the sum of the Dice loss function and the cross-entropy loss function. The expression for the Dice loss function is: Where |X| and |Y| represent the number of elements in the expert annotation and segmentation results, and |X∩Y| represents the number of elements in their intersection. The expression for the cross-entropy loss function is: Where p(x) i ) and q(x i The symbols () represent expert annotations and corresponding segmentation results at a given location. A stochastic gradient descent optimizer is used to optimize the U-shaped instance segmentation network, with an initial learning rate of 10. -2 Furthermore, a momentum update and weight decay mechanism is introduced, and the segmentation loss function L is optimized by adjusting the weight values ​​of the U-shaped instance segmentation network through gradient backpropagation.

[0039] S24. Perform multi-scale resampling on the training set images, expert annotations, and multi-organ probability density maps. Repeat the above optimization process on the U-shaped instance segmentation network at multiple scales to finally obtain the multi-scale preliminary segmentation results of the input image by the multi-scale parallel U-shaped instance segmentation network.

[0040] Step 3: Improve the initial segmentation results using a multi-scale adaptive fusion module to obtain fine multi-organ segmentation of the input image.

[0041] The design and construction of a multi-scale adaptive fusion module is shown in the diagram below. Figure 4 As shown, its weight parameters are adaptive weights for different organs at different scales. The multi-scale fusion module takes the preliminary multi-scale segmentation results as input, and uses the module weight parameters to perform weighted fusion of the multi-scale inputs to obtain the multi-scale fused segmentation result of the input image. The adaptive weights at different scales are optimized and updated based on the segmentation loss function L between multi-scale fusion segmentation and expert annotation. Specifically, the expression for the multi-scale fusion operation is as follows: Where P i P represents the probability that a certain location in the image belongs to the i-th organ after fusion. ij w represents the probability that this position belongs to the i-th organ in the preliminary segmentation results at the j-th scale. ijLet w represent the adaptive weight parameters for the i-th organ at the j-th scale, and n represent all scales used in step two. The adaptive weight parameters w for different organs at multiple scales are based on the segmentation loss function L between multi-scale fusion segmentation and expert annotation. ij For optimization and updates, the segmentation loss function L is the sum of the Dice loss function and the cross-entropy loss function. The expression for the Dice loss function is: Where |X| and |Y| represent the number of elements in the expert annotation and segmentation results, and |X∩Y| represents the number of elements in their intersection. The expression for the cross-entropy loss function is: Where p(x) i ) and q(x i The symbols () represent expert annotations and segmentation results at a given location. The initial learning rate is set to 10. -3 Furthermore, momentum update and weight decay mechanisms are introduced, and the segmentation loss function L is optimized by adjusting multi-scale adaptive weight parameters through gradient backpropagation. The final output is the refined segmentation result of the input image obtained after multi-scale preliminary segmentation and learnable fusion module.

[0042] To visually demonstrate the effects of the present invention, Figures 5a-5d This invention demonstrates the two-dimensional visualization results of multi-organ segmentation on chest and abdominal CT image data, wherein... Figure 5a This is a CT scan of the chest and abdomen. Figure 5b This is the segmentation result of the liver in the image according to the present invention. Figure 5c This invention provides the segmentation results of the stomach in an image. Figure 5d This is the result of the spleen segmentation in the image according to the present invention. Figures 6a-6b This invention demonstrates the three-dimensional visualization results of multi-organ segmentation on chest and abdominal CT image data, wherein... Figure 6a Multi-organ expert annotation for images. Figure 6b This invention demonstrates the method for multi-organ segmentation in identical images. It shows that the method achieves accurate segmentation of multiple major organs in the chest and abdomen, while maintaining high consistency with expert annotation results. Using 2858 unlabeled images for pre-training, and randomly selecting 136 images from 170 labeled images as the training set and 34 as the test set, the method achieved an average Dice score of 85.748% on the multi-organ segmentation task. This invention addresses the difficulty of utilizing prior knowledge of organ location by introducing probability density maps for guidance, employing contrastive learning self-supervised pre-training to fully utilize the large-scale unlabeled dataset, and utilizing a multi-scale adaptive fusion module to improve segmentation performance for organs at different scales. Therefore, this invention can achieve accurate multi-organ segmentation in medical images. This invention can be closely integrated with applications in clinical diagnosis and radiotherapy / chemotherapy planning, possessing broad market prospects and application value.

Claims

1. A multi-organ segmentation method based on parallel deep U-shaped networks and probability density maps, characterized in that: The steps of this method are as follows: Step 1: Perform self-supervised pre-training of the U-shaped instance segmentation network encoder based on contrastive learning on a large-scale unlabeled image dataset: A deep convolutional encoder is constructed based on a U-shaped instance segmentation network. Using a large-scale unlabeled image dataset, the deep convolutional encoder is pre-trained in a self-supervised manner under the pixel propagation contrastive learning framework. The encoder parameters are optimized based on the pixel propagation loss function, and the pre-trained encoder parameters are finally output. Step 2: Based on the organ probability density map and pre-trained encoder parameters, a multi-scale parallel U-shaped instance segmentation network is used to perform preliminary multi-organ segmentation on the input image at multiple scales. Based on the multi-organ annotations of the labeled dataset, a 3D multi-organ probability density map is statistically constructed. This probability density map, along with the image to be segmented, is used as prior guiding information and input to a U-shaped instance segmentation network. The network is trained using pre-defined manual annotations from experts. The U-shaped instance segmentation network encodes depth features into its input through an encoder, and then decodes these features to obtain a preliminary segmentation of the input image. The network parameters are optimized based on the Dice loss function and cross-entropy loss function between the segmentation results and the expert annotations. The input and annotations of the U-shaped instance segmentation network are resampled and optimized at multiple scales. The final output is a multi-scale preliminary segmentation result of the image from a multi-scale parallel U-shaped instance segmentation network. Step 3: Improve the initial segmentation results using a multi-scale adaptive fusion module to obtain fine multi-organ segmentation of the input image: A multi-scale adaptive fusion module is constructed, with parameters consisting of adaptive weights for each organ at different scales. The module takes the preliminary multi-scale segmentation results as input and uses the module weights to perform weighted fusion of the multi-scale inputs, resulting in a multi-scale fused segmentation result for the input image. The adaptive weight parameters at different scales are optimized and updated based on the cross-entropy loss function and Dice loss function between the multi-scale fused segmentation and expert annotation. The final output is the refined segmentation result of the input image obtained from the preliminary multi-scale segmentation through the multi-scale adaptive fusion module. The specific process of step one is as follows: S11. Construct a pixel propagation contrastive learning framework. This framework contains two forward paths, each of which contains an independent segmentation network encoder and a feature projection layer. Path A also includes a pixel propagation module. The input of this framework is two image patches of the same image from different viewpoints. The two image patches are processed through path A and path B to obtain two feature maps, respectively. The output of the framework is the pixel propagation loss function value between the two feature maps. S12. The unlabeled image used for self-supervised pre-training is randomly cropped twice to obtain two image patches from different viewpoints. After random data augmentation, these patches are used as input sample pairs. The segmentation network encoder is pre-trained according to the contrastive learning framework in step S11. Specifically, the encoder parameters in the contrastive learning framework are optimized based on the pixel propagation loss between the two feature maps obtained from the two image patches through path A and path B. The encoder parameters in path A are updated using the backpropagation gradient of the pixel propagation loss function, and the encoder parameters in path B are updated using the exponential moving average. The pre-trained encoder parameters are obtained after the pixel propagation loss function converges.

2. The multi-organ segmentation method based on parallel deep U-shaped networks and probability density maps according to claim 1, characterized in that: The specific process of step two is as follows: S21. Based on the multi-organ annotations of the labeled dataset, construct a three-dimensional multi-organ probability density map. Specifically, the expert annotations in the dataset are registered and aligned according to the boundaries of the labeled organs. Then, each image is resampled to the standard size. Subsequently, the probability of all organs to be segmented appearing in the dataset is calculated on a voxel-by-voxel basis, and the calculated probabilities are normalized to obtain the probability density map of each organ. S22. Construct a U-shaped instance segmentation network. The U-shaped instance segmentation network includes a deep convolutional encoder and a deep convolutional decoder. The input is the image to be segmented and the probability density map of multiple organs. The output is the preliminary segmentation result of the input image. The deep convolutional encoder in the U-shaped instance segmentation network is used to extract multi-scale convolutional features from the input. As the number of layers in the U-shaped instance segmentation network increases, the spatial size of the output feature map decreases while the number of channels increases. Shallow features contain more local detail information while deep features contain more global semantic information. The deep convolutional decoder in the U-shaped instance segmentation network fuses and decodes the multi-scale convolutional features extracted by the encoder to obtain the preliminary segmentation result of the input image. S23. Using the multi-organ probability density map obtained in step S21 and the images and expert annotation data in the training set, train and optimize the U-shaped instance segmentation network constructed in step S22. First, initialize the parameters of the U-shaped instance segmentation network, where the initialization parameters of the deep convolutional encoder are the pre-trained encoder parameters obtained in step one, and the initialization parameters of the deep convolutional decoder are obtained through random initialization. Then, calculate the segmentation loss function between the segmentation result of the U-shaped instance segmentation network on the input image and the corresponding expert annotation. Segmentation loss function The sum of the Dice loss function and the cross-entropy loss function is given by the expression for the Dice loss function. ,in and This represents the number of elements in the expert annotation and segmentation results. Let represent the number of elements in the intersection of the two elements. The expression for the cross-entropy loss function is: ,in and These represent the expert annotations and corresponding segmentation results at a given location. The U-shaped instance segmentation network is optimized using a stochastic gradient descent optimizer, with an initial learning rate of 10. -2 Furthermore, a momentum update and weight decay mechanism is introduced, and the weight values ​​of the U-shaped instance segmentation network are adjusted for the segmentation loss function through gradient backpropagation. Optimize; S24. Perform multi-scale resampling on the training set images, expert annotations, and multi-organ probability density maps. Repeat the optimization process of step S23 on the U-shaped instance segmentation network at multiple scales to finally obtain the multi-scale preliminary segmentation results of the input image by the multi-scale parallel U-shaped instance segmentation network.

3. The multi-organ segmentation method based on parallel deep U-shaped networks and probability density maps according to claim 1, characterized in that: The specific process of step three is as follows: The expression for the fusion operation performed by the multi-scale fusion module is as follows: ,in This indicates that a certain position in the image belongs to the first position after the fusion operation. The probability of an organ Indicates the first In the preliminary segmentation results of the [number] scale, this position belongs to the [number]th [scale]. The probability of an organ Indicates the first The first scale Adaptive weight parameters for each organ This represents all scales used in step two; the segmentation loss function is based on multi-scale fusion segmentation and expert annotation. Adaptive weight parameters for different organs at multiple scales Optimize and update, segmentation loss function The sum of the Dice loss function and the cross-entropy loss function is given by the expression for the Dice loss function. ,in and This represents the number of elements in expert annotations and network segmentation. Let represent the number of elements in the intersection of the two elements. The expression for the cross-entropy loss function is: ,in and These represent expert annotations at a specific location and the corresponding network segmentation results; the initial learning rate is set to 10. -3 Furthermore, momentum update and weight decay mechanisms are introduced, and the segmentation loss function is adjusted using multi-scale adaptive weight parameters through gradient backpropagation. The process is optimized; the final output is the fine segmentation result of the input image obtained after multi-scale preliminary segmentation and multi-scale adaptive fusion module.

Images