Medical image segmentation device based on generative adversarial network and multi-scale feature fusion

Abstract

The present invention discloses a medical image segmentation device based on a generative adversarial network and multi-scale feature fusion, comprising: the input of the segmentation framework is the original 3D CT image, which is preprocessed to obtain a standardized image; the liver is segmented by using a discriminator in a feature map-based synthetic generative adversarial network after training, and a predicted probability map is output; the value of each pixel in the probability map represents the probability that the pixel belongs to the liver, and more information is learned through adversarial learning between the generator and the discriminator; the liver ROI is automatically extracted: the liver 3D segmentation result and the standardized image are dot-multiplied, other non-related organs are shielded, the minimum circumscribed cuboid of the liver area is calculated and clipped, and livers of different sizes are resampled to the same size; the liver ROI is used as input, and a trained three-channel cascade network based on an improved V-Net is used to fuse multi-scale features, expand the receptive field, and process the differences in the position, shape and size of the target area in different data and the fuzzy boundary of the lesion area, so as to finally obtain the tumor segmentation result.

Description

Technical Field

[0001] The present invention relates to the field of computer vision of machine learning, and in particular to a medical image segmentation device based on a generative adversarial network and multi-scale feature fusion. Background Art

[0002] With the development of computer technology and biomedicine, medical imaging equipment has gradually become popular, and a variety of medical imaging technologies have been widely used in clinical practice. For example: magnetic resonance imaging (MRI), computed tomography (CT), ultrasound, X-ray, etc., are all non-invasive ways to develop various organs, different tissues and lesion areas inside the human body. Medical image segmentation plays an indispensable auxiliary role in disease diagnosis, case analysis, surgical planning, and prognosis evaluation. It can not only provide doctors with valuable information such as the location of organ lesions, the size of lesion areas, and the severity of lesions, but also play a role in real-time imaging during surgical procedures. Deep learning has developed rapidly in the field of medical image segmentation. It uses nonlinear combinations to combine low-level features into abstract high-level features, solving the difficulty of analyzing low-resolution and high-complexity medical images. Therefore, the construction, upgrading and reasonable interpretation of deep learning networks have become one of the hot spots in the current research of the intersection of artificial intelligence and medicine.

[0003] At present, how to achieve high-precision automatic segmentation of liver tumors is still one of the most challenging tasks in medical image processing. Due to labor costs and professional knowledge limitations, it is difficult to annotate the liver and liver tumors at the voxel level in a large liver image dataset. For data-driven deep learning models, the lack of labeled data is undoubtedly a problem that needs to be solved urgently. Secondly, compared with the liver, liver tumors are not only small in size and uneven in grayscale distribution, but also vary in shape, number and location from person to person, and the boundaries between tumors and organs are very blurred. These characteristics make it more difficult to perform fine segmentation of liver tumors.

[0004] At present, the methods of medical image segmentation at home and abroad can be divided into: traditional technology, technology based on shallow machine learning and technology based on deep learning. Traditional image segmentation technology uses the grayscale, texture, edge and other features of the image to segment the target area by manually setting the feature values. The quality of the segmentation result is closely related to the manually set features, which usually limits the prediction performance of complex scenes and ignores a large amount of available original image information. In the face of the limitations of traditional segmentation technology, the development of machine learning has provided a new solution for the segmentation of medical images. Shallow machine learning technologies include: clustering, support vector machines, etc., which all rely on manual feature extraction, with high labor and time costs, and the quality of feature selection directly affects the segmentation effect. Research on deep learning has undergone long-term development and evolution. Since the MP (logic neuron) model was proposed in 1943, researchers have successively proposed network models such as perceptron, back propagation algorithm, convolutional neural network, generative adversarial network, residual network, etc. The emergence of these methods has injected fresh blood into machine learning. With the application and promotion of artificial intelligence in the medical field, good results have been achieved in segmenting the liver and its tumors from medical images using deep learning algorithms. The segmentation accuracy has been greatly improved compared with traditional image segmentation methods and segmentation methods based on shallow machine learning, but there are still some problems that hinder its further development. For example, for liver tumors with uneven density distribution or at different scales, the existing segmentation technology still has a lot of room for improvement; in addition, there is currently no large labeled data set that can fully meet the training needs of deep networks. Therefore, it is necessary to study how to learn with less sample data.

[0005] For medical images, especially three-dimensional images, obtaining labeled data requires a lot of labor and time costs. The small sample problem has hindered the development of AI (artificial intelligence) medical imaging to a certain extent. In recent years, researchers have tried various methods to make full use of incomplete data sets and proposed many high-performance models to reduce the demand for labeled data in medical image segmentation. Data determines the upper limit of model performance. When the amount of labeled data is very limited, how to make a small amount of data play a greater role is an urgent problem to be solved in medical image segmentation research.

[0006] At present, deep learning has achieved remarkable results in the field of liver tumor segmentation. However, due to the characteristics of medical imaging itself, the segmentation task has the following problems:

[0007] 1) In order to reduce the amount of computation, most 3D segmentation models splice 2D segmentation results into 3D results. The optimization target of the model is replaced by the boundary curve of the ROI (region of interest) in the 2D image slice instead of the ROI surface in the 3D image, which undoubtedly reduces the segmentation accuracy;

[0008] 2) In abdominal CT images of different cases, the size, location, texture and other information of liver tumors vary greatly, and it is very difficult to accurately locate the tumor area using an end-to-end network directly;

[0009] 3) Training deep neural networks relies on a large number of labeled medical images. Manually completing accurate voxel-level annotation is a very time-consuming and labor-intensive process, and has a certain degree of subjectivity. In addition, since medical images have different imaging protocols, the training set images annotated for one research project are difficult to use as training sets for another research project, and often need to be re-labeled. Therefore, it is worth further exploring how to make full use of the existing small amount of label information when the labeled data is incomplete.

[0010] 4) Most segmentation algorithms work well only in images with clear boundaries and strong contrast. However, due to the influence of local volume effects, tissue motion, noise and artifacts, the boundaries of liver tumors in CT images are blurred, the contrast with the liver is low, and the data complexity is high. The neural network model needs to extract a large number of features to divide the edges well. How to use data of different scales to improve the model representation ability and achieve fine segmentation of tumor edges is also a problem that needs to be solved. Summary of the invention

[0011] The present invention provides a medical image segmentation device based on generative adversarial networks and multi-scale feature fusion. Aiming at the problem of small sample size of medical images, the present invention uses a semi-supervised learning method, combines multi-scale semantic information of medical images, uses a generative adversarial network based on feature map synthesis and an improved multi-scale V-Nets, to achieve the segmentation of liver and tumor from abdominal CT images. See the following description for details.

[0012] A medical image segmentation device based on generative adversarial network and multi-scale feature fusion, the device comprising: a coarse-to-fine liver tumor segmentation framework,

[0013] The input of the segmentation framework is the original 3D CT image, which is preprocessed to obtain a standardized image; the liver is segmented using the discriminator in the feature map-based synthetic generative adversarial network after training, and a predicted probability map is output;

[0014] The value of each pixel in the probability map represents the probability that the pixel belongs to the liver, and the generative adversarial network learns more information through adversarial learning between the generator and the discriminator;

[0015] Automatic extraction of liver ROI: perform dot multiplication between the liver 3D segmentation result and the standardized image, shield other non-related organs, calculate the minimum circumscribed cuboid of the liver area and crop it, and resample livers of different sizes to the same size;

[0016] The liver ROI is taken as input, and the trained three-channel cascade network based on the improved V-Net is used to fuse multi-scale features, expand the receptive field, and deal with the differences in the position, shape and size of the target area in different data and the blurred boundary of the lesion area, and finally obtain the tumor segmentation result.

[0017] Among them, the discriminator is a V-Net network that incorporates a pyramid pooling module, and the generator is a neural network based on feature map synthesis. The generator is trained using the feature map output by spatial pyramid pooling to learn the distribution of CT images from unlabeled data and generate pseudo image data.

[0018] The discriminator uses the divided labeled data, unlabeled data and pseudo data generated by the generator for semi-supervised learning; the discriminator network and the generator network learn against each other until the training ends when the discriminator and the generator reach a dynamic balance.

[0019] Furthermore, the V-Net network of the pyramid pooling module is:

[0020] Using a spatial shape-aware module to improve pyramid pooling, the spatial shape-aware module is used to capture the long-range dependencies between different lesion regions in the image and capture the relevance of local context;

[0021] For the input tensor, three mutually perpendicular slices are used to process it to obtain three outputs, which are expanded to the same size as the input tensor and fused to obtain a new feature vector.

[0022] Furthermore, the fusion process is a dot product operation followed by a Softmax activation function, and the fused tensor is subjected to the same fusion operation as the original input tensor to obtain the final output tensor;

[0023] Using three-scale three-dimensional pooling layers, the input feature maps are pooled to three scales of 1×1×1, 2×2×2, and 3×3×3 respectively; the number of channels of the three-scale pooling results are reduced by convolution; then they are upsampled to the original feature map size and fused with the original feature map and the output result of the spatial shape perception module; the number of channels is reduced again by convolution to obtain a feature map containing multi-scale information.

[0024] Among them, the three-channel cascade network based on the improved V-Net is: a multi-scale segmentation network,

[0025] The input of the first branch is to reduce the three dimensions of the original input data to 0.5 times each. After outputting the corresponding segmentation result, the result is upsampled by 2 times, and the results of the other two branches are fused to output the segmentation map of the liver tumor.

[0026] The beneficial effects of the technical solution provided by the present invention are:

[0027] 1. The present invention uses the idea of ​​cascade network to directly segment the liver image area from the 3D image, and further segment the tumor image area more finely from the liver image area, thereby realizing end-to-end detection of medical images;

[0028] 2. The present invention uses a semi-supervised learning method, which can make full use of the information of unlabeled data and solve the problem of excessive dependence of deep learning models on labeled data. The model can use less labeled data to obtain higher segmentation accuracy;

[0029] 3. The present invention uses a spatial shape perception module to improve the original pyramid pooling module, which can match the appropriate target size through a variety of receptive fields of different sizes from a global and local perspective, effectively play the role of features of different scales, use the semantic information of high-level features to make the feature map of low-level features more complete, and use the details of low-level features to refine the edges of high-level features, avoiding the interference caused by background noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the structure of a medical image segmentation device based on a generative adversarial network and multi-scale feature fusion;

[0031] Figure 2 Schematic diagram of the liver segmentation model;

[0032] Figure 3 Schematic diagram of the training process of the liver segmentation model;

[0033] Figure 4 Schematic diagram of the pyramid pooling module structure improved by using spatial shape perception;

[0034] Figure 5 This is a schematic diagram of the spatial shape perception module;

[0035] Figure 6 Schematic diagram of the tumor segmentation model;

[0036] Figure 7 Schematic diagram of the visualization of liver tumor segmentation results of each model. DETAILED DESCRIPTION

[0037] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention are described in further detail below.

[0038] Example 1

[0039] The embodiment of the present invention provides a medical image segmentation device based on a generative adversarial network and multi-scale feature fusion, and describes a coarse-to-fine liver tumor segmentation framework, such as Figure 1 As shown in the figure, the input of the framework is the original 3D CT image, which is preprocessed to obtain a standardized image; then the discriminator in the trained feature map synthesis generative adversarial network (FRGAN) is used to segment the liver and output a predicted probability map. The value of each pixel in the probability map represents the probability that the pixel belongs to the liver. FRGAN learns more information from less labeled data through adversarial learning between the generator and the discriminator; then the liver ROI is automatically extracted: the liver 3D segmentation result and the standardized image are dot-multiplied, other non-related organs are shielded, the minimum circumscribed cuboid of the liver area is calculated and cropped, and livers of different sizes are resampled to the same size; then the liver ROI is used as input, and a trained three-channel cascade network based on the improved V-Net (fully convolutional neural network for volumetric medical image segmentation) is used. This part integrates multi-scale features, expands the receptive field, and better handles the differences in the position, shape and size of the target area in different data and the fuzzy boundary of the lesion area, and finally obtains the tumor segmentation result.

[0040] In summary, tumor segmentation through the above framework is completed in a liver region that is smaller and more accurate than the original image, so it can effectively reduce the situation of incorrect tumor segmentation and improve the accuracy of tumor segmentation.

[0041] Example 2

[0042] Combine the following Figure 2-Figure 5 The scheme in Example 1 is further introduced, as described below:

[0043] 1. Liver Region Extraction

[0044] The embodiment of the present invention uses a feature graph synthesis generative adversarial network (FRGAN) as Figure 1 Liver region segmentation module in the segmentation framework. Figure 2 The network structure of FRGAN is shown, which is mainly composed of a generator and a discriminator. The V-Net network with a pyramid pooling module is used as the discriminator, and the segmentation results are output at the same time. The neural network based on the feature map synthesis method is used as the generator. The generator is trained using the feature map output by the S-PPM (spatial pyramid pooling) module to learn the distribution of CT images from unlabeled data and generate pseudo image data; to verify the performance of the model, the experimental data is divided into labeled data and unlabeled data in a ratio of 4:6, and the discriminator uses the divided labeled data, unlabeled data and pseudo data generated by the generator for semi-supervised learning; the discriminator network and the generator network learn against each other until the training ends when the discriminator and the generator reach a dynamic balance. Figure 3 The three colored arrows in the figure represent the training process of the network.

[0045] Figure 2 The left half of the figure is the network structure of the generator, which uses the feature map synthesis method to generate fake images. The S-PPM module in the discriminator network aggregates multi-level and multi-scale feature information, and uses its output feature map as the input of the generator. Through 4 upsampling operations and 4 stages of convolution operations, a fake image of the same size as the real data is generated. Then the real and fake images are respectively passed through the encoder part of the discriminator to extract the feature map of the unlabeled image (Unlabeled Feature-map) and the feature map of the fake image (Fake Feature-map), and the mean difference between the two is used as the loss. After multiple iterations, the generator can synthesize a fake image that is closer to the real image through the feature map.

[0046] like Figure 4 The network structure of the improved pyramid pooling module S-PPM is shown. The organ tumors in medical images are irregular in shape and unevenly distributed. The original pyramid pooling that completely relies on the stacking of pooling layers of different sizes is difficult to learn the target area features and key location information from the medical images. Therefore, the embodiment of the present invention uses the spatial shape perception module (SSAM) to improve the pyramid pooling module. Among them, the network structure of the spatial shape perception module SSAM is as follows Figure 5 As shown in the figure, it can flexibly capture the long-distance dependencies between different lesion areas in the image, effectively capture the relevance of local context, and prevent interference from irrelevant areas. For the input tensor, three mutually perpendicular sheet submodules are used to process it to obtain three outputs, which are expanded to the same size as the input tensor and fused to obtain a new feature vector. The fusion process is a dot product operation followed by a Softmax activation function; finally, the fused tensor is subjected to the same fusion operation as the original input tensor to obtain the final output tensor. On this basis, three-dimensional pooling layers of three scales are used to pool the input feature maps to three scales of 1×1×1, 2×2×2, and 3×3×3 respectively; then the pooling results of the three scales are convolved to reduce the number of channels; then they are upsampled to the original feature map size, and fused with the original feature map and the output result of SSAM; the number of channels is reduced again by convolution to obtain a feature map containing multi-scale information.

[0047] Among them, the loss function of the generator is l G as follows:

[0048] l G =‖E x′～G(x) f(x)+E x′～G(x) f(x ′ )‖ 2 (1)

[0049] Among them, x represents the input unlabeled data, x ′Denote the pseudo data generated by the generator based on unlabeled data, and f(x) represents the unlabeled feature map extracted after the pseudo data passes through the S-PPM module in the discriminator, f(x ′ ) represents the pseudo feature map extracted after the pseudo data passes through the S-PPM module. E represents the expected value of the loss between the unlabeled data and the pseudo data, and G(x) represents the distribution of the images generated by the generator.

[0050] The discriminator calculates the supervised loss between the labels of the real annotated samples and the predicted target region segmentation results of the annotated samples, and at the same time calculates the unsupervised loss of the predicted unannotated samples and the target region segmentation results of the pseudo samples. Then the loss l of the discriminator D is composed of the loss l of the labeled images Labeled , the loss l of the unlabeled images Unlabeled and the loss l of the three types of data of the pseudo images Fake :

[0051] l D = l Labeled + l Unlabeled + l Fake (2)

[0052] For the input data, x G represents the label data corresponding to the labeled image. Using x, x G ~ P data (x, x G ) indicates that the input is labeled data, x ~ P data (x) indicates that the input is unlabeled data, x ′ ~ G(f) indicates that the input is pseudo data, and G(f) represents the distribution of the pseudo data. Using the cross-entropy loss function (well-known to those skilled in the art), the losses of the three types of input images can be calculated by the following formula:

[0053]

[0054] where p D (y i |x, y i <N + 1) represents the probability that the voxel prediction category in the image x is y i , p D (y i = N + 1|x) represents the probability that the voxel prediction category in the unlabeled data is the pseudo image, p D (y i = N + 1|x ′ ) represents the probability that the voxel prediction category in the pseudo data is the pseudo image. Here, N takes the value of 1, and p D represents the prediction probability of the discriminator network for various types of data.

[0055] The role of the above loss function is to calculate the gap between the forward calculation result and the true value during each iterative training process of the network, so as to guide the next step of training in the right direction. During the training process, the neural network updates its parameters. After training, the network can learn the distribution of the data image and has the ability to extract the liver area from the image.

[0056] 2. Tumor Segmentation

[0057] After the liver region is extracted, the liver tumor is segmented. In order to better utilize multi-scale information to improve the segmentation performance of the network model, a three-channel parallel liver tumor segmentation network (Multi-channel Parallel V-Nets, MP V-Nets) is designed based on S-PPM V-Net. The structure is as follows Figure 6 There are three similar segmentation branches in the multi-scale segmentation network, the difference lies in the different input scales and the different sampling operations performed when the final segmentation results are fused. Figure 6 Taking the first branch as an example, its input is the original input data, each of the three dimensions is reduced to 0.5 times. After outputting the corresponding segmentation result, the result needs to be upsampled by 2 times, and the results of the other two branches are fused, and finally the segmentation map of the liver tumor is output.

[0058] Define the loss function L of the network MP V-Nets MPV is the sum of the loss functions of the three parallel branches, each of which uses the cross entropy loss function. ij represents the input data of the i-th branch, N represents the total number of voxels in the data, x Gij represents the true value label of the jth voxel in the data, p(x ij ) represents the predicted probability vector of the point, then the loss function is calculated by the following formula:

[0059] L MPV =L 0.5 +L 1 +L 2 (4)

[0060]

[0061] Through the loss function L MPV The gap between the prediction result of the tumor segmentation network and the actual label is calculated, and the parameters of each layer of the network are updated during the back-propagation process. After multiple iterative updates, the network is able to distinguish whether a pixel belongs to the foreground or the background, thereby accurately segmenting the tumor from the image.

[0062] Example 3

[0063] Combine the following Figure 7Table 1 further introduces the schemes in Examples 1 and 2, as described below:

[0064] The algorithm experiment code in the embodiment of the present invention uses the Python programming language, and the algorithm framework is built in the efficient and flexible Pytorch deep learning framework. Since the solution of the embodiment of the present invention is based on a deep neural network, the experimental link needs to be configured with a GPU processor, and at the same time cooperate with the Cuda acceleration computing platform launched by NVIDIA and the GPU database cuDNN for deep neural networks to complete complex neural network calculations.

[0065] Since the FRGAN liver segmentation network uses a semi-supervised training method, the 100 pre-processed training set data are randomly divided into 40 and 60 images, which are used as labeled data and unlabeled data for training the discriminator, respectively. The Adam optimization algorithm is used in the network training process, and the initial learning rate of the generator is set to 1×10 -3 , the initial learning rate of the discriminator is set to 1×10 -5 , BatchSize is set to 2, and the maximum number of training rounds is set to 20000. The segmented liver image area is used as the training data for the tumor segmentation stage, and the liver in the label is set to 0 and the tumor is set to 1.

[0066] Then, the three-channel parallel tumor segmentation model MP V-Nets was trained. The Adam optimization algorithm was used in the network training process, the BatchSize was set to 2, and the initial learning rate was set to 1×10 -3 , the gradient coefficient is set to 0.9, the square gradient coefficient is set to 0.999, and the weight decay coefficient is 1×10 -8 , the maximum number of training rounds is set to 200, training is stopped when obvious overfitting occurs, and all weight parameters of the network are saved.

[0067] The dataset used is LiTS2017, which has a total of 131 training samples, including 100 training sets and 31 test sets. DICE (DICE coefficient), ACC (accuracy), VOE (voxel overlap error), RVD (relatively increased error), SP (specificity) and SE (Lin sensitivity) are used as evaluation indicators. Among them, the DICE number is a commonly used similarity measure in medical image segmentation. The larger the value, the closer the output result is to the real result.

[0068] Five different advanced 3D segmentation algorithms are selected for comparative experiments, and the test results of each experimental method are shown in Table 1. The five models are: DenseVoxNet, Rubik's cube++, 3D U-Net, V-Net, Vox2Vox, corresponding to [a] to [e] in the table.

[0069] Table 1 Comparison of liver tumor segmentation indicators

[0070]

[0071] It can be seen from the above table:

[0072] (1) By comprehensively considering all indicators, the three-channel parallel liver tumor segmentation model MP V-Nets proposed in the embodiments of the present invention performed well and was generally superior to other methods, proving that the coarse-to-fine segmentation framework is an excellent choice for achieving accurate liver tumor segmentation.

[0073] (2) Compared with Rubik's cube++, although MP V-Nets is slightly inferior in relative volume error, it has higher performance in DICE coefficient, ACC, VOE and SE, and improves the most important indicator DICE by 3.79%, indicating that the segmentation results of this model are closer to the true value.

[0074] (3) Compared with V-Net, the integration of the spatial shape perception module and the pyramid pooling module not only improves the DICE index, but also significantly reduces VOE and RVD. It can be seen that the spatial shape feature module and the improved pyramid pooling module stimulate more spatial information of the feature map, fully mine the high-level semantic information of the image, and retain more image edge information, which proves the accuracy and superiority of the liver tumor segmentation of this scheme.

[0075] In order to observe and compare the performance of each model more intuitively, the segmentation effects of each model on several difficult segmentation cases are compared, and the segmentation results of different models are visualized, such as Figure 7 As shown. Five specific cases with high segmentation difficulty in the experimental test were selected for display. It can be clearly seen that compared with the liver, tumors have the characteristics of wide distribution, large number, and different shapes and sizes, which increase the difficulty of tumor segmentation. The first four columns in the figure are the segmentation results of the original data, liver area, gold standard and this device, and the next five columns are the segmentation results of five comparison models. By comparison, when the tumor is very small and distributed very densely, other models tend to treat multiple small tumors as a whole and segment them as a whole, while this model can more accurately eliminate irrelevant areas between tumors; for cases where the grayscale contrast between the tumor and the surrounding tissue is small, this model can obtain a tumor boundary that is very similar to the gold standard, while other models find it difficult to accurately segment the edge of the tumor, resulting in under-segmentation.

[0076] Unless otherwise specified, the models of the components in the embodiments of the present invention are not limited, and any device that can perform the above functions may be used.

[0077] Those skilled in the art will appreciate that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the serial numbers of the embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0078] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A medical image segmentation device based on generative adversarial network and multi-scale feature fusion, characterized in that: The device comprises: a coarse-to-fine liver tumor segmentation framework, The input of the segmentation framework is the original 3D CT image, which is preprocessed to obtain a standardized image; the liver is segmented using the discriminator in the generative adversarial network based on feature map synthesis after training, and a prediction probability map is output; The value of each pixel in the probability map represents the probability that the pixel belongs to the liver, and the generative adversarial network learns more information through adversarial learning between the generator and the discriminator; Automatic extraction of liver ROI: perform dot multiplication between the liver 3D segmentation result and the standardized image, shield other non-related organs, calculate the minimum circumscribed cuboid of the liver area and crop it, and resample livers of different sizes to the same size; Taking the liver ROI as input, the trained three-channel cascade network based on the improved V-Net is used to fuse multi-scale features, expand the receptive field, and deal with the differences in the position, shape, and size of the target area in different data and the blurred boundary of the lesion area, and finally obtain the tumor segmentation result; The discriminator is a V-Net network that incorporates a pyramid pooling module, and the V-Net network is: A spatial shape perception module is used to improve pyramid pooling and attached to the skip connection of the V-Net network. The spatial shape perception module is used to capture the long-range dependencies between different lesion areas in the image and the relevance of the local context. For the input tensor, three mutually perpendicular slices are used to process it to obtain three outputs, which are then expanded to the same size as the input tensor and fused to obtain a new feature vector.

2. The medical image segmentation device based on generative adversarial network and multi-scale feature fusion according to claim 1, characterized in that: The generator uses a neural network based on feature map synthesis. The generator is trained using the feature map output by spatial pyramid pooling to learn the distribution of CT images from unlabeled data and generate pseudo image data.

3. A medical image segmentation device based on generative adversarial network and multi-scale feature fusion according to claim 1 or 2, characterized in that: The discriminator performs semi-supervised learning using the divided labeled data, unlabeled data, and pseudo data generated by the generator; The discriminator network and the generator network learn against each other until the training ends when the discriminator and the generator reach a dynamic balance.

4. The medical image segmentation device based on generative adversarial network and multi-scale feature fusion according to claim 1, characterized in that: The fusion process is a dot product operation followed by a Softmax activation function, and the fused tensor is subjected to the same fusion operation as the original input tensor to obtain the final output tensor; Use three-dimensional pooling layers of three scales to pool the input feature maps to three scales of 1×1×1, 2×2×2, and 3×3×3 respectively; The three-scale pooling results are convolved to reduce the number of channels; then they are upsampled to the original feature map size and fused with the original feature map and the output results of the spatial shape perception module; the number of channels is reduced again through convolution to obtain a feature map containing multi-scale information.

5. The medical image segmentation device based on generative adversarial network and multi-scale feature fusion according to claim 1, characterized in that: The three-channel cascade network based on the improved V-Net is: a multi-scale segmentation network, The input of the first branch is to reduce the three dimensions of the original input data to 0.5 times each. After outputting the corresponding segmentation result, the result is upsampled by 2 times, and the results of the other two branches are fused to output the segmentation map of the liver tumor.

Images