Two-stage full 3D abdominal organ segmentation method and system based on dual-resolution network

Abstract

The present invention belongs to the technical field of medical image segmentation, and specifically discloses a two-stage full 3D abdominal organ segmentation method and system based on a dual-resolution network. The method preprocesses a data set and an original image, performs random data enhancement on the preprocessed data set and the original image, trains a coarse segmentation network and a fine segmentation network according to the data set, scales the original image after data enhancement to a preset size and then inputs it into the coarse segmentation network to perform abdominal organ segmentation, obtains an ROI image according to the ROI area obtained by the coarse segmentation and scales it to a preset size, and then inputs it into the fine segmentation network to obtain a segmentation result. The present technical solution uses a two-stage method to achieve the segmentation of full 3D abdominal organs.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image segmentation, and relates to a two-stage full 3D abdominal organ segmentation method and system based on a dual-resolution network. Background Art

[0002] With the rapid development of modern society, people's pace of life has accelerated, and health issues have become more and more important. At the same time, modern medical technology is developing rapidly, and the use of medical image processing technology to assist diagnosis has become a very important treatment method in medical units.

[0003] In traditional methods, the standard of medical image segmentation is completed manually by doctors with many years of medical experience, which is time-consuming and laborious, and requires doctors to examine a large number of slice images one by one and outline volume images. In addition, the manual labeling results have the defects of low repeatability and strong subjectivity. Different doctors have different clinical treatments, and their respective manual segmentation results are also different.

[0004] Since the 21st century, image segmentation technology based on deep learning has been widely used in the medical field. However, due to the serious imbalance between classes and unclear differences between organs in abdominal medical images, some organs that are not easy to segment have the characteristics of small shape and large deformation, and the deformation between different cases is particularly large, and the boundaries are not clear, so that the effect of image model segmentation is often unsatisfactory. For example: the accuracy of the pancreas has been below 90%, and the liver, spleen, and left and right kidneys near the pancreas can reach about 33% to 95%. At the same time, due to the large amount of information and large number of bytes in a single set of 3D abdominal medical image data, the image segmentation model training process is very long, which greatly reduces the efficiency and timeliness of intelligent diagnosis of abdominal medicine. Therefore, there is an urgent need for a high-efficiency and high-precision automatic segmentation processing method.

[0005] In recent years, deep learning theoretical methods have been introduced into the intelligent automatic diagnosis of various medical images. Methods based on deep learning have become the mainstream research methods due to their powerful feature expression capabilities. Many existing medical image detection methods are based on the Unet (semantic segmentation) model and improved on this basis, such as: Dense, Res and other structures, feature pyramid and other modules, attention mechanism, deep supervision and other strategies, to improve the convolutional neural network's ability to extract image features, avoid losing more features and semantic information, etc. to improve the speed and performance of segmentation.

[0006] Although these methods and strategies have effectively improved the segmentation effect, the following problems still exist: 1) The improvement effect is not significant, and it is difficult to bring about a more obvious performance improvement in practical applications; 2) The network has a large number of parameters, and training requires a large data set, but the amount of available medical data is very small, and there is also a problem of overfitting; 3) After multiple downsamplings, the model loses serious image detail information, which has a great impact on accuracy; 4) The model uses too many activation layers and normalization layers, which may bring adverse effects, such as loss of feature details.

[0007] In addition, through further research and analysis of 3D abdominal organ segmentation, we found that there are still the following problems: 1) Sample collection is difficult, sample quality is uneven, and there are some samples of poor quality, which requires high generalization of the network; 2) There are certain difficulties in detecting small organs; 3) High prediction accuracy is required. 4) The single-stage reasoning using a sliding window has high computational cost and will lose the three-dimensional background. Summary of the invention

[0008] The purpose of the present invention is to provide a two-stage full 3D abdominal organ segmentation method and system based on a dual-resolution network, so as to achieve the segmentation of full 3D abdominal organs and shorten the segmentation task cycle.

[0009] In order to achieve the above object, the basic scheme of the present invention is: a two-stage full 3D abdominal organ segmentation method based on a deep dual-resolution network, comprising the following steps:

[0010] Obtain the data set and the original image, and preprocess the data set and the original image;

[0011] Perform random data augmentation on the preprocessed data set and original images;

[0012] According to the data set, train the coarse segmentation network and the fine segmentation network;

[0013] The original image after data enhancement is scaled to a preset size and then input into the coarse segmentation network for abdominal organ segmentation;

[0014] According to the ROI area obtained by rough segmentation, the ROI image is obtained and scaled to a preset size, and then input into the fine segmentation network to obtain the segmentation result.

[0015] The working principle and beneficial effects of this basic solution are: obtaining the corresponding data and performing preprocessing and random data enhancement can improve the generalization ability of the model and facilitate the subsequent use of the data. The two-stage method from coarse to fine is used to solve the problem of long cycle and time-consuming training of full 3D abdominal segmentation tasks, and can quickly and accurately segment abdominal organs.

[0016] Further, the method of preprocessing the dataset and original images is as follows:

[0017] The data with the smallest 5% of all layers in the z-axis direction were removed, and the data with the z-axis layer spacing higher than 3 were removed to obtain the experimental data set;

[0018] The experimental data set is divided into training set and test set in a ratio of 8:2. Before training the coarse segmentation network and the fine segmentation network, the images are adjusted to the preset size and standardized and normalized:

[0019]

[0020] Among them, μ and σ are the mean and variance of the original image data, respectively, and x and x * are the pixel values ​​of the original image and the pixel values ​​after preprocessing, respectively.

[0021] The operation is simple and easy to use. It can filter data and ensure that the data obtained by the network model is displayed smoothly in three dimensions.

[0022] Furthermore, the method of performing random data augmentation on the dataset and the original image is as follows:

[0023] Perform spatial geometric transformation and pixel transformation on the data set and the original image, and pre-set the probability value of the corresponding transformation;

[0024] Spatial geometric transformation increases the number of training samples, and pixel transformation generates pixel mask points of different sizes filled with different values, which are then mixed with the original image to disrupt some features of the original image.

[0025] Using random enhancement to enhance images can bring about a regularization effect, enhance the robustness of the model, and reduce the sensitivity of decisions to model parameters.

[0026] Further, the method of training the coarse segmentation network is as follows:

[0027] Reduce the images of the dataset to a preset size and input them into the coarse segmentation network, which is the 3DUnet network;

[0028] The ROI area of ​​the coarse segmentation network is cropped and adjusted to obtain the ROI image of the required size and input it into the fine segmentation network.

[0029] Reducing the image to a preset size can speed up the training process. The coarse segmentation network will obtain the ROI image and input it into the fine segmentation network to facilitate accurate segmentation by the fine segmentation network.

[0030] Further, the method of training the fine segmentation network is as follows:

[0031] Establish a deep dual-resolution branch network: The encoder downsamples and extracts features from the image through an improved 3D residual convolution module and a downsampling module. The i-th high-resolution feature map X Hi and low-resolution feature map X Li for:

[0032]

[0033] Among them, F H and F L Corresponding to the high-resolution and low-resolution residual basic block sequences, T L-H and T H-L They represent low-to-high and high-to-low conversion functions respectively, and R represents the Relu activation function;

[0034] Dual-resolution branch feature fusion: In the third stage of the encoder, dual-branch feature extraction is performed. The low-resolution branch continues to downsample to obtain more deep features and semantic information. The high-resolution branch extracts features and maintains the size and number of channels of the feature map unchanged. The two branches perform multiple bilateral feature fusions at different stages to fully integrate spatial and semantic information.

[0035] The anisotropic pyramid pooling module is used to capture the anisotropy and contextual information in the abdominal scene: before the dual-resolution branch is summed point by point, the low-resolution branch is input into the anisotropic pyramid pooling module. The anisotropic pyramid pooling module includes anisotropic strip pooling and standard spatial pooling. Anisotropic strip pooling captures the anisotropy and contextual information in the abdominal scene, thereby capturing the spatial relationship between multiple organs. Standard spatial pooling fuses multi-scale features.

[0036] The outputs of anisotropic strip pooling and standard spatial pooling are combined, and the semantic feature information extracted by the encoder is restored to the original image size through continuous upsampling, and the classification task of the corresponding pixels is completed to obtain the final output.

[0037] In the fine segmentation network, the residual module with less activation and normalization is used to avoid the loss of feature details. The dual-resolution branch and cross-fusion method are used to avoid the loss of more detail information during downsampling. The low-resolution branch supplements the detail information, and the high-resolution branch supplements the semantic information. The anisotropic pyramid pooling module is used in the low-resolution branch to capture more spatial information and realize multi-scale feature fusion.

[0038] Furthermore, multiple bilateral feature fusions are performed between the two branches at different stages to fuse spatial and semantic information. For fusion from high resolution to low resolution, the high-resolution feature map is downsampled by a 3×3×3 convolution sequence with a stride of 2 before point-wise summation;

[0039] From low resolution to high resolution fusion, the low resolution feature map is first compressed by 1×1×1 convolution and then upsampled by trilinear interpolation.

[0040] The operation is simple and easy to use, and it avoids the loss of image detail information caused by continuous downsampling.

[0041] Furthermore, the anisotropic pyramid pooling module captures the anisotropy and contextual information in the abdominal scene. The input feature map passes through two 1×1×1 convolution modules and is sent to anisotropic strip pooling and standard spatial pooling respectively.

[0042] Anisotropic strip pooling has three anisotropic strip pools with different receptive fields: 1×N×N, N×1×N, and N×N×1. After 3×1×1, 1×3×1, and 1×1×3 inter-slice convolutions and upsampling, they are finally added together and sent to a convolution module. This convolution module can capture the anisotropy and contextual information in the abdominal scene, thereby capturing the spatial relationship between multiple organs;

[0043] Standard spatial pooling uses two average poolings with step sizes of 2×2×2 and 4×4×4 respectively. After the same inter-slice convolution and upsampling as strip pooling, they are finally fused with the residual branch to achieve multi-scale feature fusion;

[0044] The outputs of anisotropic strip pooling and standard spatial pooling are combined, passed through a 1×1×1 convolution module, added with the input features, and passed through a ReLU activation function to obtain the final output.

[0045] Simple structure and easy to operate.

[0046] Furthermore, it also includes damage functions and deep supervision strategies:

[0047] Dice Loss is used as the loss function of the network. The calculation of Dice Loss is:

[0048]

[0049] Then the mixed loss function is:

[0050] Loss(y,p)=DiceLoss(y,p)

[0051] Among them, y represents the label of the original image, and p represents the prediction result of the network model;

[0052] Using the deep supervision strategy, auxiliary loss is added to the third stage of the high-resolution branch, and the overall loss function of the network is:

[0053] Loss total =Loss main (y,p)+λ1Loss aux1 (y,p)

[0054] Among them, Loss main 、Loss aux1 They represent the main loss and the auxiliary loss of the third stage respectively; λ1 is the loss weight.

[0055] Use deep supervision strategies to avoid gradient explosion and gradient disappearance, and solve the problem of slow convergence.

[0056] Furthermore, it also includes a method for evaluating the segmentation accuracy of the fine segmentation network:

[0057] The Adam optimizer is used to optimize the fine segmentation network, with the Dropout rate set to 0.2, the learning rate set to 0.01, the batch set to 1, and the maximum number of iterations set to 200 epochs;

[0058] Save the weight file generated in the round with the highest average DSC and NSC results on the validation set. When the maximum number of iterations is reached, terminate the network training;

[0059] in, G is the label of the original image, and S is the prediction result of the network model; G is the label of the original image, and S is the prediction result of the network model; and Represents the number of segmented voxels; and They represent the boundary area between the label and segmentation surface distance under the tolerance τ; The overlap of the boundary region representing the distance between the label voxel and the predicted surface, The overlap of the boundary region representing the distance between the predicted voxel and the label surface;

[0060] The DSC and NSC indicators are used to evaluate the accuracy of network segmentation on the test set.

[0061] Evaluating the segmentation accuracy of the network is helpful for analyzing whether the segmentation results output by the network are accurate, and is also convenient for subsequent optimization of the network.

[0062] The present invention also provides a two-stage full 3D abdominal organ segmentation system based on a deep dual-resolution network, comprising a data acquisition unit and a processing unit, wherein the data acquisition unit is used to acquire a data set and an original image, the output end of the data acquisition unit is connected to the input end of the processing unit, and the processing unit executes the method of the present invention to complete the full 3D abdominal organ segmentation.

[0063] The system uses a two-stage full 3D abdominal organ segmentation method based on a deep dual-resolution network to achieve full 3D abdominal organ segmentation, which is simple to operate and easy to use. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a flowchart of a two-stage full 3D abdominal organ segmentation method based on a deep dual-resolution network of the present invention;

[0065] Figure 2 Schematic diagram of the deep dual-resolution branch network structure of the two-stage full 3D abdominal organ segmentation method based on the deep dual-resolution network of the present invention;

[0066] Figure 3 It is a structural schematic diagram of a 3D residual convolution module of a two-stage full 3D abdominal organ segmentation method based on a deep dual-resolution network of the present invention;

[0067] Figure 4 It is a schematic diagram of a dual-branch feature fusion structure of a two-stage full 3D abdominal organ segmentation method based on a deep dual-resolution network of the present invention;

[0068] Figure 5 It is a structural schematic diagram of the anisotropic pyramid pooling module of the two-stage full 3D abdominal organ segmentation method based on a deep dual-resolution network of the present invention. DETAILED DESCRIPTION

[0069] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0070] In the description of the present invention, it is necessary to understand that the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0071] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0072] The present invention discloses a two-stage full 3D abdominal organ segmentation method based on a deep dual-resolution network, which realizes fast and accurate segmentation of full 3D abdominal organs. Figure 1 As shown, the method comprises the following steps:

[0073] Obtain the data set and the original image, and preprocess the data set and the original image;

[0074] Perform random data augmentation on the preprocessed data set and original images;

[0075] According to the data set, train the coarse segmentation network and the fine segmentation network;

[0076] The original image after data enhancement is scaled to a preset size and then input into the coarse segmentation network for abdominal organ segmentation;

[0077] According to the ROI area obtained by rough segmentation, the ROI (range of interest) image is obtained and scaled to a preset size, and then input into the fine segmentation network to obtain the segmentation result.

[0078] In a preferred embodiment of the present invention, the method for preprocessing the data set and the original image is as follows:

[0079] The experiment prepared more than 1,000 sets of self-built data sets. In order to ensure that the data obtained by the network model is displayed smoothly in three dimensions, some bad data needs to be screened. All data with the least spacing layers in the z-axis direction among all layers are removed, and all data with z-axis layer spacing higher than 3 are removed to obtain the experimental data set;

[0080] The experimental data set is divided into training set and test set in a ratio of 8:2. Before training the coarse segmentation network and the fine segmentation network, the image is resized to the preset size and standardized:

[0081]

[0082] Among them, μ and σ are the mean and variance of the original image data, respectively, and x *, x. For example, before training the coarse segmentation network model, the image is resized to (160×160×160); before training the fine segmentation model, the cropped ROI area is scaled to a large size (192×192×192) as the input data of the fine segmentation network.

[0083] In a preferred embodiment of the present invention, the method for performing random data enhancement on the data set and the original image is as follows:

[0084] Perform spatial geometric transformation and pixel transformation on the data set and the original image, and pre-set the probability value of the corresponding transformation, that is, perform random data enhancement on the pre-processed data, and set a probability value for each enhancement method; for example, the probability of flipping change is 0.5, so as to achieve random use of one or several data enhancements;

[0085] Image rotation, deformation, and mirroring are used in geometric transformation. Spatial geometric transformation of the data set increases the number of training samples and improves the generalization ability of the model. At the same time, it makes it easier for the model to learn the characteristics of different samples, enhances the translation invariance of the model, and can capture the target position of the enhanced image to avoid overfitting. Contrast enhancement, Gaussian blur, and brightness enhancement are used in pixel transformation. Through these transformations, pixel mask points of different sizes filled with different values ​​are generated, and then mixed with the original image to disrupt some features of the original image. These transformations can bring about regularization, enhance the robustness of the model, and reduce the sensitivity of the decision to the model parameters.

[0086] In a preferred embodiment of the present invention, the method for training the coarse segmentation network is as follows:

[0087] Reduce the images in the dataset to a preset size (such as [128×128×128] or [160×160×160]) to speed up the training and input them into the coarse segmentation network. The coarse segmentation network is a 3DUnet network with a simple model. It is used to learn the global information of the image and facilitate the reasoning of the fine segmentation to find a suitable ROI area.

[0088] The ROI area of ​​the coarse segmentation network is cropped and resized to obtain an ROI image of the required size (such as [192×192×192], the actual size can be determined according to the specific task of the dataset) and input into the fine segmentation network.

[0089] In a preferred embodiment of the present invention, the method for training the fine segmentation network is as follows:

[0090] The fine segmentation network needs to divide the image more accurately. In this stage, a method based on the deep dual-resolution branch network (DRUnet) is used. The overall architecture adopts the classic Encoder-Decoder framework. During training, the ROI area of ​​the original image is cropped using the Ground truth label data, and then scaled to [192, 192, 192] and fed into the model. In this way, the fine segmentation model can learn local context information more easily and accurately.

[0091] like Figure 2 As shown in Figure 2, a deep dual-resolution branch network is established: the encoder extracts features by stacking and independently downsampling feature extraction modules. The convolution block uses an improved 3D residual convolution module (such as Figure 3 As shown in the figure, the convolution block consists of two identical improved residual modules stacked together. The 3D residual convolution module uses only one normalization (Instance Norm) and one Relu activation function in the entire residual structure. At the same time, the size of each convolution kernel is 3×3×3, and the step size is 1, which does not change the size and number of channels of the feature map. The downsampling module consists of a normalization and a convolution block with a step size of 2, which reduces the feature map to 1 / 2 and the number of channels becomes twice that of the input. The feature map first passes through the downsampling module and then the feature extraction module. The input feature map passes through a convolution block composed of a convolution with a step size of 1 and an Instance Norm, and then passes through a convolution block composed of a convolution with a step size of 1 and Relu, and finally is added to the initial input as the input of the next residual module. The i-th high-resolution feature map X Hi and low-resolution feature map X Li for:

[0092]

[0093] Among them, F H and F L Corresponding to the high-resolution and low-resolution residual basic block sequences, T L-H and T H-L They represent low-to-high and high-to-low conversion functions respectively, and R represents the Relu function;

[0094] Dual-resolution branch feature fusion: Figure 4As shown in the figure, in order to avoid the loss of image detail information caused by continuous downsampling, dual-branch feature extraction is performed in the third stage of the encoder part. The low-resolution branch continues to downsample through the improved 3D residual convolution module and the independent downsampling module, and the number of channels is set to 16, 32, 64, 128, and 256 respectively to obtain more deep features and semantic information; the high-resolution branch is downsampled by two convolution modules with a convolution kernel size of 3×3×3 and a step size of 1 to extract features, maintaining the size and number of channels of the feature map unchanged, and multiple bilateral feature fusions are performed between the two branches at different stages to fully integrate spatial information and semantic information; for fusion from high resolution to low resolution, the high-resolution feature map is downsampled by a 3×3×3 convolution sequence with a step size of 2 before point-by-point summation; for fusion from low resolution to high resolution, the low-resolution feature map is first compressed by 1×1×1 convolution and then upsampled by trilinear interpolation;

[0095] The anisotropic pyramid pooling module is used to capture the anisotropy and contextual information in the abdominal scene: Figure 5 As shown in the figure, ConINReLU represents that the module is composed of convolution, normalization, and activation function; Avg-pool represents average pooling; ConINUpsample represents that the module is composed of convolution, normalization, and upsampling; 1*1*1, 1*12*12, etc. represent the size of the convolution kernel or pooling kernel; Relu represents the activation function; ConvIn represents that the module is composed of convolution and normalization. Before the dual-resolution branches are summed point by point, the low-resolution branch is input into the anisotropic pyramid pooling module. The anisotropic pyramid pooling module includes anisotropic strip pooling and standard spatial pooling. Anisotropic strip pooling captures the anisotropy and contextual information in the abdominal scene, thereby capturing the spatial relationship between multiple organs. Standard spatial pooling fuses multi-scale features.

[0096] Finally, the low-resolution branch passes through an anisotropic pyramid pool and is summed point by point with the high-resolution branch before being sent to the decoder module. The outputs of anisotropic strip pooling and standard spatial pooling are combined. The decoder module separates a standard 3D convolution with a kernel size of 3×3×3 into 3×3×1 intra-slice convolution and 1×1×3 inter-slice convolution, and uses trilinear interpolation to achieve upsampling of feature maps. By continuously upsampling, the semantic feature information extracted by the encoder is restored to the original image size and the classification task of the corresponding pixels is completed to obtain the final output.

[0097] More preferably, the anisotropic pyramid pooling module captures the anisotropy and contextual information in the abdominal scene, and the input feature map is sent to the anisotropic strip pooling and standard spatial pooling respectively after passing through two 1×1×1 convolution modules;

[0098] Anisotropic strip pooling has three anisotropic strip pools with different receptive fields: 1×N×N, N×1×N, and N×N×1. After 3×1×1, 1×3×1, and 1×1×3 inter-slice convolutions and upsampling, they are finally added together and sent to a convolution module. This convolution module can capture the anisotropy and contextual information in the abdominal scene, thereby capturing the spatial relationship between multiple organs;

[0099] Standard spatial pooling uses two average poolings with step sizes of 2×2×2 and 4×4×4 respectively. After the same inter-slice convolution and upsampling as strip pooling, they are finally fused with the residual branch to achieve multi-scale feature fusion;

[0100] The outputs of anisotropic strip pooling and standard spatial pooling are combined, passed through a 1×1×1 convolution module, added with the input features, and passed through a ReLU activation function to obtain the final output.

[0101] In a preferred embodiment of the present invention, the two-stage full 3D abdominal organ segmentation method further includes a damage function and a deep supervision strategy:

[0102] Dice Loss is used as the loss function of the network. The calculation of Dice Loss is:

[0103]

[0104] Then the mixed loss function is:

[0105] Loss(y,p)=DiceLoss(y,p)

[0106] Among them, y represents the label of the original image, and p represents the prediction result of the network model;

[0107] In order to accelerate the convergence of the network, a deep supervision strategy is adopted to add auxiliary loss to the third stage of the high-resolution branch. The overall loss function of the network is:

[0108] Loss total =Loss main (y,p)+λ1Loss aux1 (y,p)

[0109] Among them, Loss main 、Loss aux1 They represent the main loss and the auxiliary loss of the third stage respectively; λ1 is the loss weight, which can be set to 0.5.

[0110] In a preferred embodiment of the present invention, the two-stage full 3D abdominal organ segmentation method further includes a method for evaluating the segmentation accuracy of the fine segmentation network:

[0111] If we set x∈R d×w×h and y∈R d×w×h They represent the original image and its corresponding manual annotation result respectively, d, w, h represent the height, length and width of the image respectively; M:p=f(θ,x) is used to represent the segmentation model, where θ represents the network parameters, p represents the probability map of the prediction result, and M is the abbreviation of the model. Represents the prediction results:

[0112]

[0113] Randomly select 200 sets as training sets and 40 sets as test sets;

[0114] The Adam optimizer is used to optimize the fine segmentation network. (Adam is a first-order optimization algorithm that can replace the traditional stochastic gradient descent process. It can iteratively update the weights of the neural network based on the training data.) The dropout rate is set to 0.2, the learning rate is set to 0.01, the batch is set to 1, and the maximum number of iterations is 200 epochs (1 epoch means training once with all the samples in the training set);

[0115] Save the weight file generated by the round with the highest average value of DSC (Dice Similariy Coefficient) and NSC (Normalized Surface Dice) on the validation set. When the maximum number of iterations is reached, terminate the network training.

[0116] in, G is the label of the original image, and S is the prediction result of the network model; G is the label of the original image, and S is the prediction result of the network model; and Represents the number of segmented voxels; and They represent the boundary area between the label and segmentation surface distance under the tolerance τ; The overlap of the boundary region representing the distance between the label voxel and the predicted surface, The overlapping part of the boundary area representing the distance between the voxel of the prediction result and the label surface; the above parameters are all obtained through data statistics, the parameters with G are obtained through label statistics, and the parameters with S are obtained through prediction result data statistics.

[0117] The DSC and NSC indicators are used to evaluate the accuracy of network segmentation on the test set.

[0118] The present invention also provides a two-stage full 3D abdominal organ segmentation system based on a deep dual-resolution network, including a data acquisition unit and a processing unit. The data acquisition unit is used to acquire data sets and original images. The output end of the data acquisition unit is electrically connected to the input end of the processing unit. The processing unit executes the method described in the present invention to complete the full 3D abdominal organ segmentation.

[0119] This solution uses a two-stage method from coarse to fine to solve the problem of long cycle and very time-consuming training of full 3D abdominal segmentation tasks. According to the characteristics of the data, a random data enhancement method is used to improve the generalization ability of the model. In the fine segmentation network, the convolution module implements a residual module with fewer activations and normalizations to avoid the loss of feature details. At the same time, in order to avoid losing more detail information during the downsampling process, dual-resolution branches and cross-fusion methods are used. The low-resolution branch supplements the detail information, and the high-resolution branch supplements the semantic information. The anisotropic pyramid pooling module is used in the low-resolution branch to capture more spatial information and realize multi-scale feature fusion. At the same time, as the depth of the network increases, in order to solve the problems of "gradient disappearance" and "gradient explosion" and slow convergence, deep supervision is used in the decoder part.

[0120] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0121] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A two-stage full 3D abdominal organ segmentation method based on a deep dual-resolution network, characterized in that: The steps include: Obtain the data set and the original image, and preprocess the data set and the original image; Perform random data augmentation on the preprocessed data set and original images; According to the data set, train the coarse segmentation network and the fine segmentation network; The original image after data enhancement is scaled to a preset size and then input into the coarse segmentation network for abdominal organ segmentation; According to the ROI area obtained by rough segmentation, the ROI area image is obtained and scaled to a preset size, and then input into the fine segmentation network to obtain the segmentation result; The method for training the fine segmentation network is as follows: Establish a deep dual-resolution branch network: The encoder downsamples and extracts features from the image through an improved 3D residual convolution module and a downsampling module. The i-th high-resolution feature map X Hi and low-resolution feature map X Li for: Among them, F H and F L Corresponding to the high-resolution and low-resolution residual basic block sequences, T L-H and T H-L They represent low-to-high and high-to-low conversion functions respectively, and R represents the Relu activation function; Dual-resolution branch feature fusion: In the third stage of the encoder, dual-branch feature extraction is performed. The low-resolution branch continues to downsample to obtain more deep features and semantic information. The high-resolution branch extracts features and maintains the size and number of channels of the feature map unchanged. The two branches perform multiple bilateral feature fusions at different stages to fully integrate spatial and semantic information. The anisotropic pyramid pooling module is used to capture the anisotropy and contextual information in the abdominal scene: before the dual-resolution branch is summed point by point, the low-resolution branch is input into the anisotropic pyramid pooling module. The anisotropic pyramid pooling module includes anisotropic strip pooling and standard spatial pooling. Anisotropic strip pooling captures the anisotropy and contextual information in the abdominal scene, thereby capturing the spatial relationship between multiple organs. Standard spatial pooling fuses multi-scale features. The outputs of anisotropic strip pooling and standard spatial pooling are combined, and the semantic feature information extracted by the encoder is restored to the original image size through continuous upsampling, and the classification task of the corresponding pixels is completed to obtain the final output.

2. The two-stage full 3D abdominal organ segmentation method based on a deep dual-resolution network as claimed in claim 1, characterized in that: The method for preprocessing the dataset and original images is as follows: The data with the smallest 5% of all layers in the z-axis direction were removed, and the data with the z-axis layer spacing higher than 3 were removed to obtain the experimental data set; The experimental data set is divided into training set and test set in a ratio of 8:

2. Before training the coarse segmentation network and the fine segmentation network, the images are adjusted to the preset size and standardized and normalized: Among them, μ and σ are the mean and variance of the original image data, respectively, and x and x * are the pixel values ​​of the original image and the pixel values ​​after preprocessing, respectively.

3. The two-stage full 3D abdominal organ segmentation method based on a deep dual-resolution network as claimed in claim 1, characterized in that: The method of performing random data augmentation on the dataset and the original image is as follows: Perform spatial geometric transformation and pixel transformation on the data set and the original image, and pre-set the probability value of the corresponding transformation; Spatial geometric transformation increases the number of training samples, and pixel transformation generates pixel mask points of different sizes filled with different values, which are then mixed with the original image to disrupt some features of the original image.

4. The two-stage full 3D abdominal organ segmentation method based on a deep dual-resolution network as claimed in claim 1, characterized in that: The method for training the coarse segmentation network is as follows: Reduce the images of the dataset to a preset size and input them into the coarse segmentation network, which is the 3DUnet network; The ROI area of ​​the coarse segmentation network is cropped and adjusted to obtain the ROI image of the required size and input it into the fine segmentation network.

5. The two-stage full 3D abdominal organ segmentation method based on deep dual-resolution network as claimed in claim 1, characterized in that: Multiple bilateral feature fusions are performed between the two branches at different stages to fuse spatial and semantic information. For fusion from high resolution to low resolution, the high resolution feature map is downsampled by a 3×3×3 convolution sequence with a stride of 2 before point-wise summation. From low resolution to high resolution fusion, the low resolution feature map is first compressed by 1×1×1 convolution and then upsampled by trilinear interpolation.

6. The two-stage full 3D abdominal organ segmentation method based on a deep dual-resolution network as claimed in claim 1, characterized in that: The anisotropic pyramid pooling module captures the anisotropy and contextual information in the abdominal scene. The input feature map passes through two 1×1×1 convolution modules and is sent to anisotropic strip pooling and standard spatial pooling respectively. Anisotropic strip pooling has three anisotropic strip pools with different receptive fields: 1×N×N, N×1×N, and N×N×1. After 3×1×1, 1×3×1, and 1×1×3 inter-slice convolutions and upsampling, they are finally added together and sent to a convolution module. This convolution module can capture the anisotropy and contextual information in the abdominal scene, thereby capturing the spatial relationship between multiple organs; Standard spatial pooling uses two average poolings with step sizes of 2×2×2 and 4×4×4 respectively. After the same inter-slice convolution and upsampling as strip pooling, they are finally fused with the residual branch to achieve multi-scale feature fusion; The outputs of anisotropic strip pooling and standard spatial pooling are combined, passed through a 1×1×1 convolution module, and then added with the input features and passed through a ReLU function to obtain the final output.

7. The two-stage full 3D abdominal organ segmentation method based on a deep dual-resolution network as claimed in claim 1, characterized in that: Also includes damage functions and deep supervision strategies: Dice Loss is used as the loss function of the network. The calculation of Dice Loss is: Then the mixed loss function is: Loss(y,p)=DiceLoss(y,p) Among them, y represents the label of the original image, and p represents the prediction result of the network model; Using the deep supervision strategy, auxiliary loss is added to the third stage of the high-resolution branch, and the overall loss function of the network is: Loss total =Loss main (y,p)+λ1Loss aux1 (y,p) Among them, Loss main 、Loss aux1 They represent the main loss and the auxiliary loss of the third stage respectively; λ1 is the loss weight.

8. The two-stage full 3D abdominal organ segmentation method based on a deep dual-resolution network as claimed in claim 1, characterized in that: It also includes a method for evaluating the segmentation accuracy of the fine segmentation network: The Adam optimizer is used to optimize the fine segmentation network, with the Dropout rate set to 0.2, the learning rate set to 0.01, the batch set to 1, and the maximum number of iterations set to 200 epochs; Save the weight file generated in the round with the highest average DSC and NSC results on the validation set. When the maximum number of iterations is reached, terminate the network training; in, G is the label of the original image, and S is the prediction result of the network model; G is the label of the original image, and S is the prediction result of the network model; and Represents the number of segmented voxels; and They represent the boundary area between the label and segmentation surface distance under the tolerance τ; The overlap of the boundary region representing the distance between the label voxel and the predicted surface, The overlap of the boundary region representing the distance between the predicted voxel and the label surface; The DSC and NSC indicators are used to evaluate the accuracy of network segmentation on the test set.

9. A two-stage full 3D abdominal organ segmentation system based on a deep dual-resolution network, characterized in that: It comprises a data acquisition unit and a processing unit, wherein the data acquisition unit is used to acquire a data set and an original image, the output end of the data acquisition unit is connected to the input end of the processing unit, and the processing unit executes the method according to one of claims 1-8 to complete full 3D abdominal organ segmentation.

Images