Method and device for automatic segmentation of small tumors in kidney magnetic resonance images

Abstract

The present invention relates to a method for automatic segmentation of small tumors in a renal magnetic resonance image, the method comprising: S1. pre-processing a training image and a tumor segmentation map; S2. constructing a 3D U-Net segmentation network model with multi-scale supervision; S3. using a Model Genesis self-supervised learning pre-training method to pre-train the 3D U-Net segmentation network model to obtain model parameters; S4. using the model parameters obtained by pre-training, using five-fold cross validation to train CP, DP, T1WI, NP, and T2W1 sequences on the pre-processed training image and tumor segmentation map so that each sequence obtains five segmentation network models; S5. pre-processing the image to be segmented; S6. based on the five segmentation network models, using a sliding window method to infer the pre-processed image to be segmented, and averaging the obtained initial segmentation results to obtain the final segmentation result. The present invention can realize accurate automatic segmentation of small tumors in renal magnetic resonance images.

Description

Technical Field

[0001] The invention relates to magnetic resonance kidney image processing technology in the medical field, and in particular to a method and a device for automatically segmenting small tumors in kidney magnetic resonance images. Background Art

[0002] With the development of imaging technology, the detection rate of small renal tumors (in this application, small renal tumors with a maximum diameter of less than 4 cm) is getting higher and higher. Preoperative diagnosis of benign and malignant small renal tumors (SRM) can help benign patients avoid unnecessary surgery. Radiomics can extract image features with high throughput and show great potential in tumor diagnosis. Some previous studies have verified the value of radiomics in the diagnosis of benign and malignant SRM. In these studies, accurate segmentation of small renal tumors is a key step. Small renal tumor segmentation refers to the identification and output of labels of small renal tumors on magnetic resonance images. In this study, it refers to the classification of voxels on magnetic resonance images into small renal tumors and normal tissues. The automatic segmentation of small renal tumors will greatly shorten the development time of radiomics and lay the foundation for the establishment of fully automatic segmentation and diagnosis processes of subsequent radiomics.

[0003] The segmentation of small kidney tumors is challenging. This is not only because compared with other organs, abdominal organs not only have complex anatomical structures, but also the boundaries between organs may appear blurred and may be affected by respiratory motion to produce motion artifacts. In addition, small kidney tumors are smaller in size and more difficult to detect and segment. In related technologies, the kidney segmentation model built based on CNN (Convolutional Neural Network) shows good performance, but the segmentation performance of kidney tumors is relatively poor, and as the tumor size decreases, the segmentation performance further decreases. Summary of the invention

[0004] In view of the above existing technical problems, the present application proposes a method for automatically segmenting small tumors in kidney magnetic resonance images, including:

[0005] S1. Preprocessing the training image and the tumor segmentation map by resampling and normalization, wherein the training image is a kidney magnetic resonance image for training purposes, and the tumor segmentation map is a small tumor image for training purposes manually segmented from the training image;

[0006] S2. constructing a 3D U-Net segmentation network model with multi-scale supervision, wherein the structure of the 3D U-Net segmentation network model consists of multiple resolution levels;

[0007] S3. Pre-train the 3D U-Net segmentation network model using the Model Genesis self-supervised learning pre-training method to obtain model parameters;

[0008] The pre-training includes: using 623 CT images in LUNA 2016 as pre-training materials, normalizing the CT image intensity to the range of [0,1]; randomly cropping image blocks from the normalized CT images with a size of 64*64*32; using mean square error as reconstruction loss; and performing multiple transformations on the cropped image blocks with preset probabilities;

[0009] S4. Using the model parameters obtained by pre-training, five-fold cross validation is used to train each of the CP, DP, T1WI, NP, and T2W1 sequences on the pre-processed training images and tumor segmentation maps, so that five segmentation network models are finally obtained for each sequence;

[0010] S5. Preprocessing the image to be segmented by resampling and normalizing, wherein the image to be segmented is a renal magnetic resonance image from which a small tumor is to be automatically segmented;

[0011] S6. Based on the five segmentation network models, the preprocessed image to be segmented is inferred using a sliding window method, and the initial segmentation results respectively obtained by the five segmentation network models are averaged to obtain a final segmentation result, wherein the initial segmentation result includes a tumor probability map.

[0012] According to some embodiments of the present invention, the resampling comprises:

[0013] Using cubic spline interpolation method to perform intra-layer interpolation on the training image and the image to be segmented to achieve resampling, and using nearest neighbor interpolation method to perform inter-layer interpolation on the training image and the image to be segmented to achieve resampling; and

[0014] The tumor segmentation map is resampled as follows: the tumor segmentation map is transformed into a one-hot encoding map, and the one-hot vector is interpolated using a linear interpolation method, and the one-hot map is reconverted into a tumor segmentation map using an argmax method;

[0015] The resampling further includes: all training images and images to be segmented are resampled to the same target in-plane resolution and inter-layer spacing; for the in-plane resolution, the median resolution of all training image data is used as the target in-plane resolution; and the 10th percentile of the inter-layer spacing of all training cases is used as the target inter-layer spacing;

[0016] The normalization includes using z-score to perform a normalized linear transformation on the signal intensity of the training image and the image to be segmented so that their mean is 0 and their variance is 1.

[0017] According to some embodiments of the present invention, before the preprocessed images are input into the 3D U-Net segmentation network model, each image is standardized by subtracting its mean intensity value and then dividing it by its standard deviation, and then all foreground voxels are collected, 0.5% and 99.5% of the foreground voxels are used for cropping, and the image is normalized using the mean and standard deviation of the cropped foreground, and the formula for the z-score is as follows:

[0018]

[0019] Where Z is the standardized value, χ i is the original value, μ is the mean value of the cropped foreground, and σ is the standard deviation.

[0020] According to some embodiments of the present invention, each resolution level contains two blocks, each block consists of a 3*3*3 convolution, IN and an activation function, and downsampling and upsampling are achieved through strided convolution and transposed convolution, wherein a LeakyReLU function with a negative slope of 0.01 is used as the activation function.

[0021] According to some embodiments of the present invention, the number of feature maps at the beginning is set to 32, and the number of feature maps is increased or decreased by half each time downsampling or upsampling occurs; and a 3D U-Net segmentation network model with 6 layers of resolution is used to segment the CP, DP, T1WI and NP sequences, the first layer resolution level of the segmentation network model uses 1x3x3 convolution, and the remaining five layers resolution levels use 3x3x3 convolution; a segmentation network with 7 layers of resolution is used to segment the T2WI sequence, the first and second layers of the segmentation network use 1x3x3 convolution, and the remaining layers use 3x3x3 convolution.

[0022] According to some embodiments of the present invention, the following loss function is used To guide the optimization process of model parameters:

[0023]

[0024] in is the cross-entropy loss, is soft dice loss, and the calculation formulas of the two are

[0025]

[0026] Among them, y true Represents the tumor segmentation map, ypred Represents the prediction result;

[0027] The method also includes: adding 1x1x1 convolution and Softmax activation functions on the last four resolution levels of the decoder area to introduce deep supervision.

[0028] According to some embodiments of the present invention, step S6 further includes: predicting the segmentation result in each window by sliding the image to be segmented according to the size of the image block cropped during training; the window size is the same as the size of the image block cropped during training, and the step size is equal to half of the image block cropped during training; and obtaining a complete predicted segmentation result by sliding the window over the entire image to be segmented in sequence;

[0029] According to some embodiments of the present invention, the tumor is a small solitary renal tumor, and the method further comprises: using connected component analysis to optimize the initial segmentation result of each segmentation network model to obtain a final segmentation result.

[0030] The present application also proposes a device for automatically segmenting small tumors in kidney magnetic resonance images, which includes a processor and a memory; the processor executes the above method based on a computer program stored in the memory.

[0031] The present application also proposes a computer program product, including a computer program, characterized in that the steps of the aforementioned method are implemented when the computer program is executed by a processor.

[0032] The present invention can realize accurate automatic segmentation of small tumors in kidney magnetic resonance images. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for describing the embodiments are briefly introduced below.

[0034] Figure 1 A flowchart showing a method for automatic segmentation of small tumors in kidney magnetic resonance images according to some embodiments of the present invention;

[0035] Figure 2 A schematic diagram showing a neural network architecture used in a method according to some embodiments of the present invention;

[0036] Figure 3 Demonstrate the manner of model integration according to some embodiments of the present invention;

[0037] Figure 4 The segmentation result after 3D U-net segmentation is shown;

[0038] Figure 5A schematic structural diagram showing an apparatus for automatically segmenting small tumors in a kidney magnetic resonance image according to some embodiments of the present invention. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0040] Before introducing the technical solution of the present application, the relevant terms involved in the embodiments of the present application are first introduced.

[0041] The small renal tumor in this application refers to a small renal tumor with a maximum diameter of less than 4 cm.

[0042] Convolution layer: It extracts features from input data and contains multiple convolution kernels. Each element of the convolution kernel corresponds to a weight coefficient and a bias.

[0043] Convolutional Neural Network: A type of feedforward neural network with a deep structure that contains convolution calculations.

[0044] Segmentation model: A mathematical model obtained by training labeled samples using machine learning technology. During the training process, the parameters in the model are updated according to the data samples. When performing the segmentation task, the parameters of the mathematical model are loaded and the segmentation results of the input data are calculated and generated.

[0045] Magnetic Resonance Imaging (MRI): Utilizing the principle of nuclear magnetic resonance, based on the different attenuation of the released energy in different structural environments inside a substance, and by detecting the emitted electromagnetic waves through an external gradient magnetic field, the position and type of the atomic nuclei that make up the object can be determined, and a structural image (magnetic resonance image) of the object's interior can be drawn accordingly.

[0046] Figure 1 A flowchart showing a method for automatic segmentation of small tumors in a kidney magnetic resonance image according to some embodiments of the present invention is shown, the method comprising the following steps:

[0047] S1. The training images and tumor segmentation maps were preprocessed by resampling and normalization.

[0048] The training image is a kidney magnetic resonance image for training purposes, and the tumor segmentation map is a small tumor image for training purposes that is manually segmented from the training image.

[0049] The preprocessing is intended to eliminate image differences caused by individual differences among clinical patients and different scanning devices, imaging protocols and scanning parameters.

[0050] According to some embodiments of the present invention, the intra-layer interpolation of the image is resampled using the third-order spline interpolation method, and the inter-layer interpolation is resampled using the nearest-neighbor interpolation method. The sampling method of the tumor segmentation map (the manually segmented tumor label in this application) used for training is to transform the segmentation map into a one-hot encoding map, and use the linear interpolation method to interpolate the one-hot vector, and use the argmax method to convert the interpolated one-hot map back into the tumor segmentation map to complete the resampling in the plane.

[0051] According to some embodiments of the present invention, all images will be resampled to the same target resolution. For the in-plane resolution, the median resolution of all training set data is used as the target spacing. Since the inter-layer voxel spacing and the intra-layer voxel spacing of the kidney magnetic resonance image are quite different, the spatial voxel anisotropy is presented. The present invention selects the 10th percentile of the inter-layer spacing of all training cases as the target inter-layer spacing. The main reasons for this selection are: 1. If the selected percentile is too high (such as the 50th percentile or greater), the information of the low-resolution axis may not be effectively represented; if the selection is too low (such as the 1st percentile), although the resolution can be improved, it will generate excessive computational overhead, which will affect the training efficiency; 2. The interpolation operation can improve the resolution of the image, but if the low-resolution axis is over-interpolated, noise may be introduced, blurring the details, and even making it difficult for the network to distinguish between important medical features and useless interpolation noise. Through the present invention, a reasonable balance is found between retaining sufficient spatial information and reducing the consumption of computing resources. The inter-layer spacing refers to the distance between adjacent slices (layers) in a three-dimensional image.

[0052] According to some embodiments of the present invention, the z-score may be used to perform a normalized linear transformation on the signal intensity of the image so that its mean is 0 and its variance is 1, thereby making the data in the same scale and training the network faster.

[0053] According to some embodiments of the present invention, before feeding the image into the network, each image is standardized by subtracting its mean intensity value and then dividing by its standard deviation. Afterwards, all foreground voxels are collected, 0.5% and 99.5% of the foreground voxels are used for cropping, and the image is normalized using the mean and standard deviation of the cropped foreground. The formula for the z-score is as follows:

[0054]

[0055] Where Z is the standardized value, χ i is the original value, μ is the average value of the cropped foreground, and σ is the standard deviation. Step S1 can also be moved to before step S4.

[0056] S2. Construct a 3D U-Net segmentation network model with multi-scale supervision, wherein the structure of the 3D U-Net segmentation network model consists of multiple resolution levels.

[0057] like Figure 2 As shown, the basic neural network architecture of the 3DU-Net training model for the segmentation of small renal tumors according to some embodiments of the present invention is shown. The 3D U-Net structure consists of multiple resolution levels. Each level contains two blocks, each of which consists of a 3*3*3 convolution (Conv), IN (instance normalization) and an activation function, and downsampling and upsampling are achieved through step convolution and transposed convolution. Here, the present invention uses the LeakyReLU function (IReLU) with a negative slope of 0.01 as the activation function, instead of using the ReLU function as the activation function, which comes from two reasons. On the one hand, when the input of the neural network is less than 0, the output of the ReLU function is always 0, which will cause the gradient of the neuron to be 0 during back propagation. If the network weights are improperly initialized or the learning rate is high, a part of the network will be permanently inactivated, so that effective features cannot be learned. The LeakyReLU function introduces a negative slope to ensure that the gradient flow of the network is smoother, and there will be no problem of gradient disappearance, which is particularly important in deep networks. On the other hand, in kidney MRI, the contrast, structure, and feature distribution of the data are usually very uneven, and sometimes may contain a lot of noise or tiny features. If ReLU is used, when encountering certain complex or noisy areas, some neurons may fail, affecting the segmentation effect. Leaky ReLU can better retain the sensitivity to these subtle information.

[0058] According to some embodiments of the present invention, the number of feature maps at the beginning is set to 32, and the number of feature maps is increased or decreased by half each time downsampling or upsampling. To prevent the introduction of too many feature maps, the number of feature maps is limited to 320. Since the CP, DP, T1WI and NP sequences have the same target spacing, the present invention uses the same segmentation network to segment the CP, DP, T1WI and NP sequences. Specifically, the segmentation network for the CP, DP, T1WI and NP sequences has 6 layers of resolution, the first layer of resolution level uses 1x3x3 convolution, and the remaining five layers of resolution level use 3x3x3 convolution, and the stride size (Stride) used for each layer of convolution is as follows: Figure 2In addition, since the T2WI sequence has a higher intra-layer resolution, in order to fully extract the intra-layer information of T2WI, the present invention uses a segmentation network with a 7-layer resolution to segment the T2WI sequence. The first and second layers of the network use 1x3x3 convolutions, and the remaining layers use 3x3x3 convolutions. The step size used in each layer of convolution is as follows: Figure 2 shown.

[0059] According to some embodiments of the present invention, the following loss function is used To guide the optimization process of model parameters to minimize prediction error and improve segmentation accuracy:

[0060]

[0061] in is cross-entropyloss, is soft dice loss, and the calculation formulas of the two are

[0062]

[0063] Among them, y true Represents the true label, i.e., the tumor segmentation map, y pred Represents the prediction result.

[0064] In order to allow the model to better learn and adjust features of different resolutions, prevent gradient vanishing and promote deep training in the network, the present invention uses a deep supervision strategy to train the model, that is, adding additional loss functions to feature maps at different levels for training. Specifically, the present invention adds 1x1x1 convolution and Softmax activation functions to the last four resolution levels in the decoder area to introduce deep supervision, such as Figure 2 By calculating the segmentation loss at different resolutions at four levels to further optimize the model parameters, the total loss function is It is obtained by weighted summation of loss functions at each resolution.

[0065]

[0066] in n is the number of deep supervision layers, is the loss at the penultimate i-th resolution.

[0067] S3. The 3D U-Net segmentation network model is pre-trained using the Model Genesis self-supervised learning pre-training method to obtain model parameters; the pre-training includes: using 623 CT images in LUNA 2016 as pre-training materials, normalizing the CT image intensity to the range of [0,1]; randomly cropping image blocks from the normalized CT images with a size of 64*64*32; using mean square error (MSE) as reconstruction loss; and performing multiple transformations on the cropped image blocks with preset probabilities.

[0068] Pre-training the model using a self-supervised learning pre-training method aims to reduce training time while improving the performance of the 3DU-Net segmentation network model.

[0069] Since the tumor annotation process of renal magnetic resonance images is costly and time-consuming, labeled medical image data is scarce. Self-supervised learning allows the model to be pre-trained without manual annotation, thereby improving the performance of image segmentation.

[0070] Specifically, the present invention uses the Model Genesis self-supervised learning method to pre-train the 3D U-Net segmentation network model, which includes learning the general 3D anatomical information of medical images by reconstructing the input image, thereby providing pre-trained model parameters for training the downstream 3D U-Net segmentation network model. More specifically, the present invention uses 623 CT images in LUNA 2016 with a large amount of sample data, rather than magnetic resonance images, as pre-training materials. Not only that, the present invention also randomly cuts image patches (patches) from the CT images, with a size of 64*64*32, to further improve the training effect. The inventors found that when the patch is too small, it is impossible to obtain enough contextual information. If it is too large, it will consume computing power, and the computing power consumption under 64*64*32 or left and right image patches is the least. Before training, the image intensity is normalized to the range of [0,1]. The present invention directly chooses to use the mean square error (MSE) as the reconstruction loss, because for self-supervised learning, the MSE loss is simple and efficient, enough to learn the three-dimensional spatial structure of three-dimensional medical images. The image blocks were transformed with a preset probability, including random flip with a probability of 0.4, random local pixel shuffling with a probability of 0.5, random non-linear appearance transformation with a probability of 0.9, and random image painting with a probability of 0.9, including in-painting with a probability of 0.2 and out-painting with a probability of 0.8. Each 3D U-Net segmentation network model was trained for 200 epochs, with a patch size of 64. The network parameters were optimized using stochastic gradient descent with a momentum of 0.9, with an initial learning rate set to 0.1 and decayed by 0.5 every 12 epochs.

[0071] S4. Using the model parameters obtained by pre-training, five-fold cross validation is used to train each of the CP, DP, T1WI, NP, and T2W1 sequences on the pre-processed training images and tumor segmentation maps, so that five segmentation network models are finally obtained for each sequence.

[0072] After completing the pre-training of the model, the present invention begins to train each sequence. Taking into account the large differences between different sequences, the present invention uses five-fold cross validation to train each sequence on the image and tumor segmentation map training set, so that five segmentation network models are finally obtained for each sequence. Here, the image and tumor segmentation map training set refers to the pre-processed image and tumor segmentation map for training. Afterwards, each segmentation network model is trained for 500 rounds (epoch), and in each epoch, 250 mini-batch trainings (mini-patch) are performed, and the size of each mini-patch is 2. Afterwards, the stochastic gradient descent (SGD) algorithm with Nesterov momentum is used to optimize the network parameters (the value of the momentum parameter μ is 0.99). The initial learning rate is set to 0.01 and gradually decays during the training process. The learning rate decay follows the "poly" learning rate strategy, that is, the learning rate gradually decreases as the training proceeds in an exponential function manner.

[0073] In order to solve the problem of class imbalance, the present invention samples image patches from random positions on the image. Among them, 2 / 3 of the patches are randomly cut, and 1 / 3 of the patches contain SRM. This study applies scaling, rotation, brightness, contrast, Gaussian noise, Gaussian blur, low-resolution simulation, mirroring and gamma correction to enhance the data (data augmentation).

[0074] S5. Preprocessing the image to be segmented by resampling and normalizing, wherein the image to be segmented is a kidney magnetic resonance image from which a small tumor is to be automatically segmented.

[0075] Here, the image to be segmented needs to be preprocessed, and the processing method is as mentioned in the previous step S1, that is, the image to be segmented is resampled and normalized. The resampling includes:

[0076] The images to be segmented are interpolated within the layers using the cubic spline interpolation method to achieve resampling, and the training images and the images to be segmented are interpolated between the layers using the nearest neighbor interpolation method to achieve resampling.

[0077] The resampling further includes: resampling the image to be segmented to the same target in-plane resolution and inter-layer spacing; using the median resolution of all training image data as the target in-plane resolution for the in-plane resolution; and using the 10th percentile of the inter-layer spacing of all training cases as the target inter-layer spacing;

[0078] The normalization includes using z-score to perform a normalized linear transformation on the signal intensity of the image to be segmented so that its mean is 0 and its variance is 1.

[0079] The normalization may also include standardizing each image by subtracting its mean intensity value and then dividing it by its standard deviation before inputting the preprocessed image to be segmented into the 3D U-Net segmentation network model, then collecting all foreground voxels, using 0.5% and 99.5% of the foreground voxels for cropping, and using the mean and standard deviation of the cropped foreground to normalize the image, and the formula for the z-score is as follows:

[0080]

[0081] Where Z is the standardized value, χ i is the original value, μ is the mean value of the cropped foreground, and σ is the standard deviation.

[0082] S6. Based on the five segmentation network models, the preprocessed image to be segmented is inferred using a sliding window method, and the initial segmentation results respectively obtained by the five segmentation network models are averaged to obtain a final segmentation result, wherein the initial segmentation result includes a tumor probability map.

[0083] The input image to be segmented is slid according to the size of the image patch cropped during training, and the segmentation result in each window is predicted. The window size is the same as the size of the image patch cropped during training, and the step size is equal to half of the image patch cropped during training. By sliding the window over the entire image in sequence, a complete segmentation prediction result can be obtained. During the reasoning process, all directions of the image are also mirrored (mirroring all axes). The present invention adopts an average ensemble strategy, and the final segmentation result (Figure) is obtained by averaging the initial segmentation results of the five segmentation network models obtained in step S4. The initial segmentation results of the five segmentation network models can be, for example, a probability map, which is used to represent the probability that each pixel in the image belongs to a specific category (such as tumor or non-tumor). In addition, in the task of renal tumor segmentation, the model may mistakenly classify some noise or irrelevant small areas as tumors, and these misclassified areas are called false positives. Since the present invention is aimed at the task of segmenting a single small renal tumor, in order to ensure that only one tumor region is segmented while ensuring the spatial coherence of the tumor region, the present invention uses connected component analysis to optimize the initial segmentation result of the model to obtain the final segmentation result.

[0084] The figure shows the segmentation results of the segmentation model on small renal tumors (SRM) of three patients, from left to right, respectively T2WI, T1WI, enhanced three-phase images. The yellow and red contour lines represent the tumor boundaries of manual segmentation and automatic segmentation, respectively. It can be seen that the segmentation model of the present application effectively segments the SRM. In medical image analysis, the Dice coefficient is an important evaluation index used to evaluate the performance of the segmentation algorithm. The closer the value of the Dice coefficient is to 1, the closer the segmentation result is to the actual situation, and the better the segmentation effect.

[0085] Figure 5 FIG. 5 is a schematic structural diagram of an automatic small tumor segmentation device 500 for a kidney magnetic resonance image according to some embodiments of the present invention. Figure 5 As shown, the device includes a processor 51 , a memory 52 and a bus 53 .

[0086] In some examples, the device may also include an input device 501, an input port 502, an output port 503, and an output device 504. The input port 502, the processor 51, the memory 52, and the output port 503 are interconnected through a bus 53, and the input device 501 and the output device 504 are connected to the bus 53 through the input port 502 and the output port 503, respectively, and then connected to other components of the device. It should be noted that the output interface and the input interface here can also be represented by an I / O interface. Specifically, the input device 501 receives input information from the outside, such as an image, and transmits the input information to the processor 51 through the input port 502; the processor 51 processes the input information based on the computer executable instructions stored in the memory 52 to generate output information, temporarily or permanently stores the output information in the memory 52, and then transmits the output information to the output device 504 through the output port 503; the output device 504 outputs the output information to the outside of the device.

[0087] The above-mentioned memory 52 includes a large capacity memory for data or instructions. For example, but not limitation, the memory 52 may include a HDD, a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 52 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 52 may be inside or outside the device. In a specific embodiment, the memory 52 is a non-volatile solid-state memory. In a specific embodiment, the memory 52 includes a read-only memory (ROM). In appropriate cases, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.

[0088] The bus 53 includes hardware, software or both, and multiple components are coupled to each other. For example, but not limitation, the bus 53 may include an accelerated graphics port (AGP) or other graphics bus, an enhanced industrial standard architecture (EISA) bus, a front-side bus (FSB), a hypertransport (HT) interconnect, an industrial standard architecture (ISA) bus, an infinite bandwidth interconnect, a low pin count (LPC) bus, a memory bus, a microchannel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus or other suitable bus or a combination of two or more of these. Although the embodiments of the present invention describe and illustrate a specific bus, the present invention considers any suitable bus or interconnect.

[0089] The processor 51 executes a method for automatically segmenting small tumors in a kidney magnetic resonance image based on a computer program stored in the memory 52 .

[0090] According to some further embodiments of the present invention, the computer program may be divided into one or more units in various ways and stored in the memory, and executed by the processor to complete the present invention. The one or more units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the device. The computer program may be divided into multiple units according to the functions of the various units in the various embodiments described above with reference to the various figures, or include the various units in the various embodiments described above with reference to the various figures. For simplicity, it will not be repeated here.

[0091] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc. The processor is the control center of the device, and various interfaces and lines are used to connect the various parts of the entire device. The device may be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server, or a part thereof. The device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the schematic diagram is merely an example of a device and does not constitute a limitation on the device.

[0092] The corresponding detailed descriptions made above with reference to the various figures are incorporated herein by reference and will not be repeated here.

[0093] The present application also proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the method for automatically segmenting small tumors in kidney magnetic resonance images is implemented.

[0094] The corresponding detailed descriptions made above with reference to the various figures are incorporated herein by reference and will not be repeated here.

[0095] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0096] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention. It should be noted that although the structure of the device of the present invention and the method of its operation are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in this specific order (for example, step S1 can be moved before step S4), or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flowchart can change the order of execution. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution.

Claims

1. A method for automatic segmentation of small tumors in renal magnetic resonance images, including: S1. Preprocessing the training image and the tumor segmentation map by resampling and normalization, wherein the training image is a kidney magnetic resonance image for training purposes, and the tumor segmentation map is a small tumor image for training purposes manually segmented from the training image; S2. constructing a 3D U-Net segmentation network model with multi-scale supervision, wherein the structure of the 3D U-Net segmentation network model consists of multiple resolution levels; S3. Pre-train the 3D U-Net segmentation network model using the Model Genesis self-supervised learning pre-training method to obtain model parameters; The pre-training includes: using 623 CT images in LUNA 2016 as pre-training materials, normalizing the CT image intensity to the range of [0,1]; randomly cropping image blocks from the normalized CT images with a size of 64*64*32; using mean square error as reconstruction loss; and performing multiple transformations on the cropped image blocks with preset probabilities; S4. Using the model parameters obtained by pre-training, five-fold cross validation is used to train each of the CP, DP, T1WI, NP, and T2W1 sequences on the pre-processed training images and tumor segmentation maps, so that five segmentation network models are finally obtained for each sequence; S5. Preprocessing the image to be segmented by resampling and normalizing, wherein the image to be segmented is a renal magnetic resonance image from which a small tumor is to be automatically segmented; S6. Based on the five segmentation network models, the preprocessed image to be segmented is inferred using a sliding window method, and the initial segmentation results respectively obtained by the five segmentation network models are averaged to obtain a final segmentation result, wherein the initial segmentation result includes a tumor probability map.

2. The method according to claim 1, wherein: The resampling includes: Using a cubic spline interpolation method, the training image and the image to be segmented are interpolated within a layer to achieve resampling, and using a nearest neighbor interpolation method, the training image and the image to be segmented are interpolated between layers to achieve resampling; and The tumor segmentation map is resampled as follows: the tumor segmentation map is transformed into a one-hot encoding map, and the one-hot vector is interpolated using a linear interpolation method, and the one-hot map is reconverted into a tumor segmentation map using an argmax method; The resampling further includes: all training images and images to be segmented are resampled to the same target in-plane resolution and inter-layer spacing; for the in-plane resolution, the median resolution of all training image data is used as the target in-plane resolution; and the 10th percentile of the inter-layer spacing of all training cases is used as the target inter-layer spacing; The normalization includes using z-score to perform a normalized linear transformation on the signal intensity of the training image and the image to be segmented so that their mean is 0 and their variance is 1.

3. The method according to claim 2, wherein: Before the images preprocessed in steps S1 and S5 are fed into the 3D U-Net segmentation network model, each image is standardized by subtracting its mean intensity value and then dividing by its standard deviation. After that, all foreground voxels are collected, 0.5% and 99.5% of the foreground voxels are used for cropping, and the image is normalized using the mean and standard deviation of the cropped foreground. The formula for the z-score is as follows: Where Z is the standardized value. is the original value, μ is the mean value of the cropped foreground, and σ is the standard deviation.

4. The method according to claim 1 or 3, wherein: Each resolution level contains two blocks, each block consists of a 3*3*3 convolution, IN and an activation function. Downsampling and upsampling are achieved through strided convolution and transposed convolution, where the LeakyReLU function with a negative slope of 0.01 is used as the activation function.

5. The method according to claim 4, wherein: The number of feature maps at the beginning is set to 32, and the number of feature maps is increased or decreased by half each time downsampling or upsampling; a 3D U-Net segmentation network model with a 6-layer resolution is used to segment the CP, DP, T1WI and NP sequences. The first layer resolution level of the segmentation network model uses 1x3x3 convolution, and the remaining five layers resolution level use 3x3x3 convolution; a segmentation network with a 7-layer resolution is used to segment the T2WI sequence. The first and second layers of the segmentation network use 1x3x3 convolution, and the remaining layers use 3x3x3 convolution.

6. The method according to claim 5, wherein: Use the following loss function To guide the optimization process of model parameters: in is cross-entropyloss, is soft diceloss, and the calculation formulas for the two are Among them, y true Represents the tumor segmentation map, y pred Represents the prediction result; The method also includes: adding 1x1x1 convolution and Softmax activation functions on the last four resolution levels of the decoder area to introduce deep supervision.

7. The method according to claim 6, wherein step S6 further comprises: The image to be segmented is slid according to the size of the image block cropped during training, and the segmentation result in each window is predicted; The window size is the same as the size of the image block cropped during training, and the step size is equal to half of the image block cropped during training; by sliding the window over the entire image to be segmented in sequence, a complete predicted segmentation result is obtained.

8. The method according to claim 7, wherein: The tumor is a small solitary renal tumor, and the method further comprises: using connected component analysis to optimize the initial segmentation result of each segmentation network model to obtain a final segmentation result.

9. An automatic segmentation device for small tumors in a kidney magnetic resonance image, comprising a processor and a memory; the processor executes the method according to any one of claims 1 to 8 based on a computer program stored in the memory.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images