A method for prostate magnetic resonance image region segmentation fusing prior information

Abstract

The application relates to a prostate magnetic resonance image region segmentation method fusing prior information, which comprises the following steps: data set preparation, image preprocessing, data expansion processing; constructing a shape reconstruction network; constructing a preliminary prostate region segmentation network fusing prior information; adjusting and optimizing the preliminary segmentation network through the preprocessed verification set image, obtaining a prostate region segmentation network; and using the preprocessed test set image to perform prostate region segmentation, obtaining a prostate region segmentation result image. The method can realize fine segmentation of the prostate region in the manner of fusing prior information, and has good segmentation effect and generalization ability.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical image segmentation and relates to a prostate magnetic resonance image region segmentation method fusing prior information. BACKGROUND

[0002] Prostate cancer is an epithelial malignant tumor occurring in the prostate and is one of the most common malignant tumor diseases endangering the life and health of middle-aged and old men. Studies have shown that tumors exist in different forms in different regions of the prostate, and the incidence probability and severity also differ. Generally, from the perspective of radiology, the entire prostate can be divided into a central zone and a peripheral zone, and about 70%-80% of the incidence zones occur in the peripheral zone.

[0003] However, the current examination of prostate diseases needs to rely on radiologists to manually delineate the region contour according to the prostate anatomical structure. The manual delineation process is not only time-consuming and laborious, but also subjective differences exist between different doctors. With the development of medical image processing technology, the realization of automatic segmentation of the prostate region by using computer technology can avoid the influence of subjective factors of doctors and greatly improve the disease diagnosis efficiency, which has very important research significance for the subsequent clinical treatment of prostate diseases and the grading evaluation of prostate cancer.

[0004] Medical image segmentation methods mainly include traditional segmentation methods and deep learning-based segmentation methods. The traditional segmentation methods generally have poor robustness, especially in the case of blurred boundary information. In recent years, deep learning-based medical image segmentation methods have been widely proposed due to their excellent feature representation ability, overcoming the limitations of traditional methods relying on manual feature extraction and selection. However, considering the small area proportion of the prostate region in magnetic resonance images, blurred boundary information, uneven gray scale distribution, and low contrast between the gland and the surrounding tissue, it is difficult to achieve accurate segmentation of the prostate region relying only on the bottom features of the image. Based on this, the present application combines the anatomical structure characteristics of the prostate and fully integrates prior information into the deep learning model to propose a prostate magnetic resonance image region segmentation method fusing prior information. SUMMARY

[0005] In order to solve the shortcomings and deficiencies in the prior art, the present application proposes a prostate magnetic resonance image region segmentation method fusing prior information, which fully integrates prior information and deep learning models to achieve fine segmentation of the prostate region.

[0006] The present application is realized by the following technical solutions, and the present application is specifically:

[0007] A prostate magnetic resonance image region segmentation method fusing prior information, characterized in that the method comprises:

[0008] Step 1: Obtain the prostate magnetic resonance data set, divide it into three parts of training set, validation set and test set, and convert the three-dimensional magnetic resonance image of the prostate into a two-dimensional axial sequence slice image;

[0009] Step 2: Image preprocessing: sequentially perform gray scale normalization processing, central cropping processing, size normalization processing and contrast limit adaptive histogram equalization processing on all sequence slice images;

[0010] Step 3: Data augmentation processing is performed on the training set images and corresponding label images obtained in step 2, which specifically includes: horizontal flip, small amplitude random translation, small angle random rotation and elastic deformation;

[0011] Step 4: Construct a shape reconstruction network, including the following steps:

[0012] 1) Use a convolutional autoencoder as the basic architecture to construct a shape reconstruction network;

[0013] 2) Use the training set label image obtained in step 3 as the input of the shape reconstruction network in step 1) to pre-train the shape reconstruction network;

[0014] Step 5: Construct a segmentation network that fuses prior information, including the following steps:

[0015] 1) Use U-Net as the basic architecture of the segmentation network, connect the shape reconstruction network as an extended network to U-Net, and embed a spatial constraint module at the bottom of U-Net to construct a segmentation network that fuses prior information;

[0016] 2) Use the training set image and its corresponding label image obtained in step 3 as the input of the segmentation network in step 1) to start the training of the prostate region segmentation network that fuses prior information;

[0017] 3) Train the network by forward propagation, then output the predicted probability map through the softmax classifier; calculate the loss according to the loss function, update the parameter values in the network through back propagation, and repeat this process until the training loss value converges, complete a network training, and generate a preliminary prostate region segmentation network that fuses prior information;

[0018] Step 6: Use the validation set image and its corresponding label image obtained in step 3 as the input of the network in step 3) in step 5, adjust and optimize the hyperparameters of the segmentation network according to the output result, and obtain the prostate region segmentation network that fuses prior information;

[0019] Step 7: Use the segmentation network obtained in step 6 to perform prostate region segmentation on the test set image preprocessed in step 2 to obtain the prostate peripheral zone and central zone segmentation result image.

[0020] Further, in step 2, the pixel value x of the normalized image is: *

[0021]

[0022] wherein x represents the pixel value of the image, μ represents the average value of the pixel of the image, and σ represents the standard deviation of the pixel of the image.

[0023] Further, in step 2, the center of the original image is taken as the clipping center, and the central clipping is performed according to the ratio of 2 / 3 of the length and width of the image.

[0024] Further, in step 2, the image size is uniformly normalized to 240x240.

[0025] Further, in step 3, the image is subjected to data augmentation processing, including the following steps:

[0026] 1) Horizontal flip: flip the image along the horizontal direction of the image;

[0027] 2) Small amplitude random translation: randomly translate the image along the up, down, left and right directions, and set the translation range to 4-8 pixel points;

[0028] 3) Small angle random rotation: randomly rotate the image with the center of the image as the origin, and set the rotation angle range to -10°-10°;

[0029] 4) Elastic deformation: randomly generate a displacement vector for each pixel point in the image, perform Gaussian convolution operation on the randomly generated displacement vector using a Gaussian function with a mean of 0 and a standard deviation of σ, and apply the result to the original image to obtain the elastically deformed image, wherein σ ranges from 2 to 10.

[0030] Further, in step 5, the loss function of step 3) comes from three parts: the segmentation network, the shape reconstruction network and the spatial constraint module, which are described as follows:

[0031] L=L Seg +ω SR L SR +ω SC L SC

[0032] wherein L Seg , L SR and L SC represent the segmentation loss, the shape reconstruction loss and the spatial constraint loss, respectively, ω SR and ω SC represent the weight coefficients of the shape reconstruction loss and the spatial constraint loss, respectively, and are as follows:​

[0033] i) Segmentation Loss: An exponential logarithmic loss is used, which combines the weighted cross-entropy loss and the DSC loss. The formula is described as follows:

[0034] L Seg =λ WCE L WCE +λ DSC L DSC

[0035] Where, λ WCE and λ DSC L represents the balance coefficients of the weighted cross-entropy loss function and the DSC loss function, respectively. WCE and L DSC Let represent the weighted cross-entropy loss and DSC loss after exponential and logarithmic transformations, respectively, and their formulas are described below:

[0036]

[0037] in, N represents the total number of pixels in the image, and C represents the total number of segmentation categories. This represents the probability that pixel i is predicted to be of class c. γ represents the true value of pixel i belonging to class c. WCE and γ DSC These represent two hyperparameters used to control the nonlinearity of the two-part loss function.

[0038] ii) Shape reconstruction loss:

[0039] The shape reconstruction loss uses a convolutional autoencoder to regularize the segmentation results, and its formula is described as follows:

[0040]

[0041] Among them, R i This represents the reconstructed value of pixel i in the actual label. This represents the reconstructed value of pixel i in the predicted image after being processed by the segmentation network, ||·|| F The Frobenius norm of a matrix is ​​defined as the square root of the sum of the squares of the absolute values ​​of its elements.

[0042] iii) Spatial constraint loss:

[0043] Spatial constraint loss uses the spatial location information of slices to regularize the segmentation results, and its formula is described as follows:

[0044]

[0045] Among them, P idenotes the spatial location of slice i corresponding to the ground truth label, denotes the spatial location of slice i predicted by the model.

[0046] Compared with the prior art, the method has the beneficial effects that:

[0047] The application provides a prostate magnetic resonance image region segmentation method fusing prior information, and improves a network architecture of U-Net, fuses a shape reconstruction network and a spatial constraint module into the network model, designs a loss function, and the total loss of the network comprises three parts of a segmentation loss, a shape reconstruction loss and a spatial constraint loss, and the integration of the prior information into the loss function can constrain the prediction result to be more accurate in an anatomical output. Compared with existing popular segmentation methods, the method improves the accuracy of prostate region segmentation, enhances the label consistency of the segmentation result, can realize more accurate prostate region segmentation, and has good generalization ability. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 The method flowchart is provided in the application;

[0049] Figure 2 The prostate axial magnetic resonance images from different medical centers and the corresponding real label examples used in the application are as follows: (a) slice images with a rectal coil at 1.5T and the corresponding real labels; (b) slice images with a rectal coil at 3.0T and the corresponding real labels;

[0050] Figure 3 The information flow between the shape reconstruction network and the segmentation network is provided in the application;

[0051] Figure 4 The prostate region segmentation network fusing prior information is provided in the application, and the specific architecture is as follows:

[0052] Figure 5 The prostate region segmentation result examples are provided in the application: (a) an original slice image; (b) a real label image; (c) a segmentation result image. DETAILED DESCRIPTION

[0053] In order to make the structure and advantages of the application clearer, the technical solutions of the application will be further described below with reference to the drawings and embodiments.

[0054] Figure 1 The method flowchart is provided in the application, and specifically comprises the following steps:

[0055] Step 1: data set preparation:

[0056] The prostate magnetic resonance images used in this embodiment come from the 2013 NCI-ISBI prostate structure automatic segmentation challenge dataset. The dataset contains a total of 60 T2-weighted axial magnetic resonance images and their corresponding true label images from two medical centers, half of which were obtained using a rectal coil at a 1.5T magnetic field strength, and the other half were obtained using a surface coil at a 3T magnetic field strength. Figure 2 For the prostate magnetic resonance images used in this embodiment and their corresponding true label images, the gray area represents the peripheral zone of the prostate, the white area represents the central zone of the prostate, and the black area represents the background.

[0057] This embodiment divides the above magnetic resonance dataset into a training set, a validation set and a test set in a ratio of 4:1:1, and converts it into a two-dimensional axial sequence slice image.

[0058] Step 2: Image preprocessing:

[0059] The following preprocessing operations are sequentially performed on all sequence slice images obtained in step 1:

[0060] 1) Gray scale normalization processing: Since the gray scale of magnetic resonance images from different medical centers differs greatly, the present application adopts Z-score standardization method to perform gray scale normalization processing on each slice image. The normalized pixel value x * is:

[0061]

[0062] Where x represents the image pixel value, μ represents the average value of the image pixel, and σ represents the standard deviation of the image pixel.

[0063] 2) Central cropping processing: Since the prostate organ occupies a small area in the image, and considering the prior information that the organ is usually located at the center of the image, the present application takes the center of the original image as the cropping center, and crops each slice image according to the 2 / 3 proportion of the image length and width.

[0064] 3) Size normalization processing: In this embodiment, the size of all slice images is normalized to 240x240.

[0065] 4) Contrast limited adaptive histogram equalization processing: In view of the problems of fuzzy boundary in the prostate region, low contrast between the gland and the surrounding tissue, etc., contrast limited adaptive histogram equalization processing is adopted, which equalizes each pixel by calculating the histogram of the image pixel neighborhood to improve the local contrast of the image, and simultaneously trims the gray levels in the histogram that exceed the threshold value, and distributes the part exceeding the threshold value to each gray level.

[0066] Step 3: Data augmentation processing:

[0067] Since the deep learning network needs to learn the deep semantic features in the image through a large amount of training data, and the data set used in the embodiment is very small, directly using the deep learning network will cause a serious overfitting problem. Therefore, the training set image and the corresponding label image obtained in step 2 are subjected to data augmentation processing, which specifically includes the following steps:

[0068] 1) horizontal flip: flip the image along the horizontal direction of the image; horizontal flip, small amplitude random translation, small angle random rotation and elastic deformation;

[0069] 2) small amplitude random translation: randomly translate the image in the up, down, left and right four directions, and set the translation range to 4-8 pixel points;

[0070] 3) small angle random rotation: randomly rotate the image with the center of the image as the origin, and set the rotation angle range to -15°-15°;

[0071] 4) elastic deformation: a displacement vector is randomly generated for each pixel point in the image, a Gaussian function with a mean of 0 and a standard deviation of σ is used to perform Gaussian convolution operation on the randomly generated displacement vector, and the result is applied to the original image to obtain the elastically deformed image, wherein σ is in the range of 2-10.

[0072] The training data augmented by the above four methods is 30 times the original training data.

[0073] Step 4: Constructing a shape reconstruction network:

[0074] 1) The shape reconstruction network reconstructs the original input by learning the intermediate representation, and the goal is to force the incorrectly predicted segmentation image to conform to the shape of the training label. The shape reconstruction network is constructed using the architecture of the convolutional autoencoder. The shape reconstruction module includes two parts of shape encoder and decoder. The shape encoder projects the segmentation result into the latent representation using the combination of convolution operation and nonlinear mapping, and then the shape decoder reconstructs the input from the latent space representation, thereby constraining the prediction result to a more accurate anatomical output. Compared with U-Net, there is no skip connection between the shape encoder and the decoder. Figure 3 The information flow between the shape reconstruction network and the segmentation network.

[0075] 2) The training set label image obtained in step 3 is used as the input of the shape reconstruction network in step 1), and the shape reconstruction network is pre-trained.

[0076] Step 5: Constructing a segmentation network that fuses prior information:

[0077] 1) Network structure building:

[0078] The present application combines the anatomical features of the prostate, fuses shape prior information and spatial prior information into the deep learning network, improves the basic framework U-Net, connects the shape reconstruction network obtained in step 5 as an extension network to U-Net, and embeds the spatial constraint module at the bottom of U-Net. Figure 4 The specific architecture of the prostate region segmentation network fusing prior information according to the present application.

[0079] 2) Loss function setting:

[0080] The total loss function of the prostate region segmentation network fusing prior information comes from three parts: the segmentation network, the shape reconstruction network and the spatial constraint module, and the formula description is as follows:

[0081] L = L Seg + ω SR L SR + ω SC L SC

[0082] Wherein, L Seg , L SR and L SC represent the segmentation loss, shape reconstruction loss and spatial constraint loss respectively, λ SR and λ SC represent the weight coefficients of shape reconstruction loss and spatial constraint loss, and the specific description is as follows:

[0083] i) Segmentation loss: exponential logarithmic loss is adopted, which combines weighted cross entropy loss and DSC loss, and the formula description is as follows:

[0084] L Seg = λ WCE L WCE + λ DSC L DSC

[0085] Wherein, λ WCE and λ DSC represent the balance coefficients of weighted cross entropy loss function and DSC loss function, L WCE and L DSC represent the weighted cross entropy loss and DSC loss after exponential and logarithmic conversion, and the formula description is as follows:

[0086]

[0087] Wherein, N represents the total number of pixel points in the image, C represents the total number of segmentation categories, represents the probability of pixel point i being predicted as class c, γ represents the true value of pixel i belonging to class c. WCE and γ DSC These represent two hyperparameters used to control the nonlinearity of the two-part loss function.

[0088] ii) Shape reconstruction loss:

[0089] The shape reconstruction loss uses a convolutional autoencoder to regularize the segmentation results, and its formula is described as follows:

[0090]

[0091] Among them, R i This represents the reconstructed value of pixel i in the actual label. This represents the reconstructed value of pixel i in the predicted image after being processed by the segmentation network, ||·|| F The Frobenius norm of a matrix is ​​defined as the square root of the sum of the squares of the absolute values ​​of its elements.

[0092] iii) Spatial constraint loss:

[0093] Spatial constraint loss uses the spatial location information of slices to regularize the segmentation results, and its formula is described as follows:

[0094]

[0095] Among them, P i This indicates the spatial location of the actual label corresponding to slice i. This represents the spatial location predicted for slice i.

[0096] 3) Network training strategy and key parameter settings:

[0097] In this invention, images are imported in random mini-batches during network training, with a batch size of 32. The optimizer uses parameters β1 = 0.9, β2 = 0.999, and epsilon = 10. -8 The Adam optimizer. The total number of segmentation categories in the network described in this invention is C=3, and the loss weight coefficient is set to ω. SR =5×10 -4 ω SC =1×10 -6 , λ WCE =0.2, λ DSC =0.8, initial learning rate set to 1×10 -3 The number of iterations is 50.

[0098] 4) Online training:

[0099] The training set image obtained in step 3 and the corresponding label image are input into the network with a size of 240x240. The channel number is doubled and the image size is halved after each encoding layer in the encoding stage. With the increase of the number of convolutions, the receptive field of the feature map is multiplied, and the network tends to pay more attention to the deep semantic features. Then in the decoding stage, the channel number is doubled and the image size is halved after each decoding layer. The output of the final segmentation network is a segmentation image with the same size as the input image. The training loss value is calculated according to the loss function, and the parameter value in the network is updated through back propagation. Repeat this process until the training loss value converges, complete a network training, and generate a preliminary prostate region segmentation network fused with prior information.

[0100] Step 6: The validation set image obtained in step 3 and the corresponding label image are used as the input of the segmentation network in step 3), and the hyperparameters of the segmentation network are adjusted and optimized according to the output result to obtain a prostate region segmentation network fused with prior information.

[0101] Step 7: The prostate region segmentation of the test set image preprocessed in step 2 is performed using the network obtained in step 6 to obtain the prostate peripheral zone and central zone segmentation result image. Figure 5 The prostate region segmentation result of the test set is shown in the following table.

[0102] As can be seen from the segmentation result obtained in step 7 above, the segmentation result of the method of the application basically fits the true label, overcoming the under-segmentation or over-segmentation problem caused by the fuzzy boundary information and non-uniform gray scale within the region, and being able to realize the relatively fine segmentation of the prostate region. The average DSC value of the prostate central zone and peripheral zone in the test set reaches 89.6% and 83.2%, respectively. Compared with other methods, the prostate magnetic resonance image region segmentation method fused with prior information according to the application achieves better segmentation performance.

Claims

1. A method for prostate magnetic resonance image region segmentation that integrates prior information, characterized in that, The method includes: Step 1: Obtain the prostate MRI dataset, divide it into three parts: training set, validation set and test set, and convert the prostate 3D MRI images into 2D axial sequence slice images; Step 2: Image preprocessing: Perform grayscale normalization, center cropping, size normalization, and contrast-limited adaptive histogram equalization on all sequence slice images in sequence. Step 3: Data augmentation: Perform data augmentation on the training set images and corresponding label images obtained in Step 2, specifically including: horizontal flipping, small-amplitude random translation, small-angle random rotation, and elastic deformation; Step 4: Construct the shape reconstruction network, including the following steps: 1) A shape reconstruction network is built using a convolutional autoencoder as the basic architecture; 2) Use the training set label images obtained in step 3 as input to the shape reconstruction network in step 1) to pre-train the shape reconstruction network; Step 5: Construct a segmentation network that integrates prior information, including the following steps: 1) Using U-Net as the basic architecture of the segmentation network, the shape reconstruction network is connected to U-Net as an extension network, and the spatial constraint module is embedded at the bottom of U-Net to build a segmentation network that integrates prior information; 2) Use the training set images obtained in step 3 and their corresponding label images as input to the segmentation network in step 1) to start training the prostate region segmentation network that integrates prior information. 3) Train the network through forward propagation, and then output the predicted probability map through the softmax classifier; calculate the loss according to the loss function, update the parameter values ​​in the network through backpropagation, repeat this process until the training loss value converges, complete one network training, and generate a preliminary prostate region segmentation network that integrates prior information. Step 6: Use the validation set images and their corresponding label images obtained in Step 3 as input to the network in Step 5 (Step 3). Adjust and optimize the network hyperparameters based on the output results to obtain a prostate region segmentation network with fused priors. Step 7: Use the segmentation network obtained in Step 6 to segment the prostate region of the preprocessed test set images in Step 2, and obtain the segmentation results of the peripheral zone and central zone of the prostate.

2. The method for prostate magnetic resonance image region segmentation fusing prior information as described in claim 1, characterized in that: In step 2, when performing grayscale normalization on the image, the normalized pixel value x * for: Where x represents the image pixel value, μ represents the average value of the image pixels, and σ represents the standard deviation of the image pixels.

3. The method for prostate magnetic resonance image region segmentation fusing prior information as described in claim 1, characterized in that: In step 2, the center of the original image is used as the cropping center, and the image is cropped in the center while retaining 2 / 3 of its length and width.

4. The method for prostate magnetic resonance image region segmentation fusing prior information as described in claim 1, characterized in that: In step 2, the image size is uniformly normalized to 240×240.

5. The method for prostate magnetic resonance image region segmentation fusing prior information as described in claim 1, characterized in that: Step 3 involves data augmentation of the image, including the following steps: 1) Horizontal Flip: Flips the image along its horizontal direction; 2) Small-amplitude random translation: Randomly translate the image in the four directions of up, down, left, and right, setting the translation range to 4 to 8 pixels; 3) Small-angle random rotation: Rotate the image randomly with the image center as the origin, and set the rotation angle range to -15° to 15°; 4) Elastic deformation: A displacement vector is randomly generated for each pixel in the image. A Gaussian function with a mean of 0 and a standard deviation of σ is used to perform a Gaussian convolution operation on the randomly generated displacement vector. The result is then applied to the original image to obtain the image after elastic deformation, where σ ranges from 2 to 10.

6. The method for prostate magnetic resonance image region segmentation fusing prior information as described in claim 1, characterized in that: In step 5, the loss function in step 3) comes from three parts: the segmentation network, the shape reconstruction network, and the spatial constraint module. Its formula is described as follows: L=L Seg +oh SR L SR +oh SC L SC Among them, L Seg L SR and L SC Let ω represent the segmentation loss, shape reconstruction loss, and spatial constraint loss, respectively. SR and ω SC The weighting coefficients for shape reconstruction loss and spatial constraint loss are as follows: i) Segmentation Loss: An exponential logarithmic loss is used, which combines the weighted cross-entropy loss and the DSC loss. The formula is described as follows: L Seg =λ WCE L WCE +λ DSC L DSC Where, λ WCE and λ DSC L represents the balance coefficients of the weighted cross-entropy loss function and the DSC loss function, respectively. WCE and L DSC Let represent the weighted cross-entropy loss and DSC loss after exponential and logarithmic transformations, respectively, and their formulas are described below: in, N represents the total number of pixels in the image, and C represents the total number of segmentation categories. This represents the probability that pixel i is predicted to be of class c. γ represents the true value of pixel i belonging to class c. WCE and γ DSC These represent two hyperparameters used to control the nonlinearity of the two-part loss function; ii) Shape reconstruction loss: The shape reconstruction loss uses a convolutional autoencoder to regularize the segmentation results, and its formula is described as follows: Among them, R i This represents the reconstructed value of pixel i in the actual label. This represents the reconstructed value of pixel i in the predicted image after being processed by the segmentation network, ||·|| F The Frobenius norm of a matrix is ​​defined as the square root of the sum of the squares of the absolute values ​​of its elements. iii) Spatial constraint loss: Spatial constraint loss uses the spatial location information of slices to regularize the segmentation results, and its formula is described as follows: Among them, P i This indicates the spatial location of the actual label corresponding to slice i. This represents the spatial location predicted for slice i.

Images