Deep learning based uterine, endometrial and diverticulum segmentation method and system

Abstract

The application provides a uterus, endometrium and diverticulum segmentation method and system based on deep learning, comprising the following steps: S1, pre-processing collected MRI three-dimensional images to obtain pre-processed image data; S2, building an improved three-dimensional cascaded Unet network segmentation model; S3, training the improved three-dimensional cascaded Unet network segmentation model by using the pre-processed image data to obtain a trained improved three-dimensional cascaded Unet network segmentation model; and S4, segmenting the pre-processed MRI three-dimensional images to be segmented by using the trained improved three-dimensional cascaded Unet network segmentation model to obtain a uterus segmentation image, an endometrium segmentation image and a diverticulum segmentation image.

Description

Technical Field

[0001] This invention relates to the technical field of medical image processing and applications, specifically to a method and system for segmenting the uterus, endometrium, and diverticulum based on deep learning. Background Technology

[0002] With the development of modern medicine, medical image segmentation has become a crucial part of it. Before diagnosing a diverticulum, doctors often need to review a large number of images, searching for key areas in each MRI image, and then manually annotating and creating 3D models of the diverticulum portion in abdominal MRI images to aid in subsequent surgical planning and treatment evaluation. However, relying solely on doctors for this arduous task is clearly unwise. Layer-by-layer annotation is not only tedious but also requires precision, and the more precise the medical image, the more slices it contains. Annotating diverticula is even more challenging. Diverticula are caused by incomplete healing of incisions in the endometrium. They exist on the surface of the endometrium and are not clearly distinguishable from surrounding organs and tissues, lacking distinctive geometric features. Furthermore, relying solely on laboratory data lacks visual information, easily leading to missed diagnoses. Therefore, using deep learning to automatically segment MRI images of diverticula and integrating image and feature value information in a one-stop website can greatly reduce the workload of doctors while providing valuable auxiliary diagnostic information.

[0003] For traditional algorithms, the biggest limitation in image segmentation is feature selection. However, in deep learning models, the network can spontaneously learn the features of the input object. In recent years, Fully Convolutional Neural Networks (FCNs) have become the first choice for such problems due to their powerful feature detection and extraction. For example, the UNet network, proposed in 2015, quickly became a benchmark in the field of medical image segmentation due to its simple and successful structure, and has received much attention in recent years. VNet was subsequently developed in 2016, which added residual learning and increased network parameters to UNet, achieving better edge segmentation results than UNet. Other methods included adding dense connections, attention mechanisms, residual learning, changing the objective function from a mathematical perspective, or introducing the concept of probabilistic random fields. By maximizing the combination of existing optimization methods and strategies, all of these methods achieved state-of-the-art results at the time. However, some scholars disagree, because the success of neural networks is not limited to network optimization but also requires the design of the entire process. Data analysis, preprocessing, data augmentation, network design, and prediction design are all indispensable steps. Network design is certainly one of the most effective ways to showcase research findings, but it can also lead to overfitting and decreased robustness. Therefore, in real-time applications such as patient registration and consultation, network performance may be challenged. Conversely, designing other components to match the data type can achieve excellent results and high stability, which is crucial in medical diagnosis. However, effective segmentation models only facilitate doctors' annotation but cannot provide real-time feedback, and even with well-packaged interfaces, deep learning networks still require some experience to use.

[0004] Patent document CN113066093A (application number: 202110378833.1) discloses an automatic segmentation, typing, and thickness measurement method based on endometrial ultrasound images, including the following steps: 1) Acquire a set of endometrial ultrasound scan images, manually select standard longitudinal sections of the uterus, delineate the boundaries of the endometrium, and simultaneously create typing labels to establish training and testing sample sets; 2) Perform image preprocessing such as filtering on each image to remove noise and achieve image enhancement; 3) Construct deep learning models for segmenting endometrial regions and typing, and input them into the training set for training; 5) Perform the same image preprocessing on the test set images, input them into the trained deep learning models, perform segmentation, and obtain the segmented endometrial regions and typing results; 6) Calculate the endometrial thickness using digital image morphology algorithms and the segmentation results of endometrial ultrasound images. Summary of the Invention

[0005] In view of the deficiencies in the prior art, the purpose of this invention is to provide a method and system for segmenting the uterus, endometrium and diverticulum based on deep learning.

[0006] A deep learning-based method for segmenting the uterus, endometrium, and diverticulum according to the present invention includes:

[0007] Step S1: Preprocess the collected MRI 3D images to obtain preprocessed image data;

[0008] Step S2: Construct an improved 3D cascaded Unet network segmentation model;

[0009] Step S3: Train the improved 3D cascaded Unet network segmentation model using the preprocessed image data to obtain the trained improved 3D cascaded Unet network segmentation model.

[0010] Step S4: Use the trained and improved 3D cascaded Unet network segmentation model to segment the preprocessed MRI 3D image to be segmented, and obtain the uterine segmentation image, endometrial segmentation image and diverticulum segmentation image;

[0011] The three-dimensional cascaded Unet network segmentation model includes a first-level subnetwork, a second-level subnetwork, and a third-level subnetwork. The first-level subnetwork segments the uterine region from the three-dimensional abdominal MRI image. The second-level subnetwork further segments the uterine region obtained from the first subnetwork to obtain the endometrial region. The third-level subnetwork further segments the endometrial region segmented by the second subnetwork to obtain the diverticulum.

[0012] Preferably, step S1 employs:

[0013] Step S1.1: Perform downsampling and normalization processing on the collected MRI 3D images to obtain initial samples of a preset size;

[0014] Step S1.2: Perform data augmentation processing on the initial sample image.

[0015] Preferably, each sub-network level includes downsampling convolutional layers and upsampling convolutional layers; wherein, the downsampling convolutional layers are implemented using max-pooling and convolutions with a kernel size of 3*3*3; and the upsampling convolutional layers are implemented using convolutions with a kernel size of 3*3*3 and a stride of 2.

[0016] In this subnetwork, the nonlinear activation units use Leaky ReLU to prevent the derivative from being zero in the less-than-zero part. During backpropagation, some neuron parameters are not modified, which enhances the network's weight adjustment ability and makes it more robust.

[0017] Instance normalization is used to normalize the feature tensor, maintaining the independence of each sample and preventing gradient vanishing. This addresses the problem that medical image datasets consume a lot of GPU computing memory, making large-scale training impossible.

[0018] Each sub-network level uses Dice loss as its loss function, which is defined as:

[0019]

[0020] Where X represents the prediction result; Y represents the actual sample labeling result; and D represents the dice loss coefficient.

[0021] The predictions of each sub-network are calculated using a soft max layer, outputting a probability density map of the same size as the input image, which classifies the images within it as foreground and background; its definition is:

[0022]

[0023] Among them, S i This represents the output value of the i-th node; e i Indicates the predicted value; e j represents the actual value; j represents the number of categories.

[0024] Preferably, step S3 employs the following methods:

[0025] The preprocessed image data is input into the first-level sub-network, and the input image is used for training to obtain a low-resolution uterine image I1;

[0026] The center of the low-resolution uterine image I1 is calculated and mapped to the corresponding position in the original image to obtain the region where the uterus is located in the original image. Sample images are cropped from the original image according to a preset size, and data augmentation processing is performed on the cropped sample images. The data is then input into the second-level sub-network for training to obtain the uterine segmentation image I2.

[0027] The center of the low-resolution uterine image is calculated based on the low-resolution uterine image I1. The uterine segmentation image I2 is mapped to the relative position in the original image to obtain the uterine segmentation image of the original image size. The center point size of the uterus is recalculated. Using the recalculated uterine center point size, a sample image containing the endometrium is cropped from the original image according to a preset size. The cropped sample image containing the endometrium is then subjected to data augmentation processing and input into the third-level sub-network for training to obtain the endometrial segmentation image I3.

[0028] Based on the uterine center point calculated from the uterine segmentation image I2, the endometrial segmentation image I3 is mapped to the relative position in the original image to obtain an endometrial segmentation image of the original image size. The size of the endometrial center point is calculated. Using the size of the endometrial center point, a sample image containing the diverticulum is cropped from the original image according to a preset size. The sample image containing the diverticulum is then subjected to data augmentation processing and input into the fourth-level network for training to obtain the diverticulum segmentation image I4.

[0029] Preferably, the length, width, depth, and volume of the diverticulum are obtained by calculating its smallest three-dimensional bounding rectangle based on the segmented image I4.

[0030] Preferably, seeding is performed on the segmented image I4 of the diverticulum to obtain the largest three-dimensional connected region volume; the convex hull of the three-dimensional point set is calculated using the Sklansky algorithm to obtain the minimum convex polygon, and then the minimum bounding rectangle is obtained using the rotating caliper algorithm; the depth, length, and width of the diverticulum are obtained by calculating the length, width, and height of the minimum bounding rectangle; and the volume of the diverticulum is obtained by counting the voxels within the minimum bounding rectangle.

[0031] Preferably, the distance TAM between the diverticulum and the uterus in the sagittal two-dimensional slice is calculated based on the uterine segmentation image I2 and the diverticulum segmentation image I4; and the distances TRM1 and TRM2 between the two ends of the diverticulum and the uterus in the same sagittal two-dimensional slice are calculated.

[0032] Preferably, based on the uterine segmentation image I2 and the diverticulum segmentation image I4, the average number of pixels of the segmented objects in each slice is calculated, and slice images with values ​​greater than the average are selected. In the selected slice images, the maximum contour of the uterus and diverticulum in the binary image is found in order from left to right and from top to bottom. The distance from all diverticulum contour points to the uterine contour points is calculated and sorted to obtain the minimum distance TAM. The distance between all diverticulum contour points is calculated and sorted to obtain the two endpoints of the maximum distance. Then, the shortest distance from the two endpoints to the uterine contour points is calculated to obtain TRM1 and TRM2.

[0033] Preferably, the segmented images I2 (uterus segmentation), I3 (endometrial segmentation), and I4 (diverticulum segmentation) of the uterus, endometrium, and diverticulum are used to perform 3D modeling using the Marching cubes algorithm to obtain the perspective relationship and relative positional relationship between the three organ components.

[0034] A deep learning-based system for segmenting the uterus, endometrium, and diverticulum, according to the present invention, comprises:

[0035] Module M1: Preprocesses the collected MRI 3D images to obtain processed image data;

[0036] Module M2: Constructs an improved 3D cascaded Unet network segmentation model;

[0037] Module M3: Trains the improved 3D cascaded Unet network segmentation model using preprocessed image data to obtain the trained improved 3D cascaded Unet network segmentation model;

[0038] Module M4: The pre-processed 3D cascaded Unet network segmentation model is used to segment the 3D MRI images to be segmented, resulting in segmented images of the uterus, endometrium, and diverticulum.

[0039] The three-dimensional cascaded Unet network segmentation model includes a first-level subnetwork, a second-level subnetwork, and a third-level subnetwork. The first-level subnetwork segments the uterine region from the three-dimensional abdominal MRI image. The second-level subnetwork further segments the uterine region obtained from the first subnetwork to obtain the endometrial region. The third-level subnetwork further segments the endometrial region segmented by the second subnetwork to obtain the diverticulum.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. This invention starts with the structure of the uterus in the lower abdomen of the human body, analyzes its shape and location characteristics, and connects them with the endometrium to construct a reasonable segmentation strategy, proposing a cascaded Unet network segmentation method. Then, a detailed analysis is conducted on the network construction, framework, and data preparation, resulting in a reasonable cascaded segmentation network. The cascaded Unet network segmentation method far surpasses commonly used large image segmentation training strategies, whether in terms of 3D spatial understanding, segmentation accuracy, or network generalization. Since medical image data is often very limited, through the cascaded segmentation network training method, we train each sub-network to focus on only one segmentation target; each level of sub-network further segments based on the segmentation of the previous level, thereby improving the overall segmentation accuracy of the network.

[0042] 2. To address the issue of limited data volume, we optimized the overfitting layers of the network by incorporating instance normalization to ensure that the features of each MRI image can be learned by the network. Furthermore, in clinical diagnosis, poorly acquired MRI images are inevitable. Cascaded segmentation networks effectively avoid performance degradation caused by unclear local images, whereas a single multi-object segmentation network is prone to performance degradation due to the inability to segment a particular object. In addition, clinically acquired MRI images are typically large in size; training a single multi-object segmentation network would require significant GPU computing resources and computation time, failing to meet the real-time requirements of clinical applications.

[0043] 3. By combining deep learning with web pages, this system closely aligns with practical user needs. Users require no coding experience; simply clicking a few buttons provides access to relevant information. The 3D perspective and relative positions of the uterus, endometrium, and diverticulum are displayed in real-time on the webpage. Relevant characteristic values ​​of the diverticulum are returned in list format. This one-stop integration of effective diagnostic information via web pages assists doctors in real-time diagnosis, significantly reducing their workload. Attached Figure Description

[0044] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0045] Figure 1 This is a flowchart of a diverticulum-assisted diagnostic system.

[0046] Figure 2 This is a diagram of a segmented cascaded network structure.

[0047] Figure 3 This is a schematic diagram of the network structure at each level. Detailed Implementation

[0048] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0049] Example 1

[0050] A deep learning-based method for segmenting the uterus, endometrium, and diverticulum according to the present invention includes:

[0051] Step S1: Preprocess the collected MRI 3D images to obtain preprocessed image data;

[0052] Step S2: Construct an improved 3D cascaded Unet network segmentation model;

[0053] Step S3: Train the improved 3D cascaded Unet network segmentation model using the preprocessed image data to obtain the trained improved 3D cascaded Unet network segmentation model.

[0054] Step S4: Use the trained and improved 3D cascaded Unet network segmentation model to segment the preprocessed MRI 3D image to be segmented, and obtain the uterine segmentation image, endometrial segmentation image and diverticulum segmentation image;

[0055] The three-dimensional cascaded Unet network segmentation model includes a first-level subnetwork, a second-level subnetwork, and a third-level subnetwork. The first-level subnetwork segments the uterine region from the three-dimensional abdominal MRI image. The second-level subnetwork further segments the uterine region obtained from the first subnetwork to obtain the endometrial region. The third-level subnetwork further segments the endometrial region segmented by the second subnetwork to obtain the diverticulum.

[0056] More specifically, the network model is trained using complete abdominal MRI images and their corresponding segmented images as the training set; the structure of each network level is the same, such as... Figure 3 As shown, each layer includes four downsampling convolutional layers and four upsampling convolutional layers. The downsampling convolutional layers are implemented using max-pooling and convolutions with a kernel size of 3*3*3. The upsampling convolutional layers are implemented using convolutions with a kernel size of 3*3*3 and a stride of 2.

[0057] In this network, Leaky ReLU is used for the nonlinear activation units in each level to prevent the derivative from remaining zero for parts less than zero. This ensures that some neuron parameters remain unchanged during backpropagation, enhancing the network's weight adjustment capability and robustness. Instance normalization is used to normalize the feature tensors during training, maintaining the independence of each sample, preventing gradient vanishing, and addressing the issue that medical image datasets consume significant GPU memory, hindering large-scale training. The mean μ and variance σ of each image are calculated along each channel. 2 And normalize the input.

[0058] The training model uses dice loss as the loss function, which is defined as follows:

[0059]

[0060] Where D represents the dice loss coefficient, X represents the prediction result, and Y represents the actual sample labeling result.

[0061] Each layer of the network calculates its predictions using a soft max layer, outputting a probability density map of the same size as the input image, which then classifies the images within it as foreground or background. This map is defined as follows:

[0062]

[0063] Among them, S i This represents the output value of the i-th node; e i Indicates the predicted value; e j represents the actual value; j represents the number of categories.

[0064] Specifically, step S1 employs the following:

[0065] Step S1.1: Perform downsampling and normalization processing on the collected MRI 3D images to obtain initial samples of a preset size;

[0066] Step S1.2: Perform data augmentation processing on the initial sample image, including random rotation, adding noise, etc.

[0067] Specifically, step S3 employs the following:

[0068] The preprocessed image data is input into the first-level sub-network, and the input image is used for training to obtain a low-resolution uterine image I1;

[0069] The center of the low-resolution uterine image I1 is calculated and mapped to the corresponding position in the original image to obtain the region where the uterus is located in the original image. Sample images are cropped from the original image according to a preset size, and data augmentation processing is performed on the cropped sample images, including random rotation and noise addition. The data is then input into the second-level sub-network for training to obtain the uterine segmentation image I2.

[0070] The center of the low-resolution uterine image I1 is calculated, and the relative position of the segmented uterine image I2 is mapped onto the original image to obtain a segmented uterine image of the original size. The center point size of the uterus is recalculated. Using the recalculated center point size, a sample image containing the endometrium is cropped from the original image according to a preset size. Data augmentation processing is performed on the cropped sample image containing the endometrium, including random rotation and noise addition. The data is then input into the third-level sub-network for training to obtain the endometrial segmentation image I3.

[0071] Based on the uterine center point calculated from the uterine segmentation image I2, the endometrial segmentation image I3 is mapped to the relative position in the original image to obtain an endometrial segmentation image of the original image size. The size of the endometrial center point is calculated. Using the size of the endometrial center point, sample images containing diverticula are cropped from the original image according to a preset size. Data augmentation processing is performed on the sample images containing diverticula, including random rotation and noise addition. The data is then input into the fourth-level network for training to obtain the diverticulum segmentation image I4.

[0072] Specifically, the length, width, depth, and volume of the diverticulum are obtained by calculating its minimum three-dimensional bounding rectangle based on the segmented image I4.

[0073] Specifically, seeding of eight neighborhoods is performed on the segmented image I4 of the diverticulum to obtain the largest three-dimensional connected region volume; the Sklansky algorithm is used to calculate the convex hull of the three-dimensional point set to obtain the minimum convex polygon, and then the rotating caliper algorithm is used to obtain the minimum bounding rectangle; the depth, length, and width of the diverticulum are obtained by calculating the length, width, and height of the minimum bounding rectangle; the volume of the diverticulum is obtained by counting the voxels within the minimum bounding rectangle.

[0074] Specifically, based on the uterine segmentation image I2 and the diverticulum segmentation image I4, the closest distance TAM between the diverticulum and the uterus in the sagittal two-dimensional slice is calculated; and the closest distances TRM1 and TRM2 between the two ends of the diverticulum and the uterus in the same sagittal two-dimensional slice are calculated.

[0075] Specifically, based on the uterine segmentation image I2 and the diverticulum segmentation image I4, the average number of pixels of the segmented objects in each slice is calculated, and slice images with values ​​greater than the average are selected. In the selected slice images, the maximum contour of the uterus and diverticulum in the binary image is found in order from left to right and from top to bottom. The distance from all diverticulum contour points to the uterine contour points is calculated and sorted to obtain the minimum distance TAM. The distance between all diverticulum contour points is calculated and sorted to obtain the two endpoints of the maximum distance. Then, the shortest distance from the two endpoints to the uterine contour points is calculated to obtain TRM1 and TRM2.

[0076] Specifically, the segmented images of the uterus (I2), endometrium (I3), and diverticulum (I4) obtained from the segmentation are used to perform 3D modeling of the three-dimensional images of the uterus, endometrium, and diverticulum, and the Marching cubes algorithm is used to obtain the perspective relationship and relative positional relationship between the three organ components.

[0077] Furthermore, by using 3D image segmentation, doctors can obtain accurate 3D organ models and the relative positional relationships between various components, and construct a clinically applicable diverticulum-assisted diagnostic system: the 3D segmented images of the uterus, endometrium, and diverticulum obtained from the segmentation are displayed on a webpage in real time, and the feature parameters required for diverticulum diagnosis are calculated. The system also quickly integrates the 3D image information and geometric feature parameter information of the diverticulum region required by doctors for diagnosis, thereby assisting doctors in diagnostic analysis and improving the accuracy and efficiency of clinical diagnosis of diverticulum.

[0078] A deep learning-based system for segmenting the uterus, endometrium, and diverticulum, according to the present invention, comprises:

[0079] Module M1: Preprocesses the collected MRI 3D images to obtain processed image data;

[0080] Module M2: Constructs an improved 3D cascaded Unet network segmentation model;

[0081] Module M3: Trains the improved 3D cascaded Unet network segmentation model using preprocessed image data to obtain the trained improved 3D cascaded Unet network segmentation model;

[0082] Module M4: The pre-processed 3D cascaded Unet network segmentation model is used to segment the 3D MRI images to be segmented, resulting in segmented images of the uterus, endometrium, and diverticulum.

[0083] The three-dimensional cascaded Unet network segmentation model includes a first-level subnetwork, a second-level subnetwork, and a third-level subnetwork. The first-level subnetwork segments the uterine region from the three-dimensional abdominal MRI image. The second-level subnetwork further segments the uterine region obtained from the first subnetwork to obtain the endometrial region. The third-level subnetwork further segments the endometrial region segmented by the second subnetwork to obtain the diverticulum.

[0084] Example 2

[0085] Example 2 is a preferred example of Example 1.

[0086] like Figure 1-2 As shown, this invention provides a deep learning-based method for diverticulum segmentation and an auxiliary diagnostic method, comprising the following steps:

[0087] Step 1: Categorize the uploaded data according to the NMR number;

[0088] Step 2: Preprocess the collected data, including normalization, downsampling, random rotation, and random noise addition;

[0089] Step 3: Build a uterine segmentation model based on the processed downsampled data and train it to obtain the segmentation results;

[0090] Step 4: Calculate the uterine center in the downsampled image, map the corresponding position of the uterine center in the original image proportionally, crop the sample image according to the preset size, perform data augmentation on the sample image, build a second-level uterine segmentation network, obtain the segmented image of the uterus in the original image, and further calculate the uterine center.

[0091] Step 5: Based on the obtained uterine center position and preset size, crop a sample image containing the endometrium from the original image, perform data augmentation on the sample image, and construct a segmentation network for the endometrium. The segmented image of the endometrium in the original image is then obtained.

[0092] Step 6: Based on the obtained endometrial image, calculate the endometrial center, cut out a pre-defined diverticulum segmentation network, and segment to obtain the diverticulum image.

[0093] Step 7: Based on the obtained 3D segmentation image of the diverticulum, calculate the relevant feature parameters: length, width, depth, and volume;

[0094] Step 8: Based on the segmented images of the uterus and diverticulum, calculate TAM, TRM1, and TRM2 respectively;

[0095] Step 9: Based on the segmented three-dimensional images of the uterus, endometrium, and diverticulum, perform three-dimensional reconstruction to display perspective relationships and relative positional relationships;

[0096] Step 10: Return the relevant feature value information to the webpage for display.

[0097] Specifically, in step 1, the collected data is classified to distinguish the preoperative and postoperative MRI images of the same patient, which facilitates comparison and diagnosis by doctors.

[0098] Specifically, in step 2, the MRI image is first normalized, then the 256*256*N MRI image is downsampled to half the original size, and the slice dimensions are padded to create a 128*128*128 three-dimensional image for storage. Due to the limited data volume of medical images, to enhance the network's generalization ability, the data is augmented by randomly adding noise and performing random angle rotations while ensuring the organs remain undistorted.

[0099] Specifically, in step 4, after obtaining the downsampled segmented uterine image, the image is upsampled and restored to a 256*256*256 three-dimensional image. Then, the center of the uterus is calculated, and a 128*128*128 slice is cropped from the original resolution image based on this center to obtain the original resolution uterine dataset.

[0100] Specifically, the constructed segmentation network is an improvement upon UNet. The overall structure of the sub-network is a symmetrical encoding and decoding process. For the input feature image, each layer undergoes downsampling after two 3x3x3 convolutions, halving the image size and doubling the feature image size. After four downsampling iterations, decoding (upsampling) begins. Unlike typical networks, this upsampling uses convolution instead of pooling, which improves computational speed while reducing computational cost, and more importantly, increases image resolution. Finally, a 1x1 fully convolutional layer outputs a probability density map of the same size as the original image. Each sub-network uses Leaky ReLU as the non-linear activation unit and instance normalization instead of batch normalization. Dice Loss is used as the loss function to compensate for the small size of the medical target segmentation image and its disproportionate relationship with the surrounding foreground information.

[0101] Specifically, in step 5, after obtaining the accurately segmented original resolution uterine image, the position of the 128*128*128 slice image in the original image is first located based on the uterine center calculated in step 4, and then restored to obtain a 256*256*N MRI image. The uterine center is then recalculated. The uterine center obtained from the downsampling segmentation and the recalculated uterine center will not be far apart in terms of Euclidean distance, thus ensuring the integrity of the uterine segmentation. Then, based on the newly calculated uterine center, an 80*80*128 endometrial dataset is cropped from the original resolution image. The 80*80 size is the average size of the endometrium obtained from statistical analysis of the collected dataset.

[0102] Specifically, in step 6, after obtaining the 3D segmented image of the diverticulum, some scattered points will exist. Therefore, it is necessary to first find the maximum 3D connected component to eliminate randomness. In this embodiment, an eight-neighbor seed growth method is used to obtain the maximum connected component. Then, the 3D convex set of the coordinate points of the connected component is calculated according to the Sklansky algorithm, and the three points are converted into convex polygons. Next, the minimum bounding rectangle of the convex polygon is calculated, and the shape parameters and volume of the diverticulum are obtained based on this minimum bounding rectangle.

[0103] Specifically, in step 7, after obtaining the three-dimensional segmented images of the uterus and diverticulum, the average number of pixels that each slice should contain is calculated based on the total number of pixels of both, ensuring that the selected slices contain a certain amount of organ tissue and obtaining more general statistical results. Then, in the candidate slices, the contour points of the uterus and diverticulum are calculated, and the feature values ​​between the diverticulum and the uterus are obtained based on the Euclidean distance and the distance between them.

[0104] Specifically, the three-dimensional segmented images of the uterus, diverticulum, and endometrium obtained in step 8 do not provide a very direct visual experience. Therefore, it is necessary to perform three-dimensional modeling of these three components to reconstruct their relative positions within the body and the visual information of key locations. Commonly used MRI image interpretation software typically represents three-dimensional information by displaying three cross-sections, which is not intuitive enough. Therefore, this embodiment uses the Marching cubes algorithm for three-dimensional modeling, displaying the reconstructed three-dimensional image in real time on the MRI data uploaded by the doctor.

[0105] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0106] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A deep learning-based uterine, endometrial, and diverticulum segmentation method, characterized by, The application relates to a method for segmenting MRI three-dimensional images, and belongs to the field of medical image processing. Step S1: preprocessing collected MRI three-dimensional images to obtain preprocessed image data; Step S2: building an improved three-dimensional cascaded Unet network segmentation model; Step S3: training the improved three-dimensional cascaded Unet network segmentation model by using the preprocessed image data to obtain a trained improved three-dimensional cascaded Unet network segmentation model; Step S4: segmenting the preprocessed MRI three-dimensional images to be segmented by using the trained improved three-dimensional cascaded Unet network segmentation model to obtain a uterus segmentation image, an endometrium segmentation image and a diverticulum segmentation image; The three-dimensional cascaded Unet network segmentation model comprises a first-level subnetwork, a second-level subnetwork and a third-level subnetwork; the first-level subnetwork segments a uterus region from an abdominal MRI three-dimensional image; the second-level subnetwork continues to segment an endometrium region from the uterus region obtained from the first-level subnetwork; and the third-level subnetwork continues to segment a diverticulum from the endometrium region segmented by the second-level subnetwork; The length, width, depth and volume of the diverticulum are calculated according to the diverticulum segmentation image I4; The eight-neighborhood seed growth is performed on the diverticulum segmentation image I4 to obtain the volume of the largest three-dimensional connected domain; the Sklansky algorithm is used to calculate the three-dimensional point set convex hull to obtain the minimum convex polygon, and then the rotating caliper algorithm is used to obtain the minimum circumscribed rectangle; the depth, length and width of the diverticulum are obtained by calculating the length, width and height of the minimum circumscribed rectangle; and the volume of the diverticulum is obtained by counting the voxels in the minimum circumscribed rectangle. 2.The deep learning based uterus, endometrium and diverticulum segmentation method of claim 1, wherein, The step S1 adopts: Step S1.1: performing down-sampling and normalization processing on the collected MRI three-dimensional images to obtain initial samples with a preset size; Step S1.2: performing data enhancement processing on the initial sample images. 3.The deep learning based uterus, endometrium and diverticulum segmentation method of claim 1, wherein, Each subnetwork comprises a down-sampling convolutional layer and an up-sampling convolutional layer; wherein the down-sampling convolutional layer is realized by max-pooling and a convolution kernel of 3*3*3; and the up-sampling convolutional layer is realized by a convolution kernel of 3*3*3 and a step of 2; The non-linear activation unit in each subnetwork uses Leaky ReLu, avoids the derivative of the part less than zero being zero all the time, and enhances the weight adjustment ability and robustness of the network; The feature tensor is normalized by using instance normaliztion, the independence of each sample is maintained, the gradient vanishing is prevented, and the problem that the medical image data set occupies a large amount of GPU computing memory and cannot be trained in large batches is solved; Each subnetwork uses dice loss as a loss function, and the definition is as follows: Wherein, X represents a prediction result; Y represents a true sample label result; and D represents a dice loss dice coefficient; The prediction of each subnetwork is calculated by using a soft max layer, and a probability density graph with the same size as the input image is output, so that the foreground and background of the image are classified; and the definition is as follows: wherein S i represents the output value of the i-th node; e i represents the predicted value; e j represents the actual value; and j represents the number of categories of classification. 4.The deep learning based uterus, endometrium and diverticulum segmentation method of claim 1, wherein, The step S3 adopts: The pre-processed image data is input into the first-level sub-network, and the input image is trained to obtain a low-resolution uterus image I1; The center of the low-resolution uterus image I1 is calculated, and the corresponding position in the original image is mapped to obtain the region of the uterus in the original image. A sample image is cropped from the original image according to the preset size, and data enhancement processing is performed on the cropped sample image. The second-level sub-network is trained to obtain a uterus segmentation image I2; The center of the low-resolution uterus image I1 is calculated, and the corresponding position in the original image is mapped to obtain the region of the uterus in the original image. A sample image is cropped from the original image according to the preset size, and data enhancement processing is performed on the cropped sample image. The second-level sub-network is trained to obtain a uterus segmentation image I2; The center of the low-resolution uterus image I1 is calculated, and the corresponding position in the original image is mapped to obtain the region of the uterus in the original image. A sample image is cropped from the original image according to the preset size, and data enhancement processing is performed on the cropped sample image. The second-level sub-network is trained to obtain a uterus segmentation image I2; 5.The deep learning based uterus, endometrium and diverticulum segmentation method of claim 1, wherein, The center of the low-resolution uterus image I1 is calculated, and the corresponding position in the original image is mapped to obtain the region of the uterus in the original image. A sample image is cropped from the original image according to the preset size, and data enhancement processing is performed on the cropped sample image. The second-level sub-network is trained to obtain a uterus segmentation image I2; 6. The deep learning based uterus, endometrium and diverticulum segmentation method according to claim 5, characterized in that, The center of the low-resolution uterus image I1 is calculated, and the corresponding position in the original image is mapped to obtain the region of the uterus in the original image. A sample image is cropped from the original image according to the preset size, and data enhancement processing is performed on the cropped sample image. The second-level sub-network is trained to obtain a uterus segmentation image I2; 7.The deep learning based uterus, endometrium and diverticulum segmentation method of claim 1, wherein, The center of the low-resolution uterus image I1 is calculated, and the corresponding position in the original image is mapped to obtain the region of the uterus in the original image. A sample image is cropped from the original image according to the preset size, and data enhancement processing is performed on the cropped sample image. The second-level sub-network is trained to obtain a uterus segmentation image I2; 8. A system for implementing the method of uterine, endometrial and diverticulum segmentation based on deep learning according to any one of claims 1-7, characterized in that, The center of the low-resolution uterus image I1 is calculated, and the corresponding position in the original image is mapped to obtain the region of the uterus in the original image. A sample image is cropped from the original image according to the preset size, and data enhancement processing is performed on the cropped sample image. The second-level sub-network is trained to obtain a uterus segmentation image I2; The center of the low-resolution uterus image I1 is calculated, and the corresponding position in the original image is mapped to obtain the region of the uterus in the original image. A sample image is cropped from the original image according to the preset size, and data enhancement processing is performed on the cropped sample image. The second-level sub-network is trained to obtain a uterus segmentation image I2; The segmented uterus segmentation image I2, endometrial segmentation image I3, and diverticulum segmentation image I4 are used to perform three-dimensional modeling by using the Marching cubes algorithm to obtain the perspective relationship and relative position relationship between the three organ components. It comprises: Module M1: pre-processing the collected MRI three-dimensional image to obtain processed image data; Module M2: building an improved three-dimensional cascaded Unet network segmentation model; Module M3: training the improved three-dimensional cascaded Unet network segmentation model using the pre-processed image data to obtain a trained improved three-dimensional cascaded Unet network segmentation model; The module M4: using the trained improved three-dimensional cascade Unet network segmentation model to segment the preprocessed MRI three-dimensional image to be segmented, to obtain a uterus segmentation image, an endometrium segmentation image and a diverticulum segmentation image; The three-dimensional cascade Unet network segmentation model comprises a first-level subnetwork, a second-level subnetwork and a third-level subnetwork; the first-level subnetwork segments a uterus region from the abdominal MRI three-dimensional image; the second-level subnetwork continues to segment an endometrium region from the uterus region obtained by the first subnetwork; and the third-level subnetwork continues to segment a diverticulum from the endometrium region segmented by the second-level subnetwork.

Images