Automatic segmentation method and automatic segmentation system for pelvic fracture image

Abstract

The application discloses a kind of automatic segmentation method and automatic segmentation system of pelvic fracture image.The method comprises the following steps: obtaining pelvic fracture three-dimensional CT image, pre-processing the three-dimensional CT image, after processing, it is segmented by first convolutional neural network model, then it is registered to three-dimensional CT image, and the intensity information of non-bone tissue in registration image is shielded by Boolean operation, to obtain the three-dimensional CT image of bone tissue;The bone tissue three-dimensional CT image is segmented by second convolutional neural network model, to obtain all the main fragment images and secondary fragment group images of pelvis;The secondary fragment group image is solved and filtered by connected region, to obtain all the secondary fragment images of pelvis, and output all the main fragment images and secondary fragment images.The application realizes the full-automatic segmentation of pelvic fracture image, can accurately segment the main fragment image and secondary fragment image of each bone of pelvis, greatly simplifies the preoperative segmentation process in clinic, shortens the image processing duration.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image segmentation, and more particularly relates to an automatic segmentation method and system for pelvic fracture images. BACKGROUND

[0002] Pelvic fracture is a severe high-energy injury with a mortality rate of more than 50%, ranking first among all complex fractures. The complex pelvic structure and the surrounding muscle groups, ligaments, nerves, blood vessels and other tissues pose challenges to surgical planning and reduction tasks. In the fracture reduction surgery, the diagnosis, planning and other aspects of the fracture often need orthopedic experts to manually review the film for planning, and the planning process is time-consuming and laborious, and the results have strong subjectivity.

[0003] In recent years, robotic fracture reduction surgery has been researched and applied in clinical practice, successfully improving the reduction accuracy and reducing radiation exposure. Whether manual reduction or automatic reduction, accurate segmentation of the pelvic fracture is needed to find the best target position and restore the healthy shape of the pelvic bone. Due to the uncertainty of the shape and position of the bone fragments and the complexity of the collision fracture surface, it is challenging to segment the pelvic fragments from CT. Therefore, surgeons usually annotate the anatomical structure of the pelvic fracture in a semi-automatic way. First, the adaptive threshold method and region growing method are used to extract the bone region by adjusting the threshold and selecting the seed point. Then, the broken surface is manually delineated by outlining the fragments in the 3D view or even modifying the mask in a layer-by-layer manner. Usually, this tedious process may take more than 30 minutes, especially when the broken fragments collide or are not completely separated.

[0004] In order to provide users with more effective tools, someone has proposed a semi-automatic graph cut method based on continuous maximum flow for pelvic fracture segmentation, but it still needs manual selection of seed points and tracking errors. Fully automatic maximum flow segmentation based on graph cut and boundary enhancement filter is useful when fragments are separated, but often fails when fragments collide or are compressed. Learning-based bone segmentation has been successfully applied to various anatomies, including pelvis, ribs, skull, etc. Someone has proposed some deep learning methods to detect cracks, but the results of these methods cannot provide a fully automated solution for subsequent operations. In FracNet, rib fracture detection is formulated as a segmentation problem, but the final Dice is 71.5%, which only roughly depicts the fracture site without depicting the crack surface, and learning-based methods that directly handle crack segmentation are rarely studied.

[0005] For learning-based methods, fracture segmentation is still a challenging task because, compared with more common organ / tumor segmentation tasks, the model can implicitly learn the shape prior of the object, but due to the great variation of fracture types and shapes, it is difficult to learn the shape information of the bone fragments; the fracture surface itself can exhibit a variety of forms such as large space (fragment isolation movement), small gap (fragment isolation does not move), crease (fragment is not completely isolated), compression (fragment collision) and their combinations, resulting in large differences in image intensity distribution around the fracture; the change in the number of bone fragments in pelvic fractures makes it difficult to specify a consistent labeling strategy that applies to every type and case.

[0006] The information disclosed in the Background section of the present invention is only intended to deepen the understanding of the general background of the present invention and should not be regarded as acknowledging or implying in any form that this information constitutes prior art known to those skilled in the art. SUMMARY

[0007] The purpose of the present invention is to provide an automatic segmentation method and system for pelvic fracture images, which realizes full-automatic segmentation of pelvic fracture images and can accurately segment the main fragment image and the secondary fragment image of each bone of the pelvis, greatly simplifying the preoperative segmentation process in clinical practice and shortening the image processing time.

[0008] To achieve the above-mentioned purpose, the present invention provides an automatic segmentation method and system for pelvic fracture images.

[0009] According to the first aspect of the present invention, an automatic segmentation method for pelvic fracture images is provided, comprising:

[0010] obtaining a three-dimensional CT image of a pelvic fracture;

[0011] preprocessing the three-dimensional CT image of the pelvic fracture;

[0012] segmenting the three-dimensional CT image after preprocessing by a first convolutional neural network model after training, registering the three-dimensional CT image after segmentation to the three-dimensional CT image, and shielding the intensity information of non-bone tissue in the three-dimensional CT image by Boolean operation to obtain a three-dimensional CT image of pelvic bone tissue;

[0013] segmenting the three-dimensional CT image of pelvic bone tissue by a second convolutional neural network model after training to obtain all main fragment images and secondary fragment group images of the pelvic fracture;

[0014] solving the connected region of the secondary fragment group image, and filtering according to the set filtering condition to obtain all secondary fragment images of the pelvic fracture.

[0015] output all of the primary patch images and the secondary patch images.

[0016] Optionally, the preprocessing comprises:

[0017] The pelvic fracture three-dimensional CT image is cropped and format-converted to obtain an input file in NIFTI format;

[0018] The CT value data in the input file is subjected to Z-Score standardization.

[0019] Optionally, the first convolutional neural network model comprises:

[0020] The 3D-UNet pelvic anatomical segmentation network model comprises a first convolutional neural network and a second convolutional neural network;

[0021] The first convolutional neural network and the second convolutional neural network each comprise:

[0022] 3 layers of up-sampling layers and 3 layers of down-sampling layers, and the up-sampling layers and the down-sampling layers in each layer are connected by skip connection;

[0023] The loss function of the network model is a cross-entropy-Dice hybrid loss function, and five-fold cross-validation is used in the training process of the network model to verify the stability of the model.

[0024] Optionally, the second convolutional neural network model comprises:

[0025] The 3D-UNet multi-scale pelvic fracture segmentation network model comprises 5 layers of up-sampling layers and 5 layers of down-sampling layers, and the up-sampling layers and the down-sampling layers in each layer are connected by skip connection;

[0026] In addition to the up-sampling layer with the lowest resolution, an auxiliary loss function is added to each up-sampling layer to realize multi-scale deep supervision;

[0027] Five-fold cross-validation is used in the training process of the network model to verify the stability of the model.

[0028] Optionally, the loss function of the second convolutional neural network model is a cross-entropy-Dice hybrid loss function based on fracture distance feature map weighting;

[0029] Before the training starts, the primary patch and secondary patch groups of the training image are manually labeled, and according to the labeling result, a filter with a side length of 3*3*3 is used to traverse the training image to obtain the fracture surface of the training image.

[0030] calculate distances from each voxel containing bone tissue intensity information in the training image to the fracture surface to obtain a fracture distance feature map;

[0031] calculate the fracture distance feature map by a nonlinear mapping function to obtain distance feature weights.

[0032] Optionally, the expression of the total loss function of the second convolutional neural network model is:

[0033] Loss=L dice +L cross-Entropy ;

[0034] Wherein, Loss is the total loss function, L dice is a 0.5 times Dice loss function based on the fracture distance feature map weighting, L cross-Entropy is a 0.5 times cross entropy loss function based on the fracture distance feature map weighting.

[0035] Optionally, in the initial stage of training the 3D-UNet multi-scale pelvic fracture segmentation network model, a smooth transition is adopted, the network model maintains attention to global features in the early stage of training, and gradually shifts the focus of attention to the fracture site as the training progresses.

[0036] The smooth transition is to dynamically adjust the proportion of the fracture distance weight in the overall weight matrix according to the iteration number of the training.

[0037] Optionally, in the initial stage of training the second convolutional neural network model, the initial learning rate is set to 0.001, and the learning rate gradually decreases as the iteration number of the training increases.

[0038] Optionally, the set filtering conditions include:

[0039] The connected region in which the number of voxels in the connected region is less than 50.

[0040] According to the second aspect of the application, an automatic segmentation system for pelvic fracture images is provided for performing the automatic segmentation method of the pelvic fracture images, and the system comprises:

[0041] An acquisition module is configured to acquire a three-dimensional CT image of a pelvic fracture;

[0042] A preprocessing module is configured to preprocess the three-dimensional CT image of the pelvic fracture;

[0043] a first image segmentation module, configured to segment the three-dimensional CT image after the preprocessing by using a first convolutional neural network model after training, register the segmented three-dimensional CT image to the three-dimensional CT image, and shield the intensity information of non-bone tissue in the three-dimensional CT image by using a Boolean operation to obtain a three-dimensional CT image of pelvic bone tissue;

[0044] a second image segmentation module, configured to segment the three-dimensional CT image of pelvic bone tissue by using a second convolutional neural network model after training to obtain a main fragment image and a secondary fragment group image of pelvic fracture;

[0045] a connected region solving and filtering module, configured to solve the connected region of the secondary fragment group image and filter according to a set filtering condition to obtain a secondary fragment of each pelvic fracture;

[0046] an output module, configured to output all the main fragment images and the secondary fragment images.

[0047] The application has the advantages that: firstly, a first convolutional neural network model is established according to a pelvic fracture three-dimensional CT database, and the intensity information of non-bone tissue in the three-dimensional CT image is shielded by using a Boolean operation to automatically segment the three-dimensional CT images of the left ilium, the right ilium and the sacrum from the three-dimensional CT image of pelvic fracture, then a second convolutional neural network model based on a fracture distance feature map is used to segment the three-dimensional CT images of the left ilium, the right ilium and the sacrum to segment the main fragment and the secondary fragment group of each bone from the three-dimensional CT image of each bone, and then the connected region solving and filtering of the secondary fragment group are performed to obtain the secondary fragment of each bone, so that the full-automatic segmentation of the pelvic fracture image is realized, the main fragment image and the secondary fragment image of each bone of the pelvis can be accurately segmented, the preoperative segmentation process in the clinic is greatly simplified, and the image processing time is shortened.

[0048] The system of the application has other characteristics and advantages that will become apparent from and / or will be elucidated with respect to the drawings accompanying and the detailed description that follows, which together illustrate by way of example the principles of the application. BRIEF DESCRIPTION OF DRAWINGS

[0049] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which like reference characters refer to like parts throughout and in which:

[0050] Figure 1 A flow chart showing the steps of an automatic segmentation method of a pelvic fracture image according to the present application is shown.

[0051] Figure 2 A flow chart showing the steps of the automatic segmentation method of a pelvic fracture image of embodiment 1 of the present application is shown.

[0052] Figure 3 A workflow diagram of the 3D-UNet pelvic anatomical segmentation network model of embodiment 1 of the present application is shown.

[0053] Figure 4 A structural schematic diagram of the 3D-UNet pelvic anatomical segmentation network model of embodiment 1 of the present application is shown.

[0054] Figure 5 A structural schematic diagram of the 3D-UNet multi-scale pelvic fracture segmentation network model of embodiment 1 of the present application is shown.

[0055] Figure 6 The main fragment images and the secondary fragment images of all left iliac bones, right iliac bones and sacral bones of embodiment 1 of the present application are shown.

[0056] Figure 7 A structural schematic diagram of the automatic segmentation system of a pelvic fracture image of embodiment 2 of the present application is shown. DETAILED DESCRIPTION

[0057] The present application will be described in more detail by referring to the attached drawings. Although preferred embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0058] As shown in Figure 1 An automatic segmentation method of a pelvic fracture image according to the present application comprises:

[0059] acquiring a three-dimensional CT image of a pelvic fracture;

[0060] preprocessing the three-dimensional CT image of the pelvic fracture;

[0061] segmenting the three-dimensional CT image after preprocessing by a first convolutional neural network model after training, registering the three-dimensional CT image after segmentation to the three-dimensional CT image, and shielding the intensity information of non-bone tissue in the three-dimensional CT image by Boolean operation to obtain a three-dimensional CT image of pelvic bone tissue;

[0062] segmenting the three-dimensional CT image of pelvic bone tissue by a second convolutional neural network model after training to obtain all main fragment images and secondary fragment group images of the pelvic fracture;

[0063] The connected region solving is performed on the sub-fragment group image, and filtering is performed according to the set filtering condition to obtain all the sub-fragment images of the pelvic fracture.

[0064] All the main fragment images and sub-fragment images are outputted.

[0065] Specifically, the three-dimensional CT image of the pelvic fracture of the patient is acquired, and the three-dimensional CT image of the pelvic fracture is preprocessed. Firstly, the three-dimensional CT image of the pelvic fracture is converted into an input file in NIFTI format meeting the requirements through cropping and format conversion, and the input file in NIFTI format is used for a neural network; then, the CT value data in the input file is subjected to Z-Score standardization, which can accelerate the solving speed of gradient descent and improve the convergence speed of the model; the input file subjected to standardization is inputted into a first convolutional neural network model, and the neural network model is established based on a convolutional neural network according to a CT image database, for example, a convolutional neural network based on U-Net, V-Net and SegNet; the input file subjected to standardization is subjected to image segmentation through the first convolutional neural network model, the segmented image is registered with the three-dimensional CT image of the pelvic fracture, and the intensity information of non-bone tissue in the three-dimensional CT image of the pelvic fracture is shielded through Boolean operation to obtain a three-dimensional CT image of pelvic bone tissue, which is a three-dimensional CT image of left iliac bone tissue, a three-dimensional CT image of right iliac bone tissue and a three-dimensional CT image of sacral bone tissue; the three-dimensional CT images of the left iliac bone, the right iliac bone and the sacral bone are inputted into a second convolutional neural network model for segmentation to obtain the main fragment images and the sub-fragment group images of the left iliac bone, the right iliac bone and the sacral bone, the connected region solving is performed on the sub-fragment group image, and the connected region not meeting the requirements, i.e. small sub-fragments having no influence on the operation, is filtered out to obtain all the sub-fragment images of the left iliac bone, the right iliac bone and the sacral bone, and finally all the main fragment images and sub-fragment images are outputted.

[0066] In one example, the preprocessing includes:

[0067] The three-dimensional CT image of the pelvic fracture is converted into an input file in NIFTI format through cropping and format conversion;

[0068] The CT value data in the input file is subjected to Z-Score standardization.

[0069] Specifically, firstly, the three-dimensional CT image of the pelvic fracture is cropped to a proper size and all the images of the pelvis are retained to reduce the image size, reduce irrelevant information and reduce memory consumption; then, the input file in NIFTI format is obtained through format conversion; finally, the CT value data in the input file in NIFTI format is subjected to Z-Score standardization to accelerate the solving speed of gradient descent and improve the convergence speed of the model, for example, Min-Max standardization, decimal scaling standardization, mean normalization and other standardization methods can also be used.

[0070] In one example, the first convolutional neural network model comprises:

[0071] 3D-UNet pelvic anatomy segmentation network model, the model comprising a first convolutional neural network and a second convolutional neural network;

[0072] The first convolutional neural network and the second convolutional neural network each comprise:

[0073] 3 layers of up-sampling layers and 3 layers of down-sampling layers, each layer of up-sampling layer and down-sampling layer adopting skip connection;

[0074] The loss function of the network model is a cross-entropy-Dice hybrid loss function, and five-fold cross-validation is used in the training process of the network model to verify the stability of the model.

[0075] Specifically, the first convolutional neural network model is a 3D-UNet pelvic anatomy segmentation network model, which is used to segment the preprocessed pelvic fracture three-dimensional CT image to obtain a three-dimensional CT image of left iliac bone tissue, a three-dimensional CT image of right iliac bone tissue, and a three-dimensional CT image of sacral bone tissue. The network model comprises two identical convolutional neural networks, each convolutional neural network having a 3-layer down-sampling layer as an encoder and a 3-layer up-sampling layer as a decoder. Each layer of down-sampling layer and up-sampling layer adopts skip connection to provide more pixel information for the up-sampling process. Different numbers of down-sampling layers and up-sampling layers can also be used, for example, 5 layers of down-sampling layers and up-sampling layers. The network model training uses different pelvic fracture three-dimensional CT images with calibration as training samples, i.e., manual calibration of left iliac bone tissue, right iliac bone tissue, and sacral bone tissue in the pelvic fracture three-dimensional CT image before training. The network model is trained according to the calibration results. The loss function used in the training of the network model is a cross-entropy-Dice hybrid loss function, and five-fold cross-validation is used in the training process of the network model to verify the stability of the model.

[0076] In one example, the second convolutional neural network model comprises:

[0077] 3D-UNet multi-scale pelvic fracture segmentation network model, the network model comprising 5 layers of up-sampling layers and 5 layers of down-sampling layers, each layer of up-sampling layer and down-sampling layer adopting skip connection;

[0078] In addition to the lowest resolution up-sampling layer, an auxiliary loss function is added to each up-sampling layer to realize multi-scale deep supervision.

[0079] Five-fold cross-validation is used in the training process of the network model to verify the stability of the model.

[0080] Specifically, the second convolutional neural network model is a 3D-UNet multi-scale pelvic fracture segmentation network model for segmenting the three-dimensional CT images of the left iliac bone tissue, the three-dimensional CT images of the right iliac bone tissue, and the three-dimensional CT images of the sacral bone tissue. The network model is a single convolutional neural network, the encoder of the neural network is 5 layers of down-sampling layers, and the decoder is 5 layers of up-sampling layers. Each layer of the down-sampling layer and the same layer of the up-sampling layer adopts a jump connection, or different layers of down-sampling layers and up-sampling layers can be used, for example, 6 layers of down-sampling layers and up-sampling layers. In addition to the up-sampling layer with the lowest resolution, an auxiliary loss function is added to the other up-sampling layers to realize multi-scale deep supervision. Network hyperparameters are set in the network model. By changing the parameters of the nonlinear mapping function of the fracture distance map, the degree of attention to the whole / local is adjusted. The lower the resolution, the more attention is paid to global features, and the higher the resolution, the more attention is paid to local (especially near the fracture collision fracture surface) features. Five-fold cross-validation is used in the training process of the network model to verify the stability of the model.

[0081] In one example, the loss function of the second convolutional neural network model is a cross-entropy-Dice hybrid loss function based on the fracture distance feature map weighting;

[0082] Before the start of training, the main fragments and the secondary fragment groups of the training images are manually calibrated. According to the calibration results, a filter with a side length of 3*3*3 is used to traverse the training images to obtain the fracture fracture surfaces of the training images.

[0083] The distance of each voxel containing bone tissue intensity information in the training image to the fracture fracture surface is calculated to obtain the fracture distance feature map.

[0084] The fracture distance feature map is calculated by a nonlinear mapping function to obtain distance feature weights.

[0085] Specifically, the loss function of the second convolutional neural network model is a cross-entropy-Dice hybrid loss function based on the fracture distance feature map weighting. The training samples of the network model are three-dimensional CT images of left iliac bone tissue, three-dimensional CT images of right iliac bone tissue, and three-dimensional CT images of sacral bone tissue. Before the start of training, the main fragments and the secondary fragment groups of each training sample are manually calibrated. According to the calibration results, a filter with a side length of 3*3*3 is used to traverse the training images to obtain the fracture fracture surfaces of each training sample. Then, the distance of each voxel containing bone tissue intensity information in each training sample to the fracture fracture surface is calculated. The distance is the shortest distance of each voxel containing bone tissue intensity information to the fracture fracture surface in three-dimensional space. Finally, the distance feature weights are calculated by a nonlinear mapping function. The farther the distance to the fracture surface, the lower the weight. The closer the distance to the fracture surface, the higher the weight. Then, the Dice and cross-entropy loss are weighted to obtain the loss function.

[0086] In one example, the expression of the total loss function of the second convolutional neural network model is:

[0087] Loss=L dice +L cross-Entropy ;

[0088] Wherein, Loss is the total loss function, L dice is the 0.5 times the Dice loss function based on the fracture distance feature map weighting, L cross-Entropy is the 0.5 times the cross entropy loss function based on the fracture distance feature map weighting.

[0089] Specifically, the expression of the total loss function of the second convolutional neural network model is Loss=L dice +L cross-Entropy , the distance feature weight is weighted to the Dice loss function, and then multiplied by 0.5 (denoted as a) to obtain L dice ; the distance feature weight is weighted to the cross entropy loss function, and then multiplied by 0.5 (denoted as b) to obtain L cross-Entropy , wherein a+b=1, the values of a and b can be adjusted according to the situation.

[0090] In one example, in the initial stage of training the 3D-UNet multi-scale pelvic fracture segmentation network model, a smooth transition method is used to maintain the attention of the network model to global features in the early stage of training, and gradually transfer the attention center to the fracture site as the training proceeds.

[0091] The smooth transition is to dynamically adjust the proportion of the fracture distance weight in the overall weight matrix according to the number of iterations of the training.

[0092] Specifically, in the initial stage of training the 3D-UNet multi-scale pelvic fracture segmentation network model, a smooth transition method is used to maintain the attention of the network model to global features in the early stage of training, and the proportion of the fracture distance weight in the overall weight matrix is dynamically adjusted according to the number of iterations of the training, so that the proportion gradually decreases, and the attention center is gradually transferred to the fracture site.

[0093] In one example, in the initial stage of training the second convolutional neural network model, the initial learning rate is set to 0.001, and the learning rate gradually decreases as the number of iterations of the training increases.

[0094] Specifically, in the initial stage of training, the learning rate is larger, so that the network converges quickly, and in the later stage of training, the learning rate is smaller, so that the network avoids oscillation around the optimal point when converging to the optimal solution, thereby better converging to the optimal solution.

[0095] In one example, the set filtering conditions include:

[0096] A connected region with fewer than 50 voxels.

[0097] Specifically, when the number of voxels in a certain connected region is less than 50, it means that the fragment is very small and will not have any impact on the patient. It can be ignored during the operation. Therefore, connected regions with less than 50 voxels are filtered out to improve the segmentation speed and efficiency.

[0098] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the invention. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present invention can be combined with each other.

[0099] Example 1

[0100] like Figure 2 As shown, this embodiment provides an automatic segmentation method for pelvic fracture images, including:

[0101] Acquire raw 3D CT images of pelvic fractures;

[0102] The original 3D CT images of pelvic fractures were cropped and converted to obtain NIFTI format input files, and the CT value data in the input files were Z-Score standardized.

[0103] Establish a 3D-UNet pelvic anatomy segmentation network model, such as Figure 3 As shown, the network model includes a U-Net1 convolutional neural network and a U-Net2 convolutional neural network. Both convolutional neural networks consist of 3 downsampling layers and 3 upsampling layers. Each downsampling layer and the upsampling layer in the same layer use skip connections to provide more pixel information for the upsampling process, such as... Figure 4 As shown; the loss function used in this network model is the cross-entropy-Dice hybrid loss function. Five-fold cross-validation is used during the training process of the network model to verify the stability of the model. First, the input file is coarsely segmented by the U-Net1 convolutional neural network. Then, the coarsely segmented image is combined with the original 3D CT image. Next, the combined image is finely segmented by the U-Net2 convolutional neural network. Finally, Boolean operations are used to mask the intensity information of non-bone tissue in the finely segmented 3D CT image to obtain the 3D CT images of the left iliac bone tissue, the right iliac bone tissue, and the sacral bone tissue.

[0104] Three-dimensional CT images of the left iliac bone, right iliac bone, and sacral bone were input into a 3D-UNet multi-scale pelvic fracture segmentation network model, such as... Figure 5As shown, the network model is a single convolutional neural network, which includes 5 layers of down-sampling layers and 5 layers of up-sampling layers, each layer of down-sampling layer is connected to the same layer of up-sampling layer by skip connection, the loss function of the network model is a cross-entropy-Dice hybrid loss function based on fracture distance feature map weighting, the training samples of the network model are left ilium bone tissue three-dimensional CT image samples, right ilium bone tissue three-dimensional CT image samples and sacrum bone tissue three-dimensional CT image samples segmented by the 3D-UNet pelvis dissection segmentation network model, before the training starts, the main fragment and the secondary fragment group of each training sample are manually calibrated, according to the calibration result, a filter with a side length of 3*3*3 is used to traverse the training sample, the fracture surface of each training sample is obtained, then the distance from each voxel containing bone tissue intensity information to the fracture surface in each training sample is calculated, the distance is the shortest distance from each voxel containing bone tissue intensity information to the fracture surface in the three-dimensional space, finally the distance feature weight is calculated by a nonlinear mapping function, the farther the distance from the fracture surface, the lower the weight, the closer the distance from the fracture surface, the higher the weight; then the Dice and cross-entropy loss are weighted to obtain the loss function; the expression of the total loss function of the network model is: Loss=L dice +L cross-Entropy , wherein, Loss is the total loss function, L dice is the Dice loss function based on fracture distance feature map weighting, L cross-Entropy is the cross-entropy loss function based on fracture distance feature map weighting; in the initial stage of training the 3D-UNet multi-scale pelvis fracture segmentation network model, a smooth transition method is used, the network model pays attention to global features in the early stage of training, and gradually shifts the focus of attention to the fracture site as the training progresses; the smooth transition is to dynamically adjust the proportion of fracture distance weight in the overall weight matrix according to the number of iterations of training; in the initial stage of training the network model, the initial learning rate is set to 0.001, and the learning rate gradually decreases as the number of iterations of training increases; after the three-dimensional CT images of the left ilium bone tissue, the right ilium bone tissue and the sacrum bone tissue are segmented by the network model, the main fragment image and the secondary fragment group image of the left ilium bone, the right ilium bone and the sacrum bone are obtained respectively;

[0105] The secondary fragment group image is solved by connected region, and the connected regions with a voxel number less than 50 are filtered out to obtain the secondary fragment images of all left ilium bones, right ilium bones and sacrum bones;

[0106] The main fragment images and the secondary fragment images of all left ilium bones, right ilium bones and sacrum bones are output, as shown in Figure 6 , wherein, the boxed part is the secondary fragment, and the unboxed part is the main fragment.

[0107] Embodiment 2

[0108] As Figure 7 shown, the embodiment provides an automatic segmentation system for pelvic fracture image, for performing the automatic segmentation method of the pelvic fracture image according to any one of the embodiments 1, comprising:

[0109] An acquisition module is configured to acquire a three-dimensional CT image of a pelvic fracture;

[0110] A preprocessing module is configured to preprocess the three-dimensional CT image of the pelvic fracture;

[0111] A 3D-UNet pelvic anatomy segmentation network model is configured to segment the preprocessed three-dimensional CT image by using the trained 3D-UNet pelvic anatomy segmentation network model, register the segmented three-dimensional CT image to the three-dimensional CT image, and shield the intensity information of non-bone tissue in the three-dimensional CT image by using Boolean operation, to obtain a three-dimensional CT image of pelvic bone tissue, i.e., a three-dimensional CT image of left iliac bone tissue, a three-dimensional CT image of right iliac bone tissue, and a three-dimensional CT image of sacral bone tissue;

[0112] A 3D-UNet multi-scale pelvic fracture segmentation network model is configured to segment the three-dimensional CT image of pelvic bone tissue by using the trained 3D-UNet multi-scale pelvic fracture segmentation network model, to obtain a main fragment image and a secondary fragment group image of pelvic fracture, i.e., a main fragment image and a secondary fragment group image of left iliac bone, right iliac bone, and sacral bone;

[0113] A connected region solving and filtering module is configured to solve the connected region of the secondary fragment group image, and filter out the connected region with a voxel number less than 50, to obtain a secondary fragment image of all left iliac bone, right iliac bone, and sacral bone;

[0114] An output module is configured to output the main fragment image and the secondary fragment image of all left iliac bone, right iliac bone, and sacral bone.

[0115] The above has described the embodiments of the present application, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A method for automatic segmentation of pelvic fracture imagery, the method comprising: The method comprises the following steps: obtaining a three-dimensional CT image of a pelvic fracture; preprocessing the three-dimensional CT image of the pelvic fracture; segmenting the three-dimensional CT image after preprocessing by using a first trained convolutional neural network model, registering the segmented three-dimensional CT image to the three-dimensional CT image, and shielding the intensity information of non-bone tissue in the three-dimensional CT image by using a Boolean operation to obtain a three-dimensional CT image of pelvic bone tissue; segmenting the three-dimensional CT image of pelvic bone tissue by using a second trained convolutional neural network model to obtain all primary fragment images and secondary fragment group images of the pelvic fracture; solving the connected region of the secondary fragment group image, and filtering according to the set filtering condition to obtain all secondary fragment images of the pelvic fracture; outputting all primary fragment images and secondary fragment images.

2. The method of automatic segmentation of pelvic fracture imagery of claim 1, wherein, The preprocessing comprises the following steps: obtaining an input file in NIFTI format by cropping and format conversion of the three-dimensional CT image of the pelvic fracture; performing Z-Score standardization on the CT value data in the input file.

3. The method of automatic segmentation of pelvic fracture imagery of claim 1, wherein, The first convolutional neural network model comprises: a 3D-UNet pelvic anatomy segmentation network model, which comprises a first convolutional neural network and a second convolutional neural network; both the first convolutional neural network and the second convolutional neural network comprise: three up-sampling layers and three down-sampling layers, and the up-sampling layer and the down-sampling layer of each layer are connected by jump connection; the loss function of the network model is a cross-entropy-Dice hybrid loss function, and five-fold cross-validation is used in the training process of the network model to verify the stability of the model.

4. The method of automatic segmentation of pelvic fracture imagery of claim 1, wherein, The second convolutional neural network model comprises: a 3D-UNet multi-scale pelvic fracture segmentation network model, which comprises five up-sampling layers and five down-sampling layers, and the up-sampling layer and the down-sampling layer of each layer are connected by jump connection; an auxiliary loss function is added to each up-sampling layer except the up-sampling layer with the lowest resolution to realize multi-scale deep supervision; five-fold cross-validation is used in the training process of the network model to verify the stability of the model.

5. The method of automatic segmentation of pelvic fracture imagery of claim 1, wherein, The loss function of the second convolutional neural network model is a cross-entropy-Dice hybrid loss function based on a fracture distance feature map weighting; Before the training starts, the primary fragments and secondary fragment groups of the training image are manually labeled, and according to the labeling result, a filter with a side length of 3*3*3 is used to traverse the training image to obtain the fracture surface of the training image; the distance of each voxel containing bone tissue intensity information in the training image to the fracture surface is calculated to obtain the fracture distance feature map; the fracture distance feature map is calculated by using a nonlinear mapping function to obtain a distance feature weight.

6. The method of automatic segmentation of pelvic fracture imagery of claim 5, wherein, The expression of the total loss function of the second convolutional neural network model is: Loss = L dice + L cross-Entropy ; wherein Loss is a total loss function, L dice is a 0.5 times Dice loss function based on fracture distance feature map weighting, L cross-Entropy is a 0.5 times cross-entropy loss function based on fracture distance feature map weighting.

7. The method of automatic segmentation of pelvic fracture imagery of claim 6, wherein, In the initial stage of training the second convolutional neural network model, a smooth transition method is used to maintain the attention of the network model to global features in the early stage of training, and gradually shift the focus to the fracture site as the training progresses; The smooth transition is dynamically adjusting the proportion of the fracture distance weight in the overall weight matrix according to the number of iterations of the training.

8. The method of automatic segmentation of pelvis fracture imagery of claim 1, wherein, In the initial stage of training the second convolutional neural network model, the initial learning rate is set to 0.001, and the learning rate gradually decreases as the number of iterations of the training increases.

9. The method of automatic segmentation of pelvis fracture imagery of claim 1, wherein, The set filtering condition includes: A connected region with a voxel number less than 50.

10. An automatic segmentation system of pelvic fracture images for performing the automatic segmentation method of pelvic fracture images according to any one of claims 1 to 9, characterized in that, The system comprises: An acquisition module for acquiring a pelvic fracture three-dimensional CT image; A preprocessing module for preprocessing the pelvic fracture three-dimensional CT image; A first image segmentation module for segmenting the three-dimensional CT image after preprocessing by using a first convolutional neural network model after training, registering the segmented three-dimensional CT image to the three-dimensional CT image, and shielding the intensity information of non-bone tissue in the three-dimensional CT image by Boolean operation to obtain a pelvic bone tissue three-dimensional CT image; A second image segmentation module for segmenting the pelvic bone tissue three-dimensional CT image by using a second convolutional neural network model after training to obtain a main fragment image and a secondary fragment group image of the pelvic fracture; A connected region solving and filtering module for solving the connected region of the secondary fragment group image and filtering according to a set filtering condition to obtain a secondary fragment of each pelvic fracture; An output module for outputting all the main fragment images and the secondary fragment images.

Images