Pelvic fracture CT image segmentation method based on dynamic spatial feature enhancement module

Abstract

The application provides a pelvis fracture CT image segmentation method based on a dynamic spatial feature enhancement module, relates to the technical field of medical image segmentation, and solves the technical problem of low accuracy of the current pelvis fracture CT image segmentation result data. The method comprises the following steps: extracting a pelvis skeleton from pelvis CT scan data by using a specified anatomical segmentation network; performing image segmentation on bone fragments in each pelvis region of the pelvis skeleton by using a fracture segmentation network comprising a dynamic spatial feature enhancement module, and using the dynamic spatial feature enhancement module to capture global background data and local detail data in the pelvis region and retain object boundaries in the process of image segmentation by means of a multi-path structure, an attention mechanism and residual learning, so as to obtain key fracture fragments of a target bone in the pelvis CT scan data through the process of image segmentation of the fracture segmentation network.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of medical image segmentation, in particular to a pelvis fracture CT image segmentation method based on a dynamic spatial feature enhancement module. BACKGROUND

[0002] Pelvis fracture segmentation is a key link in the diagnosis and treatment planning of pelvis trauma. Accurate segmentation of the pelvis fracture site in the medical image (usually a CT image) of a fracture patient can provide detailed cross-sectional information for a clinician, which helps to accurately assess the fracture type, fracture displacement degree and fracture fragment position, thereby improving the accuracy of preoperative planning and postoperative evaluation. However, the pelvis has a complex anatomical structure, various fracture types, and significant individual differences among different patients, which makes it difficult for automatic segmentation. In addition, the pelvis fracture is usually accompanied by other soft tissue injury or bleeding, and there may be interference such as tomographic artifacts, noise and metal implants in the CT image, which puts higher requirements on the robustness of the segmentation algorithm.

[0003] At present, most of the traditional pelvis segmentation methods rely on manual or semi-automatic strategies based on threshold, region growing or morphological operation. By adjusting the threshold and selecting the seed point, the adaptive threshold segmentation and region growing method is used to extract the bone region, and then the fracture surface is manually depicted by outlining the fragments in the three-dimensional view or modifying the mask slice by slice. Although these methods have certain feasibility in relatively regular anatomical structures, they still require a large amount of interactive operation by professional doctors when facing complex fracture morphology, which leads to the accuracy of the segmentation result being easily affected by subjective factors, so that the accuracy of the current pelvis fracture CT image segmentation result is low. SUMMARY

[0004] The purpose of the present application is to provide a pelvis fracture CT image segmentation method based on a dynamic spatial feature enhancement module to solve the technical problem of low accuracy of the current pelvis fracture CT image segmentation result.

[0005] In a first aspect, the application provides a pelvis fracture CT image segmentation method based on a dynamic spatial feature enhancement module, which comprises:

[0006] Obtaining pelvis CT scan data to be segmented and processed;

[0007] Extracting the pelvis skeleton from the pelvis CT scan data by using a specified anatomical segmentation network; wherein the specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture, and the cascaded 3D nn-Unet architecture is generated by pre-training using a normal pelvis CT image data set and adjusting a fracture pelvis data set;

[0008] The fracture segmentation network comprising a dynamic spatial feature enhancement block (DSFE-Block) is used to perform image segmentation on the bone fragments in each pelvic region in the pelvic skeleton, and the dynamic spatial feature enhancement block is used to capture global background data and local detail data in the pelvic region and retain object boundaries in the process of image segmentation by means of a multi-path structure, an attention mechanism and residual learning, so as to obtain key fracture fragments of the target bone in the pelvic CT scan data through the process of image segmentation of the fracture segmentation network; wherein the multi-path structure corresponds to a plurality of parallel paths for processing an initial input tensor input into the dynamic spatial feature enhancement block, and the plurality of parallel paths comprise a local feature path, a parallel pooling path and an attention path.

[0009] In one possible implementation, the initial input tensor is represented by X∈R B×C×D×H×W , wherein B represents a batch size, C represents a channel number, and D, H and W represent spatial dimensions;

[0010] The local feature path is used to process the initial input tensor by means of a 3×3×3 convolution layer, a batch normalization (BN) layer and a ReLU activation function through the following formula:

[0011] X1=ReLU(BN(Conv3D(X)));

[0012] wherein X1 represents a local feature path processing result of the initial input tensor processed by the local feature path, X represents the initial input tensor, ReLU represents an activation function, BN represents a batch normalization layer function, and Conv3D represents an additional 3×3×3 convolution layer.

[0013] In one possible implementation, the parallel pooling path is used to perform maximum pooling processing and average pooling processing on the local feature path processing result through the following formula, and the maximum pooling processing result and the average pooling processing result are spliced along the channel dimension:

[0014] X pool =Concat(MaxPool3D(X1),AvgPool3D(X1));

[0015] wherein X1 represents the local feature path processing result, Convcat represents a splicing processing function for two convolution operations, MaxPool3D represents a maximum pooling convolution layer function with a pooling kernel size of 3×3×3, AvgPool3D represents an average pooling convolution layer function with a pooling kernel size of 3×3×3, and X Poolrepresenting a merged feature after concatenating the max-pooling result and the average-pooling result;

[0016] The parallel pooling path is further configured to further process the merged feature by using the activation function, the batch normalization layer function and the additional 3x3x3 convolution layer according to the following formula:

[0017] X2 = ReLU(BN(Conv3D(X pool )));

[0018] wherein X2 represents the result data after further processing of the merged feature, ReLU represents the activation function, BN represents the batch normalization layer function, X Pool represents the merged feature, and Conv3D represents the additional 3x3x3 convolution layer.

[0019] In one possible implementation, the attention path is configured to calculate an attention coefficient by using the additional 3x3x3 convolution layer and a Sigmoid activation function according to the following formula:

[0020] X att = σ(Conv3D(GAP(X)));

[0021] wherein σ represents the Sigmoid activation function, Conv3D represents the additional 3x3x3 convolution layer, X represents the initial input tensor, Xatt represents the attention coefficient, and GAP represents the global average pooling function.

[0022] In one possible implementation, after obtaining the attention coefficient, the attention coefficient is applied to the result data by using element-level multiplication according to the following formula to obtain a weighted feature:

[0023] X weighted = X2 ⊙ X att ;

[0024] wherein ⊙ represents element-level multiplication, X2 represents the result data after further processing of the merged feature, X att represents the attention coefficient, and X weighted represents the weighted feature.

[0025] The residual learning manner execution process includes combining the weighted feature and the initial input tensor by using residual connection according to the following formula to preserve original information:

[0026] Y = X + X weighted ;

[0027] wherein X weightedrepresents a weighted feature, X represents the initial input tensor, and Y represents the final result after combining the weighted feature with the initial input tensor.

[0028] In one possible implementation, the loss function of the fracture segmentation network is a hybrid DC_and_BCE_loss, which is used to simultaneously optimize the classification accuracy at the voxel level and the segmentation quality at the region level; the hybrid DC_and_BCE_loss is used to ignore the loss calculation of a specific region through a mask mechanism, so as to enhance the applicability to processing a specified complex segmentation task; wherein the specified complex segmentation task includes a case where a part of the region is ignored.

[0029] In one possible implementation, the loss function of the fracture segmentation network includes a binary cross-entropy (BCE) loss function and a Soft Dice loss function; the BCE loss function is used to measure the classification difference between the predicted segmentation result and the target segmentation region at the voxel level; the Soft Dice loss function is used to evaluate the overlap degree between the predicted segmentation result and the target segmentation region through the following formula, so as to optimize the matching quality of the overall region:

[0030]

[0031] wherein, represents a binary cross-entropy loss function, which is used for classification loss at the voxel level; represents a Soft Dice loss function, which is used to measure the overlap of the segmentation region; and α and β are two weight hyperparameters, which are respectively used to control the contribution proportion of the BCE loss and the Dice loss to the total loss.

[0032] In a second aspect, the present application provides a pelvis fracture CT image segmentation device based on a dynamic spatial feature enhancement module, comprising:

[0033] An acquisition module is configured to acquire pelvis CT scan data to be segmented and processed.

[0034] An extraction module is configured to extract a pelvis skeleton from the pelvis CT scan data by using a specified anatomical segmentation network; wherein the specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture, and the cascaded 3D nn-Unet architecture is generated by pre-training using a normal pelvis CT image data set and adjusting a fracture pelvis data set.

[0035] The segmentation module is configured to perform image segmentation on the bone fragments in each pelvic region in the pelvic skeleton by using a bone fracture segmentation network comprising a dynamic spatial feature enhancement module, and to enable the dynamic spatial feature enhancement module to capture global background data and local detail data in the pelvic region and retain object boundaries in the process of image segmentation by means of a multi-path structure, an attention mechanism and residual learning, so as to obtain key bone fracture fragments of a target bone in the pelvic CT scan data through the process of image segmentation by using the bone fracture segmentation network.

[0036] In a third aspect, the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program capable of running on the processor, and the processor executes the computer program to implement the method of the first aspect.

[0037] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores computer executable instructions, and the computer executable instructions, when invoked and run by a processor, cause the processor to run the method of the first aspect.

[0038] The present application brings the following beneficial effects:

[0039] The application provides a pelvis fracture CT image segmentation method based on a dynamic spatial feature enhancement module. The method can obtain pelvis CT scan data to be segmented. A specified anatomical segmentation network is used to extract pelvis bones from the pelvis CT scan data. The specified anatomical segmentation network is a cascaded 3D nn-Unet architecture that is generated by pre-training on a normal pelvis CT image dataset and adjusting a pelvis fracture dataset. A fracture segmentation network containing a dynamic spatial feature enhancement module is used to perform image segmentation on bone fragments in each pelvis region of the pelvis bones. The dynamic spatial feature enhancement module captures global background data and local detail data in the pelvis region and retains object boundaries in the image segmentation process through a multi-path structure, an attention mechanism, and residual learning. The key fracture fragments of target bones in the pelvis CT scan data are obtained through the image segmentation process of the fracture segmentation network. The multi-path structure corresponds to multiple parallel paths that process an initial input tensor input to the dynamic spatial feature enhancement module. The multiple parallel paths include a local feature path, a parallel pooling path, and an attention path. In this scheme, the multi-path design, attention mechanism, and residual learning of the dynamic spatial feature enhancement module enable the fracture segmentation network to effectively capture fine details and retain object boundaries, achieve more accurate segmentation results, improve the data accuracy of pelvis fracture CT image segmentation results, and solve the technical problem of low data accuracy of current pelvis fracture CT image segmentation results.

[0040] In order to make the above objectives, characteristics and advantages of the present application more apparent, the following describes a preferred embodiment, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0042] Figure 1 A flowchart of the pelvis fracture CT image segmentation method based on the dynamic spatial feature enhancement module provided by the embodiments of the present application is shown in the figure.

[0043] Figure 2 Another flowchart of the pelvis fracture CT image segmentation method based on the dynamic spatial feature enhancement module provided by the embodiments of the present application is shown in the figure.

[0044] Figure 3 An example of the internal architecture of the dynamic spatial feature enhancement module DSFE-Block in the pelvis fracture CT image segmentation method based on the dynamic spatial feature enhancement module provided in the embodiments of the present application is shown.

[0045] Figure 4 A structural schematic diagram of a pelvis fracture CT image segmentation device based on a dynamic spatial feature enhancement module provided in the embodiments of the present application is shown.

[0046] Figure 5 A structural schematic diagram of an electronic device provided in the embodiments of the present application is shown. DETAILED DESCRIPTION

[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions of the present application will be described below in connection with the drawings, obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0048] The terms “include” and “have” and any variations thereof mentioned in the embodiments of the present application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device including a series of steps or units is not limited to the listed steps or units, but can optionally include other steps or units not listed, or can optionally include other steps or units inherent to the process, method, product, or device.

[0049] With the rapid development of medical image processing and artificial intelligence technology, more and more researches have begun to try to improve the accuracy of pelvis fracture segmentation through deep learning-based automatic segmentation algorithms. Typical methods include using convolutional neural networks (CNN) or hybrid network structures for automatic segmentation and fracture detection of the pelvis region. Currently, deep convolutional neural networks (CNN) can be used in combination with multi-directional block images for pelvis fracture detection, through multi-view processing of CT images, the network can more comprehensively capture the features of the pelvis region, and thus improve the accuracy of fracture detection. Although this method provides a fast and accurate fracture diagnosis tool for clinical use, its output cannot provide a fully automated solution for subsequent operations.

[0050] In the pelvic fracture segmentation task, due to the complex pelvic anatomical structure, the various forms of fracture region and the high uncertainty, the current pelvic fracture segmentation method has many problems. For example, the local and global feature expression is insufficient: the form and position of the pelvic fracture region have significant uncertainty, and its distribution can cover a large range of pelvic region, and contain small and complex local features such as bone cracks, fragments or irregular boundaries. The traditional segmentation method has the problem of insufficient local features when capturing local details: the lack of accurate capture ability of local features such as fine cracks and edges leads to inaccurate segmentation of small fractures. Therefore, the current pelvic fracture CT image segmentation result data has low accuracy.

[0051] In addition, the space and channel dependence of the traditional segmentation method is weakened, and the fracture region may exhibit discontinuity in the spatial dimension (for example, bone cracks may be distributed in different positions of the pelvis), while there may be implicit association in the channel dimension (for example, different channels of multi-modal medical images or feature channels of the segmentation model). The following aspects are generally ignored by conventional segmentation models when extracting and fusing features: first, the discontinuity of spatial features: the spatial distribution of the fracture region is complex and may be discontinuous, and existing models are difficult to capture these nonlinear features through a single scale or simple convolution. Second, the interaction between channels is insufficient: the associated features between different channels (such as different window widths and window levels of CT images) are not fully explored, resulting in reduced completeness of feature expression. Therefore, the current pelvic fracture CT image segmentation result data has low accuracy.

[0052] Based on this, the embodiment of the present application provides a pelvic fracture CT image segmentation method based on a dynamic spatial feature enhancement module, which can solve the technical problem of low accuracy of the current pelvic fracture CT image segmentation result data.

[0053] The embodiments of the present application will be further described below with reference to the accompanying drawings.

[0054] Figure 1 A flowchart of a pelvic fracture CT image segmentation method based on a dynamic spatial feature enhancement module provided by an embodiment of the present application is shown in FIG. 1. As shown in the figure, the method comprises the following steps. Figure 1

[0055] Step S110, acquiring pelvic CT scan data to be segmented.

[0056] As a possible implementation, in order to automatically segment the main fragments of the target bone from the CT scan, first, the pelvic CT scan data to be segmented is acquired. The pelvic CT scan data is the pelvic fracture CT image (pelvic fracture CT image).

[0057] ​Step S120, extracting the pelvis skeleton from the pelvis CT scan data by using the specified anatomy segmentation network.

[0058] The specified anatomy segmentation network is an optimized cascaded 3D nn-Unet architecture, which is generated by pre-training on a normal pelvis CT image dataset and adjusting on a fractured pelvis dataset.

[0059] As shown in Figure 2 The method provided by the embodiment of the present application includes two stages. In the first stage, the pelvis skeleton is first extracted from the CT scan by using an anatomy segmentation network (specified anatomy segmentation network), which adopts a cascaded 3D nn-Unet architecture. The architecture is pre-trained on a healthy pelvis CT image dataset (normal pelvis CT image dataset) and further fine-tuned on a fractured pelvis dataset.

[0060] Step S130, performing image segmentation on the bone fragments in each pelvis region of the pelvis skeleton by using a fracture segmentation network containing a dynamic spatial feature enhancement module, and using the dynamic spatial feature enhancement module to capture global background data and local detail data in the pelvis region and retain object boundaries in the process of image segmentation by means of a multi-path structure, an attention mechanism and residual learning, so as to obtain the key fracture fragments of the target skeleton in the pelvis CT scan data through the process of image segmentation of the fracture segmentation network.

[0061] As shown in Figure 2 In the second stage, a fracture segmentation network (FractureSeg network) is then applied to segment the bone fragments in each pelvis region. As an example, the fracture segmentation network is composed of four key components: an encoder, a decoder, a bottleneck layer (Vision-LSTM) and a dynamic spatial feature enhancement module (DSFE-Block). The architecture of the DSFE-Block is composed of three parallel paths: a local feature path, a parallel pooling path and an attention path, which process the input tensor respectively. It can also be understood that the multi-path structure corresponds to multiple parallel paths that process the initial input tensor input to the dynamic spatial feature enhancement module, and the multiple parallel paths include the local feature path, the parallel pooling path and the attention path.

[0062] Due to the diversity of the size and shape of the fracture blocks, capturing local details and global context is crucial for pelvis fracture segmentation. In the embodiments of the present application, a dynamic spatial feature enhancement module is used in the skip connection of the fracture segmentation network. Through the multi-path design, attention mechanism and residual learning of the dynamic spatial feature enhancement module, the fracture segmentation network can effectively capture fine details and preserve object boundaries, achieving more accurate segmentation results and improving the data accuracy of pelvis fracture CT image segmentation results, solving the technical problem of low data accuracy of current pelvis fracture CT image segmentation results.

[0063] For the discontinuity of spatial features, the spatial distribution of the fracture region is complex and discontinuous, while the dynamic spatial feature enhancement module captures nonlinear spatial features through multi-path feature extraction; for the insufficient interaction between channels, fracture segmentation needs to fully exploit the correlation between different channels (such as medical image multi-modal data or feature channels), while the dynamic spatial feature enhancement module models the interaction between channels through the attention mechanism, i.e. for the problem of weak dependence on space and channel, the dynamic spatial feature enhancement module (DSFE-Block) effectively solves the problems of discontinuity of spatial dimension, insufficient interaction of channel dimension and insufficient comprehensive of cross-space and channel in fracture region segmentation. Through the dynamic spatial feature enhancement module, the problems of discontinuity of spatial dimension, insufficient interaction of channel dimension and insufficient comprehensive of cross-space and channel in fracture region segmentation are solved.

[0064] The fracture segmentation network provided in the embodiments of the present application effectively fuses the bottleneck layer and the dynamic spatial feature enhancement module, significantly improving the effect of 3D pelvis fracture CT segmentation. Not only does it overcome the problem of feature loss when processing complex fracture regions in conventional networks, but it also effectively preserves key anatomical information and can capture local details and global semantic information of the pelvis fracture region at the same time, achieving an effective balance between detail performance and overall integrity and significantly improving the performance and stability of the fracture segmentation task. The scheme provided in the embodiments of the present application provides important technical support for fracture diagnosis and treatment planning in medical image processing and has broad clinical application prospects.

[0065] The above steps are described in detail below.

[0066] In some embodiments, the initial input tensor is processed by X∈R B×C×D×H×W , where B represents the batch size, C represents the number of channels, and D, H and W represent the spatial dimensions; the local feature path is used to process the initial input tensor through a 3x3x3 convolution layer, a batch normalization (BN) layer and a ReLU activation function according to the following formula:

[0067] X1=ReLU(BN(Conv3D(X)));

[0068] wherein, X1 represents a local feature path processing result of the initial input tensor processed by the local feature path, X represents the initial input tensor, ReLU represents an activation function, BN represents a batch normalization layer function, and Conv3D represents an additional 3x3x3 convolution layer.

[0069] In view of the broken nature of the spatial features, the spatial distribution of the fracture region is complex and discontinuous. The dynamic spatial feature enhancement module in the embodiment of the application captures nonlinear spatial features through multi-path feature extraction, such as Figure 3 As shown in the figure, in the local feature path, based on the initial input tensor X, a convolution operation (3x3x3 Conv) is adopted, combined with batch normalization (BN) and ReLU activation, to strengthen the feature expression of the local region, so as to more efficiently and accurately capture the local details of the fracture region.

[0070] In some embodiments, the parallel pooling path is used to perform maximum pooling processing and average pooling processing on the local feature path processing result through the following formula, and the maximum pooling processing result and the average pooling processing result are spliced along the channel dimension:

[0071] X pool = Concat (MaxPool3D (X1), AvgPool3D (X1));

[0072] wherein, X1 represents the local feature path processing result, Convcat represents a splicing processing function for two convolution operations, MaxPool3D represents a maximum pooling convolution layer function with a pooling kernel size of 3x3x3, AvgPool3D represents an average pooling convolution layer function with a pooling kernel size of 3x3x3, and X Pool represents the merged features after splicing the maximum pooling processing result and the average pooling processing result;

[0073] The parallel pooling path is also used to further process the merged features by using an activation function, a batch normalization layer function, and an additional 3x3x3 convolution layer through the following formula:

[0074] X2=ReLU(BN(Conv3D(X pool )));

[0075] wherein, X2 represents the result data after further processing of the merged features, ReLU represents an activation function, BN represents a batch normalization layer function, and X Pool represents the merged features, and Conv3D represents an additional 3x3x3 convolution layer.

[0076] In the parallel pooling path, as Figure 3As shown, the spatial distribution of multi-scale information is retained through the combined operation of max-pooling (3x3x3 MaxPool) and average-pooling (3x3x3 AvgPool), and different pooling features are combined through a concatenation operation. Through this multi-scale mechanism, the spatial distribution of the fracture region can be more comprehensively addressed, and the perception ability of the fracture segmentation network model for complex shapes can be improved.

[0077] In some embodiments, the attention path is used to calculate the attention coefficient by using an additional 3x3x3 convolution layer and a Sigmoid activation function as follows:

[0078] X att = σ(Conv3D(GAP(X)));

[0079] wherein σ represents the Sigmoid activation function, Conv3D represents the additional 3x3x3 convolution layer, X represents the initial input tensor, Xatt represents the attention coefficient, and GAP represents the global average pooling function.

[0080] In order to enhance the importance of specific features, the dynamic spatial feature enhancement module in the embodiments of the present application generates a context-aware attention map by performing a global average pooling (GAP) operation, as shown in Figure 3 As shown, the attention coefficient is calculated using a 3x3x3 convolution layer (3x3x3 Conv) and a Sigmoid activation function. In order to address the lack of interaction between channels, the fracture segmentation network needs to fully exploit the correlation between different channels (such as multi-modal data or feature channels of medical images). The dynamic spatial feature enhancement module achieves modeling of the interaction between channels through an attention mechanism, i.e., the global average pooling (GAP) globally aggregates the input features to generate a context-aware attention map, thereby more efficiently and accurately extracting the correlation between channels.

[0081] In some embodiments, after obtaining the attention coefficient, the attention coefficient is applied to the result data by using element-level multiplication as follows to obtain the weighted feature:

[0082] X weighted = X2⊙X att ;

[0083] wherein represents element-level multiplication, X2 represents the result data after further processing of the merged features, X att represents the attention coefficient, and X weighted represents the weighted feature.

[0084] For feature fusion and residual learning, the attention coefficient Xatt is applied to X2 through element-level multiplication to highlight the most important features. To address the lack of interaction between channels, the output of GAP is further processed using convolution to generate attention coefficients, which are then applied to the pooled features to dynamically adjust the feature weights of each channel and strengthen the implicit relationships between channels.

[0085] The execution process of the above residual learning mode includes combining the weighted features with the initial input tensor using residual connection through the following formula to preserve the original information:

[0086] Y=X+X weighted ;

[0087] where X weighted represents the weighted features, X represents the initial input tensor, and Y represents the final result after combining the weighted features with the initial input tensor.

[0088] To address the lack of integration across space and channels, the dynamic spatial feature enhancement module improves the efficiency of feature fusion through collaborative modeling across space and channels. The multi-path architecture of the dynamic spatial feature enhancement module enables joint expression of spatial information (local details, multi-scale features) and channel information (contextual attention, channel weights). In the residual connection, as shown in Figure 3 , the dynamic spatial feature enhancement module adds the initial input X to the weighted features, achieving feature enhancement while preserving the integrity of the original information. This residual mechanism compensates for the limitations of single feature paths, enabling collaborative modeling of spatial dependencies and channel implicit relationships to improve the working efficiency of the fracture segmentation network.

[0089] In the embodiments of the present application, the multi-scale feature modeling, spatial-channel dependency integration, and remote context information modeling capabilities can improve the efficiency of the fracture segmentation network in segmenting fracture fragments in pelvic CT scans.

[0090] In some embodiments, the loss function of the fracture segmentation network is a hybrid DC_and_BCE_loss, which simultaneously optimizes the classification accuracy at the voxel level and the segmentation quality at the region level. The hybrid DC_and_BCE_loss is used to ignore loss calculation in specific regions through a mask mechanism to enhance the applicability of processing specified complex segmentation tasks. The specified complex segmentation tasks include cases where some regions are ignored.

[0091] The loss function of the fracture segmentation network (FractureSeg network) adopts a hybrid DC_and_BCE_loss, so as to simultaneously optimize the classification accuracy at the voxel level and the segmentation quality at the region level. In addition, the DC_and_BCE_loss supports loss calculation ignoring specific regions through a mask mechanism, thereby enhancing its applicability in processing complex segmentation tasks (for example, in the case of partial regions being ignored). By combining the advantages of BCE and Dice loss, both the local voxel classification accuracy and the overall segmentation quality can be optimized.

[0092] In some embodiments, the loss function of the fracture segmentation network includes a binary cross-entropy (BCE) loss function and a Soft Dice loss function; the BCE loss function is used to measure the classification difference between the predicted segmentation result and the target segmentation region at the voxel level; the Soft Dice loss function is used to evaluate the overlap between the predicted segmentation result and the target segmentation region by the following formula, so as to optimize the matching quality of the overall region:

[0093]

[0094] wherein, represents a binary cross-entropy loss function for voxel-level classification loss; represents a Soft Dice loss function for measuring the overlap of the segmentation region; α and β are two weight hyperparameters for controlling the contribution proportion of the BCE loss and the Dice loss to the total loss.

[0095] For example, the loss function consists of two parts: the first part is the binary cross-entropy (BCE) loss, which is used to measure the classification difference between the predicted segmentation result and the target segmentation region at the voxel level; the second part is the Soft Dice loss, which is used to evaluate the overlap between the predicted segmentation result and the target segmentation region, so as to optimize the matching quality of the overall pelvic region.

[0096] Figure 4 A structural schematic diagram of a pelvic fracture CT image segmentation device based on a dynamic spatial feature enhancement module is provided. As shown in Figure 4 The pelvic fracture CT image segmentation device based on the dynamic spatial feature enhancement module 400 includes:

[0097] The acquisition module 401 is configured to acquire pelvic CT scan data to be segmented and processed.

[0098] The extraction module 402 is configured to extract a pelvic skeleton from the pelvic CT scan data by using a specified anatomical segmentation network, wherein the specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture, and the cascaded 3D nn-Unet architecture is generated by pre-training using a normal pelvic CT image dataset and adjusting a pelvic fracture dataset.

[0099] The segmentation module 403 is configured to perform image segmentation on bone fragments in each pelvic region of the pelvic skeleton by using a pelvic fracture segmentation network comprising a dynamic spatial feature enhancement module, and to enable the pelvic fracture segmentation network to capture global background data and local detail data in the pelvic region and retain object boundaries in the process of image segmentation by using the dynamic spatial feature enhancement module in a manner of multi-path structure, attention mechanism and residual learning, so as to obtain key pelvic fracture fragments of a target skeleton in the pelvic CT scan data by the process of image segmentation of the pelvic fracture segmentation network, wherein the multi-path structure corresponds to a plurality of parallel paths for processing an initial input tensor input into the dynamic spatial feature enhancement module, and the plurality of parallel paths comprise a local feature path, a parallel pooling path and an attention path.

[0100] The pelvic fracture CT image segmentation device based on the dynamic spatial feature enhancement module provided by the embodiments of the present application has the same technical features as the pelvic fracture CT image segmentation method based on the dynamic spatial feature enhancement module provided by the above-mentioned embodiments, and can solve the same technical problems and achieve the same technical effects.

[0101] The electronic device provided by the embodiments of the present application, as shown in Figure 5 The electronic device 500 comprises a processor 502 and a memory 501, wherein the memory stores a computer program executable on the processor, and the processor implements the steps of the method provided by the above-mentioned embodiments when executing the computer program.

[0102] Referring to Figure 5 , the electronic device further comprises a bus 503 and a communication interface 504, and the processor 502, the communication interface 504 and the memory 501 are connected through the bus 503; the processor 502 is configured to execute the executable modules stored in the memory 501, such as computer programs.

[0103] The memory 501 can include a high-speed random access memory (RAM) and can also include a non-volatile memory such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 504 (which can be wired or wireless), and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used.

[0104] The bus 503 can be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the figure to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0105] The memory 501 is used to store programs, and the processor 502 executes the programs after receiving execution instructions. The method executed by the device defined by the process disclosed in any embodiment of the present application can be applied to the processor 502 or implemented by the processor 502.

[0106] The processor 502 can be an integrated circuit chip having a processing capability of signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware or the instruction in the form of software in the processor 502. The processor 502 described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. Each method, step and logic block diagram disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register, etc. The storage medium in the art. The storage medium is located in the memory 501, and the processor 502 reads the information in the memory 501, and combines the hardware to complete the steps of the above method.

[0107] Corresponding to the above-mentioned pelvic fracture CT image segmentation method based on dynamic spatial feature enhancement module, the embodiment of the present application further provides a computer readable storage medium, the computer readable storage medium stores computer executable instructions, when the computer executable instructions are called and run by the processor, the computer executable instructions prompt the processor to run the steps of the above-mentioned pelvic fracture CT image segmentation method based on dynamic spatial feature enhancement module.

[0108] The pelvic fracture CT image segmentation device based on the dynamic spatial feature enhancement module provided by the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The device provided by the embodiment of the present application has the same implementation principle and technical effect as the foregoing method embodiments. For the sake of brevity, the part of the device embodiment not mentioned in the foregoing method embodiments can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can be referred to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0109] In the embodiments of the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. The embodiments described above are merely specific implementation manners of the present application, and for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation; for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, or the among different units, can be indirect couplings or communication connections through some interfaces, communication interfaces, or a combination of other forms, which can be electric, mechanical, or in other forms.

[0110] For another example, the flowcharts and block diagrams in the drawings show the possible implementation architectures, functions and operations of the apparatus, method and computer program product according to the embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders from that shown in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and the combination of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system for executing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0111] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0112] In addition, each functional unit in the embodiments of the present application can be integrated into one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated into one unit.

[0113] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the pelvis fracture CT image segmentation method based on the dynamic spatial feature enhancement module described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0114] It should be noted that: similar labels and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings, in addition, the terms "first", "second", "third" and the like are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0115] Finally, it should be noted that: the above-described embodiments are only specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, but not to limit them, the protection scope of the present application is not limited thereto, although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art within the technical scope disclosed by the present application can still modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features thereof; and these modifications, changes or replacements do not make the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application. All should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for CT image segmentation of pelvic fracture based on dynamic spatial feature enhancement module, characterized in that, The method comprises: obtaining a pelvis CT scan data to be segmented; extracting a pelvis skeleton from the pelvis CT scan data by using a specified anatomical segmentation network; wherein the specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture generated by pre-training using a normal pelvis CT image dataset and adjusting a fractured pelvis dataset; performing image segmentation on bone fragments in each pelvis region of the pelvis skeleton by using a fracture segmentation network comprising a dynamic spatial feature enhancement module, and using the dynamic spatial feature enhancement module to capture global background data and local detail data in the pelvis region and retain object boundaries in the process of image segmentation by means of a multi-path structure, attention mechanism and residual learning, so as to obtain key fracture fragments of a target bone in the pelvis CT scan data through the process of image segmentation of the fracture segmentation network; wherein the multi-path structure corresponds to a plurality of parallel paths for processing an initial input tensor input to the dynamic spatial feature enhancement module, and the plurality of parallel paths comprise a local feature path, a parallel pooling path and an attention path; The initial input tensor is represented by X ∈ R where B denotes batch size, C denotes number of channels, and D, H, and W denote spatial dimensions. the local feature path is used to process the initial input tensor by using a 3*3*3 convolution layer, a batch normalization BN layer and a ReLU activation function through the following formula: ； wherein X1 represents a local feature path processing result of the local feature path after processing the initial input tensor, X represents the initial input tensor, ReLU represents an activation function, BN represents a batch normalization layer function, and Conv3D represents an additional 3*3*3 convolution layer; the attention path is used to calculate an attention coefficient by using the additional 3*3*3 convolution layer and a Sigmoid activation function; the parallel pooling path is used to perform maximum pooling processing and average pooling processing on the local feature path processing result, and the maximum pooling processing result and the average pooling processing result are spliced along the channel dimension; the maximum pooling processing result and the average pooling processing result are spliced to obtain a merged feature; the merged feature is further processed to obtain a result data; after obtaining the attention coefficient, the attention coefficient is applied to the result data by element-level multiplication through the following formula to obtain a weighted feature: ； wherein represents element-level multiplication, X2 represents the result data after further processing the merged feature, Xatt represents the attention coefficient, and Xweighted represents the weighted feature; the residual learning mode execution process comprises combining the weighted feature and the initial input tensor by residual connection through the following formula to retain original information: ； wherein Xweighted represents the weighted feature, X represents the initial input tensor, and Y represents a final result after combining the weighted feature and the initial input tensor.

2. The method of claim 1, wherein, The parallel pooling path is used for maximum pooling processing and average pooling processing on the local feature path processing result by the following formula, and the maximum pooling processing result and the average pooling processing result are spliced along the channel dimension: ； Wherein, X1 represents the local feature path processing result, Convcat represents a splicing processing function for two convolution operations, MaxPool3D represents a maximum pooling convolution layer function with a pooling kernel size of 3*3*3, AvgPool3D represents an average pooling convolution layer function with a pooling kernel size of 3*3*3, and XPool represents the merged features after splicing the maximum pooling processing result and the average pooling processing result; The parallel pooling path is also used for further processing of the merged features by the activation function, the batch normalization layer function and the additional 3*3*3 convolution layer by the following formula: ； Wherein, X2 represents the result data after further processing of the merged features, ReLU represents the activation function, BN represents the batch normalization layer function, XPool represents the merged features, and Conv3D represents the additional 3*3*3 convolution layer.

3. The method of claim 2, wherein, The attention path is used for calculating attention coefficients by the additional 3*3*3 convolution layer and the Sigmoid activation function by the following formula: ； Wherein, sigma represents the Sigmoid activation function, Conv3D represents the additional 3*3*3 convolution layer, X represents the initial input tensor, Xatt represents the attention coefficients, and GAP represents the global average pooling function.

4. The method of claim 1, wherein, The loss function of the fracture segmentation network is a hybrid DC_and_BCE_loss, which simultaneously optimizes the classification accuracy at the voxel level and the segmentation quality at the region level; the hybrid DC_and_BCE_loss is used to ignore the loss calculation of specific regions through a mask mechanism to enhance the applicability of processing specified complex segmentation tasks; wherein the specified complex segmentation task includes the case where part of the region is ignored.

5. The method of claim 4, wherein, The loss function of the fracture segmentation network includes a binary cross-entropy (BCE) loss function and a Soft Dice loss function; the BCE loss function is used to measure the classification difference between the predicted segmentation result and the target segmentation region at the voxel level; the Soft Dice loss function is used to evaluate the overlap between the predicted segmentation result and the target segmentation region by the following formula to optimize the matching quality of the overall region: ； wherein, represents a binary cross-entropy loss function for voxel-level classification loss; represents a SoftDice loss function for measuring the overlap of segmented regions; and a and b are two weight hyperparameters for controlling the contribution proportion of the BCE loss and the Dice loss to the total loss, respectively.

6. A pelvis fracture CT image segmentation device based on a dynamic spatial feature enhancement module, characterized in that, It includes: An acquisition module is configured to acquire pelvic CT scan data to be segmented; An extraction module is configured to extract a pelvic skeleton from the pelvic CT scan data by using a specified anatomical segmentation network; wherein the specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture, which is generated by pre-training on a normal pelvic CT image dataset and adjusting a fracture pelvic dataset; The segmentation module is configured to perform image segmentation on bone fragments in each pelvic region in the pelvic skeleton by using a bone fracture segmentation network comprising a dynamic spatial feature enhancement module, and to enable the dynamic spatial feature enhancement module to capture global background data and local detail data in the pelvic region and preserve object boundaries in the process of image segmentation by using a multi-path structure, an attention mechanism, and a residual learning manner, so as to obtain key bone fracture fragments of a target bone in the pelvic CT scan data through the process of image segmentation by using the bone fracture segmentation network. The initial input tensor is represented by X ∈ R where B denotes batch size, C denotes number of channels, and D, H, and W denote spatial dimensions. The local feature path is configured to process the initial input tensor by using a 3*3*3 convolution layer, a batch normalization (BN) layer, and a ReLU activation function according to the following formula: ； wherein X1 represents a local feature path processing result obtained by processing the initial input tensor by using the local feature path, X represents the initial input tensor, ReLU represents an activation function, BN represents a batch normalization layer function, and Conv3D represents an additional 3*3*3 convolution layer. The attention path is configured to calculate an attention coefficient by using the additional 3*3*3 convolution layer and a Sigmoid activation function, and the parallel pooling path is configured to perform maximum pooling processing and average pooling processing on the local feature path processing result, and concatenate the maximum pooling processing result and the average pooling processing result along a channel dimension. The application module is configured to apply the attention coefficient to the result data by using element-level multiplication according to the following formula to obtain weighted features after the attention coefficient is obtained: ； wherein represents element-level multiplication, X2 represents the result data obtained by further processing the merged features, Xatt represents the attention coefficient, and Xweighted represents the weighted features. The residual learning manner execution process comprises combining the weighted features and the initial input tensor by using residual connection according to the following formula to preserve original information: ； wherein Xweighted represents the weighted features, X represents the initial input tensor, and Y represents a final result obtained by combining the weighted features and the initial input tensor.

7. An electronic device comprising a memory, a processor, the memory having stored therein a computer program executable on the processor, characterized in that, The processor implements the steps of the method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer executable instructions, and the computer executable instructions enable the processor to execute the method according to any one of claims 1 to 5 when the computer executable instructions are called and executed by the processor.

Images