Pelvic fracture image segmentation method based on dynamic convolutional long short-term memory module

Abstract

The present application provides a pelvic fracture image segmentation method based on a dynamic convolutional long short-term memory module, which relates to the field of medical image segmentation technology and solves the current technical problem of low segmentation efficiency of pelvic fracture CT images. The method comprises: extracting pelvic bones from pelvic CT scan data using a specified anatomical segmentation network; performing image segmentation on bone fragments in each pelvic region of the pelvic bones using a fracture segmentation network including a dynamic convolutional long short-term memory module; integrating multi-scale contextual features and capturing spatial and channel dependencies through the dynamic convolutional long short-term memory module during the image segmentation process, so as to complete feature fusion in the processing layer using a dynamic weighting method, and finally obtaining the key fracture fragments of the target bone in the pelvic CT scan data through the image segmentation process of the fracture segmentation network; the fracture segmentation network comprises an encoder, a decoder, and a bottleneck layer, and the bottleneck layer comprises a dynamic convolutional long short-term memory module.

Description

Technical Field

[0001] The present application relates to the technical field of medical image segmentation, and in particular to a pelvic fracture image segmentation method based on a dynamic convolution long short-term memory module. Background Art

[0002] Pelvic fracture segmentation is a key step in the diagnosis and treatment planning of pelvic trauma. Accurate segmentation of the pelvic fracture site in medical images (usually CT images) of fracture patients can provide clinicians with detailed cross-sectional information, helping them to accurately assess the fracture type, degree of fracture displacement, and location of fracture fragments, thereby improving the accuracy of pre-operative planning and post-operative evaluation. However, the complex pelvic anatomy and diverse fracture types, coupled with significant individual differences between different patients, make automated segmentation more difficult. In addition, pelvic fractures are often accompanied by other soft tissue injuries or bleeding, and CT images may contain interference such as tomographic artifacts, noise, and metal implants. These factors will place higher demands on the robustness of the segmentation algorithm.

[0003] Currently, traditional pelvic segmentation methods mostly rely on manual or semi-automatic strategies based on thresholding, region growing, or morphological operations. By adjusting the threshold and selecting seed points, adaptive threshold segmentation and region growing methods are used to extract bone regions. Subsequently, the fracture surface is manually delineated by outlining fragments in a 3D view or modifying the mask slice by slice. Although these methods have certain feasibility for relatively regular anatomical structures, complex fracture morphologies still require a lot of interactive operations by professional doctors, resulting in low efficiency in the current segmentation of pelvic fracture CT images. Summary of the Invention

[0004] The purpose of the present invention is to provide a pelvic fracture image segmentation method based on a dynamic convolution long short-term memory module to solve the current technical problem of low efficiency in segmentation of pelvic fracture CT images.

[0005] In a first aspect, the present application provides a pelvic fracture image segmentation method based on a dynamic convolutional long short-term memory module, the method comprising:

[0006] Acquire pelvic CT scan data to be segmented;

[0007] Extracting pelvic bones from the pelvic CT scan data using a specified anatomical segmentation network; wherein the specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture generated by pre-training on a normal pelvic CT image dataset and adjusting it for a fractured pelvis dataset;

[0008] A fracture segmentation network including a dynamic convolutional long short-term memory module (DyCoLSTM) is used to perform image segmentation on bone fragments in each pelvic region of the pelvic skeleton. During the image segmentation process, the dynamic convolutional long short-term memory module is used to integrate multi-scale contextual features and capture spatial and channel dependencies, so as to complete feature fusion in the processing layer using a dynamic weight method. Finally, the image segmentation process of the fracture segmentation network is used to obtain key fracture fragments of the target bone in the pelvic CT scan data; wherein, the fracture segmentation network includes an encoder, a decoder and a bottleneck layer, and the bottleneck layer includes the dynamic convolutional long short-term memory module.

[0009] In one possible implementation, integrating multi-scale context features and capturing spatial and channel dependencies through the dynamic convolutional long short-term memory module to complete feature fusion using a dynamic weighting method in the processing layer includes:

[0010] Normalizing the feature map input into the dynamic convolutional long short-term memory module to obtain a normalized result;

[0011] Performing dimension expansion based on the normalized processing result through projection to obtain an expanded result feature map, and inputting the expanded result feature map into a causal convolution path and a matrix long short-term memory path; wherein the causal convolution path and the matrix long short-term memory path are respectively used to model different aspects of spatial and temporal dependencies;

[0012] The output information of the causal convolution path and the matrix long short-term memory path is integrated through a learnable residual connection method to obtain the feature fusion result of the dynamic convolution long short-term memory module.

[0013] In one possible implementation, integrating multi-scale context features and capturing spatial and channel dependencies through the dynamic convolutional long short-term memory module to complete feature fusion using a dynamic weighting method in the processing layer includes:

[0014] Performing initial dimension expansion on the original input dimension of the input tensor, and performing high-dimensional feature extraction based on the expanded dimension using a causal convolutional layer to obtain a feature extraction result; wherein the causal convolutional layer uses a spatial filter on the sequence;

[0015] The feature extraction results are converted into query data, key data, and target values ​​by means of shared linear projection through parallel paths, and tensors of the query data, the key data, and the target value are processed to generate a context-aware target feature map; wherein the multiple paths corresponding to the parallel paths include the causal convolution path and the matrix long short-term memory path;

[0016] Based on the target feature map, a learnable residual connection method is used to perform feature fusion of multi-path outputs to obtain fused features; wherein the multiple paths corresponding to the multi-path outputs include the causal convolution path and the matrix long short-term memory path;

[0017] Based on the additional features obtained from the initial dimensional expansion, the fused features are scaled by dynamic weighting to obtain a scaled result, and the scaled result is projected to the original input dimension to obtain the output data of the dynamic convolutional long short-term memory module.

[0018] In one possible implementation, the initial dimension expansion of the original input dimension of the input tensor and the high-dimensional feature extraction using a causal convolution layer based on the expanded dimension to obtain a feature extraction result include performing the initial dimension expansion of the original input dimension of the input tensor and the high-dimensional feature extraction using a causal convolution layer based on the expanded dimension to obtain a feature extraction result according to the following formula:

[0019] X conv =SiLU(CausalConv1D(X in ));

[0020] Among them, X in Represents the input tensor, Xconv represents the feature extraction result of the convolution output after SiLU activation, SiLU represents the activation function, and CausalConv1D represents the causal convolution layer in the convolutional neural network.

[0021] In one possible implementation, the query data, the key data, and the target value are expressed by the following formula:

[0022] Q=q_proj(X conv ), K=k_proj(X conv ), V=v_proj(X in );

[0023] Wherein, Q represents the query data, K represents the key data, V represents the target value, X in Represents the input tensor, X conv Represents the feature extraction result of the convolution output after SiLU activation;

[0024] The processing of the tensors of the query data, the key data, and the target value to generate a context-aware target feature map includes processing the tensors of the query data, the key data, and the target value to generate a context-aware target feature map using the following formula:

[0025] H mLSTM=MatrixLSTMCell(Q,K,V);

[0026] Wherein, Q represents the query data, K represents the key data, V represents the target value, and H mLSTM represents the context-aware target feature map, and MatrixLSTMCell represents a matrix processing function.

[0027] In one possible implementation, the feature fusion of multi-path outputs based on the target feature map using a learnable residual connection method to obtain the fused feature includes balancing the weights between the output information of the causal convolution path and the output information of the matrix long short-term memory path by the following formula:

[0028] H skip =H mLSTM +(learnable_skip×X conv );

[0029] H skip Represents the fused features, H mLSTM represents the context-aware target feature map, learnableskip represents the learnable residual connection data, and Xconv represents the feature extraction result;

[0030] Projecting the scaling result to the original input dimension includes down-projecting the scaling result to the original input dimension by the following formula:

[0031] X out =proj_down(H skip ×SiLU(Z));

[0032] Among them, Xout represents the enhanced feature representation, H skip Represents the fused features, Z represents the additional features, projdowm represents the down-projection function, and SiLU represents the activation function.

[0033] In one possible implementation, the causal convolution path extracts subtle local features at a high-resolution dimension higher than a specified resolution through depthwise separable convolution, so that the fracture segmentation network can capture small fracture regions smaller than a specified size; wherein the subtle local features include any one or more of the following: bone cracks, fragments, and irregular boundaries;

[0034] The matrix long short-term memory path captures the anatomical structure information of the entire pelvis by modeling the long-term dependency of input features, so that the fracture segmentation network can complete global semantic expression.

[0035] In a second aspect, the present application provides a pelvic fracture image segmentation device based on a dynamic convolutional long short-term memory module, comprising:

[0036] An acquisition module is used to acquire pelvic CT scan data to be segmented;

[0037] an extraction module for extracting pelvic bones from the pelvic CT scan data using a specified anatomical segmentation network; wherein the specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture generated by pre-training on a normal pelvic CT image dataset and adjusting it for a fractured pelvis dataset;

[0038] A segmentation module is used to perform image segmentation on bone fragments within each pelvic region of the pelvic skeleton using a fracture segmentation network including a dynamic convolutional long short-term memory module. During the image segmentation process, the dynamic convolutional long short-term memory module is used to integrate multi-scale contextual features and capture spatial and channel dependencies, so as to complete feature fusion in the processing layer using a dynamic weight method. Finally, the image segmentation process of the fracture segmentation network is used to obtain key fracture fragments of the target bone in the pelvic CT scan data; wherein, the fracture segmentation network includes an encoder, a decoder, and a bottleneck layer, and the bottleneck layer includes the dynamic convolutional long short-term memory module.

[0039] In a third aspect, the present application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0040] In a fourth aspect, the present application further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the method described in the first aspect above.

[0041] This application brings the following beneficial effects:

[0042] The present application provides a pelvic fracture image segmentation method based on a dynamic convolutional long short-term memory module, which can obtain pelvic CT scan data to be segmented and processed; use a specified anatomical segmentation network to extract pelvic bones from the pelvic CT scan data; wherein the specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture, and the cascaded 3D nn-Unet architecture is generated by pre-training with a normal pelvic CT image dataset and adjusting a fractured pelvic dataset; use a fracture segmentation network including a dynamic convolutional long short-term memory module to perform image segmentation on bone fragments in each pelvic region of the pelvic skeleton, and during the image segmentation process, the dynamic convolutional long short-term memory module integrates multi-scale contextual features and captures spatial and channel dependencies, so as to complete feature fusion in the processing layer using a dynamic weight method, and finally obtains the key fracture fragments of the target bone in the pelvic CT scan data through the image segmentation process of the fracture segmentation network; wherein the fracture segmentation network includes an encoder, a decoder and a bottleneck layer, and the bottleneck layer includes the dynamic convolutional long short-term memory module. In this scheme, the dynamic convolutional long short-term memory module with visual feature representation effect in the fracture segmentation network is used to integrate multi-scale contextual features and capture spatial and channel dependencies, thereby realizing the modeling of multi-scale features, the integration of spatial and channel dependencies, and the modeling of remote contextual information. It enhances the ability to model local and global dependencies in the bottleneck layer, ensures efficient feature fusion in its processing layer, solves the problem of insufficient expression of local and global features, and thus improves the overall segmentation efficiency of the fracture segmentation network for fracture fragments in pelvic CT scans, solving the current technical problem of low segmentation efficiency of pelvic fracture CT images.

[0043] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the specific implementation methods or the description of the prior art. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0045] Figure 1 A schematic diagram of a flow chart of a pelvic fracture image segmentation method based on a dynamic convolutional long short-term memory module provided in an embodiment of the present application;

[0046] Figure 2Another flowchart of the pelvic fracture image segmentation method based on the dynamic convolution long short-term memory module provided in an embodiment of the present application;

[0047] Figure 3 This is an example of the internal structure of the dynamic convolutional long short-term memory module DyCoLSTM in the pelvic fracture image segmentation method based on the dynamic convolutional long short-term memory module provided in the embodiment of the present application;

[0048] Figure 4 A schematic diagram of the structure of a pelvic fracture image segmentation device based on a dynamic convolutional long short-term memory module provided in an embodiment of the present application;

[0049] Figure 5 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0050] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0051] The terms "including," "having," and any variations thereof, as used in the embodiments of this application, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not limited to the listed steps or units but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to the process, method, product, or apparatus.

[0052] With the rapid development of medical image processing and artificial intelligence technology, more and more studies have begun to try to improve the efficiency and accuracy of pelvic fracture segmentation through automatic segmentation algorithms based on deep learning. Typical methods include using convolutional neural networks (CNN) or hybrid network structures to automatically segment and detect fractures in the pelvic region. Currently, deep convolutional neural networks (CNN) can be used in combination with multi-directional block images for pelvic fracture detection. By processing CT images from multiple perspectives, the network can more comprehensively capture the characteristics of the pelvic region, thereby improving the accuracy of fracture detection. Although this method provides a fast and accurate fracture diagnosis tool for clinical practice, its output cannot provide a fully automated solution for subsequent operations.

[0053] In the task of pelvic fracture segmentation, due to the complex anatomical structure of the pelvis, the diverse morphology of the fracture area and the high uncertainty, the current pelvic fracture segmentation methods have many problems. For example, the expression of local and global features is insufficient: the morphology and position of the pelvic fracture area have significant uncertainty, and its distribution may cover a large range of the pelvic area. Furthermore, the traditional segmentation method also has the following problems when capturing global structure and local details. First, the global semantic expression is insufficient: it is difficult to fully utilize the overall anatomical structure information of the pelvis, resulting in poor global consistency of the segmentation results. Second, there is a lack of local and global collaborative expression: existing methods often cannot establish an effective balance between local details and global context, resulting in deviations in the details and integrity of the segmentation results. It can be seen that the current segmentation efficiency of pelvic fracture CT images is low.

[0054] In addition, the spatial and channel dependencies of traditional segmentation methods are weakened. The fracture area may appear discontinuous in the spatial dimension (for example, bone fractures may be distributed in different positions of the pelvis), while there may be implicit associations in the channel dimension (for example, different channels of multimodal medical images or feature channels of segmentation models). Conventional segmentation models generally ignore the following aspects during feature extraction and fusion: First, the discontinuity of spatial features: the spatial distribution of the fracture area is complex and may be discontinuous, and it is difficult for existing models to capture these nonlinear features through a single scale or simple convolution. Second, the interactivity between channels is insufficient: the correlation features between different channels (such as different window widths and window positions of CT images) have not been fully mined, resulting in reduced integrity of feature expression. Third, the comprehensiveness across space and channels is insufficient: existing methods find it difficult to collaboratively model spatial dependencies and implicit channel relationships, and feature fusion efficiency is low. Therefore, the current segmentation efficiency of pelvic fracture CT images is low.

[0055] Based on this, an embodiment of the present application provides a pelvic fracture image segmentation method based on a dynamic convolution long short-term memory module, which can solve the current technical problem of low segmentation efficiency of pelvic fracture CT images.

[0056] The embodiments of the present invention are further described below with reference to the accompanying drawings.

[0057] Figure 1 The flowchart of the pelvic fracture image segmentation method based on the dynamic convolution long short-term memory module provided in the embodiment of the present application is as follows. Figure 1 As shown, the method includes:

[0058] Step S110: Acquire pelvic CT scan data to be segmented.

[0059] In an embodiment of the present application, pelvic CT scan data to be segmented is first obtained, wherein the pelvic CT scan data is a pelvic fracture CT image (pelvic fracture CT image), so as to automatically segment the main fragments of the target bone from the pelvic fracture CT image.

[0060] Step S120 , extracting pelvic bones from the pelvic CT scan data using a specified anatomical segmentation network.

[0061] The designated anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture, which is generated by pre-training on a normal pelvic CT image dataset and adjusting it on a fractured pelvis dataset.

[0062] The overall segmentation process of pelvic fracture CT images includes two stages, such as Figure 2 As shown, in the first stage, the pelvic bones are extracted from the CT scan using an anatomical segmentation network (designated anatomical segmentation network), which adopts a cascaded 3D nn-Unet architecture, which is pre-trained on a healthy pelvic CT image dataset (normal pelvic CT image dataset) and further fine-tuned on a fractured pelvis dataset.

[0063] Step S130, using a fracture segmentation network including a dynamic convolutional long short-term memory module to perform image segmentation on the bone fragments in each pelvic region of the pelvic skeleton. During the image segmentation process, the dynamic convolutional long short-term memory module is used to integrate multi-scale context features and capture spatial and channel dependencies, so as to complete feature fusion in the processing layer using a dynamic weight method. Finally, the key fracture fragments of the target bone in the pelvic CT scan data are obtained through the image segmentation process of the fracture segmentation network.

[0064] The fracture segmentation network includes an encoder, a decoder, and a bottleneck layer, and the bottleneck layer includes a dynamic convolutional long short-term memory module (Dynamic Convolutional LSTM, DyCoLSTM).

[0065] In one possible implementation, Figure 2 As shown in FIG, in the second stage of the overall segmentation process of the pelvic fracture CT image, the fracture segmentation network (FractureSeg network) is applied to segment the bone fragments in each pelvic region.

[0066] Through the dynamic convolutional long short-term memory module with visual feature representation effect, multi-scale contextual features are integrated and spatial and channel dependencies are captured, thereby realizing the modeling of multi-scale features, the integration of spatial and channel dependencies, and the modeling of long-range contextual information. It enhances the ability to model local and global dependencies in the bottleneck layer, ensures efficient feature fusion in its processing layer, solves the problem of insufficient expression of local and global features, and can improve the overall segmentation efficiency of the fracture segmentation network for fracture fragments in pelvic CT scans.

[0067] In the embodiments of this application, the dynamic convolutional long short-term memory module combines convolution to propose an efficient feature modeling method to address the uncertainty of the morphology and position of the pelvic fracture area. This method can simultaneously solve the problems of insufficient local feature capture, lack of global semantic expression, and insufficient local and global collaborative expression. The solution provided by the embodiments of this application provides important technical support for fracture diagnosis and treatment planning in medical image processing and has broad clinical application prospects.

[0068] The above steps are described in detail below.

[0069] In some embodiments, in step S130, the multi-scale context features are integrated and spatial and channel dependencies are captured by the dynamic convolution long short-term memory module, so as to complete feature fusion in the processing layer using a dynamic weighting method. Specifically, the following steps may be included:

[0070] The feature map input to the dynamic convolutional long short-term memory module is normalized to obtain a normalized result; based on the normalized result, the dimension is expanded by projection to obtain an expanded result feature map, and the expanded result feature map is input to the causal convolution path and the matrix long short-term memory path; wherein, the causal convolution path and the matrix long short-term memory path are used to model different aspects of spatial and temporal dependencies respectively; the output information of the causal convolution path and the matrix long short-term memory path is integrated through a learnable residual connection method to obtain the feature fusion result of the dynamic convolutional long short-term memory module.

[0071] For example, Figure 3 As shown in the figure, in the dynamic convolutional long short-term memory module DyCoLSTM, the input feature map is first normalized and then dimensionally expanded through a series of projections. The resulting feature map is fed into two independent paths: a causal convolution path and an mLSTM (matrix long short-term memory) path, responsible for modeling different aspects of spatial and temporal dependencies, respectively. The final feature fusion of DyCoLSTM integrates the information of the two paths through a learnable residual connection to generate a rich representation of the input. Thus, in terms of the coordinated expression of local and global features, feature fusion is achieved by using a dynamic weighting method, realizing the adaptive fusion of features extracted from different dimensions.

[0072] A dynamic weight mechanism is introduced through the dynamic convolution long short-term memory module, and a fusion strategy of matrix long short-term memory mLSTM and convolution path is used to establish an effective balance between local details and global context, ensuring that the segmentation results have both local refinement and global structural consistency.

[0073] In some embodiments, in step S130, the multi-scale context features are integrated and spatial and channel dependencies are captured by the dynamic convolution long short-term memory module, so as to complete feature fusion in the processing layer using a dynamic weighting method. Specifically, the following steps may be included:

[0074] The original input dimension of the input tensor is initially expanded, and a causal convolution layer is used to extract high-dimensional features based on the expanded dimension to obtain feature extraction results; wherein the causal convolution layer uses a spatial filter on the sequence;

[0075] The feature extraction results are converted into query data, key data, and target values ​​through a parallel path using shared linear projection. The tensors of query data, key data, and target values ​​are processed to generate a context-aware target feature map. The multiple paths corresponding to the parallel path include a causal convolution path and a matrix long short-term memory path.

[0076] Based on the target feature map, a learnable residual connection method is used to fuse the features of multi-path outputs to obtain the fused features; wherein the multiple paths corresponding to the multi-path outputs include causal convolution paths and matrix long short-term memory paths;

[0077] Based on the additional features obtained from the initial dimensional expansion, the fused features are scaled by dynamic weighting to obtain the scaled result, and the scaled result is projected to the original input dimension to obtain the output data of the dynamic convolutional long short-term memory module.

[0078] Regarding dimensionality expansion and high-dimensional feature extraction, in order to support multi-scale processing, the embodiment of the present application first expands the dimension of the input tensor. For high-dimensional feature extraction, the embodiment of the present application uses a causal convolution layer, which applies a spatial filter on the sequence.

[0079] For the matrix long short-term memory mLSTM path modeling time dependency, in the parallel path, a shared linear projection is used to convert the feature map into query (Q), key (K) and value (V) to support long-term dependency modeling and enable the dynamic convolutional long short-term memory module DyCoLSTM to focus on the recent and long-term features in the input sequence. Figure 3 As shown, the matrix long short-term memory mLSTM module then processes these query, key, and value tensors to generate context-aware feature maps.

[0080] For feature fusion and dynamic weighting, in order to effectively fuse the outputs of the two paths, the dynamic convolutional long short-term memory module DyCoLSTM uses a learnable residual connection (learnable skip) to balance the weights between the matrix long short-term memory mLSTM and the convolution output, such as Figure 3 As shown, finally, the fused output is scaled using the additional features derived from the initial dimensional expansion through a dynamic weighting mechanism, and the result is down-projected to the original input dimension using the following formula.

[0081] Through shared projection and multi-dimensional processing configuration, the dynamic convolutional long short-term memory module DyCoLSTM can efficiently fuse spatial and channel dependencies, which is crucial for effective visual feature modeling.

[0082] In some embodiments, the above-mentioned initial dimension expansion of the original input dimension of the input tensor and the high-dimensional feature extraction using the causal convolution layer based on the expanded dimension to obtain the feature extraction result may specifically include performing the initial dimension expansion of the original input dimension of the input tensor and the high-dimensional feature extraction using the causal convolution layer based on the expanded dimension to obtain the feature extraction result using the following formula:

[0083] X conv =SiLU(CausalConv1D(X in ));

[0084] Among them, X in Represents the input tensor, X conv Represents the feature extraction result of the convolution output after SiLU activation, SiLU represents the activation function, and CausalConv1D represents the causal convolution layer in the convolutional neural network.

[0085] In the embodiment of the present application, the data processing method of the above formula can make the extraction result data of high-dimensional features more accurate.

[0086] In some embodiments, the query data, key data, and target value are represented by the following formula:

[0087] Q=q_proj(X conv ), K=k_proj(X conv ), V=v_proj(X in );

[0088] Among them, Q represents query data, K represents key data, V represents target value, X in Represents the input tensor, X conv Represents the feature extraction result of the convolution output after SiLU activation;

[0089] The query data, key data, and target value tensors are processed to generate a context-aware target feature map, including processing the query data, key data, and target value tensors to generate a context-aware target feature map using the following formula:

[0090] H mLSTM =MatrixLSTMCell(Q,K,V);

[0091] Among them, Q represents query data, K represents key data, V represents target value, and H mLSTM Represents the context-aware target feature map, and MatrixLSTMCell represents the matrix processing function.

[0092] In the embodiment of the present application, the data processing method of the above formula can make the generated target feature map with context awareness more accurate.

[0093] In some embodiments, the above-mentioned method of performing feature fusion of multi-path outputs based on the target feature map using a learnable residual connection method to obtain the fused features may specifically include balancing the weights between the output information of the causal convolution path and the output information of the matrix long short-term memory path using the following formula:

[0094] H skip =H mLSTM +(learnable_skip×X conv );

[0095] H skip Represents the fused features, H mLSTM represents the context-aware target feature map, learnableskip represents the learnable residual connection data, X conv Represents the feature extraction results;

[0096] Project the scaled result to the original input dimension, including down-projecting the scaled result to the original input dimension using the following formula:

[0097] X out =proj_down(H skip ×SiLU(Z));

[0098] Among them, X out represents the enhanced feature representation, H skip Represents the fused features, Z represents the additional features, projdowm represents the down-projection function, and SiLU represents the activation function.

[0099] In the embodiment of the present application, the data processing method of the above formula can make the feature fusion results and the scaling results projected to the data of the original input dimension more accurate.

[0100] In some embodiments, the causal convolution path extracts subtle local features at a high-resolution dimension higher than a specified resolution through depthwise separable convolution, so that the fracture segmentation network can capture small fracture areas smaller than a specified size; wherein, the subtle local features include any one or more of the following: bone cracks, fragments, and irregular boundaries; the matrix long short-term memory path captures the anatomical structure information of the entire pelvis by modeling the long-term dependencies of the input features, so that the fracture segmentation network can complete the global semantic expression.

[0101] To accurately capture local features, the causal convolution path of the dynamic convolutional long short-term memory module (DyCoLSTM) uses large-kernel depthwise separable convolution to extract subtle local features such as bone cracks, fragments, and irregular boundaries at high resolution. This design enhances the model's ability to capture small fracture areas and improves segmentation accuracy.

[0102] To enhance global semantic expression, the matrix long short-term memory (mLSTM) path of the dynamic convolutional long short-term memory (DyCoLSTM) module captures the overall anatomical structure of the pelvis by modeling long-term dependencies between input features, strengthening the model's global semantic expression capabilities. By introducing shared linear projections, DyCoLSTM establishes global correlations between feature dimensions, ensuring globally consistent segmentation results.

[0103] Figure 4 A schematic diagram of the structure of a pelvic fracture image segmentation device based on a dynamic convolutional long short-term memory module is provided. Figure 4 As shown, the pelvic fracture image segmentation device 400 based on the dynamic convolution long short-term memory module includes:

[0104] An acquisition module 401 is used to acquire pelvic CT scan data to be segmented;

[0105] an extraction module 402 for extracting pelvic bones from the pelvic CT scan data using a specified anatomical segmentation network; wherein the specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture generated by pre-training using a normal pelvic CT image dataset and adjusting it for a fractured pelvis dataset;

[0106] Segmentation module 403 is used to perform image segmentation on bone fragments in each pelvic region of the pelvic skeleton using a fracture segmentation network including a dynamic convolutional long short-term memory module. During the image segmentation process, the dynamic convolutional long short-term memory module is used to integrate multi-scale context features and capture spatial and channel dependencies, so as to complete feature fusion in the processing layer using a dynamic weight method, and finally obtain the key fracture fragments of the target bone in the pelvic CT scan data through the image segmentation process of the fracture segmentation network; wherein, the fracture segmentation network includes an encoder, a decoder and a bottleneck layer, and the bottleneck layer includes the dynamic convolutional long short-term memory module.

[0107] The pelvic fracture image segmentation device based on the dynamic convolution long short-term memory module provided in the embodiment of the present application has the same technical features as the pelvic fracture image segmentation method based on the dynamic convolution long short-term memory module provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects.

[0108] An electronic device provided in an embodiment of the present application is Figure 5 As shown, the electronic device 500 includes a processor 502 and a memory 501 , wherein the memory stores a computer program that can be run on the processor, and the processor implements the steps of the method provided in the above embodiment when executing the computer program.

[0109] See also Figure 5 The electronic device further includes: a bus 503 and a communication interface 504, a processor 502, a communication interface 504 and a memory 501 connected via the bus 503; the processor 502 is used to execute executable modules stored in the memory 501, such as computer programs.

[0110] The memory 501 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage. The system network element communicates with at least one other network element via at least one communication interface 504 (which may be wired or wireless), and may utilize the Internet, a wide area network, a local area network, a metropolitan area network, or the like.

[0111] The bus 503 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0112] Among them, the memory 501 is used to store programs, and the processor 502 executes the program after receiving the execution instruction. The method executed by the device defined by the process disclosed in any embodiment of the present application can be applied to the processor 502 or implemented by the processor 502.

[0113] The processor 502 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 502 or by instructions in the form of software. The above-mentioned processor 502 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 501, and processor 502 reads the information in memory 501 and, in conjunction with its hardware, completes the steps of the above method.

[0114] Corresponding to the above-mentioned pelvic fracture image segmentation method based on dynamic convolution long short-term memory module, an embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by the processor, the computer-executable instructions prompt the processor to execute the steps of the above-mentioned pelvic fracture image segmentation method based on dynamic convolution long short-term memory module.

[0115] The pelvic fracture image segmentation device based on the dynamic convolution long short-term memory module provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in the embodiment of the present application are the same as those in the aforementioned method embodiment. For the sake of brief description, for parts not mentioned in the device embodiment, reference can be made to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0116] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interface, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0117] For another example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0118] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0119] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0120] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the pelvic fracture image segmentation method based on the dynamic convolution long-short-term memory module described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0121] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0122] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. However, these modifications, changes, or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application. They should all be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A pelvic fracture image segmentation method based on dynamic convolutional long short-term memory module, characterized in that: The method comprises: Acquire pelvic CT scan data to be segmented; Extracting pelvic bones from the pelvic CT scan data using a specified anatomical segmentation network; wherein the specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture generated by pre-training on a normal pelvic CT image dataset and adjusting it for a fractured pelvis dataset; A fracture segmentation network including a dynamic convolutional long short-term memory module is used to perform image segmentation on bone fragments in each pelvic region of the pelvic skeleton. During the image segmentation process, the dynamic convolutional long short-term memory module is used to integrate multi-scale contextual features and capture spatial and channel dependencies, so as to complete feature fusion in the processing layer using a dynamic weighting method. Finally, the image segmentation process of the fracture segmentation network is used to obtain key fracture fragments of the target bone in the pelvic CT scan data. The fracture segmentation network includes an encoder, a decoder, and a bottleneck layer, and the bottleneck layer includes the dynamic convolutional long short-term memory module. The method integrates multi-scale context features and captures spatial and channel dependencies through the dynamic convolutional long short-term memory module to complete feature fusion in the processing layer using a dynamic weight method, including: Performing initial dimension expansion on the original input dimension of the input tensor, and performing high-dimensional feature extraction based on the expanded dimension using a causal convolutional layer to obtain a feature extraction result; wherein the causal convolutional layer uses a spatial filter on the sequence; The feature extraction results are converted into query data, key data, and target values ​​by means of shared linear projection through parallel paths, and tensors of the query data, the key data, and the target value are processed to generate a context-aware target feature map; wherein the multiple paths corresponding to the parallel paths include a causal convolution path and a matrix long short-term memory path; Based on the target feature map, a learnable residual connection method is used to perform feature fusion of multi-path outputs to obtain fused features; wherein the multiple paths corresponding to the multi-path outputs include the causal convolution path and the matrix long short-term memory path; Based on the additional features obtained from the initial dimensional expansion, the fused features are scaled by dynamic weighting to obtain a scaled result, and the scaled result is projected to the original input dimension to obtain the output data of the dynamic convolutional long short-term memory module.

2. The method according to claim 1, characterized in that The method integrates multi-scale context features and captures spatial and channel dependencies through the dynamic convolutional long short-term memory module to complete feature fusion in the processing layer using a dynamic weight method, including: Normalizing the feature map input into the dynamic convolutional long short-term memory module to obtain a normalized result; Performing dimension expansion based on the normalized processing result through projection to obtain an expanded result feature map, and inputting the expanded result feature map into a causal convolution path and a matrix long short-term memory path; wherein the causal convolution path and the matrix long short-term memory path are respectively used to model different aspects of spatial and temporal dependencies; The output information of the causal convolution path and the matrix long short-term memory path is integrated through a learnable residual connection method to obtain the feature fusion result of the dynamic convolution long short-term memory module.

3. The method according to claim 2, characterized in that The initial dimension expansion of the original input dimension of the input tensor and the high-dimensional feature extraction using the causal convolution layer based on the expanded dimension to obtain the feature extraction result include performing the initial dimension expansion of the original input dimension of the input tensor and the high-dimensional feature extraction using the causal convolution layer based on the expanded dimension to obtain the feature extraction result according to the following formula: ； Among them, X Represents the input tensor, Xconv represents the feature extraction result of the convolution output after SiLU activation, SiLU represents the activation function, and CausalConv1D represents the causal convolution layer in the convolutional neural network.

4. The method according to claim 3, characterized in that The query data, the key data and the target value are expressed by the following formula: ； Wherein, Q represents the query data, K represents the key data, V represents the target value, X Represents the input tensor, Xconv represents the feature extraction result of the convolution output after SiLU activation; The processing of the tensors of the query data, the key data, and the target value to generate a context-aware target feature map includes processing the tensors of the query data, the key data, and the target value to generate a context-aware target feature map using the following formula: ； Wherein, Q represents the query data, K represents the key data, V represents the target value, and H represents the context-aware target feature map, and MatrixLSTMCell represents a matrix processing function.

5. The method according to claim 4, characterized in that The method of performing feature fusion of multi-path outputs based on the target feature map using a learnable residual connection method to obtain fused features includes balancing weights between the output information of the causal convolution path and the output information of the matrix long short-term memory path using the following formula: ； H Represents the fused features, H represents the context-aware target feature map, learnable skip represents learnable residual connection data, and Xconv represents the feature extraction result; Projecting the scaling result to the original input dimension includes down-projecting the scaling result to the original input dimension by the following formula: ； Among them, Xout represents the enhanced feature representation, H Represents the fused features, Z represents the additional features, proj_dowm represents the down-projection function, and SiLU represents the activation function.

6. The method according to claim 2, characterized in that The causal convolutional path extracts subtle local features at a high-resolution dimension higher than a specified resolution through depthwise separable convolution, so that the fracture segmentation network can capture small fracture regions smaller than a specified size; wherein the subtle local features include any one or more of the following: bone cracks, fragments, and irregular boundaries; The matrix long short-term memory path captures the anatomical structure information of the entire pelvis by modeling the long-term dependency of input features, so that the fracture segmentation network can complete global semantic expression.

7. A pelvic fracture image segmentation device based on a dynamic convolutional long short-term memory module, characterized in that: include: An acquisition module is used to acquire pelvic CT scan data to be segmented; an extraction module for extracting pelvic bones from the pelvic CT scan data using a specified anatomical segmentation network; wherein the specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture generated by pre-training on a normal pelvic CT image dataset and adjusting it for a fractured pelvis dataset; a segmentation module for performing image segmentation on bone fragments within each pelvic region of the pelvic skeleton using a fracture segmentation network including a dynamic convolutional long short-term memory module, integrating multi-scale contextual features and capturing spatial and channel dependencies through the dynamic convolutional long short-term memory module during the image segmentation process, so as to complete feature fusion in a processing layer using a dynamic weighting method, and ultimately obtaining key fracture fragments of the target bone in the pelvic CT scan data through the image segmentation process of the fracture segmentation network; wherein the fracture segmentation network includes an encoder, a decoder, and a bottleneck layer, and the bottleneck layer includes the dynamic convolutional long short-term memory module; The segmentation module is specifically used to: perform initial dimension expansion on the original input dimension of the input tensor, and use a causal convolution layer to perform high-dimensional feature extraction based on the expanded dimension to obtain a feature extraction result; wherein the causal convolution layer uses a spatial filter on the sequence; the feature extraction result is converted into query data, key data and target value by a parallel path using a shared linear projection method, and the tensors of the query data, the key data and the target value are processed to generate a context-aware target feature map; wherein the multiple paths corresponding to the parallel path include the causal convolution path and the matrix long short-term memory path; based on the target feature map, the features of the multi-path output are fused using a learnable residual connection method to obtain a fused feature; wherein the multiple paths corresponding to the multi-path output include a causal convolution path and a matrix long short-term memory path; based on the additional features obtained from the initial dimension expansion, the fused feature is scaled by a dynamic weighting method to obtain a scaled result, and the scaled result is projected to the original input dimension to obtain the output data of the dynamic convolution long short-term memory module.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the method according to any one of claims 1 to 6.

Images