Pelvic fracture image segmentation method based on dynamic convolution long-short term memory module

Abstract

The invention provides a pelvic fracture image segmentation method based on a dynamic convolution long-short term memory module, relates to the technical field of medical image segmentation, and solves the technical problem of low segmentation efficiency of a pelvic fracture CT image at present. The method comprises the following steps: extracting pelvic bones from pelvic CT scanning data by using a specified anatomical segmentation network; a fracture segmentation network comprising a dynamic convolution long-short-term memory module is used for carrying out image segmentation on bone fragments in each pelvic area in a pelvic skeleton, and in the image segmentation process, multi-scale context features are integrated through the dynamic convolution long-short-term memory module, and a space and channel dependency relationship is captured; feature fusion is completed in the processing layer by using a dynamic weight method, and finally key fracture fragments of a target skeleton in pelvis CT scanning data are obtained through an image segmentation process of a fracture segmentation network; the fracture segmentation network comprises an encoder, a decoder and a bottleneck layer, and the bottleneck layer comprises a dynamic convolution long-short time memory module.

Description

Technical Field

[0001] This application relates to the technical field of medical image segmentation, and in particular, to a pelvic fracture image segmentation method based on a dynamic convolutional long short-term memory module. Background Art

[0002] Pelvic fracture segmentation is a key link in the diagnosis and treatment planning of pelvic trauma. Accurately segmenting the pelvic fracture site in the medical images (usually CT images) of fracture patients can provide detailed cross-sectional information for clinicians, helping them accurately evaluate the fracture type, fracture displacement degree, and fracture fragment position, thereby improving the accuracy of pre-operative planning and post-operative evaluation. However, the pelvic anatomical structure is complex, the fracture types are diverse, and there are significant individual differences among different patients, making it difficult to perform automated segmentation. In addition, pelvic fractures are usually accompanied by other soft tissue injuries or bleeding, and there may be interference such as tomographic artifacts, noise, and metal implants in CT images, which all pose higher requirements for the robustness of the segmentation algorithm.

[0003] Currently, most traditional pelvic segmentation methods rely on manual or semi-automatic strategies based on thresholding, region growing, or morphological operations. By adjusting the threshold and selecting seed points, adaptive threshold segmentation and region growing methods are used to extract the bone region. Subsequently, the fracture surface is manually delineated by outlining the fragments in a three-dimensional view or modifying the mask slice by slice. Although these methods are feasible to some extent in relatively regular anatomical structures, a large number of interactive operations by professional doctors are still required when facing complex fracture morphologies, resulting in low segmentation efficiency of current pelvic fracture CT images. Summary of the Invention

[0004] The purpose of the present invention is to provide a pelvic fracture image segmentation method based on a dynamic convolutional long short-term memory module to solve the technical problem of low segmentation efficiency of current pelvic fracture CT images.

[0005] In a first aspect, the present application provides a pelvic fracture image segmentation method based on a dynamic convolutional long short-term memory module, and the method includes:

[0006] Obtain pelvic CT scan data to be segmented;

[0007] Extract pelvic bones from the pelvic CT scan data by using a specified anatomical segmentation network; wherein, the specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture, and the cascaded 3D nn-Unet architecture is generated by pre-training with a normal pelvic CT image dataset and adjusting the fractured pelvic dataset.

[0008] Use a fracture segmentation network containing a Dynamic Convolutional LSTM (DyCoLSTM) module to perform image segmentation on bone fragments in each pelvic region of the pelvic bone. During the image segmentation process, integrate multi-scale context features and capture spatial and channel dependencies through the Dynamic Convolutional LSTM module, so as to complete feature fusion using the dynamic weight method in the processing layer. Finally, obtain the key fracture fragments of the target bone in the pelvic CT scan data through the image segmentation process of the fracture segmentation network. Among them, the fracture segmentation network includes an encoder, a decoder, and a bottleneck layer, and the bottleneck layer contains the Dynamic Convolutional LSTM module.

[0009] In a possible implementation, the integration of multi-scale context features and capture of spatial and channel dependencies through the Dynamic Convolutional LSTM module to complete feature fusion using the dynamic weight method in the processing layer includes:

[0010] Normalize the feature map input to the Dynamic Convolutional LSTM module to obtain the normalization result.

[0011] Based on the normalization result, perform dimension expansion through projection to obtain an expanded result feature map, and input the expanded result feature map into the causal convolution path and the matrix long short-term memory path. Among them, the causal convolution path and the matrix long short-term memory path are respectively used to model different aspects of spatial and temporal dependencies.

[0012] Integrate the output information of the causal convolution path and the matrix long short-term memory path through a learnable residual connection method to obtain the feature fusion result of the Dynamic Convolutional LSTM module.

[0013] In a possible implementation, the integration of multi-scale context features and capture of spatial and channel dependencies through the Dynamic Convolutional LSTM module to complete feature fusion using the dynamic weight method in the processing layer includes:

[0014] Perform initial dimension expansion on the original input dimension of the input tensor, and perform high-dimensional feature extraction using a causal convolutional layer based on the expanded dimension to obtain the feature extraction result. Among them, the causal convolutional layer uses a spatial filter in the sequence.

[0015] Convert the feature extraction result into query data, key data, and target values through a parallel path using a shared linear projection method, and process the tensors of the query data, the key data, and the target values to generate a context-aware target feature map. Among them, the multiple paths corresponding to the parallel path include the causal convolution path and the matrix long short-term memory path.

[0016] Perform feature fusion with multi-path output using learnable residual connections based on the target feature map to obtain the fused feature; wherein, the multiple paths corresponding to the multi-path output include the causal convolution path and the matrix long short-term memory path;

[0017] Based on the additional features obtained from the initial dimension expansion, scale the fused feature through dynamic weighting to obtain a scaling result, and project the scaling result to the original input dimension to obtain the output data of the dynamic convolutional long short-term memory module.

[0018] In a possible implementation, the initial dimension expansion of the original input dimension of the input tensor, and high-dimensional feature extraction using a causal convolution layer based on the expanded dimension to obtain a feature extraction result, including initial dimension expansion of the original input dimension of the input tensor through the following formula and high-dimensional feature extraction using a causal convolution layer based on the expanded dimension to obtain a feature extraction result:

[0019] X conv = SiLU(CausalConv1D(X in ));

[0020] wherein, X in represents the input tensor, Xconv represents the feature extraction result of the convolution output after being activated by SiLU, SiLU represents the activation function, and CausalConv1D represents the causal convolution layer in the convolutional neural network.

[0021] In a possible implementation, the query data, the key data, and the target value are represented by the following formula:

[0022] Q = q_proj(X conv ), K = k_proj(X conv ), V = v_proj(X in );

[0023] wherein, Q represents the query data, K represents the key data, V represents the target value, X in represents the input tensor, and X conv represents the feature extraction result of the convolution output after being activated by SiLU;

[0024] The processing of the tensors of the query data, the key data, and the target value to generate a context-aware target feature map includes processing the tensors of the query data, the key data, and the target value through the following formula to generate a context-aware target feature map:

[0025] H mLSTM= MatrixLSTMCell(Q, K, V);

[0026] Among them, Q represents the query data, K represents the key data, V represents the target value, and H mLSTM represents the target feature map with context awareness, and MatrixLSTMCell represents a matrix processing function.

[0027] In a possible implementation, the feature fusion of multi-path output is performed on the basis of the target feature map by using a learnable residual connection method to obtain the fused feature, including balancing the weights between the output information of the causal convolution path and the output information of the matrix long short-term memory path through the following formula:

[0028] H skip = H mLSTM + (learnable_skip × X conv );

[0029] H skip represents the fused feature, H mLSTM represents the target feature map with context awareness, learnable_skip represents learnable residual connection data, and Xconv represents the feature extraction result;

[0030] The projection of the scaling result to the original input dimension includes projecting the scaling result down to the original input dimension through the following formula:

[0031] X out = proj_down(H skip × SiLU(Z));

[0032] Among them, Xout represents the enhanced feature representation, H skip represents the fused feature, Z represents the additional feature, proj_down represents the down-projection function, and SiLU represents the activation function.

[0033] In a possible implementation, the causal convolution path extracts fine-grained local features in the high-resolution dimension higher than the specified resolution through depthwise separable convolution, so that the fracture segmentation network can capture small fracture regions smaller than the specified size; among them, the fine-grained local features include any one or more of the following: bone cracks, fragments, and irregular boundaries;

[0034] The matrix long short-term memory path captures the anatomical structure information of the whole pelvis by modeling the long-term dependence relationship of the input features, so that the fracture segmentation network can complete the global semantic expression.

[0035] In a second aspect, the present application provides a pelvic fracture image segmentation device based on a dynamic convolutional long short-term memory module, including:

[0036] An acquisition module for acquiring pelvic CT scan data to be segmented.

[0037] An extraction module for extracting pelvic bones from the pelvic CT scan data by using a specified anatomical segmentation network; wherein, the specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture, and the cascaded 3D nn-Unet architecture is generated by pre-training with a normal pelvic CT image dataset and adjusting a fractured pelvic dataset.

[0038] A segmentation module for performing image segmentation on bone fragments in each pelvic region of the pelvic bones by using a fracture segmentation network including a dynamic convolutional long short-term memory module, integrating multi-scale context features and capturing spatial and channel dependencies through the dynamic convolutional long short-term memory module during the image segmentation process, to complete feature fusion by using a dynamic weight method in the processing layer, and finally obtaining key fracture fragments of the target bones in the pelvic CT scan data through the image segmentation process of the fracture segmentation network; wherein, the fracture segmentation network includes an encoder, a decoder, and a bottleneck layer, and the bottleneck layer includes the dynamic convolutional long short-term memory module.

[0039] In a third aspect, the present application further provides an electronic device including a memory and a processor, where a computer program executable on the processor is stored in the memory, and when the processor executes the computer program, the method described in the first aspect above is implemented.

[0040] In a fourth aspect, the present application further provides a computer-readable storage medium storing computer-executable instructions, and when the computer-executable instructions are called and run by a processor, the computer-executable instructions cause the processor to run the method described in the first aspect above.

[0041] The present application brings the following beneficial effects:

[0042] A pelvic fracture image segmentation method based on a dynamic convolutional long short-term memory module provided by the present application can obtain pelvic CT scan data to be segmented; extract pelvic bones from the pelvic CT scan data by using a specified anatomical segmentation network; wherein, the specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture, and the cascaded 3D nn-Unet architecture is generated by pre-training with a normal pelvic CT image dataset and adjusting a fractured pelvic dataset; use a fracture segmentation network containing a dynamic convolutional long short-term memory module to perform image segmentation on bone fragments in each pelvic region of the pelvic bones, and integrate multi-scale context features and capture spatial and channel dependencies through the dynamic convolutional long short-term memory module during the image segmentation process, so as to complete feature fusion by using a dynamic weight method in the processing layer, and finally obtain key fracture fragments of the target bones in the pelvic CT scan data through the image segmentation process of the fracture segmentation network; wherein, the fracture segmentation network includes an encoder, a decoder and a bottleneck layer, and the bottleneck layer includes the dynamic convolutional long short-term memory module. In this solution, the dynamic convolutional long short-term memory module with a visual feature representation effect in the fracture segmentation network integrates multi-scale context features, captures spatial and channel dependencies, thereby realizing the modeling of multi-scale features, the integration of spatial and channel dependencies, and the modeling of remote context information, enhancing the ability to model local and global dependencies in the bottleneck layer, ensuring efficient feature fusion in its processing layer, solving the problem of insufficient local and global feature expression, thereby improving the overall segmentation efficiency of the fracture segmentation network for fracture fragments in pelvic CT scans, and solving the technical problem of low segmentation efficiency of current pelvic fracture CT images.

[0043] To make the above objects, features and advantages of the present application more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, makes a detailed description as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0045] Figure 1 It is a schematic flowchart of the pelvic fracture image segmentation method based on the dynamic convolutional long short-term memory module provided by the embodiment of the present application;

[0046] Figure 2Another flowchart of the pelvic fracture image segmentation method based on the dynamic convolutional long short-term memory module provided by the embodiments of the present application;

[0047] Figure 3 An example of the internal structure of the dynamic convolutional long short-term memory module DyCoLSTM in the pelvic fracture image segmentation method based on the dynamic convolutional long short-term memory module provided by the embodiments of the present application;

[0048] Figure 4 A structural diagram of a pelvic fracture image segmentation device based on the dynamic convolutional long short-term memory module provided by the embodiments of the present application;

[0049] Figure 5 A structural diagram of an electronic device provided by the embodiments of the present application is shown. Detailed implementation manners

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings. Apparently, the described embodiments are some but not all of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0051] The terms "including" and "having" and any variations thereof mentioned in the embodiments of the present application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes other steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0052] With the rapid development of medical image processing and artificial intelligence technologies, more and more research has begun to attempt to improve the efficiency and accuracy of pelvic fracture segmentation through automatic segmentation algorithms based on deep learning. Typical methods include using convolutional neural networks (CNNs) or hybrid network structures to automatically segment the pelvic region and detect fractures. Currently, deep convolutional neural networks (CNNs) can be used in combination with multi-directional block images for pelvic fracture detection. By processing CT images from multiple perspectives, the network can capture the characteristics of the pelvic region more comprehensively, thereby improving the accuracy of fracture detection. Although this method provides a fast and accurate fracture diagnosis tool for clinical use, its output cannot provide a fully automated solution for subsequent operations.

[0053] In the pelvic fracture segmentation task, due to the complex pelvic anatomical structure, diverse fracture region morphologies, and high uncertainty, there are many problems with current pelvic fracture segmentation methods. For example, the expression of local and global features is insufficient: the morphology and position of the pelvic fracture region have significant uncertainty, and its distribution may cover a large range of the pelvic region. Moreover, traditional segmentation methods also have the following problems when capturing global structures and local details. First, the global semantic expression is insufficient: it is difficult to make full use of the anatomical structure information of the entire pelvis, resulting in poor performance in global consistency of the segmentation results. Second, the lack of coordinated expression between local and global: existing methods often cannot establish an effective balance between local details and global context, resulting in deviations in details and integrity of the segmentation results. Thus, it can be seen that the current segmentation efficiency of pelvic fracture CT images is low.

[0054] In addition, the spatial and channel dependencies of traditional segmentation methods are weakened. The fracture region may show discontinuity in the spatial dimension (for example, bone fractures may be distributed at different positions in the pelvis), while in the channel dimension (for example, different channels of multi-modal medical images or feature channels of segmentation models), there may be implicit associations. Conventional segmentation models generally ignore the following aspects during feature extraction and fusion: First, the discontinuity of spatial features: the spatial distribution of the fracture region is complex and may be discontinuous, and existing models are difficult to capture these non-linear features through a single scale or simple convolution. Second, the lack of interaction between channels: the associated features between different channels (such as different window widths and window levels of CT images) are not fully explored, resulting in a reduction in the integrity of feature expression. Third, the lack of comprehensiveness across space and channels: existing methods are difficult to co-model spatial dependencies and channel implicit relationships, and the feature fusion efficiency is not high. Therefore, the current segmentation efficiency of pelvic fracture CT images is low.

[0055] Based on this, the embodiments of this application provide a pelvic fracture image segmentation method based on a dynamic convolutional long short-term memory module, which can solve the technical problem of the low current segmentation efficiency of pelvic fracture CT images.

[0056] The embodiments of the present invention will be further introduced below with reference to the accompanying drawings.

[0057] Figure 1 It is a schematic flowchart of a pelvic fracture image segmentation method based on a dynamic convolutional long short-term memory module provided by the embodiments of this application. As Figure 1 shown, the method includes:

[0058] Step S110, obtain pelvic CT scan data to be segmented.

[0059] In the embodiment of the present application, first, pelvic CT scan data to be segmented is obtained, and the pelvic CT scan data herein is the pelvic fracture CT image (pelvic fracture CT picture), so as to automatically segment the main fragments of the target bone from the pelvic fracture CT image.

[0060] Step S120: Extract pelvic bones from the pelvic CT scan data by using a specified anatomical segmentation network.

[0061] The specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture, and the cascaded 3D nn-Unet architecture is generated by pre-training with a normal pelvic CT image dataset and adjusting the fractured pelvic dataset.

[0062] The overall segmentation process of the pelvic fracture CT image includes two stages. As Figure 2 shown, in the first stage, first use an anatomical segmentation network (specified anatomical segmentation network) to extract pelvic bones from the CT scan. The anatomical segmentation network adopts a cascaded 3D nn-Unet architecture, which is pre-trained on a healthy pelvic CT image dataset (normal pelvic CT image dataset) and further fine-tuned on the fractured pelvic dataset.

[0063] Step S130: Use a fracture segmentation network containing a dynamic convolutional long short-term memory module to perform image segmentation on the bone fragments in each pelvic region of the pelvic bones. In the image segmentation process, integrate multi-scale context features and capture spatial and channel dependencies through the dynamic convolutional long short-term memory module, so as to complete feature fusion by using the dynamic weight method in the processing layer. Finally, obtain the key fracture fragments of the target bone in the pelvic CT scan data through the image segmentation process of the fracture segmentation network.

[0064] Among them, the fracture segmentation network includes an encoder, a decoder, and a bottleneck layer, and the bottleneck layer includes a dynamic convolutional long short-term memory module (Dynamic Convolutional LSTM, DyCoLSTM).

[0065] In a possible implementation manner, as Figure 2 shown, in the second stage of the overall segmentation process of segmenting the pelvic fracture CT image, apply a fracture segmentation network (FractureSeg network) to segment the bone fragments in each pelvic region.

[0066] The dynamic convolutional long short-term memory module with visual feature representation effect integrates multi-scale context features, captures spatial and channel dependencies, thereby realizing the modeling of multi-scale features, the integration of spatial and channel dependencies, and the modeling of remote context information, enhancing the ability to model local and global dependencies in the bottleneck layer, ensuring efficient feature fusion in its processing layer, solving the problem of insufficient local and global feature expressions, and being able to improve the overall segmentation efficiency of the fracture segmentation network for fracture fragments in pelvic CT scans.

[0067] In the embodiments of the present application, aiming at the uncertainty of the morphology and position of the pelvic fracture area, the dynamic convolutional long short-term memory module proposes an efficient feature modeling method by combining convolutions to simultaneously solve the problems of insufficient local feature capture, lack of global semantic expression, and insufficient local-global collaborative expression. The solution provided by the embodiments of the present application provides important technical support for fracture diagnosis and treatment planning in medical image processing and has broad clinical application prospects.

[0068] The above steps will be introduced in detail below.

[0069] In some embodiments, in the above step S130, multi-scale context features are integrated through the dynamic convolutional long short-term memory module and spatial and channel dependencies are captured to complete feature fusion using the dynamic weight method in the processing layer, which may specifically include the following steps:

[0070] The feature map input to the dynamic convolutional long short-term memory module is normalized to obtain the normalization result; based on the normalization result, dimensionality expansion is performed through projection to obtain the expanded result feature map, and the expanded result feature map is input to the causal convolutional path and the matrix long short-term memory path; wherein, the causal convolutional path and the matrix long short-term memory path are respectively used to model different aspects of spatial and temporal dependencies; the output information of the causal convolutional path and the matrix long short-term memory path is integrated through a learnable residual connection method to obtain the feature fusion result of the dynamic convolutional long short-term memory module.

[0071] Exemplarily, as Figure 3 shown, in the dynamic convolutional long short-term memory module DyCoLSTM, the input feature map is first normalized and then dimensionality expanded through a series of projections. The resulting feature map is fed into two independent paths: a causal convolutional path and an mLSTM (matrix long short-term memory) path, which are respectively responsible for modeling different aspects of spatial and temporal dependencies. The final feature fusion of DyCoLSTM integrates the information of the two paths through a learnable residual connection to generate a rich representation of the input. Thus, in terms of the collaborative expression of local and global features, feature fusion is completed using the dynamic weight method, realizing the design of adaptively fusing features extracted from different dimensions.

[0072] Introduce a dynamic weight mechanism through a dynamic convolutional long short-term memory module, and use the fusion strategy of matrix long short-term memory mLSTM and convolutional paths to establish an effective balance between local details and global context, ensuring that the segmentation results are both locally refined and globally structurally consistent.

[0073] In some embodiments, in the above step S130, multi-scale context features are integrated through a dynamic convolutional long short-term memory module, and spatial and channel dependencies are captured to complete feature fusion using a dynamic weight method in the processing layer, which may specifically include the following steps:

[0074] Perform an initial dimension expansion on the original input dimension of the input tensor, and based on the expanded dimension, use a causal convolutional layer for high-dimensional feature extraction to obtain a feature extraction result; wherein, the causal convolutional layer uses a spatial filter on the sequence.

[0075] Convert the feature extraction result into query data, key data, and target values through parallel paths using a shared linear projection, and process the tensors of the query data, key data, and target values to generate a context-aware target feature map; wherein, the multiple paths corresponding to the parallel paths include a causal convolutional path and a matrix long short-term memory path.

[0076] Based on the target feature map, perform feature fusion of multi-path outputs using a learnable residual connection method to obtain a fused feature; wherein, the multiple paths corresponding to the multi-path outputs include a causal convolutional path and a matrix long short-term memory path.

[0077] Based on the additional features obtained from the initial dimension expansion, scale the fused feature through a dynamic weighting method to obtain a scaled result, and project the scaled result to the original input dimension to obtain the output data of the dynamic convolutional long short-term memory module.

[0078] For dimension expansion and high-dimensional feature extraction, to support multi-scale processing, in the embodiments of the present application, the dimension of the input tensor is first expanded. For high-dimensional feature extraction, in the embodiments of the present application, a causal convolutional layer is used, and this layer applies a spatial filter on the sequence.

[0079] For the matrix long short-term memory mLSTM path to model temporal dependencies, in the parallel paths, a shared linear projection is used to convert the feature map into queries (Q), keys (K), and values (V) to support long-term dependency modeling and enable the dynamic convolutional long short-term memory module DyCoLSTM to focus on recent and long-term features in the input sequence. As Figure 3 shown, the matrix long short-term memory mLSTM module then processes these query, key, and value tensors to generate a context-aware feature map

[0080] For feature fusion and dynamic weighting, to effectively fuse the outputs of the two paths, the dynamic convolutional long short-term memory module DyCoLSTM uses learnable skip connections to balance the weights between the matrix long short-term memory mLSTM and the convolutional output, as Figure 3 shown. Finally, the fused output is scaled through a dynamic weighting mechanism using additional features derived from the initial dimension expansion, and the result is projected down to the original input dimension through the following formula.

[0081] Through the shared projection and multi-dimensional processing configuration, the dynamic convolutional long short-term memory module DyCoLSTM can efficiently fuse spatial and channel dependencies, which is crucial for effective visual feature modeling.

[0082] In some embodiments, the above-mentioned initial dimension expansion of the original input dimension of the input tensor is performed, and high-dimensional feature extraction is carried out using a causal convolutional layer based on the expanded dimension to obtain a feature extraction result. Specifically, it can include performing the initial dimension expansion of the original input dimension of the input tensor through the following formula and using a causal convolutional layer based on the expanded dimension to carry out high-dimensional feature extraction to obtain a feature extraction result:

[0083] X conv = SiLU(CausalConv1D(X in ));

[0084] where X in represents the input tensor, X conv represents the feature extraction result of the convolutional output after SiLU activation, SiLU represents the activation function, and CausalConv1D represents the causal convolutional layer in the convolutional neural network.

[0085] In the embodiments of the present application, through the data processing method of the above formula, the data of the high-dimensional feature extraction result can be made more accurate.

[0086] In some embodiments, the query data, key data, and target value are represented by the following formula:

[0087] Q = q_proj(X conv ), K = k_proj(X conv ), V = v_proj(X in );

[0088] where Q represents the query data, K represents the key data, V represents the target value, X in represents the input tensor, X conv represents the feature extraction result of the convolutional output after SiLU activation;

[0089] Process the tensors of query data, key data, and target values to generate a context-aware target feature map, including processing the tensors of query data, key data, and target values through the following formula to generate a context-aware target feature map:

[0090] H mLSTM = MatrixLSTMCell(Q, K, V);

[0091] where Q represents query data, K represents key data, V represents target values, and H mLSTM represents the context-aware target feature map, and MatrixLSTMCell represents a matrix processing function.

[0092] In the embodiments of the present application, through the data processing method of the above formula, the data of the generated context-aware target feature map can be made more accurate.

[0093] In some embodiments, the above-mentioned feature fusion of multi-path output using a learnable residual connection method based on the target feature map to obtain the fused feature may specifically include balancing the weights between the output information of the causal convolution path and the output information of the matrix long short-term memory path through the following formula:

[0094] H skip = H mLSTM + (learnable_skip × X conv );

[0095] H skip represents the fused feature, H mLSTM represents the context-aware target feature map, learnable_skip represents learnable residual connection data, and X conv represents the feature extraction result;

[0096] Project the scaling result to the original input dimension, including projecting the scaling result down to the original input dimension through the following formula:

[0097] X out = proj_down(H skip × SiLU(Z));

[0098] where X out represents the enhanced feature representation, H skip represents the fused feature, Z represents additional features, proj_down represents a down-projection function, and SiLU represents an activation function.

[0099] In the embodiments of the present application, through the data processing method of the above formula, the data of the feature fusion result and the projection of the scaling result to the original input dimension can be made more accurate.

[0100] In some embodiments, the above-mentioned causal convolution path extracts subtle local features in the high-resolution dimension higher than the specified resolution through depthwise separable convolution, so that the fracture segmentation network can capture small fracture regions smaller than the specified size; wherein, the subtle local features include any one or more of the following: bone fractures, fragments, and irregular boundaries; the matrix long short-term memory path captures the anatomical structure information of the whole pelvis by modeling the long-term dependence relationship of the input features, so that the fracture segmentation network can complete global semantic expression.

[0101] For the accurate capture of local features, the causal convolution path of the dynamic convolution long short-term memory module DyCoLSTM extracts subtle local features such as bone fractures, fragments, or irregular boundaries in the high-resolution dimension through large kernel depthwise separable convolution. This design enhances the model's ability to capture small fracture regions and improves the accuracy of segmentation.

[0102] For the enhancement of global semantic expression, the matrix long short-term memory mLSTM path of the dynamic convolution long short-term memory module DyCoLSTM captures the anatomical structure information of the whole pelvis by modeling the long-term dependence relationship of the input features, enhancing the model's global semantic expression ability. By introducing shared linear projection, DyCoLSTM establishes global associations between feature dimensions, ensuring the performance of the segmentation result in terms of global consistency.

[0103] Figure 4 A structural schematic diagram of a pelvic fracture image segmentation device based on a dynamic convolution long short-term memory module is provided. As Figure 4 shown, the pelvic fracture image segmentation device 400 based on the dynamic convolution long short-term memory module includes:

[0104] An acquisition module 401, configured to acquire pelvic CT scan data to be segmented.

[0105] An extraction module 402, configured to extract pelvic bones from the pelvic CT scan data by using a specified anatomical segmentation network; wherein, the specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture, and the cascaded 3D nn-Unet architecture is generated by pre-training with a normal pelvic CT image dataset and adjusting the fractured pelvic dataset.

[0106] A segmentation module 403 is configured to perform image segmentation on bone fragments within each pelvic region of the pelvic bone by using a fracture segmentation network including a dynamic convolutional long short-term memory module. During the image segmentation process, the dynamic convolutional long short-term memory module integrates multi-scale context features and captures spatial and channel dependencies to complete feature fusion by using a dynamic weight method in a processing layer. Finally, key fracture fragments of the target bone in the pelvic CT scan data are obtained through the image segmentation process of the fracture segmentation network. The fracture segmentation network includes an encoder, a decoder, and a bottleneck layer, and the bottleneck layer includes the dynamic convolutional long short-term memory module.

[0107] The pelvic fracture image segmentation device based on the dynamic convolutional long short-term memory module provided by the embodiments of the present application has the same technical features as the pelvic fracture image segmentation method based on the dynamic convolutional long short-term memory module provided by the above embodiments. Therefore, it can also solve the same technical problems and achieve the same technical effects.

[0108] An electronic device provided by an embodiment of the present application, as Figure 5 shown, the electronic device 500 includes a processor 502 and a memory 501. A computer program that can run on the processor is stored in the memory. When the processor executes the computer program, the steps of the method provided by the above embodiments are implemented.

[0109] See Figure 5 , the electronic device further includes: a bus 503 and a communication interface 504. The processor 502, the communication interface 504, and the memory 501 are connected through the bus 503. The processor 502 is configured to execute an executable module stored in the memory 501, such as a computer program.

[0110] Among them, the memory 501 may include a high-speed random access memory (Random Access Memory, abbreviated as RAM), and may also include a non-volatile memory, such as at least one disk memory. Through at least one communication interface 504 (which can be wired or wireless), a communication connection between the system network element and at least one other network element is realized, and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used.

[0111] The bus 503 may be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 5 only a bidirectional arrow is used in

[0112] Among them, the memory 501 is used to store a program. After receiving an execution instruction, the processor 502 executes the program. The method executed by the device defined by any of the processes disclosed in the foregoing embodiments of the present application can be applied to or implemented by the processor 502.

[0113] The processor 502 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor 502 or instructions in software form. The above-mentioned processor 502 may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 501, and the processor 502 reads the information in the memory 501 and combines its hardware to complete the steps of the above method.

[0114] Corresponding to the above-mentioned pelvic fracture image segmentation method based on a dynamic convolutional long short-term memory module, an embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and run by a processor, the computer-executable instructions cause the processor to run the steps of the above-mentioned pelvic fracture image segmentation method based on a dynamic convolutional long short-term memory module.

[0115] The pelvic fracture image segmentation device based on the dynamic convolutional long short-term memory module provided by the embodiments of the present application can be specific hardware on a device or software or firmware installed on the device. For the device provided by the embodiments of the present application, its implementation principle and the technical effects produced are the same as those of the foregoing method embodiments. For the sake of brief description, for the parts not mentioned in the device embodiments, reference may be made to the corresponding content in the foregoing method embodiments. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the foregoing-described systems, devices, and units can all refer to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0116] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces. The indirect coupling or communication connection of the devices or units can be in an electrical, mechanical, or other form.

[0117] For another example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0118] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0119] In addition, each functional unit in the embodiments provided in this application may be integrated into a processing unit, may exist separately as individual physical units, or two or more units may be integrated into one unit.

[0120] If the above-mentioned function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the pelvic fracture image segmentation method based on the dynamic convolutional long short-term memory module described in each embodiment of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0121] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0122] Finally, it should be noted that the above-mentioned embodiments are only specific implementation manners of this application, used to illustrate the technical solutions of this application, and are not intended to limit it. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed in this application can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or make equivalent replacements for some of the technical features; and these modifications, changes, or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A pelvic fracture image segmentation method based on dynamic convolution long short-term memory module, characterized in that: The method comprises: Acquire pelvic CT scan data to be segmented; Extracting pelvic bones from the pelvic CT scan data using a specified anatomical segmentation network; wherein the specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture, and the cascaded 3D nn-Unet architecture is generated by pre-training with a normal pelvic CT image dataset and adjusting a fractured pelvic dataset; A fracture segmentation network including a dynamic convolutional long short-term memory module is used to perform image segmentation on bone fragments in each pelvic region of the pelvic skeleton. During the image segmentation process, the dynamic convolutional long short-term memory module is used to integrate multi-scale context features and capture spatial and channel dependencies, so as to complete feature fusion in the processing layer using a dynamic weight method, and finally the key fracture fragments of the target bone in the pelvic CT scan data are obtained through the image segmentation process of the fracture segmentation network; wherein the fracture segmentation network includes an encoder, a decoder and a bottleneck layer, and the bottleneck layer includes the dynamic convolutional long short-term memory module.

2. The method according to claim 1, characterized in that The method integrates multi-scale context features and captures spatial and channel dependencies through the dynamic convolution long short-term memory module to complete feature fusion in the processing layer using a dynamic weight method, including: Normalizing the feature map input into the dynamic convolutional long short-term memory module to obtain a normalized result; Based on the normalized processing result, dimension expansion is performed by projection to obtain an extended result feature map, and the extended result feature map is input into a causal convolution path and a matrix long short-term memory path; wherein the causal convolution path and the matrix long short-term memory path are respectively used to model different aspects of spatial and temporal dependencies; The output information of the causal convolution path and the matrix long short-term memory path is integrated through a learnable residual connection method to obtain the feature fusion result of the dynamic convolution long short-term memory module.

3. The method according to claim 2, characterized in that The method integrates multi-scale context features and captures spatial and channel dependencies through the dynamic convolution long short-term memory module to complete feature fusion in the processing layer using a dynamic weight method, including: Performing initial dimension expansion on the original input dimension of the input tensor, and performing high-dimensional feature extraction using a causal convolution layer based on the expanded dimension to obtain a feature extraction result; wherein the causal convolution layer uses a spatial filter on the sequence; The feature extraction result is converted into query data, key data and target value by means of shared linear projection through parallel paths, and tensors of the query data, the key data and the target value are processed to generate a context-aware target feature map; wherein the multiple paths corresponding to the parallel paths include the causal convolution path and the matrix long short-term memory path; Based on the target feature graph, a learnable residual connection method is used to perform feature fusion of multi-path outputs to obtain fused features; wherein the multiple paths corresponding to the multi-path outputs include the causal convolution path and the matrix long short-term memory path; Based on the additional features obtained from the initial dimensional expansion, the fused features are scaled by a dynamic weighting method to obtain a scaled result, and the scaled result is projected to the original input dimension to obtain output data of the dynamic convolutional long short-term memory module.

4. The method according to claim 3, characterized in that The initial dimension expansion is performed on the original input dimension of the input tensor, and a high-dimensional feature extraction is performed using a causal convolution layer based on the expanded dimension to obtain a feature extraction result, including performing an initial dimension expansion on the original input dimension of the input tensor and a high-dimensional feature extraction is performed using a causal convolution layer based on the expanded dimension to obtain a feature extraction result by using the following formula: X conv =YesLU(CausalConv1D(X in )): Among them, X in represents the input tensor, X conv It represents the feature extraction result of the convolution output after SiLU activation, SiLU represents the activation function, and CausalConv1D represents the causal convolution layer in the convolutional neural network.

5. The method according to claim 4, characterized in that The query data, the key data and the target value are expressed by the following formula: Q=q_proj(X conv ), K = k_proj(X conv ), V = v_proj(X in ); Wherein, Q represents the query data, K represents the key data, V represents the target value, and X in represents the input tensor, X conv Represents the feature extraction result of the convolution output after SiLU activation; The processing of the tensors of the query data, the key data, and the target value to generate a context-aware target feature map includes processing the tensors of the query data, the key data, and the target value to generate a context-aware target feature map by the following formula: H mLSTM =MatrixLSTMCell(Q,K,V); Wherein, Q represents the query data, K represents the key data, V represents the target value, and H mLSTM represents the context-aware target feature map, and MatrixLSTMCell represents the matrix processing function.

6. The method according to claim 5, characterized in that The method of performing feature fusion of multi-path outputs based on the target feature graph using a learnable residual connection method to obtain fused features includes balancing the weights between the output information of the causal convolution path and the output information of the matrix long short-term memory path by the following formula: H skip =H mLSTM +(learnable_skip×X conv ); H skip represents the fused features, H mLSTM represents the context-aware target feature map, learnableskip represents the learnable residual connection data, X conv represents the feature extraction result; Projecting the scaling result to the original input dimension includes down-projecting the scaling result to the original input dimension by the following formula: X out =proj_down(H skip ×SiLU(Z)); Among them, X out represents the enhanced feature representation, H skip Represents the fused features, Z represents the additional features, projdowm represents the downward projection function, and SiLU represents the activation function.

7. The method according to claim 2, characterized in that The causal convolution path extracts subtle local features at a high-resolution dimension higher than a specified resolution through a depthwise separable convolution, so that the fracture segmentation network captures small fracture regions smaller than a specified size; wherein the subtle local features include any one or more of the following: bone cracks, fragments, and irregular boundaries; The matrix long short-term memory path captures the anatomical structure information of the entire pelvis by modeling the long-term dependency of input features, so that the fracture segmentation network completes global semantic expression.

8. A pelvic fracture image segmentation device based on a dynamic convolution long short-term memory module, characterized in that: include: An acquisition module, used for acquiring pelvic CT scan data to be segmented; An extraction module for extracting pelvic bones from the pelvic CT scan data using a specified anatomical segmentation network; wherein the specified anatomical segmentation network is an optimized cascaded 3D nn-Unet architecture, and the cascaded 3D nn-Unet architecture is generated by pre-training with a normal pelvic CT image dataset and adjusting a fractured pelvic dataset; A segmentation module is used to perform image segmentation on bone fragments in each pelvic region of the pelvic bone using a fracture segmentation network including a dynamic convolutional long short-term memory module. In the image segmentation process, multi-scale context features are integrated and spatial and channel dependencies are captured through the dynamic convolutional long short-term memory module, so as to complete feature fusion in the processing layer using a dynamic weight method, and finally obtain key fracture fragments of the target bone in the pelvic CT scan data through the image segmentation process of the fracture segmentation network; wherein the fracture segmentation network includes an encoder, a decoder and a bottleneck layer, and the bottleneck layer includes the dynamic convolutional long short-term memory module.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the method according to any one of claims 1 to 7.

Images