Lumbar multi-plane MR image reconstruction segmentation method, device and equipment and storage medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- YANSHAN UNIV
- Filing Date
- 2026-05-08
- Publication Date
- 2026-07-10
Smart Images
Figure CN122368076A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and storage medium for multiplanar MR image reconstruction and segmentation of the lumbar spine. Background Technology
[0002] In the clinical analysis and auxiliary diagnosis of lumbar spine MRI images, high-quality image reconstruction and accurate anatomical structure segmentation are two key tasks. However, existing technologies typically treat reconstruction and segmentation as two separate processes: first, the k-space data obtained from undersampled acquisition is reconstructed, and then the reconstructed image is input into a segmentation network for analysis. This separate processing approach has significant drawbacks: First, the reconstruction network only aims at fidelity or visual quality, lacking awareness of the requirements of the downstream segmentation task, resulting in reconstruction results that may not be optimal for segmentation in terms of edge details and contrast. Second, existing segmentation methods usually rely on high-quality input images, but under undersampled conditions, noise, artifacts, and missing details in the reconstructed image reduce the segmentation accuracy of structures such as the vertebral body, intervertebral disc, and dural sac. Traditional reconstruction methods struggle to preserve the fine features of anatomical structures while suppressing artifacts, affecting segmentation accuracy. Third, lumbar spine image acquisition often involves two different planes: sagittal and axial. The morphology, category settings, and imaging characteristics of target structures differ in these two planes, and existing methods often design models for a single plane, making it difficult to effectively generalize between the two planes. The aforementioned issues limit the accuracy of automated analysis of lumbar spine MR images under undersampling conditions, and also affect the efficiency and reliability of clinical diagnosis. Summary of the Invention
[0003] This invention provides a method, apparatus, device, and storage medium for lumbar spine multiplanar MR image reconstruction and segmentation to solve the problem of poor reconstruction and segmentation accuracy of lumbar spine MR images under undersampling conditions.
[0004] In a first aspect, embodiments of the present invention provide a method for multiplanar MR image reconstruction and segmentation of the lumbar spine, including: Acquire undersampled observation data; The undersampled observation data is input into the reconstruction network to obtain the reconstructed image corresponding to the undersampled observation data. The reconstruction network includes multiple stages, each stage including a data fidelity module, a priori enhancement module, and a fusion update module. The priori enhancement module is an encoder-decoder structure based on Mamba units. The reconstructed image is input into the segmentation network to obtain the segmentation result corresponding to the reconstructed image. The segmentation result includes multiple lumbar spine regions. The segmentation network and the reconstruction network are trained end-to-end based on undersampled observation data of the lumbar spine multi-plane.
[0005] In one possible implementation, the segmentation network includes a Mamba-based encoder-decoder structure, a sagittal output header, and an axial output header; The reconstructed image is input into the segmentation network to obtain the segmentation results corresponding to the reconstructed image, including: When the acquisition plane corresponding to the undersampled observation data is sagittal, the segmentation network outputs the sagittal segmentation result corresponding to the reconstructed image through the sagittal output head; When the acquisition plane corresponding to the undersampled observation data is axial, the segmentation network outputs the axial segmentation result corresponding to the reconstructed image through the axial output head.
[0006] In one possible implementation, before inputting the undersampled observation data into the reconstruction network to obtain the reconstructed image corresponding to the undersampled observation data, the following steps are also included: The VSS block is self-supervised and trained based on the first training sample set and the SimMIM strategy to obtain the basic model weights of the VSS block; the training sample set includes MR image data of the lumbar spine in multiple planes. The weights of the base model are loaded into each prior enhancement module and the segmentation network to obtain the initialized reconstruction network and segmentation network. The initial reconstruction network and segmentation network are jointly trained based on the second training sample set and the joint loss function to obtain the trained reconstruction network and segmentation network.
[0007] In one possible implementation, the VSS block includes a first normalization layer, a second normalization layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, a depthwise separable convolutional layer, and an SS2D layer. The input features are divided into two paths after passing through the first normalization layer. The first path passes through the first fully connected layer, the depthwise separable convolutional layer, the SS2D layer, and the second normalization layer. The second path passes through the third fully connected layer. The two paths are multiplied element-wise and then mapped through the second fully connected layer, and residual connections are made with the input features.
[0008] In one possible implementation, the joint loss function includes reconstruction loss and segmentation loss; The reconstruction loss is calculated based on the difference between the reconstructed image and the original image. The segmentation loss includes Dice loss, cross-entropy loss, and depth supervision loss.
[0009] In one possible implementation, before jointly training the initialized reconstruction network and segmentation network based on the second training sample set and the joint loss function to obtain the trained reconstruction network and segmentation network, the following steps are also included: Acquire multiplanar MR images of the lumbar spine with segmentation labels as the raw images; Apply a preset undersampling mask to the multi-coil k-space data corresponding to the original image to obtain undersampling observation data; The undersampled observation data, the original image, and the segmentation label are matched one-to-one to form training samples, and multiple training samples are combined to form a second training sample set.
[0010] In one possible implementation, the data fidelity module is used to update the data based on the consistency constraints between the current reconstruction results and the undersampled observation data; The fusion update module is used to perform a weighted fusion of the output of the data fidelity module and the output of the prior enhancement module to obtain the update result of the current stage.
[0011] Secondly, embodiments of the present invention provide a lumbar spine multiplanar MR image reconstruction and segmentation device, comprising: The acquisition module is used to acquire undersampled observation data; The reconstruction module is used to input the undersampled observation data into the reconstruction network to obtain the reconstructed image corresponding to the undersampled observation data. The reconstruction network includes multiple stages, each stage including a data fidelity module, a priori enhancement module, and a fusion update module. The priori enhancement module is an encoder-decoder structure based on Mamba units. The segmentation module is used to input the reconstructed image into the segmentation network to obtain the segmentation result corresponding to the reconstructed image; wherein, the segmentation result includes multiple lumbar spine regions, and the segmentation network and the reconstruction network are jointly trained end-to-end based on undersampled observation data of multiple planes of the lumbar spine.
[0012] Thirdly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect or any possible implementation thereof.
[0013] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect or any possible implementation thereof.
[0014] The lumbar spine multiplanar MR image reconstruction and segmentation method, apparatus, device, and storage medium provided in this invention solves the problem of disconnect between reconstruction and segmentation by jointly training the reconstruction network and the segmentation network end-to-end based on undersampled observation data from multiple lumbar spine planes. This allows the reconstruction process to directly perceive the needs of the segmentation task, thereby generating reconstructed images that are more conducive to subsequent structural recognition. Secondly, the reconstruction network adopts a multi-stage deep unfolding structure, with the data fidelity module of each stage working collaboratively with the encoder-decoder prior enhancement module based on Mamba units. This can suppress undersampled artifacts while preserving the edge information of key structures such as the lumbar vertebral body, intervertebral disc, and dural sac, significantly improving reconstruction quality. Thirdly, the state-space modeling capability of Mamba units endows the prior enhancement module with global context awareness, enabling it to stably extract image priors under different acquisition planes. Combined with end-to-end joint training, a single model can simultaneously process sagittal and axial images without the need to train independent models for different planes, enhancing the method's cross-plane generalization ability and improving the reconstruction and segmentation accuracy of lumbar spine MR images under undersampled conditions. Attached Figure Description
[0015] Figure 1 This is a flowchart illustrating the implementation of the lumbar spine multiplanar MR image reconstruction and segmentation method provided in this embodiment of the invention. Figure 2 This is a system block diagram of the lumbar spine multiplanar MR image reconstruction and segmentation method provided in the embodiments of the present invention; Figure 3 This is a schematic diagram of the network unfolding structure of the lumbar spine multiplanar MR image reconstruction and segmentation method provided in the embodiment of the present invention; Figure 4A This is a schematic diagram of a real sagittal image of the lumbar spine provided in an embodiment of the present invention; Figure 4B This is a schematic diagram of the sagittal reconstruction results of the lumbar spine using the VS-Net+SymTc method provided in an embodiment of the present invention; Figure 4C This is a schematic diagram of the sagittal reconstruction results of the lumbar spine using the VS-Net+LKM method provided in an embodiment of the present invention; Figure 4D This is a schematic diagram of the sagittal reconstruction results of the lumbar spine using the PromptIR+SymTc method provided in an embodiment of the present invention. Figure 4E This is a schematic diagram of the sagittal reconstruction results of the lumbar spine using the PromptIR+LKM method provided in an embodiment of the present invention; Figure 4F This is a schematic diagram of the sagittal reconstruction result of the lumbar spine using the proposed algorithm, provided in an embodiment of the present invention.
[0016] Figure 5A This is a schematic diagram of a real lumbar axial image provided in an embodiment of the present invention; Figure 5B This is a schematic diagram of the lumbar axial reconstruction results using the VS-Net+SymTc method provided in an embodiment of the present invention; Figure 5C This is a schematic diagram of the lumbar axial reconstruction results using the VS-Net+LKM method provided in an embodiment of the present invention; Figure 5D This is a schematic diagram of the lumbar axial reconstruction results using the PromptIR+SymTc method provided in an embodiment of the present invention. Figure 5E This is a schematic diagram of the lumbar axial reconstruction results using the PromptIR+LKM method provided in an embodiment of the present invention; Figure 5F This is a schematic diagram of the lumbar axial reconstruction result using the proposed algorithm provided in an embodiment of the present invention.
[0017] Figure 6 This is a schematic diagram of the lumbar spine multiplanar MR image reconstruction and segmentation device provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0018] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
[0019] See Figure 1 The flowchart illustrating the implementation of the lumbar spine multiplanar MR image reconstruction and segmentation method provided in this embodiment of the invention is described in detail below: Step 101: Obtain undersampled observation data.
[0020] In this embodiment, in practical applications, T2-weighted sagittal and T2-weighted axial scans of the lumbar spine region are performed using conventional magnetic resonance imaging (MRI) equipment to obtain T2 sagittal and T2 axial MR images of the lumbar spine as raw images. Based on these, a preset undersampling mask is applied to the multi-coil k-space data corresponding to the raw images to simulate undersampling observation data. This undersampling observation data includes undersampling k-space data, the undersampling mask, and coil sensitivity information. The undersampling acceleration factor corresponding to the undersampling mask is set to 4, and approximately 8% of the sampling points in the low-frequency region at the center of the k-space are fully sampled to ensure the integrity of low-frequency information. The undersampling observation data obtained in this way can simulate actual acquisition data in a rapid clinical scanning scenario, providing input for subsequent reconstruction and segmentation. In the model application stage, this undersampling observation data is directly used as input to be processed without additional annotation or preprocessing.
[0021] Step 102: Input the undersampled observation data into the reconstruction network to obtain the reconstructed image corresponding to the undersampled observation data. The reconstruction network includes multiple stages, and the number of stages can be determined according to actual needs. Each stage includes a data fidelity module, a priori enhancement module, and a fusion update module. The priori enhancement module is an encoder-decoder structure based on Mamba units.
[0022] In this embodiment, Figure 2 The system composition block diagram of the present invention is shown. As can be seen from the figure, the entire system consists of four main parts: a training sample construction module, a Mamba vision basic model pre-training module, an integrated reconstruction network, and a segmentation network. The parts are connected by data flow to form a complete end-to-end processing flow. Figure 3 The diagram shows the expanded structure of the reconstruction network and the segmentation network of the present invention. The diagram clearly shows the serial relationship of the K iteration stages, as well as the data flow of the data fidelity module, the prior enhancement module and the fusion update module within each stage.
[0023] Specifically, the reconstruction network adopts a model-driven deep unfolding structure, which unfolds the traditional iterative optimization algorithm into a deep network form, containing K iteration stages. Each stage consists of three core components: a data fidelity module, a prior enhancement module, and a fusion update module. The data fidelity module is responsible for constraining the consistency between the current reconstruction result and the undersampled observation data in k-space. Its processing principle is based on minimizing the mean square error between the reconstructed image and the observation data. In specific implementation, this module calculates the residual between the current reconstruction result and the undersampled observation data in k-space, and backprojects this residual back into the image domain through the conjugate transpose operator to form the gradient direction. Then, the current reconstruction result is updated using gradient descent, and the update step size can be designed as a learnable parameter that varies with the stage. The prior enhancement module is used to extract prior information of the image and suppress noise and artifacts introduced by undersampling. This module adopts an encoder-decoder structure based on Mamba units, which can perform global context modeling of image features, thereby effectively enhancing the anatomical details of the reconstructed image. The fusion update module is responsible for weighted fusion of the output of the data fidelity module and the output of the prior enhancement module to obtain the update result of the current stage. The fusion weights can be designed as learnable parameters, enabling the network to adaptively balance the contributions of fidelity and prior. After K iterations, the reconstruction network outputs a high-quality reconstructed image that effectively eliminates artifacts caused by undersampling while preserving the structural details of the original image.
[0024] Step 103: Input the reconstructed image into the segmentation network to obtain the segmentation result corresponding to the reconstructed image; wherein, the segmentation result includes multiple lumbar spine regions, and the segmentation network and the reconstruction network are jointly trained end-to-end based on undersampled observation data of the lumbar spine multi-plane.
[0025] In this embodiment, the segmentation network also adopts an encoder-decoder structure based on Mamba units. Its overall architecture is isomorphic to the prior enhancement module in the reconstruction network, thus ensuring consistency in feature extraction methods. The segmentation network takes the reconstructed image as input and, through stepwise feature extraction and spatial resolution restoration, finally outputs segmentation results containing multiple lumbar vertebrae. During the training phase, the segmentation network and the reconstruction network are jointly trained end-to-end based on undersampled observation data of the lumbar multi-plane, enabling the reconstruction network to fully consider the needs of downstream segmentation tasks while optimizing reconstruction quality, thereby obtaining both high-quality reconstructed images and accurate segmentation results under undersampled conditions.
[0026] This invention addresses the disconnect between reconstruction and segmentation by jointly training the reconstruction and segmentation networks end-to-end based on undersampled observation data from multiple planes of the lumbar spine. This allows the reconstruction process to directly perceive the segmentation task's requirements, generating reconstructed images more conducive to subsequent structural recognition. Secondly, the reconstruction network employs a multi-stage deep unfolding structure, where each stage's data fidelity module collaborates with a Mamba-based encoder-decoder prior enhancement module. This suppresses undersampled artifacts while preserving edge information of key structures such as the vertebral body, intervertebral discs, and dural sac, significantly improving reconstruction quality. Thirdly, the state-space modeling capability of the Mamba unit endows the prior enhancement module with global context awareness, enabling stable extraction of image priors across different acquisition planes. Combined with end-to-end joint training, a single model can simultaneously process sagittal and axial images without requiring independent model training for different planes, enhancing the method's cross-plane generalization ability and improving the reconstruction and segmentation accuracy of lumbar spine MR images under undersampled conditions.
[0027] In one possible implementation, the segmentation network includes a Mamba-based encoder-decoder structure, a sagittal output header, and an axial output header; The reconstructed image is input into the segmentation network to obtain the segmentation results corresponding to the reconstructed image, including: When the acquisition plane corresponding to the undersampled observation data is sagittal, the segmentation network outputs the sagittal segmentation result corresponding to the reconstructed image through the sagittal output head; When the acquisition plane corresponding to the undersampled observation data is axial, the segmentation network outputs the axial segmentation result corresponding to the reconstructed image through the axial output head.
[0028] In this embodiment, the segmentation network has two types of mapping output heads at its end: a sagittal output head and an axial output head, to adapt to segmentation tasks on different acquisition planes. When the acquisition plane of the undersampled observation data corresponding to the input reconstructed image is sagittal, the segmentation network automatically switches to the sagittal output head and outputs the sagittal segmentation result. This sagittal segmentation result includes 13 categories, including the background, specifically 5 intervertebral discs, 5 vertebral bodies, 1 sacrum, and the dural sac, which can completely cover the main anatomical structures in the lumbar sagittal image.
[0029] When the undersampled observation data acquisition plane corresponding to the input reconstructed image is axial, the segmentation network switches to the axial output head and outputs the axial segmentation result. This axial segmentation result includes four categories, including the background, specifically the dural sac, ligamentum flavum, and muscle, which can meet the segmentation requirements of the perispinal structures under axial images. Through the above output head design, a single segmentation network can process images from both sagittal and axial planes simultaneously, achieving unified modeling across planes.
[0030] In one possible implementation, before inputting the undersampled observation data into the reconstruction network to obtain the reconstructed image corresponding to the undersampled observation data, the following steps are also included: The VSS block is self-supervised and trained based on the first training sample set and the SimMIM strategy to obtain the basic model weights of the VSS block; the training sample set includes MR image data of the lumbar spine in multiple planes. The weights of the base model are loaded into each prior enhancement module and the segmentation network to obtain the initialized reconstruction network and segmentation network. The initial reconstruction network and segmentation network are jointly trained based on the second training sample set and the joint loss function to obtain the trained reconstruction network and segmentation network.
[0031] In this embodiment, before inputting the undersampled observation data into the reconstruction network, a model training process is required, which is divided into two stages: pre-training and joint training.
[0032] The pre-training phase was implemented based on the first training sample set, which is a large-scale lumbar spine multiplanar MR image dataset. A masked image modeling strategy was used to perform self-supervised training on the VSS blocks. Specifically, firstly, the first dataset for self-supervised pre-training was constructed. Real lumbar spine MRI images of patients were collected from multiple hospitals. After data processing and extraction steps such as de-privacy, format unification, and quality screening, a total of one million two-dimensional PNG images were obtained. All images in this dataset are original acquisition images without any manually labeled segmentation tags. The SimMIM (Simple Masked Image Modeling) strategy was adopted, dividing the input image into several image blocks. A portion of these blocks was randomly masked at a certain ratio before being input into the model. The model learns the deep structural features of the image by predicting the original pixel values of the masked regions. The pre-trained network uses a complete encoder-decoder structure including an encoder and a decoder. Through iterative optimization on a large-scale unlabeled dataset, the encoder learns the general visual representations of lumbar spine images, including edges, texture, contrast, and the spatial relationships between anatomical structures. After numerous iterations, only the model weights of the encoder portion are retained, while the decoder portion is discarded, resulting in the basic model weights of the VSS block. These weights embody the general representation capability of lumbar multiplanar images. This encoder has the exact same network structure as the encoders in the subsequent reconstruction and segmentation networks, therefore its weights can be directly used as initialization parameters for the second stage.
[0033] The joint training phase then begins, where the base model weights of the VSS blocks obtained from the pre-training are loaded into the encoder-decoder structures of each prior augmentation module and the segmentation network as initial weights, resulting in initialized reconstruction and segmentation networks. Based on this, end-to-end joint training is performed on the initialized reconstruction and segmentation networks using a second training sample set and a joint loss function. This allows the two networks to learn specific features for their respective tasks while sharing general representations. This combination of pre-training and joint training effectively improves the model's generalization ability in small-sample scenarios and enhances the synergistic effect between reconstruction and segmentation tasks.
[0034] In one possible implementation, the VSS block includes a first normalization layer, a second normalization layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, a depthwise separable convolutional layer, and an SS2D layer. The input features are divided into two paths after passing through the first normalization layer. The first path passes through the first fully connected layer, the depthwise separable convolutional layer, the SS2D layer, and the second normalization layer. The second path passes through the third fully connected layer. The two paths are multiplied element-wise and then mapped through the second fully connected layer, and residual connections are made with the input features.
[0035] In this embodiment, as Figure 3 As shown, the Visual State Space Block (VSSBlock), or VSS block, is a basic building block of the Mamba unit, and its internal structure is as follows: The VSS block includes a first normalization layer, a second normalization layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, a depthwise separable convolutional layer, and an SS2D layer. The input features are first normalized by the first normalization layer, and then split into two parallel branches: the first branch sequentially passes through the first fully connected layer, the depthwise separable convolutional layer, the SS2D layer, and the second normalization layer. The depthwise separable convolutional layer is used for local feature extraction, and the SS2D layer recursively models the global context based on the state space model; the second branch passes through the third fully connected layer. The features from both branches are then element-wise multiplied and mapped by the second fully connected layer. Finally, the result is residually concatenated with the input features. Through this structural design, the VSS block can effectively capture long-range dependencies in the image while maintaining linear computational complexity.
[0036] In one possible implementation, the joint loss function includes reconstruction loss and segmentation loss; The reconstruction loss is calculated based on the difference between the reconstructed image and the original image. The segmentation loss includes Dice loss, cross-entropy loss, and depth supervision loss.
[0037] In this embodiment, the joint loss function consists of a weighted sum of reconstruction loss and segmentation loss. The reconstruction loss constrains the difference between the reconstructed image and the original image; specifically, it can employ mean squared error loss or L1 loss. Minimizing this loss forces the reconstruction network output to approximate the fully sampled original image as closely as possible. The segmentation loss supervises the consistency between the segmentation output and the manually labeled data; specifically, it uses a combination of Dice loss, cross-entropy loss, and deep supervision loss. Dice loss focuses on the intersection-union ratio of segmented regions and is suitable for handling class imbalance problems; cross-entropy loss provides classification supervision at the pixel level; and deep supervision loss applies auxiliary supervision signals at multiple levels of the decoder, helping to alleviate gradient vanishing and accelerate convergence. In actual training, the weight coefficient of the segmentation loss can be set to 0.0002 to balance the relative importance of the reconstruction and segmentation tasks in the joint optimization.
[0038] In one possible implementation, before jointly training the initialized reconstruction network and segmentation network based on the second training sample set and the joint loss function to obtain the trained reconstruction network and segmentation network, the following steps are also included: Acquire multiplanar MR images of the lumbar spine with segmentation labels as the raw images; Apply a preset undersampling mask to the multi-coil k-space data corresponding to the original image to obtain undersampling observation data; The undersampled observation data, the original image, and the segmentation label are matched one-to-one to form training samples, and multiple training samples are combined to form a second training sample set.
[0039] In this embodiment, the training data used in the second stage is a labeled dataset. Before training begins, a second training sample set needs to be constructed. Real lumbar spine MRI images from the same source as those in the first stage (but with different samples) are manually labeled. Specifically, sagittal images are labeled with 13 categories (including background), including 5 intervertebral discs, 5 vertebral bodies, 1 sacrum, and the dural sac; axial images are labeled with 4 categories (including background), including the dural sac, ligamentum flavum, and muscles. After labeling, a dataset containing the original images and corresponding segmentation labels is formed. Due to the high cost and time-consuming nature of manual labeling, the amount of data in this labeled dataset is much smaller than the millions of unlabeled images in the first stage. Then, a preset undersampling mask is applied to the multi-coil k-space data corresponding to the original images to obtain undersampled observation data. This undersampled observation data, the original images, and the segmentation labels correspond one-to-one, constituting a training sample. Multiple training samples are aggregated to form the second training sample set. This training sample set is used for end-to-end supervised learning in the joint training stage, ensuring the consistency and integrity of the data during model training.
[0040] In one possible implementation, the data fidelity module is used to update the data based on the consistency constraints between the current reconstruction results and the undersampled observation data; The fusion update module is used to perform a weighted fusion of the output of the data fidelity module and the output of the prior enhancement module to obtain the update result of the current stage.
[0041] In this embodiment, each functional module in the reconstruction network has a clearly defined mechanism of action. The data fidelity module is used to update the image based on the consistency constraints between the current reconstruction result and the undersampled observation data. Essentially, it matches the reconstructed image with the observation data in k-space, ensuring that the reconstruction result conforms to the physical acquisition process. The fusion update module weights and fuses the outputs of the data fidelity module and the prior enhancement module to obtain the update result for the current stage. This fusion process can be designed as an adaptive weighting method, with weight coefficients dynamically adjusted according to the stage index or input data. This allows the reconstruction network to flexibly adjust the emphasis on fidelity and prior enhancement at different iteration stages. Through the collaborative work of these modules, the reconstruction network can gradually approach a high-quality reconstructed image in multi-stage iterations.
[0042] like Figure 3 As shown, the data fidelity module is structured to receive the reconstruction results from the previous stage. First, calculate its relationship with undersampled observation data in k-space. The residual, i.e. ,in This is the undersampled Fourier encoding matrix. Then, the residual is passed through the conjugate transpose operator. Backprojecting back into the image domain yields the gradient direction. Finally, gradient descent is used to... Perform an update, update step size These are learnable parameters. The processing procedure can be expressed as follows:
[0043] Its goal is to find an updated image. This allows the system to maintain prior information at the current stage while minimizing inconsistencies with undersampled observation data.
[0044] Figure 3 The auxiliary variable update module in the code is the fusion update module, and its processing can be represented as follows:
[0045] in The prior augmented image output by the prior augmentation module. These are learnable fusion weights used to adaptively balance the contributions of data fidelity terms and prior terms to the reconstruction results at the current stage.
[0046] In one specific embodiment, the method provided by this invention is compared with existing lumbar spine MR image reconstruction methods for sagittal and axial lumbar spine MR images, respectively. The reconstruction results are as follows: Figures 4A-4F , Figures 5A-5F As shown in the figure, compared to the comparative methods, the method provided by this invention can more effectively suppress aliasing artifacts and noise interference caused by undersampling, making the lumbar anatomical structures in the reconstructed images clearer, and maintaining the outlines of intervertebral disc boundaries, vertebral endplates, and structures around the spinal canal more completely. This effect is evident in both sagittal and axial images. Segmentation based on the reconstructed images obtained by this invention results in more continuous segmentation at the boundaries of target structures, better consistency between the segmented regions and corresponding anatomical structures, and reduces the impact of artifacts and structural blurring on the segmentation results.
[0047] As can be seen from the end-to-end joint training method of this invention, the high-quality reconstructed images output by the reconstruction network can provide clearer structural boundaries and texture information for the segmentation network; at the same time, the supervision information of the segmentation task can encourage the reconstruction network to pay more attention to local details related to anatomical structure recognition during joint training. Thus, the reconstruction task and the segmentation task in this invention can form a mutually reinforcing relationship, thereby improving the reconstruction and segmentation effect of lumbar spine MR images under undersampling conditions.
[0048] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0049] The following are device embodiments of the present invention. For details not described in detail, please refer to the corresponding method embodiments described above.
[0050] Figure 6 A schematic diagram of the lumbar spine multiplanar MR image reconstruction and segmentation device provided in an embodiment of the present invention is shown. For ease of explanation, only the parts related to the embodiment of the present invention are shown, and are described in detail below: like Figure 6 As shown, the lumbar spine multiplanar MR image reconstruction and segmentation device 6 includes: Module 61 is used to acquire undersampled observation data; The reconstruction module 62 is used to input the undersampled observation data into the reconstruction network to obtain the reconstructed image corresponding to the undersampled observation data; the reconstruction network includes multiple stages, each stage including a data fidelity module, a priori enhancement module, and a fusion update module, wherein the priori enhancement module is an encoder-decoder structure based on Mamba units; The segmentation module 63 is used to input the reconstructed image into the segmentation network to obtain the segmentation result corresponding to the reconstructed image; wherein, the segmentation result includes multiple lumbar spine regions, and the segmentation network and the reconstruction network are jointly trained end-to-end based on undersampled observation data of the lumbar spine multi-plane.
[0051] In one possible implementation, the segmentation network includes a Mamba-based encoder-decoder structure, a sagittal output header, and an axial output header; The segmentation module 63 is specifically used for: When the acquisition plane corresponding to the undersampled observation data is sagittal, the segmentation network outputs the sagittal segmentation result corresponding to the reconstructed image through the sagittal output head; When the acquisition plane corresponding to the undersampled observation data is axial, the segmentation network outputs the axial segmentation result corresponding to the reconstructed image through the axial output head.
[0052] In one possible implementation, the reconstruction module 62 is also used for: Before inputting the undersampled observation data into the reconstruction network to obtain the reconstructed image corresponding to the undersampled observation data, the VSS block is self-supervised based on the first training sample set and the SimMIM strategy to obtain the basic model weights of the VSS block; wherein, the training sample set includes MR image data of the lumbar spine multiplane; The weights of the base model are loaded into each prior enhancement module and the segmentation network to obtain the initialized reconstruction network and segmentation network. The initial reconstruction network and segmentation network are jointly trained based on the second training sample set and the joint loss function to obtain the trained reconstruction network and segmentation network.
[0053] In one possible implementation, the VSS block includes a first normalization layer, a second normalization layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, a depthwise separable convolutional layer, and an SS2D layer. The input features are divided into two paths after passing through the first normalization layer. The first path passes through the first fully connected layer, the depthwise separable convolutional layer, the SS2D layer, and the second normalization layer. The second path passes through the third fully connected layer. The two paths are multiplied element-wise and then mapped through the second fully connected layer, and residual connections are made with the input features.
[0054] In one possible implementation, the joint loss function includes reconstruction loss and segmentation loss; The reconstruction loss is calculated based on the difference between the reconstructed image and the original image. The segmentation loss includes Dice loss, cross-entropy loss, and depth supervision loss.
[0055] In one possible implementation, the reconstruction module 62 is also used for: Before the initial reconstruction network and segmentation network are jointly trained based on the second training sample set and the joint loss function to obtain the trained reconstruction network and segmentation network, lumbar multiplanar MR images with segmentation labels are acquired as the original images. Apply a preset undersampling mask to the multi-coil k-space data corresponding to the original image to obtain undersampling observation data; The undersampled observation data, the original image, and the segmentation label are matched one-to-one to form training samples, and multiple training samples are combined to form a second training sample set.
[0056] In one possible implementation, the data fidelity module is used to update the data based on the consistency constraints between the current reconstruction results and the undersampled observation data; The fusion update module is used to perform a weighted fusion of the output of the data fidelity module and the output of the prior enhancement module to obtain the update result of the current stage.
[0057] This invention addresses the disconnect between reconstruction and segmentation by jointly training the reconstruction and segmentation networks end-to-end based on undersampled observation data from multiple planes of the lumbar spine. This allows the reconstruction process to directly perceive the segmentation task's requirements, generating reconstructed images more conducive to subsequent structural recognition. Secondly, the reconstruction network employs a multi-stage deep unfolding structure, where each stage's data fidelity module collaborates with a Mamba-based encoder-decoder prior enhancement module. This suppresses undersampled artifacts while preserving edge information of key structures such as the vertebral body, intervertebral discs, and dural sac, significantly improving reconstruction quality. Thirdly, the state-space modeling capability of the Mamba unit endows the prior enhancement module with global context awareness, enabling stable extraction of image priors across different acquisition planes. Combined with end-to-end joint training, a single model can simultaneously process sagittal and axial images without requiring independent model training for different planes, enhancing the method's cross-plane generalization ability and improving the reconstruction and segmentation accuracy of lumbar spine MR images under undersampled conditions.
[0058] Figure 7 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Figure 7 As shown, the electronic device 7 of this embodiment includes a processor 70 and a memory 71. The memory 71 stores a computer program 72. When the processor 70 executes the computer program 72, it implements the steps in the various method embodiments described above. Alternatively, when the processor 70 executes the computer program 72, it implements the functions of each module / unit in the various device embodiments described above.
[0059] For example, computer program 72 may be divided into one or more modules / units, which are stored in memory 71 and executed by processor 70 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 72 in electronic device 7.
[0060] Electronic device 7 may include, but is not limited to, processor 70 and memory 71. Those skilled in the art will understand that... Figure 7 This is merely an example of electronic device 7 and does not constitute a limitation on electronic device 7. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device 7 may also include input / output devices, network access devices, buses, etc.
[0061] The processor 70 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.
[0062] The memory 71 can be an internal storage unit of the electronic device 7, such as a hard disk or RAM. The memory 71 can also be an external storage device of the electronic device 7, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 71 can include both internal and external storage units of the electronic device 7. The memory 71 is used to store the computer program 72 and other programs and data required by the electronic device 7. The memory 71 can also be used to temporarily store data that has been output or will be output.
[0063] For the sake of simplicity and clarity, only the above-described functional modules / units are used as examples. In practical applications, the functions described above can be assigned to different functional modules / units as needed. These modules / units can be implemented in hardware, software, or a combination of both.
[0064] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the methods described in the above-described method embodiments.
[0065] This invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the methods described in the above-described method embodiments.
[0066] Computer programs include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. Computer-readable media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.
[0067] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not detailed or described in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Unless otherwise specified or in conflict with logic, the terminology and / or descriptions between different embodiments are consistent and can be referenced interchangeably. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.
[0068] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A method for multiplanar MR image reconstruction and segmentation of the lumbar spine, characterized in that, include: Acquire undersampled observation data; The undersampled observation data is input into the reconstruction network to obtain the reconstructed image corresponding to the undersampled observation data. The reconstruction network includes multiple stages, each stage including a data fidelity module, a priori enhancement module, and a fusion update module. The priori enhancement module is an encoder-decoder structure based on Mamba units. The reconstructed image is input into a segmentation network to obtain the segmentation result corresponding to the reconstructed image; wherein, the segmentation result includes multiple lumbar spine regions, and the segmentation network and the reconstruction network are jointly trained end-to-end based on undersampled observation data of multiple planes of the lumbar spine.
2. The method for lumbar spine multiplanar MR image reconstruction and segmentation according to claim 1, characterized in that, The segmentation network includes a Mamba-based encoder-decoder structure, a sagittal output header, and an axial output header; The step of inputting the reconstructed image into a segmentation network to obtain the segmentation result corresponding to the reconstructed image includes: When the acquisition plane corresponding to the undersampled observation data is sagittal, the segmentation network outputs the sagittal segmentation result corresponding to the reconstructed image through the sagittal output head; When the acquisition plane corresponding to the undersampled observation data is axial, the segmentation network outputs the axial segmentation result corresponding to the reconstructed image through the axial output head.
3. The method for lumbar spine multiplanar MR image reconstruction and segmentation according to claim 1, characterized in that, Before inputting the undersampled observation data into the reconstruction network to obtain the reconstructed image corresponding to the undersampled observation data, the method further includes: The VSS block is self-supervised and trained based on the first training sample set and the SimMIM strategy to obtain the basic model weights of the VSS block; wherein, the training sample set includes MR image data of the lumbar spine in multiple planes; The weights of the base model are loaded into each prior enhancement module and the segmentation network to obtain the initialized reconstruction network and segmentation network. The initial reconstruction network and segmentation network are jointly trained based on the second training sample set and the joint loss function to obtain the trained reconstruction network and segmentation network.
4. The lumbar spine multiplanar MR image reconstruction and segmentation method according to claim 3, characterized in that, The VSS block includes a first normalized layer, a second normalized layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, a depthwise separable convolutional layer, and an SS2D layer; The input features are divided into two paths after passing through the first normalization layer. The first path passes through the first fully connected layer, the depthwise separable convolutional layer, the SS2D layer, and the second normalization layer. The second path passes through the third fully connected layer. The two paths are multiplied element-wise and then mapped through the second fully connected layer, and residual connections are made with the input features.
5. The method for lumbar spine multiplanar MR image reconstruction and segmentation according to claim 3, characterized in that, The joint loss function includes reconstruction loss and segmentation loss; The reconstruction loss is calculated based on the difference between the reconstructed image and the original image, and the segmentation loss includes Dice loss, cross-entropy loss, and depth supervision loss.
6. The method for lumbar spine multiplanar MR image reconstruction and segmentation according to claim 3, characterized in that, Before jointly training the initialized reconstruction network and segmentation network based on the second training sample set and the joint loss function to obtain the trained reconstruction network and segmentation network, the method further includes: Acquire multiplanar MR images of the lumbar spine with segmentation labels as the raw images; Apply a preset undersampling mask to the multi-coil k-space data corresponding to the original image to obtain undersampling observation data; The undersampled observation data, the original image, and the segmentation label are matched one-to-one to form training samples, and multiple training samples are combined to form the second training sample set.
7. The method for lumbar spine multiplanar MR image reconstruction and segmentation according to claim 1, characterized in that, The data fidelity module is used to update the data based on the consistency constraints between the current reconstruction results and the undersampled observation data. The fusion update module is used to perform weighted fusion of the output of the data fidelity module and the output of the prior enhancement module to obtain the update result of the current stage.
8. A lumbar spine multiplanar MR image reconstruction and segmentation device, characterized in that, include: The acquisition module is used to acquire undersampled observation data; The reconstruction module is used to input the undersampled observation data into the reconstruction network to obtain the reconstructed image corresponding to the undersampled observation data. The reconstruction network includes multiple stages, each stage including a data fidelity module, a priori enhancement module, and a fusion update module. The priori enhancement module is an encoder-decoder structure based on Mamba units. The segmentation module is used to input the reconstructed image into the segmentation network to obtain the segmentation result corresponding to the reconstructed image; wherein, the segmentation result includes multiple lumbar spine regions, and the segmentation network and the reconstruction network are jointly trained end-to-end based on undersampled observation data of multiple planes of the lumbar spine.
9. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.