A method, system, and storage medium for unfolding MRI images with joint boundaries
By combining image boundary information to design a reconstruction model and utilizing optimization theory and deep learning technology, the contradiction between data acquisition volume and imaging quality in MRI image reconstruction was resolved, achieving efficient and refined image reconstruction results while improving the interpretability and reliability of the model.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGDONG UNIV OF TECH
- Filing Date
- 2023-01-30
- Publication Date
- 2026-05-26
AI Technical Summary
Existing MRI image reconstruction methods present a trade-off between data acquisition volume and image quality. Traditional methods have high computational complexity and insufficient image quality, while deep learning-based methods lack interpretability and fail to effectively utilize image boundary information.
An unfolded MRI image reconstruction method based on joint boundaries is adopted. By designing an energy model and combining optimization theory and deep learning technology, image boundary information is used for alternating optimization, including pre-training of boundary information repair network and denoising network. The variable solution is performed using semi-quadratic splitting method and single-step gradient descent method to achieve image reconstruction.
It enables high-quality and rapid MRI image reconstruction, improving imaging accuracy and enhancing the interpretability and reliability of the model.
Smart Images
Figure CN115984402B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of medical image optimization technology, and more specifically, to a method, system, and storage medium for unfolded MRI image reconstruction with joint boundaries. Background Technology
[0002] Magnetic resonance imaging (MRI) is one of the most important medical imaging technologies in modern medicine. It is a non-invasive imaging technique widely used for anatomical observation and pathological diagnosis. However, due to the limitations of its imaging physical mechanisms, in clinical settings, acquiring sufficient data often requires prolonged patient scans. This not only causes discomfort for patients but may also introduce motion artifacts, compromising imaging accuracy. Conversely, reducing scan time decreases the amount of data acquired, directly leading to a decline in MRI image quality, and consequently affecting the accuracy of diagnosis and treatment.
[0003] To resolve the contradiction between the amount of data acquired and the quality of imaging, researchers in related fields have conducted numerous studies and explorations on MRI image reconstruction methods using modern computational science, in order to achieve accelerated imaging of MRI images.
[0004] Early researchers in MRI image reconstruction focused on using compressed sensing theory to design sparse constraints to construct MRI reconstruction models [Michael Lustiget et al. Sparse MRI: The Application of Compressed Sensing for Rapid MR Imaging, 2007.]. With the development of deep learning technology, some researchers proposed methods to reconstruct undersampled MRI images using convolutional neural networks alone [S. Wang et al. Accelerating magnetic resonance imaging via deep learning, 2016]. Others proposed MRI reconstruction methods based on diffusion models [Chung H, Ye J C. Score-based diffusion models for accelerated MRI, 2022]. However, traditional MRI reconstruction methods based on compressed sensing theory suffer from high computational complexity and insufficient image quality. While deep learning-based MRI reconstruction methods offer better visual effects, their lack of interpretability hinders clinical application and widespread adoption, and they generally fail to effectively utilize image boundary information to improve reconstruction results.
[0005] Therefore, existing technologies have shortcomings and urgently need improvement. Summary of the Invention
[0006] In view of the above problems, the purpose of this invention is to provide a method, system and storage medium for unfolding MRI image reconstruction with joint boundaries, which can realize MRI image imaging more effectively and quickly.
[0007] The first aspect of the present invention provides a method for reconstructing unfolded MRI images with joint boundaries, comprising:
[0008] Acquire undersampled MRI image information;
[0009] An energy model based on a pre-defined joint MRI image boundary; the optimization variables of the energy model consist of reconstructed images and image boundaries; the energy model consists of four weighted terms, namely, a K-space fidelity term, a reconstructed image implicit regularization term, an image boundary implicit regularization term, and a fusion constraint term for reconstructed image and boundary information;
[0010] Based on optimization theory, the energy model is decomposed into variables, resulting in subproblems 1 and 2 that update individual variables.
[0011] Based on the preset subproblem optimization algorithm, the networkable sub-modules in the algorithm are extracted and expanded; auxiliary variables are introduced for the two subproblems according to the semi-quadratic splitting method; the subproblems are solved by alternately optimizing the auxiliary variables and the target variable; wherein the module for solving the auxiliary variables is expanded by the neural network, and the solution of the target variable is further optimized by the single-step gradient descent method.
[0012] Based on a pre-defined loss function for networkable submodules, the networkable submodules are pre-trained; the specific steps include: pre-setting the loss function l for the boundary information repair network. r The loss function l of the denoising network d Based on supervised learning techniques, the boundary information repair network and the denoising network are pre-trained to obtain the corresponding pre-trained models.
[0013] Based on a preset number of iterations, the loss function and the hyperparameter set in the energy model are designed according to the preset overall joint boundary information expansion reconstruction model, and the overall joint boundary information expansion reconstruction model is obtained.
[0014] The undersampled MRI images are input into an unfolded reconstruction model with global joint boundary information to obtain the MRI image reconstruction results.
[0015] In this scheme, the energy model is specifically as follows:
[0016]
[0017] Where, x∈C N Represents reconstructed image; Ω∈BN B = [0,1] represents the boundary information of the reconstructed image, indicating the degree of correlation between the corresponding pixel and the image boundary. The higher the correlation, the closer the corresponding value is to 1; Ω C = J - Ω, where J is a one matrix, i.e., Ω C This indicates the degree of correlation between the corresponding pixel and the non-image boundary; the higher the correlation, the closer the value is to 1. The first term on the right side of the equation is the K-space fidelity term, which constrains the consistency between the reconstructed image and the observed data in K-space, where y∈C M (M < N) represents the downsampled K-space data, F is the Fourier transform operator, and U is the downsampling operator; R in the second term on the right-hand side is the implicit regularization term with respect to the reconstructed image, where λ1 is the weight coefficient; P in the third term on the right-hand side is the implicit regularization term with respect to the image boundary, where λ2 is the weight coefficient; the fourth term on the right-hand side is the constraint term for the fusion of reconstructed image and boundary information, where... The Hadamard product operation is represented by W, where W is the wavelet decomposition operator and ρ is its weighting coefficient. The smoothness of the image in the non-edge region is reconstructed in the wavelet domain using the square reduction of the second norm.
[0018] In this solution, the subproblems subproblem1 and subproblem2 that update individual variables are specifically as follows:
[0019] The subproblem 1 for updating individual variables is transformed into the subproblem 1 for updating image boundaries.
[0020] The subproblem 2 that updates individual variables is transformed into the subproblem 2 that updates the reconstructed image.
[0021] They are as follows:
[0022] subproblem1:
[0023] subproblem2:
[0024] Where n represents the iteration stage.
[0025] In this scheme, the algorithm for solving the image boundary subproblem1 is as follows:
[0026] Based on the semi-quadratic splitting method, an auxiliary variable V is introduced, so subproblem1 is transformed into the following optimization problem:
[0027]
[0028] Where V is an auxiliary variable; Ω CLet V be the target variable; α be the weighting coefficient; the solution to this problem can be derived from the auxiliary variable V and the target variable Ω. C The approximation is achieved through alternating iterative updates, with the following update methods:
[0029]
[0030]
[0031] Among them, for the auxiliary variable V (n) The solution can be expanded using a neural network, which can be used to optimize the image boundary information. The repair is performed and its relevant regular expressions are learned implicitly, as follows:
[0032]
[0033] Among them, for the target variable The solution can be further optimized using single-step gradient descent, as follows:
[0034]
[0035] Where s1 is the gradient descent step size hyperparameter.
[0036] In this scheme, the algorithm for solving the subproblem 2 of updating and reconstructing the image is as follows:
[0037] Based on the semi-quadratic splitting method, an auxiliary variable Z is introduced, so subproblem2 is transformed into the following optimization problem:
[0038]
[0039] Where Z is the auxiliary variable; x is the target variable; β is the weighting coefficient; the solution to this problem can be approximated by iterative updates of the auxiliary variable Z and the target variable x, with the update methods as follows:
[0040]
[0041]
[0042] Among them, for the auxiliary variable Z (n) The solution can be expanded using a neural network, which is used to optimize the reconstructed image x. (n-1) Denoising is performed and its relevant regularization is implicitly learned, as follows:
[0043] Z (n) =DENOISER(x (n-1) )
[0044] Where, for the target variable x (n) The solution can be further optimized using single-step gradient descent, as follows:
[0045]
[0046] Where s2 is the gradient descent step size; F H W is the inverse Fourier transform operator. H For wavelet reconstruction operators.
[0047] In this scheme, the loss function l of the preset boundary information repair network r The loss function l of the denoising network d Specifically:
[0048]
[0049]
[0050] Where K represents the number of samples; This is an undersampled MRI image; This is a fully sampled real MRI image; Represents the boundary information of undersampled MRI images; This represents the boundary information of a fully sampled MRI image.
[0051] A second aspect of the present invention provides a joint boundary unfolding MRI image reconstruction system, including a memory and a processor. The memory stores a joint boundary unfolding MRI image reconstruction method program, which, when executed by the processor, performs the following steps:
[0052] Acquire undersampled MRI image information;
[0053] An energy model based on a pre-defined joint MRI image boundary; the optimization variables of the energy model consist of reconstructed images and image boundaries; the energy model consists of four weighted terms, namely, a K-space fidelity term, a reconstructed image implicit regularization term, an image boundary implicit regularization term, and a fusion constraint term for reconstructed image and boundary information;
[0054] Based on optimization theory, the energy model is decomposed into variables, resulting in subproblems 1 and 2 that update individual variables.
[0055] Based on the preset subproblem optimization algorithm, the networkable sub-modules in the algorithm are extracted and expanded; auxiliary variables are introduced for the two subproblems according to the semi-quadratic splitting method; the subproblems are solved by alternately optimizing the auxiliary variables and the target variable; wherein the module for solving the auxiliary variables is expanded by the neural network, and the solution of the target variable is further optimized by the single-step gradient descent method.
[0056] Based on a pre-defined loss function for networkable submodules, the networkable submodules are pre-trained; the specific steps include: pre-setting the loss function l for the boundary information repair network. r The loss function l of the denoising network d Based on supervised learning techniques, the boundary information repair network and the denoising network are pre-trained to obtain the corresponding pre-trained models.
[0057] Based on a preset number of iterations, the loss function and the hyperparameter set in the energy model are designed according to the preset overall joint boundary information expansion reconstruction model, and the overall joint boundary information expansion reconstruction model is obtained.
[0058] The undersampled MRI images are input into an unfolded reconstruction model with global joint boundary information to obtain the MRI image reconstruction results.
[0059] In this scheme, the energy model is specifically as follows:
[0060]
[0061] Where, x∈C N Represents reconstructed image; Ω∈B N B = [0,1] represents the boundary information of the reconstructed image, indicating the degree of correlation between the corresponding pixel and the image boundary. The higher the correlation, the closer the corresponding value is to 1; Ω C = J - Ω, where J is a one matrix, i.e., Ω C This indicates the degree of correlation between the corresponding pixel and the non-image boundary; the higher the correlation, the closer the value is to 1. The first term on the right side of the equation is the K-space fidelity term, which constrains the consistency between the reconstructed image and the observed data in K-space, where y∈C M (M < N) represents the downsampled K-space data, F is the Fourier transform operator, and U is the downsampling operator; R in the second term on the right-hand side is the implicit regularization term with respect to the reconstructed image, where λ1 is the weight coefficient; P in the third term on the right-hand side is the implicit regularization term with respect to the image boundary, where λ2 is the weight coefficient; the fourth term on the right-hand side is the constraint term for the fusion of reconstructed image and boundary information, where... The Hadamard product operation is represented by W, where W is the wavelet decomposition operator and ρ is its weighting coefficient. The smoothness of the image in the non-edge region is reconstructed in the wavelet domain using the square reduction of the second norm.
[0062] In this solution, the subproblems subproblem1 and subproblem2 that update individual variables are specifically as follows:
[0063] The subproblem 1 for updating individual variables is transformed into the subproblem 1 for updating image boundaries.
[0064] The subproblem 2 that updates individual variables is transformed into the subproblem 2 that updates the reconstructed image.
[0065] They are as follows:
[0066] subproblem1:
[0067] subproblem2:
[0068] Where n represents the iteration stage.
[0069] In this scheme, the algorithm for solving the image boundary subproblem1 is as follows:
[0070] Based on the semi-quadratic splitting method, an auxiliary variable V is introduced, so subproblem1 is transformed into the following optimization problem:
[0071]
[0072] Where V is an auxiliary variable; Ω C Let V be the target variable; α be the weighting coefficient; the solution to this problem can be derived from the auxiliary variable V and the target variable Ω. C The approximation is achieved through alternating iterative updates, with the following update methods:
[0073]
[0074]
[0075] Among them, for the auxiliary variable V (n) The solution can be expanded using a neural network, which can be used to optimize the image boundary information. The repair is performed and its relevant regular expressions are learned implicitly, as follows:
[0076]
[0077] Among them, for the target variable The solution can be further optimized using single-step gradient descent, as follows:
[0078]
[0079] Where s1 is the gradient descent step size hyperparameter.
[0080] In this scheme, the algorithm for solving the subproblem 2 of updating and reconstructing the image is as follows:
[0081] Based on the semi-quadratic splitting method, an auxiliary variable Z is introduced, so subproblem2 is transformed into the following optimization problem:
[0082]
[0083] Where Z is the auxiliary variable; x is the target variable; β is the weighting coefficient; the solution to this problem can be approximated by iterative updates of the auxiliary variable Z and the target variable x, with the update methods as follows:
[0084]
[0085]
[0086] Among them, for the auxiliary variable Z (n) The solution can be expanded using a neural network, which is used to optimize the reconstructed image x. (n-1) Denoising is performed and its relevant regularization is implicitly learned, as follows:
[0087] Z (n) =DENOISER(x (n-1) )
[0088] Where, for the target variable x (n) The solution can be further optimized using single-step gradient descent, as follows:
[0089]
[0090] Where s2 is the gradient descent step size; F H W is the inverse Fourier transform operator. H For wavelet reconstruction operators.
[0091] In this scheme, the loss function l of the preset boundary information repair network r The loss function l of the denoising network d Specifically:
[0092]
[0093]
[0094] Where K represents the number of samples; This is an undersampled MRI image; This is a fully sampled real MRI image; Represents the boundary information of undersampled MRI images; This represents the boundary information of a fully sampled MRI image.
[0095] A third aspect of the present invention provides a computer storage medium storing a program for a method of reconstructing an unfolded MRI image with joint boundaries, wherein when the program is executed by a processor, it implements the steps of the method of reconstructing an unfolded MRI image with joint boundaries as described in any of the preceding claims.
[0096] This invention discloses a method, system, and storage medium for unfolded MRI image reconstruction based on joint boundaries. It designs a reconstruction model by incorporating image boundary information and alternately optimizes the reconstructed image and image boundaries using traditional optimization theory. Simultaneously, it expands the networkable sub-modules in the optimization process using deep learning technology, which has strong fitting capabilities. Driven by large-scale data, and combining network module pre-training and overall reconstruction model fine-tuning, an optimized model is trained that can effectively utilize image boundary information to perform high-quality reconstruction of undersampled MRI images. Compared to methods solely based on deep learning, this reconstruction model exhibits higher interpretability and reliability. Attached Figure Description
[0097] Figure 1 A flowchart of a joint boundary unfolding MRI image reconstruction method is shown;
[0098] Figure 2 A model architecture diagram of an unfolded MRI image reconstruction method with joint boundaries is shown;
[0099] Figure 3 A block diagram of a joint boundary unfolding MRI image reconstruction system is shown. Detailed Implementation
[0100] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.
[0101] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.
[0102] Figure 1 A flowchart of a joint boundary unfolding MRI image reconstruction method is shown.
[0103] like Figure 1As shown, this invention discloses a method for unfolding MRI image reconstruction with joint boundaries, comprising:
[0104] S102, acquire undersampled MRI image information;
[0105] S104, an energy model based on a preset joint MRI image boundary; the optimization variables of the energy model consist of reconstructed images and image boundaries; the energy model consists of four weighted terms, namely, a K-space fidelity term, a reconstructed image implicit regularization term, an image boundary implicit regularization term, and a reconstructed image and boundary information fusion constraint term;
[0106] S106. Based on optimization theory, the energy model is decomposed into variables to obtain subproblems subproblem1 and subproblem2 for updating individual variables;
[0107] S108, Based on the preset subproblem optimization algorithm, extract the networkable sub-modules in the algorithm and expand them; according to the semi-quadratic splitting method, introduce auxiliary variables for the two subproblems; solve the subproblems by alternately optimizing the auxiliary variables and the target variable; wherein the module for solving the auxiliary variables is expanded by a neural network, and the solution of the target variable is further optimized by the single-step gradient descent method;
[0108] S110, Based on the preset loss function for networkable submodules, pre-train the networkable submodules; specific steps include: preset the loss function l for the boundary information repair network. r The loss function l of the denoising network d Based on supervised learning techniques, the boundary information repair network and the denoising network are pre-trained to obtain the corresponding pre-trained models.
[0109] S112, Based on the preset number of iterations, design the loss function and the hyperparameter set in the energy model according to the preset overall joint boundary information expansion reconstruction model to obtain the overall joint boundary information expansion reconstruction model;
[0110] S114, input the undersampled MRI image into the expansion reconstruction model of the global joint boundary information to obtain the MRI image reconstruction result.
[0111] It should be noted that an energy model of the joint MRI image boundary is established, and the energy model is decomposed into variables using optimization theory to obtain subproblems for updating and reconstructing the image and updating the image boundary. Then, a pre-defined subproblem optimization algorithm is used, with auxiliary variables introduced based on the semi-quadratic splitting method, to solve the subproblems. Next, the networkable sub-modules in the pre-defined subproblem optimization algorithm are pre-trained using a pre-defined loss function to obtain pre-trained models for the boundary information restoration network and the denoising network, completing the corresponding parameter initialization. When the pre-defined number of iterations S = 3, a loss function l is designed for the overall joint boundary information expansion reconstruction model. total The l total The loss function is composed of the loss functions of each stage, and the loss function of each stage is a weighted sum of the image loss and the boundary loss. total The definition is as follows:
[0112]
[0113] in, and These are the weighting coefficients. To improve the model's flexibility, the hyperparameter set in the reconstructed model is extracted. These parameters are set as learnable parameters and optimized and fine-tuned together with the neural network parameters during backpropagation to obtain an unfolded reconstruction model of the overall joint boundary information. The boundary information repair network has a residual network structure, specifically composed of 8 sub-modules connected in series. Except for the last sub-module which only has a convolutional network layer, each of the other sub-modules consists of a convolutional network layer and a linear rectified activation function layer connected in series. The last sub-module is added pixel by pixel to the network input. This residual structure avoids gradient vanishing during training. Finally, the network output is reduced to [0,1] by the sigmoid activation function. The overall structure of the denoising network is an encoder-decoder structure, which performs three downsampling operations and three upsampling operations on the image in sequence to maintain the original size of the image. The downsampling method is max pooling, and the upsampling method is deconvolution. The output features of the downsampling max pooling layer are connected to the initial input features of the upsampling deconvolution layer, which together serve as the input of the deconvolution layer.
[0114] According to an embodiment of the present invention, the energy model is specifically as follows:
[0115]
[0116] Where, x∈C N Represents reconstructed image; Ω∈B N B = [0,1] represents the boundary information of the reconstructed image, indicating the degree of correlation between the corresponding pixel and the image boundary. The higher the correlation, the closer the corresponding value is to 1; ΩC = J - Ω, where J is a one matrix, i.e., Ω C This indicates the degree of correlation between the corresponding pixel and the non-image boundary; the higher the correlation, the closer the value is to 1. The first term on the right side of the equation is the K-space fidelity term, which constrains the consistency between the reconstructed image and the observed data in K-space, where y∈C M (M < N) represents the downsampled K-space data, F is the Fourier transform operator, and U is the downsampling operator; R in the second term on the right-hand side is the implicit regularization term with respect to the reconstructed image, where λ1 is the weight coefficient; P in the third term on the right-hand side is the implicit regularization term with respect to the image boundary, where λ2 is the weight coefficient; the fourth term on the right-hand side is the constraint term for the fusion of reconstructed image and boundary information, where... The Hadamard product operation is represented by W, where W is the wavelet decomposition operator and ρ is its weighting coefficient. The smoothness of the image in the non-edge region is reconstructed in the wavelet domain using the square reduction of the second norm.
[0117] It should be noted that an energy model for the joint MRI image boundary is established, and the optimization variables of the energy model consist of the reconstructed image and the image boundary. The energy model consists of four weighted terms, namely, the K-space fidelity term, the reconstructed image implicit regularization term, the image boundary implicit regularization term, and the reconstructed image and boundary information fusion constraint term.
[0118] According to an embodiment of the present invention, the subproblems subproblem1 and subproblem2 that update individual variables specifically refer to:
[0119] The subproblem 1 for updating individual variables is transformed into the subproblem 1 for updating image boundaries.
[0120] The subproblem 2 that updates individual variables is transformed into the subproblem 2 that updates the reconstructed image.
[0121] They are as follows:
[0122] subproblem1:
[0123] subproblem2:
[0124] Where n represents the iteration stage.
[0125] It should be noted that, based on the semi-quadratic splitting method, the energy model is minimized by iteratively updating the variables in the energy model, namely the reconstructed image and the image boundary. The problem of minimizing the energy model is then transformed into iterative optimization of the variables, establishing subproblems subproblem1 and subproblem2 for iteratively solving the image boundary and the reconstructed image, respectively. When the iteration stage is 0, the initialization of the image boundary and the reconstructed image... and x (0) They are respectively:
[0126]
[0127] x (0) =F H y
[0128] Where `normalize()` is the normalization function, reducing the input to [0,1]; F H is the inverse Fourier transform operator; |·| denotes the modulo operation on complex numbers.
[0129] According to an embodiment of the present invention, the algorithm for solving the image boundary subproblem1 is as follows:
[0130] Based on the semi-quadratic splitting method, an auxiliary variable V is introduced, so subproblem1 is transformed into the following optimization problem:
[0131]
[0132] Where V is an auxiliary variable; Ω C Let V be the target variable; α be the weighting coefficient; the solution to this problem can be derived from the auxiliary variable V and the target variable Ω. C The approximation is achieved through alternating iterative updates, with the following update methods:
[0133]
[0134]
[0135] Among them, for the auxiliary variable V (n) The solution can be expanded using a neural network, which can be used to optimize the image boundary information. The repair is performed and its relevant regular expressions are learned implicitly, as follows:
[0136]
[0137] Among them, for the target variable The solution can be further optimized using single-step gradient descent, as follows:
[0138]
[0139] Where s1 is the gradient descent step size hyperparameter.
[0140] It should be noted that the auxiliary variable V is introduced through the semi-quadratic splitting method to solve the subproblem 1 of updating the image boundary.
[0141] According to an embodiment of the present invention, the algorithm for solving the subproblem 2 of updating and reconstructing the image is specifically as follows:
[0142] Based on the semi-quadratic splitting method, an auxiliary variable Z is introduced, so subproblem2 is transformed into the following optimization problem:
[0143]
[0144] Where Z is the auxiliary variable; x is the target variable; β is the weighting coefficient; the solution to this problem can be approximated by iterative updates of the auxiliary variable Z and the target variable x, with the update methods as follows:
[0145]
[0146]
[0147] Among them, for the auxiliary variable Z (n) The solution can be expanded using a neural network, which is used to optimize the reconstructed image x. (n-1) Denoising is performed and its relevant regularization is implicitly learned, as follows:
[0148] Z (n) =DENOISER(x (n-1) )
[0149] Where, for the target variable x (n) The solution can be further optimized using single-step gradient descent, as follows:
[0150]
[0151] Where s2 is the gradient descent step size; F H W is the inverse Fourier transform operator. H For wavelet reconstruction operators.
[0152] It should be noted that the auxiliary variable Z is introduced through the semi-quadratic splitting method to solve the subproblem 2 of updating and reconstructing images.
[0153] According to an embodiment of the present invention, the loss function of the preset boundary information repair network is l r The loss function l of the denoising networkd Specifically:
[0154]
[0155]
[0156] Where K represents the number of samples; This is an undersampled MRI image; This is a fully sampled real MRI image; Represents the boundary information of undersampled MRI images; This represents the boundary information of a fully sampled MRI image.
[0157] It should be noted that the network submodule includes a boundary information restoration network and a denoising network, and the loss function of the boundary information restoration network is l. r The boundary information restoration network is pre-trained to obtain a pre-trained model of the boundary information restoration network; the loss function of the denoising network is l. d The denoising network is pre-trained to obtain a pre-trained model of the denoising network.
[0158] Figure 2 A model architecture diagram of an unfolded MRI image reconstruction method with joint boundaries is shown.
[0159] like Figure 2 As shown, the upper part of the figure represents the optimization of the image boundary in the (n-1)th stage, and the lower part represents the optimization of the reconstructed image.
[0160] Figure 3 A block diagram of a joint boundary unfolding MRI image reconstruction system is shown.
[0161] like Figure 3 As shown, a second aspect of the present invention provides a joint boundary unfolding MRI image reconstruction system 3, including a memory 31 and a processor 32. The memory stores a joint boundary unfolding MRI image reconstruction method program, which, when executed by the processor, performs the following steps:
[0162] Acquire undersampled MRI image information;
[0163] An energy model based on a pre-defined joint MRI image boundary; the optimization variables of the energy model consist of reconstructed images and image boundaries; the energy model consists of four weighted terms, namely, a K-space fidelity term, a reconstructed image implicit regularization term, an image boundary implicit regularization term, and a fusion constraint term for reconstructed image and boundary information;
[0164] Based on optimization theory, the energy model is decomposed into variables, resulting in subproblems 1 and 2 that update individual variables.
[0165] Based on the preset subproblem optimization algorithm, the networkable sub-modules in the algorithm are extracted and expanded; auxiliary variables are introduced for the two subproblems according to the semi-quadratic splitting method; the subproblems are solved by alternately optimizing the auxiliary variables and the target variable; wherein the module for solving the auxiliary variables is expanded by the neural network, and the solution of the target variable is further optimized by the single-step gradient descent method.
[0166] Based on a pre-defined loss function for networkable submodules, the networkable submodules are pre-trained; the specific steps include: pre-setting the loss function l for the boundary information repair network. r The loss function l of the denoising network d Based on supervised learning techniques, the boundary information repair network and the denoising network are pre-trained to obtain the corresponding pre-trained models.
[0167] Based on a preset number of iterations, the loss function and the hyperparameter set in the energy model are designed according to the preset overall joint boundary information expansion reconstruction model, and the overall joint boundary information expansion reconstruction model is obtained.
[0168] The undersampled MRI images are input into an unfolded reconstruction model with global joint boundary information to obtain the MRI image reconstruction results.
[0169] It should be noted that an energy model of the joint MRI image boundary is established, and the energy model is decomposed into variables using optimization theory to obtain subproblems for updating and reconstructing the image and updating the image boundary. Then, a pre-defined subproblem optimization algorithm is used, with auxiliary variables introduced based on the semi-quadratic splitting method, to solve the subproblems. Next, the networkable sub-modules in the pre-defined subproblem optimization algorithm are pre-trained using a pre-defined loss function to obtain pre-trained models for the boundary information restoration network and the denoising network, completing the corresponding parameter initialization. When the pre-defined number of iterations S = 3, a loss function l is designed for the overall joint boundary information expansion reconstruction model. total The l total The loss function is composed of the loss functions of each stage, and the loss function of each stage is a weighted sum of the image loss and the boundary loss. total The definition is as follows:
[0170]
[0171] in, and These are the weighting coefficients. To improve the model's flexibility, the hyperparameter set in the reconstructed model is extracted. These parameters are set as learnable parameters and optimized and fine-tuned together with the neural network parameters during backpropagation to obtain an unfolded reconstruction model of the overall joint boundary information. The boundary information repair network has a residual network structure, specifically composed of 8 sub-modules connected in series. Except for the last sub-module which only has a convolutional network layer, each of the other sub-modules consists of a convolutional network layer and a linear rectified activation function layer connected in series. The last sub-module is added pixel by pixel to the network input. This residual structure avoids gradient vanishing during training. Finally, the network output is reduced to [0,1] by the sigmoid activation function. The overall structure of the denoising network is an encoder-decoder structure, which performs three downsampling operations and three upsampling operations on the image in sequence to maintain the original size of the image. The downsampling method is max pooling, and the upsampling method is deconvolution. The output features of the downsampling max pooling layer are connected to the initial input features of the upsampling deconvolution layer, which together serve as the input of the deconvolution layer.
[0172] According to an embodiment of the present invention, the energy model is specifically as follows:
[0173]
[0174] Where, x∈C N Represents reconstructed image; Ω∈B N B = [0,1] represents the boundary information of the reconstructed image, indicating the degree of correlation between the corresponding pixel and the image boundary. The higher the correlation, the closer the corresponding value is to 1; Ω C = J - Ω, where J is a one matrix, i.e., Ω C This indicates the degree of correlation between the corresponding pixel and the non-image boundary; the higher the correlation, the closer the value is to 1. The first term on the right side of the equation is the K-space fidelity term, which constrains the consistency between the reconstructed image and the observed data in K-space, where y∈C M (M < N) represents the downsampled K-space data, F is the Fourier transform operator, and U is the downsampling operator; R in the second term on the right-hand side is the implicit regularization term with respect to the reconstructed image, where λ1 is the weight coefficient; P in the third term on the right-hand side is the implicit regularization term with respect to the image boundary, where λ2 is the weight coefficient; the fourth term on the right-hand side is the constraint term for the fusion of reconstructed image and boundary information, where... The Hadamard product operation is represented by W, where W is the wavelet decomposition operator and ρ is its weighting coefficient. The smoothness of the image in the non-edge region is reconstructed in the wavelet domain using the square reduction of the second norm.
[0175] It should be noted that an energy model for the joint MRI image boundary is established, and the optimization variables of the energy model consist of the reconstructed image and the image boundary. The energy model consists of four weighted terms, namely, the K-space fidelity term, the reconstructed image implicit regularization term, the image boundary implicit regularization term, and the reconstructed image and boundary information fusion constraint term.
[0176] According to an embodiment of the present invention, the subproblems subproblem1 and subproblem2 that update individual variables specifically refer to:
[0177] The subproblem 1 for updating individual variables is transformed into the subproblem 1 for updating image boundaries.
[0178] The subproblem 2 that updates individual variables is transformed into the subproblem 2 that updates the reconstructed image.
[0179] They are as follows:
[0180] subproblem1:
[0181] subproblem2:
[0182] Where n represents the iteration stage.
[0183] It should be noted that, based on the semi-quadratic splitting method, the energy model is minimized by iteratively updating the variables in the energy model, namely the reconstructed image and the image boundary. The problem of minimizing the energy model is then transformed into iterative optimization of the variables, establishing subproblems subproblem1 and subproblem2 for iteratively solving the image boundary and the reconstructed image, respectively. When the iteration stage is 0, the initialization of the image boundary and the reconstructed image... and x (0) They are respectively:
[0184]
[0185] x (0) =F H y
[0186] Where `normalize()` is the normalization function, reducing the input to [0,1]; F H is the inverse Fourier transform operator; |·| denotes the modulo operation on complex numbers.
[0187] According to an embodiment of the present invention, the algorithm for solving the image boundary subproblem1 is as follows:
[0188] Based on the semi-quadratic splitting method, an auxiliary variable V is introduced, so subproblem1 is transformed into the following optimization problem:
[0189]
[0190] Where V is an auxiliary variable; Ω C Let V be the target variable; α be the weighting coefficient; the solution to this problem can be derived from the auxiliary variable V and the target variable Ω. C The approximation is achieved through alternating iterative updates, with the following update methods:
[0191]
[0192]
[0193] Among them, for the auxiliary variable V (n) The solution can be expanded using a neural network, which can be used to optimize the image boundary information. The repair is performed and its relevant regular expressions are learned implicitly, as follows:
[0194]
[0195] Among them, for the target variable The solution can be further optimized using single-step gradient descent, as follows:
[0196]
[0197] Where s1 is the gradient descent step size hyperparameter.
[0198] It should be noted that the auxiliary variable V is introduced through the semi-quadratic splitting method to solve the subproblem 1 of updating the image boundary.
[0199] According to an embodiment of the present invention, the algorithm for solving the subproblem 2 of updating and reconstructing the image is specifically as follows:
[0200] Based on the semi-quadratic splitting method, an auxiliary variable Z is introduced, so subproblem2 is transformed into the following optimization problem:
[0201]
[0202] Where Z is the auxiliary variable; x is the target variable; β is the weighting coefficient; the solution to this problem can be approximated by iterative updates of the auxiliary variable Z and the target variable x, with the update methods as follows:
[0203]
[0204]
[0205] Among them, for the auxiliary variable Z (n) The solution can be expanded using a neural network, which is used to optimize the reconstructed image x. (n-1) Denoising is performed and its relevant regularization is implicitly learned, as follows:
[0206] Z (n) =DENOISER(x (n-1) )
[0207] Where, for the target variable x (n) The solution can be further optimized using single-step gradient descent, as follows:
[0208]
[0209] Where s2 is the gradient descent step size; F H W is the inverse Fourier transform operator. H For wavelet reconstruction operators.
[0210] It should be noted that the auxiliary variable Z is introduced through the semi-quadratic splitting method to solve the subproblem 2 of updating and reconstructing images.
[0211] According to an embodiment of the present invention, the loss function of the preset boundary information repair network is l r The loss function l of the denoising network d Specifically:
[0212]
[0213]
[0214] Where K represents the number of samples; This is an undersampled MRI image; This is a fully sampled real MRI image; Represents the boundary information of undersampled MRI images; This represents the boundary information of a fully sampled MRI image.
[0215] It should be noted that the network submodule includes a boundary information restoration network and a denoising network, and the loss function of the boundary information restoration network is l. r The boundary information restoration network is pre-trained to obtain a pre-trained model of the boundary information restoration network; the loss function of the denoising network is l. d The denoising network is pre-trained to obtain a pre-trained model of the denoising network.
[0216] A third aspect of the present invention provides a computer storage medium storing a program for a method of reconstructing an unfolded MRI image with joint boundaries, wherein when the program is executed by a processor, it implements the steps of the method of reconstructing an unfolded MRI image with joint boundaries as described in any of the preceding claims.
[0217] This invention discloses a method, system, and storage medium for unfolded MRI image reconstruction based on joint boundaries. It designs a reconstruction model by incorporating image boundary information and alternately optimizes the reconstructed image and image boundaries using traditional optimization theory. Simultaneously, it expands the networkable sub-modules in the optimization process using deep learning technology, which has strong fitting capabilities. Driven by large-scale data, and combining network module pre-training and overall reconstruction model fine-tuning, an optimized model is trained that can effectively utilize image boundary information to perform high-quality reconstruction of undersampled MRI images. Compared to methods solely based on deep learning, this reconstruction model exhibits higher interpretability and reliability.
[0218] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.
[0219] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.
[0220] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.
[0221] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0222] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.
Claims
1. A method for reconstructing unfolded MRI images with joint boundaries, characterized in that, include: Acquire undersampled MRI image information; An energy model based on a pre-defined joint MRI image boundary; the optimization variables of the energy model consist of reconstructed images and image boundaries; the energy model consists of four weighted terms, namely, a K-space fidelity term, a reconstructed image implicit regularization term, an image boundary implicit regularization term, and a fusion constraint term for reconstructed image and boundary information; Based on optimization theory, the energy model is decomposed into variables, resulting in subproblems 1 and 2 that update individual variables. Based on the preset subproblem optimization algorithm, networkable sub-modules in the algorithm are extracted and expanded; auxiliary variables are introduced for the two subproblems according to the semi-quadratic splitting method; the subproblems are solved by alternately optimizing the auxiliary variables and the target variable; The auxiliary variable solution module is expanded by a neural network, while the target variable solution is further optimized using the single-step gradient descent method. Based on a pre-defined loss function for networkable submodules, the networkable submodules are pre-trained; the specific steps include: pre-setting the loss function for the boundary information repair network. loss function of denoising network ; Based on supervised learning techniques, the boundary information repair network and the denoising network are pre-trained to obtain the corresponding pre-trained models. Based on a preset number of iterations, the loss function and the hyperparameter set in the energy model are designed according to the preset overall joint boundary information expansion reconstruction model, and the overall joint boundary information expansion reconstruction model is obtained. The undersampled MRI images are input into an unfolded reconstruction model with global joint boundary information to obtain the MRI image reconstruction results.
2. The method for reconstructing joint boundaries in MRI images according to claim 1, characterized in that, The energy model is as follows: in, Indicates the reconstructed image; To reconstruct the boundary information of the image, which represents the degree of correlation between the corresponding pixel and the image boundary, the higher the degree of correlation, the closer the corresponding value is to 1; ,in It is a matrix of all ones, that is This indicates the degree of correlation between the corresponding pixel and the non-image boundary; the higher the correlation, the closer the value is to 1. The first term on the right side of the equation is the K-space fidelity term, which constrains the consistency between the reconstructed image and the observed data in K-space. For downsampling K-space data, For Fourier transform operators, For the downsampling operator; the second term on the right-hand side of the equation... Let be the implicit regularization term for the reconstructed image, where These are the weighting coefficients; the third term on the right side of the equation... Let be the implicit regularization term with respect to the image boundary, where These are the weighting coefficients; the fourth term on the right side of the equation is the constraint term for the fusion of reconstructed image and boundary information, where... This represents the Hadamard product operation, where W is the wavelet decomposition operator. Using these as weighting coefficients, the smoothness of the image in the non-edge region in the wavelet domain is reconstructed using the squared reduction of the second norm.
3. The method for reconstructing joint boundaries in MRI images according to claim 2, characterized in that, The subproblems subproblem1 and subproblem2 that update individual variables are specifically as follows: The subproblem 1 for updating individual variables is transformed into the subproblem 1 for updating image boundaries. The subproblem 2 that updates individual variables is transformed into the subproblem 2 that updates the reconstructed image. They are as follows: Updating the image boundary subproblem ; Update and reconstruct image sub-problem ; Where n represents the iteration stage.
4. The method for reconstructing joint boundaries in MRI images according to claim 3, characterized in that, The algorithm for solving the image boundary update subproblem1 is as follows: Introducing auxiliary variables based on the semi-quadratic splitting method Therefore, subproblem1 is transformed into the following optimization problem: in, As an auxiliary variable; The target variable; These are the weighting coefficients; the solution to this problem can be derived from the auxiliary variable V and the target variable. The approximation is achieved through alternating iterative updates, with the following update methods: Among them, for auxiliary variables The solution can be expanded using a neural network, which can be used to optimize the image boundary information. The repair is performed and its relevant regular expressions are learned implicitly, as follows: Among them, for the target variable The solution can be further optimized using single-step gradient descent, as follows: in, This is the gradient descent step size hyperparameter.
5. The method for reconstructing joint boundaries in an unfolded MRI image according to claim 3, characterized in that, The algorithm for solving the subproblem 2 of updating and reconstructing the image is as follows: Introducing auxiliary variables based on the semi-quadratic splitting method Therefore, subproblem2 is transformed into the following optimization problem: in, As an auxiliary variable; The target variable; These are the weighting coefficients; the solution to this problem can be derived from auxiliary variables. and target variable The iterative update approximation is performed using the following methods: Among them, for auxiliary variables The solution can be expanded using neural networks, which are used to optimize the reconstructed image. Denoising is performed and its relevant regularization is implicitly learned, as follows: Among them, for the target variable The solution can be further optimized using single-step gradient descent, as follows: in, This is the gradient descent step size; This is the inverse Fourier transform operator; For wavelet reconstruction operators.
6. The method for reconstructing joint boundaries in an unfolded MRI image according to claim 5, characterized in that, The loss function of the preset boundary information repair network loss function of denoising network Specifically: in, Indicates the number of samples; This is an undersampled MRI image; This is a fully sampled real MRI image; This represents the boundary information of the undersampled MRI image; , representing the boundary information of a fully sampled MRI image.
7. A joint boundary unfolding MRI image reconstruction system, characterized in that, The system includes a memory and a processor. The memory stores a program for a method of unfolded MRI image reconstruction with joint boundaries. When the program is executed by the processor, the method performs the following steps: Acquire undersampled MRI image information; An energy model based on a pre-defined joint MRI image boundary; the optimization variables of the energy model consist of reconstructed images and image boundaries; the energy model consists of four weighted terms, namely, a K-space fidelity term, a reconstructed image implicit regularization term, an image boundary implicit regularization term, and a fusion constraint term for reconstructed image and boundary information; Based on optimization theory, the energy model is decomposed into variables, resulting in subproblems 1 and 2 that update individual variables. Based on the preset subproblem optimization algorithm, networkable sub-modules in the algorithm are extracted and expanded; auxiliary variables are introduced for the two subproblems according to the semi-quadratic splitting method; the subproblems are solved by alternately optimizing the auxiliary variables and the target variable; The auxiliary variable solution module is expanded by a neural network, while the target variable solution is further optimized using the single-step gradient descent method. Based on a pre-defined loss function for networkable submodules, the networkable submodules are pre-trained; the specific steps include: pre-setting the loss function for the boundary information repair network. loss function of denoising network ; Based on supervised learning techniques, the boundary information repair network and the denoising network are pre-trained to obtain the corresponding pre-trained models. Based on a preset number of iterations, the loss function and the hyperparameter set in the energy model are designed according to the preset overall joint boundary information expansion reconstruction model, and the overall joint boundary information expansion reconstruction model is obtained. The undersampled MRI images are input into an unfolded reconstruction model with global joint boundary information to obtain the MRI image reconstruction results.
8. The unfolded MRI image reconstruction system with joint boundaries according to claim 7, characterized in that, The energy model is as follows: in, Indicates the reconstructed image; To reconstruct the boundary information of the image, which represents the degree of correlation between the corresponding pixel and the image boundary, the higher the degree of correlation, the closer the corresponding value is to 1; ,in It is a matrix of all ones, that is This indicates the degree of correlation between the corresponding pixel and the non-image boundary; the higher the correlation, the closer the value is to 1. The first term on the right side of the equation is the K-space fidelity term, which constrains the consistency between the reconstructed image and the observed data in K-space. For downsampling K-space data, For Fourier transform operators, For the downsampling operator; the second term on the right-hand side of the equation... Let be the implicit regularization term for the reconstructed image, where These are the weighting coefficients; the third term on the right side of the equation... Let be the implicit regularization term with respect to the image boundary, where These are the weighting coefficients; the fourth term on the right side of the equation is the constraint term for the fusion of reconstructed image and boundary information, where... This represents the Hadamard product operation, where W is the wavelet decomposition operator. Using these as weighting coefficients, the smoothness of the image in the non-edge region in the wavelet domain is reconstructed using the squared reduction of the second norm.
9. A joint boundary unfolding MRI image reconstruction system according to claim 8, characterized in that, The subproblems subproblem1 and subproblem2 that update individual variables are specifically as follows: The subproblem 1 for updating individual variables is transformed into the subproblem 1 for updating image boundaries. The subproblem 2 that updates individual variables is transformed into the subproblem 2 that updates the reconstructed image. They are as follows: Updating the image boundary subproblem ; Update and reconstruct image sub-problem ; Where n represents the iteration stage.
10. A computer storage medium, characterized in that, The computer storage medium stores a program for a method of reconstructing an unfolded MRI image with a joint boundary. When the program is executed by a processor, it implements the steps of the method of reconstructing an unfolded MRI image with a joint boundary as described in any one of claims 1 to 6.