Image reconstruction method based on magnetic resonance fingerprint rapid imaging and deep learning model

Abstract

The present application provides an image reconstruction method, device and storage medium based on magnetic resonance fingerprint rapid imaging and deep learning model. The image reconstruction method includes: obtaining a magnetic resonance fingerprint imaging data set, and a magnetic resonance image reconstructed based on the magnetic resonance fingerprint imaging data set; extracting the relaxation characteristics of the magnetic resonance image; constructing a mapping function based on a partially observable Markov decision process based on the relaxation characteristics of the magnetic resonance image and the time information of the magnetic resonance image; solving the mapping function using a deep reinforcement learning model, and selecting an optimized trajectory from the solution result for image reconstruction of magnetic resonance fingerprint imaging. The present application improves the utilization of K-space information under high-speed mapping by converting the magnetic resonance image reconstruction mapping into a partially observable Markov decision process solution, so that more abundant detailed information is learned during the model learning process, and the loss of image details after reconstruction is reduced, so that magnetic resonance fingerprint imaging has better results in high-speed reconstruction.

Description

Technical Field

[0001] The present application relates to the field of magnetic resonance imaging technology, and in particular to an image reconstruction method, device, and storage medium based on magnetic resonance fingerprint rapid imaging and deep learning models. Background Art

[0002] Quantitative MRI holds great potential for the detection and diagnosis of various diseases, including cancer, edema, and sclerosis. Despite its long-recognized importance, the adoption of quantitative imaging in clinical practice remains a challenge, primarily due to its long acquisition times. Magnetic resonance fingerprinting (MRF) is a relatively new MRI framework for quantitative MRI that can rapidly and simultaneously quantify multiple tissue properties. Compared to traditional quantitative imaging methods, MRF technology uses pseudo-random acquisition parameters, such as flip angles and TRs, to obtain a unique signal evolution for each tissue type and estimates tissue property values using a template matching approach. Specifically, each observed signal evolution is matched against a precomputed MRF dictionary, which contains a large set of tissue property values and their corresponding signal evolutions modeled by the Bloch equation. During the template matching process, the dictionary entry with the highest correlation with the observed signal is selected, and the corresponding tissue property value is retrieved.

[0003] Magnetic resonance fingerprinting (MRF) is a quantitative imaging technique that can simultaneously measure multiple important tissue properties in the human body. While significantly improving scanning efficiency compared to traditional magnetic resonance imaging (MRI), this technique still requires further acceleration before it can be translated into routine clinical practice. Most existing work focuses on pre-determined spiral trajectories, leveraging the evolution of the MRF signal at each pixel to estimate tissue properties and design deep learning models for reconstruction, without considering trajectory optimization. Summary of the Invention

[0004] The present application provides an image reconstruction method, device, and storage medium based on magnetic resonance fingerprint rapid imaging and deep learning model.

[0005] The present application provides an image reconstruction method based on magnetic resonance fingerprint rapid imaging and a deep learning model, the image reconstruction method comprising:

[0006] Acquiring a magnetic resonance fingerprint imaging dataset and a magnetic resonance image reconstructed based on the magnetic resonance fingerprint imaging dataset;

[0007] extracting relaxation features of the magnetic resonance image;

[0008] constructing a mapping function based on a partially observable Markov decision process based on the relaxation characteristics of the magnetic resonance image and the time information of the magnetic resonance image;

[0009] The mapping function is solved using a deep reinforcement learning model, and an optimized trajectory is selected from the solution results for image reconstruction of magnetic resonance fingerprint imaging.

[0010] The magnetic resonance image reconstructed based on the magnetic resonance fingerprint imaging data set includes:

[0011] The magnetic resonance fingerprint imaging data set is reconstructed by using non-uniform fast Fourier transform to obtain the magnetic resonance image.

[0012] The step of extracting the relaxation characteristics of the magnetic resonance image includes:

[0013] Using a fully connected neural network to convert the high-order signal of the magnetic resonance image into a low-dimensional feature vector;

[0014] Preset tissue attributes are screened out from the low-dimensional feature vector, including a first relaxation feature and a second relaxation feature.

[0015] The step of filtering out the preset tissue attributes from the low-dimensional feature vector, including the first relaxation feature and the second relaxation feature, comprises:

[0016] Set cross-correlation as the similarity measure;

[0017] selecting, based on the similarity measure, a dictionary signal that best matches the observed signal at each pixel of the magnetic resonance image;

[0018] assigning a first relaxation time in the dictionary signal to the pixel point to generate a mapping of the first relaxation feature;

[0019] The second relaxation time in the dictionary signal is assigned to the pixel point to generate a mapping of the second relaxation feature.

[0020] The step of constructing a mapping function based on a partially observable Markov decision process based on the relaxation characteristics of the magnetic resonance image and the time information of the magnetic resonance image includes:

[0021] Using the relaxation characteristics of the magnetic resonance image as input to the mapping function, and setting the output of the mapping function to be a de-aliased image;

[0022] The time information is introduced into the mapping function to construct a final mapping function.

[0023] Wherein, taking the relaxation characteristics of the magnetic resonance image as the input of the mapping function includes:

[0024] Construct a fully sampled K-space;

[0025] masking the fully sampled K space with a partially observable K space mask to simulate a partially observable fully sampled K space generated by the partially observed K space;

[0026] A non-uniform fast Fourier transform is performed on the partially observable fully sampled K space to obtain the relaxation characteristics of the magnetic resonance image.

[0027] The mapping function based on the partially observable Markov decision process is expressed as:

[0028]

[0029] in, For the anti-aliased image, M t is the partially observable K-space mask, y is the fully sampled K-space, and t represents the time during the acquisition process.

[0030] Wherein, solving the mapping function by using a deep reinforcement learning model includes:

[0031] A policy function for solving the mapping function is learned using a deep reinforcement learning model to map the de-aliased image and a portion of the observable K-space mask to an action;

[0032] Whenever an action is observed, the anti-aliased image and the partial observable K-space mask are updated based on time to observe the next action;

[0033] A series of motion trajectories obtained through observation within a preset time period.

[0034] The present application also provides a terminal device, the terminal device comprising a memory and a processor, wherein the memory is coupled to the processor;

[0035] The memory is used to store program data, and the processor is used to execute the program data to implement the above-mentioned image reconstruction method.

[0036] The present application also provides a computer storage medium, which is used to store program data. When the program data is executed by a processor, it is used to implement the above-mentioned image reconstruction method.

[0037] The beneficial effects of the present application are as follows: a terminal device acquires a magnetic resonance fingerprint imaging dataset and a magnetic resonance image reconstructed based on the magnetic resonance fingerprint imaging dataset; extracts relaxation features of the magnetic resonance image; constructs a mapping function based on a partially observable Markov decision process based on the relaxation features of the magnetic resonance image and the time information of the magnetic resonance image; solves the mapping function using a deep reinforcement learning model, and selects an optimized trajectory from the solution result for image reconstruction of magnetic resonance fingerprint imaging. The present application improves the utilization of K-space information under high-speed mapping by converting the magnetic resonance image reconstruction mapping into a partially observable Markov decision process solution, enables the model learning process to learn richer detailed information, reduces the loss of image details after reconstruction, and makes magnetic resonance fingerprint imaging have better effects in high-speed reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts. Among them:

[0039] Figure 1 This is a flow chart of an embodiment of an image reconstruction method based on magnetic resonance fingerprint rapid imaging and deep learning model provided by the present application;

[0040] Figure 2 Schematic diagram of brain MRF imaging in different axial planes provided by this application;

[0041] Figure 3 Schematic diagram of brain MRF imaging of different slices provided by this application;

[0042] Figure 4 Schematic diagram of MRF imaging of the abdominal liver provided in this application;

[0043] Figure 5 This is a schematic structural diagram of an embodiment of a terminal device provided by this application;

[0044] Figure 6 It is a structural diagram of an embodiment of a computer storage medium provided by this application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0046] In order to solve the technical problem of image quality in MRF (Magnetic Resonance Fingerprinting) accelerated imaging, this application designs a method based on active sampling for MRF accelerated imaging. First, a fully connected network is used to realize feature extraction. Secondly, the active MRF acquisition problem is converted into a partially observable Markov decision process (POMDP, partially observable Markov decision processes) mapping. Then, a deep reinforcement learning model is used to solve the mapping process. Finally, the trajectory optimization of the MRF image at high speed is realized to obtain better image results. Compared with previous methods, the imaging results are better, the acceleration speed is higher, and the generalization ability is better.

[0047] Please refer to the following for details: Figure 1 , Figure 1 This is a flow chart of an embodiment of an image reconstruction method based on magnetic resonance fingerprint rapid imaging and deep learning model provided in this application.

[0048] like Figure 1 As shown, the image reconstruction method based on magnetic resonance fingerprint imaging in the embodiment of the present application specifically includes the following steps:

[0049] Step S11: Acquire a magnetic resonance fingerprint imaging dataset and a magnetic resonance image reconstructed based on the magnetic resonance fingerprint imaging dataset.

[0050] In an embodiment of the present application, the terminal device uses the dictionary matching method introduced in the original MRF framework to obtain the organizational attribute graph of Ground Truth from the MRF data collected at the time point.

[0051] Specifically, the terminal device scans the human body through a magnetic resonance device to obtain a large amount of magnetic resonance data, and then processes the magnetic resonance data using a non-uniform fast Fourier transform (NUFFT) to obtain a reconstructed magnetic resonance image.

[0052] Step S12: extracting relaxation features of the magnetic resonance image.

[0053] In this embodiment of the present application, the terminal device uses cross-correlation as a similarity metric and selects the signal evolution in the dictionary that best matches the evolution of the observed signal at each pixel in the magnetic resonance image, that is, matches the pixel with a pre-calculated MRF dictionary. The T1 value and T2 value corresponding to the best match obtained by matching the MRF dictionary are assigned to the corresponding pixel.

[0054] The above matching process is repeated by traversing all pixels on the reconstructed magnetic resonance image, and quantitative T1 and T2 maps are generated at the same time. Figures 2 to 4 MRF imaging shown.

[0055] Specifically, the feature extraction module in the terminal device of the embodiment of the present application uses a fully connected neural network (FNN, Feedforward Neural Network) to convert the high-dimensional signal evolution of the magnetic resonance image into a low-dimensional feature vector, wherein the low-dimensional feature vector contains useful information for estimating the magnetic resonance data. Each fully connected neural network of the feature extraction module is used to extract features and estimate a specific tissue attribute. In the embodiment of the present application, the feature extraction module includes two fully connected neural networks for estimating two specific tissue attributes, namely T1 relaxation time and T2 relaxation time.

[0056] Step S13: constructing a mapping function based on a partially observable Markov decision process based on the relaxation characteristics of the magnetic resonance image and the time information of the magnetic resonance image.

[0057] In an embodiment of the present application, the terminal device converts active MRF acquisition into a partially observable Markov decision process (POMDP).

[0058] Specifically, suppose Is a representation of the full sampling K space, x is obtained by non-uniform fast Fourier transform The reconstruction of y obtained is Then, the partially observable K-space mask M is used to simulate the partially observed K-space generation

[0059] Using correlation as a similarity measure, the signal evolution in the dictionary that best matches the observed signal evolution at each pixel is selected. The T1 and T2 values corresponding to the best match are assigned to that pixel. This process is repeated throughout the image, generating both a quantitative T1 map and a T2 map.

[0060] In the embodiment of the present application, the terminal device uses deep learning to reconstruct the model need As input, the output is a de-aliased image The reconstructed model is Parameterize Convolutional Neural Networks (CNN). Finally, use subscripts to indicate time, for example represents the reconstruction obtained during the acquisition process at time step t.

[0061] In active MRF acquisition, the goal is to learn a policy function This formula means that at time t, the de-aliased image is reconstructed and represents the observation frequency M t The mask of is mapped to action a t In active non-Cartesian MRF sampling, the action is composed of each pixel in the fully sampled K space, and y represents each mask M t The value is 0. Once an action is observed, the mask and anti-aliased image reconstruction are updated in is a binary matrix. For all mappings, a series of actions [a t ,a t+1 ,…,a T The parameters θ of the optimization strategy are used to reduce the sampling loss function (maximize the activation function). The loss function can be a conventional loss function such as SSIM, PSNR, etc. Therefore, the goal is to minimize the loss within the acceleration range of the entire MRF acquisition.

[0062] Step S14: using the deep reinforcement learning model to solve the mapping function, and selecting the optimized trajectory from the solution results for image reconstruction of magnetic resonance fingerprint imaging.

[0063] In the embodiments of the present application, DDQN (Double Deep Q-Network) is a representative deep reinforcement learning network. Here, DDQN is taken as an example. Other networks are also applicable to this application and are not listed here one by one.

[0064] The POMDP strategy is a function of the observation history, however, in active non-Cartesian MRF acquisition, the entire history of observations is captured by the current observation o t Therefore, we use o t As a single input value network, the value network is used to receive the income for reconstructing the image and mask M t .

[0065] In order to obtain the reconstructed image X at each time step t , a pre-trained reconstruction network can be used. In addition, a variant for MRF is considered that only takes the time step information as input, which is equivalent to considering the mask M tThe number of non-zero elements in , therefore, the same acquisition trajectory is used for the slices in the dataset. In both cases, the value network is constrained to choose from valid actions by setting the value of all previously observed k-space elements to -∞. In addition, a modified ε-greedy strategy is used as the exploration strategy to fill the repeated memory buffer. This strategy selects the best action with probability 1-ε and selects an action from a set of valid actions with probability ε.

[0066] In this way, the terminal device solves the mapping function through the deep reinforcement learning model and selects several optimal actions from the solution results to form an optimized trajectory to compensate for the information lost in the dimension when decomposing the dictionary.

[0067] In an embodiment of the present application, a terminal device obtains a magnetic resonance fingerprint imaging dataset and a magnetic resonance image reconstructed based on the magnetic resonance fingerprint imaging dataset; extracts relaxation features of the magnetic resonance image; constructs a mapping function based on a partially observable Markov decision process based on the relaxation features of the magnetic resonance image and the time information of the magnetic resonance image; solves the mapping function using a deep reinforcement learning model, and selects an optimized trajectory from the solution result for image reconstruction of magnetic resonance fingerprint imaging. This application improves the utilization of K-space information under high-speed mapping by cutting into the sampling trajectory in K-space, converting the MRF tissue attribute mapping into a partially observable Markov decision process solution, thereby improving the utilization of K-space information under high-speed mapping, enabling the model to learn more detailed information during learning, reducing the loss of image details after reconstruction, and making MRF have better effects in high-speed reconstruction.

[0068] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0069] In order to implement the image reconstruction method based on magnetic resonance fingerprint imaging of the above embodiment, this application also proposes a terminal device, please refer to Figure 5 , Figure 5 It is a structural diagram of an embodiment of a terminal device provided by this application.

[0070] The terminal device 500 of the embodiment of the present application includes a memory 51 and a processor 52, wherein the memory 51 and the processor 52 are coupled.

[0071] The memory 51 is used to store program data, and the processor 52 is used to execute the program data to implement the image reconstruction method based on magnetic resonance fingerprint imaging described in the above embodiment.

[0072] In this embodiment, the processor 52 may also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip having signal processing capabilities. The processor 52 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor, or the processor 52 may be any conventional processor.

[0073] This application also provides a computer storage medium, such as Figure 6 As shown, the computer storage medium 600 is used to store program data 61. When the program data 61 is executed by the processor, it is used to implement the image reconstruction method based on magnetic resonance fingerprint imaging as described in the above embodiment.

[0074] The present application also provides a computer program product, wherein the computer program product includes a computer program, and the computer program is operable to cause a computer to execute the image reconstruction method based on magnetic resonance fingerprint imaging as described in the embodiments of the present application. The computer program product can be a software installation package.

[0075] The image reconstruction method based on magnetic resonance fingerprint imaging described in the above embodiment of the present application, when implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0076] The above description is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image reconstruction method based on magnetic resonance fingerprint rapid imaging and deep learning model, characterized in that: The image reconstruction method comprises: Acquiring a magnetic resonance fingerprint imaging dataset and a magnetic resonance image reconstructed based on the magnetic resonance fingerprint imaging dataset; extracting relaxation features of the magnetic resonance image; constructing a mapping function based on a partially observable Markov decision process based on the relaxation characteristics of the magnetic resonance image and the time information of the magnetic resonance image; Solving the mapping function using a deep reinforcement learning model and selecting an optimized trajectory from the solution results for image reconstruction of magnetic resonance fingerprint imaging; The step of extracting the relaxation characteristics of the magnetic resonance image includes: Using a fully connected neural network to convert the high-order signal of the magnetic resonance image into a low-dimensional feature vector; Set cross-correlation as the similarity measure; selecting, based on the similarity measure, a dictionary signal that best matches the observed signal at each pixel of the magnetic resonance image; assigning a first relaxation time in the dictionary signal to the pixel point to generate a mapping of a first relaxation feature; The second relaxation time in the dictionary signal is assigned to the pixel point to generate a mapping of the second relaxation feature.

2. The image reconstruction method according to claim 1, wherein: The magnetic resonance image reconstructed based on the magnetic resonance fingerprint imaging data set includes: The magnetic resonance fingerprint imaging data set is reconstructed by using non-uniform fast Fourier transform to obtain the magnetic resonance image.

3. The image reconstruction method according to claim 1, wherein: The constructing of a mapping function based on a partially observable Markov decision process based on the relaxation characteristics of the magnetic resonance image and the time information of the magnetic resonance image includes: Using the relaxation characteristics of the magnetic resonance image as input to the mapping function, and setting the output of the mapping function to be a de-aliased image; The time information is introduced into the mapping function to construct a final mapping function.

4. The image reconstruction method according to claim 3, wherein: The step of using the relaxation characteristics of the magnetic resonance image as input of the mapping function comprises: Construct a fully sampled K-space; masking the fully sampled K space with a partially observable K space mask to simulate a partially observable fully sampled K space generated by the partially observed K space; A non-uniform fast Fourier transform is performed on the partially observable fully sampled K space to obtain the relaxation characteristics of the magnetic resonance image.

5. The image reconstruction method according to claim 3, wherein: The mapping function based on the partially observable Markov decision process is expressed as: ， in, To de-alias the image, is the partially observable K-space mask, y is the fully sampled K-space, and t represents the time during the acquisition process.

6. The image reconstruction method according to claim 5, characterized in that: Solving the mapping function using a deep reinforcement learning model includes: A policy function for solving the mapping function is learned using a deep reinforcement learning model to map the de-aliased image and a portion of the observable K-space mask to an action; Whenever an action is observed, the anti-aliased image and the partial observable K-space mask are updated based on time to observe the next action; A series of motion trajectories obtained through observation within a preset time period.

7. A terminal device, characterized in that: The terminal device includes a memory and a processor, wherein the memory is coupled to the processor; The memory is used to store program data, and the processor is used to execute the program data to implement the image reconstruction method according to any one of claims 1 to 6.

8. A computer storage medium, characterized in that The computer storage medium is used to store program data, and when the program data is executed by the processor, it is used to implement the image reconstruction method according to any one of claims 1 to 6.

Images