Magnetic resonance fingerprint data processing method, device, terminal and medium based on cascade neural network

Through the cascade neural network, the relaxation time characteristics are extracted and simulation correction is performed, and the magnetic resonance fingerprint data acquisition time and noise problems in the existing technology are solved, and efficient T1 and T2 quantification is achieved, which improves the time Resolution and calculation speed.

CN115100695BActive Publication Date: 2024-11-15SHANGHAI TECH UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210869824.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-22
Publication Date
2024-11-15
Estimated Expiration
2042-07-22

AI Technical Summary

Technical Problem

In the prior art, the magnetic resonance fingerprint data acquisition time is long, and quantitative images of T1 and T2 cannot be quickly calculated. Especially in preclinical magnetic resonance imaging experiments in mouse brains, the signal dictionary method requires a long data acquisition time, and The reconstructed image contains strong noise and aliasing artifacts.

Method used

Using magnetic resonance fingerprint data processing method based on cascade neural network, supervised learning is performed through multi-layer perceptron and convolutional neural networks, quantitative feature images of relaxation time are extracted, and simulated according to the Bloch equation, using the actual collected k-space The signal is corrected and the T1 and T2 quantitative images output by the neural network are optimized step by step.

Benefits of technology

The accurate quantities T1 and T2 are realized from 15 seconds fingerprint signal and 2 spiral arms acquisition data, which improves the time resolution to 30 seconds/layer, reduces the storage space of neural network parameters, improves the computing speed, and reduces the Effects of noise and aliasing artifacts in quantitative results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115100695B_ABST
    Figure CN115100695B_ABST
Patent Text Reader

Abstract

The present application provides a magnetic resonance fingerprint data processing method, device, terminal and medium based on a cascade neural network, including: obtaining an initial reconstructed magnetic resonance fingerprint image, and connecting the real values ​​corresponding to the real and imaginary parts of the initial reconstructed magnetic resonance fingerprint image and inputting them into the first-level cascade unit to output the corresponding relaxation time quantitative image and the first cascade reconstructed magnetic resonance fingerprint image obtained based on the relaxation time quantitative image; starting from the second-level cascade unit, the cascade reconstructed magnetic resonance fingerprint image output by the previous cascade unit is used as the input image of the current-level cascade unit, and the corresponding relaxation time quantitative image and the corresponding cascade reconstructed magnetic resonance fingerprint image obtained based on the relaxation time quantitative image are output. In the present invention, the storage of neural network parameters can reduce the space occupied by one to two orders of magnitude compared with the storage of signal dictionaries; the quantitative accuracy is high, and it is less affected by noise and aliasing artifacts in the reconstructed image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a magnetic resonance fingerprint data processing method, device, terminal and medium based on a cascade neural network. Background Art

[0002] The two-dimensional magnetic resonance fingerprint technology can simultaneously achieve the relaxation time T1 and T2 quantification of the mouse brain at a time resolution of 2 minutes / layer, making it possible to simultaneously and dynamically track two magnetic resonance tracers, providing convenience for studying the material transport mode of biological tissues, drug metabolism and other studies. However, the time resolution of 2 minutes / layer limits the range of brain areas that can be imaged within the feasible scanning time, and it is also difficult to accurately reflect the real-time changes in the concentration of magnetic resonance tracers. Directly reducing the data acquisition time will inevitably reduce the quality of the reconstructed magnetic resonance fingerprint image and affect the accurate quantification of T1 and T2.

[0003] Two-dimensional magnetic resonance fingerprinting technology has been shown to be able to simultaneously quantify relaxation times T1 and T2 in the human brain, heart, and abdomen with a temporal resolution of 13 seconds per layer, providing a potential quantitative standard for the diagnosis of a series of diseases. However, when this technology is applied to preclinical magnetic resonance experiments, it takes 2 minutes per layer to obtain quantitative T1 and T2 images of the same quality, which reduces the practicality of this technology in preclinical magnetic resonance experiments. Therefore, we hope to further improve the temporal resolution of preclinical magnetic resonance fingerprinting technology.

[0004] Similar to ordinary magnetic resonance imaging, the raw data of the magnetic resonance fingerprint image is also collected in the k-space, and the reconstructed magnetic resonance fingerprint image is obtained after Fourier transformation. In order to achieve the quantification of T1 and T2, the magnetic resonance fingerprint method needs to collect the changes in the magnetic resonance signal intensity caused by T1 and T2 at different time points. Therefore, the two-dimensional magnetic resonance fingerprint image is composed of two-dimensional spatial information and one-dimensional time information, which can be recorded as f(x,y,t|T1(x,y),T2(x,y)). The acquisition time of the magnetic resonance fingerprint data can be obtained by multiplying the fingerprint signal duration by the number of spatial encodings. The shorter the fingerprint signal duration, the fewer the number of spatial encodings, and the shorter the required sampling time; however, the T contained in the short signal 1、 The small amount of T2 information and the small number of spatial encoding times lead to severe reconstruction aliasing artifacts, making it difficult to accurately quantify T 1、 The difficulty of T2 increases.

[0005] Currently, three major types of methods are applied to human MRI fingerprint data processing to quantify T1 and T2 from reconstructed MRI fingerprint images.

[0006] The first category is the pattern recognition method based on signal dictionary. The signal dictionary contains fingerprint signals corresponding to all possible physiological combinations of T1 and T2. The signal of each pixel point of the acquired magnetic resonance fingerprint image is compared with all signal dictionary entries, and the most similar dictionary entry can provide the corresponding T1 and T2 values. Specifically, the signal dictionary simulates the physical model of magnetic resonance signals described by the Bloch equation, with the discretized T1, T2 values ​​and fingerprint signal imaging parameters as input variables. The simulated signal obtained in the dictionary is a complex signal and does not contain any noise or artifacts. On the other hand, the similarity between the acquired signal and the signal dictionary entry is quantitatively evaluated by the amplitude of the inner product normalized by the 2-norm. Therefore, if the noise and artifacts in the time domain are very small in correlation with the fingerprint signal, we can still accurately quantify the T1 and T2 values ​​from the low-quality reconstructed magnetic resonance fingerprint image. This method can accurately and stably quantify T1 and T2 from the human brain k-space data collected by spatial encoding of the fingerprint signal of 13 seconds and 1 spiral arm. The disadvantages of this type of method are that the signal dictionary occupies a large storage space, and the computational overhead of the similarity between the actual collected signal and the signal dictionary entries is large. At the same time, the quantification of T2 is not stable enough in fingerprint signals with a duration of less than 13 seconds, and further reducing the data collection time may produce large errors in the quantitative results.

[0007] The second type of method improves the image reconstruction process to assist in accurate quantification of T 1、 T2. This type of method uses compressed sensing technology to reduce aliasing artifacts in the reconstructed magnetic resonance fingerprint image, and then uses a dictionary-based method to quantify T1 and T2 from the reconstructed magnetic resonance fingerprint image after artifact removal. Since the same biological tissue has similar T1 and T2, the fingerprint signals in the corresponding pixel groups are also highly similar, so the magnetic resonance fingerprint image has low rank in the time domain. Under the premise of ensuring that the k-space information corresponding to the reconstructed magnetic resonance fingerprint image and the actual collected k-space information have data consistency, the compressed sensing method utilizes this low rank in the time domain, and at the same time introduces prior knowledge about image smoothness and sparsity in the spatial domain to regularize the image reconstruction results, so as to reconstruct an image without aliasing artifacts with very little sampling data. This type of compressed sensing image reconstruction method can accurately quantify T1 and T2 from the human brain k-space data collected by spatial encoding of fingerprint signals of about 4-6 seconds and 1 spiral arm, which makes the time resolution about twice as high as the first type of dictionary method. The disadvantage of this type of method is that the compressed sensing reconstruction calculation time is long and has high requirements on the computing power of the computer's central processing unit and the memory space. At the same time, the prior knowledge in the spatial domain used by the regularization term and the balance between regularization and data consistency need to be adjusted according to the actual collected data.

[0008] The third type of method uses deep learning technology to learn the mapping of magnetic resonance fingerprint signals to quantitative values ​​of T1 and T2, so there is no need to store signal dictionaries, and the calculation speed can be greatly improved. Initially, this type of method used a neural network based on a multi-layer perceptron. The multi-layer perceptron extracts the time domain features of the fingerprint signal for the quantification of T1 and T2. It usually uses simulated signals generated according to the physical model and the T1 and T2 values ​​of the generated simulated signals for supervised learning to alleviate the problem of insufficient magnetic resonance fingerprint data. This method can obtain quantitative measurement results consistent with the dictionary method, and the calculation time can be shortened by nearly two orders of magnitude compared with the dictionary method. However, the multi-layer perceptron's ability to handle noise and aliasing artifacts is similar to that of the dictionary method, and it is impossible to further shorten the sampling time of the magnetic resonance fingerprint signal. With the rapid development of deep learning technology, convolutional neural networks have also been used in the data processing of magnetic resonance fingerprint signals. Convolutional neural networks are usually trained directly on reconstructed magnetic resonance fingerprint images containing aliasing artifacts, and at the same time extract temporal and spatial features for T1 and T2 quantification. Through high-quality T 1、 The use of convolutional neural networks greatly reduces the impact of aliasing artifacts and noise in the reconstructed magnetic resonance fingerprint images on the T2 quantitative images of the network output. 1、 In addition to the advantages of saving storage space and fast calculation, the convolutional neural network method can also accurately quantify T1 and T2 from the human brain k-space data collected by spatial encoding of fingerprint signals of 4-6 seconds and 1 spiral arm, just like the compressed sensing method. The bottleneck of this method is that the current application of mouse magnetic resonance fingerprint data is relatively small, so the number of samples available for convolutional neural network training is small, and the high-quality T images of mouse brain used for supervised training are limited. 1、 The acquisition of T2 quantitative images requires a long data acquisition time.

[0009] In preclinical magnetic resonance fingerprint imaging experiments conducted on mouse brains, only the first type of data processing methods based on signal dictionaries are currently used for the quantification of T1 and T2. Due to hardware limitations, preclinical magnetic resonance imaging equipment cannot use multi-channel signal receiving coils, and mouse magnetic resonance imaging requires higher spatial resolution, resulting in stronger noise and aliasing artifacts in the reconstructed magnetic resonance fingerprint images. Therefore, the signal dictionary method requires 20 seconds of fingerprint signals and 6 spiral arms for spatial encoding of mouse magnetic resonance fingerprint k-space data to obtain T1 and T2 quantitative images of similar quality to those in human magnetic resonance fingerprint experiments, so that the maximum time resolution is 2 minutes / layer. In order to shorten the data acquisition time and quickly calculate the quantitative images of T1 and T2, drawing on the experience of human brain magnetic resonance fingerprint data processing technology, the present invention hopes to develop a data processing method based on deep learning to accurately quantify T1 and T from low-quality reconstructed magnetic resonance fingerprint images. 2。 Summary of the invention

[0010] In view of the shortcomings of the prior art described above, the purpose of the present application is to provide a magnetic resonance fingerprint data processing method, device, terminal and medium based on a cascade neural network, so as to solve the technical problems in the prior art that the magnetic resonance fingerprint data acquisition time is long and the quantitative images of T1 and T2 cannot be calculated quickly.

[0011] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides a magnetic resonance fingerprint data processing method based on a cascade neural network, comprising: obtaining an initial reconstructed magnetic resonance fingerprint image, and connecting the real values ​​corresponding to the real part and the imaginary part of the initial reconstructed magnetic resonance fingerprint image and inputting them into a first-level cascade unit to output a corresponding relaxation time quantitative image and a first cascade reconstructed magnetic resonance fingerprint image obtained based on the relaxation time quantitative image; starting from the second-level cascade unit, the cascade reconstructed magnetic resonance fingerprint image output by the previous cascade unit is used as the input image of the current-level cascade unit, and the corresponding relaxation time quantitative image and the corresponding cascade reconstructed magnetic resonance fingerprint image obtained based on the relaxation time quantitative image are output.

[0012] In some embodiments of the first aspect of the present application, the processing process of each cascade unit for reconstructing the magnetic resonance fingerprint image includes: extracting a characteristic image of relaxation time quantification in the time direction from the input reconstructed magnetic resonance fingerprint image; using two independent convolutional neural networks to perform supervised learning respectively to map the characteristic image of relaxation time quantification to the relaxation time quantitative image; based on the relaxation time quantitative image, and according to the magnetic resonance physical model in the Bloch equation, simulating to obtain the magnetic resonance fingerprint image; using the actually collected k-space signal, correcting the accuracy of the simulated magnetic resonance fingerprint image.

[0013] In some embodiments of the first aspect of the present application, the extraction of a feature image of relaxation time quantification in the time direction from the input reconstructed magnetic resonance fingerprint image includes: training a multilayer perceptron to extract feature images when there is a lot of training data; using the multilayer perceptron, taking the fingerprint signal of a single pixel of the reconstructed magnetic resonance fingerprint image as input, and taking the feature image of the pixel as output; the multilayer perceptron includes an input layer, an output layer and two hidden layers, each layer containing neuron nodes equivalent to the number of output eigenvalues; when there is less training data, feature extraction is performed through sliding average processing, and after sliding average, the k-space of the feature image is fully sampled to restore the structural information of the feature image.

[0014] In some embodiments of the first aspect of the present application, the use of two independent convolutional neural networks to perform supervised learning to map the characteristic image of the relaxation time quantification to the relaxation time quantitative image, including: the convolutional neural network uses a four-layer U-net structure to extract relevant information of the relaxation time at four spatial scales from local to global, so as to respectively and supervisely learn the mapping from the characteristic image to the relaxation time quantitative value.

[0015] In some embodiments of the first aspect of the present application, the method of simulating a magnetic resonance fingerprint image based on the relaxation time quantitative image and the magnetic resonance physical model in the Bloch equation includes: using a multilayer perceptron to learn the mapping from the relaxation time quantitative image to the simulated magnetic resonance fingerprint image; the network parameters of the multilayer perceptron are supervised by using the simulated signals obtained by solving a sufficient number of discrete relaxation times as the gold standard.

[0016] In some embodiments of the first aspect of the present application, the loss function used in the supervised learning is:

[0017]

[0018] Where f represents the gold standard simulation signal obtained by solving the Bloch equation, Represents the output signal of the multilayer perceptron, n is the training data number, and a total of N data are used in the training process; the training of the multilayer perceptron is independent of the training of the cascade neural network, and its network parameters will not change after the independent training is completed and are shared by all cascade units.

[0019] In some embodiments of the first aspect of the present application, the use of the actually acquired k-space signal to correct the accuracy of the simulated magnetic resonance fingerprint image includes: using the following optimization problem for correction:

[0020]

[0021] in, is the fingerprint reconstruction image after data consistency correction, subscript N t is the number of sampling time points, and the superscript c is the cascade unit number; It is the simulated image output by the fingerprint simulation module; is the actual acquired k-space signal, and the superscript S indicates the actual acquired signal; is the Fourier transform matrix; U is the k-space sampling mask; λ c is a regularization parameter used to balance the consistency between the reconstructed image and the actual acquired signal and the output image of the simulation module.

[0022] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides a magnetic resonance fingerprint data processing device based on a cascade neural network, comprising: a multi-stage cascade unit; wherein: the first stage cascade unit is used to obtain an initial reconstructed magnetic resonance fingerprint image, and connect the real values ​​corresponding to the real part and the imaginary part of the initial reconstructed magnetic resonance fingerprint image and input them into the first stage cascade unit to output a corresponding relaxation time quantitative image and a first cascade reconstructed magnetic resonance fingerprint image obtained based on the relaxation time quantitative image; from the second stage cascade unit onwards, the cascade reconstructed magnetic resonance fingerprint image output by the previous stage cascade unit is used as the input image of the current stage cascade unit, and the corresponding relaxation time quantitative image and the corresponding cascade reconstructed magnetic resonance fingerprint image obtained based on the relaxation time quantitative image are output.

[0023] In some embodiments of the second aspect of the present application, each cascade unit includes: a feature extraction module, which is used to extract a feature image of relaxation time quantification in the time direction from an input reconstructed magnetic resonance fingerprint image; a parameter quantification module, which is used to use two independent convolutional neural networks to perform supervised learning respectively to map the feature image of relaxation time quantification to a relaxation time quantitative image; a fingerprint image simulation module, which is used to simulate a magnetic resonance fingerprint image based on the relaxation time quantitative image and according to the magnetic resonance physical model in the Bloch equation; and a data consistency correction module, which is used to correct the accuracy of the simulated magnetic resonance fingerprint image using an actually acquired k-space signal.

[0024] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the magnetic resonance fingerprint data processing method based on a cascade neural network is implemented.

[0025] To achieve the above-mentioned purpose and other related purposes, the fourth aspect of the present application provides an electronic terminal, including: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes the magnetic resonance fingerprint data processing method based on the cascade neural network.

[0026] As described above, the magnetic resonance fingerprint data processing method, device, terminal and medium based on cascade neural network of the present application have the following beneficial effects:

[0027] (1) The storage space occupied by neural network parameters can be reduced by one to two orders of magnitude compared to the storage space occupied by signal dictionaries. At the same time, the operation process of the neural network is relatively simple, the degree of parallel operation on the graphics processor is high, and the operation speed is one to two orders of magnitude faster than the dictionary-based pattern recognition method. Furthermore, the current research direction of magnetic resonance fingerprint technology focuses on simultaneous multi-parameter quantification. In addition to the basic T1 and T2 relaxation times, more magnetic resonance parameters such as T 1ρ 、T2 * Relaxation time, fat fraction, etc. can also help diagnose related diseases. As the number of quantitative parameters increases, the size of the signal dictionary increases exponentially, and the amount of computation required to find dictionary entries that match the collected fingerprint signal also increases exponentially. However, using deep learning methods for multi-parameter quantification may only require a slight increase in the number of convolution kernels or training a new neural network for the newly added parameters, and the growth in storage space and computation is closer to a linear increase.

[0028] (2) High quantitative accuracy, less affected by noise and aliasing artifacts in the reconstructed image. The T1 and T2 quantitative images obtained using the deep learning method based on the convolutional neural network have the smallest error with the gold standard, and are significantly smaller than the errors obtained by the other two methods. This advantage will be confirmed in the following examples. On the data set used in the "Example", the deep learning method proposed in the present invention can accurately quantify the values ​​of T1 and T2 from the data collected by spatial encoding of the fingerprint signal with a duration of 15 seconds and 2 spiral arms, so that the time resolution of T1 and T2 quantification is increased to 30 seconds / layer, which solves the problem that the prior art cannot obtain accurate, low-noise, and artifact-free parameter quantitative images from the mouse fingerprint data collected within 30 seconds. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1A Shown is a schematic diagram of the overall cascade network connection structure in one embodiment of the present application.

[0030] Figure 1B Shown is a schematic diagram of the structure of a single cascade unit in an embodiment of the present application.

[0031] Figure 2 Shown is a schematic flow chart of a magnetic resonance fingerprint data processing method based on a cascade neural network in one embodiment of the present application.

[0032] Figure 3A Shown is a schematic diagram of a feature extraction module in one embodiment of the present application using a multi-layer perceptron to extract feature images when training data.

[0033] Figure 3B It is a schematic diagram showing the use of a sliding average method to extract feature images when there is little training data in one embodiment of the present application.

[0034] Figure 4 Shown is a schematic diagram of the structure of U-net in one embodiment of the present application.

[0035] Figure 5 Shown is a schematic diagram of the structure of a multi-layer perceptron used in a fingerprint image simulation module in one embodiment of the present application.

[0036] Figure 6 Shown is a schematic diagram of the structure of an electronic terminal in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0038] As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms "comprise", "include" indicate the presence of the described features, operations, elements, components, items, kinds, and / or groups, but do not exclude the presence, occurrence, or addition of one or more other features, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Therefore, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". Exceptions to this definition will only occur when the combination of elements, functions, or operations is inherently mutually exclusive in some way.

[0039] This paper proposes a deep learning-based MRI fingerprint data processing method that can accurately quantify T1 and T2 from low-quality images reconstructed within a 30-second / layer data acquisition time, thereby increasing the temporal resolution of data acquisition by 4 times. The aim is to solve the problem of how to improve the current mouse brain MRI fingerprint method while performing T 1、 The temporal resolution of T2 quantification, that is, shortening the fingerprint data acquisition time of 2 minutes / layer under current technical means. Shorter data acquisition time means that the MRI fingerprint method can achieve isotropic T1 and T2 quantification of the whole brain within a feasible data acquisition time, or simultaneously and quantitatively track two MRI tracers in the mouse brain with high temporal resolution.

[0040] Specifically, the present invention provides a deep learning method based on a cascaded convolutional neural network to learn to reconstruct magnetic resonance fingerprint images. The data processing method can accurately measure the T1 and T2 values ​​from the 15-second fingerprint signal and the mouse brain k-space data collected by two spiral arms for spatial encoding, and improve the temporal resolution of the mouse magnetic resonance fingerprint method to 30 seconds / layer, which is 4 times higher than the traditional method.

[0041] like Figure 1A and 1B As shown, a schematic diagram of the overall framework structure of a magnetic resonance fingerprint data processing method based on a cascade neural network in an embodiment of the present invention is shown. Figure 1A The diagram shows the overall cascade network connection structure. Figure 1B The diagram shows the structure of a single cascade unit. Among them, f represents the magnetic resonance fingerprint image reconstructed by the magnetic resonance fingerprint, the superscript c is the cascade number, and the subscript N t is the number of sampling time points; F represents the feature image, the superscript c is the cascade number, and the subscript N F is the characteristic image length; y represents the magnetic resonance fingerprint k-space data, the superscript c is the cascade number, s represents the actual sampling signal, and the subscript N t is the number of sampling time points.

[0042] In this embodiment, the process of the magnetic resonance fingerprint data processing method based on the cascade neural network is as follows: Figure 2 As shown:

[0043] Step S21: obtaining an initial reconstructed magnetic resonance fingerprint image, and connecting the real values ​​corresponding to the real part and the imaginary part of the initial reconstructed magnetic resonance fingerprint image and inputting them into the first cascade unit to output the corresponding relaxation time quantitative image and the first cascade reconstructed magnetic resonance fingerprint image obtained based on the relaxation time quantitative image.

[0044] Step S22: starting from the second level cascade unit, the cascade reconstructed magnetic resonance fingerprint image output by the previous level cascade unit is used as the input image of the current level cascade unit, and the corresponding relaxation time quantitative image is output and the corresponding cascade reconstructed magnetic resonance fingerprint image is obtained based on the relaxation time quantitative image.

[0045] Combination Figure 1A Further explanation is as follows: In this example, there are C-level cascade units, namely cascade unit 1, cascade unit 2, ..., cascade unit C; the actual sampled k-space signal After inverse Fourier transform, the initial reconstructed magnetic resonance fingerprint image is obtained

[0046] Initially reconstruct the magnetic resonance fingerprint image The input is cascade unit 1, and the cascade 1 quantitative image of T1 and T2 predicted by the neural network is output. The reconstructed magnetic resonance fingerprint image obtained after simulation and data consistency correction of the cascade 1 quantitative image is recorded as cascade 1 resonance fingerprint image Cascade 1 resonance fingerprint image As the input image of cascade unit 2, the cascade 2 quantitative image is output, and the reconstructed magnetic resonance fingerprint image obtained after simulation and data consistency correction of the cascade 2 quantitative image is recorded as the cascade 2 resonance fingerprint image And so on, until the cascade unit C at the end of the cascade, the cascade (C-1) vibration fingerprint image output by the cascade (C-1) unit As the input image of the cascade unit C, the cascade C quantitative image is output, and the reconstructed magnetic resonance fingerprint image obtained after simulation and data consistency correction of the cascade C quantitative image is recorded as the cascade C resonance fingerprint image

[0047] In this embodiment, the processing process of each cascade unit for reconstructing the magnetic resonance fingerprint image includes: extracting the characteristic image of relaxation time quantification in the time direction from the input reconstructed magnetic resonance fingerprint image; using two independent convolutional neural networks to perform supervised learning respectively to map the characteristic image of relaxation time quantification to the relaxation time quantitative image; based on the relaxation time quantitative image, and according to the magnetic resonance physical model in the Bloch equation, simulating to obtain the magnetic resonance fingerprint image; using the actual collected k-space signal, correcting the accuracy of the simulated magnetic resonance fingerprint image. The above steps will be executed by each module inside the corresponding cascade unit, and the structure of each cascade unit is as follows: Figure 1B As shown, it includes four data processing modules, namely: feature extraction module, parameter quantification module, fingerprint image simulation module and data consistency correction module. The operation process and principle of each module will be explained and illustrated in detail below.

[0048] In this embodiment, the feature extraction module is used to extract the feature image of relaxation time quantification in the time direction from the input reconstructed magnetic resonance fingerprint image. When there is a lot of training data, a multi-layer perceptron can be trained to extract feature images. The feature extraction module uses a multi-layer perceptron to reconstruct the fingerprint signal of a single pixel of the magnetic resonance fingerprint image as input and the feature image of the pixel as output; the multi-layer perceptron includes an input layer, an output layer and two hidden layers, and each layer contains neuron nodes equivalent to the number of output feature values.

[0049] It is worth mentioning that the so-called multi-layer perceptron, also known as MLP (Multi-layer Perceptron), is a feed-forward artificial neural network model that maps multiple input data to a single output data set. For example, each layer of the multi-layer perceptron contains N F nodes, and the final output feature image contains N F The multilayer perceptron is applied to all pixels to obtain N eigenvalues. F feature image.

[0050] For ease of understanding, now combined Figure 3A The process of extracting feature images using a multilayer perceptron when training data is illustrated in the above feature extraction module. The process is as follows: the multilayer perceptron includes an input layer, two hidden layers, and an output layer; the input layer inputs the fingerprint signal of pixel i (including the real part and the imaginary part), and after being processed by the hidden layer, the feature image of pixel i is output to the output layer, and finally the output layer outputs the feature image of relaxation time quantification and Among them, N t Indicates the number of sampling time points; N F Represents the feature image length.

[0051] In some examples, when the training data is small (such as 100-200 data), it is difficult for the multi-layer perceptron to learn a highly universal feature extraction method, so the feature extraction can be effectively performed through sliding average processing. The average window of the sliding average processing in this embodiment contains 48 reconstructed magnetic resonance fingerprint images, and the sliding distance is 16 images. After sliding average, the k-space of the feature image is fully sampled, thereby restoring more detailed structural information.

[0052] Figure 3B When the training data is small, the sliding average method is used to extract the feature image. The process includes: selecting a total of 48 sampled k-spaces (the images are numbered from 1 to 48), and obtaining the corresponding reconstructed magnetic resonance fingerprint image after inverse Fourier transform; the sliding distance is 16 images, and the k-space of the feature image is fully sampled after sliding averaging, thereby restoring more detailed structural information.

[0053] In this embodiment, the parameter quantification module uses two independent convolutional neural networks, which use a four-layer U-net structure to extract relevant information about relaxation time at four spatial scales from local to global, so as to learn the mapping from feature images to relaxation time quantitative values ​​in a supervised manner. It should be noted that the structure of U-net is as follows: Figure 4 As shown in the figure, it is a symmetrical network structure shaped like the letter U. The left half is feature extraction, and the right half is upsampling, forming an Encoder-Decoder structure; where Nx , N y is the number of pixels in the x and y directions of the two-dimensional image, N F is the feature image length, N U is the number of U-net features.

[0054] Specifically, the U-net structure in this embodiment is composed of four encoding units and four decoding units connected in series; each encoding unit uses a 3×3 convolution kernel to extract the features of the feature image in spatial distribution, and the number of convolution kernels used by the lower-layer encoding unit is twice that of the upper-layer encoding unit, so that the number of features continues to increase during the encoding process. At the same time, the feature image processed by the upper-layer encoding unit is input into the lower-layer encoding unit after the 2×2 maximum pooling operation to reduce the spatial resolution by half, and obtains features on a larger spatial scale. Therefore, the spatial resolution of the feature image output by the bottom-layer encoding unit is reduced to 1 / 8 of the input image, while the number of features is increased to 8 times the input image. During the decoding process, the decoding unit first doubles the spatial resolution of the feature image with low spatial resolution through a 2×2 transposed convolution operation, and the number of transposed convolution kernels is half the number of feature images. The feature image obtained after the transposed convolution is connected to the feature image of the same spatial resolution in the encoding process to obtain a feature image combined with encoding and decoding. This combined feature image can use the high spatial resolution information retained in the encoding process to guide the recovery of image details in the decoding process. At the same time, the decoding unit uses a 3×3 convolution kernel to extract the spatial distribution features of the encoding-decoding combined feature image, and the number of convolution kernels is half the number of combined feature images. The feature image output by the bottom decoder is passed through a 3×3 convolution layer to obtain a quantitative image of T1 or T2.

[0055] In this embodiment, the fingerprint image simulation module is based on the relaxation time quantitative image (T1 and T2 quantitative images) output by the parameter quantification module, and simulates the magnetic resonance fingerprint image according to the magnetic resonance physical model in the Bloch equation to prepare for the input of the next level cascade unit.

[0056] Furthermore, considering that the calculation process of solving the Bloch equation for each sampling time point of the fingerprint signal is very time-consuming, this embodiment preferably uses a multilayer perceptron that can simplify the calculation overhead of gradient backpropagation during cascade neural network training so that the calculation time required for the simulation process is greatly reduced to learn the mapping from the values ​​of T1 and T2 to the simulated fingerprint signal. The network parameters of the multilayer perceptron used in the fingerprint image simulation module are supervised by the simulated signals obtained by solving a sufficient number of discrete T1 and T2 as the gold standard; it should be understood that the gold standard refers to the most reliable method for diagnosing diseases recognized by the current clinical medical community. The more commonly used gold standards include biopsy, surgical findings, microbial culture, autopsy, special examinations and imaging diagnosis, as well as the results of long-term follow-up. The loss function used for supervised learning in this embodiment is defined as:

[0057]

[0058] Where f represents the gold standard simulation signal obtained by solving the Bloch equation, represents the output signal of the multilayer perceptron, n is the training data number, and a total of N data are used in the training process. The training of the multilayer perceptron is independent of the training of the cascade neural network. Its network parameters will not change after the independent training is completed and are shared by all cascade units. The structure of the multilayer perceptron used in the fingerprint image simulation module is as follows Figure 5 As shown, N t Indicates the fingerprint signal length.

[0059] The data consistency correction module uses the actually acquired k-space signal to correct the accuracy of the magnetic resonance fingerprint image obtained by the fingerprint image simulation module, so that the subsequent cascade units can quantify the relaxation times T1 and T2 from a more accurate fingerprint image.

[0060] Specifically, the correction process of the data consistency correction module can be expressed using the following optimization problem:

[0061]

[0062] in, is the fingerprint reconstruction image after data consistency correction, subscript N t is the number of sampling time points, and the superscript c is the cascade unit number; It is the simulated image output by the fingerprint simulation module; is the actual acquired k-space signal, and the superscript S indicates the actual acquired signal; is the Fourier transform matrix; U is the k-space sampling mask; λ c is a regularization parameter used to balance the consistency between the reconstructed image and the actual acquired signal and the output image of the simulation module.

[0063] This optimization problem has an explicit solution in:

[0064] is the inverse code of the k-space sampling mask U. That is, the k-space data corresponding to the image output by the simulation module is the position where the k-space is not sampled. Provided, and the sampled position is the actual acquisition of k-space data Simulation k-space data The weighted average of .

[0065] In this embodiment, the magnetic resonance fingerprint data processing method based on the cascade neural network adopts a step-by-step training method for training, and the training process includes: the first training only involves the first-level cascade unit; in each subsequent training, all cascade units before the cascade unit that needs to be trained this time use the latest training results as initial values, and the cascade unit that needs to be trained this time is connected in series with the previous cascade unit for training, so that the network parameters of the cascade unit that needs to be trained this time are updated. For example, in the second training, the network parameters of the first-level cascade unit are fixed, and the second-level cascade unit is connected in series with the first-level cascade unit to participate in the training process, and the training updates the network parameters of the second-level cascade unit; in the third training, the network parameters of the first two cascade units use the results obtained in the second training process as initial values, and are simultaneously updated in the third training process; the training process of more cascade units is obtained by analogy.

[0066] In some examples, when training is performed based on a single cascade unit, if the feature extraction module uses a sliding average operation, the network parameters that can be trained only exist in the U-net structure of the parameter quantification module. In this regard, this embodiment uses the mean absolute percentage error as the loss function: The T1 and T2 quantitative networks are used as gold standard for high-quality quantitative images. and The network parameters are updated under the supervision of B Represents the number of all pixels in the mouse brain, and the loss function is only calculated for the pixels in the mouse brain.

[0067] Furthermore, if the feature extraction module uses a multi-layer perceptron, the series-connected multi-layer perceptron and U-net are trained using a step-by-step method:

[0068] First, multi-layer perception opportunities are involved in training; since there is no gold standard for feature images, a new fully connected layer (such as Figure 3AAfter the hollow arrows in the figure are added to the multi-layer perceptron, T1 and T2 are quantified by the extracted feature image. The effectiveness of the feature image is evaluated by the quantitative effect of T1 and T2, and the loss function uses the T1, T2 predicted by the perceptron and the gold standard The mean absolute percentage error between .

[0069] Secondly, the parameters of the multi-layer perceptron were fixed, the newly added fully connected layer was removed, and it was connected in series with the U-net for T1 and T2 quantification. The network parameters of the U-net were also supervised by the mean absolute percentage error between its output T1 and T2 quantitative images and the gold standard.

[0070] Finally, the cascade network of the multilayer perceptron and U-net uses the network parameters obtained from the first two steps of training as initial values, and uses the mean absolute percentage error between the T1 and T2 quantitative images output by the cascade network and the gold standard as the loss function to update the network parameters of the cascade network as a whole.

[0071] In some examples, when multiple cascade units are trained simultaneously, the loss function of the cascade network is defined as the average of the loss functions of a single cascade unit, that is:

[0072] Where C is the number of cascade units involved in the training. In addition, in the data consistency module of each unit, the regularization parameter λ c As a hyperparameter, the appropriate value is manually selected based on the training results.

[0073] It is worth noting that in the method of processing mouse brain magnetic resonance fingerprint data of the present invention, the deep learning method has the following advantages compared with the dictionary-based pattern recognition method and the method combining compressed sensing image reconstruction with dictionary pattern recognition:

[0074] Advantages 1) The storage space occupied by neural network parameters can be reduced by one to two orders of magnitude compared to the storage space of signal dictionaries. At the same time, the operation process of the neural network is relatively simple, the degree of parallel operation on the graphics processor is high, and the operation speed is one to two orders of magnitude faster than the dictionary-based pattern recognition method. Furthermore, the current research direction of magnetic resonance fingerprint technology focuses on simultaneous multi-parameter quantification. In addition to the basic T1 and T2 relaxation times, more magnetic resonance parameters such as T 1ρ 、T2 * Relaxation time, fat fraction, etc. can also help diagnose related diseases. As the number of quantitative parameters increases, the size of the signal dictionary increases exponentially, and the amount of computation required to find dictionary entries that match the collected fingerprint signal also increases exponentially. However, using deep learning methods for multi-parameter quantification may only require a slight increase in the number of convolution kernels or training a new neural network for the newly added parameters, and the growth in storage space and computation is closer to a linear increase.

[0075] Advantage 2) High quantitative accuracy, less affected by noise and aliasing artifacts in the reconstructed image. The T1 and T2 quantitative images obtained using the deep learning method based on convolutional neural networks have the smallest error with the gold standard, and are significantly smaller than the errors obtained by the other two methods. This advantage will be confirmed in the following examples. On the data set used in the "Example", the deep learning method proposed in the present invention can accurately quantify the values ​​of T1 and T2 from the data collected by spatial encoding of the fingerprint signal with a duration of 15 seconds and 2 spiral arms, so that the time resolution of T1 and T2 quantification is increased to 30 seconds / layer. The parameter quantitative images obtained by the other two processing methods for the fingerprint data collected within 30 seconds all contain large noise.

[0076] It should also be noted that all cascade units in the cascade neural network of the present invention use independent parameter quantitative networks, and these networks are trained using a step-by-step training method, which is obviously and essentially different from other existing cascade units sharing the same parameter quantitative network.

[0077] To further illustrate the advantages of the technical solution of the present invention, the following will be described in conjunction with specific experimental examples:

[0078] The effectiveness of the present invention was tested in the mouse brain for manganese (Mn 2+ ) has been verified on a full-sampled magnetic resonance fingerprint data set obtained in an enhanced magnetic resonance imaging experiment. Manganese, as a magnetic resonance tracer, can significantly reduce the T1 value of the target tissue, while the decrease in the T2 value is not obvious. This experiment aims to prove that the magnetic resonance fingerprint data processing method based on the cascade neural network provided by the present invention can accurately quantify the values ​​of T1 and T2 from the fingerprint signal data collected within 30 seconds, and effectively detect the changes in T1 and T2 caused by the magnetic resonance tracer.

[0079] Manganese-enhanced MRI experiments were performed on 14 C57BL / 6 mice (4-6 months old, weighing about 30 g) with bicine-buffered manganese chloride solution (pH 7.4) at a concentration of 60 mg / kg body weight injected intraperitoneally. The animal experiments were approved by the Animal Care and Use Committee of the performing institution. Each mouse underwent 4 intraperitoneal injections, with an interval of 2 weeks between each intraperitoneal injection. MRI experiments were performed before each intraperitoneal injection and 24 hours after the intraperitoneal injection, that is, each mouse provided 8 sets of MRI data, of which 4 sets were baseline data and 4 sets were enhanced data. All 14 mice provided a total of 112 sets of data.

[0080] Magnetic resonance data were collected using a Bruker Biospec 7T horizontal MRI scanner and a single-channel Bruker 35 mm aperture orthogonal birdcage coil. The parameters of the two-dimensional MRI fingerprint imaging were as follows: the field of view was 2 × 2 cm 2, the image size is 128×128, and the spatial resolution is 0.16×0.16mm 2 , with a layer thickness of 1mm. Each magnetic resonance fingerprint experiment takes 20 seconds to collect fingerprint signals containing 1024 sampling time points, and is spatially encoded by 48 spiral arms to achieve full sampling of the k-space, completing the scan of a layer of brain tissue within 16 minutes. We will select the corresponding sampling mask for the above-mentioned fully sampled data according to the target data acquisition time, and perform retrospective downsampling to simulate the situation where the data acquisition time and the amount of collected data are reduced. Therefore, the gold standard T1 and T2 quantitative images will be obtained from the fully sampled data using a dictionary-based pattern recognition method (see the technical background section for details), and will be used for supervised training of neural networks, as well as to evaluate the T1 and T2 quantitative images obtained by various data processing methods under the condition of undersampling of data.

[0081] For the currently used dataset, the cascaded convolutional neural network was trained and validated using a 7-fold cross-validation method. Each fold was trained using 96 sets of data from 12 mice. This experiment used the left-right flipping data augmentation method to double the amount of training data to obtain 192 sets of training data. The cascaded convolutional neural network was programmed and installed using PyTorch software and trained using NVIDIA's 3080 graphics card. Due to the small amount of training data, the feature extraction module of the neural network used a simple sliding average operation. The neural network was trained using the ADAM optimizer, and the learning rate was set to 2e-4. During the training process, the full 128×128×1024 (N x ,N y ,N t ) will be input into the network, and 3 sets of data are used for each training batch, so the network parameters will be iterated 64 times in one training period. In the actual training process, we found that the loss function of the network on the training set tends to be stable within 200 training periods. On the other hand, the 16 sets of data from the remaining 2 mice are used as validation sets to test the effectiveness of network training. Among them, the performance of the deep learning method is evaluated by the following three indicators:

[0082] Mean absolute percentage error Peak signal-to-noise ratio The mean and variance of the evaluation indicators obtained on each fold validation set will be used to evaluate the effectiveness of the deep learning method. In the above calculation formula, n is the number of pixels inside the mouse brain, and the brain contains N B pixels; It is the T1 (i=1) or T2 (i=2) quantitative image finally output by the cascade neural network; It is a quantitative image of T1 or T2 as the gold standard. During the experiment, it was found that when the number of cascade units is greater than 2, due to the increase in network parameters, it is easy to overfit the training set data, which is not helpful to improve the T1 and T2 quantification of the validation set. Therefore, the experiment uses a cascade network composed of 2 cascade units to quantify T1 and T2. So that the proposed deep learning method can obtain T1 and T2 quantitative images with matching quality from the dictionary method from the 20-second fingerprint signal and 6 spiral arms of the mouse brain magnetic resonance fingerprint data for spatial encoding. Because the latter is currently a commonly used data acquisition and processing method for preclinical mouse brain magnetic resonance fingerprint experiments.

[0083] At the same time, the experiment also used a method based on compressed sensing image reconstruction and dictionary pattern recognition to quantify T1 and T2, which was used to compare with the quantitative images obtained by deep learning methods. Specifically, the image f can be recorded as Since the fingerprint signals of pixels corresponding to the same tissue in f have certain similarities, f has low rank in the time domain. Therefore, we can use To approximate the real image f, where represents a matrix of r time-domain basis vectors of rank r, and Represents the projection of the fingerprint signal corresponding to each pixel on the time domain basis vector. Each column of V is reshaped to a size of N x ×N y The singular value image of . The noise-free time domain basis vector U is obtained from the simulated signal dictionary by the singular value decomposition method. r , so we only need to solve the matrix V to get the reconstructed image Because U r The aliasing artifacts caused by k-space undersampling in the image are not noise-free and will be filtered out during the reconstruction process as long as they are not sufficiently correlated with the time domain basis vectors, thereby achieving the effect of removing reconstruction artifacts.

[0084] In order to further reduce the noise in the reconstructed image, it is proposed that the singular value image contained in the matrix V has local low rank and is sparse after wavelet transform. Therefore, the matrix V can be obtained by solving the following optimization problem: In the first term, M is the sampling mask. is the Fourier transform matrix, is the acquired k-space signal, so this term ensures data consistency. The second term is a regularization term based on local low rank, R b The pixels in the 7×7 neighborhood around pixel b are selected; the third term is a regularization term based on the sparsity of wavelet transform, W is the Dobesi-4 wavelet transform operation. V is obtained by iterative operation of the alternating direction multiplier method.

[0085] The experiment retrospectively downsampled the full sampling data in terms of fingerprint signal duration and number of spiral arms. First, the fingerprint signal duration was fixed to 20 seconds, the number of spiral arms of the sampled k-space signal was reduced, and the performance of the dictionary method, compressed sensing reconstruction method, and deep learning method was compared (see Table 1). Among them, the experiment in Table 1 used 20 seconds of fingerprint signals and 2 spiral arms for k-space sampling of mouse brain magnetic resonance fingerprint data. The expected effect was obtained by the dictionary method in the data with 20 seconds of fingerprint signals and 6 spiral arms. It can be found that the compressed sensing reconstruction method has a small improvement compared with the dictionary method, and the average absolute percentage error is about 15% lower than that of the dictionary method, which does not reach the expected level. The deep learning method can quantify T1 and T2 with the expected accuracy in the k-space data sampled by 2 spiral arms, and the average absolute percentage error is about 38% lower than that of the dictionary method. At the same time, we can see that the quantitative error of the deep learning method in cerebrospinal fluid is much higher than that of other brain tissues, especially T2, which also makes its T2 quantitative results slightly lower than the expected level in the evaluation of peak signal-to-noise ratio and structural similarity.

[0086] Table 1: Evaluation of the accuracy of quantitative T1 and T2 of dictionary method, compressed sensing image reconstruction method, and deep learning method

[0087]

[0088] Subsequently, the experiment reduced the duration of the fingerprint signal on the basis of sampling k-space with two spiral arms, that is, reduced the number of time domain sampling points (see Table 2). Among them, the experiment used mouse brain magnetic resonance fingerprint data with two spiral arms for k-space sampling, and the fingerprint signal duration was selected as 20, 15, and 10 seconds. The expected effect was obtained by the dictionary method in the data of 20-second fingerprint signals and 6 spiral arm sampling. The deep learning method can quantify T1 and T2 with the expected accuracy in the fingerprint signal with a duration of 15 seconds. When the fingerprint signal duration is less than 15 seconds, the T2 quantification of cerebrospinal fluid in the ventricles will have a large deviation.

[0089] Table 2: Evaluation of the accuracy of deep learning methods in quantifying T1 and T2 in MRI fingerprint data with different signal durations

[0090]

[0091] Therefore, the deep learning method proposed in the present invention can accurately quantify T1 and T2 from the mouse brain magnetic resonance fingerprint data collected within 30 seconds, shortening the minimum data sampling time to 1 / 4 of the existing method.

[0092] The magnetic resonance fingerprint data processing method based on the cascade neural network provided in the embodiment of the present invention can be implemented on the terminal side or the server side. As for the hardware structure of the server side device, please refer to Figure 6 :The electronic terminal 600 can be a mobile phone, an electronic terminal, a tablet device, a personal digital processing, a server device, etc. The electronic terminal 600 includes: at least one processor 601, a memory 602, at least one network interface 604 and a user interface 606. The various components in the image classification device are coupled together through a bus system 605. It can be understood that the bus system 605 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 605 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, in Figure 6 In the specification, various buses are labeled as bus systems.

[0093] The user interface 606 may include a display, a keyboard, a mouse, a trackball, a click wheel, keys, buttons, a touch pad or a touch screen.

[0094] It is understood that the memory 602 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable categories of memory.

[0095] The memory 602 in the embodiment of the present invention is used to store various categories of data to support the operation of the electronic terminal 600. Examples of these data include: any executable program used to operate on the electronic terminal 600, such as an operating system 6021 and an application 6022; an original image; a new image obtained by transforming the original image, etc.; wherein the operating system 6021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 6022 may include various applications, such as a media player (MediaPlayer), a browser (Browser), etc., for implementing various application services. The image classification method provided in the embodiment of the present invention may be included in the application 6022.

[0096] The method disclosed in the above embodiment of the present invention can be applied to the processor 601, or implemented by the processor 601. The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 601 or the instruction in the form of software. The above processor 601 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 601 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor 601 may be a microprocessor or any conventional processor, etc. The steps of the image classification method provided in combination with the embodiment of the present invention can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0097] In an exemplary embodiment, the electronic terminal 600 may be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD) to execute the aforementioned method.

[0098] The embodiment of the present invention also provides a magnetic resonance fingerprint data processing device based on a cascade neural network, and its specific structure can be referred to in the attached Figure 1A The magnetic resonance fingerprint data processing device based on the cascade neural network comprises: a multi-stage cascade unit; wherein the first stage cascade unit is used to obtain an initial reconstructed magnetic resonance fingerprint image, and the real values ​​corresponding to the real part and the imaginary part of the initial reconstructed magnetic resonance fingerprint image are connected and input into the first stage cascade unit to output a corresponding relaxation time quantitative image and a first cascade reconstructed magnetic resonance fingerprint image obtained based on the relaxation time quantitative image; from the second stage cascade unit onwards, the cascade reconstructed magnetic resonance fingerprint image output by the previous stage cascade unit is used as the input image of the current stage cascade unit, and the corresponding relaxation time quantitative image and the corresponding cascade reconstructed magnetic resonance fingerprint image obtained based on the relaxation time quantitative image are output.

[0099] In some examples, the structure of each cascade unit can be referred to in the attached Figure 1B, including: a feature extraction module, used to extract a feature image of relaxation time quantification in the time direction from an input reconstructed magnetic resonance fingerprint image; a parameter quantification module, used to use two independent convolutional neural networks to perform supervised learning respectively to map the feature image of relaxation time quantification to a relaxation time quantitative image; a fingerprint image simulation module, used to simulate a magnetic resonance fingerprint image based on the relaxation time quantitative image and according to the magnetic resonance physical model in the Bloch equation; a data consistency correction module, used to use the actually collected k-space signal to correct the accuracy of the simulated magnetic resonance fingerprint image.

[0100] It should be noted that the implementation method of the magnetic resonance fingerprint data processing device based on the cascade neural network in this embodiment is similar to the magnetic resonance fingerprint data processing method based on the cascade neural network, so it will not be described in detail.

[0101] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the magnetic resonance fingerprint data processing method based on the cascade neural network is implemented.

[0102] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0103] In the embodiments provided in the present application, the computer readable and writable storage medium may include a read-only memory, a random access memory, an EEPROM, a CD-ROM or other optical disk storage device, a disk storage device or other magnetic storage device, a flash memory, a USB flash drive, a mobile hard disk, or any other medium that can be used to store a desired program code in the form of an instruction or data structure and can be accessed by a computer. In addition, any connection can be appropriately referred to as a computer-readable medium. For example, if the instruction is sent from a website, a server or other remote source using a coaxial cable, an optical fiber cable, a twisted pair, a digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, the coaxial cable, optical fiber cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of the medium. However, it should be understood that computer readable and writable storage media and data storage media do not include connections, carriers, signals, or other temporary media, but are intended to be non-temporary, tangible storage media. Disk and disc, as used in this application, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers.

[0104] In summary, the present application provides a method, device, terminal and medium for processing magnetic resonance fingerprint data based on a cascade neural network. The storage space occupied by the neural network parameters of the present invention can be reduced by one to two orders of magnitude compared to the storage of the signal dictionary. At the same time, the operation process of the neural network is relatively simple, the degree of parallel operation on the graphics processor is high, and the operation speed is one to two orders of magnitude faster than the dictionary-based pattern recognition method. Furthermore, the current research direction of magnetic resonance fingerprint technology focuses on simultaneous multi-parameter quantification. In addition to the basic T1 and T2 relaxation times, more magnetic resonance parameters such as T 1ρ 、T2 *Relaxation time, fat fraction, etc. are also helpful for the diagnosis of related diseases. As the number of quantitative parameters increases, the size of the signal dictionary increases exponentially, and the amount of calculation to find dictionary entries that match the collected fingerprint signal also increases exponentially. However, using a deep learning method for multi-parameter quantification may only require a slight increase in the number of convolution kernels, or training a new neural network for the newly added parameters, and the growth of storage space and computational complexity is closer to a linear increase; the quantitative accuracy is high, and it is less affected by noise and aliasing artifacts in the reconstructed image. The T1 and T2 quantitative maps obtained using the deep learning method based on the convolutional neural network have the smallest error with the gold standard, and are significantly smaller than the errors obtained by the other two methods. This advantage will be confirmed in the following examples. On the data set used in the "Example", the deep learning method proposed in the present invention can accurately quantify the values ​​of T1 and T2 from the fingerprint signal with a duration of 15 seconds and the data collected by spatial encoding of 2 spiral arms, so that the time resolution of T1 and T2 quantification is increased to 30 seconds / layer, which solves the technical problem that the parameter quantitative maps obtained by the processing method of fingerprint data collected within 30 seconds in the prior art all contain large noise. Therefore, the present application effectively overcomes various shortcomings in the prior art and has high industrial utilization value.

[0105] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A magnetic resonance fingerprint data processing method based on a cascade neural network, characterized in that: include: Acquire an initial reconstructed magnetic resonance fingerprint image, and connect the real values ​​corresponding to the real part and the imaginary part of the initial reconstructed magnetic resonance fingerprint image and input them into the first cascade unit to output the corresponding relaxation time quantitative image and the first cascade reconstructed magnetic resonance fingerprint image obtained based on the relaxation time quantitative image; From the second level cascade unit onwards, the cascade reconstructed magnetic resonance fingerprint image output by the previous level cascade unit is used as the input image of the current level cascade unit, and the corresponding relaxation time quantitative image is output and the corresponding cascade reconstructed magnetic resonance fingerprint image is obtained based on the relaxation time quantitative image; The processing of each cascade unit for reconstructing the magnetic resonance fingerprint image includes: Extracting a relaxation time quantitative feature image in the time direction from the input reconstructed magnetic resonance fingerprint image; Two independent convolutional neural networks are used to respectively perform supervised learning to map the characteristic image of the relaxation time quantification to the relaxation time quantitative image, which includes: the convolutional neural network uses a four-layer U-net structure to extract relevant information of the relaxation time at four spatial scales from local to global, so as to respectively supervise and learn the mapping from the characteristic image to the relaxation time quantitative value; Based on the relaxation time quantitative image, a magnetic resonance fingerprint image is obtained by simulation according to the magnetic resonance physical model in the Bloch equation, which includes: using a multilayer perceptron to learn the mapping from the relaxation time quantitative image to the simulated magnetic resonance fingerprint image; the network parameters of the multilayer perceptron are supervised by using the simulated signals obtained by solving a sufficient number of discrete relaxation times as the gold standard; The accuracy of the magnetic resonance fingerprint image obtained by simulation is corrected using the actually acquired k-space signal.

2. The magnetic resonance fingerprint data processing method based on cascade neural network according to claim 1 is characterized in that: The step of extracting a relaxation time quantitative feature image from the input reconstructed magnetic resonance fingerprint image in the time direction comprises: When there is a lot of training data, a multilayer perceptron is trained to extract feature images; through the multilayer perceptron, a fingerprint signal of a single pixel point of the reconstructed magnetic resonance fingerprint image is used as input, and a feature image of the pixel point is used as output; the multilayer perceptron includes an input layer, an output layer and two hidden layers, and each layer includes neuron nodes equivalent to the number of output feature values; When the training data is small, feature extraction is performed through sliding average processing, and the k-space of the feature image after sliding average is fully sampled to restore the structural information of the feature image.

3. The magnetic resonance fingerprint data processing method based on cascade neural network according to claim 1 is characterized in that: The loss function used in the supervised learning is: Where f represents the gold standard simulation signal obtained by solving the Bloch equation, Represents the output signal of the multilayer perceptron, n is the training data number, and a total of N data are used in the training process; the training of the multilayer perceptron is independent of the training of the cascade neural network, and its network parameters will not change after the independent training is completed and are shared by all cascade units.

4. The magnetic resonance fingerprint data processing method based on cascade neural network according to claim 1 is characterized in that: The method of using the actually acquired k-space signal to correct the accuracy of the magnetic resonance fingerprint image obtained by simulation includes: The following optimization problem is used for calibration: in, is the fingerprint reconstruction image after data consistency correction, subscript N t is the number of sampling time points, and the superscript c is the cascade unit number; It is the simulated image output by the fingerprint simulation module; is the actual acquired k-space signal, and the superscript S indicates the actual acquired signal; is the Fourier transform matrix; U is the k-space sampling mask; λ c is a regularization parameter used to balance the consistency between the reconstructed image and the actual acquired signal and the output image of the simulation module.

5. A magnetic resonance fingerprint data processing device based on a cascade neural network, characterized in that: include: A multi-stage cascade unit; wherein: The first-level cascade unit is used to obtain an initial reconstructed magnetic resonance fingerprint image, and connect the real values ​​corresponding to the real part and the imaginary part of the initial reconstructed magnetic resonance fingerprint image and input them into the first-level cascade unit to output the corresponding relaxation time quantitative image and the first cascade reconstructed magnetic resonance fingerprint image obtained based on the relaxation time quantitative image; From the second level cascade unit onwards, the cascade reconstructed magnetic resonance fingerprint image output by the previous level cascade unit is used as the input image of the current level cascade unit, and the corresponding relaxation time quantitative image is output and the corresponding cascade reconstructed magnetic resonance fingerprint image is obtained based on the relaxation time quantitative image; Each cascade unit includes: A feature extraction module is used to extract a feature image of relaxation time quantification in the time direction from the input reconstructed magnetic resonance fingerprint image; A parameter quantification module, for using two independent convolutional neural networks to respectively perform supervised learning to map the characteristic image of the relaxation time quantification to the relaxation time quantitative image, comprising: the convolutional neural network uses a four-layer U-net structure to extract relevant information of the relaxation time at four spatial scales from local to global, so as to respectively perform supervised learning on the mapping from the characteristic image to the relaxation time quantitative value; A fingerprint image simulation module is used to simulate a magnetic resonance fingerprint image based on the relaxation time quantitative image and the magnetic resonance physical model in the Bloch equation, which includes: using a multi-layer perceptron to learn the mapping from the relaxation time quantitative image to the simulated magnetic resonance fingerprint image; the network parameters of the multi-layer perceptron are supervised by the simulation signals obtained by solving a sufficient number of discrete relaxation times as the gold standard; The data consistency correction module is used to correct the accuracy of the magnetic resonance fingerprint image obtained by simulation using the actually collected k-space signal.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the magnetic resonance fingerprint data processing method based on a cascade neural network as described in any one of claims 1 to 4 is implemented.

7. An electronic terminal, characterized in that: include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes the magnetic resonance fingerprint data processing method based on a cascade neural network as described in any one of claims 1 to 4.