Magnetic resonance image reconfiguration device and magnetic resonance image reconfiguration method

Abstract

To improve an image quality of a reconfigured image.SOLUTION: A magnetic resonance image reconfiguration device according to the embodiment reconfigures the magnetic resonance image data in which an artifact resulted from an under-sampling is removed or reduced, based on the under-sampled k-space data. The device includes a reconfiguration part for reconfiguring the magnetic resonance image data using a reconfiguration network having a correction module. The correction module includes: a regularization block that performs a regularization process to a first image data using a first neural network and that generates second image data; and a data matching block that performs a data matching process and generates third image data such that the k-space data corresponding to the second image data becomes closer to the under-sampled k-space data. The correction module further includes at least one of the data matching adjustment block for adjusting the data matching process and a regularization adjustment block for adjusting the regularization process.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The embodiments disclosed in this specification and the drawings relate to a magnetic resonance image reconstruction apparatus and a magnetic resonance image reconstruction method. [Background technology]

[0002] A magnetic resonance imaging (MRI) system is a non-invasive medical imaging system that utilizes the magnetic resonance phenomenon, in which hydrogen nuclei in a static magnetic field resonate with a radio-frequency magnetic field of a specific frequency. Magnetic resonance imaging technology has the advantages of high resolution, no trauma, and no radiation, and can examine various solid organs in the human body. Therefore, it is widely used in the diagnosis of clinical diseases. A magnetic resonance imaging system includes a magnetic resonance scanning device that scans a subject and a magnetic resonance image reconstruction device that reconstructs an image based on the scan data.

[0003] Conventional magnetic resonance imaging devices use raster scanning to acquire k-space data in k-space, and after the k-space data collection is complete, they perform an inverse Fourier transform on the k-space data to perform imaging. Furthermore, conventional magnetic resonance technology requires the collection of complete k-space data, which results in problems such as long scan times, discomfort to patients, and a tendency for motion artifacts to occur.

[0004] A technique is known for accelerating the scan time of a magnetic resonance imaging device by sampling only a portion of k-space data and reconstructing an image using undersampled k-space data. Magnetic resonance image reconstruction devices that use this technique include those based on algorithms such as parallel imaging or compressed sensing. However, such techniques result in low imaging quality due to the lack of information in some k-space.

[0005] In recent years, with the development of machine learning technology, magnetic resonance image reconstruction devices using deep learning technology have been proposed. Deep learning technology can be used to correct image space or k-space data, or to implement end-to-end reconstruction neural networks. As a deep learning-based magnetic resonance image reconstruction technology, a technique has been proposed for reconstructing images from undersampled k-space data using an end-to-end unrolled learning network. This technology has advantages such as high reconstruction accuracy and easy learning. Here, the learning network refers to a large neural network that includes multiple interconnected sub-neural networks.

[0006] Non-Patent Document 1 discloses a magnetic resonance image reconstruction device based on an end-to-end unrolled learning network, which uses a convolutional neural network with residual connections as a prior term to reconstruct image data from undersampled k-space data. The invention in Non-Patent Document 1 performs the same processing on input data with different resolutions, signal-to-noise ratios, scan targets, etc.

[0007] Patent Document 1 discloses a medical image processing device based on a neural network that can handle input data with different signal-to-noise ratios, by outputting a control signal based on attributes related to the signal-to-noise ratio of input data and adjusting the parameters of the neural network according to the control signal during neural network inference. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] US Patent Application Publication No. 2018 / 0349759 [Non-patent literature]

[0009] [Non-Patent Document 1] “MoDL: Model Based Deep Learning Architecture for Inverse Problems”, Aggarwal HK, Mani MP, Jacob M, IEEE Trans Med Imaging, 2019;38(2):p394-405 Summary of the Invention [Problem to be solved by the invention]

[0010] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to improve the image quality of a reconstructed image. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0011] A magnetic resonance image reconstruction apparatus according to an embodiment reconstructs magnetic resonance image data from undersampled k-space data, with artifacts caused by undersampling removed or reduced, and includes a reconstruction unit that reconstructs the magnetic resonance image data using a reconstruction network having a correction module. The correction module includes: a regularization block that performs a regularization process on first image data using a first neural network to generate second image data; and a data consistency block that performs data consistency processing so that k-space data corresponding to the second image data approaches the undersampled k-space data to generate third image data. The correction module further includes at least one of a data consistency adjustment block that adjusts the data consistency processing and a regularization adjustment block that adjusts the regularization processing. [Brief explanation of the drawings]

[0012] [Figure 1]FIG. 1 is a diagram showing an example of the configuration of a magnetic resonance image reconstruction apparatus according to the first embodiment. [Figure 2] FIG. 2 is a data flow diagram for explaining the processing executed by the preprocessing unit according to the first embodiment. [Figure 3] FIG. 3 is a data flow diagram for explaining the reconstruction process executed by the reconstruction unit according to the first embodiment. [Figure 4] FIG. 4 is a block diagram illustrating the configuration of the correction module according to the first embodiment. [Figure 5] FIG. 5 is a data flow diagram for explaining processing by the image space regularization adjustment block included in the correction module according to the first embodiment. [Figure 6] FIG. 6 is a data flow diagram for explaining processing by the image space regularization block included in the correction module according to the first embodiment. [Figure 7] FIG. 7 is a data flow diagram for explaining the convolutional activation process executed by the convolutional activation unit included in the image space regularization neural network according to the first embodiment. [Figure 8] FIG. 8 is a data flow diagram for explaining the processing by the data consistency adjustment block included in the correction module according to the first embodiment. [Figure 9] FIG. 9 is a data flow diagram for explaining the processing by the correction module according to the first embodiment. [Figure 10] FIG. 10 is a flowchart showing the flow of the magnetic resonance image reconstruction method according to the first embodiment. [Figure 11] FIG. 11 is a diagram comparing complete image data reconstructed by the magnetic resonance image reconstruction apparatus according to the first embodiment with complete image data reconstructed by the prior art. [Figure 12] FIG. 12 is a diagram comparing complete image data reconstructed by the magnetic resonance image reconstruction apparatus according to the first embodiment with complete image data reconstructed by the prior art. [Figure 13]FIG. 13 is a block diagram illustrating the configuration of a correction module according to the second embodiment. [Figure 14] FIG. 14 is a data flow diagram for explaining the processing by the correction module according to the second embodiment. [Figure 15] FIG. 15 is a flowchart showing the flow of the magnetic resonance image reconstruction method according to the second embodiment. [Figure 16] FIG. 16 is a block diagram illustrating the configuration of a correction module according to the third embodiment. [Figure 17] FIG. 17 is a data flow diagram for explaining the processing by the correction module according to the third embodiment. [Figure 18] FIG. 18 is a flowchart showing the flow of the magnetic resonance image reconstruction method according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, a magnetic resonance image reconstruction apparatus and a magnetic resonance image reconstruction method according to the present invention will be described with reference to the drawings.

[0014] (First embodiment) The magnetic resonance image reconstruction apparatus according to this embodiment performs image reconstruction of image space image data based on undersampled k-space data obtained by scanning a subject with a magnetic resonance scanning apparatus. The objective is to reconstruct image data equivalent to fully sampled k-space data (hereinafter referred to as complete image data). Here, the k-space data is obtained by the magnetic resonance scanning apparatus transmitting pulse signals to a subject in a frequency-encoded and phase-encoded magnetic field and receiving echo signals due to specific nuclear magnetic resonance from multiple receiving coils. The complete image data can be rephrased as magnetic resonance image data from which artifacts due to undersampling have been removed or reduced.

[0015] 1 is a diagram showing an example of the configuration of a magnetic resonance image reconstruction device 1 according to the first embodiment. The magnetic resonance image reconstruction device 1 of the first embodiment includes an input / output interface 10, a display interface 20, a communication interface 30, a storage unit 40, a pre-processing unit 50, and a reconstruction unit 60. The input / output interface 10, the display interface 20, the communication interface 30, the storage unit 40, the pre-processing unit 50, and the reconstruction unit 60 are connected to each other so that they can communicate with each other.

[0016] The input / output interface 10 is an interface for connecting the magnetic resonance image reconstruction apparatus 1 to an input device (not shown), receives a user's input operation from the input device, and transmits a signal based on the received input operation to the magnetic resonance image reconstruction apparatus 1. The input / output interface 10 is, for example, a serial bus interface such as a USB. The input device includes a mouse, keyboard, trackball, switch, button, joystick, touch screen, microphone, etc. The input / output interface 10 may also be connected to a storage device to read and write various data to and from the storage device. The storage device may be, for example, a hard disk drive (HDD), a solid state drive (SSD), etc.

[0017] The display interface 20 is an interface for connecting the magnetic resonance image reconstruction apparatus 1 to a display device (not shown), and transmits data to the display device to display an image. The display interface 20 is, for example, a video output interface such as a DVI (Digital Visual Interface) or an HDMI (registered trademark) (High-Definition Multimedia Interface). The display device includes an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence) display. The display device displays a user interface for accepting input operations from a user and complete image data output by the magnetic resonance image reconstruction apparatus 1, and the user interface is, for example, a GUI (Graphical User Interface).

[0018] The communication interface 30 is an interface for connecting the magnetic resonance image reconstruction apparatus 1 to a server (not shown), and is capable of transmitting and receiving various data to and from the server. The communication interface 30 is, for example, a network card such as a wireless network card or a wired network card.

[0019] The storage unit 40 stores user data such as image data and k-space data used for image reconstruction. The storage unit 40 also stores parameters, such as neural network parameters, used by the magnetic resonance image reconstruction device 1 when performing image reconstruction. The storage unit 40 also stores training data for learning each neural network and other trainable parameters used by the magnetic resonance image reconstruction device 1. The storage unit 40 is implemented by a storage device such as a ROM, flash memory, random access memory (RAM), hard disk drive (HDD), solid state drive (SSD), or register. Flash memory, HDD, SSD, etc. are non-volatile storage media. These non-volatile storage media may also be implemented by other storage devices connected via a network, such as a network-attached storage (NAS) or an external storage server device. Here, the network includes, for example, the Internet, a wide area network (WAN), a local area network (LAN), a carrier terminal, a wireless communication network, a wireless base station, a dedicated line, etc.

[0020] The preprocessing unit 50 performs preprocessing on the undersampled k-space data K0, which is input data to the magnetic resonance image reconstruction device 1. In the following description, the k-space data K0 is described as three-dimensional tensor data with a width W, a height H, and a number of channels (number of receiving coils) C, where the width direction is the frequency encoding direction and the height direction is the phase encoding direction. Typically, to shorten scan time, magnetic resonance scanning is performed by undersampling, omitting specific phase encoding. Therefore, during magnetic resonance scanning, some coordinates in the phase encoding direction (height direction) are skipped. As a result, data does not exist at some coordinates in the height direction (phase encoding direction) in the k-space data K0, and zero padding is performed on the data at these coordinates. Because data near the center of the k-space data has a significant impact on the contrast of the reconstructed image data, undersampling typically involves concentrating sampling of data near the center in the phase encoding direction and skipping data at some positions far from the center.

[0021] The following describes the processing executed by the pre-processing unit 50. Fig. 2 is a data flow diagram for explaining the processing executed by the pre-processing unit 50 according to the first embodiment. In Fig. 2, the flow of data is indicated by solid arrows.

[0022] 2, the preprocessing unit 50 includes an inverse Fourier transform unit 51 and a channel merging unit 52. The inverse Fourier transform unit 51 performs an inverse Fourier transform on the data using an algorithm such as an inverse fast Fourier transform. The channel merging unit 52 merges multi-channel data corresponding to each receive coil of the magnetic resonance scanning device into single-channel data.

[0023] The preprocessing unit 50 preprocesses the undersampled k-space data K0 to generate primary image data X0.

[0024] First, the preprocessing unit 50 reads the k-space data K0 from the storage unit 40, and performs an inverse Fourier transform on the k-space data K0 using the inverse Fourier transform means 51 to generate multi-channel image data I0. The multi-channel image data I0 is image space data that has the same width, height, and number of channels as the k-space data K0. The data for each channel of the multi-channel image data I0 is image space data converted from the k-space data collected by each receive coil.

[0025] The pre-processing unit 50 then uses the channel integrating means 52 to integrate the data from multiple channels of the multi-channel image data I0 into data for one channel based on the sensitivity of each receive coil, thereby generating primary image data X0. The primary image data X0 is two-dimensional image data with a width W and a height H. Because the primary image data X0 is generated directly from the undersampled k-space data K0, it contains artifacts and noise, resulting in problems such as a lack of detail and blurred images.

[0026] Returning to the explanation of Fig. 1, the reconstruction unit 60 reconstructs complete image data, which is magnetic resonance image data equivalent to fully sampled k-space data, based on the initial image data X0. Note that the embodiment is not limited to this, and the reconstructed magnetic resonance image data may be image data with image quality equivalent to a reconstructed image based on fully sampled k-space data, or image data from which artifacts due to undersampling have been removed.

[0027] The following describes the reconstruction processing executed by the reconstruction unit 60. Fig. 3 is a data flow diagram for explaining the reconstruction processing executed by the reconstruction unit 60 according to the first embodiment. In Fig. 3, the flow of data is indicated by solid arrows.

[0028] 3, the reconfiguration unit 60 includes a reconfiguration network RN. The reconfiguration network RN includes n (n is an integer equal to or greater than 1) correction modules UM1 to UM2 connected in series. nThe number n of the reconfigurable networks RN is preferably 8 to 10. Hereinafter, each of the modification modules UM1 to UM n When there is no distinction between these, they are collectively referred to as the modification module UM. The modification module UM performs modification processing on the image data so that the image data approaches the perfect image data.

[0029] The reconstruction unit 60 reconstructs complete image data equivalent to fully sampled k-space data by correcting the initial image data X0 using the reconstruction network RN. First, the reconstruction unit 60 inputs the initial image data X0 and k-space data K0 acquired from the preprocessing unit 50 to the reconstruction network RN.

[0030] The reconstruction network RN then inputs the initial image data X0 and k-space data K0 to a correction module UM1 located at the front stage. The correction module UM1 corrects the initial image data X0 based on the initial image data X0, the k-space data K0, and mask data (not shown), generates corrected image data X1, and outputs it to a correction module UM2. The corrected image data X1 is an image closer to complete image data than the initial image data X0. The mask data indicates which phase encodes of the k-space data K0 were sampled and which phase encodes were omitted in the magnetic resonance scan. The mask data is, for example, a matrix with width W x height H, where the width direction is the frequency encode direction and the height direction is the phase encode direction. In the mask data, the value of the elements of the coordinates of the frequency encodes and phase encodes that have been sampled is set to 1, and the value of the elements of the coordinates of the frequency encodes and phase encodes that have not been sampled is set to 0.

[0031] And each correction module UM2~UM n-1 The correction module UM further corrects the corrected image data output from the previous correction module based on the corrected image data, k-space data K0, and mask data output from the previous correction module, and outputs the further corrected corrected image data to the next stage. t(t is an integer between 2 and N) The operation of the correction module UM will be explained using the example of correction module UM t is the previous correction module UM t-1 Corrected image data X output from t-1 , based on the k-space data K0 and the mask data, the corrected image data X t-1 is corrected to approach the complete image data, and the corrected image data X t Generate the modified module UM t+1 Output to.

[0032] Finally, the fix module UM n is the previous correction module UM n-1 Corrected image data X output by n-1 , corrected image data X based on the k-space data K0 and mask data n and outputs this as an estimate of the complete image data.

[0033] Below is the correction module UM t The configuration of the correction module UM will be explained using the configuration of the correction module UM as an example. t 4 shows the configuration of the correction module UM according to the first embodiment, and therefore a duplicated description will be omitted here. t 4 is a block diagram for explaining the configuration of Fig. 4. In Fig. 4, the connections between the various components are indicated by dashed arrows.

[0034] Referring to Figure 4, the correction module UM t includes an image space regularization adjustment block IRMB, an image space regularization block IRB, a data consistency adjustment block DCMB, and a data consistency block DCB.

[0035] The image space regularization adjustment block IRMB adjusts the regularization process by the image space regularization block IRB. The image space regularization adjustment block IRMB adjusts the regularization process by the image space regularization adjustment neural network IRMN. t-1Corrected image data X input from t-1 The number of feature adjustment tensors and activation adjustment tensors is determined by the number of convolutional activation units in the image space regularization neural network IRN of the image space regularization block IRB described later. The feature adjustment tensors adjust the output results of the convolutional layers in the convolutional activation units of the image space regularization neural network IRN described later. The activation adjustment tensors adjust the processing of each activation layer in the convolutional activation units of the image space regularization neural network IRN described later. Each parameter used by the image space regularization adjustment neural network IRMN is stored in the storage unit 40. The image space regularization adjustment neural networks IRMN included in different correction modules UM may have different parameters or may share parameters. It is preferable that the image space regularization adjustment neural networks IRMN included in different correction modules UM have different parameters.

[0036] The image space regularization adjustment neural network IRMN includes a subnetwork SN1 and a fully connected layer FCL1. The subnetwork SN1 functions as an image feature extractor and is configured using a feedforward neural network, a convolutional neural network, a Transformer, or the like. The subnetwork SN1 is preferably configured using a convolutional neural network. The fully connected layer FCL1 has multiple nodes connected to all output nodes of the subnetwork SN1 and integrates the image features extracted by the subnetwork SN1.

[0037] Next, the processing by the image space regularization adjustment block IRMB will be described. t 5 is a data flow diagram for explaining the processing by the image space regularization adjustment block IRMB included in Fig. 5. In Fig. 5, the flow of data is indicated by solid arrows.

[0038] Referring to Figure 5, the image space regularization adjustment block IRMB is a pre-modification module UM t-1 Corrected image data X input from t-1 is input to the image space regularization adjustment neural network IRMN, which then calculates the corrected image data X based on the trained neural network. t-1 Analyze and correct image data X t-1 Feature adjustment tensor FI for t、1 ~FI t、3 and the activation adjustment tensor FA t、1 ~FA t、3 Generate.

[0039] Specifically, the image space regularization neural network IRMN calculates the corrected image data X t-1 is input to the sub-network SN1, its image features are extracted and passed to the fully connected layer FCL1, and the image features extracted by the sub-network SN1 are integrated in the fully connected layer FCL1 to generate the feature adjustment tensor FI t、1 ~FI t、3 and activation adjustment tensor FA t、1 ~FA t、3 Each feature adjustment tensor FI t、1 ~FI t、3 The size of each of the activation adjustment tensors FA matches the size of each convolution layer in the convolution activation unit of the image space regularization neural network IRN described later. t、1 ~FA t、3 The size of each of the activation layers in the convolutional activation unit of the image space regularization neural network (IRN) described below corresponds to the size of each activation layer.

[0040] Returning to the explanation of Figure 4, the image space regularization block IRB executes the image space regularization process. The image space regularization block IRB uses the image space regularization neural network IRN to t-1 Corrected image data X input fromt-1 The image space regularization neural network IRN performs regularization processing in image space on the image space regularization neural network IRN. The image space regularization neural network IRN is a convolutional neural network including three convolutional activation units CU1 to CU3, hidden layers IL1 and IL2, and an output layer OL. The output layer OL of the image space regularization neural network IRN outputs two-dimensional image data with a width W and a height H. The hidden layers IL1 and IL2 and the output layer OL may be, for example, pooling layers, batch normalization layers, fully connected layers, activation layers, or combinations thereof. Details of the convolutional activation units CU1 to CU3 will be described later. In this embodiment, for convenience of explanation, the number of convolutional activation units in the image space regularization neural network IRN is three. However, the number of convolutional activation units in the image space regularization neural network IRN is not limited to three and may be any integer equal to or greater than one. Note that, for convenience of explanation, the configuration of the image space regularization neural network IRN is simplified, but the configuration of the image space regularization neural network IRN is not limited to this. The image space regularization neural network IRN is preferably a U-net. Parameters used by the image space regularization neural network IRN are stored in the storage unit 40. The image space regularization neural networks IRN included in different modification modules UM may have different parameters or may share parameters. The image space regularization neural networks IRN included in different modification modules UM preferably have different parameters.

[0041] Next, the processing by the image space regularization block IRB will be described. t 6 is a data flow diagram for explaining the processing by the image space regularization block IRB included in Fig. 6. In Fig. 6, the flow of data is indicated by solid arrows.

[0042] Referring to Figure 6, the image space regularization block IRB is a pre-modification module UM t-1Corrected image data X input from t-1 and the feature adjustment tensor FI input from the image space regularization adjustment neural network IRMN. t、1 ~FI t、3 and the activation adjustment tensor FA t、1 ~FA t、3 and are input to the image space regularization neural network IRN, which then generates the corrected image data X based on the trained neural network. t-1 Regularization is performed in image space to obtain the regularized image data XR t The image space regularization neural network IRN processes the corrected image data X t-1 It is considered that artifact removal and noise removal are performed on the image data.

[0043] Specifically, the image space regularization neural network IRN first calculates the corrected image data X t-1 is input to the convolutional activation unit CU1, which then generates the feature adjustment tensor FI t、1 and activation adjustment tensor FA t、1 Therefore, the corrected image data X t-1 For the corrected image data X t-1 After performing the convolutional activation process adjusted based on the image data, the extracted image feature data is passed to the hidden layer IL1. Details of the convolutional activation process will be described later. The hidden layer IL1 performs the necessary processing on the received image feature data and passes the processed image feature data to the convolutional activation unit CU2. The processing performed by the convolutional activation units CU2 and CU3 and the hidden layer IL2 is the same as that performed by the convolutional activation unit CU1 and the hidden layer IL1, so a duplicated explanation will be omitted here. Finally, the output layer OL converts the image feature data output by the convolutional activation unit CU3 into two-dimensional image data of width W × height H and outputs regularized image data XR. t Output as

[0044] The convolutional activation process performed by the convolutional activation units CU1 to CU3 will be described below using the convolutional activation unit CU2 as an example. The structure of the convolutional activation units CU1 and CU3 and the convolutional activation process they perform are similar to those of the convolutional activation unit CU2, so a duplicated description will be omitted here. Figure 7 is a data flow diagram for explaining the convolutional activation process performed by the convolutional activation unit CU2 included in the image space regularization neural network IRN according to the first embodiment. In Figure 7, the flow of data is indicated by solid arrows.

[0045] 7, the convolution activation unit CU2 includes a convolution layer CL, a multiplier M1, and an activation layer AL. When performing the convolution activation process, the convolution activation unit CU2 first converts the modified image data X received from the previous layer (hidden layer IL1) into t-1 The image features related to are passed to the convolution layer CL to perform convolution processing to calculate higher-level image features. Then, the convolution activation unit CU2 uses the multiplier M1 to multiply the image features output from the convolution layer CL with the feature adjustment tensor FI t、2 By calculating the Hadamard product with t-1 The activation layer AL performs adjustment based on the activation adjustment tensor FA t、2 The adjusted image features are activated using an activation function, such as a Rectified Linear Unit, a Leaky Rectified Linear Unit, a SoftShrink function, or an Exponential Linear Unit. The activation layer AL activates the adjusted image features using an activation adjustment tensor FA t、2 Based on this, we adjust the threshold of the activation function (e.g., SoftShrink function), and calculate the image features activated by the activation function and the activation adjustment tensor FA t、2 By calculating the Hadamard product with t-1 Finally, the convolutional activation unit CU2 passes the activated image features to the next layer (hidden layer IL2).

[0046] The corrected image data X is calculated by calculating the Hadamard product of the image features with the feature adjustment tensor and the activation adjustment tensor. t-1 Based on this information, we can make element-level adjustments to image features, which allows us to adjust the convolutional activation process according to the characteristics of the image.

[0047] Returning to the explanation of Figure 4, the data consistency adjustment block DCMB adjusts the data consistency processing by the data consistency block DCB. The data consistency adjustment block DCMB adjusts the regularized image data XR input from the image space regularization block IRB by the data consistency adjustment neural network DCMN. t The data consistency weight is a scalar quantity for weighting and adjusting the data consistency strength constant of the data consistency block DCB, which will be described later. Each parameter used by the data consistency adjustment neural network DCMN is stored in the storage unit 40. The data consistency adjustment neural networks DCMN included in different modification modules UM may have different parameters or may share parameters. It is preferable that the data consistency adjustment neural networks DCMN included in different modification modules UM have different parameters from each other.

[0048] The data consistency adjustment neural network DCMN includes a subnetwork SN2, a fully connected layer FCL2, and a sigmoid function activation layer SAL. The subnetwork SN2 operates as an image feature extractor and is configured using a feedforward neural network, a convolutional neural network, a Transformer, or the like. The subnetwork SN2 is preferably configured using a convolutional neural network. The fully connected layer FCL2 has multiple nodes connected to all output nodes of the subnetwork SN2 and integrates the image features extracted by the subnetwork SN2. The sigmoid function activation layer SAL converts the tensor data output by the fully connected layer FCL2 into a single scalar quantity.

[0049] Next, the processing by the data consistency adjustment block DCMB will be described. t 8 is a data flow diagram for explaining the processing by the data consistency control block DCMB included in Fig. 8. In Fig. 8, the flow of data is indicated by solid arrows.

[0050] Referring to FIG. 8, the data consistency adjustment block DCMB adjusts the regularized image data XR input from the image space regularization block IRB. t is input to the data consistency adjustment neural network DCMN, which then calculates the regularized image data XR based on the trained neural network. t Analyze and regularize image data XR t The data consistency weight W based on t Generate.

[0051] Specifically, the data consistency adjustment neural network DCMN is a regularized image data XR t is input to the subnetwork SN2, which extracts the image features and passes them to the fully connected layer FCL2. The fully connected layer FCL2 integrates the image features extracted by the subnetwork SN2 and passes the results to the sigmoid function activation layer SAL. Based on the results, the sigmoid function activation layer SAL determines the data consistency weights W used by the data consistency block DCB. t Generate.

[0052] Returning to the explanation of Figure 4, the data consistency block DCB performs data consistency processing. The data consistency block DCB has a data consistency weight W t The regularized image data XR output by the image space regularization block IRB is based on t Data consistency processing is performed so that the k-space data corresponding to the corrected image data X t The data consistency process generates the corrected image data X so that it matches the original scan data, k-space data K0, as closely as possible. tSpecifically, the data consistency block DCB is used to calculate the corrected image data X using the following formula (1): t Calculate.

[0053]

number

[0054] where λ is the data consistency strength and A is the forward operator. The forward operation A(Xt) first calculates the corrected image data X t The forward operator A is calculated by applying a Fourier transform to the corrected image data X t is converted into k-space data corresponding to the k-space data K0, i.e., the coordinates of the frequency encoding and phase encoding where sampling is performed have normal values, and the coordinates of the frequency encoding and phase encoding where sampling is not performed have values ​​of 0. The data consistency strength λ is calculated by the following formula (2).

[0055]

number

[0056] Here, λ0 is a data consistency strength constant, and the data consistency strength constant λ0 may be a preset fixed value or may be a learnable value. t The corrected image data X t-1 The strength of the data consistency process can be adjusted based on this information.

[0057] The data consistency block DCB can solve equation (1) by using an optimization algorithm such as gradient descent or near-end mapping, where the near-end mapping uses the conjugate gradient method.

[0058] Below is the correction module UM t 9 is a flow chart of the process performed by the correction module UM according to the first embodiment. t 9 is a data flow diagram for explaining the processing by . In Fig. 9, the flow of data is indicated by solid arrows.

[0059] Referring to Figure 9, the correction module UM t performs the following processes in order. (1) The image space regularization adjustment block IRMB and the image space regularization block IRB are supplied with the corrected image data X t-1 Enter. (2) The image space regularization adjustment block IRMB receives the corrected image data X t-1 Based on the feature adjustment tensor F I t、1 ~FI t、3 and activation adjustment tensor FA t、1 ~FA t、3 and outputs it to the image space regularization block IRB. (3) The image space regularization block IRB calculates the corrected image data X t-1 , feature adjustment tensor FI t、1 ~FI t、3 and activation adjustment tensor FA t、1 ~FA t、3 Regularized image data based on XR t and outputs it to the data consistency adjustment block DCMB and the data consistency block DCB. (4) The data consistency adjustment block DCMB uses the regularized image data XR t Based on the data consistency weight W t and outputs it to the data consistency block DCB. (5) The data integrity block DCB is the regularized image data XR t and the data consistency weight W t and the k-space data K0, the corrected image data X t Generate.

[0060] The following describes the flow of the magnetic resonance image reconstruction method according to the first embodiment. Fig. 10 is a flowchart showing the flow of the magnetic resonance image reconstruction method according to the first embodiment.

[0061] In step S101, based on the user interface displayed on the display device, the user uses the input device to select undersampled k-space data K0 and mask data stored in the storage unit 40 or input from an external source, and inputs these to the magnetic resonance image reconstruction device 1. When the processing of step S101 is completed, the process proceeds to step S102.

[0062] In step S102, the preprocessing unit 50 generates initial image data X0 based on the undersampled k-space data K0. When the process of step S102 is completed, the process proceeds to step S103.

[0063] In step S103, the reconstruction unit 60 loads the front-stage correction module UM of the reconstruction network RN from the storage unit 40, and inputs the undersampled k-space data K0, mask data, and initial image data X0 to the front-stage correction module UM. When the processing of step S103 is completed, the process proceeds to step S104.

[0064] In step S104, the modification module UM causes the image space regularization adjustment block IRMB to generate a feature adjustment tensor and an activation adjustment tensor based on the input image data. After the process of step S104 is completed, the process proceeds to step S105.

[0065] In step S105, the correction module UM causes the image space regularization block IRB to generate a regularized image based on the input image data, feature adjustment tensor, and activation adjustment tensor. After the process of step S105 is completed, the process proceeds to step S106.

[0066] In step S106, the correction module UM causes the data consistency adjustment block DCMB to generate data consistency weights based on the regularized image. When the process of step S106 is completed, the process proceeds to step S107.

[0067] In step S107, the correction module UM causes the data consistency block DCB to generate corrected image data based on the regularized image, the data consistency weights, and the undersampled k-space data K0. After the process of step S107 is completed, the process proceeds to step S108.

[0068] In step S108, it is determined whether or not there is a next-stage correction module UM. If there is a next-stage correction module UM (determined as YES), the process proceeds to step S109. If there is no next-stage correction module UM (determined as NO), the process proceeds to step S110.

[0069] In step S109, the next-stage correction module UM is loaded, and the undersampled k-space data K0, mask data, and corrected image data are input to the next-stage correction module UM. When the process of step S109 is completed, the process proceeds to step S104.

[0070] In step S110, the reconstructed image data is displayed as complete image data on the display device via the display interface 20. When the process of step S110 is completed, the process of the magnetic resonance image reconstruction method ends.

[0071] In the above description, the magnetic resonance image reconstruction apparatus and magnetic resonance image reconstruction method of this embodiment use the image space regularization adjustment neural network IRMN, the image space regularization neural network IRN, the data consistency adjustment neural network DCMN, and the data consistency strength constant λ0, and these neural networks and parameters require prior training to operate normally. Below, we will explain the training method for the above-mentioned neural networks and parameters.

[0072] First, multiple sets of pre-stored training data are read from the storage unit 40. Each set of training data includes, as input data, undersampled k-space data K0 obtained by scanning with a magnetic resonance scanning device, mask data associated with the data, and true values ​​of complete image data as output data. The training data preferably include image data with different characteristics.

[0073] Then, the multiple sets of teacher data are divided into a training set and a test set. Examples of the ratio between the training set and the test set include 80%, 20%, 90%, and 10%. For example, if the total number of sets of teacher data is 10,000, the teacher data of data #1 to #10,000 is divided into data #1 to #8,000 as the training set and data #8001 to #10,000 as the test set. In such a case, input data in each set of teacher data in the training set is input to the magnetic resonance image reconstruction device 1, the magnetic resonance image reconstruction method of this embodiment is executed to calculate an estimated value of the complete image data, and the difference between the estimated value of the complete image data and the true value of the complete image data is calculated. Backpropagation is performed based on the difference value, and the parameters of each neural network and other learnable parameters are changed so that the difference between the estimated value of the complete image data output by the magnetic resonance image reconstruction device 1 and the true value of the complete image data becomes smaller. The above process is repeated for most of the data in the training set until the difference between the estimated value of the complete image data output by the magnetic resonance image reconstruction device 1 and the true value of the complete image data becomes smaller than a preset threshold, at which point it is determined that the training of each neural network and parameter is complete.

[0074] Then, test data (data #8000 to #10000) is input into the trained magnetic resonance image reconstruction device 1, and the difference between the estimated value of the complete image data output by the magnetic resonance image reconstruction device 1 and the true value of the complete image data is calculated as evaluation data.

[0075] The effects of the magnetic resonance image reconstruction apparatus and magnetic resonance image reconstruction method of this embodiment will be described below.

[0076] In conventional magnetic resonance imaging systems, frequency-domain k-space information is undersampled to generate undersampled k-space data in order to shorten scan times. Reconstructing complete image data using undersampled k-space data is an ill-posed problem, resulting in countless solutions and making it impossible to uniquely identify accurate complete image data. A magnetic resonance image reconstruction system and method based on deep learning technology solve this problem using a compressive sensing algorithm and adds a trained neural network as a sparsity constraint to uniquely identify a single appropriate solution.

[0077] Conventional magnetic resonance image reconstruction devices and magnetic resonance image reconstruction methods using deep learning technology perform similar processing regardless of the characteristics of the input image because the parameters of the neural network and algorithm that perform correction processing, including regularization processing and image consistency processing, are fixed values ​​when reconstructing complete image data using a trained neural network.

[0078] In the present invention, the correction modules UM1 to UM n The complete image data is reconstructed using the above, and each correction module UM further includes an image space regularization adjustment block IRMB and a data consistency adjustment block DCMB that adjust the regularization processing and image consistency processing, in addition to an image space regularization block IRB and a data consistency block DCB that perform regularization processing and image consistency processing.

[0079] The image space regularization adjustment block IRMB generates feature adjustment tensors and activation adjustment tensors based on image data undergoing correction processing, thereby dynamically adjusting the operation of the convolutional layers and activation layers in the image space regularization neural network IRN of the image space regularization block IRB. The data consistency adjustment block DCMB generates data consistency weights based on image data undergoing correction processing, thereby dynamically adjusting the data consistency strength of the data consistency block DCB. The feature adjustment tensors, activation adjustment tensors, and data consistency weights are not fixed values ​​but values ​​corresponding to information on the image data undergoing correction processing, allowing processing to be performed according to the characteristics of the image data. According to the present invention, optimal image reconstruction processing can be performed according to the characteristics of the input data, thereby improving the accuracy of image reconstruction.

[0080] Compared to conventional technology, the MSE between the estimated value of complete image data generated by the magnetic resonance image reconstruction device 1 of this embodiment and the true value of the complete image data is lower than that of the magnetic resonance image reconstruction method based on conventional deep learning technology, and the SSIM and PSNR are higher than that of the magnetic resonance image reconstruction method based on conventional deep learning technology.

[0081] FIG. 11 is a diagram comparing complete image data reconstructed by the magnetic resonance image reconstruction apparatus 1 of the first embodiment with complete image data reconstructed by the prior art.

[0082] Figure 11(a) shows local actual image data of the cerebellum, Figure 11(b) shows local image data of the cerebellum reconstructed using conventional technology when undersampling by 5x, and Figure 11(c) shows local image data of the cerebellum reconstructed by the magnetic resonance image reconstruction device 1 of this embodiment when undersampling by 5x.

[0083] 11, when undersampling is performed by 5x, the image data reconstructed by the magnetic resonance image reconstruction device 1 of this embodiment has a relatively high degree of reproduction of the actual image data, and compared to image data reconstructed by conventional technology, more details are restored, the edges of the reconstructed cerebellar tissue are sharper, and the image clarity is higher. Furthermore, the image data reconstructed by the magnetic resonance image reconstruction device 1 of this embodiment well restores the sulcal structure of the cerebellum.

[0084] FIG. 12 is a diagram comparing complete image data reconstructed by the magnetic resonance image reconstruction apparatus 1 of the first embodiment with complete image data reconstructed by the prior art.

[0085] Figure 12(a) shows actual image data of the knee, Figure 12(b) shows image data of the knee reconstructed using conventional technology with 5x undersampling, and Figure 12(c) shows image data of the knee reconstructed using the magnetic resonance image reconstruction device 1 of this embodiment with 5x undersampling.

[0086] 12, when undersampling is performed by 5x, the image data reconstructed by the magnetic resonance image reconstruction device 1 of this embodiment has a relatively high degree of reproduction of the actual image data, and more details are restored and the image is clearer than image data reconstructed by conventional technology. In addition, the image data reconstructed by the magnetic resonance image reconstruction device 1 of this embodiment well restores the texture structure of the patella.

[0087] (Second embodiment) A magnetic resonance image reconstruction apparatus and a magnetic resonance image reconstruction method according to the second embodiment will be described below, focusing on the differences from the first embodiment and omitting a description of the commonalities with the first embodiment. In the description of the second embodiment, the same parts as those in the first embodiment will be denoted by the same reference numerals.

[0088] The magnetic resonance image reconstruction apparatus of the second embodiment has correction modules UM1-UM n Instead, UMA1-UMA n It has.

[0089] Below is the correction module UMA t The configuration of the correction module UMA will be explained using the configuration of the above as an example. The configuration of other correction module UMAs is t 13 shows the configuration of the correction module UMA according to the second embodiment, and therefore, a duplicated description will be omitted here. t FIG. 2 is a block diagram for explaining the configuration of the

[0090] Correction module UMA t is the correction module UM of the first embodiment. t In comparison with the first embodiment, the data consistency adjustment block DCMB is not included, and the other configuration is the correction module UM t is the same as

[0091] Next, fix module UMA t 14 is a flow chart of the process performed by the correction module UMA according to the second embodiment. t 14 is a data flow diagram for explaining the process by the modification module UMA. t performs the following processes in order.

[0092] (1) The image space regularization adjustment block IRMB and the image space regularization block IRB are supplied with the corrected image data X t-1 Enter. (2) The image space regularization adjustment block IRMB receives the corrected image data X t-1 Based on the feature adjustment tensor F I t、1 ~FI t、3 and activation adjustment tensor FA t、1 ~FA t、3 and outputs it to the image space regularization block IRB. (3) The image space regularization block IRB calculates the corrected image data X t-1, feature adjustment tensor FI t、1 ~FI t、3 and activation adjustment tensor FA t、1 ~FA t、3 Regularized image data based on XR t and outputs it to the data consistency block DCB. (4) The data integrity block DCB is the regularized image data XR t and the k-space data K0, the corrected image data X t Generate.

[0093] 15 is a flowchart showing the flow of the magnetic resonance image reconstruction method according to the second embodiment. Compared to the first embodiment, the magnetic resonance image reconstruction method according to the second embodiment does not include step S106, and includes step S107A instead of step S107.

[0094] In step S107, the correction module UM causes the data consistency block DCB to generate corrected image data based on the regularized image and the undersampled k-space data K0.

[0095] (Third embodiment) A magnetic resonance image reconstruction apparatus and a magnetic resonance image reconstruction method according to the third embodiment will be described below, focusing on the differences from the first embodiment and omitting a description of the commonalities with the first embodiment. In the description of the third embodiment, the same parts as those in the first embodiment will be denoted by the same reference numerals.

[0096] The magnetic resonance image reconstruction apparatus of the third embodiment has correction modules UM1-UM n Instead of UMB1-UMB n It has.

[0097] Below is the correction module UMB t The configuration of the correction module UMB will be explained using the configuration of the correction module UMB. t16 shows the configuration of the correction module UMA according to the third embodiment, and therefore, a duplicated description will be omitted here. t FIG. 2 is a block diagram for explaining the configuration of the

[0098] Correction module UMB t is the correction module UM of the first embodiment. t 1. In comparison with the first embodiment, the image space regularization adjustment block IRMB is not included, and the other components are the same as the correction module UM t is the same as

[0099] Next, fix module UMB t 17 is a flow chart of the process performed by the correction module UMB according to the third embodiment. t 17 is a data flow diagram for explaining the process by the modification module UMB t performs the following processes in order.

[0100] (1) The image space regularization block IRB receives the corrected image data X t-1 Enter. (2) The image space regularization block IRB calculates the corrected image data X t-1 Regularized image data based on XR t and outputs it to the data consistency adjustment block DCMB and the data consistency block DCB. (3) The data consistency adjustment block DCMB uses the regularized image data XR t Based on the data consistency weight W t and outputs it to the data consistency block DCB. (4) The data integrity block DCB is the regularized image data XR t and the data consistency weight W t and the k-space data K0, the corrected image data X t Generate.

[0101] 18 is a flowchart showing the flow of a magnetic resonance image reconstruction method according to the third embodiment. Compared to the first embodiment, the magnetic resonance image reconstruction method according to the third embodiment does not include step S104, and includes step S105B instead of step S105.

[0102] In step S105B, the correction module UM causes the image space regularization block IRB to generate a regularized image based on the input image data. When the process of step S109 is completed, the process proceeds to step S105B.

[0103] (Variation) In the above embodiment, the reconstruction network RN corrects image data in image space and reconstructs complete image data. However, the reconstruction network RN may correct k-space data in k-space and reconstruct complete image data. In this case, each correction module UM included in the reconstruction network RN corrects the k-space data K0 and reconstructs complete image data. Here, the correction module UM may include a k-space regularization adjustment block having a k-space regularization adjustment neural network and a k-space regularization block having a k-space regularization neural network, instead of the image space regularization adjustment block IRMB and the image space regularization block IRB.

[0104] In the above embodiment, the image space regularization adjustment block IRMB generates a feature-adjusted tensor and an activation-adjusted tensor. However, the image space regularization adjustment block IRMB may generate only a feature-adjusted tensor or only an activation-adjusted tensor. In this case, the image space regularization block IRB performs image space regularization processing based on only either the feature-adjusted tensor or the activation-adjusted tensor.

[0105] In the above embodiment, the corrected image data X tAs an example, we have shown that the regularized image data XR is output as an estimate of the complete image data. t may be output as an estimate of the complete image data. In other words, in a plurality of iterations of the process, the process using the data consistency block DCB may not be performed in the final iteration.

[0106] With respect to the above embodiment, the following supplementary notes are disclosed as one aspect and optional features of the invention.

[0107] (Appendix 1) One aspect of the present invention provides a magnetic resonance image reconstruction device that reconstructs magnetic resonance image data based on undersampled k-space data, with artifacts caused by undersampling removed or reduced, and has a reconstruction unit that reconstructs the magnetic resonance image data using a reconstruction network having a correction module, and the correction module includes a regularization block that uses a first neural network to perform regularization processing on first image data and generate second image data, and a data consistency block that performs data consistency processing so that k-space data corresponding to the second image data approaches the undersampled k-space data and generates third image data, and the correction module further includes at least one of a data consistency adjustment block that adjusts the data consistency processing and a regularization adjustment block that adjusts the regularization processing.

[0108] (Appendix 2) The regularization block may perform the regularization process in image space using the first neural network so that the first image data approximates magnetic resonance image data with artifacts due to undersampling removed or reduced.

[0109] (Appendix 3) The data consistency adjustment block may generate a data consistency weight using a second neural network based on the second image data, and the data consistency block may perform the data consistency processing based on the data consistency weight.

[0110] (Appendix 4) The regularization adjustment block may generate a first tensor using a third neural network based on the first image data, and the regularization block may adjust the calculation of a convolutional layer in the first neural network based on the first tensor.

[0111] (Appendix 5) The regularization adjustment block may generate a second tensor using a fourth neural network based on the first image data, and the regularization block may adjust the calculation of an activation layer in the first neural network based on the second tensor.

[0112] (Appendix 6) The regularization adjustment block may include a first image feature extractor that extracts image features in the first image data and generates the first tensor based on the image features.

[0113] (Appendix 7) The regularization adjustment block may include a first image feature extractor that extracts image features in the first image data and generates the second tensor based on the image features.

[0114] (Appendix 8) The data consistency adjustment block may include a second image feature extractor for extracting image features in the second image data, and may generate the data consistency weights based on the image features.

[0115] (Appendix 9) The third neural network may be a convolutional neural network.

[0116] (Appendix 10) The reconstruction network may include a plurality of the correction modules, and the first neural network of each of the plurality of correction modules may have different parameters.

[0117] (Appendix 11) The activation function of the activation layer may be a ReLU function.

[0118] (Appendix 12) The regularization block may use the first neural network to perform the regularization process in k-space to generate the second image data so that k-space data corresponding to the first image data approaches fully sampled k-space data.

[0119] (Appendix 13) A magnetic resonance image reconstruction method provided in one aspect of the present invention is a magnetic resonance image reconstruction method for reconstructing magnetic resonance image data based on undersampled k-space data, in which artifacts caused by undersampling have been removed or reduced, and includes a regularization step for performing a regularization process on first image data using a first neural network to generate second image data, a data consistency step for performing a data consistency process to generate third image data so that k-space data corresponding to the second image data approaches the undersampled k-space data, and an adjustment step for adjusting at least one of the regularization process and the data consistency process.

[0120] According to at least one of the embodiments described above, the image quality of the reconstructed image can be improved.

[0121] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0122] 1. Magnetic resonance image reconstruction device 10 Input / Output Interface 20 Display Interface 30 Communication Interface 40 Storage section 50 Pretreatment section 60 Reconstruction section

Claims

1. A magnetic resonance image reconstruction device that reconstructs magnetic resonance image data in which artifacts caused by undersampling are removed or reduced based on undersampled k-space data, comprising: a reconstruction unit for reconstructing the magnetic resonance image data using a reconstruction network having a correction module; The correction module: a regularization block that performs a regularization process on the first image data using a first neural network to generate second image data; a data consistency block that performs data consistency processing to generate third image data so that k-space data corresponding to the second image data is closer to the undersampled k-space data; Including, The magnetic resonance image reconstruction apparatus, wherein the correction module further includes at least one of a data consistency adjustment block that adjusts the data consistency processing and a regularization adjustment block that adjusts the regularization processing.

2. 2. The magnetic resonance image reconstruction device of claim 1, wherein the regularization block performs the regularization processing in image space using the first neural network so that the first image data approaches magnetic resonance image data in which artifacts due to undersampling have been removed or reduced.

3. The data consistency adjustment block generates a data consistency weight based on the second image data using a second neural network; The magnetic resonance image reconstruction apparatus according to claim 1 , wherein the data consistency block performs the data consistency processing based on the data consistency weights.

4. The regularization adjustment block generates a first tensor using a third neural network based on the first image data; The magnetic resonance image reconstruction device of claim 1 , wherein the regularization block adjusts the calculation of a convolutional layer in the first neural network based on the first tensor.

5. The regularization adjustment block generates a second tensor using a fourth neural network based on the first image data; The magnetic resonance image reconstruction apparatus of claim 1 , wherein the regularization block adjusts the calculation of an activation layer in the first neural network based on the second tensor.

6. The magnetic resonance image reconstruction device according to claim 4 , wherein the regularization adjustment block includes a first image feature extractor that extracts image features in the first image data and generates the first tensor based on the extracted image features.

7. The magnetic resonance image reconstruction device according to claim 5 , wherein the regularization adjustment block includes a first image feature extractor that extracts image features in the first image data and generates the second tensor based on the extracted image features.

8. 4. The magnetic resonance image reconstruction device according to claim 3, wherein the data consistency adjustment block includes a second image feature extractor that extracts image features in the second image data and generates the data consistency weights based on the corresponding image features.

9. 6. The magnetic resonance image reconstruction apparatus according to claim 4, wherein the third neural network is a convolutional neural network.

10. the reconstruction network has a plurality of the modification modules; 9. The magnetic resonance image reconstruction apparatus according to claim 1, wherein the first neural network of each of the plurality of correction modules has different parameters.

11. 8. The magnetic resonance image reconstruction apparatus according to claim 5, wherein the activation function of the activation layer is a ReLU function.

12. 2. The magnetic resonance image reconstruction device of claim 1, wherein the regularization block uses the first neural network to perform the regularization process in k-space to generate the second image data so that k-space data corresponding to the first image data approaches fully sampled k-space data.

13. A magnetic resonance image reconstruction method for reconstructing magnetic resonance image data in which artifacts due to undersampling are removed or reduced based on undersampled k-space data, comprising: a regularization step of performing a regularization process on the first image data using a first neural network to generate second image data; a data consistency step of performing data consistency processing to generate third image data so that k-space data corresponding to the second image data is closer to the undersampled k-space data; an adjusting step of adjusting at least one of the regularization process and the data consistency process.

Images