Image processing apparatus and image processing method

Abstract

The image processing apparatus includes: an acquisition unit that acquires K-space data obtained by undersampling using a plurality of coils, the K-space data corresponding to each of a plurality of frames; a classification unit that classifies the auto-calibration signal data in the K-space data corresponding to each of the plurality of frames into a plurality of groups; and a sensitivity profile calculation unit that generates a sensitivity profile corresponding to each of the plurality of coils on the basis of data based on the auto-calibration signal data classified into the first group among the plurality of groups and data based on the auto-calibration signal data classified into the second group among the plurality of groups.

Description

Technical Field

[0001] This invention relates to an image processing apparatus and an image processing method. Background Technology

[0002] Magnetic resonance imaging (MRI) is a non-invasive medical imaging technique that utilizes the magnetic resonance phenomenon—the resonance between hydrogen nuclei placed in a static magnetic field and a high-frequency magnetic field of a specific frequency. Dynamic magnetic resonance imaging (DRAM) is a type of MRI technique that acquires magnetic resonance images in a time-series manner, allowing for the observation and analysis of dynamic changes in organs or tissues over a specific period.

[0003] In dynamic magnetic resonance imaging, multiple K-space frame data that are continuous in time are obtained by transmitting pulse signals to the subject (patient) in a frequency-coded and phase-coded magnetic field and receiving echo signals generated by specific atomic nuclear magnetic resonance from multiple receiving coils. The magnetic resonance image time series data is then reconstructed based on the K-space time series data composed of these multiple K-space frame data.

[0004] However, obtaining complete K-space frame data requires a long magnetic resonance imaging (MRI) scan. In this case, the reconstructed MRI time series data, containing multiple MRI image frames, becomes discontinuous in time (the intervals increase), failing to accurately represent the changes of the scanned object over time. Therefore, in MRI, partial K-space frame data is typically obtained through undersampling. The MRI time series data is then reconstructed based on the K-space time series data composed of the undersampled K-space frame data, thereby shortening the MRI scan time.

[0005] When reconstructing magnetic resonance image time series data based on undersampled K-space time series data, the true value of the magnetic resonance image time series data cannot be obtained due to insufficient information for reconstruction; only an estimate of the true value can be made.

[0006] Previously, it was known to use an end-to-end unrolled reconstruction network to reconstruct magnetic resonance image time series data from undersampled K-space time series data. This method requires obtaining the sensitivity distribution maps of multiple receiving coils that receive echo signals, and then reconstructing the magnetic resonance image time series data based on these sensitivity distribution maps.

[0007] Non-Patent Document 1 discloses an image processing method for magnetic resonance image reconstruction by a neural network that calculates a sensitivity profile of a plurality of receive coils, an end-to-end unfolded reconstruction network. In the image processing method of Non-Patent Document 1, a convolutional neural network is used to calculate a sensitivity profile based on an auto-calibration signal. Non-Patent Document 2 discloses an image processing method for magnetic resonance image reconstruction by a neural network that calculates a sensitivity profile of a plurality of receive coils, an end-to-end unfolded reconstruction network. In the image processing method of Non-Patent Document 2, a 2D U-Net model is used to refine a sensitivity profile generated based on an auto-calibration signal.

[0008] Prior Art Documents

[0009] Non-Patent Documents

[0010] Non-Patent Document 1: k-t CLAIR: Self-Consistency Guided Multi-Prior Learning for Dynamic Parallel MR Image Reconstruction, Liping Zhang, & Weitian Chen. (2024). arXiv: 2310.11050

[0011] Non-Patent Document 2: Deep Cardiac MRI Reconstruction with ADMM, George Yiasemis, Nikita Moriakov, Jan-Jakob Sonke, & Jonas Teuwen. (2023). arXiv: 2310.06628 SUMMARY

[0012] Problems to be Solved by the Invention

[0013] In scanning of dynamic magnetic resonance imaging, the scanning environment can change over time, and the scan object can also move over time. For example, in scanning of the heart, the heart contracts and relaxes with a cardiac cycle. When scanning a scan object that moves like this, the auto-calibration signal in each K-space frame data obtained by the scan can become discontinuous.

[0014] In the existing image processing method for magnetic resonance image reconstruction, there is a problem that the stability and accuracy of the reconstruction of the magnetic resonance image will decrease in the case of discontinuity of the automatic calibration signal. The image processing method for magnetic resonance image reconstruction of Non-Patent Literature 1 and Non-Patent Literature 2 uses a reconstruction network to perform reconstruction of the magnetic resonance image based on the automatic calibration signal of any one of the plurality of K-space frame data or the average of the automatic calibration signals of the plurality of K-space frame data. If the automatic calibration signal becomes discontinuous, the magnetic resonance image reconstructed by the image processing method for magnetic resonance image reconstruction of Non-Patent Literature 1 and Non-Patent Literature 2 can produce obvious artifacts, and the stability and accuracy of the reconstruction decrease.

[0015] Figure 8 is a diagram showing the problem of the existing image processing method for magnetic resonance image reconstruction. In Figure 8 , Prior Art 1 is a technology that uses a reconstruction network to perform reconstruction of the magnetic resonance image based on the average of the automatic calibration signals of the plurality of K-space frame data, and Prior Art 2 is a technology that uses a reconstruction network to perform reconstruction of the magnetic resonance image based on the automatic calibration signal of any one of the plurality of K-space frame data. In addition, in Figure 8 , magnetic resonance images 1-2 are respectively reconstructed based on the K-space frame data contained in different undersampled K-space time series data.

[0016] As shown in Figure 8 , in the case where there is no obvious discontinuity between the automatic calibration signals, there is no obvious artifact in the magnetic resonance images reconstructed using Prior Art 1 and Prior Art 2, and they are basically consistent with the true value of the magnetic resonance image. On the other hand, in the case where there is obvious discontinuity between the automatic calibration signals, there are different degrees of artifacts in the magnetic resonance images reconstructed using Prior Art 1 and Prior Art 2.

[0017] Therefore, it is desirable to propose an image processing method for magnetic resonance image reconstruction and an image processing device for magnetic resonance image reconstruction that can suppress the decrease in the stability and accuracy of the reconstruction of the magnetic resonance image caused by the discontinuity of the automatic calibration signal.

[0018] Means for solving the problem

[0019] The image processing apparatus of the embodiment has: an acquisition unit that acquires K-space data obtained by undersampling using a plurality of coils, the K-space data corresponding to a plurality of frames respectively; a classification unit that classifies auto-calibration signal data in the K-space data corresponding to the plurality of frames respectively into a plurality of groups; a sensitivity profile calculation unit that generates sensitivity profiles corresponding to the plurality of coils respectively, based on data based on auto-calibration signal data classified into a first group among the plurality of groups and data based on auto-calibration signal data classified into a second group among the plurality of groups; an image generation unit that generates an image corresponding to a first frame among the plurality of frames based on the sensitivity profiles calculated by the sensitivity profile calculation unit and the K-space data corresponding to the first frame, and generates an image corresponding to a second frame among the plurality of frames based on the sensitivity profiles calculated by the sensitivity profile calculation unit and the K-space data corresponding to the second frame; a regularization unit that performs image space regularization processing on the image corresponding to the first frame and performs image space regularization processing on the image corresponding to the second frame; and a data consistency processing unit that performs data consistency processing on the image corresponding to the first frame to which the image space regularization processing is applied, based on the K-space data corresponding to the first frame acquired by the acquisition unit and the sensitivity profiles, and performs data consistency processing on the image corresponding to the second frame to which the image space regularization processing is applied, based on the K-space data corresponding to the second frame acquired by the acquisition unit and the sensitivity profiles.

[0020] The image processing method of the embodiment includes: a step of acquiring K-space data obtained by undersampling using a plurality of coils, the K-space data corresponding to a plurality of frames, respectively; a step of classifying auto-calibration signal data in the K-space data corresponding to the plurality of frames, respectively, into a plurality of groups; a step of generating a sensitivity profile corresponding to each of the plurality of coils based on data of auto-calibration signal data classified into a first group among the plurality of groups and data of auto-calibration signal data classified into a second group among the plurality of groups; a step of generating an image corresponding to a first frame among the plurality of frames based on the sensitivity profile calculated in the step of calculating a sensitivity profile and the K-space data corresponding to the first frame, and generating an image corresponding to a second frame among the plurality of frames based on the sensitivity profile calculated in the step of calculating a sensitivity profile and the K-space data corresponding to the second frame; a step of performing image space regularization processing on the image corresponding to the first frame and performing image space regularization processing on the image corresponding to the second frame; and a step of performing data consistency processing on the image corresponding to the first frame to which the image space regularization processing is applied based on the K-space data corresponding to the first frame acquired in the step of acquiring and the sensitivity profile, and performing data consistency processing on the image corresponding to the second frame to which the image space regularization processing is applied based on the K-space data corresponding to the second frame acquired in the step of acquiring and the sensitivity profile.

[0021] Effects of Invention

[0022] According to the present application, it is possible to provide an image processing method for magnetic resonance image reconstruction and an image processing device for magnetic resonance image reconstruction that can suppress a decrease in stability and precision of reconstruction of a magnetic resonance image due to discontinuity of an auto-calibration signal. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a diagram showing an example of the structure of a magnetic resonance image reconstruction device of the first embodiment.

[0024] Figure 2 is a flowchart showing the flow of a magnetic resonance image reconstruction method of the first embodiment

[0025] Figure 3 is a diagram showing an example of the structure of K-space time series data and K-space frame data.

[0026] Figure 4 is a dataflow diagram for explaining the processing of steps S102 to S106 of the magnetic resonance image reconstruction method of the first embodiment.

[0027] Figure 5 is a dataflow diagram for explaining the process of step S107 of the magnetic resonance image reconstruction method of the first embodiment.

[0028] Figure 6 is a dataflow diagram for explaining the process of steps S109-S110 of the magnetic resonance image reconstruction method of the first embodiment.

[0029] Figure 7 is a graph for comparing the magnetic resonance image reconstructed by the magnetic resonance image reconstruction method of the present application with the magnetic resonance image reconstructed by the prior art.

[0030] Figure 8 is a graph showing the problem of the prior art magnetic resonance image reconstruction method. DETAILED DESCRIPTION

[0031] Hereinafter, the magnetic resonance image reconstruction apparatus and the magnetic resonance image reconstruction method of the present application will be described with reference to the accompanying drawings.

[0032] The magnetic resonance image reconstruction apparatus of the present embodiment is an image processing apparatus that reconstructs magnetic resonance image time series data of an image space based on K-space time series data obtained by using a plurality of coils for under-sampling. The purpose is to estimate the true value of the magnetic resonance image time series data corresponding to the completely sampled K-space time series data, thereby reconstructing magnetic resonance image time series data close to the true value of the magnetic resonance image time series data.

[0033] Figure 1 is a graph showing an example of the structure of the magnetic resonance image reconstruction apparatus 1 of the embodiment of the present application. Referring to Figure 1 The structure of the magnetic resonance image reconstruction apparatus 1 of the embodiment of the present application will be described. The magnetic resonance image reconstruction apparatus 1 of the embodiment of the present application has an input / output interface 10, a display interface 20, a communication interface 30, a storage section 40, an acquisition section 50, a sensitivity profile calculation section 60, a preprocessing section 70, an image space regularization section 80, a data consistency processing section 90, and an output section 100. The input / output interface 10, the display interface 20, the communication interface 30, the storage section 40, the acquisition section 50, the sensitivity profile calculation section 60, the preprocessing section 70, the image space regularization section 80, the data consistency processing section 90, and the output section 100 are communicably connected to each other.

[0034] The input / output interface 10 connects the magnetic resonance image reconstructing apparatus 1 and an input device not shown, receives an input operation from a user from the input device, and transmits a signal based on the received input operation to the magnetic resonance image reconstructing apparatus 1. The input / output interface 10 is, for example, a serial bus interface such as USB. The input device includes a mouse and a keyboard, a trackball, a switch, a button, a lever, a touch panel, a microphone, and the like. In addition, the input / output interface 10 can also be connected to a storage device and perform reading and writing of various data with the storage device. The storage device is, for example, a hard disk (HDD), a solid state drive (SSD), or the like.

[0035] The display interface 20 connects the magnetic resonance image reconstructing apparatus 1 and a display device not shown, transmits data to the display device, and causes the display device to display an image. The display interface 20 is, for example, a video output interface such as Digital Visual Interface (DVI) or High-Definition Multimedia Interface (HDMI (registered trademark)). The display device includes an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence) display, and the like. The display device displays a user interface for receiving an input operation from a user, magnetic resonance image data output from the magnetic resonance image reconstructing apparatus 1, and the like, and the user interface is, for example, a GUI (Graphical User Interface) or the like.

[0036] The communication interface 30 connects the magnetic resonance image reconstructing apparatus 1 and a server not shown and can transmit and receive various data with the server. The communication interface 30 is, for example, a network card such as a wireless network card or a wired network card.

[0037] The storage section 40 stores data such as image data, K-space data, and the like used for image reconstruction. Further, the storage section 40 stores parameters used when the magnetic resonance image reconstruction apparatus 1 performs image reconstruction, such as parameters of a neural network and the like. Further, the storage section 40 stores teacher data used for training each neural network and other learnable parameters used by the magnetic resonance image reconstruction apparatus 1. The storage section 40 is realized by, for example, a storage device such as a ROM, a flash memory, a RAM (Random Access Memory), a HDD (Hard Disc Drive), a SSD (Solid State Drive), a register, or the like. The flash memory, the HDD, the SSD, and the like are non-volatile storage media. These non-volatile storage media can be realized by other storage devices connected via a network such as a NAS (Network Attached Storage) or an external storage server apparatus. Among them, the above network includes, for example, the Internet, a WAN (Wide Area Network), a LAN (Local Area Network), a carrier terminal, a wireless communication network, a wireless base station, a dedicated line, and the like.

[0038] The acquisition section 50 acquires K-space time series data obtained by using a plurality of coils to perform under-sampling.

[0039] The under-sampled K-space time series data is constituted by a plurality of under-sampled K-space frame data that are continuous in time. The under-sampled K-space frame data indicates K-space data acquired by each reception coil at a specific timing in a scan of dynamic magnetic resonance imaging.

[0040] The sensitivity profile calculation section 60 generates sensitivity profile time series data indicating sensitivities of a plurality of reception coils used in a dynamic magnetic resonance scan, based on the under-sampled K-space time series data.

[0041] The sensitivity profile time series data is constituted by a plurality of sensitivity profile frame data that are continuous in time. The sensitivity profile frame data indicates a sensitivity of each reception coil for each scan position in a scan range at a specific timing in a scan of dynamic magnetic resonance imaging.

[0042] The sensitivity profile calculation section 60 has an automatic calibration signal extraction unit 61, a grouping unit 62, an inverse Fourier transform unit 63, a plurality of first neural networks 64-1 to 64-g, and a sensitivity profile generation unit 65.

[0043] The automatic calibration signal extraction unit 61 extracts an automatic calibration signal time series data from the K-space time series data. The automatic calibration signal is used for calibrating a phase or amplitude error of K-space data caused by magnetic field inhomogeneity, coil characteristics, and the like. The automatic calibration signal is data located near the center position of the K-space.

[0044] The grouping unit 62 groups a plurality of automatic calibration signal frame data included in the automatic calibration signal time series data, and generates automatic calibration signal group data in the same number as a predetermined group number g (g is an integer of 2 or more).

[0045] The inverse Fourier transform unit 63 performs inverse Fourier transform on the automatic calibration signal group data by an algorithm such as fast inverse Fourier transform, and generates automatic calibration signal image group data.

[0046] The first neural networks 64-1 to 64-g respectively perform image processing on the automatic calibration signal image group data, and generate sensitivity calculation image group data used for calculating the sensitivity profile time series data. The plurality of first neural networks 64-1 to 64-g are, for example, a convolutional neural network or a Transformer. It is preferable that the plurality of first neural networks 64-1 to 64-g be a convolutional neural network. It is more preferable that the plurality of first neural networks 64-1 to 64-g be a U-Net. In the present embodiment, it is assumed that the plurality of first neural networks 64-1 to 64-g are convolutional neural networks including an input layer, an output layer, a convolutional layer, an activation layer, a pooling layer, a batch normalization layer, a fully connected layer, and the size of the input layer and the output layer is equal. The plurality of first neural networks 64-1 to 64-g realize the function of image processing by loading the neural network parameters dedicated to the plurality of first neural networks 64-1 to 64-g stored in the storage section 40. The parameters of the plurality of first neural networks 64-1 to 64-g can be shared or not shared, and it is preferable that the plurality of first neural networks 64-1 to 64-g share the parameters. In addition, the number of the first neural networks is equal to the number of the automatic calibration signal group data generated by the grouping unit 62.

[0047] The sensitivity profile generation unit 65 calculates the sensitivity profile time series data based on the sensitivity calculation image group data.

[0048] The preprocessing section 70 generates an initial image time series data by preprocessing the undersampled K-space time series data based on the sensitivity profile time series data. The initial image time series data is time series data of an image space corresponding to the undersampled K-space time series data.

[0049] The preprocessing section 70 has an inverse Fourier transform unit 71 and a channel merging unit 72. The inverse Fourier transform unit 71 performs inverse Fourier transform on the K-space time series data by an algorithm such as fast inverse Fourier transform. The channel merging unit 72 merges the multi-channel data of the image time series data respectively corresponding to each of the reception coils into single-channel data based on the sensitivity profile time series data.

[0050] The image space regularization section 80 generates, as the regularized image time series data that is the image time series data that has undergone the regularization processing, regularized image time series data by performing image space regularization processing on the input image time series data using the second neural network 81 in the image space. The image space regularization section 80 has the second neural network 81. The second neural network 81 is, for example, a feedforward neural network, a convolutional neural network, or a Transformer, or the like. It is preferable that the second neural network 81 be a convolutional neural network. It is more preferable that the second neural network 81 be a U-Net. In the present embodiment, it is assumed that the second neural network 81 is a convolutional neural network that includes an input layer, an output layer, a convolutional layer, an activation layer, a pooling layer, a batch normalization layer, a fully connected layer, and the size of the input layer and the output layer is equal. The second neural network 81 realizes the function of the image space regularization processing by loading the neural network parameters specific to the second neural network 81 stored in the storage section 40. By loading different neural network parameters, the second neural network 81 can perform different regularization processing.

[0051] The data consistency processing section 90 generates the corrected image time series data by performing data consistency processing on the regularized image time series data in such a manner that the K-space time series data corresponding to the regularized image time series data approaches the undersampled K-space time series data based on the undersampled K-space time series data and the sensitivity profile time series data.

[0052] The output section 100 outputs, as the reconstructed magnetic resonance image time series data, the image time series data based on the corrected image time series data.

[0053] Figure 2 is a flowchart showing the flow of the magnetic resonance image reconstruction method of the embodiment of the present application. The following describes the flow of the magnetic resonance image reconstruction method of the embodiment of the present application with reference to Figure 2 The flow of the magnetic resonance image reconstruction method of the embodiment of the present application is described.

[0054] The magnetic resonance image reconstruction method of the present embodiment is an image processing method that reconstructs magnetic resonance image time series data of an image space based on K-space time series data obtained by undersampling using a plurality of coils.

[0055] First, the processing proceeds to step S101.

[0056] In step S101, the acquisition unit 50 acquires the undersampled K-space time series data K0 and the scan mask time series data M corresponding thereto stored in the storage unit 40 or input from an external storage device, and inputs them to the magnetic resonance image reconstruction device 1.

[0057] The K-space time series data K0 includes frame number f pieces of K-space frame data F1 to Ff. f The frame number f indicates the number of K-space frame data acquired in the dynamic magnetic resonance scan. The K-space frame data F1 to Ff f indicate K-space data acquired at a plurality of continuous timings.

[0058] Figure 3 is a diagram showing a structure example of the K-space time series data K0 and the K-space frame data F1 to Ff f . The K-space time series data K0 and the K-space frame data F1 to Ff f of the present embodiment will be described with reference to Figure 3 . As shown in Figure 3 , in the present embodiment, the K-space time series data K0 is set as four-dimensional tensor data of width w x height h x channel number (number of reception coils) c x frame number f. The K-space time series data K0 includes frame number f pieces of K-space frame data F1 to Ff f as three-dimensional tensor data of width w x height h x channel number c. The K-space frame data F1 to Ff f each include channel number c pieces of two-dimensional matrix data each indicating K-space data received by each reception coil.

[0059] Here, the width direction of the matrix is the phase encoding direction, and the height direction is the frequency encoding direction. The dynamic magnetic resonance scan performs undersampling in which a specific frequency encoding or phase encoding is omitted in order to reduce the scan time, and thus in the magnetic resonance scan, scanning is performed while skipping positions corresponding to the specific frequency encoding and phase encoding. As a result thereof, in the K-space time series data K0, there is no data at positions corresponding to a part of the frequency encoding and phase encoding, and the data at the part of the positions is subjected to zero padding. Since data near the center position of the K-space data has a large influence on the contrast of the reconstructed image data, when undersampling is performed, data near the center position in the frequency encoding direction and the phase encoding direction is generally concentratedly sampled, and data at a part of positions far from the center position is skipped.

[0060] The scan mask time series data M indicates which positions in the K-space time series data K0 are sampled and which positions are omitted in the magnetic resonance scan. The scan mask time series data M is a four-dimensional tensor having the same size as the K-space time series data K0. The scan mask time series data M contains channel number c x frame number f number of two-dimensional matrix data of width w x height h which are identical. In these two-dimensional matrix data, the value of the position where the frequency encoding and the phase encoding are sampled is set to 1, and the value of the position where the frequency encoding and the phase encoding are not sampled is set to 0.

[0061] After the process of step S101 is completed, the processing proceeds to step S102.

[0062] In steps S102 to S106 (one example of a sensitivity profile calculation step), the sensitivity profile calculation section 60 generates the sensitivity profile time series data S indicating the sensitivity of the plurality of reception coils used in the dynamic magnetic resonance scan, based on the undersampled K-space time series data K0.

[0063] Figure 4 is a data flow diagram for explaining the process of steps S102 to S106 of the magnetic resonance image reconstruction method of the embodiment of the present application. In Figure 4 , the flow of data is indicated by solid arrows. Hereinafter, the process of steps S102 to S106 will be explained with reference to Figure 4 .

[0064] In step S102, the auto-calibration signal extraction unit 61 of the sensitivity profile calculation section 60 extracts the auto-calibration signal time series data Q from the K-space time series data K0.

[0065] Specifically, the auto-calibration signal extraction unit 61 calculates the Hadamard product (corresponding element multiplication) of the K-space time series data K0 and the auto-calibration signal mask time series data V read out from the storage section 40, thereby generating the auto-calibration signal time series data Q. The auto-calibration signal time series data Q and the auto-calibration signal mask time series data V are both four-dimensional tensors having the same size as the K-space time series data K0. The auto-calibration signal mask time series data V contains channel number c x frame number f number of two-dimensional matrix data of width w x height h which are identical, in which the value of the position in the region where the auto-calibration signal is present is set to 1, and the value of the remaining position is set to 0. The auto-calibration signal time series data Q contains channel number c x frame number f number of two-dimensional matrix data of width w x height h, each of which indicates the auto-calibration signal of each coil at each timing.

[0066] After the process of step S102 is completed, the processing proceeds to step S103.

[0067] In step S103, the grouping unit 62 of the sensitivity distribution map calculation unit 60 groups the automatic calibration signal frame data based on the similarity between the number f automatic calibration signal frame data contained in the automatic calibration signal time series data Q, generating an automatic calibration signal group data G1 to G2 that is the same as the predetermined group number g. g .

[0068] Specifically, firstly, grouping unit 62 clusters the automatic calibration signal frame data of number f based on the similarity between them, dividing the automatic calibration signal frame data into a predetermined number of groups g. Each group contains a number r of automatic calibration signal frame data within the group, where the number of frames within the group r = the number of frames f / the number of groups g.

[0069] Then, grouping unit 62 concatenates r autocalibration signal frame data within each group along the frame dimension to generate autocalibration signal group data G1 to G2. g .

[0070] Grouping unit 62 can group data based on at least one of the differences in pixel values ​​between automatically calibrated signal frame data, Frobenius norm, cosine similarity, and Euclidean distance.

[0071] After the processing in step S103 is completed, proceed to step S104.

[0072] In step S104, the inverse Fourier transform unit 63 of the sensitivity distribution calculation unit 60 processes the automatic calibration signal group data G1 to G... g The automatic calibration signal image group data H1~H is generated by performing inverse Fourier transform. g Automatic calibration signal image group data H1~H g It is the size and automatic calibration signal group data G1~G g The same four-dimensional tensor. Automatically calibrated signal-image group data H1~H g Two-dimensional image data containing two-dimensional matrix data of width w × height h, with the number of channels c × the number of frames in the group r, implicitly contains information about magnetic field inhomogeneity, coil sensitivity, etc. in each two-dimensional image data.

[0073] After the processing in step S104 is completed, proceed to step S105.

[0074] In step S105, the sensitivity distribution map calculation unit 60 processes the data H1 to H2 of each automatic calibration signal image group using the first neural network 64-1 to 64-g. g Image processing is performed to generate image set data L1 to L2 for sensitivity calculation of the time series data S used to calculate the sensitivity distribution map. g .

[0075] sensitivity calculation image group data L1~L g Each of the sensitivity calculation image group data L1~L

[0076] Specifically, first, the sensitivity profile calculation section 60 reads the neural network parameters of the first neural networks 64-1~64-g from the storage section 40 and causes the first neural networks 64-1~64-g to load the parameters. Then, the sensitivity profile calculation section 60 reads the auto-calibration signal image group data H1~H g from the storage section 40 and inputs the auto-calibration signal image group data H1~H g to the first neural networks 64-1~64-g, respectively, which have loaded the parameters. Then, the first neural networks 64-1~64-g perform forward propagation, respectively, according to the respective auto-calibration signal image group data H1~H g and calculate the sensitivity calculation image group data L1~L g as output data of the second neural network 81. The sensitivity calculation image group data L1~L g have the same size as the auto-calibration signal image group data H1~H g .

[0077] After the processing of step S105 is completed, the process proceeds to step S106.

[0078] In step S106, the sensitivity profile generation unit 65 of the sensitivity profile calculation section 60 calculates the sensitivity profile time series data S on the basis of the sensitivity calculation image group data L1~L g .

[0079] Specifically, first, the sensitivity profile generation unit 65 of the sensitivity profile calculation section 60 integrates the total number f of sensitivity calculation image frame data included in the respective sensitivity calculation image group data L1~L g and generates sensitivity calculation image time series data. In this integration, the total number f of sensitivity calculation image frame data included in the sensitivity calculation image group data L1~L g is reordered in such a manner that the order of the frame data included in the sensitivity calculation image time series data is the same as the order of the frame data in the K-space time series data K0, and the total number f of sensitivity calculation image frame data is spliced in the frame dimension to generate the sensitivity calculation image time series data.

[0080] Then, the sensitivity profile generation unit 65 divides each pixel value in the f sensitivity calculation image frame data included in the sensitivity calculation image time series data by the square root of the square sum of the pixel values of the channel number c (including the pixel itself) having the same position as the pixel in the width direction and the height direction for each of the f sensitivity calculation image frame data. Thus, the sensitivity profile time series data S is calculated.

[0081] After the processing of Step S106 is completed, the process proceeds to Step S107.

[0082] In Steps S102 to S106, by adjusting the size of the group number g, the model size of the first neural network and the strength of data sharing between the frame data can be adjusted. In the case where the group number g is set small, the number of the frame number r within the group is large, the strength of data sharing between the frame data is large, and the model size of the first neural network is large. Thus, the accuracy of the calculated sensitivity profile time series data S can be improved, but the number of parameters of the model increases, and the calculation amount and the training difficulty of the neural network increase. On the other hand, in the case where the group number g is set large, the number of the frame number r within the group is small, the strength of data sharing between the frame data is small, and the model size of the first neural network is small. Thus, the number of parameters of the model can be reduced, and the calculation amount and the training difficulty of the neural network can be reduced, but the accuracy of the calculated sensitivity profile time series data S can be reduced accordingly.

[0083] In Step S107, the pre-processing unit 70 generates the initial image time series data X0 from the undersampled K-space time series data K0. Figure 5 is a data flow diagram for explaining the processing of Step S107 of the magnetic resonance image reconstruction method of the embodiment of the present application. In Figure 5 In, the flow direction of data is indicated by solid arrows. The processing of Step S107 will be explained below with reference to Figure 5 The processing of Step S107 will be explained.

[0084] First, the pre-processing unit 70 reads the K-space time series data K0 from the storage unit 40, and generates the multi-channel image space time series data I0 by performing inverse Fourier transform on the K-space time series data K0 by the inverse Fourier transform unit 71. The multi-channel image space time series data I0 is image space data having the same size as the K-space time series data K0. The data of each channel of the multi-channel image space time series data I0 is image space data converted from the K-space data acquired by each receiving coil.

[0085] Then, the preprocessing section 70 merges data of the plurality of channels of the multi-channel image spatial time series data I0 into data of a single channel based on the sensitivity profile time series data S by the channel merging unit 72, and generates initial image time series data X0. The initial image time series data X0 is three-dimensional tensor data of width w x height h x frame number f, which is directly generated based on the undersampled K-space time series data K0, and contains frame number f pieces of two-dimensional image data that are two-dimensional matrix data of width w x height h. The two-dimensional image data contained in the initial image time series data X0 has problems such as presence of a large amount of artifacts and noise, lack of details, and image blurring.

[0086] After the processing of step S107 is completed, the processing proceeds to step S108.

[0087] In steps S108 to S112, the correction processing is repeatedly performed on the image data a predetermined number of times based on the initial image time series data X0. Here, the predetermined number of times is preferably 8 to 10 times. Hereinafter, the image time series data obtained after the initial image time series data X0 has been corrected t (t is an integer greater than or equal to 0) times will be referred to as corrected image time series data X t The initial image time series data X0 and the corrected image time series data X0 in the case where t is 0 are the same image.

[0088] In step S108, the magnetic resonance image reconstruction apparatus 1 sets the current correction number (iteration number) to 0. After the processing of step S108 is completed, the processing proceeds to step S109.

[0089] Figure 6 is a data flow diagram for explaining the processing of steps S109 to S110 of the magnetic resonance image reconstruction method of the embodiment of the present application. In Figure 6 , the flow of data is indicated by solid arrows. Hereinafter, the processing of steps S109 to S110 will be explained with reference to Figure 6 .

[0090] In step S109 (one example of an image space regularization step), the image space regularization section 80 generates regularized image time series data Z t by performing regularization processing on the corrected image time series data X t in the image space using the second neural network 81, where t is the current correction number.

[0091] Specifically, first, the image space regularization section 80 reads the neural network parameters of the second neural network 81 corresponding to the current correction number from the storage section 40, and causes the second neural network 81 to load the parameters. Then, the image space regularization section 80 reads the corrected image time series data Xt and the corrected image time series data X t is input into the second neural network 81 whose parameters have been loaded. Then, the image space regularization unit 80 causes the second neural network 81 to perform forward propagation on the corrected image time series data X t and calculate the regularization image time series data Z t as output data of the second neural network 81. t The regularization image time series data Z t is three-dimensional tensor data having the same size as the corrected image time series data X

[0092] It can be considered that the processing performed by the second neural network 81 is de-noising, de-artifacting, and anti-aliasing processing for the corrected image time series data X t . Therefore, it can also be considered that the regularization image time series data Z t is the corrected image time series data X t that has undergone de-noising, de-artifacting, and anti-aliasing processing.

[0093] It is preferable that the neural network parameters used by the second neural network 81 be different in the correction processing of different correction times. By causing the second neural network 81 to use different parameters in different correction stages, appropriate processing can be performed on the corrected image time series data X t of different correction stages.

[0094] After the processing of step S109 is completed, the process proceeds to step S110.

[0095] In step S110 (one example of a data consistency processing step), the data consistency processing unit 90 performs data consistency processing on the regularization image time series data Z t in such a manner that the K-space data corresponding to the regularization image time series data Z t becomes close to the under-sampled K-space time series data K0, and generates the corrected image time series data X t+1 .

[0096] The data consistency processing unit 90 calculates the corrected image time series data X t+1 using the following expression (1).

[0097]

[0098] In Formula (1), λ is a data consistency coefficient, and A is a forward operator. λ can be a fixed value set in advance or a trainable value. The operation A(X) performed by the forward operator A on the single-channel image data X indicates that the image data X is first converted into multi-channel image data based on the sensitivity profile time series data S, and then Fourier transform is performed on the multi-channel image data to obtain K-space data corresponding to the image data X, and then the Hadamard product of the obtained K-space data and the scan mask time series data M is calculated.

[0099] Formula (1) can be solved by an optimization algorithm such as a Gradient Descent method or a Proximal Mapping method. The Proximal Mapping method can be further solved by a Conjugate Gradient method.

[0100] The corrected image time series data X t+1 generated through the data consistency processing has pixel values close to the pixel values of the regularized image time series data Z t , but the K-space data corresponding to the corrected image time series data X t+1 is closer to the undersampled K-space time series data K0 than the K-space data corresponding to the regularized image time series data Z t .

[0101] After the processing of step S110 is completed, the process proceeds to step S111.

[0102] In step S111, the current correction number is incremented by 1.

[0103] In the magnetic resonance image reconstruction method of the present embodiment, the corrected image time series data X t is generated by performing a complete correction process once on the corrected image time series data X t+1 through the processing of steps S109 to S110. The corrected image time series data X t+1 is closer to the true value of the magnetic resonance image time series data than the corrected image time series data X t .

[0104] After the processing of step S111 is completed, the process proceeds to step S112.

[0105] In step S112, it is determined whether the current correction number has reached a predetermined number, and in the case where it is determined that the predetermined number has been reached, the process proceeds to step S113; in the case where it is determined that the predetermined number has not been reached, the process proceeds to step S109.

[0106] In step S113, the corrected image time series data that has been corrected a predetermined number of times is output as an estimate of the true value of the magnetic resonance image time series data.

[0107] After the process of step S113 is completed, the process of the magnetic resonance image reconstruction method is ended.

[0108] Next, the effects of the magnetic resonance image reconstruction apparatus and the magnetic resonance image reconstruction method of the present application will be described.

[0109] In the present application, based on the auto-calibration signal time series data included in the undersampled K-space time series data, sensitivity profile time series data that indicates the sensitivity of each receiving coil at each timing in the scan of dynamic magnetic resonance imaging is generated. Therefore, even if the auto-calibration signal becomes discontinuous, accurate sensitivity profile time series data can be calculated, and correct correction by magnetic field inhomogeneity and inconsistency between coils and the like can be performed. According to the present application, it is possible to suppress the decrease in stability and accuracy of reconstruction of the magnetic resonance image due to discontinuity of the auto-calibration signal.

[0110] Further, in the present application, based on the similarity between the plurality of auto-calibration signal frame data included in the auto-calibration signal time series data, the auto-calibration signal frame data are grouped, auto-calibration signal group data is generated, and the sensitivity profile time series data is calculated based on the auto-calibration signal group data. Therefore, the accuracy of the calculated sensitivity profile time series data can be improved, and further the accuracy of the reconstructed magnetic resonance image time series data can be improved.

[0111] Figure 7 is a graph for comparing the magnetic resonance image reconstructed by the magnetic resonance image reconstruction method of the present application with the magnetic resonance image reconstructed by the prior art.

[0112] In Figure 7 , prior art 1 is a technique of reconstructing a magnetic resonance image using a reconstruction network based on an auto-calibration signal that is an average of a plurality of K-space frame data, and prior art 2 is a technique of reconstructing a magnetic resonance image using a reconstruction network based on an auto-calibration signal of any one of a plurality of K-space frame data. Further, in Figure 7 , magnetic resonance images 1 to 3 are respectively reconstructed based on K-space frame data included in different undersampled K-space time series data. In the reconstruction of magnetic resonance images 2 and 3, the auto-calibration signals in the plurality of K-space frame data included in the K-space time series data are discontinuous.

[0113] As Figure 7As shown, magnetic resonance images 1 to 3 were reconstructed by the prior art 1, 2 and the magnetic resonance image reconstruction method of the present application. The prior art 1, 2 generated obvious artifacts (positions indicated by arrows in the figures) in the reconstruction of the magnetic resonance images 2, 3. On the other hand, the magnetic resonance image reconstruction method of the present application did not generate artifacts in the reconstruction of the magnetic resonance images 1 to 3.

[0114] In the case of 4X, 8X undersampling, the stability and precision of the reconstruction of the magnetic resonance image reconstruction method of the present application are higher than those of the prior art. In each test, the Structural Similarity Index (SSIM) and the Peak Signal-to-Noise Ratio (PSNR) of the magnetic resonance image reconstruction method of the present application are higher and the normalized mean squared error is lower than those of the magnetic resonance image reconstruction method of the prior art.

[0115] (Method for training and performance evaluation of neural network)

[0116] In the above description, the magnetic resonance image reconstruction apparatus and the magnetic resonance image reconstruction method of the present application use a plurality of first neural networks 64-1 to 64-g, a second neural network 81 and a data consistency coefficient λ, and these neural networks and parameters need to be trained in advance to work normally. Next, the training method of the above neural networks and parameters will be described.

[0117] First, a plurality of sets of teacher data stored in advance are read from the storage unit 40. In each set of teacher data, undersampled K-space time series data K0 and a scan mask time series data M corresponding thereto are included as input data, and a true value of magnetic resonance image time series data is included as output data.

[0118] Then, the plurality of sets of teacher data is divided into a training set, a test set, and a cross validation set. As an example of the proportions of the training set, the test set, and the cross validation set, 80%, 10%, 10% or 90%, 5%, 5% or the like can be given. For example, assuming that the total number of the teacher data is 10,000 sets, the teacher data of data #1 to #10,000 is divided into data #1 to #8,000 as the training set, data #8,001 to #9,000 as the test set, and data #9,001 to #10,000 as the cross validation set. In this case, the input data in each set of the teacher data in the training set is input to the magnetic resonance image reconstruction apparatus 1, and the magnetic resonance image reconstruction method of the present embodiment is executed to calculate the estimated value of the magnetic resonance image time series data, and then the difference value between the estimated value of the magnetic resonance image time series data and the true value of the magnetic resonance image time series data is calculated, and the difference value is back-propagated. Thereby, the parameters of each neural network and other parameters that can be subjected to machine learning are changed so that the difference value between the estimated value of the magnetic resonance image time series data output from the magnetic resonance image reconstruction apparatus 1 and the true value of the magnetic resonance image time series data becomes smaller. The above steps are repeated until the difference value between the estimated value of the magnetic resonance image time series data output from the magnetic resonance image reconstruction apparatus 1 and the true value of the magnetic resonance image time series data is smaller than a threshold value set in advance for most of the data in the test set. At this time, it is determined that the training of each neural network and the parameters is completed.

[0119] Then, the input data of the cross validation data (data #9,001 to #10,000) is input to the magnetic resonance image reconstruction apparatus 1 whose learning has been completed, and the peak signal-to-noise ratio of the estimated value of the magnetic resonance image time series data output from the magnetic resonance image reconstruction apparatus 1 and the structural similarity index and the normalized mean square error between the estimated value and the true value of the magnetic resonance image time series data are calculated as evaluation data.

[0120] The above illustrates several embodiments of the present application, but these embodiments are given as examples and are not intended to limit the scope of the application. These new embodiments can be implemented in other various forms, and various omissions, substitutions, modifications, and the like can be made within the scope of the gist of the application. These embodiments and modifications thereof are included in the scope or gist of the application, and are included in the scope of the application and equivalents thereof recited in the claims. In addition, the above-described embodiments can be implemented in combination with each other.

Claims

1. An image processing apparatus comprising: an acquisition section that acquires K-space data obtained by undersampling using a plurality of coils, the K-space data corresponding to a plurality of frames, respectively; a classification section that classifies auto-calibration signal data in the K-space data corresponding to the plurality of frames, respectively, into a plurality of groups; a sensitivity profile calculation section that generates sensitivity profiles corresponding to the plurality of coils, respectively, based on data based on auto-calibration signal data classified into a first group among the plurality of groups and data based on auto-calibration signal data classified into a second group among the plurality of groups; an image generation section that generates an image corresponding to a first frame among the plurality of frames based on the sensitivity profiles calculated by the sensitivity profile calculation section and the K-space data corresponding to the first frame, and generates an image corresponding to a second frame among the plurality of frames based on the sensitivity profiles calculated by the sensitivity profile calculation section and the K-space data corresponding to the second frame; a regularization section that performs image-space regularization processing on the image corresponding to the first frame and performs image-space regularization processing on the image corresponding to the second frame; and a data consistency processing section that performs data consistency processing on the image corresponding to the first frame to which the image-space regularization processing is applied based on the K-space data corresponding to the first frame acquired by the acquisition section and the sensitivity profiles, and performs data consistency processing on the image corresponding to the second frame to which the image-space regularization processing is applied based on the K-space data corresponding to the second frame acquired by the acquisition section and the sensitivity profiles. 2.The image processing apparatus according to claim 1, wherein the classification section classifies the auto-calibration signal data in the K-space data corresponding to the plurality of frames, respectively, into the plurality of groups based on similarity of the auto-calibration signal data in the K-space data corresponding to the plurality of frames, respectively, to each other. 3.The image processing apparatus according to claim 2, wherein the classification section classifies the auto-calibration signal data in the K-space data corresponding to the plurality of frames, respectively, into the plurality of groups based on at least one of a difference in pixel values, a Frobenius norm, a cosine similarity, and an Euclidean distance between the auto-calibration signal data in the K-space data corresponding to the plurality of frames, respectively. 4.The image processing apparatus according to claim 1, wherein the classification section classifies the auto-calibration signal data in the K-space data corresponding to the plurality of frames, respectively, into the plurality of groups by clustering the auto-calibration signal data in the K-space data corresponding to the plurality of frames, respectively. 5.The image processing apparatus according to claim 1, wherein ​ The sensitivity profile calculation section has a first neural network and a second neural network, the first neural network corresponding to the first group, generating sensitivity calculation image set data for calculating the sensitivity profile by image processing of auto-calibration signal data classified into the first group, the second neural network corresponding to the second group, generating sensitivity calculation image set data for calculating the sensitivity profile by image processing of auto-calibration signal data classified into the second group, The sensitivity calculation image set data indicates the relative magnitude of the sensitivity between coils, The sensitivity profile calculation section generates the sensitivity profile based on the sensitivity calculation image set data generated by the first neural network and the sensitivity calculation image set data generated by the second neural network.

6. The image processing apparatus according to claim 5, wherein The first neural network and the second neural network share parameters.

7. An image processing method comprising: an acquisition step of acquiring K-space data obtained by undersampling using a plurality of coils, the K-space data corresponding to a plurality of frames respectively; a classification step of classifying auto-calibration signal data in the K-space data corresponding to the plurality of frames respectively into a plurality of groups; a sensitivity profile calculation step of generating sensitivity profiles corresponding to the plurality of coils respectively from data based on auto-calibration signal data classified into a first group among the plurality of groups and data based on auto-calibration signal data classified into a second group among the plurality of groups; an image generation step of generating an image corresponding to a first frame among the plurality of frames based on the sensitivity profiles calculated in the sensitivity profile calculation step and the K-space data corresponding to the first frame, and generating an image corresponding to a second frame among the plurality of frames based on the sensitivity profiles calculated in the sensitivity profile calculation step and the K-space data corresponding to the second frame; a regularization step of performing image space regularization processing on the image corresponding to the first frame, and performing image space regularization processing on the image corresponding to the second frame; and a data consistency processing step of performing data consistency processing on the image corresponding to the first frame to which the image space regularization processing is applied, based on the K-space data corresponding to the first frame acquired in the acquisition step and the sensitivity profiles, and performing data consistency processing on the image corresponding to the second frame to which the image space regularization processing is applied, based on the K-space data corresponding to the second frame acquired in the acquisition step and the sensitivity profiles. ​

Images