Image processing apparatus and image processing method

Abstract

To improve image quality.SOLUTION: An image processing apparatus includes an acquisition part, a classification part, a sensitivity map calculation part, an image generation part, a regularization part, and a data consistency processing part. The processing circuitry acquires k-space data. The processing circuitry classifies ACS data in the k-space data corresponding to each of the frames into a plurality of groups. The sensitivity map calculation unit generates a sensitivity map. The image generation unit generates an image. The regularizer performs image-space regularization processing. The data consistency process performs a data consistency process.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The embodiments disclosed in this specification and the drawings relate to an image processing device and an image processing method. [Background technology]

[0002] Magnetic resonance imaging (MRI) is a non-invasive medical imaging technique that utilizes the magnetic resonance phenomenon, in which hydrogen nuclei placed in a static magnetic field resonate with a radio-frequency magnetic field of a specific frequency. Dynamic magnetic resonance imaging (DMRI) is a type of MRI technique that can acquire magnetic resonance images over time, allowing for the observation and analysis of dynamic changes in organs or tissues over a certain period of time.

[0003] In dynamic magnetic resonance imaging, a pulse signal is sent to a subject (patient) in a frequency-encoded and phase-encoded magnetic field, and echo signals due to specific nuclear magnetic resonance are received from multiple receiving coils to obtain multiple temporally consecutive k-space frame data. Magnetic resonance image time series data is then reconstructed based on the k-space time series data consisting of these k-space frame data.

[0004] However, acquiring all of the k-space frame data requires a long magnetic resonance scan, and in this case, the multiple magnetic resonance image frame data included in the reconstructed magnetic resonance image time series data become discontinuous in time (the intervals become longer), making it impossible to accurately represent the changes over time of the scanned object. Therefore, in magnetic resonance imaging, some of the k-space frame data are usually acquired by undersampling, and the magnetic resonance image time series data is reconstructed based on the k-space time series data consisting of the undersampled k-space frame data, thereby shortening the magnetic resonance scan time.

[0005] Non-Patent Document 1 discloses an image processing method for magnetic resonance image reconstruction using an end-to-end unrolled reconstruction network, which calculates sensitivity maps of multiple receiver coils via a neural network. In the image processing method of Non-Patent Document 1, sensitivity maps are calculated using a convolutional neural network based on an auto-calibration signal (ACS). Non-Patent Document 2 discloses an image processing method for magnetic resonance image reconstruction using an end-to-end unrolled reconstruction network, which calculates sensitivity maps of multiple receiver coils via a neural network. In the image processing method of Non-Patent Document 2, sensitivity maps generated based on the ACS are refined using a 2D U-Net model. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] kt CLAIR:Self-Consistency Guided Multi-Print Learning for Dynamic Parallel MR Image Reconstruction, Lipping Zhang, &Weitian Chen.(2024).arXiv:2310.11050

[0007] [Non-patent document 2] Deep Cardiac MRI Reconstruction with ADMM, George Yiasemis, Nikita Moriaov, Jan-Jakob Sonke, &Jonas Teuwen.(2023).arXiv:2310.06628 Summary of the Invention [Problem to be solved by the invention]

[0008] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to improve image quality. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0009] An image processing apparatus according to an embodiment includes an acquisition unit, a classification unit, a sensitivity map calculation unit, an image generation unit, a regularization unit, and a data consistency processing unit. The acquisition unit acquires k-space data obtained by undersampling using multiple coils, the k-space data corresponding to each of multiple frames. The classification unit classifies ACS data in the k-space data corresponding to each of the multiple frames into multiple groups. The sensitivity map calculation unit generates a sensitivity map corresponding to each of the multiple coils based on data based on ACS data classified into a first group included in the multiple groups and data based on ACS data classified into a second group included in the multiple groups. The image generation unit generates an image corresponding to a first frame based on the sensitivity map and the k-space data corresponding to a first frame of the multiple frames, and generates an image corresponding to a second frame based on the sensitivity map and the k-space data corresponding to a second frame of the multiple frames. The regularization unit performs image space regularization processing on the image corresponding to the first frame and performs image space regularization processing on the image corresponding to the second frame. The data consistency processing unit performs data consistency processing on an image corresponding to the first frame to which the image space regularization processing has been applied, based on the k-space data corresponding to the first frame and the sensitivity map, and performs data consistency processing on an image corresponding to the second frame to which the image space regularization processing has been applied, based on the k-space data corresponding to the second frame and the sensitivity map. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a magnetic resonance image reconstruction apparatus according to an embodiment. [Figure 2] FIG. 2 is a flowchart showing the flow of the magnetic resonance image reconstruction method according to the embodiment. [Figure 3] FIG. 3 is a diagram showing an example of the configuration of k-space time series data and k-space frame data. [Figure 4] FIG. 4 is a data flow diagram for explaining the processing of steps S102 to S106 of the magnetic resonance image reconstruction method according to the embodiment. [Figure 5] FIG. 5 is a data flow diagram for explaining the processing of step S107 of the magnetic resonance image reconstruction method according to the embodiment. [Figure 6] FIG. 6 is a data flow diagram for explaining the processing of steps S109 to S110 of the magnetic resonance image reconstruction method according to the embodiment. [Figure 7] FIG. 7 is a diagram for comparing a magnetic resonance image reconstructed using the magnetic resonance image reconstruction method with a magnetic resonance image reconstructed using the prior art. [Figure 8] FIG. 8 is a diagram illustrating the problems with the conventional magnetic resonance image reconstruction method. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of an image processing device and an image processing method will be described in detail with reference to the drawings.

[0012] The magnetic resonance image reconstruction apparatus of this embodiment is an image processing apparatus that reconstructs magnetic resonance image time series data in image space based on k-space time series data obtained by undersampling using multiple coils. Its purpose is to estimate the ground truth (GT) of the magnetic resonance image time series data corresponding to the fully sampled k-space time series data and reconstruct magnetic resonance image time series data close to the GT of the magnetic resonance image time series data.

[0013] FIG. 1 is a diagram showing an example of the configuration of a magnetic resonance image reconstruction apparatus 1 according to an embodiment. The configuration of the magnetic resonance image reconstruction apparatus 1 according to the embodiment will be described with reference to FIG. 1. The magnetic resonance image reconstruction apparatus 1 according to the embodiment includes an input / output interface 10, a display interface 20, a communication interface 30, a storage unit 40, an acquisition unit 50, a sensitivity map calculation unit 60, a preprocessing unit 70, an image space regularization unit 80, a data consistency processing unit 90, and an output unit 100. The input / output interface 10, the display interface 20, the communication interface 30, the storage unit 40, the acquisition unit 50, the sensitivity map calculation unit 60, the preprocessing unit 70, the image space regularization unit 80, the data consistency processing unit 90, and the output unit 100 are connected to each other so that they can communicate with each other.

[0014] The input / output interface 10 is an interface for connecting the magnetic resonance image reconstruction apparatus 1 to an input device (not shown), receives a user's input operation from the input device, and transmits a signal based on the received input operation to the magnetic resonance image reconstruction apparatus 1. The input / output interface 10 is, for example, a serial bus interface such as a USB. The input device includes a mouse, keyboard, trackball, switch, button, joystick, touch screen, microphone, etc. The input / output interface 10 may also be connected to a storage device to read and write various data to and from the storage device. The storage device may be, for example, a hard disk drive (HDD), a solid state drive (SSD), etc.

[0015] The display interface 20 is an interface for connecting the magnetic resonance image reconstruction apparatus 1 to a display device (not shown), and transmits data to the display device to display an image. The display interface 20 is, for example, a video output interface such as a DVI (Digital Visual Interface) or an HDMI (registered trademark) (High-Definition Multimedia Interface). The display device includes an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence) display. The display device displays a user interface for accepting input operations from a user and magnetic resonance image data output from the magnetic resonance image reconstruction apparatus 1, and the user interface is, for example, a GUI (Graphical User Interface).

[0016] The communication interface 30 is an interface for connecting the magnetic resonance image reconstruction apparatus 1 to a server (not shown) and is capable of transmitting and receiving various data to and from the server. The communication interface 30 is, for example, a network card such as a wireless network card or a wired network card.

[0017] The storage unit 40 stores data such as image data and k-space data used for image reconstruction. The storage unit 40 also stores parameters, such as neural network parameters, used by the magnetic resonance image reconstruction device 1 when performing image reconstruction. The storage unit 40 also stores training data for training each neural network and other learnable parameters used by the magnetic resonance image reconstruction device 1. The storage unit 40 is implemented by a storage device such as a read-only memory (ROM), a flash memory, a random access memory (RAM), a hard disk drive (HDD), a solid state drive (SSD), or a register. Flash memory, HDD, SSD, etc. are non-volatile storage media. These non-volatile storage media may also be implemented by other storage devices connected via a network, such as a network-attached storage (NAS) or an external storage server device. Here, the network includes, for example, the Internet, a wide area network (WAN), a local area network (LAN), a carrier terminal, a wireless communication network, a wireless base station, a dedicated line, etc.

[0018] In this embodiment, the processing functions performed by the sensitivity map calculation unit 60 (ACS extraction means 61, grouping means 62, inverse Fourier transform means 63, first neural networks 64-1 to 64-g, sensitivity map generation means 65), acquisition unit 50, preprocessing unit 70 (inverse Fourier transform means 71, channel integrating means 72), image space regularization unit 80 (second neural network 81), data consistency processing unit 90, and output unit 100 are stored in the storage unit 40 in the form of computer-executable programs. These functions are realized by a processing circuit including a processor that reads and executes the programs from the storage unit 40 to realize the functions corresponding to each program. In other words, the processing circuit in a state in which each program has been read has each function shown in FIG. 1. Note that, in FIG. 1, these processing functions are described as being realized by a single processing circuit; however, a processing circuit may be configured by combining multiple independent processors, and the functions may be realized by each processor executing a program. In other words, each of the above functions may be configured as a program, and a single processing circuit may execute each program. As another example, a specific function may be implemented in a dedicated independent program execution circuit. Note that in Fig. 1, the grouping means 62, the preprocessing unit 70, and the data consistency processing unit are examples of a classification unit, an image generation unit, and a regularization unit, respectively.

[0019] The term "processor" used in the above description refers to circuits such as a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). The processor realizes its functions by reading and executing programs stored in the storage unit 40.

[0020] The acquisition unit 50 acquires k-space time series data obtained by undersampling using multiple coils. The undersampled k-space time series data consists of multiple undersampled k-space frame data that are consecutive in time. The undersampled k-space frame data represents k-space data collected by each receive coil at a specific timing in a dynamic magnetic resonance imaging scan.

[0021] The sensitivity map calculation unit 60 generates sensitivity map time series data indicating the sensitivities of a plurality of receive coils used in a dynamic magnetic resonance scan based on the undersampled k-space time series data.

[0022] The sensitivity map time-series data is made up of a plurality of time-sequential sensitivity map frame data, each of which indicates the sensitivity of each receive coil to each scan position within the scan range at a specific timing in a dynamic magnetic resonance imaging scan.

[0023] The sensitivity map calculation unit 60 includes an ACS extraction means 61, a grouping means 62, an inverse Fourier transform means 63, a plurality of first neural networks 64-1 to 64-g, and a sensitivity map generation means 65.

[0024] The ACS extraction means 61 extracts ACS time series data from the k-space time series data. The ACS is used to correct phase and amplitude errors in the k-space data caused by magnetic field inhomogeneity, coil characteristics, etc. The ACS is data located near the center position of the k-space.

[0025] The grouping means 62 groups a plurality of ACS frame data included in the ACS time-series data, and generates ACS group data of a predetermined number g (g is an integer of 2 or more).

[0026] The inverse Fourier transform means 63 performs inverse Fourier transform on the ACS group data using an algorithm such as inverse fast Fourier transform, to generate ACS image group data.

[0027] Each of the first neural networks 64-1 to 64-g performs image processing on the ACS image group data to generate sensitivity calculation image group data for calculating sensitivity map time-series data. is, for example, a convolutional neural network or a Transformer. The multiple first neural networks 64-1 to 64-g are preferably convolutional neural networks. More preferably, the multiple first neural networks 64-1 to 64-g are U-Nets. In this embodiment, the multiple first neural networks 64-1 to 64-g are convolutional neural networks including an input layer, an output layer, a convolutional layer, an excitation layer, a pooling layer, a batch normalization layer, and a fully connected layer, and the input layer and output layer are equal in size. The multiple first neural networks 64-1 to 64-g realize image processing functions by loading neural network parameters dedicated to the multiple first neural networks 64-1 to 64-g stored in the storage unit 40. The parameters of the multiple first neural networks 64-1 to 64-g may or may not be shared, and it is preferable that the multiple first neural networks 64-1 to 64-g share parameters. The number of first neural networks is equal to the number of ACS group data generated by the grouping means 62.

[0028] The sensitivity map generating means 65 calculates the sensitivity map time series data based on the sensitivity calculation image group data.

[0029] The preprocessing unit 70 preprocesses the undersampled k-space time series data based on the sensitivity map time series data to generate initial image time series data, which is time series data in image space corresponding to the undersampled k-space time series data.

[0030] The preprocessing unit 70 includes an inverse Fourier transform unit 71 and a channel integrating unit 72. The inverse Fourier transform unit 71 performs an inverse Fourier transform on the k-space time-series data using an algorithm such as an inverse fast Fourier transform. The channel integrating unit 72 integrates multi-channel data corresponding to each receiving coil of the image time-series data into single-channel data based on the sensitivity map time-series data.

[0031] The image space regularization unit 80 performs image space regularization processing on the input image time series data in image space using a second neural network 81, generating regularized image time series data, which is the image time series data that has been regularized. The image space regularization unit 80 includes the second neural network 81. The second neural network 81 is, for example, a feedforward neural network, a convolutional neural network, or a Transformer. The second neural network 81 is preferably a convolutional neural network. More preferably, the second neural network 81 is a U-Net. In this embodiment, the second neural network 81 is a convolutional neural network that includes an input layer, an output layer, a convolutional layer, an excitation layer, a pooling layer, a batch normalization layer, and a fully connected layer, and the input layer and output layer are equal in size. The second neural network 81 realizes the function of image space regularization processing by loading neural network parameters dedicated to the second neural network 81 stored in the storage unit 40. By loading different neural network parameters, the second neural network 81 can be made to perform different regularization processes.

[0032] The data consistency processing unit 90 performs data consistency processing on the regularized image time series data based on the undersampled k-space time series data and the sensitivity map time series data so that the k-space time series data corresponding to the regularized image time series data approaches the undersampled k-space time series data, thereby generating corrected image time series data.

[0033] The output unit 100 outputs image time series data based on the corrected image time series data as reconstructed magnetic resonance image time series data.

[0034] 2 is a flowchart showing the flow of the magnetic resonance image reconstruction method according to the embodiment. Hereinafter, the flow of the magnetic resonance image reconstruction method according to the embodiment will be described with reference to FIG.

[0035] The magnetic resonance image reconstruction method of this embodiment is an image processing method for reconstructing magnetic resonance image time series data in image space based on k-space time series data obtained by undersampling using a plurality of coils.

[0036] First, the process proceeds to step S101. In step S101, the acquisition unit 50 acquires the undersampled k-space time series data K0 and the corresponding scan mask time series data M input from the storage unit 40 or an external storage device, and inputs them to the magnetic resonance image reconstruction device 1. That is, the acquisition unit 50 acquires k-space data obtained by undersampling using multiple coils and corresponding to each of multiple frames.

[0037] The k-space time series data K0 is k-space frame data F1 to F2 with a number f of frames. f The number of frames f is the number of k-space frame data collected in the dynamic magnetic resonance scan. f indicates k-space data acquired at multiple consecutive timings.

[0038] Figure 3 shows the k-space time series data K0 and the k-space frame data F1 to F f 3 is a diagram showing an example of the configuration of k-space time series data K0 and k-space frame data F1 to F2 of this embodiment. f As shown in Fig. 3, in this embodiment, the k-space time series data K0 is four-dimensional tensor data of width w × height h × number of channels (number of receiving coils) c × number of frames f. The k-space time series data K0 is divided into k-space frame data F1 to F2, which are three-dimensional tensor data of width w × height h × number of channels c. f Contains f frames of k-space frame data F1 to F f Each of the channels includes c pieces of two-dimensional matrix data indicating the k-space data received by each receiving coil.

[0039] Here, the width direction of the matrix is ​​the phase encoding direction, and the height direction is the frequency encoding direction. Dynamic magnetic resonance scanning involves undersampling, in which specific frequency encodings and phase encodings are omitted, to reduce scan time. Therefore, in magnetic resonance scanning, positions corresponding to specific frequency encodings and phase encodings are skipped. As a result, in the k-space time series data K0, no data exists at positions corresponding to some frequency codes and phase codes, and the data at these positions is zero-filled. Because data near the center of the k-space data has a significant impact on the contrast of the reconstructed image data, undersampling typically involves sampling data near the center in the frequency encoding and phase encoding directions, while skipping data at positions far from the center.

[0040] The scan mask time series data M indicates which positions in the k-space time series data K0 were sampled and which positions were omitted in the magnetic resonance scan. The scan mask time series data M is a four-dimensional tensor of the same size as the k-space time series data K0. The scan mask time series data M includes two-dimensional matrix data of width w × height h, number of channels c × number of frames f, that perfectly match. In this two-dimensional matrix data, the value of the frequency encoding and phase encoding positions where sampling was performed is set to 1, and the value of the frequency encoding and phase encoding positions where sampling was not performed is set to 0.

[0041] When the process of step S101 is completed, the process proceeds to step S102. In steps S102 to S106 (an example of a sensitivity map calculation step), the sensitivity map calculation unit 60 generates sensitivity map time series data S indicating the sensitivities of multiple receive coils used in a dynamic magnetic resonance scan, based on the undersampled k-space time series data K0.

[0042] Fig. 4 is a data flow diagram for explaining the processing of steps S102 to S106 of the magnetic resonance image reconstruction method according to the embodiment. In Fig. 4, the flow of data is indicated by solid arrows. Next, the processing of steps S102 to S106 will be explained with reference to Fig. 4.

[0043] In step S102, the ACS extraction means 61 of the sensitivity map calculation unit 60 extracts ACS time series data Q from the k-space time series data K0.

[0044] Specifically, the ACS extraction means 61 calculates the Hadamard product (product of corresponding elements) of the k-space time series data K0 and the ACS mask time series data V read out from the storage unit 40 to generate ACS time series data Q. Both the ACS time series data Q and the ACS mask time series data V are four-dimensional tensors of the same size as the k-space time series data K0. The ACS mask time series data V includes two-dimensional matrix data of width w × height h, number of channels c × number of frames f, which are completely identical to each other. In these two-dimensional matrix data, the value of a position in the region where the ACS exists is set to 1, and the value of the remaining positions is set to 0. The ACS time series data Q includes two-dimensional matrix data of width w × height h, number of channels c × number of frames f, and each two-dimensional matrix data indicates the ACS of each coil at each timing.

[0045] When the process of step S102 is completed, the process proceeds to step S103. In step S103, the grouping means 62 of the sensitivity map calculation unit 60 groups the ACS frame data based on the similarity between the ACS frame data of the number f of frames included in the ACS time-series data Q, and generates ACS group data G1 to G2 of the number g of groups. g That is, the grouping means 62 as a classification unit classifies the ACS time-series data Q, which is ACS data in the k-space data corresponding to each of the plurality of frames, into a plurality of groups.

[0046] As an example, the grouping means 62 as a classification unit classifies each of the ACS data in the k-space data corresponding to each of the multiple frames into multiple groups based on the similarity between the ACS data in the k-space data corresponding to each of the multiple frames. Specifically, the grouping means 62 as a classification unit first performs clustering on the ACS frame data (number of frames: f) based on the similarity between the ACS frame data, which is the ACS data in the k-space data corresponding to each of the multiple frames, thereby classifying each of the ACS data in the k-space data corresponding to each of the multiple frames into multiple groups, and divides the ACS frame data into a predetermined number of groups: g. Each group contains r frames of ACS frame data, where r is the number of frames in a group = f frames / g groups.

[0047] Then, the grouping means 62 joins together the ACS frame data, the number of which is r, in each group in the frame dimension to form ACS group data G1 to G2. g Generate.

[0048] The grouping means 62 as a classification unit may perform grouping based on at least one of the difference value of pixel values ​​between a plurality of ACS frame data, the Frobenius norm, the cosine similarity, and the Euclidean distance, and classify each of the ACS data in the k-space data corresponding to each of the plurality of frames into a plurality of groups.

[0049] When the process of step S103 is completed, the process proceeds to step S104. In step S104, the inverse Fourier transform means 63 of the sensitivity map calculation unit 60 performs the inverse Fourier transform on the ACS group data G1 to G g is inverse Fourier transformed to obtain ACS image group data H1~H g Generate ACS image group data H1~H g ACS group data G1~G g It is a 4-dimensional tensor of the same size as ACS image group data H1~H gcontains two-dimensional image data, which is two-dimensional matrix data of width w and height h, with the number of channels c times the number of frames in the group r. Each two-dimensional image data contains hidden information about magnetic field inhomogeneity, coil sensitivity, etc.

[0050] When the process of step S104 is completed, the process proceeds to step S105. In step S105, the sensitivity map calculation unit 60 calculates the ACS image group data H1 to H2 by using the first neural networks 64-1 to 64-g, respectively. g Image processing is performed on each of the sensitivity calculation image group data L1 to L2 to calculate the sensitivity map time series data S. g Generate.

[0051] Image group data for sensitivity calculation L1~L g Each of the ACS image group data H1 to H2 includes two-dimensional image data, which is two-dimensional matrix data of width w and height h, with the number of channels c times the number of frames r in the group, and the two-dimensional image data of the number of channels c in each frame indicates the relative magnitude of sensitivity between each receiving coil. Specifically, first, the sensitivity map calculation unit 60 reads out the neural network parameters of the first neural networks 64-1 to 64-g from the storage unit 40 and loads the parameters into the first neural networks 64-1 to 64-g. Then, the sensitivity map calculation unit 60 calculates the ACS image group data H1 to H2. g is read from the storage unit 40, and the ACS image group data H1 to H g The first neural networks 64-1 to 64-g are then respectively input with the ACS image group data H1 to H2. g , and the output data of the second neural network 81 is the sensitivity calculation image group data L1 to L2. g Calculate the sensitivity calculation image group data L1 to L g The size of the ACS image group data H1 to H g is the same as

[0052] When the process of step S105 is completed, the process proceeds to step S106. In step S106, the sensitivity map generating means 65 of the sensitivity map calculation unit 60 generates the sensitivity calculation image group data L1 to L2. g The sensitivity map time series data S is calculated based on the above.

[0053] Specifically, first, the sensitivity map generating means 65 of the sensitivity map calculation unit 60 generates the sensitivity calculation image group data L1 to L2. g In the above integration, the image group data L1 to L2 for sensitivity calculation are integrated so that the order of the frame data included in the image time series data for sensitivity calculation is the same as the order of the frame data in the k-space time series data K0. g The total number of frames of image frame data for sensitivity calculation, f, included in the table is rearranged, and further, the total number of frames of image frame data for sensitivity calculation, f, are joined together in the frame dimension to generate image time-series data for sensitivity calculation.

[0054] Then, for each of the f pieces of image frame data for sensitivity calculation included in the image time-series data for sensitivity calculation, the sensitivity map generation means 65 divides each pixel value in the image frame data for sensitivity calculation by the square root of the sum of the squares of the pixel values ​​of the c number of channels (including the pixel itself) whose positions in the width and height directions are the same as the position of the pixel in question. This calculates the sensitivity map time-series data S. In this way, the sensitivity map generation means 65 as a sensitivity map calculation unit generates sensitivity maps (sensitivity map time-series data S) corresponding to each of the multiple coils based on data based on ACS data classified into a first group included in the multiple groups and data based on ACS data classified into a second group included in the multiple groups. For example, the sensitivity map calculation unit includes a first neural network corresponding to a first group that performs image processing on the ACS data classified into the first group to generate sensitivity calculation image group data for calculating a sensitivity map, and a second neural network corresponding to a second group that performs image processing on the ACS data classified into the second group to generate sensitivity calculation image group data for calculating a sensitivity map, the sensitivity calculation image group data being data indicating the relative magnitude of sensitivity between each coil, and sensitivity map generation means 65 serving as the sensitivity map calculation unit generates a sensitivity map based on the sensitivity calculation image group data generated by the first neural network and the sensitivity calculation image group data generated by the second neural network. The first and second neural networks may share parameters.

[0055] When the process of step S106 is completed, the process proceeds to step S107. In steps S102 to S106, the size of the number of groups g is adjusted, thereby adjusting the model scale of the first neural network and the strength of data sharing between each frame data. When the number of groups g is set to a small value, the number of frames r within a group is large, the strength of data sharing between frame data is high, and the model scale of the first neural network is large. This improves the accuracy of the calculated sensitivity map time-series data S, but increases the number of model parameters, increasing the computational complexity and training difficulty of the neural network. On the other hand, when the number of groups g is set to a large value, the number of frames r within a group is small, the strength of data sharing between frame data is low, and the model scale of the first neural network is small. This reduces the number of model parameters, reducing the computational complexity and training difficulty of the neural network, but correspondingly reduces the accuracy of the calculated sensitivity map time-series data S.

[0056] In step S107, the preprocessing unit 70 generates initial image time-series data X0 from the undersampled k-space time-series data K0. That is, the preprocessing unit 70 as an image generating unit generates an image corresponding to a first frame based on, for example, a sensitivity map and k-space data corresponding to a first frame of the multiple frames, and generates an image corresponding to a second frame based on the sensitivity map and k-space data corresponding to a second frame of the multiple frames. FIG. 5 is a data flow diagram for explaining the processing of step S107 of the magnetic resonance image reconstruction method according to the embodiment. In FIG. 5, the flow of data is indicated by solid arrows. Next, the processing of step S107 will be explained with reference to FIG. 5.

[0057] First, the preprocessing unit 70 reads the k-space time series data K0 from the storage unit 40, and performs an inverse Fourier transform on the k-space time series data K0 using the inverse Fourier transform means 71 to generate multi-channel image space time series data I0. The multi-channel image space time series data I0 is image space data of the same size as the k-space time series data K0. Data for each channel of the multi-channel image time series data I0 is image space data converted from k-space data collected by each receive coil.

[0058] The preprocessing unit 70 then uses the channel integration means 72 to integrate the data from multiple channels of the multi-channel image space time series data I0 into single-channel data based on the sensitivity map time series data S, thereby generating initial image time series data X0. The initial image time series data X0 is three-dimensional tensor data with width w × height h × number of frames f that is generated directly from the undersampled k-space time series data K0, and includes f frames of two-dimensional image data, which is two-dimensional matrix data with width w × height h. The two-dimensional image data included in the initial image time series data X0 has problems such as many artifacts and noise, a lack of detail, and blurred images.

[0059] When the process of step S107 is completed, the process proceeds to step S108. In steps S108 to S112, the image data is subjected to a predetermined number of correction processes based on the initial image time series data X0. Here, the predetermined number of times is preferably 8 to 10 times. Hereinafter, the image time series data that has been subjected to correction processes t times (t is an integer equal to or greater than 0) based on the initial image time series data X0 is referred to as corrected image time series data X0. t The initial image time series data X0 is the same image as the corrected image time series data X0 when t is equal to 0.

[0060] In step S108, the magnetic resonance image reconstruction apparatus 1 sets the current number of corrections (number of iterations) to 0. When the process of step S108 is completed, the process proceeds to step S109.

[0061] Fig. 6 is a data flow diagram for explaining the processing of steps S109 to S110 of the magnetic resonance image reconstruction method according to the embodiment. In Fig. 6, the flow of data is indicated by solid arrows. Next, the processing of steps S109 to S110 will be explained with reference to Fig. 6.

[0062] In step S109 (an example of an image space regularization step), the image space regularization unit 80 uses the second neural network 81 to regularize the corrected image time series data X t Regularization is performed on the regularized image time series data Z t where t is the current number of corrections. That is, image space regularization unit 80 as a regularization unit performs image space regularization processing on an image corresponding to the first frame, and performs image space regularization processing on an image corresponding to the second frame.

[0063] Specifically, first, the image space regularization unit 80 reads out the neural network parameters of the second neural network 81 corresponding to the current number of corrections from the storage unit 40, and loads the parameters into the second neural network 81. Then, the image space regularization unit 80 reads out the corrected image time series data X t and the corrected image time series data X t is input to the second neural network 81 with the loaded parameters. Then, the image space regularization unit 80 regularizes the second neural network 81 to the corrected image time series data X t By performing forward propagation based on the t Calculate the regularized image time series data Z t is the corrected image time series data X t It is a 3D tensor data of the same size as

[0064] The second neural network 81 performs the process of correcting the image time series data X tTherefore, the regularized image time series data Z t is the corrected image time series data X that has undergone noise removal, artifact removal, and anti-aliasing processing. t It is thought that this is the case.

[0065] It is preferable that the second neural network 81 uses different neural network parameters for the correction processes with different correction times. By making the second neural network 81 use different parameters at different correction stages, the corrected image time series data X t Appropriate processing can be performed for the above.

[0066] When the process of step S109 is completed, the process proceeds to step S110. In step S110 (an example of a data consistency processing step), the data consistency processing unit 90 performs the regularized image time series data Z t Regularize the image time series data Z so that the k-space data corresponding to t The data consistency process is performed on the corrected image time series data X t+1 That is, the data consistency processing unit 90 performs data consistency processing on an image corresponding to the first frame to which the image space regularization processing has been applied, based on the k-space data and sensitivity map corresponding to the first frame, and performs data consistency processing on an image corresponding to the second frame to which the image space regularization processing has been applied, based on the k-space data and sensitivity map corresponding to the second frame.

[0067] The data consistency processing unit 90 calculates the corrected image time series data X t+1 Calculate.

[0068]

number

[0069] In equation (1), λ is a data consistency coefficient, and A is a forward operator. λ may be a preset fixed value or a trainable value. The calculation A(X) by the forward operator A on single-channel image data X means first converting the image data X into multi-channel image data based on the sensitivity map time-series data S, then Fourier transforming the multi-channel image data to obtain k-space data corresponding to the image data X, and then calculating the Hadamard product of the obtained k-space data and the scan mask time-series data M.

[0070] Equation (1) can be solved by optimization algorithms such as Gradient Descent or Proximal Mapping, where Proximal Mapping can be further solved using Conjugate Gradient.

[0071] Corrected image time series data X generated by data consistency processing t+1 The pixel values ​​of the regularized image time series data Z t The pixel values ​​of the corrected image time series data X t+1 The k-space data corresponding to the regularized image time series data Z t is closer to the undersampled k-space time series data K0 than the k-space data corresponding to

[0072] When the process of step S110 is completed, the process proceeds to step S111, where 1 is added to the current number of corrections.

[0073] In the magnetic resonance image reconstruction method of this embodiment, the corrected image time series data X t Correction processing is performed once on the corrected image time series data X t+1 Generate the corrected image time series data X t+1 is the corrected image time series data X t This image data is closer to GT, which is magnetic resonance imaging time series data.

[0074] When the process of step S111 is completed, the process proceeds to step S112. In step S112, it is determined whether the current number of revisions has reached a predetermined number, and if it is determined that the predetermined number has been reached, the process proceeds to step S113, and if it is determined that the predetermined number has not been reached, the process proceeds to step S109.

[0075] In step S113, the corrected image time series data that has been corrected a predetermined number of times is output as an estimated value for the GT of the magnetic resonance image time series data.

[0076] When the process of step S113 is completed, the process of the magnetic resonance image reconstruction method ends.

[0077] The effects of the magnetic resonance image reconstruction apparatus and magnetic resonance image reconstruction method according to the embodiment will be described below.

[0078] In the embodiment, sensitivity map time series data indicating the sensitivity of each receive coil at each timing in a dynamic magnetic resonance imaging scan is generated based on ACS time series data included in the undersampled k-space time series data. Therefore, even if the ACS becomes discontinuous, accurate sensitivity map time series data can be calculated, and accurate correction can be made for magnetic field inhomogeneity, mismatch between coils, etc. According to the embodiment, it is possible to suppress deterioration in the stability and accuracy of magnetic resonance image reconstruction caused by ACS discontinuity.

[0079] In addition, in the embodiment, the ACS frame data are grouped based on the similarity between multiple ACS frame data included in the ACS time-series data to generate ACS group data, and sensitivity map time-series data are calculated based on the ACS group data, thereby improving the accuracy of the calculated sensitivity map time-series data and the accuracy of the reconstructed magnetic resonance image time-series data.

[0080] FIG. 7 is a diagram for comparing a magnetic resonance image reconstructed using the magnetic resonance image reconstruction method of the embodiment with a magnetic resonance image reconstructed using a conventional technique.

[0081] 7, prior art 1 is a technique for reconstructing a magnetic resonance image using a reconstruction network based on an average ACS of multiple k-space frame data, and prior art 2 is a technique for reconstructing a magnetic resonance image using a reconstruction network based on one of the ACSs of the multiple k-space frame data. Note that in FIG. 7, magnetic resonance images 1 to 3 are reconstructed based on k-space frame data included in different undersampled k-space time series data. In the reconstruction of magnetic resonance images 2 and 3, the ACSs of the multiple k-space frame data included in the k-space time series data are discontinuous.

[0082] As shown in Fig. 7, magnetic resonance images 1 to 3 were reconstructed using conventional techniques 1 and 2 and the magnetic resonance image reconstruction method of the embodiment. In conventional techniques 1 and 2, significant artifacts (at positions indicated by arrows in the figure) were generated in the reconstruction of magnetic resonance images 2 and 3. On the other hand, in the magnetic resonance image reconstruction method of the embodiment, no artifacts were generated in the reconstruction of magnetic resonance images 1 to 3.

[0083] In the case of 4X and 8X undersampling, the magnetic resonance image reconstruction method of the embodiment has higher reconstruction stability and accuracy than the conventional technique. In each test, the magnetic resonance image reconstruction method of the embodiment has a higher Structural Similarity Index (SSIM) and Peak Signal-to-Noise Ratio (PSNR), and a lower normalized mean squared error, compared to the conventional magnetic resonance image reconstruction method.

[0084] (Methods for training and evaluating neural networks) In the above description, the magnetic resonance image reconstruction device and magnetic resonance image reconstruction method of the embodiment use a plurality of first neural networks 64-1 to 64-g, a second neural network 81, and a data consistency coefficient λ, but these neural networks and parameters must be trained in advance to operate normally. Below, we will explain the training method for the above neural networks and parameters.

[0085] First, multiple sets of pre-stored training data are read out from the storage unit 40. Each set of training data includes undersampled k-space time series data K0 and corresponding scan mask time series data M as input data, and magnetic resonance image time series data GT as output data.

[0086] Next, the multiple sets of teacher data are divided into a training set, a test set, and a cross-validation set. Examples of the ratios of the training set, the test set, and the cross-validation set include 80%, 10%, 10%, or 90%, 5%, and 5%. For example, if the total number of sets of teacher data is 10,000, the teacher data of data #1 to #10,000 is divided so that data #1 to #8000 are the training set, data #8001 to #9000 are the test set, and data #9001 to #10,000 are the cross-validation set. In this case, input data of each set of teacher data in the training set is input to the magnetic resonance image reconstruction device 1, the magnetic resonance image reconstruction method of this embodiment is executed to calculate an estimated value of the magnetic resonance image time series data, and a difference value between the estimated value of the magnetic resonance image time series data and the GT of the magnetic resonance image time series data is calculated, and backpropagation is performed based on the difference value. As a result, the parameters of each neural network and other machine-learnable parameters are changed so as to reduce the difference between the estimated value of the magnetic resonance image time-series data output by the magnetic resonance image reconstruction device 1 and the GT of the magnetic resonance image time-series data. The above procedure is repeated for the majority of data in the test set until the difference between the estimated value of the magnetic resonance image time-series data output by the magnetic resonance image reconstruction device 1 and the GT of the magnetic resonance image time-series data becomes smaller than a preset threshold. Thereafter, it is determined that the training of each neural network and parameter has been completed.

[0087] Next, the input data of the cross-validation data (data #9001 to #10000) is input to the trained magnetic resonance image reconstruction device 1, and the peak signal-to-noise ratio of the estimated value of the magnetic resonance image time series data output by the magnetic resonance image reconstruction device 1, as well as the structural similarity index and normalized mean square error between the estimated value and the GT of the magnetic resonance image time series data are calculated as evaluation data.

[0088] According to at least one of the embodiments described above, image quality can be improved.

[0089] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0090] 1. Magnetic resonance image reconstruction device 10 Input / Output Interface 20 Display Interface 30 Communication Interface 40 Storage section 50 Acquisition Department 60 Sensitivity map calculation unit 61 ACS extraction means 62 Grouping Methods 63 Inverse Fourier Transform Means 64-1~64-g First neural network 65 Sensitivity map generation means 70 Pretreatment section 71 Inverse Fourier Transform Means 72 Channel Integration Method 80 Image Space Regularization Unit 81 Second Neural Network 90 Data integrity processing section 100 Output section

Claims

1. an acquisition unit that acquires k-space data obtained by undersampling using a plurality of coils and corresponding to each of a plurality of frames; a classification unit that classifies ACS data in the k-space data corresponding to each of the plurality of frames into a plurality of groups; a sensitivity map calculation unit that generates sensitivity maps corresponding to the plurality of coils based on data based on ACS data classified into a first group included in the plurality of groups and data based on ACS data classified into a second group included in the plurality of groups; an image generator that generates an image corresponding to a first frame based on the sensitivity map and the k-space data corresponding to a first frame of the plurality of frames, and generates an image corresponding to a second frame based on the sensitivity map and the k-space data corresponding to a second frame of the plurality of frames; a regularization unit that performs image space regularization processing on an image corresponding to the first frame and performs image space regularization processing on an image corresponding to the second frame; a data consistency processing unit that performs data consistency processing on an image corresponding to the first frame to which the image space regularization processing has been applied, based on k-space data corresponding to the first frame and the sensitivity map, and performs data consistency processing on the image corresponding to the second frame to which the image space regularization processing has been applied, based on k-space data corresponding to the second frame and the sensitivity map; An image processing device having:

2. The image processing device according to claim 1 , wherein the classification unit classifies each of the ACS data in the k-space data corresponding to each of the plurality of frames into the plurality of groups based on similarities between the ACS data in the k-space data corresponding to each of the plurality of frames.

3. 3. The image processing device according to claim 2, wherein the classifier classifies each of the ACS data in the k-space data corresponding to each of the plurality of frames into the plurality of groups based on at least one of a difference value of pixel values ​​between ACS data in the k-space data corresponding to each of the plurality of frames, a Frobenius norm, a cosine similarity, and a Euclidean distance.

4. The image processing device according to claim 1 , wherein the classification unit classifies each of the ACS data in the k-space data corresponding to each of the plurality of frames into the plurality of groups by performing clustering on the ACS data in the k-space data corresponding to each of the plurality of frames.

5. the sensitivity map calculation unit includes a first neural network corresponding to the first group, which performs image processing on the ACS data classified into the first group to generate sensitivity calculation image group data for calculating the sensitivity map, and a second neural network corresponding to the second group, which performs image processing on the ACS data classified into the second group to generate sensitivity calculation image group data for calculating the sensitivity map, the sensitivity calculation image group data is data indicating the relative magnitude of sensitivity between each coil, 2. The image processing device according to claim 1, wherein the sensitivity map calculation unit generates the sensitivity map based on the sensitivity calculation image group data generated by the first neural network and the sensitivity calculation image group data generated by the second neural network.

6. The image processing device according to claim 5 , wherein the first neural network and the second neural network share parameters.

7. an acquisition step of acquiring k-space data obtained by undersampling using a plurality of coils and corresponding to each of a plurality of frames; a classification step of classifying ACS data in the k-space data corresponding to each of the plurality of frames into a plurality of groups; a sensitivity map calculation step of generating sensitivity maps corresponding to the plurality of coils based on data based on ACS data classified into a first group included in the plurality of groups and data based on ACS data classified into a second group included in the plurality of groups; an image generating step of generating an image corresponding to a first frame based on the sensitivity map and the k-space data corresponding to a first frame of the plurality of frames, and generating an image corresponding to the second frame based on the sensitivity map and the k-space data corresponding to a second frame of the plurality of frames; a regularization step of performing an image space regularization process on an image corresponding to the first frame and performing an image space regularization process on an image corresponding to the second frame; a data consistency processing step of performing data consistency processing on an image corresponding to the first frame to which the image space regularization processing has been applied, based on k-space data corresponding to the first frame and the sensitivity map, and performing data consistency processing on the image corresponding to the second frame to which the image space regularization processing has been applied, based on k-space data corresponding to the second frame and the sensitivity map; An image processing method comprising:

Images