Magnetic resonance electric characteristic tomography
The PCNN-EPT method addresses MREPT's numerical artifacts and inaccuracy by using a physically coupled neural network to optimize diffusion and convection coefficients, ensuring high-accuracy conductivity maps in diverse and noisy conditions.
Patent Information
- Application Number
- JP2023076845
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-08
- Publication Date
- 2024-11-20
AI Technical Summary
Existing magnetic resonance electrical property tomography (MREPT) methods face challenges with numerical artifacts and inaccuracy due to uniformity assumptions, requiring empirical selection of diffusion and convection coefficients, and lack generalizability and noise robustness, especially in medical applications where data collection is limited.
A physically coupled neural network (PCNN-EPT) that uses a data-driven mechanism to calculate diffusion and convection coefficients based on wave physics, incorporating a machine learning model to optimize these coefficients for improved reconstruction accuracy and generalizability.
PCNN-EPT provides magnetic resonance electrical property tomography with enhanced noise robustness and generalizability, achieving high-accuracy conductivity maps even in noisy environments and with varying sample types.
Smart Images

Figure 2024161803000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to magnetic resonance electrical property tomography, and in particular to physically coupled neural network magnetic resonance electrical property tomography, for the reconstruction of images using electrical properties as tissue contrast. [Background technology]
[0002] The electrical properties of human tissue (EPs; electrical conductivity σ, permittivity ε) are quantitative biomarkers that are useful for identifying various pathological conditions and their onset. Images using EPs as tissue contrast (conductivity maps, permittivity maps) may be applicable to the early diagnosis of cancerous tissue.
[0003] Magnetic resonance electric property tomography (MREPT) is an imaging diagnostic technique that reconstructs EPs using radio frequency magnetic fields (RF fields) measured by an MRI device. Conventionally, MREPT is known to reconstruct conductivity maps by numerically solving a standard analytical model (std-EPT) based on Maxwell's equations. Although such analytical numerical solutions for MREPT are simple, they have problems such as boundary artifacts at tissue boundaries due to the inaccuracy and numerical errors of the assumption of uniformity behind the analytical model, i.e., the assumption that the conductivity is constant, resulting in low reconstruction accuracy.
[0004] Non-Patent Document 1 proposes a method for solving the reconstruction problem flexibly by removing the assumption of uniformity from an analytical model based on the Maxwell equations. In detail, the reconstruction problem is formulated as a convection-reaction partial differential equation (cr-EPT) and is discretized and solved for a region of interest (ROI), which results in unknowns that can be solved without spatial constraints, making the assumption of uniformity unnecessary. However, with this method, there is a risk of numerical artifacts occurring when the resolution of the B1 field measured by the MRI device is low.
[0005] The effect of numerical artifacts in the cr-EPT method can be reduced by adding a diffusion term that acts as a viscous regularizer, but if the diffusion coefficient in the diffusion term is large, the diffusion term dominates the cr-EPT, blurring the distribution of EPs and reducing contrast. Therefore, to achieve a trade-off between suppressing the rapid fluctuations in the reconstruction of EPs and reducing contrast, it is necessary to give an appropriate diffusion coefficient according to the sample.
[0006] In Non-Patent Document 2, a method is proposed to add a convection coefficient to adjust the weight of the convection term in addition to the diffusion coefficient to cr-EPT in order to correct boundary artifacts. In detail, Non-Patent Document 2 proposes a method of adding a single convection coefficient empirically selected for the entire ROI, a so-called global convection coefficient map (d∇ 2 ρ+c∇φ·∇ρ+(∇ 2 However, the convection coefficient may require different values in different regions (boundary or non-boundary) within the ROI to improve the reconstruction accuracy, so not only must an appropriate convection coefficient be given for each sample, but the coordination with the diffusion coefficient must also be considered. [Prior art documents] [Non-patent literature]
[0007] [Non-Patent Document 1] FS Hafalir, OF Oran, N. Gurler, and YZ Ider, “Convection-reaction equation based magnetic resonance electrical properties tomography (crmrept),” IEEE Trans Med Imaging, vol. 33, no. 3, pp. 777-793, Mar. 2014. [Non-Patent Document 2] YZ Ider and MN Akyer, “Properties and implementation issues of phase based cr-MRECT for conductivity imaging,” in Proc. Intl. Soc. Mag. Reson. Med. 28, 2020, p. 3190. [Non-Patent Document 3] S. Mandija, EF Meliad`o, NRF Huttinga, PR Luijten, and CAT van den Berg, “Opening a new window on mr-based electrical properties tomography with deep learning,” Scientific Reports, vol. 9, no. 1, Dec. 2019. Summary of the Invention [Problem to be solved by the invention]
[0008] As mentioned above, the artifacts associated with the analytical MREPT numerical solution can be reduced by adding stabilization coefficients, i.e., appropriate diffusion and convection coefficients depending on the sample, to the analytical model. However, because these stabilization coefficients have been selected empirically, it has been difficult to improve the generalizability of the analytical model.
[0009] In addition, in Non-Patent Document 3, in order to avoid artifacts associated with the analytical numerical solution of MREPT described above, B is calculated from training samples without using Maxwell's equations. +A data-driven end-to-end neural network (NN)-based MREPT (NN-EPT) has been proposed that learns a function that maps one field to EPs. However, the NN-EPT in Non-Patent Document 3 is a physics-uncoupled approach that does not capture the wave physics recognized in analytical MREPT, and therefore requires a huge amount of training sample data to ensure robustness to new samples, i.e., generalizability. It is difficult to collect a huge amount of data, especially in the medical field. In addition, robustness against various types of noise is required in MRI to improve reconstruction accuracy.
[0010] The present invention has been made in view of the above-mentioned problems, and has an object to provide a magnetic resonance electrical property tomography that is excellent in noise robustness and generalizability. [Means for solving the problem]
[0011] The inventors have conducted intensive research into the above-mentioned problem and have discovered that by constructing a physically-coupled neural network electrical property tomography that includes a first means for reconstructing a conductivity map using a physical model based on wave physics with the diffusion coefficient ρ and the convection coefficient β as coefficients, and a second means for learning the diffusion coefficient ρ and the convection coefficient β using a data-driven machine learning model that utilizes parameters of the physical model, it is possible to achieve magnetic resonance electrical property tomography that is highly noise robust and generalizable, thereby completing the present invention.
[0012] In addition, the physical model is β(∇φ tr ∇γ)+γ∇ 2 φ tr -ρ∇ 2 γ=ωμ0, and the parameters are φ tr , ∇φ tr , ∇ 2 φ tr Here, φ tr is the normalized transmit phase. γ is the inverse of the electrical conductivity σ (γ=1 / σ). ω is the Larmor frequency. μ0 is the magnetic permeability of a vacuum.
[0013] The machine learning model may also be a neural network.
[0014] Also, the diffusion coefficient ρ and the convection coefficient β may be obtained by different neural networks.
[0015] The apparatus may further comprise a third means for calculating a difference between the conductivity map reconstructed by the first means and a ground truth conductivity map, and feeding the difference back to the second means.
[0016] Furthermore, the diffusion coefficient ρ and the convection coefficient β may each be a global coefficient map or a local coefficient map.
[0017] In addition, the diffusion coefficient ρ and the convection coefficient β may be selectable between the global coefficient map and the local coefficient map depending on the noise condition of the test sample. Effect of the Invention
[0018] The magnetic resonance electrical properties tomography of the present invention provides a magnetic resonance electrical properties tomography with excellent noise robustness and generalizability, specifically, it is possible to reconstruct a conductivity map with high accuracy even in cases where the test sample deviates from the training sample or in a noisy environment. [Brief description of the drawings]
[0019] [Figure 1] FIG. 1 is a diagram showing the configuration of the magnetic resonance electrical property tomography of the present invention, i.e., physically coupled neural network electrical property tomography (PCNN-EPT). [Diagram 2] (a) is the artificial irregular shape (AIG) dataset, (b) is the digital human head (DHH) dataset, and (c) is a test sample dataset that is a modified version of the DHH dataset. [Diagram 3](a) is a phantom with a birdcage coil mounted on a cylindrical sample with an AIG, and (b) is a phantom with a birdcage coil mounted on a DHH. [Figure 4] FIG. 1 shows the conductivity maps reconstructed in generalization experiments 1 to 3 by the non-learning numerical analysis models (std-EPT, cr-EPT), the physically unbound direct learning model (NN-EPT), and the physically bound learning models according to the present invention (PCNN-global-EPT, PCNN-local-EPT). [Diagram 5] This figure shows the local coefficient maps, normalized local coefficient maps, and line profiles generated by PCNN-local-EPT in the sixth column of Figure 4 for five types of experiments (a) to (e) corresponding to generalization experiments 1 to 3. [Figure 6] 13 is a graph showing the average and standard deviation of the SSIM values of the conductivity maps reconstructed by the models in cases (1) to (9) in five types of experiments (a) to (e) corresponding to generalization experiments 1 to 3. [Figure 7] 13 is a graph showing the average and standard deviation of the SSIM values of the conductivity maps reconstructed by each model while decreasing the signal-to-noise ratio (SNR) of various samples from 200 to 0 to examine noise robustness in generalization experiments 1 to 3. [Figure 8] 13 shows the conductivity maps reconstructed by each model for training sample #10 of the AIG dataset at four SNR levels (SNR=∞, 150, 100, 50). [Figure 9] The first row shows the conductivity map of training sample #4 of the AIG dataset, divided into boundary and non-boundary parts. The second row shows the local diffusion coefficient map generated by dividing the three diagonal circular structures and their boundary and non-boundary parts using PCNN-local-EPT. The third row shows the local convection coefficient map generated by dividing the three circular structures and their boundary and non-boundary parts. [Figure 10](a) is a graph comparing the distribution of the average diffusion coefficient against the conductivity value of the boundary and non-boundary parts of the test sample, (b) is a graph comparing the distribution of the average convection coefficient, (c) is a graph comparing the tabulated results of the diffusion coefficient against the conductivity value of the boundary and non-boundary parts of the test sample, and (d) is a graph comparing the tabulated results of the convection coefficient. [Figure 11] The first row shows a comparison of the reconstruction results of phantom scans trained with the DHH dataset, and the second row shows a comparison of the reconstruction results of phantom scans trained with the AIG dataset. [Figure 12] (a)–(e) are graphs showing the mean and standard deviation of SSIM values of reconstructed conductivity maps under various outer boundary setting (OBS) conditions in generalization experiments 1–3. [Figure 13] (a) to (e) are graphs showing the average and standard deviation of the SSIM values of the conductivity maps reconstructed in generalization experiments 1 to 3, using either MSE or SSIM as the loss function during training of the black-box NN-EPT, and using φtr as the input, and using ∇2φtr, ∇φtr, and φtr. [Figure 14] (a) is a graph showing the loss curves during training for PCNN-local-EPT, (b) for PCNN-global-EPT, (c) for NN-EPT, and (d) for PCNN-local-EPT. Note that (d) shows the operation of the learning rate reduction mechanism (see solid line) that prevents the loss curve from getting stuck in a local minimum value (see dashed line) during training for PCNN-local-EPT and reduces the loss curve. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0020] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the present invention can be embodied in many different forms and is not limited to the following embodiments and examples.
[0021] As shown in FIG. 1, the magnetic resonance electrical properties tomography in this embodiment is a so-called physically coupled neural network electrical properties tomography (PCNN-EPT) that includes a stabilized cr-EPT as a first means for reconstructing a conductivity map using a physical model based on wave physics to which a diffusion coefficient ρ and a convection coefficient β are added as stabilization coefficients, and a coefficient generating NNs (neural networks) as a second means for learning the diffusion coefficient ρ and the convection coefficient β using a data-driven machine learning model that utilizes parameters of the physical model, and is capable of providing a magnetic resonance electrical properties tomography that is excellent in noise robustness and generalizability.
[0022] In addition, the PCNN-EPT in this embodiment reconstructs the conductivity map using a stabilized cr-EPT to which the diffusion coefficient ρ and convection coefficient β generated from the coefficient generation NNs are added, and the coefficient generation NNs learns to optimize the diffusion coefficient ρ and the convection coefficient β by feeding back to the coefficient generation NNs based on the difference between the reconstructed conductivity map and the ground truth conductivity map, and the coefficient generation NNs can reduce artifacts and reconstruct the conductivity map with high accuracy even in cases where the test sample deviates from the training sample or in a noisy environment. In other words, even if the PCNN-EPT does not know the ground truth of the diffusion coefficient ρ and the convection coefficient β for human tissue with a complex shape or an unknown sample, it is possible to generate an optimized diffusion coefficient ρ and a convection coefficient β from the coefficient generation NNs.
[0023] In addition, the PCNN-EPT in this embodiment includes a third means for calculating a loss function based on a structural similarity index measurement (SSIM) as the difference (reconstruction error) between the conductivity map reconstructed by the stabilized cr-EPT and the ground truth conductivity map, and performing feedback using the backpropagation method, and the two NNs (U-net) as data-driven machine learning models constituting the coefficient generating NNs are updated by the feedback. Note that the diffusion coefficient ρ and the convection coefficient β are generated separately from the two NNs constituting the coefficient generating NNs.
[0024] In addition, for the diffusion coefficient ρ and convection coefficient β added to the stabilized cr-EPT, there is an optimal combination of global and local coefficient maps depending on the noise conditions of the test sample. Therefore, the coefficient generation NNs can select either the global coefficient map or the local coefficient map for the diffusion coefficient ρ and convection coefficient β to be generated depending on the noise conditions of the test sample. Figure 1 shows the case where the diffusion coefficient ρ and convection coefficient β generated by the coefficient generation NNs are each a local coefficient map.
[0025] The "global coefficient map" is a scalar in which the diffusion coefficient ρ and the convection coefficient β are applied globally (to the entire ROI), in other words, a single-value coefficient, whereas the "local coefficient map" is a matrix in which the diffusion coefficient ρ and the convection coefficient β are applied locally to each pixel in the ROI.
[0026] The PCNN-EPT in this embodiment will be described in detail below.
[0027] (Construction of stabilized cr-EPT) First, we describe the construction of the stabilized cr-EPT used in PCNN-EPT. Phase-based magnetic resonance electrical property tomography (MREPT) is a fast model due to the reduced MRI scan time. As shown in the following equation (1), the simplest form corresponds to the standard EPT formulation (std-EPT) and assumes uniform conductivity (∇γ=0) in the ROI. Furthermore, we assume that the transmit and receive phases are similar and that the transmit phase can be approximated as half of the transmit and receive phase. This assumption is known as the "half-phase assumption", and the impact of this assumption is not significant at conditions below 3 Tesla.
[0028]
number
[0029] Here, φ in the above formula (1) tr is the B +is the transmit phase from the field (normalized transmit phase), ω is the Larmor frequency, γ is the inverse of the conductivity σ (γ=1 / σ), and μ0 is the permeability of a vacuum. Note that due to the assumption of uniform conductivity, boundary artifacts occur in the conductivity map reconstructed by the above equation (1). Such boundary artifacts can be attenuated by solving for the spatial variation of the conductivity map as an unknown.
[0030]
number
[0031] The formulation of the convection-reaction partial differential equation (cr-EPT) shown in equation (2) above is a method derived from Maxwell's equations. cr-EPT removes the assumption of uniformity of conductivity, but numerical artifacts occur in the solution due to numerical instability associated with discretization over the ROI. cr-EPT introduces a diffusion term (ρ∇ 2 γ), and we propose to add a convection coefficient β to the convection term to produce a more accurate contrast. This leads to the stabilized cr-EPT shown in Equation (3) below, which is used for PCNN-EPT.
[0032]
number
[0033] (PCNN-EPT flow) Next, the flow of PCNN-EPT will be described. As shown in FIG. 1, the formulation of the physical model in PCNN-EPT is based on the stabilized cr-EPT, which is a phase-based analytical model shown in the above formula (3). The input of PCNN-EPT is the normalized transmission phase (φ tr ), its gradient (∇φ tr ), and its Laplacian (∇ 2 φ tr) PCNN-EPT is composed of two main parts: a stabilized cr-EPT and coefficient generation NNs.
[0034] The coefficient generation NNs is a two-stream NN structure, and each individual NN generates either the diffusion coefficient ρ or the convection coefficient β, which are the stabilization coefficients. The generated diffusion coefficient ρ and convection coefficient β are passed to the stabilization cr-EPT, and a conductivity map is calculated as the output. The reconstructed (predicted) conductivity map is compared with the ground truth conductivity map, a loss function is calculated based on SSIM, and feedback is performed using the backpropagation method (see the dashed arrows in Figure 1), updating the parameters of the two individual NNs.
[0035] In detail, the coefficient generating NNs in this embodiment learns to generate optimized stabilization coefficients (diffusion coefficient ρ and convection coefficient β) from the difference (reconstruction error) between the input and the conductivity map without explicitly specifying the ground truth of the diffusion coefficient ρ and convection coefficient β, which are unknown in the MREPT research area. Note that the U-net that constitutes the coefficient generating NNs is a general model for semantic segmentation, so a detailed description will be omitted.
[0036] The output of the NN may be optimized by various settings, for example bounded by a sigmoid activation function at the output so that the stabilization coefficient does not overwhelm the stabilization cr-EPT in equation (3) above. Specifically, the diffusion coefficient ρ is set to 1×10 because the diffusion term (viscosity term) may cause the values of neighboring pixels to be too strongly coupled, attenuating all spatial variations. -6The value is limited to 0.1. The maximum value of the diffusion coefficient ρ is selected by grid search to select a global value that does not cause numerical artifacts in the sample, and then the global value is doubled to increase the degree of freedom of the NN. The minimum value of the diffusion coefficient ρ is a non-zero value to prevent the diffusion coefficient ρ from being deleted from the following equation (4). On the other hand, the convection coefficient β is limited to 0.01 to 5 to prevent the convection term from overtaking the reconstruction during learning and the formulation from being overshadowed. The maximum value of the convection coefficient β is limited according to twice the global convection coefficient value to increase the degree of freedom of the NN, and the minimum value of the convection coefficient β is selected to prevent the NN from getting stuck in a local minimum by deleting the convection term from the following equation (4). Note that the following equation (4) shows the optimization formulation in the i-th iteration.
[0037]
number
[0038] Here, ρ θi and β θi is the NN parameter θ that belongs to the parameter space (θ). i are the diffusion coefficient ρ and the convection coefficient β at the i-th iteration when given. L is the loss function defined in (5) using the SSIM of the following equation (6). θi-1 ,β θi-1 ) is the conductivity predicted in the previous iteration. σ is the ground truth conductivity. In the above formula (4), ^σ(ρ θi-1 ,β θi-1 ) is compared to σ through the loss function L. The NN parameters θ i is optimized in the first training iteration to minimize the loss function L according to the gradient using the RMSprop optimization algorithm with a learning rate of 0.001 and momentum of 0.9.
[0039]
number
[0040]
number
[0041] where σ and ^σ are the ground truth conductivity and the reconstructed conductivity, respectively. μ is the mean value over a collective patch of pixels. SD is the standard deviation over the same patch. c1 and c2 are the prior coefficients. SSIM is chosen from various choices to construct a loss function to overcome very local artifacts that appear in the conductivity reconstructed by (3) above. This is because the calculation of μ and SD in (6) above is done over regions, i.e., over patches of pixels, which can reduce the impact of local artifacts. If a pixel-wise error measure such as the mean squared error (MSE) were used, these local artifacts would bias the training, leading to high diffusion values (ρ ≈ 0.1), which would reduce the overall spatial variation and lead to the reconstruction of a conductivity map with very low contrast. EXAMPLES
[0042] For each model of magnetic resonance electrical property tomography, three generalization experiments in which the test samples gradually deviate from the training samples, and a noise robustness experiment were performed using two simulated data sets.
[0043] In each experiment, we compared PCNN-local-EPT (see Figure 1), a physically-coupled learning model, where the stabilization coefficients are generated as local coefficient maps, which are matrices applied locally on a pixel-by-pixel basis in the ROI, with PCNN-global-EPT, where the stabilization coefficients are scalars applied globally in the ROI. In PCNN-global-EPT, two parallel two-layer fully connected architectures are used to generate the global coefficient maps from the coefficient generation NNs. In the first layer, the input is compressed to the square root of the phase image size, and in the second layer, it is further compressed to a scalar output.
[0044] In addition to the two PCNN-EPTs mentioned above, we also compared the non-learning numerical analysis models std-EPT and cr-EPT, and the direct learning model NN-EPT, which is a physics-uncoupled model. The loss function of the NN in NN-EPT is based on the mean squared error (MSE). This is because NN-EPT can benefit more from MSE than SSIM due to the absence of local artifacts in the end-to-end method (see Figure 13).
[0045] The stabilized cr-EPT used in PCNN-EPT is based on the formulation of the above formula (3), and receives stabilized coefficients (local coefficient map or global coefficient map) that are the output of the coefficient generation NNs, and generates a conductivity map as an output. In this embodiment, PCNN-EPT focuses on two-dimensional reconstruction to reduce calculation costs, but it can also be extended to three-dimensional reconstruction. The spatial resolution is 2 mm in the x, y, and z directions. The central difference of each point in the ROI is taken, and a mesh is created using the finite difference method to solve the formulation of the above formula (3). In detail, the formulation of the phase stabilized cr-EPT after discretization is as shown in the following formula (7). Here, i and j are the index numbers of the pixels in the x and y directions, respectively, and φ tr The derivatives are calculated using a second-order Savistky-Golay filter. By discretizing Eq. (7) below, we obtain a set of linear equations for the ROI, σ i,j We can form a system Ax=b where x is the unknown sequence.
[0046]
number
[0047] Here, we explain the simulated dataset used for each experiment. EPT algorithm / network φ tr , ∇φ tr , ∇ 2 φ tr The inputs are the simulations of different phantoms B + Prepare based on 1. Simulation B+ One data set is generated by Sim4Life (ZMT AG, Zurich). For the created phantom (see Fig. 3), the transmit coil is a high-pass 16-run birdcage coil shielded by a 50 cm diameter shield, with an inner diameter of 28 cm for head imaging. The birdcage coil is excited with two ports geometrically separated by 90°, with a harmonic excitation of 127.78 MHz, and a phase shift of 90° between the ports in orthogonal mode. The complex B associated with a particular slice for reconstruction is + One field is extracted, and the transmit / receive phase φ tr and the corresponding derivatives are used as inputs for the model. To generate the training samples required for the data-driven model, we mounted different phantoms (see Fig. 3(a) and (b)) on the coil and further measured the complex B + 1 field was chosen, which reduces the end-ring effect and suppresses the value of δBz, so that δBz ≈ 0 can be assumed.
[0048] (Training Dataset) In this example, two types of training datasets of numerically simulated phantoms are created: a 2D artificial irregular shape (AIG) dataset shown in Fig. 2(a) and a digital human head (DHH) dataset generated from the virtual population 3.0 shown in Fig. 2(b).
[0049] The AIG dataset consists of 30 samples in total, each with a different foreground embedded in a cylindrical main structure with a diameter of 16 cm and a length of 24 cm. The length of the cylinder is set to suppress the end effect. The size and conductivity (0.1-2.5 S / m) of the structures vary. This AIG dataset is created to present not only the main structure but also samples with different conductivity values to provide diverse information to the reconstruction model.
[0050] The DHH dataset consists of 30 samples from 15 different samples with ±40% conductivity variation. The 15 samples correspond to eight male digital models ("Duke") and seven female models ("Ella"). From each model, two consecutive slices from the ROI are selected to reconstruct a total of 30 samples (#31-46 for "Duke" and #47-60 for "Ella"). The male and female models have different heights, but the ROI is set at a constant height, so that the anatomical structures focused on are different for the male and female models. This DHH dataset is created to present samples with anatomical structures and complex boundaries similar to those in clinical settings.
[0051] (Test dataset with modifications) The test dataset used in this example is a set of 16 samples that have been modified by adding variations to some of the samples in the DHH dataset, such as rotation (within 15° left and right) and the addition of masses that mimic pathological tissue. The masses that mimic pathological tissue are spheres with diameters of 0.8 to 2.4 cm placed at random positions in the brain within the ROI, and the conductivity of the white matter is increased to 60 to 120% to mimic brain tumors. These modified samples are created to present variations that may appear in clinical practice.
[0052] (Generalization experiment) To investigate the generalizability of PCNN-EPT, we conduct generalization experiments 1 to 3. The test samples are designed with three levels of deviation from the training samples, with generalization experiment 1 having the smallest deviation, followed by generalization experiment 2 and generalization experiment 3.
[0053] In generalization experiment 1, the AIG dataset in Figure 2(a) and the DHH dataset in Figure 2(b) are split using a 5-fold cross-validation method. In this 5-fold cross-validation, we apply a training and testing split based on "leave-one-group-out", i.e., one group for testing and four groups for training. Thus, for each dataset, five training schemes (one for each test group) were performed. The average reconstruction accuracy of the five test groups summarizes the performance of the model on the dataset.
[0054] In generalization experiment 2, the model using the training group of the DHH dataset in generalization experiment 1 (Figure 2(b)) was tested using 16 test samples obtained by modifying the DHH dataset, such as by rotation and adding pathological tissue, as shown in Figure 2(c).
[0055] In generalization experiment 3, we study dataset generalization, i.e., responses in two cases that are challenging for data-driven PCNN-EPT and NN-EPT: the presented test samples are completely different from the samples used during training. In particular, in generalization experiment 3(A), we test the training group from generalization experiment 1 and the models trained on the DHH dataset in Figure 2(b) on the corresponding test group from generalization experiment 1. In generalization experiment 3(B), we test the models trained on the training group from the AIG dataset in Figure 2(b) on the modified test samples from the DHH dataset in Figure 2(c).
[0056] (Noise robustness experiment) To investigate the noise robustness of PCNN-EPT, we conducted a noise robustness experiment. Using the model trained in the generalization experiment described above, we performed a noise robustness test by adding noise to the corresponding test samples from each experiment in order to evaluate it without further training. In detail, we performed a noise robustness test by adding noise to the corresponding test samples from each experiment in order to evaluate the noise robustness without further training. tr ) is contaminated with additive zero-mean Gaussian noise, and the added noise is the double angle B+ B generated by 1 search method + It is calculated by simulating the noise of a first order image. Therefore, the standard deviation (SD) of the noise, SDφ tr The relationship between the signal-to-noise ratio (SNR) of the grade image and the image quality is expressed as follows:
[0057]
number
[0058] The noise level was set to the range of clinical MRI equipment, and the noisy transmission and reception phase (φ tr ) and its derivatives are used as the inputs of the models. Also, to avoid noise explosion, a low-pass Gaussian filter with standard deviation 1 is applied as preprocessing before numerical differentiation. This process is applied equally to all models being compared when noisy samples are tested.
[0059] (Examination of local coefficient maps) Both the diffusion coefficient map and the convection coefficient map, which are the stabilization coefficients generated in PCNN-EPT, are very important, especially near the tissue boundary regions. To analyze the variation of the coefficient maps generated in PCNN-EPT, the coefficient maps are classified according to the boundary and non-boundary parts of the conductivity map. The conductivity boundary parts are determined by calculating the sum of the gradient values of the conductivity in the x and y directions, and matching the gradient value to the conductivity difference of the boundary of each test sample, i.e., the absolute value of the difference between the two conductivities that constitute the boundary part.
[0060] The gradient values near the boundary are modeled as a Gaussian distribution, and the boundary region is defined as the region within the range of the mean value ±1 standard deviation to capture the fluctuations before and after the boundary of the coefficient map. The non-boundary conductivity region is determined by separating each test sample according to its conductivity value and deleting the previously defined boundary conductivity region. Using the boundary and non-boundary regions of the conductivity map, each coefficient map is divided into two: a boundary coefficient map and a non-boundary coefficient map.
[0061] (phantom scan) The phantom scan uses the Magphan® SMR-170 MRI phantom. The background is filled with saline (2.5 g / L NaCl, estimated conductivity 0.5 S / m), and the foreground is filled with saline (10 g / L NaCl, estimated conductivity 2.0 S / m). The measurements are performed on a 3T “Skyra” MR scanner (Siemens, Erlangen, Germany) installed at the Hubei Provincial Cancer Hospital, using a standard orthogonal body coil. A multiple gradient echo (m-GRE) sequence is applied to measure the transmit and receive phase. The imaging parameters are TR=84 ms, first echo time (TE)=1.91 ms, ΔTE=2.67 ms, flip angle=15°, and number of echoes=6. The scan resolution is 1.06×1.06×2 mm. The unwrapped phase map at the echo time is fitted to the transmit and receive phase by applying temporal fitting to obtain the phase measurement at TE=0 as shown in the following equation (9). This is used to reconstruct the electrical conductivity: where r is the phase, TE is the echo time, B0 is the constant uniform magnetic field, and B1 is the magnetic field across B0.
[0062]
number
[0063] The phase map is also filtered for the conductivity reconstruction as described in the noise robustness experiment above: a 3 mm thick wall between the background and foreground nulls the pixels between them, which introduces artifacts into the reconstruction, so the filtering process smoothes out the null pixels.
[0064] Figure 4 shows the conductivity maps reconstructed by the non-learning numerical analysis models (std-EPT, cr-EPT), the physically uncoupled direct learning model (NN-EPT), and the physically coupled learning models (PCNN-global-EPT, PCNN-local-EPT) in generalization experiments 1 to 3, together with the SSIM values of the reconstructed conductivity maps. Figure 5 also shows the local coefficient maps generated by PCNN-local-EPT, as well as the normalized local coefficient maps and line profiles.
[0065] The first column in Figure 4 is the ground truth conductivity map. The second to sixth columns in Figure 4 are the conductivity maps reconstructed by std-EPT, cr-EPT, NN-EPT, PCNN-global-EPT, and PCNN-local-EPT in generalization experiments 1 to 3, respectively. The seventh and eighth columns in Figure 5 (see the numbers in the figure) are the local diffusion coefficient map and local convection coefficient map used in PCNN-local-EPT in the sixth column in Figure 4, where each pixel has a learned diffusion coefficient ρ or convection coefficient β. The ninth to twelfth columns in Figure 5 are the normalized local diffusion coefficient map and local convection coefficient map and their respective line profiles.
[0066] In detail, the local diffusion coefficient map and the local convection coefficient map shown in the 7th and 8th columns of Fig. 5 are normalized to the maximum values of 0.1 and 1, respectively, to improve their visibility, and are shown in the 9th and 11th columns of Fig. 5. The line profiles of the local diffusion coefficient map and the local convection coefficient map shown in the 10th and 12th columns of Fig. 5 are shown together with the line profile of the normalized ground truth conductivity map of the background (see dashed line). Note that the line profile of the local diffusion coefficient map is magnified by 5 times to visualize its spatial shift.
[0067] In addition, the first and second lines of Figures 4 and 5 show the conductivity maps reconstructed in generalization experiment 1 using the 5-fold cross-validation method. The first line shows the result when sample #26 from the AIG dataset shown in Figure 2(a) was used as a test sample, and the second line shows the result when sample #33 from the DHH dataset shown in Figure 2(b) was used as a test sample. The third line shows the conductivity map reconstructed in generalization experiment 2, which shows the result when sample #75, which is a modified version of the DHH dataset shown in Figure 2(c), was used as a test sample. The fourth and fifth lines show the conductivity maps reconstructed in generalization experiment 3, which is the most difficult experiment in which the test data and training data are completely different. The fourth line for generalization experiment 3(A) shows the result when sample #26 from the AIG dataset shown in Figure 2(a) was used as a test sample, and the fifth line for generalization experiment 3(B) shows the result when sample #75, which is a modified version of the DHH dataset shown in Figure 2(c), was used as a test sample.
[0068] Moreover, the sixth line in Fig. 4 shows the correlation coefficient (CC) between the line profile at the center of mass of the largest additional tissue added in the third line and the corresponding ground truth.
[0069] Figures 6(a)-(e) show the average and standard deviation of the reconstruction accuracy for the generalization experiments shown in the first to fifth rows of Figure 4, using different cases, namely, case (1) std-EPT, case (2) cr-EPT, case (3) NN-EPT, case (4) PCNN-global-EPT, and case (5) PCNN-local-EPT. In addition, generalization experiments were also performed for cases (6)-(9) to examine the roles of the global coefficient map and the local coefficient map. Cases (6) and (7) show the average and standard deviation of the reconstruction accuracy for PCNN-global-EPT and PCNN-local-EPT, respectively, when only the diffusion coefficient ρ is learned and the convection coefficient β is set to 1 (fixed global coefficient map), case (8) shows the average and standard deviation of the reconstruction accuracy when the global diffusion coefficient map and the local convection coefficient map are learned in combination, and case (9) shows the average and standard deviation of the reconstruction accuracy when the local diffusion coefficient map and the global convection coefficient map are learned in combination.
[0070] The reconstruction results in the generalization experiment will be explained in more detail. In the following explanation, figure X(i,j) means to refer to row i, column j in figure X.
[0071] (Generalization experiment 1) Figure 4 (1,1-6) shows the reconstruction results of test sample #26 from the AIG dataset, and Figure 4 (2,1-6) shows the reconstruction results of test sample #33 from the DHH dataset. It can be seen that the conductivity maps in columns 4-6 reconstructed by the learning method (learning model) have significantly less artifacts than the conductivity maps in columns 2-3 reconstructed by the non-learning method (analysis model). In addition, comparing columns 5 and 6, it can be seen that the reconstruction accuracy of PCNN-global-EPT (Figure 4 (1-2,5)) is lower than that of PCNN-local-EPT (Figure 4 (1-2,6)). Furthermore, comparing the fourth and sixth columns, for test sample #26 from the AIG dataset shown in the first row, PCNN-local-EPT (Fig. 4(1,6)) shows higher reconstruction accuracy than NN-EPT (Fig. 4(1,4)), while for test sample #33 from the DHH dataset, which has less geometric variation, PCNN-local-EPT (Fig. 4(2,6)) shows lower reconstruction accuracy than NN-EPT (Fig. 4(2,4)). In addition, by looking at the line profiles of the local diffusive flow coefficient map and the local convective flow coefficient map in the tenth and twelfth columns, respectively, it can be confirmed that there is variation corresponding to the boundaries.
[0072] Figures 6(a) and (b) show the SSIM values, i.e., the average and standard deviation of the reconstruction accuracy, of generalization experiment 1 using the 5-fold cross-validation method for the AIG dataset and the DHH dataset, respectively. Comparing cases (1) to (5) in Figures 6(a) and (b), it can be confirmed that the learning method (cases (3) to (5)) has a higher reconstruction accuracy than the non-learning method (cases (1) to (2)). In addition, as shown in Figures 6(a) and (b), when the sample is changed from the highly diverse AIG dataset in Figure 2(a) to the low-diversity DHH dataset in Figure 2(b), it can be confirmed that the reconstruction accuracy is lower for both PCNN-global-EPT (case (4)) and PCNN-local-EPT (case (5)).
[0073] In addition, in both Figures 6(a) and (b), it can be seen that in generalization experiment 1, the test samples do not deviate from the training samples, and therefore NN-EPT (case (3)) exhibits the highest reconstruction accuracy among the learning methods.
[0074] In addition, in both Fig. 6(a) and (b), comparing the results of cases (6) and (7) with cases (4) and (5), it can be confirmed that there is almost no decrease in reconstruction accuracy when only the diffusion coefficient ρ is learned as a global diffusion coefficient map or a local diffusion coefficient map and the convection coefficient β is 1 (fixed global coefficient map). Furthermore, comparing the results of case (4) where both the diffusion coefficient ρ and the convection coefficient β are learned as global coefficient maps, cases (8) and (9) where one of the coefficients is learned as a global coefficient map, and case (5) where both coefficients are learned as local coefficient maps, it can be confirmed that the cases where at least one of the diffusion coefficient ρ and the convection coefficient β is learned as a local coefficient map lead to high reconstruction accuracy, and among these, case (5) where both coefficients are learned as local coefficient maps has the highest reconstruction accuracy.
[0075] (Generalization experiment 2) Test sample #75 used in generalization experiment 2 is a sample that has been modified by adding rotation and spherical tissue (boxed area in Figure 4 (3,1)) that mimics a brain tumor to a sample in the DHH dataset, and is more difficult to reconstruct than generalization experiment 1. As shown in the line profiles in Figure 4 (6,2-6), all of the non-learning methods (Figure 4 (6,2-3)) show wavy artifacts, but all of the learning methods (Figure 4 (6,4-6)) show that the occurrence of wavy artifacts is suppressed. Among the learning methods, the line profile of PCNN-global-EPT in Figure 4 (6,5) has a smaller waveform than the line profile of the ground truth conductivity map (see dashed line), and as shown in Figure 4 (3,5), the contrast of cerebrospinal fluid (CSF) tissue with a conductivity of 2.14 S / m cannot be sufficiently brought out during reconstruction.
[0076] In addition, the NN-EPT line profile in Figure 4(6,4) shows that the contrast is overestimated near the beginning of the line profile, and the added spherical tissue mimicking the brain tumor shows an unstable line profile for the added mass, as shown by the slope of the reconstructed conductivity. On the other hand, the PCNN-local-EPT results in Figure 4(3,6) and Figure 4(6,6) show that the contrast of the CSF tissue is only slightly overestimated, and the contrast of the added spherical tissue mimicking the brain tumor is appropriately reconstructed. In addition, the arrows in the profiles of the conductivity map and local diffusion coefficient map in Figure 4(6,6) and Figure 4(3,10) show that the local diffusion coefficient map is reconstructed adaptively to the added spherical tissue mimicking the brain tumor by increasing the value near the boundary of the added spherical tissue mimicking the brain tumor and smoothing the transition to the surrounding white matter tissue.
[0077] As shown in Fig. 6(c), cases (1) to (5) show the same tendency as Fig. 6(b), but the major difference is that the reconstruction accuracy of NN-EPT (case (3)) and PCNN-local-EPT (case (5)) is decreased, and the reconstruction accuracy of NN-EPT is particularly decreased, so the reconstruction accuracy of NN-EPT and PCNN-local-EPT are almost the same. In addition, when using a global diffusion coefficient map in PCNN-global-EPT, the effect of setting the convection coefficient β to 1 is small when comparing cases (4) and (6), and the difference in reconstruction accuracy is negligible. However, when using a local diffusion coefficient map, the reconstruction accuracy of case (7) is decreased by setting the convection coefficient β to 1 when comparing cases (5) and (7). This confirms that a fixed value of either the diffusion coefficient ρ or the convection coefficient β is insufficient for the reconstruction of the conductivity map, and that it is important to learn both. Furthermore, by comparing cases (8) and (9) with cases (4) and (5), the importance of learning the diffusion coefficient ρ as a local diffusion coefficient map can be confirmed in order to improve the reconstruction accuracy.
[0078] (Generalization experiment 3) Generalization experiment 3 is the most difficult of the three generalization experiments because the test sample is the most distant from the training sample and higher generalization is required to reconstruct the conductivity map. Figure 4 (4,1-6), which corresponds to generalization experiment 3 (A), shows the result of using the same test sample #26 used in the 5-fold cross-validation experiment shown in Figure 4 (1,1-6). By directly comparing the first and fourth rows of Figure 4, it is possible to confirm the effect of the similarity of the test sample and the training sample on the reconstruction. It can be confirmed that PCNN-local-EPT (Figure 4 (4,6)) achieves reconstruction with the same high accuracy as Figure 4 (1,6) in generalization experiment 1. On the other hand, it can be confirmed that the reconstruction of NN-EPT (Figure 4 (4,4)) has a noticeable unexpected artifact that does not appear in generalization experiment 1 (Figure 4 (1,4)).
[0079] Note that, comparing Fig. 4(4,3) and Fig. 4(4,5), the reconstruction from PCNN-global-EPT (Fig. 4(4,5)) shows similar artifacts as the cr-EPT (Fig. 4(4,3)) due to the low values of the global diffusion coefficient maps generated from the coefficient generating NNs (ρG ≈ 0).
[0080] Figure 4 (5,1-6), which corresponds to generalization experiment 3 (B), shows the results using the same test sample #75 as in generalization experiment 2 shown in Figure 4 (3,1-6). A direct comparison of the third and fifth rows of Figure 4 confirms that the SSIM values are almost the same regardless of the non-learning and learning methods. It can be confirmed that the reconstruction of PCNN-local-EPT (Figure 4 (5,6)) has a 0.08 decrease in SSIM value compared to Figure 4 (3,6). It can also be confirmed that the reconstruction of NN-EPT (Figure 4 (5,4)) has a 0.01 decrease in SSIM value compared to Figure 4 (3,4).
[0081] As shown in Fig. 6(d), in the generalization test 3(A), among the learning methods of cases (3) to (5), the reconstruction accuracy by NN-EPT (case (3)) further decreased, becoming the same as that of PCNN-global-EPT (case (4)), and lower than that of PCNN-local-EPT (case (5)). In addition, when a global diffusion coefficient map is used in PCNN-global-EPT, the impact of setting the convection coefficient β to 1 is small even when comparing cases (4) and (6), and the reconstruction accuracy is about the same. In addition, when a local diffusion coefficient map is used in PCNN-local-EPT, the impact of setting the convection coefficient β to 1 is small even when comparing cases (5) and (7), and the reconstruction accuracy is about the same. Furthermore, when comparing cases (8) and (9) with cases (4) and (5), case (5), in which both the diffusion coefficient ρ and the convection coefficient β were learned as local coefficient maps, showed the highest SSIM value, while case (4), in which both were learned as global coefficient maps, showed the lowest SSIM value. This confirms the importance of coordinating both the diffusion coefficient ρ and the convection coefficient β as local coefficient maps in order to improve the reconstruction accuracy.
[0082] In addition, as shown in Fig. 6(e), in generalization experiment 3(B), among the learning methods, cases (3) to (5), the reconstruction accuracy of NN-EPT (case (3)) is reduced by 0.08 in SSIM value compared to the result in Fig. 6(c). The reconstruction accuracy of PCNN-global-EPT and PCNN-local-EPT is significantly reduced from cases (4) to (9), so they lag behind the reconstruction accuracy of NN-EPT (case (3)). However, it can be confirmed that the reconstruction accuracy is higher than that of the non-learning methods, cases (1) and (2). In addition, when comparing cases (6) and (7) with cases (4) and (5), the reconstruction accuracy is about the same, but it was confirmed that case (7), in which the convection coefficient is set to 1 and the diffusion coefficient ρ is learned as the local diffusion coefficient map, has the highest reconstruction accuracy. Furthermore, by comparing cases (4) and (5) with cases (8) and (9), it can be confirmed that when either the diffusion coefficient ρ or the convection coefficient β is learned as a global coefficient map, the reconstruction accuracy is comparable to that when a local diffusion coefficient map is learned, and the relative reconstruction accuracy is improved. In other words, PCNN-local-EPT is the only model that can maintain the contrast of the reconstructed conductivity map across generalization experiments 1 to 3.
[0083] (Noise robustness experiment) Figure 8 shows the conductivity maps of test sample #10 reconstructed from the AIG dataset of the above-mentioned generalization experiment 1 at four SNR levels (SNR = ∞, 150, 100, 50) in the noise robustness experiment. The conductivity maps reconstructed by the non-learning numerical analysis models (std-EPT, cr-EPT), the physically uncoupled direct learning model (NN-EPT), and the physically coupled learning models (PCNN-global-EPT, PCNN-local-EPT) are shown together with the SSIM values of the reconstructed conductivity maps.
[0084] In addition, the reconstruction by std-EPT in the second column shows that boundary artifacts are characteristic even without noise (SNR = ∞). The reconstruction by cr-EPT in the third column shows that numerical artifacts are observed even without noise, and as the noise increases, the reconstruction becomes filled with artifacts. The reconstruction by NN-EPT in the fourth column shows that NN-EPT has high reconstruction accuracy in the absence of noise, but as the noise increases, the reconstruction shows random artifacts. The reconstruction by PCNN-global-EPT in the fifth column shows that the reconstruction has low contrast, and the advantage of excellent noise robustness regardless of the SNR level. The reconstruction by PCNN-local-EPT in the sixth column shows high contrast without noise, and even when the SNR increases to the standard noise level of the MRI scanner (SNR = 100), the SSIM value of the reconstructed conductivity map remains at 0.39, which shows the highest reconstruction accuracy among all the models. It can be seen that in the reconstruction by PCNN-global-EPT, destructive artifacts appear when the noise level increases to SNR=50.
[0085] Figure 7 shows the average and standard deviation of the SSIM of the conductivity maps in generalization experiments 1 to 3 in the range of SNR = 200 to 0. Figures 7(a) and (b) show the results of generalization experiment 1, Figure 7(c) shows the results of generalization experiment 2, and Figures 7(d) and (e) show the results of generalization experiments 3(A) and (B). As shown in Figure 7(a), when the AIG dataset with diverse information is used, it can be confirmed that the learning methods (NN-EPT, PCNN-global-EPT, and PCNN-local-EPT) are more noise robust than the non-learning methods (std-EPT and cr-EPT). As shown in Figure 7(b), when the DHH dataset with less diversity is used, the learning methods maintain their superior noise robustness to the non-learning methods. However, when the SNR is higher than 70, the reconstruction accuracy (SSIM value) of PCNN-global-EPT is lower than that of the other two learning methods (NN-EPT, PCNN-local-EPT). However, as the SNR decreases, the decrease in reconstruction accuracy becomes very small, so that when the SNR is 70 or lower, the reconstruction accuracy is higher than that of the other two learning methods.
[0086] Furthermore, comparing the results of Fig. 7(b) and Fig. 7(c) for generalization experiment 2, it can be seen that although the reconstruction accuracy of both NN-EPT and PCNN-local-EPT declines, PCNN-local-EPT exhibits higher reconstruction accuracy than NN-EPT up to SNR = 50. Furthermore, as shown in Fig. 7(d) for generalization experiment 3(A), when the test sample deviates significantly from the training sample, PCNN-local-EPT maintains the same trend as generalization experiment 2, confirming its excellent noise robustness and generalizability to the diversity of test samples and training samples.
[0087] In addition, comparing Fig. 7(d) for generalization experiment 3(A) with Fig. 7(e) for generalization experiment 3(B), the reconstruction accuracy of PCNN-local-EPT decreases by about 0.1 across the noise spectrum, while NN-EPT and PCNN-global-EPT increase by about 0.1, resulting in PCNN-local-EPT having a lower reconstruction accuracy than NN-EPT and PCNN-global-EPT. This is mainly due to the erroneous coefficients generated by PCNN-local-EPT, which is affected by noise, and generates artifacts and reduces the reconstruction accuracy. The artifacts caused by the erroneous coefficients can be confirmed by comparing the conductivity maps reconstructed by PCNN-local-EPT shown in the sixth column of Fig. 8 with the conductivity maps reconstructed by NN-EPT and PCNN-global-EPT shown in the fourth and fifth columns.
[0088] (Consideration of diffusion coefficient and convection coefficient) Figure 9 shows the conductivity map of sample #4 of the AIG dataset divided into boundary and non-boundary parts (first row), and the diffusion coefficient ρ (second row) and convection coefficient β (third row) generated by PCNN-local-EPT for three circular structures on the diagonal and their boundary and non-boundary parts. As shown in Figure 9, even for structures with the same conductivity, it can be seen that the coefficients generated are different because different boundary effects need to be treated depending on the surroundings.
[0089] Figure 10 shows a comparison of the coefficients in the boundary and non-boundary parts of the test sample, with Fig. 10(a) showing the distribution of the average coefficients for the diffusion coefficient and Fig. 10(b) showing the distribution of the average coefficients for the conductivity values in the boundary and non-boundary parts for the convection coefficient. In order to compare the coefficient values in the boundary and non-boundary parts, two straight lines were fitted to the average values of the diffusion coefficients for the conductivity values in the boundary and non-boundary parts, respectively, using linear approximation.
[0090] In Fig. 10(a), the average diffusivity decreases with increasing conductivity for both fitted lines, but the fitted line for the non-boundary area (dashed line) has a steeper slope.In Fig. 10(b), the average convection coefficient for the boundary area is almost flat for the fitted line for the boundary area (solid line), but increases slightly with increasing conductivity for the fitted line for the non-boundary area (dashed line).
[0091] Figure 10(c) shows the diffusion coefficients of the boundary and non-boundary regions, and it can be seen that there is a large difference between the coefficients of the boundary and non-boundary regions. As shown in Figure 10(d), the average diffusion coefficient of the boundary region is lower than that of the non-boundary region, and a similar tendency can be seen for the convection coefficient.
[0092] (Phantom scan reconstruction) Figure 11 shows the results of reconstruction of a phantom scan. The first row shows the results of learning on the DHH dataset, and the second row shows the results of learning on the AIG dataset. In detail, the first column shows the ground truth, the second column shows the reconstruction results by std-EPT with global diffusion coefficient ρ = 0 and global convection coefficient β = 0, the third column shows the reconstruction results by cr-EPT with global diffusion coefficient ρ = 0 and global convection coefficient β = 1, and the fourth column shows the reconstruction results by stabilized cr-EPT with global diffusion coefficient ρ = 1 and global convection coefficient β = 1. The fifth column shows the reconstruction results by NN-EPT, the sixth column shows the reconstruction results by PCNN-global-EPT, and the seventh column shows the reconstruction results by PCNN-local-EPT. The eighth and ninth columns show the local diffusion coefficient map and the local convection coefficient map generated by PCNN-local-EPT in the seventh column, respectively.
[0093] In Fig. 11, the std-EPT and cr-EPT reconstructions in columns 2 and 3 show typical artifacts in both rows 1 and 2, respectively, and even the stabilized cr-EPT in column 4 shows artifacts related to conductivity transition at the foreground / background partition at very high diffusion coefficient values (ρ = 1). This is presumably due to the negative influence of the global diffusion coefficient. The NN-EPT reconstructions in column 5 show poor reconstruction accuracy in both rows 1 and 2, and the PCNN-global-EPT in column 6 shows a central artifact in both rows 1 and 2, similar to the stabilized cr-EPT in column 4, but the area of the artifact is reduced for row 2, where the training samples are generated by the AIG dataset. The PCNN-local-EPT reconstruction in column 7 shows the best reconstruction result, but overestimates the foreground conductivity, indicating that this is the only way to accurately characterize the ground truth structure.
[0094] Thus, as shown in the reconstruction results of the conductivity map in the second and third columns of Figure 4, PCNN-EPT (PCNN-global-EPT, PCNN-local-EPT) includes formulations derived from std-EPT, which has severe boundary artifacts due to the assumptions of derivation, and cr-EPT, which lacks flexibility in expression such as fixed coefficients, but it can gain considerable flexibility by introducing two multidimensional variable coefficients that need to be optimized by data, namely the stabilization coefficients (diffusion coefficient ρ and convection coefficient β). This is evident by comparing Figure 4 (5,6) with Figure 4 (2,3).
[0095] Furthermore, the optimization of the two stabilization coefficients by the coefficient generation NNs combined with the analytical model (stabilized cr-EPT) provides the generalization of PCNN-EPT, which is difficult to obtain by end-to-end learning, and this is an important point that is essential for the future clinical application of this method. As shown in the results of generalization experiment 2 (Figure 4 (3,1-6) and Figure 4 (5,1-6)), for the additional brain tumor-like spherical tissue (boxed area in Figure 4 (3,1)) that was not seen in the training sample, the analytical models (std-EPT, cr-EPT) were able to roughly reconstruct the additional tissue, but it can be confirmed that large errors occurred around the tissue, as shown by the low correlation coefficient (Figure 4 (6,2-3)) of the line profile across one of the additional tissues (see Figure 4 (3,1-6)). It can be confirmed that these errors were effectively handled as shown by the line profile (see Figure 4 (5,6)) with the help of the local diffusion coefficient map and local convection coefficient map optimized by the coefficient generation NNs of PCNN-local-EPT. Therefore, the combination of a physical model-based formulation based on wave physics and learning of stabilization coefficients in PCNN-EPT plays the most important role for high generalization in the reconstruction of conductivity maps.
[0096] This is also evidenced by the results of generalization experiment 3 (A), where the reconstruction by PCNN-local-EPT showed the highest SSIM value (Figure 4 (4,1-6)), and generalization experiment 3 (B), where the reconstruction by PCNN-local-EPT showed the second highest SSIM value after the reconstruction by NN-EPT (Figure 4 (5,1-6)). Furthermore, the reconstruction results of the phantom scan in Figure 11 support this assertion, where the reconstructions by std-EPT, cr-EPT and stabilized cr-EPT in columns 2-4 of Figure 11 show harmful artifacts, and the reconstruction by end-to-end NN-EPT in column 5 also shows very noticeable artifacts, while the reconstruction by PCNN-global-EPT in column 6 has significantly reduced artifacts compared to the reconstruction by stabilized cr-EPT in column 4 by balancing the global diffusion coefficient map and the global convection coefficient map. Furthermore, it can be seen that the reconstruction by PCNN-local-EPT in the seventh column is able to remove the artifacts that first appeared in the reconstruction by stabilized cr-EPT in the fourth column by combining the local diffusion coefficient map and the local convection coefficient map, demonstrating improved reconstruction accuracy.
[0097] The line profile of the local diffusion coefficient map in Figure 4 (4,10) shows smaller values near the boundary compared to Figure 4 (1,10), but compared to Figure 4 (4,2-5), it can be confirmed that it plays a role in reducing artifacts, as shown in Figure 4 (4,6). In addition, the line profile of the local diffusion coefficient map in Figure 4 (5,10) shows smaller values near the boundary compared to Figure 4 (3,10), but this can be confirmed by the fact that the peaks of the local diffusion coefficient map trained with the AIG dataset tend to be smaller, as shown in Figure 4 (1,10). In addition, in the function mapping of the physically uncoupled NN-EPT, the reconstruction is determined only by the training sample, so the reconstruction accuracy decreases as the test sample deviates from the training sample, as shown in case (3) of Figure 6 (a)-(e). PCNN-local-EPT also uses coefficient-generating NNs, so a similar effect appears. However, due to the formulation of a physical model, as shown in case (5) of Figures 6(a) to (d), the coefficients are robust to changes in the reused test samples, i.e., to new samples, as long as they do not produce extreme artifacts that cause the accuracy degradation shown in Figure 6(e).
[0098] The noise robustness of PCNN-local-EPT is comparable to that of NN-EPT, as shown in Fig. 7(a),(b) and Fig. 8(3,6) for generalization experiment 1. The noise robustness of PCNN-local-EPT is also higher than that of NN-EPT until the noise level increases to SNR=50 in the test samples, as shown in Fig. 7(c) for generalization experiment 2 and Fig. 7(d) for generalization experiment 3(A).
[0099] Here, NN-EPT generates more spurious noise-related artifacts as the noise level increases, as shown in the fourth column of Figure 8. This is presumably because NN-EPT maps the input to the reconstructed image based on the pattern matching rules acquired by the NN, so the noise is directly transferred to the reconstructed image in an interpolated manner, further reducing the generalizability. This problem is due to the B +The process of obtaining 1 can be exacerbated by multiple sources of noise.
[0100] On the other hand, in PCNN-local-EPT, when there is noise in the test sample, even if the noise affects both the stabilization coefficient and the analytical model (stabilized cr-EPT) terms, it can be suppressed by the stabilization coefficient in the end. Therefore, in generalization experiments 2 and 3 (A), it is speculated that PCNN-local-EPT was able to perform relatively accurate reconstruction even if the local diffusion coefficient map and local convection coefficient map could not be generated accurately because the test sample contained a structure that was partially or completely different from the training sample. However, it is speculated that the inaccurate local diffusion coefficient map and local convection coefficient map generated by PCNN-local-EPT not only generate destructive artifacts when the noise level increases to SNR = 50, as shown in the fourth column of Figure 8, but also cause the reconstruction accuracy of PCNN-local-EPT to be generally lower than that of PCNN-global-EPT, as shown in Figure 7(e) compared to Figure 7(d), which may have a negative impact on the reconstruction accuracy of some samples.
[0101] In this way, PCNN-EPT not only inherits the noise robustness of the formulated stabilized cr-EPT realized by the global coefficient map, but also significantly improves the reconstruction accuracy by optimizing the local coefficient map. In addition, the diffusion coefficient ρ and convection coefficient β generated by the coefficient generation NNs in PCNN-EPT can be selected to either the global coefficient map or the local coefficient map depending on the noise situation of the test sample, thereby improving noise robustness and generalization.
[0102] The role of the diffusion coefficient ρ and the convection coefficient β in PCNN-EPT is explained in detail. In PCNN-EPT, the diffusion coefficient ρ and the convection coefficient β are the diffusion terms (∇ 2 γ) and part of the convection term (∇φ tr , ∇γ).
[0103] The diffusion coefficient ρ determines how a pixel affects its neighbors. By applying a pixel-wise diffusion coefficient ρ, i.e., a local diffusion coefficient map, the viscous effect can be set according to the circumstances of each region for a good reconstruction of the conductivity map. Specifically, in the boundary area, it is set to a low value to generate an accurate conductivity transition, and then to a high value to avoid boundary artifacts. Meanwhile, in the non-boundary area, it is set to a high value to mitigate small numerical artifacts. This is well reflected in the results of tallying the diffusion coefficient values of the boundary and non-boundary areas in Figure 10(c) and the line profiles of the local diffusion coefficient maps learned from the two training samples (AIG dataset and DHH dataset) in generalization experiment 1 (see Figure 5(1,10) and Figure 5(2,10)).
[0104] By simultaneously learning the diffusion coefficient ρ and the convection coefficient β as local coefficient maps, an effective mutual reaction between the diffusion coefficient ρ and the convection coefficient β is achieved. This can be confirmed by comparing the learned local diffusion coefficient map (see Fig. 5 (1,10) and Fig. 5 (2,10)) with the line profile of the learned local convection coefficient map (see Fig. 5 (1,12) and Fig. 5 (2,12)). This can be understood by comparing the different cases in Fig. 6 (a) and (b) of generalization experiment 1, and it can be confirmed that case (5), in which both coefficients are learned as local coefficient maps, shows higher reconstruction accuracy than cases (4), (7), and (9), in which one of the coefficients is set or learned as a global coefficient map. This is also true in generalization experiment 2 (see Fig. 6 (c)), in which pathological tissues not seen in the training sample are added to the test sample. However, as shown in Fig. 6(d) and (e), in generalization experiments 3(A) and (B) where the test samples are significantly different from the training samples, this advantage does not exist and the accuracy difference between cases (5), (7), and (9) is smaller. This is because by making the convection coefficient β a fixed global coefficient map (β = 1), we can reduce the possibility of generating erroneous coefficients that cause problems in reconstruction, especially for test samples that are significantly different from the training samples.
[0105] As described above, these embodiments and examples provide a PCNN-EPT (physically coupled neural network electrical property tomography) that effectively combines a stabilization cr-EPT and a coefficient generation NNs (U-net) and flexibly optimizes two stabilization coefficients (diffusion coefficient ρ and convection coefficient β) to generate accurate reconstruction. This can achieve high reconstruction accuracy for two simulation datasets (AIG dataset and DHH dataset) even in difficult cases for end-to-end methods, such as when the test sample is the most deviated from the training sample or in a noisy environment, making it possible to provide a magnetic resonance electrical property tomography that is excellent in noise robustness and generalizability.
[0106] In addition, PCNN-local-EPT, which takes into account different needs for tissue boundaries and optimizes the two stabilization coefficients together, especially as a local coefficient map, can provide magnetic resonance electrical property tomography with excellent noise robustness and generalizability.
[0107] In the above-mentioned embodiments and examples, the aspect in which a conductivity map is reconstructed by magnetic resonance electrical property tomography has been mainly described, but the image reconstructed by magnetic resonance electrical property tomography may be an image using EPs as tissue contrast, for example, a map based on a dielectric constant map.
[0108] In addition, although an embodiment has been described in which a neural network is used as a data-driven machine learning model, the present invention is not limited to this, and other techniques such as ensemble learning may be used as the machine learning model.
[0109] In addition, as shown in Figure 12, the results of the average and standard deviation of the SSIM values of the conductivity maps reconstructed under various outer boundary setting (OBS) conditions in generalization experiments 1 to 3 show that if the OBS is accurate, the reconstruction accuracy of each model is slightly improved, but the reconstruction is accurate even without OBS.
[0110] We also used PyTorch (registered trademark), a machine learning library, to generate the backpropagation algorithm. This NN framework creates a computational graph that engraves operations into a hierarchical chart, allowing automatic differentiation. The NN is trained for up to 1000 epochs, and the model with the best accuracy according to the training sample is saved for testing. An annealing technique is used for training, and if the loss does not decrease after 50 epochs, the learning rate is halved. Furthermore, in PCNN-EPT, to accelerate training, the learning rate is also halved if artifacts (spikes in the loss function) accumulate during reconstruction and the loss increases sharply. This further prevents getting stuck in a local optimum. Training is performed without creating a validation set for each training group because the number of training samples is small. Training of PCNN-local-EPT converges, but artifacts appear in the reconstruction when the local coefficient map is under / overestimated, resulting in spikes in the loss function, as shown in Figure 14(a). These are usually easily corrected, but the loss continues to decrease. The training of PCNN-global-EPT is slightly reduced as the reconstruction is only affected by the global parameters, as shown in Figure 14(b). That is, the coefficient generation of the NN is a very constrained modification to the reconstruction. The training of NN-EPT is very smooth, with the loss function decreasing at every epoch, as shown in Figure 14(c). In PCNN-EPT, there is a point during training where the learning stops decreasing, and at this point a spike appears and the training stops at a local minimum (see the dashed line in Figure 14(d)). After that, by lowering the learning rate, the NN further converges, and it takes more than the limit of 1000 epochs to converge, as shown by the solid line in Figure 14(d). In such a case, the model with the lowest loss is kept for testing, as long as it is within the limit of 1000 epochs. In addition, the training time ranged from 30 minutes for NN-EPT to 1 hour for PCNN-EPT, in either global or local form, and all training was generated on an Nvidia GeForce RTX 2070 GPU.
[0111] We also compared two types of inputs to create an independent comparison point for the end-to-end NN-EPT: one NN directly outputs the final conductivity map and does not perform any intermediate steps; one model uses only the transmit and receive phase φ as input to avoid the noise explosion associated with numerical differentiation calculations. tr Received (NN-EPTφ tr ) Another model is the transmit / receive phase φ tr In addition to the transmit / receive phase differential ∇ 2 φ tr ,∇φ tr These features provide relevant information based on many analytical methods, so they are concatenated into a multi-channel input (NN-EPT∇ 2 φ tr ,∇φ tr ,φ tr ) Furthermore, since NN-EPT can be trained with different loss functions, we compared training with the usual loss function MSE of the end-to-end method and SSIM used in the embodiment and examples. As shown in Figure 13, there were slight differences in NN-EPT in each experiment, but the NN-EPT∇ trained with MSE in case (3) performed better than the NN-EPT∇ 2 φ tr ,∇φ tr ,φ tr It can be confirmed that the reconstruction accuracy improves across generalization experiments 1 to 3. [Industrial Applicability]
[0112] The present invention provides magnetic resonance electrical property tomography with excellent noise robustness and generalizability by effectively combining a stabilized cr-EPT and a coefficient generation NNs, and flexibly optimizing two stabilization coefficients (diffusion coefficient ρ and convection coefficient β) to generate accurate reconstructions, thereby providing a physically coupled neural network electrical property tomography (PCNN-EPT).Since it is possible to achieve higher reconstruction accuracy than conventional end-to-end NN-EPT, the present invention has industrial applicability as an important step toward the practical use of magnetic resonance electrical property tomography in medical diagnosis.
Claims
1. 1. Magnetic resonance electrical properties tomography, comprising: A first means for reconstructing a conductivity map using a physical model based on wave physics with a diffusion coefficient ρ and a convection coefficient β as coefficients; A second means for learning the diffusion coefficient ρ and the convection coefficient β by a data-driven machine learning model using parameters of the physical model; A magnetic resonance electrical characteristic tomography device comprising:
2. The physical model is tr ・∇γ)+γ∇ 2 φ tr -ρ∇ 2 γ=ωμ 0 and The parameter is φ tr , ∇φ tr , ∇ 2 φ tr (where φ tr is the normalized transmit phase. γ is the inverse of the conductivity σ (γ=1 / σ). ω is the Larmor frequency. μ 0 2. The magnetic resonance electrical characteristic tomography according to claim 1, wherein:
3. The magnetic resonance electrical property tomography according to claim 1 , wherein the machine learning model is a learning model using a neural network.
4. 4. The magnetic resonance electrical characteristic tomography according to claim 3, wherein the diffusion coefficient ρ and the convection coefficient β are determined by different neural networks.
5. 5. The magnetic resonance electrical characteristic tomography according to claim 1, further comprising a third means for calculating a difference between the conductivity map reconstructed by the first means and a ground truth conductivity map, and feeding the difference back to the second means.
6. The magnetic resonance electrical property tomography method according to claim 1 , wherein the diffusion coefficient ρ and the convection coefficient β are each a global coefficient map.
7. The magnetic resonance electrical property tomography method according to claim 1 , wherein the diffusion coefficient ρ and the convection coefficient β are each a local coefficient map.
8. The magnetic resonance electrical characteristic tomography of claim 1, characterized in that the diffusion coefficient ρ and the convection coefficient β can be selected as either a global coefficient map or a local coefficient map according to the noise condition of a test sample.
Citation Information
Patent Citations
Information processing device and information processing method
JP2022115092A
Device and method for reconstructing low-frequency conductivity images using MRI without current injection
US20190142296A1
