Apparatus and method for improved interpretation of magnetic resonance and ultrasound shear wave elastography for monitoring liver fibrosis - Patents.com

Abstract

A method and system 100 for improved interpretation of shear wave elastography between a first imaging modality and a second imaging modality comprises performing elastography measurements with a second imaging modality 20 different from the first imaging modality 10 to obtain at least one second imaging modality elastography value 32, 60 of a region of interest 33. At least one corresponding first imaging modality elastography value 36, 38, 62 is predicted based on the obtained second imaging modality elastography value. A graphical user interface or smart report dashboard 50 is generated showing (i) a fibrosis level 521 of the region of interest, the fibrosis level being determined as a function of (i)(a) the at least one second imaging modality elastography value 32 and / or (i)(b) the predicted at least one corresponding first imaging modality elastography value 36, 38.

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE

[0001] The present embodiments relate generally to elastography, and more particularly to an apparatus and method for improved interpretation of magnetic resonance and ultrasound shear wave elastography for monitoring liver fibrosis. [Background technology]

[0002]

[0002] Liver disease is the 12th leading cause of death in the United States. Various causes of liver disease, such as Hepatitis B, Hepatitis C, and nonalcoholic fatty liver disease (NAFLD), cause scarring in the liver. If left untreated, scarring can lead to liver fibrosis, an advanced state of liver disease. Liver fibrosis results from chronic damage to the liver in conjunction with the accumulation of extracellular matrix proteins. If left untreated, fibrosis can further progress to cirrhosis, liver cancer, liver failure, and death. Therefore, accurate diagnosis and monitoring of liver fibrosis is crucial.

[0003]

[0003] The current gold standard for liver fibrosis diagnosis is liver biopsy, but there are some limitations associated with biopsy. Liver biopsy is invasive and causes anxiety. In some extreme cases, complications such as post-biopsy bleeding and liver rupture may lead to death for patients with comorbidities. Therefore, it is desirable to develop a non-invasive and accurate measurement tool for liver fibrosis evaluation and treatment monitoring for better patient management.

[0004]

[0004] Shear wave elastography has been shown to be a promising tool for non-invasively measuring soft tissue stiffness. Both ultrasound shear wave elastography (UE) and magnetic resonance shear wave elastography (MRE) are quantitative imaging techniques for diagnosing and monitoring liver fibrosis. Now referring to FIG. 1, an overview of the top system setup of a system for magnetic resonance shear wave elastography 10 is shown. For MRE, a patient is placed on the stage of an MRI scanner 12. A passive driver / transducer 14 is positioned on the patient with respect to the region of interest 16. The patient is then restrained in the MRI scanner while performing elastography measurements. Furthermore, for MRE, a passive driver is used to transmit continuous acoustic vibrations at low frequency (~60Hz for liver) to the subject's region of interest (e.g., abdomen) to generate high frequency waves with a narrow frequency band, which means that the wave energy is concentrated at the desired frequency. Examples of six different stiffness maps (collectively indicated by reference numeral 18) obtained by MRE, along with the corresponding designation of fibrosis stage, are shown at the bottom of Figure 1. Viewing the MRE stiffness maps from left to right in Figure 1 (i.e., by increasing levels of fibrosis), the respective fibrosis stage designations include: F0: normal (no fibrosis), F1: portal fibrosis without septa, F2: portal fibrosis with few septa, F3: numerous septa without cirrhosis, and F4: cirrhosis.

[0005]

[0005] Referring now to FIG. 2, an overview of the system setup for ultrasound shear elastography 20 is shown. Unlike the MRE method, the UE method uses high intensity short duration ultrasound "push" pulses to generate shear waves with a wide frequency spectrum. In the excitation step of the UE method, it is difficult to precisely control the degree to which the energy of the wave is distributed within that wide frequency spectrum. An example of a stiffness map (indicated by reference numeral 22) acquired by UE is also shown in FIG. 2 with a corresponding color-coded stiffness scale 24 in the upper right portion of the stiffness map.

[0006]

[0006] Ultrasound imaging vendors use time-of-flight methods to reconstruct shear wave velocities as well as group velocities from wide bandwidths, which are also vendor-dependent. Due to differences in shear wave frequencies, selected tissue viscoelastic mechanical models and time delay estimation methods, the shear wave velocities (m / s) and subsequent stiffness values ​​(kPa) reported by ultrasound systems, such as group velocities, do not match the velocities or stiffness values ​​(kPa) measured by MRE. These differences make MRE stiffness measurements and UE stiffness measurements non-interchangeable, and as a result, measurements obtained by UE and MRE are difficult to directly compare.

[0007]

[0007] There are no known guidelines provided by international clinical organizations regarding which elastography method (UE or MRE) to choose for diagnosing and monitoring liver fibrosis. MRE is highly reproducible in healthy subjects and patients with liver fibrosis. In addition, MRE has a higher success rate than UE in diagnosing liver fibrosis (93% for MRE vs. 82% for UE). Furthermore, MRE may have a more robust quantification performance due to its high shear wave SNR, 3D volume imaging, and by assessing a larger proportion of the liver, which would reduce sampling variability. However, MRE has some limitations. Additional time is required to position the passive driver, and the transducer causes discomfort for some patients.

[0008]

[0008] Compared to MRE, UE is cheaper, faster, more portable, widely available, and has been independently approved in many clinical centers around the world. Some clinical sites do not have access to MRI scanners, and some patients are reluctant to undergo MRE due to being confined in the MR scanner again. In these cases, UE is preferred over MRE during the monitoring phase for liver fibrosis due to its higher clinical value. Therefore, depending on the availability of the system in the clinical institution, the comfort level of the patient, and the cost of care, MRE or UE is preferred for diagnosing and monitoring liver fibrosis.

[0009]

[0009] Although both MRE and UE are used for liver fibrosis diagnosis, due to the aforementioned technical differences between MRE and UE, the use of the same image modality is preferred for monitoring and measurement consistency over time for a given patient. Using the same image modality ensures that stiffness changes over time are exclusively pathological and due to patient-related factors, and not due to technical differences between MRE and UE (excitation frequency, equipment-related dependencies, and post-processing algorithms). However, MRE is expensive, and performing MRE measurements at multiple time points or follow-up examinations for liver monitoring imposes a significant economic burden on patients. Considering the lower cost and wider availability of UE compared to MRE, it is desirable to use UE for follow-up liver fibrosis monitoring. Also, some pediatric and elderly patients may feel uncomfortable in the MRI scan, which would make MRE examination more laborious and time-consuming (i.e., many movements and repeated measurements). Summary of the Invention [Problem to be solved by the invention]

[0010]

[0010] Therefore, improved methods and devices that overcome the problems in the art are desirable. It is further desirable to provide a non-invasive and accurate measurement tool for liver fibrosis assessment and treatment monitoring for better patient management, and to remove the dependency on one imaging modality. [Means for solving the problem]

[0011]

[0011] According to one aspect, an embodiment of the present disclosure bridges the gap between MRE and UE for monitoring liver fibrosis by 1) predicting corresponding MRE stiffness values ​​and confidence levels (or scores) solely by taking UE measurements, and 2) providing a smart dashboard showing fibrosis stage, confidence scores, and changes in stiffness values ​​in plots and tables in addition to initial and predicted liver fibrosis stiffness maps.

[0012] According to one embodiment, a method for improved interpretation of shear wave elastography between a first imaging modality and a second imaging modality comprises performing elastography measurements with a second imaging modality different from the first imaging modality to obtain at least one second imaging modality elastography value of a region of interest. The method further comprises predicting, by a processor, at least one corresponding first imaging modality elastography value based solely on the obtained second imaging modality elastography value. After predicting, the method comprises generating, by a processor and a display, a graphical user interface or smart report dashboard including or indicating (i) a fibrosis level of the region of interest, the fibrosis level being determined as a function of (i)(a) the at least one second imaging modality elastography value and / or (i)(b) the predicted at least one corresponding first imaging modality elastography value.

[0013] In one embodiment, the second imaging modality is selected between (i) magnetic resonance shear wave elastography (MRE) and (ii) ultrasound shear wave elastography (UE). The first imaging modality comprises an unselected one of the second imaging modalities. The elastography values ​​acquired by the second imaging modality are not equivalent to the elastography values ​​acquired by the first imaging modality due to technical differences in excitation frequencies and post-processing algorithms of each respective imaging modality.

[0014] According to one embodiment, each of the at least one second imaging modality elastography value and the at least one corresponding predicted first imaging modality elastography value includes one or more of: (i) a stiffness map; and (ii) a stiffness value in kPa. For example, the stiffness value includes an average value of the respective stiffness map. In another embodiment, the method includes predicting a confidence score associated with the at least one second imaging modality elastography value, the predicted confidence score including a highest confidence percentage among the confidence percentages at each of a plurality of fibrosis levels F0-F4 based on the second imaging modality elastography measurements; and selecting, by a processor, a predicted fibrosis level based on the fibrosis level having the highest confidence percentage.

[0015] In yet another embodiment, the smart report dashboard further includes (or indicates) (ii) a baseline fibrosis level of the region of interest. The baseline fibrosis level is determined based on an initial baseline elastography measurement performed by a first imaging modality to obtain at least one baseline first imaging modality elastography value of the region of interest. In one embodiment, the baseline elastography measurement is performed before the elastography measurement by a second imaging modality. In yet another embodiment, the smart report dashboard further includes (or indicates) (iii) a percentage change in elastography value between (iii) (a) the at least one first imaging modality elastography value predicted based on the obtained at least one second imaging modality elastography value and (iii) (b) the at least one baseline first imaging modality elastography value. Additionally, the smart report dashboard further includes (or indicates) (iv) the at least one second imaging modality elastography value of the region of interest, (v) the predicted at least one corresponding first imaging modality elastography value of the region of interest, and (vi) a confidence score associated with the at least one second imaging modality elastography value, the confidence score being a percentage between 0 and 100 percent and representing a confidence level in the fibrosis level based on the at least one second imaging modality elastography value.

[0016] In one embodiment, predicting the at least one corresponding first imaging modality elastography value comprises initiating a deep learning based algorithm to predict the at least one corresponding first imaging modality elastography value. The deep learning based algorithm comprises a generative adversarial network (GAN) and / or a convolutional neural network (CNN). In another embodiment, the method further comprises determining, by the processor and the second deep learning based algorithm, a confidence score associated with the at least one second imaging modality elastography value. The determining of the confidence score is automatically launched simultaneously with initiating the deep learning based algorithm. The confidence score further provides a real-time classification of the fibrosis level or a confidence level of the fibrosis level based on the at least one second imaging modality elastography value.

[0017] According to another embodiment, a system for improved interpretation of shear wave elastography between a first imaging modality and a second imaging modality comprises an input for receiving imaging data related to elastography measurements acquired from a second imaging modality, and a controller configured to perform elastography measurements with a second imaging modality different from the first imaging modality to acquire at least one second imaging modality elastography value of a region of interest, the controller further configured to predict at least one corresponding first imaging modality elastography value based solely on the acquired second imaging modality elastography value. After predicting, the controller is further configured to: (i) generate a graphical user interface or smart report dashboard including (or indicating) a fibrosis level of the region of interest, the fibrosis level being determined as a function of (i)(a) the at least one second imaging modality elastography value and / or (i)(b) the predicted at least one corresponding first imaging modality elastography value. The system further comprises a display in communication with the controller for displaying the smart report dashboard.

[0018]

[0018] Still further advantages and benefits will become apparent to those of ordinary skill in the art upon reading and understanding the following detailed description.

[0019]

[0019] The embodiments of the present disclosure can take the form of various components and arrangements of components, as well as various steps and configurations of steps. Thus, the drawings are for illustrating various embodiments and should not be interpreted as limiting the embodiments. In the drawings, like reference numerals refer to like elements. Furthermore, it should be noted that the figures may not be drawn to scale. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 is an exemplary flow diagram of a method for magnetic resonance shear wave elastography (MRE). [Diagram 2]

[0021] FIG. 1 is an exemplary flow diagram of a method for ultrasound shear wave elastography (UE). [Diagram 3]

[0022] 1 is an example graph showing MRE-obtained shear modulus (or stiffness). [Figure 4]

[0023] FIG. 13 is a graph showing the shear modulus obtained by UE. [Diagram 5]

[0024] FIG. 13 is an exemplary flow diagram of a step of predicting a corresponding first imaging modality elastography value based solely on an acquired or measured second imaging modality elastography value according to an embodiment of the present disclosure. [Figure 5A] 6 is a partial reproduction of the flow diagram of FIG. 5 for clarity. [Figure 5B] 6 is a partial reproduction of the flow diagram of FIG. 5 for clarity. [Figure 5C] 6 is a partial reproduction of the flow diagram of FIG. 5 for clarity. [Figure 6]

[0025] FIG. 13 is an exemplary flow diagram of a step of predicting a confidence score according to an embodiment of the present disclosure, where the predicted confidence score is related to an elastography value acquired or measured by a second imaging modality. [Figure 6A] 7 is a partial reproduction of the flow diagram of FIG. 6 for clarity. [Figure 6B] 7 is a partial reproduction of the flow diagram of FIG. 6 for clarity. [Figure 6C] 7 is a partial reproduction of the flow diagram of FIG. 6 for clarity. [Figure 7-1]

[0026] FIG. 1 is an exemplary graphical user interface diagram of a smart report dashboard according to one embodiment of the present disclosure. [Figure 7-2] For clarity, this is a continuation of FIG. [Figure 7A] 8 is a partial reproduction of the flow diagram of FIG. 7 for clarity. [Figure 7B] 8 is a partial reproduction of the flow diagram of FIG. 7 for clarity. [Figure 7C] 8 is a partial reproduction of the flow diagram of FIG. 7 for clarity. [Figure 8]

[0027] FIG. 1 is a block diagram of a system for improved interpretation of shear wave elastography between a first imaging modality and a second imaging modality, according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021]

[0028] The embodiments of the present disclosure and various features and advantageous details thereof will be more fully described with reference to non-limiting examples described and / or illustrated in the drawings and detailed in the following description. It should be noted that the features illustrated in the drawings are not necessarily drawn to scale, and that features of one embodiment can be utilized with other embodiments, even if not explicitly described herein, as one skilled in the art will recognize. Descriptions of well-known components and processing techniques may be omitted so as not to unnecessarily obscure the embodiments of the present disclosure. The examples used herein are merely intended to facilitate understanding of the manner in which the embodiments of the present disclosure are implemented and to further enable those skilled in the art to implement the same. Thus, the examples herein should not be construed as limiting the scope of the embodiments of the present disclosure, which is defined solely by the appended claims and applicable law.

[0022]

[0029] It is to be understood that the embodiments of the present disclosure are various and are not limited to the specific methodologies, protocols, devices, apparatus, materials, applications, etc. described herein. It is also to be understood that the terminology used herein is used to describe specific embodiments only and is not intended to be limiting in scope of the embodiments described in the claims. It must be noted that as used herein and in the appended claims, the singular includes the plural unless the context clearly dictates otherwise.

[0023]

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the disclosed embodiments belong. Although preferred methods, devices, and materials are described, any methods and materials similar or equivalent to those described herein can be used in the practice or testing of the embodiments.

[0024]

[0031] To more fully understand the embodiments of the present disclosure, we first consider at least two challenges / limitations known in the art that are overcome by the embodiments, as discussed herein below. Regarding the first challenge, while there are stiffness cutoff values ​​in UE and MRE for liver fibrosis staging derived from liver biopsy as ground truth, stiffness values ​​in MRE and UE for the same patient are not comparable. For example, FIG. 3 shows a plot 110 of stiffness (kPa) versus liver fibrosis level (F0, F1, F2, F3, and F4) for magnetic resonance shear wave elastography, i.e., MRE. For plot 110, the vertical axis 112 on the left side includes a stiffness scale in kPa for the first imaging modality, and the horizontal axis 114 includes a fibrosis level scale from F0 to F4 for the first imaging modality. FIG. 4 shows a plot 130 of stiffness (kPa) versus liver fibrosis level (F0, F1, F2, F3, and F4) for ultrasound shear wave elastography, i.e., UE. For plot 130, left vertical axis 132 includes stiffness scale in kPa for the second imaging modality, and horizontal axis 134 includes fibrosis level scale F0-F4 for the second imaging modality. The underlying reasons for the difference in stiffness values ​​between MRE and UE include (i) technical differences in the respective excitation frequencies, and (ii) technical differences in the respective post-processing algorithms for stiffness map reconstruction of the imaging modalities.

[0025]

[0032] Regarding the technical differences in excitation frequencies, consider the following: The external drivers used in MRE produce harmonics with narrow frequency bandwidths, meaning that wave energy can be focused at the desired frequency (e.g., ∼60 Hz for the liver). UE methods, on the other hand, use high intensity short duration ultrasound “push” pulses to generate shear waves with a broad frequency spectrum. Although it is possible to filter out the desired shear wave frequency in UE to make it comparable to the shear wave frequency in MRE in the post-processing stage, it is difficult to control the degree to which the wave frequencies are distributed within the spectrum in the excitation step. The difference in frequency content of the resulting waves in these methods (single frequency vs. multi-frequency excitation) makes the shear group velocity in UE different from, and therefore not compatible with, the shear wave group velocity in MRE.

[0026]

[0033] Regarding the technical differences in the post-processing algorithms for stiffness map reconstruction, consider further the following: In MRE, a local frequency estimation (LFE) algorithm is typically used for stiffness map reconstruction. LFE calculates shear wave velocity by estimating the local spatial frequency of the shear wave propagation pattern by an algorithm that combines local estimates of instantaneous frequency across several scales. However, in UE, an inverse wave equation or 2D cross-correlation algorithm is used. Furthermore, in UE, the reconstructed stiffness map is 2D, whereas in MRE, the reconstructed stiffness map is 3D.

[0027]

[0034] Referring again to Figures 3 and 4, the figures further show the shear modulus acquired by MRE and UE, respectively. As can be seen, the UE acquired shear modulus (or stiffness) (Figure 4) is higher than the MRE acquired shear modulus (Figure 3) for all liver fibrosis levels (F0-F4). In further comparison, MRE has a higher success rate (93% vs. 82%) in diagnosing liver fibrosis than UE. MRE may have robust quantification performance due to its high shear wave SNR, 3D volume imaging, and by assessing a large proportion of the liver, which results in reduced sampling variation.

[0028]

[0035] However, compared to MRE, UE is cheaper, faster, more portable, and widely available, and UE has been independently approved in many clinical institutions / facilities worldwide. Some clinical care sites do not have access to MRI scanners, and some patients are reluctant to undergo MRE due to being confined to the MR scanner again. In these cases, UE is preferred over MRE. Therefore, depending on the availability of the system at the clinical institution, the comfort level of the patient, and the cost of care, MRE or UE is preferred for diagnosing and monitoring liver fibrosis.

[0029]

[0036] As in the past, if a patient undergoes MRE / UE examination, monitoring of liver fibrosis in follow-up measurements will need to remain MRE / UE (i.e., the same imaging modality) for consistency. However, MRE is expensive, and performing MRE measurements at multiple time points can cause significant financial burden and inconvenience to patients. Direct comparison of UE and MRE measurements without proper knowledge can unfortunately lead to misinterpretation. Another drawback of modality-dependent stiffness cut-off values ​​is that they only indicate separate staging or grading results (e.g., grade 2 or 3). However, it is also important to quantify the continuous stiffness value changes that treatment causes during treatment, and even in some cases within the same stage. The embodiments of the present disclosure advantageously provide a solution by allowing clinicians to use UE and MRE interchangeably at various time points to help better quantify and monitor the progression / regression of liver fibrosis.

[0030]

[0037] The second challenge overcome by the embodiments of the present disclosure is to bridge the gap between MRE and UE for follow-up fibrosis monitoring and to remove the dependency on one imaging modality. The embodiments of the present disclosure provide a smart reporting system across both imaging modalities. Contrary to this, for current clinical practice, depending on the type of elastography exam (MRE or UE) performed, different look-up tables are used to find the fibrosis level corresponding to the liver stiffness measured. Given the difference in training levels of ultrasound and MRI technicians or users, it is inconvenient for the technician or user to correlate the various stiffness levels using various imaging modalities at various time points. To address this challenge, the embodiments of the present disclosure advantageously provide a smart report / dashboard with applicability across imaging modalities that represents the fibrosis stage at all time points and the change in stiffness value showing the trend from baseline to the current exam. The dashboard presented by the embodiments of the present disclosure also advantageously removes the dependency on one imaging modality, helping the user to more easily interpret the changes in liver fibrosis regardless of their own expertise.

[0031]

[0038] In particular, the embodiments of the present disclosure provide a solution to bridge the gap between MRE and UE stiffness results by making the UE more similar to MRE without the need to perform an MRE exam. According to one embodiment, a shear wave elastography system and method predicts the corresponding MRE stiffness map (and / or stiffness value) and confidence level only by performing UE measurements, and vice versa. In particular, the shear wave elastography system and method uses the UE measured stiffness map as input and applies artificial intelligence (AI) based on an algorithm to 1) predict the corresponding MRE stiffness map and 2) predict the confidence level for the obtained stiffness map (or confidence score for the obtained stiffness value). Finally, the system and method provides a smart report / dashboard across imaging modalities (UE / MRE) showing the fibrosis stage, stiffness value changes at multiple time points for a given patient in plots and tables in addition to the measured and predicted stiffness maps.

[0032]

[0039] As can be understood from the disclosure herein, the method and system according to the embodiments of the present disclosure can be divided into three aspects. In a first aspect, the method comprises predicting the corresponding MRE stiffness value only by performing UE measurements, and vice versa. In a second aspect, the method comprises predicting a confidence score for the obtained stiffness value (e.g., the obtained liver stiffness value). Third, the method comprises providing a smart report showing the change in fibrosis stage and stiffness value for a given patient over time.

[0033]

[0040] Now referring to FIG. 5, an exemplary flow diagram of a step of predicting a corresponding first imaging modality elastography value based only on an acquired second imaging modality elastography value, indicated by reference number 30, according to an embodiment of the present disclosure is shown. For clarity, a portion of the flow diagram of FIG. 5 is reproduced in FIG. 5A, FIG. 5B and FIG. 5C. Regarding predicting a corresponding MRE stiffness value, consider the following as an example. The method starts with performing an elastography measurement with a second imaging modality different from the first imaging modality to obtain at least one second imaging modality elastography value, indicated by reference number 32, of a region of interest. In this example, the second imaging modality is UE and the first imaging modality is MRE. Referring again to FIG. 2, an ultrasound UE measurement is performed for a given region of interest (ROI) of a patient 21 using a UE device 20. The UE device 20 includes, for example, an ultrasound probe 25, a graphical user interface or ultrasound scanner touch panel 26, and a display 27. A system operator or user 23 can perform ultrasound UE measurements on a region of interest, and the resulting measurements, such as a stiffness map 22 and a corresponding color-coded stiffness scale 24, are displayed on the display device 27, as indicated by reference numeral 29 in FIG. 2. Following the UE measurements obtained by the UE device 20, a soft button 20 on the UE device's graphical user interface or ultrasound scanner touch panel 26 is pressed. A soft button 28 may be labeled, for example, "Get Corresponding MRE Stiffness Map (or Stiffness Value)."

[0034]

[0041] When the user selects the soft button 28, the ultrasound scanner 20 launches a corresponding stiffness value prediction application, indicated in Figure 5 by reference numeral 34. The stiffness value prediction application includes suitable program instructions for the processor to predict at least one corresponding first imaging modality elastography value based solely on the acquired second imaging modality elastography values. In one embodiment, the stiffness value prediction application is configured to implement a deep learning based algorithm, such as a Generative Adversarial Network (GAN) or a Convolutional Neural Network (CNN).

[0035]

[0042] Deep learning is a type of artificial neural network that is multi-layered and tries to resemble the human cognitive system. GAN and CNN are examples of deep learning-based algorithms that are known in the art and will only be briefly described herein. Now, referring to FIG. 5B, a generative adversarial network (GAN) 34 includes a deep learning architecture that includes two networks, namely a generative network 341 and a discriminative network 342. The generative network and the discriminative network are jointly trained with a backpropagation algorithm by a cost function 343. With the training, the generative network 341 becomes better at generating more realistic samples (e.g., real), and the discriminative network becomes better at differentiating artificially generated samples (e.g., generated). According to an embodiment of the present disclosure, the deep learning-based algorithm is started to predict the corresponding MRE stiffness map 36 based only on the acquired UE stiffness map (or elastography value) 32. Furthermore, the training set 344 is input to the discriminative network 342 via a processing block "x" indicated by reference numeral 345. In operation, the acquired UE stiffness map 32 (FIG. 5A) having the region of interest 33 is input to the generative network 341 of the GAN network 34 via the processing block "z" indicated by reference numeral 346. The output of the generative network 341 is input to the discriminatory network 342 via the processing block "x" indicated by reference numeral 347. The discriminatory network 342 outputs the predicted MRE stiffness map 36 and / or the corresponding predicted average value (i.e., stiffness value) 38 of the MRE stiffness map, as shown in FIG. 5C. Furthermore, the generative network and the discriminatory network are jointly trained with a backpropagation algorithm by the backpropagated output 348 of the cost function 343. The predicted MRE stiffness map 36 and the corresponding predicted average value 38 of the stiffness map are then stored in a suitable memory (not shown) of the ultrasound scanner 20.

[0036]

[0043] For training, UE and MRE measurements may be performed on the same day in a group of patients with different levels of liver fibrosis and health control. In the deep learning based algorithm, ultrasound UE and MRE stiffness maps are used as (i) input and (ii) ground truth, respectively.

[0037]

[0044] The training phase can be vendor independent and not limited to one vendor's scanner. Furthermore, since the MRE stiffness map is three-dimensional (3D) (unlike the UE), UE stiffness maps that are in the same plane shall be fed into the GAN network 34 for consistency. To match the UE and MRE planes, a suitable navigation system is preferably used for guidance to ensure that the same MR plane is selected for the UE measurement. During the UE measurement, the user or UE system operator 23 (FIG. 2) uses the UE system 20 (FIG. 2) to move the ultrasound probe 25 (FIG. 2) until a matching MR plane is reached. Once a matching MR plane is reached, the user 23 then starts acquiring stiffness measurements.

[0038]

[0045] Referring now to FIG. 6, an exemplary flow diagram of a step of predicting a confidence score, collectively indicated by reference numeral 40, according to one embodiment of the present disclosure, is shown, the predicted confidence score being related to the elastography value acquired by the second imaging modality (i.e., the actual measured elastography value). For clarity, a portion of the flow diagram is reproduced in FIG. 6.1, FIG. 6B and FIG. 6C. An automatic confidence score of the acquired stiffness value is determined as discussed below. An application or function for automatically determining a confidence score for the acquired stiffness value, indicated by reference numeral 42, is simultaneously launched with the deep learning-based application 34 (i.e., a deep learning-based algorithm) in the previous step as discussed herein with reference to FIG. 5. The confidence score, indicated by reference numeral 44, of the acquired or actual measured stiffness map is obtained using a deep learning-based algorithm, such as a convolutional neural network (CNN), for real-time classification (FIG. 6).

[0039]

[0046] Convolutional Neural Networks (CNNs) have a deep learning architecture and are used in medical imaging research due to their ability to preserve spatial relationships when filtering input images. Spatial relationships in medical imaging define tissue interfaces, structural boundaries, or joints between muscles and bones, and therefore CNNs have found special attention. A schematic diagram of a convolutional neural network 42 is shown in FIG. 6B. With reference to FIGS. 6, 6A, and 6B, the CNN 42 takes an input image of raw pixels (e.g., UE stiffness map 32) and transforms it through convolutional layers (Conv) 421 and 423, a normalized linear layer (Flatten) 425, and pooling layers (Pool) 422 and 424. This is then fed into fully connected layers (FC) 426 and 427, which assign a probability or class score to classify the input (i.e., UE stiffness map 32) into the class with the highest probability (Softmax) 428. Pixels in an image can be processed by different weights in a fully connected layer, or alternatively, all positions can be processed using the same set of weights to extract various repeating patterns across the image. These trainable weights, called kernels or filters, are applied using dot products or convolutions and then processed by nonlinear functions. Each convolution layer has several filters to extract multiple sets of patterns in each layer. Often, each convolution layer (Conv) 421 and 423 is followed by a pooling layer (Pool) 422 and 424, respectively, which reduces dimensionality and imposes translational invariance. These convolution and pooling layers can be stacked together to form a multi-layer network, which often ends in one or more fully connected layers 426 and 427.

[0040]

[0047] When the region of interest 33 (FIG. 6.1) includes the patient's liver, the output of the deep learning based algorithm is the liver fibrosis level 44 with the highest confidence score (FIG. 6C). In other words, a probability or class score for each class is assigned to classify the input (e.g., elastography value 32) into a respective fibrosis level (Level 1, Level 2, Level 3, Level 4, and No fibrosis, indicated by reference characters 441, 442, 443, 444, and 445, respectively). The class with the highest probability, representing the liver fibrosis level, is then output. For example, if the highest confidence score predicted by the AI ​​model (i.e., the convolutional neural network (CNN) of FIG. 6) belongs to fibrosis level 2 (e.g., confidence scores = 0.82 for level 2, = 0.05 for level 1, = 0A0 for level 3, = 0.02 for level 4, and 0.01 for no fibrosis), then the confidence score is equal to 82%, which means that we are 82% sure that the stiffness value measured by the second imaging modality (e.g., UE modality) represents fibrosis level 2. The highest probability and corresponding fibrosis level are stored in a suitable memory (not shown) and output or shown via the GUI 26 or on the smart report dashboard 50, as further discussed herein with respect to FIG. 7. For example, once the confidence score (e.g., 44 in FIG. 6) and stiffness map (e.g., 36 in FIG. 5) are predicted, a "Smart Report" soft button 28 is made to appear or made to appear on the ultrasound scanner touch panel 26 so that the system user 23 can click on the soft button 28 to observe the final results via the Smart Report dashboard 50 (FIG. 7). As used herein, fibrosis stage is understood to be the same as fibrosis level. Furthermore, predicted confidence level and predicted confidence score are used interchangeably herein and are further based on actual measured elastography values, e.g., measured UE stiffness map.

[0041]

[0048] In one embodiment, the predicted confidence score is related to the measured stiffness value in the second imaging modality. In particular, the predicted confidence score of the measured stiffness value in the second imaging modality indicates the confidence level of the reported fibrosis level measured by the second imaging modality. The AI ​​model predicting the confidence score of the fibrosis level is calibrated / evaluated using liver biopsies as ground truth in the training phase. In other words, for the deep learning based algorithm in the training phase, the UE stiffness map and the liver biopsy are used as (i) input and (ii) ground truth, respectively. For training, the UE and liver biopsy may be performed on the same day in a group of patients with various levels of liver fibrosis. This training applies to the determination of the "confidence score".

[0042]

[0049] Now referring to FIG. 7, a diagram of an exemplary graphical user interface of a smart report dashboard, indicated by reference numeral 50, according to one embodiment of the present disclosure is shown. For clarity, a portion of the flow diagram is reproduced in FIG. 7A and FIG. 7B. The smart report dashboard 50 has a graphical user interface that includes one or more forms of multimedia. Considering the different training levels of ultrasound and MRI users, it is inconvenient for users to correlate different stiffness levels using different imaging modalities at different time points. The smart report or smart report dashboard 50 helps users to better monitor fibrotic changes.

[0043]

[0050] When the user clicks on the "Smart Report" soft button 28 on the ultrasound scanner touch panel 26, a smart dashboard 50 is automatically generated (FIG. 7) that includes an indication of one or more of the following: a current fibrosis stage or level, indicated by reference numeral 52, and a corresponding predicted confidence score, indicated by reference numeral 54, obtained in a previous step as discussed herein with reference to FIG. 6, are provided. In one embodiment, the current fibrosis stage 52 includes an indication of the fibrosis level 521, an elastography value 522 (e.g., a measured UE stiffness value) obtained by the second imaging modality elastography measurement, and a predicted elastography value 523 (e.g., a predicted MRE stiffness value) corresponding to the first imaging modality elastography measurement based on the obtained second imaging modality elastography measurement. The acquisition of elastography values ​​522 by second imaging modality elastography measurements and the prediction of elastography values ​​523 corresponding to the first imaging modality measurements are accomplished as discussed herein with reference to at least FIG. 5, UE stiffness map 32, predicted MRE stiffness map 36, and predicted MRE stiffness values ​​38. In one embodiment, the new stiffness values ​​(i.e., measured UE values ​​522 and / or corresponding predicted MRE values ​​523) are automatically added to a plot indicated by reference numeral 56 and a table indicated by reference numeral 58 of existing stiffness values ​​measured during previous examinations. The plot 56 and / or table 58 advantageously provide a meaningful illustration of changes in liver stiffness over time. Referring now to FIG. 7.1, the plot 56 shows the acquired fibrosis levels in one or more previous examinations. With respect to plot 56, the left vertical axis 561 includes a stiffness scale in kPa for the second imaging modality, the horizontal axis 562 includes a stiffness scale in kPa for the first imaging modality, and the right vertical axis 563 includes a scale of fibrosis levels from level 0 to level 4.As shown in the example plot 56, a decrease in fibrosis level is observed from the baseline (March 10, 2019) measurement acquired by the first imaging modality to the subsequent measurement (February 4, 2020) acquired by the second imaging modality, corresponding to a change in fibrosis level from level 3 to level 2. With further reference to FIG. 7B, table 58 further includes a fibrosis stage 51 including a display of fibrosis level 511 and elastography values ​​512 acquired by the baseline (March 10, 2019) along with the first imaging modality elastography measurements (e.g., measured MRE stiffness values). In FIGS. 7A and 7B, smart dashboard 50 further illustrates a baseline stiffness map indicated by reference numeral 64 as well as measured and predicted stiffness maps indicated by reference numerals 60 and 62, respectively, acquired as discussed herein with reference to FIG. 5. It should be noted that the baseline stiffness map 64 is a previously acquired stiffness map, such as in an initial fibrosis examination, acquired by one or more of the MRE or UE imaging modalities. A hyperlink, indicated by reference numeral 66, is included within the smart dashboard 50 directing the user to more information on the MRE / UE cutoff values, such as guideline material (such as reference papers) regarding the differences between MRE / UE. Furthermore, the smart dashboard 50 provides results, such as an increase or decrease in fibrosis stiffness, which may be represented as a result field, indicated by reference numeral 68, and / or by an appropriate symbolic display, indicated by reference numeral 70 (e.g., a percentage number and a graphic showing an upward point for an increase or a downward point for a decrease). As shown in FIG. 7, the smart report dashboard 50 shows stiffness values, changes in fibrosis levels, measured and predicted stiffness maps, and confidence scores in plots and tables.

[0044]

[0051] As should be understood herein, the stiffness map can be predicted in either direction. There are clinical sites where MRI scanners are not available, or some patients are unwilling to undergo MRE due to being confined in the MR scanner. In these cases, UE is preferred over MRE during the fibrosis diagnosis stage. However, during the treatment stage, MRE may be recommended to more accurately monitor liver fibrosis, since it has a higher success rate and more robust quantification performance compared to UE due to its higher shear wave SNR, 3D volume imaging, and evaluating a larger percentage of the liver. Therefore, there are cases where a patient undergoes a UE examination in a fibrosis diagnosis test and an MRE examination in a long-term follow-up measurement. In this scenario, instead of predicting the MRE stiffness map according to an embodiment of the present disclosure, predicting the UE stiffness map and stiffness value based on the measured MRE stiffness map can be used. In this case, the same deep learning based algorithm and training process can be used, except that (i) the input and (ii) the ground truth are the MRE stiffness map and the UE stiffness map, respectively.

[0045]

[0052] Referring now to FIG. 8, a block diagram of a system 100 for improved interpretation of shear wave elastography between a first imaging modality and a second imaging modality is shown, according to one embodiment of the present disclosure. In this embodiment, the system 100 comprises at least a controller 102, a display 104, and an input / output device 106. The controller 102 is operatively connected to one or more shear wave imaging modality systems or devices, such as an MRE system / device 10 and / or a UE system / device 20, depending on the requirements of a given shear wave elastography implementation. In one embodiment, the controller 102 includes one or more of a microprocessor, a microcontroller, a field programmable gate array (FPGA), an integrated circuit, a discrete analog or digital circuit component, hardware, software, firmware, or any combination thereof, to perform various functions as discussed herein, further depending on the requirements of a given shear wave elastography implementation and / or application. The controller 102 may further include one or more of various modules, units, or subsystems. In one embodiment, the controller 102 can have system electronics, power, and memory including one or more of a variety of modules, units, or subsystems as appropriate for the requirements of a given shear wave elastography system implementation and / or application. In one embodiment, a display 104 is operatively connected to the controller 102 for displaying one or more soft buttons (e.g., soft buttons 28 as discussed herein) and the smart report dashboard 50 or one or more portions thereof (e.g., current fibrosis stage, confidence score, plots of stiffness values, stiffness change over time, acquired and predicted stiffness maps, stiffness values, tables showing increase and decrease in fibrosis stiffness, etc.) as appropriate for the requirements of a given shear wave elastography system implementation.The input / output device(s) 106 may include one or more of an input / output device, a user interface, a tactile output device, a touch screen, an optical display, a microphone, a keypad, a keyboard, a pointing device, an image capture device, a video camera, an automatic output device, and any combination thereof, depending on the requirements of a given shear wave elastography system implementation.

[0046]

[0053] As discussed herein, various aspects of the embodiments of the present disclosure involve user interface components, for example, having elastography features that use UE to predict corresponding MRE stiffness maps (or vice versa, i.e., using MRE to predict UE stiffness maps). The embodiments of the present disclosure can be incorporated into new features of ultrasound / MR scanners, i.e., can be included in ultrasound (US) or magnetic resonance imaging (MRI) scanner devices. Furthermore, the smart reporting system / dashboard or graphical user interface presented herein and its features can be incorporated into ultrasound / MR scanners.

[0047]

[0054] Thus, the embodiments of the present disclosure eliminate the need to keep the same image modality for follow-up stiffness measurements to monitor the progression of liver fibrosis. Enabling UE instead of MRE for follow-up stiffness measurements advantageously contributes to lower cost care for patients. Furthermore, the smart dashboard / report according to the embodiments of the present disclosure advantageously eliminates the dependency on one image modality in the reporting system. The method and system device of the present disclosure further advantageously allows users with various levels of training / experience to be able to monitor changes in liver fibrosis.

[0048]

[0055] Although only a few exemplary embodiments have been described in detail above, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without substantially departing from the novel teachings and advantages of the embodiments of the present disclosure. For example, the embodiments of the present disclosure can be advantageously used by performing baseline shear wave elastography measurements in both MRE and UE imaging modalities, and then performing one or the other of MRE or UE based on (i) the availability of a given shear wave elastography imaging modality at a subsequent time point and (ii) a confidence score. As discussed herein, for each predicted imaging modality elastography value, at a subsequent time point, either MRE or UE can be selected based on the acceptability of the confidence score. If the confidence score is determined to be unacceptable or not within a given acceptable range for one imaging modality, the other imaging modality can be selected, i.e., the confidence score is used as a metric to help select between MRE and UE based on the confidence score. Accordingly, all such modifications are intended to be included within the scope of the embodiments of the present disclosure as defined in the appended claims. In the claims, means-plus-function clauses are intended to cover the structures described herein as performing the recited function and to cover equivalent structures as well as structural equivalents.

[0049]

[0056] Moreover, any reference signs placed between parentheses in one or more of the claims shall not be construed as limiting the claim. The use of words such as "comprises", "including" and the like does not exclude the presence of other elements or steps than those listed in any claim or in the specification as a whole. The singular reference of an element does not exclude the plural reference of such elements and vice versa. One or more of the embodiments may be realized by means of hardware comprising several distinct elements, and / or by means of a suitably programmed computer. In a device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain means are recited in mutually different dependent claims does not indicate that a combination of these means cannot be used to advantage.

Claims

1. 1. A method for improved interpretation of shear wave elastography between a first imaging modality and a second imaging modality, the method comprising: performing elastography measurements with a second imaging modality different from the first imaging modality to obtain at least one second imaging modality elastography value of a region of interest; predicting, by a processor, at least one corresponding first imaging modality elastography value based on the acquired second imaging modality elastography values; generating, by the processor and display, (i) a smart report dashboard including a fibrosis level of the region of interest, the fibrosis level being determined as a function of (i)(a) the at least one second imaging modality elastography value and / or (i)(b) the at least one predicted corresponding first imaging modality elastography value; The method comprising:

2. 2. The method of claim 1, wherein the second imaging modality is selected between (i) magnetic resonance shear wave elastography (MRE) and (ii) ultrasound shear wave elastography (UE), and the first imaging modality comprises the non-selected one of the second imaging modalities.

3. The method of claim 1 , wherein the elastography values ​​acquired by the second imaging modality are not equivalent to the elastography values ​​acquired by the first imaging modality.

4. 2. The method of claim 1, wherein the at least one second imaging modality elastography value and the at least one predicted corresponding first imaging modality elastography value each comprise one or more of: (i) a stiffness map; and (ii) a stiffness value in kPa.

5. predicting, by the processor, a confidence score associated with the at least one second imaging modality elastography value, the predicted confidence score comprising a highest confidence percentage among confidence percentages at each of a plurality of fibrosis levels F0-F4 based on the elastography measurements of the second imaging modality; selecting, by the processor, a predicted fibrosis level based on the fibrosis level having the highest confidence percentage; The method of claim 1 further comprising:

6. 2. The method of claim 1, wherein the smart report dashboard further comprises: (ii) a baseline fibrosis level of the region of interest, the baseline fibrosis level being determined based on a baseline elastography measurement performed by the first imaging modality to obtain at least one baseline first imaging modality elastography value of the region of interest, the baseline elastography measurement being performed prior to the elastography measurement by the second imaging modality.

7. 7. The method of claim 6, wherein the smart report dashboard further comprises: (iii) a percentage change in elastography values ​​between (a) the at least one first imaging modality elastography value predicted based on the at least one acquired second imaging modality elastography value and (iii)(b) the at least one baseline first imaging modality elastography value.

8. 8. The method of claim 7, wherein the smart report dashboard further comprises: (iv) the at least one second imaging modality elastography value of the region of interest; (v) the at least one predicted corresponding first imaging modality elastography value of the region of interest; and (vi) a confidence score associated with the at least one second imaging modality elastography value, the confidence score being a percentage between 0 and 100 percent and representing a confidence level in the fibrosis level based on the at least one second imaging modality elastography value.

9. 2. The method of claim 1, wherein predicting the at least one corresponding first imaging modality elastography value comprises initiating a deep learning based algorithm to predict the at least one corresponding first imaging modality elastography value, the deep learning based algorithm comprising an algorithm selected from the group consisting of a generative adversarial network (GAN) and a convolutional neural network (CNN).

10. 10. The method of claim 9, further comprising determining, by the processor and a second deep learning based algorithm, a confidence score associated with the at least one second imaging modality elastography value, wherein determining the confidence score is automatically launched simultaneously with initiating the deep learning based algorithm, and wherein the confidence score further provides a real-time classification of the fibrosis level or a confidence level of the fibrosis level based on the at least one second imaging modality elastography value.

11. 1. A system for improved interpretation of shear wave elastography between a first imaging modality and a second imaging modality, comprising: an input for receiving imaging data relating to elastography measurements obtained from the second imaging modality; performing elastography measurements with a second imaging modality different from the first imaging modality to obtain at least one second imaging modality elastography value of a region of interest; predicting at least one corresponding first imaging modality elastography value based on the acquired second imaging modality elastography value; and a controller that generates a smart report dashboard including (i) a fibrosis level of the region of interest, the fibrosis level being determined as a function of (i)(a) the at least one second imaging modality elastography value and / or (i)(b) the at least one predicted corresponding first imaging modality elastography value; a display in communication with the controller for displaying the smart report dashboard; A system comprising:

12. 12. The system of claim 11, wherein the second imaging modality is selected between (i) magnetic resonance shear wave elastography (MRE) and (ii) ultrasound shear wave elastography (UE), and the first imaging modality comprises a non-selected one of the second imaging modalities.

13. The system of claim 11 , wherein the elastography values ​​acquired by the second imaging modality are not equivalent to the elastography values ​​acquired by the first imaging modality.

14. 12. The system of claim 11, wherein the at least one second imaging modality elastography value and the at least one predicted corresponding first imaging modality elastography value each include one or more of: (i) a stiffness map; and (ii) a stiffness value in kPa.

15. The controller further comprises: predicting a confidence score associated with the at least one second imaging modality elastography value, the predicted confidence score comprising a highest confidence percentage among confidence percentages at each of a plurality of fibrosis levels F0-F4 based on the elastography measurements of the second imaging modality; and The system of claim 11 , further comprising: selecting a predicted fibrosis level based on the fibrosis level having the highest confidence percentage.

Images