Devices and methods for enhanced interpretation of magnetic resonance and ultrasound shear wave elastography for monitoring liver fibrosis
By using deep learning algorithms in ultrasonic shear wave elastography to predict magnetic resonance stiffness values and confidence levels, the problem of incomparability between MRE and UE stiffness values is solved, achieving cross-modal liver fibrosis monitoring with improved accuracy and cost-effectiveness.
Patent Information
- Application Number
- CN202280012048.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2021-01-27
- Filing Date
- 2022-01-25
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2042-01-25
AI Technical Summary
Existing magnetic resonance shear wave elastography (MRE) and ultrasound shear wave elastography (UE) have problems with the incomparability of stiffness values in the diagnosis and monitoring of liver fibrosis, resulting in a lack of consistency and accuracy. Furthermore, MRE is expensive and inconvenient for patients to use.
By performing only UE measurements to predict the corresponding MRE stiffness value, and using deep learning-based algorithms such as Generative Adversarial Networks (GANs) and Convolutional Neural Networks (CNNs) to predict confidence levels, an intelligent reporting panel is provided to display fibrosis stages and stiffness value changes across modalities.
It enables interchangeability of stiffness values between different imaging modalities, improves the accuracy and consistency of liver fibrosis monitoring, reduces reliance on MRE, lowers costs, and improves patient comfort.
Smart Images

Figure CN116761546B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present invention generally relate to elastography, and more specifically, to apparatus and methods for enhanced interpretation of magnetic resonance and ultrasound shear wave elastography for monitoring liver fibrosis. Background Technology
[0002] Liver disease is the twelfth leading cause of death in the United States. Various causes of liver diseases (such as hepatitis B, hepatitis C, and non-alcoholic fatty liver disease (NAFLD)) lead to scarring in the liver. If this scarring remains untreated, it can lead to liver fibrosis, a late-stage stage of liver disease. Liver fibrosis is caused by chronic damage to the liver and the accumulation of extracellular matrix proteins. Without treatment, fibrosis can further progress to cirrhosis, liver cancer, liver failure, and death. Therefore, accurate diagnosis and monitoring of liver fibrosis are crucial.
[0003] The current gold standard for diagnosing liver fibrosis is liver biopsy, but several limitations exist associated with it. Liver biopsy is invasive and causes anxiety. In some extreme cases, complications such as post-biopsy bleeding and liver rupture can lead to death in patients with comorbidities. Therefore, there is a desire to develop a non-invasive and accurate measurement tool for the assessment and management of liver fibrosis to facilitate better patient management.
[0004] Shear wave elastography has shown promise as a non-invasive tool for measuring soft tissue stiffness. Ultrasound shear wave elastography (UE) and magnetic resonance shear wave elastography (MRE) are both quantitative imaging techniques used for the diagnosis and monitoring of liver fibrosis. (See reference below.) Figure 1 The diagram illustrates an overview of the advanced system setup of system 10 for magnetic resonance shear wave elastography. For MRE, the patient is placed on the platform of MRI scanner 12. A passive actuator / transducer 14 is positioned relative to the region of interest 16 on the patient. The patient is then restrained within the MRI scanner during the elastography measurement. Additionally, regarding MRE, a low-frequency (~60 Hz for the liver) continuous acoustic vibration is transmitted to the region of interest of the object (e.g., the abdomen) using a passive actuator, thereby generating harmonics with a narrow frequency bandwidth, meaning the wave energy is concentrated at the desired frequency. Figure 1 The bottom illustration shows examples of six different hardness maps (collectively designated by reference numeral 18) corresponding to the fibrosis stage obtained via MRE. Observe the MRE hardness maps... Figure 1 From left to right (i.e., as the level of fibrosis increases), the corresponding fibrosis stages are designated as including F0: normal (no fibrosis); F1: portal fibrosis without septum; F2: portal fibrosis with rare septum; F3: extensive septum without cirrhosis; and F4: cirrhosis.
[0005] Now go to Figure 2 The figure illustrates a system setup overview of system 20 for ultrasonic shear wave elastography. Unlike the MRE method, the UE method uses high-intensity, short-duration ultrasonic "push" pulses to generate shear waves with a wide spectrum. In the excitation step of the UE method, it is difficult to accurately control how the wave energy is distributed within this wide spectrum. Figure 2 The figure also illustrates an example of a hardness map obtained via the UE (as specified by reference numeral 22), in which a corresponding color-coded hardness scale 24 is present in the upper right part of the hardness map.
[0006] Ultrasound imaging providers use time-of-flight methods to reconstruct shear wave velocities similar to group velocities from a wide bandwidth; this is also provider-dependent. Due to differences in shear wave frequency, the chosen tissue viscoelasticity model, and the time delay estimation method, the shear wave group velocity (m / s) and subsequent hardness value (kPa) reported by the ultrasound system do not match the velocity or hardness value (kPa) measured by MRE. These differences make MRE and UE hardness measurements non-interchangeable and therefore difficult to directly compare measurements obtained using UE and MRE.
[0007] No international clinical organization has yet issued guidelines for selecting elastography methods (UE or MRE) for the diagnosis and monitoring of liver fibrosis. MRE is highly reproducible in both healthy individuals and patients with liver fibrosis. Furthermore, MRE has a higher success rate than UE in diagnosing liver fibrosis (93% for MRE vs. 82% for UE). Additionally, MRE offers more robust quantification performance, i.e., reduced sampling variability due to its higher shear wave SNR, 3D volumetric imaging, and the ability to assess a larger proportion of the liver. However, MRE has some limitations. Additional time is required to locate the passive actuator, and the transducer causes discomfort in some patients.
[0008] Compared to MRE, UE is cheaper, faster, highly portable, and widely available, and has been independently validated in many clinical centers worldwide. There are also clinical sites without access to MRI scanners, and some patients prefer not to undergo MRE, again due to the constraints of MRI scanners. In these cases, UE is superior to MRE during the monitoring phase of liver fibrosis due to its greater clinical value. Therefore, depending on the availability of the clinical research system, patient comfort levels, and the cost of care, either MRE or UE is preferred for the diagnosis and monitoring of liver fibrosis.
[0009] Although both MRE and UE are used for the diagnosis of liver fibrosis, due to the aforementioned technical differences between MRE and UE, using the same imaging modality for monitoring and consistency of measurements over time for a given patient is preferred. Using the same imaging modality ensures that changes in stiffness over time are only pathological and due to patient-related factors, rather than due to technical differences between MRE and UE (excitation frequency, instrument-dependent dependence, and post-processing algorithms). However, MRE is expensive, and performing MRE measurements or follow-up examinations for liver monitoring at several time points imposes a significant financial burden on patients. Given the lower cost and wider availability of UE compared to MRE, its use for subsequent liver fibrosis monitoring is expected. Furthermore, some pediatric and elderly patients may experience discomfort during MRI scans, which can make MRE examinations more challenging and time-consuming (i.e., extensive movement and repetitive measurements).
[0010] Therefore, an improved method and apparatus are desired to overcome the problems in this art. It is also desirable to provide a non-invasive and accurate measurement tool for assessing and monitoring liver fibrosis for better patient management, eliminating dependence on a single imaging modality. Summary of the Invention
[0011] According to one aspect, embodiments of the present invention bridging the gap between MRE and UE for monitoring liver fibrosis by 1) predicting the corresponding MRE stiffness value and confidence level (or score) solely by performing UE measurements and 2) providing a smart panel that shows the fibrosis stage, confidence score, changes in stiffness values on graphs and tables, and stiffness maps of the original and predicted liver fibrosis.
[0012] According to one embodiment, a method for enhanced interpretation of shear wave elastography between a first imaging mode and a second imaging mode includes: performing elastography measurements via a second imaging mode different from the first imaging mode to obtain at least one second imaging mode elastography value for a region of interest. The method further includes, via a processor, predicting at least one corresponding first imaging mode elastography value solely based on the obtained second imaging mode elastography value. After prediction, the method includes generating a graphical user interface or smart report panel via the processor and a display, the graphical user interface or smart report panel including or displaying (i) the fibrillation level of the region of interest, wherein the fibrillation level is determined based on (i)(a) the at least one second imaging mode elastography value and / or (i)(b) the predicted at least one corresponding first imaging mode elastography value.
[0013] In one embodiment, the second imaging modality is selected between (i) magnetic resonance shear wave elastography (MRE) and (ii) ultrasound shear wave elastography (UE). The first imaging modality includes the unselected imaging modality of the second imaging modality. Due to technical differences in the excitation frequency and post-processing algorithm of each respective imaging modality, the elastography values obtained via the second imaging modality are not comparable to the elastography values obtained via the first imaging modality.
[0014] According to an embodiment, both the at least one second imaging modality elastography value and the at least one corresponding predicted first imaging modality elastography value include one or more of (i) a hardness map and (ii) a hardness value in kPa. For example, the hardness value includes the average of the corresponding hardness maps. In another embodiment, the method includes: predicting a confidence score associated with the at least one second imaging modality elastography value via the processor, wherein the predicted confidence score includes the highest confidence percentage among a plurality of fibrosis levels F0-F4 measured by the second imaging modality elastography; and selecting a predicted fibrosis level via the processor based on the fibrosis level having the highest confidence percentage.
[0015] According to another embodiment, the smart reporting panel further includes (or shows) (ii) an initial baseline fibrillation level of the region of interest. The baseline fibrillation level is determined based on at least one baseline first imaging modality elastography value of the region of interest obtained via a baseline elastography measurement performed via the first imaging modality. In one embodiment, the baseline elastography measurement is performed prior to the elastography measurement via the second imaging modality. In another embodiment, the smart reporting panel further includes (or shows) (iii) a percentage change in elastography values between: (iii)(a) at least one first imaging modality elastography value predicted based on at least one second imaging modality elastography value obtained, and (iii)(b) the at least one baseline first imaging modality elastography value. Additionally, the smart reporting panel further includes (shown): (iv) the at least one second imaging modality elastography value of the region of interest, (v) at least one corresponding first imaging modality elastography value predicted for the region of interest, and (vi) a confidence score associated with the at least one second imaging modality elastography value. The confidence score is a percentage between 0% and 100%, and represents the confidence level of the fibrosis level based on the elastic imaging value of the at least one second imaging modality.
[0016] In one embodiment, predicting the at least one corresponding first imaging modality elastography value includes initiating a deep learning-based algorithm to predict the at least one corresponding first imaging modality elastography value. The deep learning-based algorithm includes a generative adversarial network (GAN) and / or a convolutional neural network (CNN). In another embodiment, the method further includes determining a confidence score associated with the at least one second imaging modality elastography value via the processor and the second deep learning-based algorithm. The determination of the confidence score is automatically activated simultaneously with the initiation of the deep learning-based algorithm. The confidence score is also used to provide a real-time classification of the fibrillation level based on the at least one second imaging modality elastography value, or a confidence level of the fibrillation level.
[0017] According to another embodiment, a system for enhanced interpretation of shear wave elastography between a first imaging mode and a second imaging mode includes an input and a controller. The input is configured to receive imaging data related to elastography measurements obtained from the second imaging mode. The controller is configured to perform elastography measurements via a second imaging mode different from the first imaging mode to obtain at least one second imaging mode elastography value for a region of interest. The controller is also configured to predict at least one corresponding first imaging mode elastography value based solely on the obtained second imaging mode elastography value. After prediction, the controller is further configured to generate a graphical user interface or smart report panel including (or showing) (i) the fibrillation level of the region of interest, wherein the fibrillation level is determined based on (i)(a) the at least one second imaging mode elastography value and / or (i)(b) the predicted at least one corresponding first imaging mode elastography value. The system also includes a display communicating with the controller to display the smart report panel.
[0018] After reading and understanding the following detailed description, those skilled in the art will appreciate the additional advantages and benefits. Attached Figure Description
[0019] The embodiments of this disclosure may take the form of various components and arrangements of components, as well as various steps and arrangements of steps. Therefore, the accompanying drawings are for illustrative purposes and should not be construed as limiting the embodiments. In the drawings, the same reference numerals denote the same elements. It should be noted that the drawings may not be drawn to scale.
[0020] Figure 1 This is an illustrative flowchart of a method for magnetic resonance shear wave elastography (MRE);
[0021] Figure 2This is an illustrative flowchart of a method for ultrasonic shear wave elastography (UE);
[0022] Figure 3 It is a graphical representation of the shear modulus (i.e., hardness) obtained through MRE;
[0023] Figure 4 It is a graphical representation of the shear modulus obtained through the UE;
[0024] Figure 5 This is an illustrative flowchart illustrating the steps of predicting a corresponding first imaging modality elastic imaging value based solely on an obtained or measured second imaging modality elastic imaging value, according to embodiments of this disclosure. For clarity, in Figure 5A , 5B The flowchart view is reproduced in 5C;
[0025] Figure 6 This is an illustrative flowchart illustrating the steps of predicting confidence scores according to embodiments of the present disclosure. For clarity, in Figure 6A , 6B The flowchart portion is reproduced in 6C, where the predicted confidence score involves elastic imaging values obtained or measured via a second imaging mode;
[0026] Figure 7 This is an illustrative graphical user interface view of a smart report panel according to embodiments of the present disclosure, wherein, for clarity, Figure 7 exist Figure 7 continues Continue on the form in the diagram. Also, for clarity, in Figure 7A and 7B The flowchart view is reproduced in the middle; and
[0027] Figure 8 This is a block diagram view of a system for enhanced interpretation of shear wave elastic imaging between first and second imaging modes according to embodiments of the present disclosure. Detailed Implementation
[0028] Embodiments of this disclosure and their various features and advantageous details will be explained more fully with reference to the non-limiting examples described and / or illustrated in the accompanying drawings and detailed in the following description. It should be noted that the features shown in the drawings are not necessarily drawn to scale, and as those skilled in the art will recognize, features of one embodiment may be used with other embodiments, even if not explicitly stated herein. Descriptions of well-known components and processing techniques may be omitted to avoid unnecessarily overwhelming the embodiments of this disclosure. The examples used herein are intended only to facilitate understanding of how embodiments of the invention can be practiced and to further enable those skilled in the art to practice embodiments of the invention. Therefore, the examples herein should not be construed as limiting the scope of embodiments of this disclosure, which is defined only by the claims and applicable law.
[0029] It should be understood that the embodiments of this disclosure are not limited to the specific methods, protocols, devices, apparatuses, materials, applications, etc., described herein, as these can vary. It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the claimed embodiments. It must be noted that the singular forms “a,” “an,” and “described” as used herein and in the claims include plural references unless the context clearly indicates otherwise.
[0030] Unless otherwise defined, all scientific and technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments of this disclosure pertain. Preferred methods, apparatus, and materials are described; however, any methods and materials similar to or equivalent to those described herein may also be used in practice or to test the embodiments.
[0031] To gain a more comprehensive understanding of the embodiments of this disclosure, we first consider at least two problems / limitations known in the art that are overcome by the embodiments, as discussed below. Regarding the first problem, although there are stiffness cutoff values in both the UE and MRE used as the true measure for staging liver fibrosis derived from liver biopsy, the stiffness values in the MRE and UE on the same patient are not comparable. For example, Figure 3 The figure shows a graph 110 comparing stiffness (kPa) of magnetic resonance shear wave elastography (MRE) with liver fibrosis levels (F0, F1, F2, F3, and F4). Regarding graph 110, the left vertical scale 112 includes a stiffness scale in kPa for the first imaging modality, and the horizontal scale 114 includes a fibrosis level scale from F0 to F4 for the first imaging modality. Figure 4The figure illustrates graph 130 comparing the stiffness (kPa) of the ultrasound shear wave elastography (MRE) UE with the level of liver fibrosis (F0, F1, F2, F3, and F4). Regarding graph 130, the left vertical scale 132 includes a stiffness scale in kPa for the second imaging modality, and the horizontal scale 134 includes a fibrosis level scale from F0 to F4 for the second imaging modality. The fundamental reasons for the difference in stiffness values between the MRE and UE include (i) technical differences in the corresponding excitation frequencies and (ii) technical differences in the corresponding post-processing algorithms used for stiffness map reconstruction of the imaging modality.
[0032] Regarding the technical differences in excitation frequencies, let's consider the following. The external actuator used in MRE generates harmonics with a narrow frequency bandwidth, meaning wave energy can be concentrated at the desired frequency (e.g., ~60 Hz for the liver). On the other hand, the UE method uses high-intensity, short-duration ultrasonic "push" pulses to generate shear waves with a broad spectrum. Although the desired shear wave frequency in the UE can be filtered out in post-processing to make it comparable to the shear wave frequency in the MRE, it is difficult to control how the wave energy is distributed within that spectrum during the excitation step. The difference in the frequency components of the waves generated in these methods (single-frequency versus multi-frequency excitation) causes the shear wave group velocity in the UE to differ from and therefore mismatch the shear wave group velocity in the MRE.
[0033] Regarding the technical differences in post-processing algorithms used for hardness map reconstruction, let's consider the following further. In MRE, the Local Frequency Estimation (LFE) algorithm is typically used for hardness map reconstruction. LFE calculates the shear wave velocity by estimating the local spatial frequencies of the shear wave propagation mode through an algorithm that combines local estimates of instantaneous frequencies at several scales. However, in UE, the inverse wave equation or a 2D cross-correlation algorithm is used. Furthermore, the reconstructed hardness map in UE is 2D, while the reconstructed hardness map in MRE is 3D.
[0034] Refer again Figure 3 and 4 These figures further illustrate the shear modulus obtained by MRE and UE, respectively. It can be observed from the figures that for all levels of liver fibrosis (F0 to F4), the shear modulus obtained by UE ( Figure 4 The shear modulus (or hardness) obtained by MRE is higher than that obtained by MRE. Figure 3 The shear modulus (or stiffness) obtained is used. In further comparisons, MRE has a higher success rate than UE in diagnosing liver fibrosis (93% vs. 82%). MRE has better quantification performance due to its higher shear wave SNR, 3D volumetric imaging, and ability to assess a larger proportion of the liver, which reduces sampling variability.
[0035] However, compared to MRE, UE is cheaper, faster, highly portable, and widely available, and has been independently validated in many clinical facilities worldwide. There are also clinical settings where MRI scanners are not accessible, and some patients prefer not to undergo MRE again due to the constraint of being inside an MRI scanner. In these cases, UE is superior to MRE. Therefore, depending on the availability of the clinical research system, patient comfort levels, and the cost of care, either MRE or UE is preferred for the diagnosis and monitoring of liver fibrosis.
[0036] As in the past, if a patient is to undergo MRE / UE testing, then for consistency, monitoring of liver fibrosis must maintain MRE / UE (i.e., the same image modality) during follow-up measurements. However, MRE is expensive, and performing MRE measurements at several time points imposes a significant financial burden on patients and can be uncomfortable. Directly comparing UE and MRE measurements without proper understanding can adversely lead to misinterpretation. Another drawback of modality-related stiffness cutoff values is that they only provide discrete, staged or graded results (e.g., grade 2 or 3). However, during treatment, it is sometimes important to quantify continuous changes in stiffness values caused by treatment, even within the same stage. Embodiments of this disclosure advantageously provide a solution to help clinicians better quantify and monitor the progression / regression of liver fibrosis by enabling the interchangeable use of UE and MRE at different time points.
[0037] The second problem overcome by embodiments of this disclosure is bridging the gap between MRE and UE for subsequent fibrosis monitoring and to eliminate dependence on a single imaging modality. Embodiments of this disclosure provide an intelligent reporting system across both imaging modalities. In contrast, current clinical practice uses different lookup tables to find the level of fibrosis corresponding to the measured liver stiffness, depending on the type of elastography performed (MRE or UE). Given the varying levels of training among ultrasound and MRI technicians or users, it is inconvenient for them to correlate different stiffness levels at different time points using various imaging modalities. To address this problem, embodiments of this disclosure advantageously provide an intelligent report / panel with applicability that represents the stage of fibrosis at all time points across imaging modalities and shows changes in stiffness values from baseline to the current examination. The panel presented by embodiments of this disclosure also advantageously eliminates dependence on a single imaging modality and helps users more easily interpret changes in liver fibrosis, regardless of their level of expertise.
[0038] In particular, embodiments of this disclosure provide a solution to bridge the gap between MRE and UE stiffness results by bringing the UE closer to the MRE without requiring an MRE examination. According to one embodiment, the shear wave elastography system and method predict the corresponding MRE stiffness map (and / or stiffness value) and confidence level by performing only UE measurements, and vice versa. Specifically, the shear wave elastography system and method use the stiffness map measured by the UE as input and apply an artificial intelligence (AI) based algorithm to 1) predict the corresponding MRE stiffness map, and 2) predict the confidence level (or confidence score regarding the obtained stiffness value) on the obtained stiffness map. Finally, the system and method provide a cross-image modality (UE / MRE) representation of the fibrosis stage at multiple time points for a given patient, the variation of stiffness values on a graph, and the measured and predicted stiffness. Figure 1 The table generates a smart report / panel.
[0039] As will be understood from the disclosure herein, the methods and systems according to embodiments of this disclosure can be divided into three aspects. In a first aspect, the method includes predicting a corresponding MRE stiffness value by performing only UE measurements, and vice versa. In a second aspect, the method includes predicting a confidence score for the obtained stiffness value (e.g., an obtained liver stiffness value). Third, the method includes providing an intelligent report showing the fibrosis stage and the change in stiffness value over time for a given patient.
[0040] Now go to Figure 5 This illustrates an illustrative flowchart, indicated by reference numeral 30, of the steps for predicting a corresponding first imaging modality elastic imaging value based solely on an obtained second imaging modality elastic imaging value, according to an embodiment of the present disclosure. For clarity, in Figure 5A , 5B And reproduced in 5C Figure 5 The flowchart view is a portion thereof. Regarding the prediction of the corresponding MRE stiffness value, consider the following as an example. The method begins by performing an elastic imaging measurement via a second imaging mode different from the first imaging mode to obtain at least one second imaging mode elastic imaging value for the region of interest, indicated by reference numeral 32. In this example, the second imaging mode is the UE, and the first imaging mode is the MRE. Refer again... Figure 2 Ultrasound UE measurements are performed on a given region of interest (ROI) of the patient 21 using UE device 20. UE device 20 includes, for example, an ultrasound probe 25, a graphical user interface or ultrasound scanner touch panel 26, and a display 27. The system operator or user 23 can perform ultrasound UE measurements on the ROI and display them on the display device 27. Figure 2The obtained measurement results, such as the hardness map 22 and the corresponding color-coded hardness scale 24, are displayed as indicated by reference numeral 29 in the accompanying drawings. After obtaining the UE measurement results via the UE device 20, a soft button 28 is provided on the graphical user interface of the UE device or the touch panel 26 of the ultrasonic scanner. The soft button 28 may be labeled with a message such as "Obtain the corresponding MRE hardness map (or hardness value)".
[0041] When the user selects soft button 28, the ultrasound scanner 20 initiates a process... Figure 5 The corresponding hardness value prediction application is specified by reference numeral 34 in the accompanying drawings. The hardness value prediction application includes suitable program instructions for predicting at least one corresponding first imaging modality elasticity value via a processor based solely on the obtained second imaging modality elasticity value. In one embodiment, the hardness value prediction application is configured to implement a deep learning-based algorithm, such as a generative adversarial network (GAN) or a convolutional neural network (CNN).
[0042] Deep learning is a type of artificial neural network that attempts to mimic the multi-layered human cognitive system. GANs and CNNs are examples of deep learning-based algorithms known in the field, and will only be briefly described in this article. Now refer to... Figure 5B The Generative Adversarial Network (GAN) 34 includes a deep learning architecture comprising two networks: a generator 341 and a discriminator 342. The generator and discriminator networks are jointly trained via a backpropagation algorithm using a cost function 343. Training enables the generative network 341 to generate more realistic samples (e.g., real), and the discriminator network to better distinguish artificially generated samples (e.g., generated). According to embodiments of this disclosure, a deep learning-based algorithm is initiated to predict the corresponding MRE stiffness map 36 based solely on the obtained UE stiffness map (or elastic imaging value) 32. Additionally, the training set 344 is input to the discriminator 342 via a processing block "x," as indicated by reference numeral 345. In operation, the obtained region of interest 33 ( Figure 5A The UE hardness map 32 is input to the generator 341 of the GAN network 34 via the processing block "z" indicated by reference numeral 346. The output of the generator 341 is input to the discriminator 342 via the processing block "x-hat" indicated by reference numeral 347. The discriminator 342 may output the predicted MRE hardness map 36 and / or the corresponding predicted average value (i.e., hardness value) 38 of the MRE hardness map, such as... Figure 5C As shown in the diagram. Additionally, the generator and discriminator networks are jointly trained with the backpropagation algorithm via the backpropagation output 348 of the cost function 343. The corresponding predicted average values of the predicted MRE hardness maps 36 and 38 are then stored in a suitable memory (not shown) of the ultrasound scanner 20.
[0043] For training, UE and MRE measurements can be performed on the same day in a group of patients with different levels of liver fibrosis and healthy controls. In the deep learning-based algorithm, ultrasound UE stiffness maps and MRE stiffness maps are used as (i) inputs and (ii) real-world data, respectively.
[0044] The training phase can be vendor-agnostic and is not limited to a single vendor's scanner. Furthermore, since the MRE hardness map is three-dimensional (3D) (unlike the UE), for consistency, UE hardness maps in the same plane should be fed into the GAN network 34. To match the UE and MRE planes, a suitable navigation system is preferably used for guidance to ensure that the same MR plane is selected for UE measurement. During UE measurement, the user or UE system operator 23 ( Figure 2 Using UE system 20 ( Figure 2 ) Moving ultrasound probe 25 ( Figure 2 The process continues until the user reaches the matching MR plane. Upon reaching the matching MR plane, the user then begins acquiring the hardness measurement results.
[0045] Now for reference Figure 6 An illustrative flowchart illustrating the steps of predicting a confidence score, uniformly designated by reference numeral 40, according to an embodiment of the present disclosure, wherein the predicted confidence score relates to an elastic imaging value (i.e., an actually measured elastic imaging value) obtained via a second imaging mode. For clarity, in Figure 6 Sections 1, 6B, and 6C reproduce portions of the flowchart view. An automatic confidence score for the obtained hardness values is determined as discussed below. (See references in this document.) Figure 5 In the preceding steps discussed, the application or function that automatically determines the confidence score for the obtained hardness value (as specified by reference numeral 42) and the deep learning-based application 34 (i.e., the deep learning-based algorithm) are simultaneously activated. A deep learning-based algorithm (such as a convolutional neural network (CNN)) is used to implement the confidence score (as specified by reference numeral 44) of the obtained or actually measured hardness map to provide real-time classification. Figure 6 ).
[0046] Convolutional Neural Networks (CNNs) encompass deep learning architectures and are used in medical imaging research due to their ability to preserve spatial relationships while filtering input images. Spatial relationships in medical imaging define tissue interfaces, structural boundaries, or joints between muscles and bones; and therefore, CNNs have received particular attention. Figure 6B A schematic diagram of a convolutional neural network 42 is shown below. (Reference) Figure 6 , 6ALike 6B, CNN 42 takes the input image of raw pixels (e.g., UE hardness map 32) and transforms it via convolutional layers (Conv) 421 and 423, a rectified linear layer (Flatten) 425, and pooling layers (Pool) 422 and 424. This is then fed into fully connected layers (FC) 426 and 427, which assign probabilities or class scores to classify the input (i.e., UE hardness map 32) into the class with the highest probability (Softmax) 428. Pixels in the image can be processed by different weights in the fully connected layers, or alternatively, the same set of weights can be used to process each location to extract various recurring patterns across the entire image. These trainable weights are called kernels or filters; and are applied using dot products or convolutions, and then processed by a nonlinear function. Each convolutional layer can have several filters to extract multiple sets of patterns at each layer. Each convolutional layer (Conv) 421 and 423 is typically followed by a pooling layer (Pool) 422 and 424, which reduces the dimensionality and applies translation invariance. These convolutional and pooling layers can be stacked together to form a multi-layer network that typically ends with one or more fully connected layers 426 and 427.
[0047] When the region of interest is 33 ( Figure 6 .1) When including the patient's liver, the output of the deep learning-based algorithm is the level of liver fibrosis with the highest confidence score of 44. Figure 6C In other words, a probability or category score is assigned to each category to classify the input (e.g., elastography value 32) into the corresponding level of fibrosis (level 1, level 2, level 3, level 4, and no fibrosis, specified by reference numerals 441, 442, 443, 444, and 445, respectively). The output is then the category with the highest probability representing the level of liver fibrosis. For example, if it is generated by an AI model (i.e., Figure 6 If the highest confidence score predicted by the convolutional neural network (CNN) belongs to fibrosis level 2 (e.g., confidence score for level 2 = 0.82, confidence score for level 1 = 0.05, confidence score for level 3 = 0A0, confidence score for level 4 = 0.02, and confidence score for no fibrosis = 0.01), then the confidence score is 82%, and this means that we are 82% confident that the stiffness value measured by the second imaging modality (e.g., UE modality) represents level 2 fibrosis. The highest probability and the corresponding fibrosis level are stored in a suitable memory (not shown) and output or displayed via GUI 26 or on the smart report panel 50, as will be discussed in this document. Figure 7 Further discussion is needed. For example, once the confidence score is predicted (e.g., Figure 6 44) and hardness chart (e.g., Figure 5(36) renders or makes the "Smart Report" soft button 28 appear on the ultrasound scanner touch panel 26, so that the system user 23 can access it via the Smart Report panel 50 ( Figure 7 Click soft button 28 to view the final results. As used herein, the fibrosis stage should be understood as the same as the fibrosis level. Furthermore, the predicted confidence level and the predicted confidence score are used interchangeably in this paper, further based on actual measured elastography values (e.g., measured UE stiffness maps).
[0048] In one embodiment, the predicted confidence score is correlated with the measured stiffness value in the second imaging modality. Specifically, the predicted confidence score of the measured stiffness value in the second imaging modality indicates the confidence level of the reported fibrosis level measured by the second imaging modality. Liver biopsies are used as the real-world scenario in the training phase to calibrate / evaluate the AI model used to predict the confidence score of fibrosis level. In other words, regarding the deep learning-based algorithm in the training phase, the UE stiffness map and liver biopsy are used as (i) input and (ii) real-world scenario, respectively. For training, UE and liver biopsies can be performed on the same day on a group of patients with different levels of liver fibrosis. This training is applied to the determination of the "confidence score".
[0049] Now for reference Figure 7 An illustrative graphical user interface view of a smart reporting panel (designated by reference numeral 50) according to an embodiment of the present disclosure is shown. For clarity, in Figure 7A and 7B The flowchart portion is reproduced in the image. The Smart Reporting Panel 50 includes a graphical user interface that can incorporate one or more forms of multimedia. Given the varying training levels of ultrasound and MRI users, it is inconvenient for them to use various imaging modalities to correlate different stiffness levels at different time points. The Smart Reporting Panel 50 helps users better monitor changes in fibrosis.
[0050] When the user clicks the "Smart Report" soft button 28 on the ultrasound scanner touch panel 26, a smart panel 50 will be automatically generated, including one or more of the following: Figure 7 (This provides references to other documents.) Figure 6The current fibrosis stage or level (as specified by reference numeral 52) and the corresponding prediction confidence score (as specified by reference numeral 54) obtained in the preceding steps discussed herein. In one embodiment, the current fibrosis stage 52 includes a fibrosis level 521, an elastic imaging value 522 (e.g., a measured UE stiffness value) obtained via a second imaging modality elastography measurement, and an indication of a predicted elastic imaging value 523 (e.g., a predicted MRE stiffness value) corresponding to a first imaging modality elastography measurement result based on the obtained second imaging modality elastography measurement result. The elastic imaging value 522 obtained via the second imaging modality elastography measurement and the predicted elastic imaging value 523 corresponding to the first imaging modality measurement result are as at least referenced herein. Figure 5 This is implemented as discussed in the UE stiffness graph 32, the predicted MRE stiffness graph 36, and the predicted MRE stiffness value 38. In one embodiment, new stiffness values (i.e., the measured UE value 522 and / or the corresponding predicted MRE value 523) can be automatically added to the graph designated by reference numeral 56 and the table designated by reference numeral 58 of existing stiffness values measured during past examinations. Graph 56 and / or table 58 can advantageously provide a meaningful illustration of the change in liver stiffness over time. Reference now Figure 7 .1. Graph 56 shows the level of fibrosis obtained in one or more previous examinations. Regarding graph 56, the left vertical scale 561 includes a hardness scale of the second imaging modality in kPa, and the horizontal scale 562 includes a hardness scale of the first imaging modality in kPa, and the right vertical scale 563 includes a scale of fibrosis levels from level 0 to level 4. As illustrated in the example of graph 56, a decrease in fibrosis level was observed from the baseline (10 / 03 / 19) obtained via the first imaging modality to the subsequent measurement (04 / 02 / 20) obtained via the second imaging modality, corresponding to a change in fibrosis level from level 3 to level 2. Still refer to Figure 7B Table 58 also includes a fibrosis stage 51, which includes indications of the fibrosis level 511 and elastography value 512 obtained via a baseline (10 / 03 / 2019) using elastography measurements (e.g., measured MRE stiffness values) in the first imaging modality. Figure 7 and 7B In the middle, the smart panel 50 also shows, as referenced in this article Figure 5The measured hardness map and predicted hardness map (specified by reference numerals 60 and 62, respectively), and the baseline hardness map (specified by reference numeral 64), are obtained as discussed. It should be noted that the baseline hardness map 64 is a previously obtained hardness map (such as at the initial fibrosis check) obtained via one or more of the MRE or UE imaging modalities. Hyperlinks (specified by reference numeral 66) are included in the smart panel 50 to redirect the user to more information about the MRE / UE cutoff values, such as guidance materials (reference documents, etc.) regarding the differences between MRE / UE. Furthermore, the smart panel 50 provides findings, such as increases or decreases in fibrosis hardness, which may be represented as a findings field (specified by reference numeral 68) and / or by appropriate symbols (e.g., percentage figures and graphs, with the graph pointing upwards to increase or downwards to decrease) (specified by reference numeral 70). Figure 7 As shown, the intelligent reporting panel 50 displays stiffness values, changes in liver fibrosis levels on graphs and tables, measured and predicted stiffness graphs, and confidence scores.
[0051] As should be understood herein, stiffness maps can be predicted in either direction. There are clinical sites where MRI scanners are inaccessible, or some patients dislike undergoing MRE due to confinement within an MRI scanner. In these cases, UE is superior to MRE during the fibrosis diagnostic phase. However, during the treatment phase, MRE can be recommended for more accurate monitoring of liver fibrosis because it offers higher success rates and more robust quantification performance compared to UE due to its higher shear wave SNR, 3D volumetric imaging, and assessment of a larger proportion of the liver. Therefore, there are situations where patients undergo UE for fibrosis diagnostic examination and MRE for longitudinal follow-up measurements. In such cases, the step of predicting the MRE stiffness map according to one embodiment of this disclosure can be replaced by predicting the UE stiffness map and stiffness value based on the measured MRE stiffness map. The same deep learning-based algorithm and training process can be used, except that in this case, (i) the input and (ii) the ground truth will be the MRE and UE stiffness maps, respectively.
[0052] Now for reference Figure 8This diagram illustrates a block diagram of a system 100 for enhanced interpretation of shear wave elastography between first and second imaging modalities according to an embodiment of the present disclosure. In this embodiment, system 100 includes at least a controller 102, a display 104, and one or more input / output devices 106. Depending on the requirements of a given shear wave elastography implementation, controller 102 is operatively coupled to one or more shear wave imaging modal systems or devices, such as MRE system / device 10 and / or UE system / device 20. In one embodiment, further depending on the requirements of a given shear wave elastography implementation and / or application, controller 102 includes one or more of a microprocessor, microcontroller, field-programmable gate array (FPGA), integrated circuit, discrete analog or digital circuit components, hardware, software, firmware, or any combination thereof for performing the various functions discussed herein. Controller 102 may also include one or more of various modules, units, or subsystems. In one embodiment, depending on the requirements of a given shear wave elastography system implementation and / or application, controller 102 may suitably include system electronics, which include one or more of various modules, units, or subsystems, power supplies, and memories. In one embodiment, depending on the requirements of a given shear wave elastography system implementation, display 104 is operatively coupled to controller 102 for displaying one or more soft buttons (e.g., soft button 28 as discussed herein) and intelligent reporting panel 50 or one or more portions thereof (e.g., current fibrosis stage, confidence score, graphs of hardness values, tables showing changes in hardness over time, obtained and predicted hardness graphs, hardness values, increases or decreases in fibrosis hardness, etc.). Depending on the requirements of a given shear wave elastography system implementation, input / output device(s) 106 may include one or more of the following: input / output devices, user interfaces, haptic output devices, touchscreens, optical displays, microphones, keypads, keyboards, indicating devices, image capture devices, cameras, audio output devices, and any combination thereof.
[0053] As discussed herein, various aspects of embodiments of this disclosure relate to, for example, user interface components with elastography features that use the user experience (UE) to predict a corresponding MRE stiffness map (or vice versa, i.e., use MRE to predict a UE stiffness map). Embodiments of this disclosure can be incorporated into novel features of ultrasound / MR scanners, i.e., included in ultrasound (US) or magnetic resonance imaging (MRI) scanner devices. Additionally, the intelligent reporting systems / panels or graphical user interfaces and their features presented herein can be incorporated into ultrasound / MR scanners.
[0054] Therefore, embodiments of this disclosure do not require maintaining the same image modality for subsequent stiffness measurements to monitor the progression of liver fibrosis. This allows the use of a UE (body image) instead of an MRE (muscle image resonator) for subsequent stiffness measurements, advantageously contributing to reduced patient care costs. Furthermore, the smart panel / reporting according to embodiments of this disclosure advantageously eliminates dependence on a single image modality in the reporting system. The methods and system apparatus of this disclosure further advantageously enable users with varying levels of training / background to monitor changes in liver fibrosis.
[0055] While only a few exemplary embodiments have been described in detail above, those skilled in the art will readily understand that many modifications can be made to the exemplary embodiments without substantially departing from the novel teachings and advantages of the embodiments of this disclosure. For example, embodiments of this disclosure can be advantageously used by performing baseline shear wave elastography measurements in both the MRE and UE imaging modalities and subsequently performing one or the other MRE or UE based on (i) the availability of the given shear wave elastography imaging modality at a subsequent time point and (ii) a confidence score. At a subsequent time point, the MRE or UE can be selected based on the acceptability of the confidence score of the corresponding predicted imaging modality elastography value, as discussed herein. If the confidence score is determined to be unacceptable for one imaging modality, or outside a given acceptable range, another imaging modality can be selected, i.e., based on the confidence score being used as a criterion to aid in the selection between the MRE and the UE. Therefore, all such modifications are intended to be included within the scope of the embodiments of this disclosure as defined in the claims. In the claims, the functional module clauses are intended to cover not only the structures described herein that perform the functions, but also equivalent structures.
[0056] Furthermore, any reference numerals enclosed in parentheses in one or more claims should not be construed as limiting the claims. The terms "comprising" and "including" do not exclude the presence of elements or steps other than those listed as a whole in any claim or specification. Referring to an element in the singular does not exclude referring to such an element in the plural, and vice versa. One or more embodiments in these examples can be implemented by means of hardware comprising several different elements and / or by means of a suitably programmed computer. In apparatus-type claims that enumerate several units, several of these units can be implemented by the same item of hardware. Although specific measures are recited in dissimilar dependent claims, this does not imply that combinations of these measures cannot be advantageously used.
Claims
1. A method for enhanced interpretation of shear wave elastic imaging between a first imaging mode and a second imaging mode, the method comprising: Elastic imaging measurements are performed via a second imaging mode (20) different from the first imaging mode (10) to obtain at least one second imaging mode elastic imaging value (32) of the region of interest (33). The processor (102) predicts at least one corresponding first imaging modality elastic imaging value (36, 38) based on the obtained second imaging modality elastic imaging value (32); and A smart report panel (50) is generated via the processor (102) and the display (104), the smart report panel including (i) the fiberization level (521) of the region of interest, wherein the fiberization level is determined based on (i)(a) at least one second imaging modality elastography value (32) and / or (i)(b) at least one corresponding first imaging modality elastography value (36, 38) predicted by (i)(b).
2. The method according to claim 1, wherein, The second imaging mode is selected between (i) magnetic resonance shear wave elastography (MRE) (10) and (ii) ultrasound shear wave elastography (UE) (20), wherein the first imaging mode includes the unselected imaging mode of the second imaging mode.
3. The method according to claim 1, wherein, The elastic imaging value (22) obtained via the second imaging mode (20) is not comparable to the elastic imaging value (18) obtained via the first imaging mode (10).
4. The method according to claim 1, wherein, The at least one second imaging modal elastic imaging value (32) and the at least one corresponding predicted first imaging modal elastic imaging value (36, 38) each include one or more of (i) a hardness map and (ii) a hardness value in kPa.
5. The method according to claim 1, further comprising: The processor (102) predicts a confidence score (44) related to the at least one second imaging modality elastography value (32), wherein the predicted confidence score includes the highest confidence percentage among the confidence percentages of each of a plurality of fibrosis levels F0-F4 (441, 442, 443, 444, 445) measured based on the second imaging modality elastography; and The processor selects the predicted fibrosis level based on the fibrosis level with the highest confidence percentage (521).
6. The method according to claim 1, wherein, The intelligent reporting panel (50) also includes (ii) the baseline fibrillation level (511) of the region of interest, which has been determined based on a baseline elastography measurement performed via the first imaging modality (10) to obtain at least one baseline first imaging modality elastography value (64) of the region of interest, which has been performed prior to the elastography measurement via the second imaging modality.
7. The method according to claim 6, wherein, The intelligent reporting panel (50) further includes (iii) a percentage change (70) in the elastic imaging value between (iii)(a) at least one first imaging modality elastic imaging value (523) predicted based on at least one second imaging modality elastic imaging value (522) and (iii)(b) the at least one baseline first imaging modality elastic imaging value (511).
8. The method according to claim 7, wherein, The intelligent reporting panel (50) further includes: (iv) at least one second imaging modality elastography value (60) of the region of interest, (v) at least one corresponding first imaging modality elastography value (62) of the predicted region of interest, and (vi) a confidence score (54) relating to the at least one second imaging modality elastography value (522), wherein the confidence score is a percentage between 0% and 100% and represents the confidence level of the fibrosis level based on the at least one second imaging modality elastography value.
9. The method according to claim 1, wherein, Predicting the elastic imaging value of the at least one corresponding first imaging modality (36, 38) includes initiating a deep learning-based algorithm (34) to predict the elastic imaging value of the at least one corresponding first imaging modality, wherein the deep learning-based algorithm includes one selected from the group consisting of generative adversarial networks (GANs) and convolutional neural networks (CNNs).
10. The method of claim 9, further comprising: A confidence score (44) relating to the at least one second imaging modality elastography value (32) is determined via the processor (102) and the second deep learning-based algorithm (42), wherein the determination of the confidence score is automatically activated simultaneously with the initiation of the deep learning-based algorithm (34), and the confidence score is also used to provide a real-time classification of the fibrosis level based on the at least one second imaging modality elastography value (32) or a confidence level of the fibrosis level.
11. A system (100) for enhanced interpretation of shear wave elastic imaging between a first imaging mode and a second imaging mode, comprising: The input unit is used to receive imaging data related to elastic imaging measurements obtained from the second imaging mode (20); The controller (102) is configured as follows: Elastic imaging measurements are performed via a second imaging mode (20) different from the first imaging mode (10) to obtain at least one second imaging mode elastic imaging value (32) of the region of interest (33). Based on the obtained second imaging modality elastic imaging value (32), at least one corresponding first imaging modality elastic imaging value (36, 38) is predicted; and A smart report panel (50) is generated, the smart report panel including (i) the fiberization level (521) of the region of interest, wherein the fiberization level is determined based on (i)(a) the at least one second imaging modality elastography value (32) and / or (i)(b) at least one corresponding first imaging modality elastography value (36, 38) predicted by (i)(b); and A display (104) communicates with the controller (102) to display the smart report panel (50).
12. The system (100) according to claim 11, wherein, The second imaging mode is selected between (i) magnetic resonance shear wave elastography (MRE) (10) and (ii) ultrasound shear wave elastography (UE) (20), wherein the first imaging mode includes the unselected imaging mode of the second imaging mode.
13. The system (100) according to claim 11, wherein, The elastic imaging value (22) obtained via the second imaging mode (20) is not comparable to the elastic imaging value (18) obtained via the first imaging mode (10).
14. The system (100) according to claim 11, wherein, The at least one second imaging modal elastic imaging value (32) and the at least one corresponding predicted first imaging modal elastic imaging value (36, 38) each include one or more of (i) a hardness map and (ii) a hardness value in kPa.
15. The system (100) according to claim 11, wherein, The controller (102) is also configured to: A confidence score (44) is predicted related to the at least one second imaging modality elastography value (32), wherein the predicted confidence score includes the highest confidence percentage among the confidence percentages of each of the plurality of fibrosis levels F0-F4 (441, 442, 443, 444, 445) based on the elastography measurement; and The predicted fibrosis level is selected based on the fibrosis level with the highest confidence percentage (521).
16. The system (100) according to claim 11, wherein, The intelligent reporting panel (50) also includes (ii) the baseline fibrillation level (511) of the region of interest, which has been determined based on a baseline elastography measurement performed via the first imaging modality (10) to obtain at least one baseline first imaging modality elastography value (64) of the region of interest, which has been performed prior to the elastography measurement via the second imaging modality.
17. The system (100) according to claim 16, wherein, The intelligent reporting panel (50) further includes (iii) a percentage change (70) in the elastic imaging value between (iii)(a) at least one first imaging modality elastic imaging value (523) predicted based on at least one second imaging modality elastic imaging value (522) and (iii)(b) the at least one baseline first imaging modality elastic imaging value (511).
18. The system (100) according to claim 17, wherein, The intelligent reporting panel (50) further includes: (iv) at least one second imaging modality elastography value (60) of the region of interest, (v) at least one corresponding first imaging modality elastography value (62) of the predicted region of interest, and (vi) a confidence score (54) relating to the at least one second imaging modality elastography value (522), wherein the confidence score is a percentage between 0% and 100% and represents the confidence level of the fibrosis level based on the at least one second imaging modality elastography value.
19. The system (100) according to claim 11, wherein, Predicting the elastic imaging value of the at least one corresponding first imaging modality (36, 38) includes initiating a deep learning-based algorithm (34) to predict the elastic imaging value of the at least one corresponding first imaging modality, wherein the deep learning-based algorithm includes one selected from the group consisting of generative adversarial networks (GANs) and convolutional neural networks (CNNs).
20. The system (100) according to claim 19, wherein, The controller (102) is also configured to: A confidence score (44) is determined in relation to the at least one second imaging modality elastic imaging value (32), wherein the determination of the confidence score is automatically activated simultaneously with the initiation of the deep learning-based algorithm (34), and the confidence score is also used to provide a real-time classification of the fibrillation level based on the at least one second imaging modality elastic imaging value (32) or a confidence level of the fibrillation level.
Citation Information
Patent Citations
Ultrasonic elastic imaging device and elastic imaging result evaluation method
CN108733857A
Reverberant shear wave field estimation of body properties
CN109996498A
