Systems and methods for tissue evaluation and classification

Abstract

The present invention provides a system and method for assessing and classifying one or more lesions formed within intravascular and / or intracardiac tissue to assist in the diagnosis and / or treatment of heart-related diseases. Aspects of the present invention analyze images obtained during procedures such as catheter ablation procedures, and thereby identify, evaluate, and classify one or more lesions formed within targeted tissue (i.e., intravascular and / or intracardiac tissue) in real-time or near real-time to assist in the diagnosis and / or treatment of heart-related diseases, which can be accomplished using a platform configured to do so.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims the priority and benefit of U.S. Provisional Application No. 63 / 343,757, filed on May 19, 2022, the content of which is incorporated herein by reference in its entirety.

[0002] The present disclosure relates to the classification of biological tissues, and more specifically, to systems and methods for evaluating and classifying one or more lesions formed within intravascular and / or intracardiac tissue to assist in the diagnosis and / or treatment of heart - related diseases.

Background Art

[0003] Catheter ablation is a treatment in which energy is applied to heart tissue to create scars or lesions in order to prevent or block the transmission of abnormal electrical signals. Catheter ablation forms an essential part of the management of cardiac arrhythmias, including supraventricular tachycardia (SVT), atrial flutter (AFL), atrial fibrillation (AF), and ventricular tachycardia (VT).

[0004] As the acceptance and implementation of catheter ablation increase on a global scale, the limitations of current technologies are becoming increasingly apparent, particularly in the treatment of complex arrhythmias such as AF. For example, due to certain complex arrhythmia mechanisms such as AF, recurrence of arrhythmia is observed in more than half of patients. The relatively low effectiveness of AF treatment is likely due to limitations in mapping, incomplete understanding of the definitive mechanisms of arrhythmia, and most importantly, the inability to create trans - wall and durable lesions.

[0005] Successful catheter ablation requires not only precise localization of the arrhythmogenic substrate but also complete and permanent elimination of that substrate without producing collateral damage. The ablation effect depends on several factors, including the applied power, the quality of the electrical contact, the local tissue properties, the presence of blood flow in proximity to the tissue surface, and the effect of perfusion. Due to the variability of these parameters, it can be difficult to obtain consistent results and understand the ablation effect within the tissue using current systems and methods for ablation.

[0006] As a result, current ablation systems may be limited due to the difficulties and challenges in assessing the results of ablation within the tissue, such as the step of identifying the damage formed within the tissue and the step of determining the various properties of the damage through the catheter. Despite years of research and the emergence of improved imaging techniques, the reliable generation of effective and permanent damage remains a challenge. SUMMARY OF THE INVENTION MEANS FOR SOLVING THE PROBLEM

[0007] The present invention recognizes the drawbacks of current ablation systems, particularly the lack of reliable and accurate means for assessing the step of damage formation. Accordingly, the present invention provides an image analysis platform configured to provide automated tissue evaluation and classification based on artificial intelligence techniques to address such drawbacks.

[0008] Aspects of the present invention can be carried out using a platform configured to analyze images obtained during procedures such as catheter ablation procedures and thereby identify, evaluate, and classify one or more lesions formed within targeted tissue (i.e., intravascular and / or intracardiac tissue) in real-time or near real-time to assist in the diagnosis and / or treatment of heart-related diseases.

[0009] In particular, the present invention provides a computing system that activates a neural network trained using a plurality of training data sets, including eligible reference data that may include clinical data. For example, each training data set includes reference image data associated with a known tissue, and further includes classification data associated with the known tissue, and the plurality of training data sets exclude digital histopathology data. More specifically, the present invention recognizes that the assessment and verification of lesion formation are generally problematic due to histological requirements. In particular, the general drawbacks and difficulties of histology include time-consuming preparation (e.g., slicing, fixing, and staining of tissues on slides), the inherent risk of human error during the preparation and analysis of tissue samples, and the lack of specificity regarding conventional histological staining, which can affect its value as a diagnostic tool.

[0010] As an attempt to overcome such difficulties associated with histology, the present invention proposes the use of phase-contrast computed tomography (CT) imaging as an alternative approach to using digital histopathology data as an input for training data. The present invention recognizes that the direct visualization of cardiac ablation via phase-contrast CT can be particularly useful for bridging the gap between conventional imaging modality data (i.e., CT, magnetic resonance imaging (MRI), and ultrasound (US)) and histology, especially using lesion formation assessment.

[0011] Thus, in one aspect of the present invention, the reference image data of the training data set includes one or more images of a known tissue acquired and processed via a phase-contrast computed tomography (CT) imaging system and at least one other imaging modality including, but not limited to, a transmission imaging system, a bright-field or dark-field imaging system, a fluorescence imaging system, a differential interference contrast imaging system, a hyperspectral imaging system, a Raman or surface-enhanced Raman imaging system, and a magnetic resonance imaging (MRI) system.

[0012] Note that the reference image data relied upon for the input for the training data (i.e., phase contrast CT image data) has been verified in that a link has been established between the conventional histological data and the reference image data. In particular, the neural network is trained from multiple training data sets such that the neural network is suitable for evaluating and classifying tissues based on the association of the classification data with the reference image data. Generally, the reference image data is associated with reference lesions formed within a known tissue, and the classification data is associated with the reference lesions. The classification data may include, but is not limited to, characteristics of the reference lesion, such as the location of the reference lesion on a known tissue, the size of the reference lesion, the path of the reference lesion, the depth of the reference lesion, and the known success of the reference lesion in the treatment of heart-related diseases.

[0013] In other words, each training data set is associated with an individual known tissue that has been collected as part of a clinical study or equivalent. The known tissue may have one or more known lesions therein. The known tissue (including the known lesions formed therein) may undergo a process for the collection of both conventional histological data and reference image data. In particular, the reference image data may be collected from a clinical tissue sample (i.e., an image of the clinical tissue sample may be obtained via a phase contrast CT imaging system and / or other imaging modalities described herein). Similarly, the clinical tissue sample may be prepared and analyzed via conventional histological techniques to collect conventional histological data. As a result, a link is established between the reference image data and the histological data, thereby verifying what is shown in the reference image data (i.e., confirmation of what is shown in the image data based on the conventional histological techniques performed on the reference tissue sample, including the presence of any lesions and the characteristics of such lesions).

[0014] The computing system is configured to receive patient tissue data, which may include image data obtained by an imaging modality used during a procedure. For example, during a catheter ablation procedure, the imaging modality may be an ultrasound machine that provides an ultrasound image of the target site to the computing system. The target site may include a targeted area of intravascular and / or intracardiac tissue that is undergoing or has already undergone catheter ablation for the formation of one or more lesions to treat a heart disease such as AF or the like.

[0015] Note that the platform of the present invention can either be directly incorporated into the imaging modality (i.e., provided as a local component to the imaging machine or equivalent) or be cloud-based and accessible by an operator via an imaging system or computing device (i.e., smartphone, tablet, personal computer, or equivalent) to provide a digital web-based application.

[0016] In response to receiving image data of a patient undergoing a procedure, a computing system is configured to analyze the image data using a neural network based on an association of classification data and reference image data. Based on such analysis, the computing system can classify the tissue within the image. In particular, the training data set may include reference image data associated with reference lesions formed within known tissue, and the classification data is associated with the reference lesions. Thus, as part of the analysis, the computing system can correlate the patient's image data with known tissue data with reference lesions so that one or more lesion formations can be identified and further classified within the patient's image data. The classification of the identified one or more lesion formations may include various characteristics of the lesion formation that may be useful in assessing and validating the effectiveness of the lesion (i.e., predicting whether the lesion will successfully treat the disease). For example, the various characteristics that may be identified include, but are not limited to, the location of one or more lesion formations on the sample tissue, the size of one or more lesion formations, the path of one or more lesion formations, the depth of one or more lesion formations, and the known success of one or more lesion formations in the treatment of heart-related diseases.

[0017] In response to identifying and classifying tissue within the patient's image data, including any lesion formation, the computing system further outputs the results of the tissue evaluation and classification to a user (i.e., a clinician or other healthcare provider associated with the patient). For example, in some embodiments, the results may be provided to the user via a report that generally provides details about any lesion formation within the targeted tissue of the captured image.

[0018] Accordingly, by providing real-time, automated tissue evaluation and classification based on artificial intelligence techniques, the present invention addresses the limitations of current ablation systems, particularly the lack of reliable and accurate means for assessing lesion formation. More specifically, the present invention reduces the need for any specialized training when evaluating image data for assessing and validating lesion formation. Further, the present invention does not require conventional histology to provide lesion classification, which can present challenges particularly with respect to complex alignment processes and issues related to deformation and / or tissue displacement that further impair the alignment process. Rather, the neural network of the present invention is trained using a plurality of training data sets that include qualified reference data excluding digital histopathology data. More specifically, the present invention utilizes phase contrast CT image data as an alternative approach to using digital histopathology data as an input for training data. Thus, the present invention provides a system that is highly effective for evaluating and classifying one or more lesions formed within intravascular and / or intracardiac tissue to assist in the diagnosis and / or treatment of heart-related diseases.

[0019] One aspect of the present invention includes a method for training a neural network to evaluate and classify tissue. The method includes providing a plurality of training data sets to a computing system, each training data set including reference image data associated with a known tissue and classification data associated with the known tissue, the plurality of training data sets excluding digital histopathology data. The method further includes training the neural network from the plurality of training data sets such that the neural network is suitable for evaluating and classifying tissue based on an association between the classification data and the reference image data.

[0020] The reference image data may include an image of one or more known tissues that is acquired and processed via an imaging modality selected from the group consisting of an ultrasonic imaging system, a computed tomography (CT) imaging system, a transmission imaging system, a bright field or dark field imaging system, a fluorescence imaging system, a phase contrast imaging system, a differential interference contrast imaging system, a hyperspectral imaging system, a Raman or surface enhanced Raman imaging system, and a magnetic resonance imaging (MRI) system. The MRI system may perform at least one of a delayed gadolinium enhanced MRI and a diffusion weighted MRI sequence.

[0021] In some embodiments, the reference image data may include an image of a known tissue that is acquired and processed via an ultrasonic imaging system and a CT imaging system. For example, the reference image data may include three-dimensional (3D) ultrasonic image data of a known tissue and computed tomography (CT) image data. In some embodiments, the CT image data includes phase contrast CT image data. For example, the CT image data may include postmortem CT image data of a known tissue for anatomical reference and phase contrast CT image data of the known tissue. The 3D ultrasonic image data may be obtained from a catheter-based ultrasonic imaging device configured to provide full circumference 3D image data.

[0022] In some embodiments, the reference image data may be associated with a reference lesion formed within a known tissue, and the classification data is associated with the reference lesion. Thus, the classification data may include characteristics of the reference lesion, including at least one of the location of the reference lesion on the known tissue, the size of the reference lesion, the path of the reference lesion, the depth of the reference lesion, and the known success of the reference lesion in the treatment of heart related diseases.

[0023] The method may further include obtaining one or more images of the sample tissue undergoing an ablation procedure, processing the one or more images, inputting the sample image data obtained in the processing step into a computing system, correlating the sample image data with reference damage and known tissue data, and outputting the result of the correlation step. The result of the correlation step may include the identification of one or more damage formations within the sample tissue and the classification of the identified one or more damage formations. Thus, the classification of the identified one or more damage formations may include at least one of the location of the one or more damage formations on the sample tissue, the size of the one or more damage formations, the path of the one or more damage formations, the depth of the one or more damage formations, and the known success of the one or more damage formations in the treatment of heart-related diseases, including the identified characteristics. It should be noted that in some embodiments, the method may further include providing a binary classification of the tissue (i.e., the ablated portion and the non-ablated portion). The method may further generally include providing a damage mapping, including reconstructing a complete 3D volume such that each point within the 3D volume includes the probability that the tissue will be ablated or not ablated, providing the likelihood or probability of damage formation as a 3D image representation. The result of the correlation step may further include further verifying one or more damage formations.

[0024] In some embodiments, the computing system may include a machine learning system selected from the group consisting of a neural network, a random forest, a support vector machine, a Bayesian classifier, a hidden Markov model, an independent component analysis method, and a clustering method. For example, in some embodiments, the computing system may include an autonomous machine learning system that associates classification data with reference image data. In some embodiments, the machine learning system may include a deep learning neural network that includes an input layer, a plurality of hidden layers, and an output layer. Still further, in some embodiments, the autonomous machine learning system may represent a training data set using a plurality of features, each feature comprising a feature vector. In some embodiments, the autonomous machine learning system may include a convolutional neural network.

[0025] In some embodiments, the method may further include operating the machine learning system to learn the relationship between reference image data, classification data, and damage formation via an ablation technique.

Brief Description of the Drawings

[0026]

Figure 1

[0027]

Figure 2

[0028]

Figure 3

[0029]

Figure 4

[0030]

Figure 5

[0031]

Figure 6

[0032]

Figure 7

[0033]

Figure 8

DETAILED DESCRIPTION OF THE INVENTION

[0034] (Detailed Description) In summary, the present invention is directed to a system and method for classifying biological tissues, including automated tissue evaluation and classification based on artificial intelligence techniques. More specifically, aspects of the present invention are configured to analyze an image of a target site of a patient undergoing a procedure and thereby identify features of the tissue within the image, including identification and classification of lesion formation at the target site, and can be performed using a platform.

[0035] In particular, the platform is configured to receive an input in the form of image data obtained by an imaging modality used during the procedure. For example, during a catheter ablation procedure, the imaging modality may be an ultrasound imaging machine in which an ultrasound image of the target site therein is provided to the platform. The target site may include a targeted area of intravascular and / or intracardiac tissue that is receiving or has already received catheter ablation for the formation of one or more lesions for treating a heart disease such as AF or the like.

[0036] The platform is configured to analyze the image data using a neural network that relies on a pre-trained dataset containing reference image data associated with known tissues and classification data associated with known tissues, and the plurality of training datasets exclude digital histopathology data. More specifically, the reference image data of the training dataset includes one or more images of known tissues obtained and processed via at least one other imaging modality including, but not limited to, a phase contrast computed tomography (CT) imaging system, an ultrasound imaging system, a transmission imaging system, a bright field or dark field imaging system, a fluorescence imaging system, a differential interference contrast imaging system, a hyperspectral imaging system, a Raman or surface enhanced Raman imaging system, and a magnetic resonance imaging (MRI) system.

[0037] In this example of a catheter ablation procedure, where the goal is to normally generate damage that results in the complete and permanent elimination of a target tissue or a part thereof, the training data set may include reference image data associated with reference damage formed within a known tissue, and the classification data is associated with the reference damage. Thus, as part of the analysis, the platform can correlate the patient's image data with known tissue data with reference damage so that one or more damage formations can be identified and further classified within the patient's image data. The classification of the identified one or more damage formations may include various characteristics of the damage formation that may be useful in assessing and validating the effectiveness of the damage (i.e., predicting whether the damage will successfully treat the disease). For example, the various characteristics that may be identified include, but are not limited to, the location of one or more damage formations on the sample tissue, the size of one or more damage formations, the path of one or more damage formations, the depth of one or more damage formations, and the known success of one or more damage formations in the treatment of heart-related diseases.

[0038] In response to identifying and classifying tissue within the patient's image data, including any damage formation, the platform further outputs the results of the tissue evaluation and classification to a user (i.e., a clinician or other healthcare provider associated with the patient). For example, in some embodiments, the results may be provided to the user via a report that generally provides details about any damage formation within the targeted tissue of the captured image.

[0039] Accordingly, by providing real-time, automated tissue evaluation and classification based on artificial intelligence techniques, the present invention addresses the limitations of current ablation systems, particularly the lack of a reliable and accurate means for assessing the formation of lesions. More specifically, the present invention reduces any need for specialized training when evaluating image data to assess and verify lesion formation. Further, the present invention does not require conventional histology to provide lesion classification, which can present challenges particularly with respect to complex alignment processes and issues associated with deformation and / or tissue displacement that can further impair the alignment process. Rather, the neural network of the present invention is trained using a plurality of training data sets that include qualified reference data that excludes digital histopathology data. More specifically, the present invention utilizes phase contrast CT image data as an alternative approach to using digital histopathology data as an input for training data. Thus, the present invention provides a highly effective system for evaluating and classifying one or more lesions formed within intravascular and / or intracardiac tissue to assist in the diagnosis and / or treatment of heart-related diseases.

[0040] While the following description focuses on the use of the present invention for classifying catheter ablation procedures and associated lesion formation, it should be noted that the systems and methods of the present invention can be used for classifying any type of tissue for any type of procedure where imaging analysis is used and / or preferred.

[0041] Figures 1A and 1B are schematic diagrams of an exemplary medical imaging system 10 that is compatible for use in conjunction with the systems and methods of the present invention to provide automated tissue evaluation and classification. The medical imaging system 10 may include any type of medical imaging modality, including but not limited to, an ultrasonic imaging system, a computed tomography (CT) imaging system, a transmission imaging system, a bright field or dark field imaging system, a fluorescence imaging system, a phase contrast imaging system, a differential interference contrast imaging system, a hyperspectral imaging system, a Raman or surface enhanced Raman imaging system, and a magnetic resonance imaging (MRI) system.

[0042] In the illustrated embodiment, the system 10 is an ultrasonic system and includes an imaging device 12 operatively coupled to a console 14 and a display 16. As generally understood, ultrasonic imaging (sonography) uses high frequency sound waves to visualize the inside of the body. Since ultrasonic images are captured in real time, they can also show the movement of internal organs and fluid flow (e.g., blood flowing through blood vessels) in the body. In sonography, an ultrasonic device 12, also referred to as a transducer probe, is placed directly on the skin or inside a body opening.

[0043] The ultrasonic transducer probe 12 is involved in transmitting and receiving sound waves that generate ultrasonic images through the piezoelectric effect, i.e., the phenomenon of rapidly vibrating a quartz crystal within the probe to send out sound waves. These waves are then bounced off an object and reflected back to the probe.

[0044] The transducer probe 12 is operatively coupled to a console 14 that generally controls the operation of the transducer probe 12 (i.e., the transmission of sound waves from the probe). The console 14 generally includes a central processing unit (CPU), a storage device, and some form of input section (i.e., a keyboard, knob, scroll wheel, or the like) through which an operator can interact to operate the machine, including adjusting the transmission characteristics of the probe, saving images, and performing other tasks. During operation, the CPU transmits a current that causes the probe 12 to emit sound waves. The CPU also analyzes electrical pulses generated in response to the reflected waves that return from the probe. This is then converted into an image (i.e., an ultrasonic image) that can be viewed on a display 16, which may be an integrated monitor. Such images may also be stored in memory and / or printed via a printer (not shown).

[0045] In the illustrated embodiment, the imaging device 12 may generally be in the form of an imaging catheter that can provide imaging and mapping capabilities and, in some embodiments, may be capable of providing energy delivery (i.e., ablation). Thus, such a device 12 may be useful in performing catheter ablation for treating heart diseases such as AF or the like. For example, in some embodiments, the catheter 12 may further include additional components that provide associated capabilities. For example, a portion of the catheter may include sensors (e.g., location and / or tracking sensors) and / or energy delivery elements (e.g., ablation elements).

[0046] The imaging catheter 12 may include a fully rotatable transducer unit consisting of an ultrasonic transducer array configured to transmit ultrasonic pulses into the circumferential intravascular tissue during a procedure and receive echoes of the ultrasonic pulses therefrom. Such ultrasonic transmissions are received by the console 14 and subsequently result in a collection of image data that is reconstructed into one or more images that provide visualization and characterization of the circumferential intravascular tissue. In particular, the console 14 may utilize the image data received from the imaging assembly of the imaging catheter 12 to reconstruct one or more images that provide 360-degree visualization of the entire circumference of the intravascular tissue. The console 14 may process the received image data using certain imaging protocols and algorithms to reconstruct the images and subsequently output to the operator, via a display, the reconstructed images (2-, 3-, or 4-dimensional images) depicting the visualization of the intravascular tissue. In addition to providing image reconstruction based on the image data received from the imaging assembly, the console 14 may further provide control of the imaging assembly, including controlling the emission (intensity, frequency, duration, etc.) of the ultrasonic pulses therefrom and controlling the movement of the ultrasonic transducer unit (i.e., controlling the rotation, including the speed and duration of the rotation). For example, the console 14 may be equipped with certain hardware and software to provide such image reconstruction and imaging assembly control as described in the international PCT application No. PCT / IB2019 / 000963 of Hennersperger et al. (published as WO 2020 / 044117), the content of which is incorporated herein by reference in its entirety.

[0047] As described above, the present invention is directed to a system and method for classifying biological tissues, including automated tissue evaluation and classification based on artificial intelligence techniques. More specifically, aspects of the present invention are configured to analyze an image of a target site of a patient undergoing a procedure and thereby identify features of the tissue in the image, including identification and classification of lesion formation at the target site, and can be performed using a platform.

[0048] As shown in FIG. 1A, the present invention may include an automated tissue evaluation and classification system 100. The system 100 may be directly incorporated into an ultrasound machine (i.e., provided as a local component to an ultrasound imaging machine) or may be cloud-based and provide a digital web-based application that can be accessed by an operator via an ultrasound machine or a computing device (i.e., a smartphone, tablet, personal computer, or the like).

[0049] FIG. 2 is a block diagram that more particularly illustrates an automated tissue evaluation and classification system 100. As shown, system 100 is embodied, for example, on a cloud-based service 102. The automated ultrasonic imaging analysis system 100 is configured to communicate and share data with an ultrasonic imaging machine 10. However, it should be noted that system 100 can also be configured to communicate and share data with a computing device 11 associated with a user. The computing device 11 may include a separate computer coupled to the ultrasonic imaging machine. Still further, in some embodiments, the computing device 11 may include a portable computing device such as a smartphone, tablet, laptop computer, or the like. For example, technological advancements have led to some ultrasonic probes that can be connected to personal and / or portable computing devices. Thus, in some embodiments, system 100 may be configured to communicate with an operator of an ultrasonic probe via an associated smartphone or tablet. In this context, the user may include a clinician such as a physician, physician assistant, nurse, or other healthcare provider who provides ultrasonic examinations and / or catheter ablation procedures to a patient. System 100 is configured to communicate and exchange data with the ultrasonic imaging machine 10 and / or the computing device 11, for example, via a network 104.

[0050] Network 104 may represent, for example, a private or non-private local area network (LAN), personal area network (PAN), storage area network (SAN), backbone network, global area network (GAN), wide area network (WAN), or any such collection of computer networks such as an intranet, extranet, or the Internet (i.e., a global system of interconnected networks, including, for example, the World Wide Web, on which various applications or services are launched). In an alternative embodiment, the communication path between the ultrasonic imaging machine 10 and the computing device 11 and / or between the machine 10, the computing device 11, and the system 100 may be a wired connection, in whole or in part, regardless of whether it is all or part of the connection.

[0051] Network 104 may be any network that conveys data. Non-limiting examples of suitable networks that may be used as Network 18 include Wi-Fi wireless data communication technology, the Internet, private networks, virtual private networks (VPNs), the public switched telephone network (PSTN), integrated services digital network (ISDN), digital subscriber line network (DSL), various second-generation (2G), third-generation (3G), fourth-generation (4G), fifth-generation (5G), and future-generation cellular-based data communication technologies, Bluetooth (R) wireless, near field communication (NFC), the most recently published version of the IEEE 802.11 transmission protocol standard, other networks capable of conveying data, and combinations thereof. In some embodiments, Network 104 is selected from the Internet, at least one wireless network, at least one cellular phone network, and combinations thereof. Thus, Network 104 may include any number of additional devices such as additional computers, routers, and switches for facilitating communication. In some embodiments, Network 104 is, or may include, a single network, and in other embodiments, Network 104 is, or may include, a collection of networks.

[0052] Note that in some embodiments, in contrast to providing a web-based application, System 100 may be directly built into, or directly connected to, an ultrasonic machine in a local configuration. For example, in some embodiments, System 100 operates in communication with a medical setting such as an examination or operating room, laboratory, or the like, and is configured to communicate directly with an instrument, including ultrasonic imaging machine 10, via either a wired or wireless connection.

[0053] FIG. 3 is a block diagram illustrating an automated tissue evaluation and classification system 100 that provides automated analysis of image data and subsequent identification, evaluation, and classification of tissue within the image data, including a machine learning system 108. System 100 is preferably implemented within a tangible computer system made to implement the various methods described herein.

[0054] As shown, system 100 may generally be accessed by a user to initiate the methods of the present invention and obtain results, for example, via interface 106. Interface 106 enables the user to connect to the platform provided via the system, provide sample ultrasound images, and receive automated analysis and feedback. System 100 may further include one or more databases with which machine learning system 108 communicates. In this example, reference database 112 includes stored reference data obtained from a plurality of training data sets, and sample database 114 includes stored sample data obtained as a result of evaluations performed on sample ultrasound images via system 100. System 100 further includes an image analysis module 110 to assist in processing input image data and correlating such data with reference image data of a trained data set of a neural network.

[0055] System 100 generally activates a neural network that has been trained using a plurality of training data sets containing eligible reference data. FIG. 4 is a block diagram, for example, illustrating the input of reference data (i.e., training data sets) into machine learning system 108. The machine learning techniques of the present invention and subsequent analysis of sample ultrasonic images based on such techniques utilize reference data. The reference data may include a plurality of training data sets 116 that are input into the machine learning system 108 of the present invention. For example, each training data set includes reference image data associated with a known tissue and further includes classification data associated with the known tissue, and the plurality of training data sets exclude digital histopathology data. More specifically, the present invention recognizes that the assessment and verification of lesion formation are generally difficult problems due to histological requirements. As an attempt to overcome such problems, the present invention proposes the use of phase contrast computed tomography (CT) imaging as an alternative approach to using digital histopathology data as an input for training data. The present invention recognizes that the direct visualization of cardiac ablation via phase contrast CT can be particularly useful for bridging the gap between conventional imaging modality data (i.e., CT, magnetic resonance imaging (MRI), and ultrasound (US)) and histology, particularly using lesion formation assessment.

[0056] Accordingly, in one aspect of the present invention, the reference image data of the training data set includes one or more images of a known tissue that are acquired and processed via a phase contrast computed tomography (CT) imaging system and at least one other imaging modality including, but not limited to, an ultrasonic imaging system, a transmission imaging system, a bright field or dark field imaging system, a fluorescence imaging system, a differential interference contrast imaging system, a hyperspectral imaging system, a Raman or surface enhanced Raman imaging system, and a magnetic resonance imaging (MRI) system.

[0057] In this embodiment, the reference image data is associated with a reference lesion formed within a known tissue, and the classification data is associated with the reference lesion. Thus, the classification data may include characteristics of the reference lesion, including at least one of the location of the reference lesion on the known tissue, the size of the reference lesion, the path of the reference lesion, the depth of the reference lesion, and the known success of the reference lesion in the treatment of heart-related diseases.

[0058] FIG. 5 shows a machine learning system according to an embodiment of the present disclosure. The machine learning system 108 accesses reference data from one or more training data sets 116 provided by any known source 200. The source 200 may include, for example, a laboratory-specific repository of reference data collected for the purpose of machine learning training. Additionally or alternatively, the source 200 may include publicly available registries and databases and / or subscription-based data sources. In this embodiment, the source 200 is clinical data, as will be further described in more detail herein with reference to FIG. 6.

[0059] In a preferred embodiment, a plurality of training data sets 116 are fed into the machine learning system 108. The machine learning system 108 may include, but is not limited to, neural networks, random forests, support vector machines, Bayesian classifiers, hidden Markov models, independent component analysis methods, and clustering methods.

[0060] For example, the machine learning system 108 is an autonomous machine learning system that associates classification data with reference image data. For example, the machine learning system may include a deep learning neural network including an input layer, a plurality of hidden layers, and an output layer. The autonomous machine learning system may represent the training data set using a plurality of features, each feature comprising a feature vector. For example, the autonomous machine learning system may include a convolutional neural network (CNN). In the described embodiment, the machine learning system 108 includes a neural network 118.

[0061] The machine learning system 108 discovers associations of data from a training dataset. In particular, the machine learning system 108 processes reference image data and classification data and associates them with each other, thereby establishing reference data in which image characteristics of known tissues, including damage, are associated with known characteristics of the tissue and the damage, i.e., the location of a reference damage on a known tissue, the size of the reference damage, the path of the reference damage, the depth of the reference damage, and the known success of the reference damage in the treatment of heart-related diseases. In particular, the machine learning system 108 can learn the relationship between the reference image data, the classification data, and damage formation via an ablation technique. The reference data is stored, for example, in the reference database 112 and is available during subsequent processing of the image data obtained during a catheter ablation procedure.

[0062] FIG. 6 is a flowchart illustrating an exemplary process for collecting, processing, and inputting reference data into the machine learning system of the present invention. The reference data is generally clinical data collected via animal trials 300.

[0063] Such reference data may include, for example, obtaining in-vivo image data 302 that includes one or more images of known tissues having one or more known damages formed therein. The image data may be obtained by any contemplated imaging modality described herein, including but not limited to an ultrasonic imaging system, a computed tomography (CT) imaging system, a transmission imaging system, a bright-field or dark-field imaging system, a fluorescence imaging system, a phase contrast imaging system, a differential interference contrast imaging system, a hyperspectral imaging system, a Raman or surface-enhanced Raman imaging system, and a magnetic resonance imaging (MRI) system. In a preferred embodiment, the reference image data obtained in-vivo is ultrasonic image data, and in some embodiments, three-dimensional (3D) ultrasonic image data.

[0064] In some embodiments, the additional reference data may include obtaining postmortem image data 304 for use as an anatomical reference, such image data may include computed tomography (CT) image data of known tissue (i.e., known tissue having one or more known lesions formed therein). For example, in some embodiments, it may be preferable to obtain postmortem video data for annotation of known lesions (e.g., using TTC staining), such postmortem obtained data can be associated (via mapping techniques) with in vivo obtained image data for a specific lesion or set of lesions at a given target site (i.e., the lesion location or equivalent).

[0065] Still further, in some embodiments, the additional reference data may include obtaining postmortem image data that includes phase contrast CT image data of a specific excised lesion 306 that is mapped to in vivo image data.

[0066] Using each set of reference data (302, 304, and 306), a certain property is ascertained that may include the location of the reference lesion on the known tissue, the size of the reference lesion, the path of the reference lesion, the depth of the reference lesion, and the known success of the reference lesion in the treatment of heart-related diseases (i.e., whether a given reference lesion is transmural, complete, and / or durable enough to effectively treat the underlying disease).

[0067] The machine learning system 108 discovers associations of data from the training data set 308. In particular, the machine learning system 108 processes the reference image data and the classification data and associates them with each other, thereby establishing reference data in which the image characteristics of known tissue, including lesions, are associated with the classification data.

[0068] FIG. 7 is a block diagram illustrating the reception of one or more images obtained during a procedure (such as an ablation procedure) performed on a patient's targeted tissue, the subsequent processing of the images via the automated tissue evaluation and classification system 100 of the present invention, and the output of the results to an operator (i.e., a clinician performing or assisting in the ablation procedure), such results including, inter alia, the identification of one or more lesion formations within the targeted tissue and the classification of such lesion formations to assist in the diagnosis and / or treatment of the associated disease.

[0069] As shown, system 100 is generally configured to receive image data that may include an image (obtained via medical imaging modality 10) of a target site of a patient undergoing a procedure. In this example, the image is, for example, an ultrasound image captured via an ultrasound imaging system with an ablation catheter. In response to receiving the ultrasound image, system 100 is configured to analyze the image using a neural network of machine learning system 108 based on an association of classification data and reference image data. Based on such analysis, the computing system is capable of evaluating and classifying one or more lesions formed within the tissue at the target site undergoing catheter ablation. More specifically, machine learning system 108 correlates sample ultrasound image data with reference data (i.e., reference image data and classification data). For example, machine learning system 108 generally receives two or more sets of clearly defined data, identifies the level of correlation to at least a certain range, and thereby is operable to associate the sets of data with each other based on the level of correlation, and may include custom, dedicated, known, and / or later developed statistical analysis code (or instruction set), hardware, and / or firmware.

[0070] In response to the step of detecting the association between the ultrasonic image data, the reference image data, and the classification data, system 100 can output the results to the clinician. The results may be in the form of a report 500 that provides details about the target site, i.e., the identification of lesion formation and its respective specific characteristics. For example, the results may include an annotated image of the target site that includes a visual rendering of the lesion formation and its respective characteristics, and may be provided to the clinician via the display 16 of the imaging modality 10.

[0071] Thus, by providing real-time and automated tissue evaluation and classification based on artificial intelligence techniques, the present invention addresses the limitations of current ablation systems, particularly the lack of a highly reliable and accurate means for assessing lesion formation. More specifically, the present invention reduces any need for specialized training when evaluating image data to assess and verify lesion formation. Furthermore, the present invention does not require conventional histology to provide lesion classification, which can present challenges related to complex alignment processes and issues related to deformation and / or tissue displacement that further impair the alignment process. Rather, the neural network of the present invention is trained using a plurality of training data sets that include eligible reference data excluding digital histopathology data. More specifically, the present invention utilizes phase contrast CT image data as an alternative approach to using digital histopathology data as an input for training data. Accordingly, the present invention provides a system that is highly effective for evaluating and classifying one or more lesions formed within intravascular and / or intracardiac tissue to assist in the diagnosis and / or treatment of heart-related diseases.

[0072] FIG. 8 shows an image of a sample tissue that has undergone histological analysis and a corresponding reference image (i.e., a phase contrast image) of the sample tissue. As described above in this specification, the reference image data relied upon for the input for the training data (i.e., phase contrast CT image data) is verified in that a link is established between the conventional histological data and the reference image data. In particular, each training data set is associated with an individual known tissue that has been collected as part of a clinical study or the like. The known tissue may have one or more known lesions therein. The known tissue (including the known lesions formed therein) may undergo a process for the collection of both conventional histological data and reference image data. In particular, the reference image data may be collected from a clinical tissue sample (i.e., an image of the clinical tissue sample may be obtained via a phase contrast CT imaging system and / or other imaging modalities described herein). Similarly, the clinical tissue sample may be prepared and analyzed via conventional histological techniques to collect conventional histological data. Consequently, a link is established between the reference image data and the histological data, thereby verifying what is shown in the reference image data (i.e., confirmation of what is shown in the image data based on the conventional histological techniques performed on the reference tissue sample, including the presence of any lesions and the characteristics of such lesions).

[0073] As shown in FIG. 8, the reference image (phase contrast CT image) can be verified based on histological analysis performed on a sample tissue, where ablation (ablation 1 and ablation 2) identified as part of the histological analysis is linked to the ablation shown in the phase contrast CT image. Thus, a link is established between histology / staining and the phase contrast CT image, thereby confirming the accuracy of the visual representation of the reference image (i.e., confirmation of what is shown in the image data based on conventional histological techniques performed on a reference tissue sample, including the presence of any damage and the characteristics of such damage). The link can be established by using specific calibration features such as a calibration rod that enables mapping of tissue properties to phase contrast CT data measured in an absolute manner. By using the calibration rod, absolute tissue properties are directly mapped, thereby improving the visualization of ablation.

[0074] As used in any embodiment of this specification, the term "module" may refer to software, firmware, and / or circuitry configured to perform any of the foregoing operations. The software may be embodied as a software package, code, instructions, instruction sets, and / or data recorded on a non-transitory computer-readable storage medium. The firmware may be embodied as code, instructions or instruction sets, and / or data hard-coded (e.g., non-volatile) within a memory device. "Circuitry," as used in any embodiment of this specification, may include, for example, wired circuitry, programmable circuitry such as a computer processor having one or more individual instruction processing cores, state machine circuitry, and / or firmware that stores instructions to be executed by the programmable circuitry, alone or in any combination. The module may be embodied as circuitry that, collectively or individually, forms part of a larger system, such as, for example, an integrated circuit (IC), a system on chip (SoC), a desktop computer, a laptop computer, a tablet computer, a server, a smartphone, etc.

[0075] Any of the operations described herein may also be implemented within a system that includes one or more storage media storing, individually or in combination, instructions that, when executed by one or more processors, perform the method. Here, the processor may include, for example, a server CPU, a mobile device CPU, and / or other programmable circuitry.

[0076] Also, the operations described herein are intended to be distributed across multiple physical devices, such as processing structures, at more than one different physical location. The memory medium can include any type of tangible medium, such as a hard disk, a floppy disk, an optical disk, a compact disc read-only memory (CD-ROM), a rewritable compact disc (CD-RW), and a magneto-optical disk, semiconductor devices such as read-only memory (ROM), random access memory (RAM) such as dynamic and static RAM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, solid state disk (SSD), magnetic or optical cards, or any other type of medium suitable for storing electronic instructions. Other embodiments may be implemented as software modules executed by a programmable control device. The memory medium may be non-transitory.

[0077] As described herein, various embodiments may be implemented using hardware elements, software elements, or any combination thereof. Examples of hardware elements may include processors, microprocessors, circuits, circuit elements (e.g., transistors, registers, capacitors, inductors, etc.), integrated circuits, application specific integrated circuits (ASICs), programmable logic devices (PLDs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), logic gates, registers, semiconductor devices, chips, microchips, chip sets, etc.

[0078] References throughout this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0079] The term "non-transitory" should be understood to exclude only transitory signals per se propagating from a claim scope, and not to waive rights to all standard computer-readable media that merely propagate transitory signals per se. In other words, the meaning of the terms "non-transitory computer-readable medium" and "non-transitory computer-readable storage medium" should be construed to exclude only those types of transitory computer-readable media that have been found to fall outside the scope of patentable subject matter under 35 U.S.C. § 101 in the In Re Nuijten case.

[0080] The terms and expressions used herein are used in an illustrative rather than a limiting sense, and it is not intended to exclude any equivalents of the features shown and described (or portions thereof) in the use of such terms and expressions, and it should be recognized that various modifications are contemplated as possible within the scope of the claims. Accordingly, the claims are intended to cover all such equivalents.

[0081] (Incorporation by reference) References and citations to patents, patent applications, patent publications, magazines, books, papers, web content, and other documents are made throughout this disclosure. All such documents are hereby incorporated by reference in their entirety for all purposes.

[0082] (Equivalents) In addition to what is shown and described herein, various modifications of the invention and many further embodiments thereof will become apparent to those skilled in the art from the entire contents of this book, including references to scientific and patent literature cited herein. The subject matter herein contains important information, exemplification, and guidance that can be adapted to the practice of the invention in its various embodiments and their equivalents.

Claims

**Claim 1** A method for training a neural network for evaluating and classifying tissues, the method comprising: providing a plurality of training data sets to a computing system, each training data set including reference image data associated with a known tissue and classification data associated with the known tissue, the plurality of training data sets excluding digital histopathology data; training a neural network from the plurality of training data sets such that the neural network is suitable for evaluating and classifying tissues based on an association between the classification data and the reference image data; and a method comprising the steps of. **Claim 2** The method of claim 1, wherein the reference image data includes one or more images of the known tissue obtained and processed via an imaging modality selected from the group consisting of an ultrasound imaging system, a computed tomography (CT) imaging system, a transmission imaging system, a brightfield or darkfield imaging system, a fluorescence imaging system, a phase contrast imaging system, a differential interference contrast imaging system, a hyperspectral imaging system, a Raman or surface enhanced Raman imaging system, and a magnetic resonance imaging (MRI) system. **Claim 3** The method of claim 2, wherein the MRI system performs at least one of a delayed gadolinium enhanced MRI and a diffusion weighted MRI sequence. **Claim 4** The method of claim 2, wherein the reference image data includes images of the known tissue obtained and processed via an ultrasound imaging system and a CT imaging system. **Claim 5** The method of claim 4, wherein the reference image data includes three-dimensional (3D) ultrasound image data of the known tissue and computed tomography (CT) image data. **Claim 6** The method of claim 5, wherein the CT image data includes phase contrast CT image data. **Claim 7** The method of claim 6, wherein the CT image data includes postmortem CT image data of the known tissue for anatomical reference and phase contrast CT image data of the known tissue. **Claim 8** The method of claim 5, wherein the 3D ultrasound image data is obtained from a catheter-based ultrasound imaging device configured to provide full circumference 3D image data. **Claim 9** The method according to claim 1, wherein the reference image data is associated with a reference lesion formed within the known tissue, and the classification data is associated with the reference lesion.

10. The method according to claim 9, wherein the classification data includes at least one of a location of the reference lesion on the known tissue, a size of the reference lesion, a path of the reference lesion, a depth of the reference lesion, and a known success of the reference lesion in the treatment of a heart-related disease, and includes characteristics of the reference lesion.

11. Obtaining one or more images of a sample tissue undergoing an ablation procedure; Processing the one or more images and inputting sample image data obtained in the processing step into the computing system; Correlating the sample image data with the reference lesion and known tissue data; Outputting the result of the correlation step The method according to claim 9, further comprising.

12. The method according to claim 11, wherein the result of the correlation step includes identification of one or more lesion formations in the sample tissue and classification of the identified one or more lesion formations.

13. The method according to claim 12, wherein the classification of the identified one or more lesion formations includes a location of the one or more lesion formations on the sample tissue, a size of the one or more lesion formations, a path of the one or more lesion formations, a depth of the one or more lesion formations, and at least one of a known success of the one or more lesion formations in the treatment of a heart-related disease, and includes identified characteristics.

14. The method according to claim 11, wherein the result of the correlation step further includes verification of one or more lesion formations.

15. The method according to claim 1, wherein the computing system comprises a machine learning system selected from the group consisting of a neural network, a random forest, a support vector machine, a Bayesian classifier, a hidden Markov model, an independent component analysis method, and a clustering method.

16. The method according to claim 1, wherein the computing system comprises an autonomous machine learning system that associates the classification data with the reference image data.

17. The method according to claim 16, wherein the machine learning system comprises a deep learning neural network including an input layer, a plurality of hidden layers, and an output layer.

18. The method according to claim 17, wherein the self-regulating machine learning system represents the training data set using a plurality of features, each feature comprising a feature vector.

19. The method according to claim 17, wherein the self-regulating machine learning system comprises a convolutional neural network.

20. The method according to claim 1, further comprising operating a machine learning system to learn a relationship between reference image data, classification data, and damage formation via an ablation technique.

21. A method for classifying tissue, the method comprising: providing, to a computer that activates a neural network, tissue data of a patient, wherein the neural network is trained to classify tissue, the neural network is trained using a plurality of training data sets, each training data set comprising reference image data associated with a known tissue and known classification data associated with the known tissue, and the plurality of training data sets excludes digital histopathology data; classifying the tissue data of the patient using the neural network based on an association between the classification data and the reference image data; and.

22. The method according to claim 21, wherein the reference image data comprises one or more images of the known tissue obtained and processed via an imaging modality selected from the group consisting of an ultrasonic imaging system, a computed tomography (CT) imaging system, a transmission imaging system, a bright field or dark field imaging system, a fluorescence imaging system, a phase contrast imaging system, a differential interference contrast imaging system, a hyperspectral imaging system, a Raman or surface enhanced Raman imaging system, and a magnetic resonance imaging (MRI) system.

23. The method according to claim 22, wherein the MRI system performs at least one of a delayed gadolinium enhanced MRI and a diffusion weighted MRI sequence.

24. The method according to claim 22, wherein the reference image data includes an image of the known tissue obtained and processed via an ultrasonic imaging system and a CT imaging system.

25. The method according to claim 24, wherein the reference image data includes three-dimensional (3D) ultrasonic image data of the known tissue and computed tomography (CT) image data.

26. The method according to claim 25, wherein the CT image data includes phase-contrast CT image data.

27. The method according to claim 26, wherein the CT image data includes postmortem CT image data of the known tissue for anatomical reference and phase-contrast CT image data of the known tissue.

28. The method according to claim 25, wherein the 3D ultrasonic image data is obtained from a catheter-based ultrasonic imaging device configured to provide full-circumference 3D image data.

29. The method according to claim 21, wherein the reference image data is associated with a reference lesion formed in the known tissue, and the classification data is associated with the reference lesion.

30. The method according to claim 29, wherein the classification data includes at least one of a location of the reference lesion on the known tissue, a size of the reference lesion, a path of the reference lesion, a depth of the reference lesion, and a known success of the reference lesion in the treatment of heart-related diseases, and includes characteristics of the reference lesion.

31. Obtaining one or more images of a sample tissue undergoing an ablation procedure; Processing the one or more images and inputting sample image data obtained in the processing step into a computing system; Correlating the sample image data with the reference lesion and known tissue data; Outputting the result of the correlating step The method according to claim 29, further comprising.

32. The method according to claim 31, wherein the result of the correlating step includes identification of one or more lesion formations in the sample tissue and classification of the identified one or more lesion formations.

33. The method according to claim 32, wherein the identified classification of one or more damage formations includes at least one of the location of the one or more damage formations on the sample tissue, the size of the one or more damage formations, the path of the one or more damage formations, the depth of the one or more damage formations, and the known success of the one or more damage formations in the treatment of heart-related diseases, and includes the identified characteristics.

34. The method according to claim 31, wherein the result of the correlation step further includes verification of one or more damage formations.

35. The method according to claim 21, wherein the computing system comprises a machine learning system selected from the group consisting of a neural network, a random forest, a support vector machine, a Bayesian classifier, a hidden Markov model, an independent component analysis method, and a clustering method.

36. The method according to claim 21, wherein the computing system comprises an autonomous machine learning system that associates the classification data with the reference image data.

37. The method according to claim 36, wherein the machine learning system comprises a deep learning neural network including an input layer, a plurality of hidden layers, and an output layer.

38. The method according to claim 37, wherein the autonomous machine learning system represents the training data set using a plurality of features, and each feature comprises a feature vector.

39. The method according to claim 37, wherein the autonomous machine learning system comprises a convolutional neural network.

40. The method according to claim 21, further comprising operating the machine learning system to learn the relationship between the reference image data, the classification data, and the damage formation via the ablation technique.

Images