Aiheal - an artificial intelligence healthcare system

Abstract

AIHeal system is a new solution creating explainable and causal AI infrastructure in a way that is accessible to physicians, solution in which they can trust, which open new routes to delivering better, faster, more reliable, secure, and more cost-effective medical care. The data from wearables and implants are transferred by mobile networks and collected by big data architecture in the innovative Electronic Health Record forming the "digital twin" of the patient. The ground-breaking innovative contribution of the proposed patent is to reach naturally interpretable AI through the synergistic AI concept. The main idea is to combine the benefits of the accuracy of deep-learning algorithms with visibility on the factors that are important to the algorithm's conclusion. Our approach is an attempt to follow the physicians thinking through the clinical decision procedure. Physicians first follow some rules and knowledge, then "call" experience (data) and make a decision.

Description

AIHeal - An Artificial Intelligence Healthcare System

[0001] Healthcare Equipment and Services: An Artificial Intelligence Healthcare System Architecture for Clinical Decision Support, Triage, Diagnosis, and Care DeliveryTechnical Problem

[0002] As longevity increases, healthcare systems face growing demand for their services (population ageing, patient expectations, lifestyle choices, and the never-ending cycle of innovation), rising the costs and requiring a workforce that is struggling to meet the needs of the patients. Artificial intelligence (AI) in healthcare can support significant improvements in Self-care / Prevention / Wellness, Triage and Diagnosis, Diagnostics, Clinical Decision Support, Care Delivery, Chronic Care Management, etc., and has the potential to transform healthcare organizations and healthcare services. The number of data sources in healthcare services has grown rapidly as a result of widespread use of mobile, wearable sensors technologies and implants, which has flooded healthcare area with a huge amount of data and changed our understanding of human biology and of how medicines work, enabling personalized and real-time treatment for all. Therefore, the high volume of diversified medical data analysis based on traditional methods becomes a non-promising solution.

[0003] Could we build an AI solution which can create explainable and causal infrastructure for Clinical Decision Support, Triage, Diagnosis and Care Delivery in a way that is accessible to physicians, solution in which they can trust, which open new routes to delivering better, faster, more reliable, secure, and more cost-effective medical care?

[0004] Financial sustainability is a core challenge for European healthcare systems. In 2018, healthcare expenditure ranged between Italy 8.8% of the GDP, Spain 8.9%, UK 9.8%, France 11.2%, Germany 11.2% and are expected to continue rising. Healthcare spending as a share of GDP has been growing since 1990, outpacing average wage growth and the growth of GDP itself. Without major structural and transformational change, healthcare systems will struggle to find the funding needed to address growing demand. Also, projected staff shortages of 9.9 million physicians and inevitable skill gaps will be limiting healthcare systems’ ability to satisfy this increasing demands.

[0005] How could these challenges be solved? Problems and possible AI solutions are recognized by the World Health Organization, EU Commission, and many governments and private companies all over the world.

[0006] Companies around the world implemented numerous products where AI is having an impact in healthcare. Some of the prominent applications available today grouped around use cases in healthcare are [Transforming healthcare with AI: The impact on the workforce andorganisations,McKinsey&Company, March2020.]:Cronic Care Management: Sensely –virtual nurse, Karantis360–automated personal monitoring and alerting system, AICure–treatment adherence, Pill Pack–personalised presorted meds for repeat prescriptions;Care Delivery: Moxi–nurse assistant robot, Amelia–virtual health assistant, Bionic Pancreas–insulin / glucagon administration for Type-1 diabetes patients, EarlySense–contact-free patient monitoring;Clinical Decision Support: IBM Watson For Oncology, DeepMind–prediction of acute kidney injury;Improving population-health management: Mount Sinai Health Systems–risk prediction for emergency admissions, Sheba Medical Cancer–prediction of complications;Improving operations: Qventus–optimisation of operating room flow;Self-care / Prevention / Wellness: AliveCor CardiaMobile–personal ECG, Activity and sleep trackers;Triage and Diagnosis: Symptom checkers: Babylon, Mediktor, Ping An Good Doctor, Ada Health, K Health;Diagnostics: Sight Diagnostics–point of care blood testing, Arterys–medical image analysis, Idx –detection of diabetic retinopathy, Detection of eye diseases: DeepMind, UCL and Moorfields.

[0007] Even though these applications have patients who use it, they are not widely accepted from patients and especially from physicians. What are common disadvantages for these applications? We would stress some of the problems which are important from the technological and users’ point of view.Applications are designed to cover only specific use cases.Applications focusing on areas that simplify routine processes are better accepted because they are more understandable for patients and physicians, and obviously free up physician time.Applications focusing on use cases such as Clinical Decision Support, Triage and Diagnosis, Diagnostics and other more complex segments of healthcare are less straightforward and are not understandable enough for physicians. Physicians cast doubt on the readiness of the technology and have no confidence to apply them in practice. Any solution perceived as a black box or if there can be a fundamental mismatch between the way machines learn and the way clinicians work may face significant barriers to adoption and is therefore something developers need to preempt.The major challenge is data. Data can be incomplete or of poor quality and consequently AI applications cannot be better than the data used. Data are often poorly suited to AI due to quality issues, inconsistent formats, or the challenges of linking data and obtaining the necessary consent to use in different use cases.

[0008] The objective of this patent is to propose the AI healthcare system infrastructure which could solve above mentioned disadvantages.

[0009] The ground-breaking innovative contribution of the proposed patent is to reach naturally interpretable AI healthcare through the synergistic AI concept. Any solution perceived as a black box or if there can be a fundamental mismatch between the way machines learn and the way clinicians work may face significant barriers to adoption and is therefore something our patent needs to preempt.

[0010] The main idea is to reach explainable and causal AI by combining the benefits of the accuracy of deep-learning algorithms with visibility on the factors that are important to the algorithm’s conclusion.

[0011] Our approach is an attempt to follow the physicians thinking through the Clinical Decision Procedure. Physicians first follow some rules and knowledge, then "call" experience (data) and make a decision. The architecture of the proposed AIHeal system is shown in.

[0012] Electronic Health Record (EHR). The major challenge is data. How is it possible to manage such a quantity of data and reach the necessary quality and interchangeability. One of the most serious obstacles for AI in healthcare is that while enough data are available from pharmacy companies and healthcare institutions, they are not connected or interoperable. The proposed EHRs are medical records for patients with any information relating to the past, present or future physical / mental health or condition of an individual and connect longitudinal data and new types of data from wearables, mobiles, NL data, sensors and genomics and other omics data. Genomics data are becoming more accessible as the costs of sequencing and bioinformatics techniques have significantly reduced. Healthcare data are typical big data which combine volume (high amounts of data), velocity (data is generated at a rapid pace), variety (data comes under different formats), veracity (data originates from trustable sources), and variability (variations in the data flow rates). Improvements in data, processing power and algorithms are rapidly changing what is feasible. The scope and quality of healthcare data produced and the potential to link datasets are opening new possibilities. Both the quality and consistency of data are improving as more data are machine generated. Focus is on consistent interconnected data infrastructure capable for data exchange and semantic interoperability of different medical fields and use cases. The proposed EHRs, unlike the traditional ones, may include structured (demographics, medications, diagnoses, laboratory tests, doctor's note, radiology documents, clinical information), semistructured (from various medical devices, implants, wearables...) and unstructured data (text, voice, biomedical images...). This data integration allows us to develop the “digital twin” of the patient and makes a prerequisite to design modular and flexible AI infrastructure capable to cover different use cases.

[0013] Extensible big data architecture. The process of extracting from big data can be broken down into five stages. These five stages form the two main sub-processes: data management and data analytics. Data management involves processes and supporting technologies to acquire and store data and to prepare and retrieve it for analysis: Acquisition and Recording; Extraction, Cleaning and Annotation; and Integration, Aggregation and Representation. Analytics, on the other hand, refers to techniques used to analyze and acquire intelligence from big data. The proposed extensible big data architecture consists of Data Acquisition, Data Semantic and AI Data Analytics modules. The Data Acquisition module is responsible for collecting data from various sources. This component is combination of HDFS, NoSQL such as MongoDB and SQL database.

[0014] The Data Semantic module is mapping heterogeneous databases into common structure and semantics. The Web Ontology Language with XML syntax can be used as standard interchange format regarding ontology. The AI Data Analytics module is responsible for Extraction, Cleaning and Annotation, Integration, Aggregation and Representation, and is based on AI mechanisms proposed through the Main AI module proposal. The AI Data Analytics module compares every processed data to predefined user's threshold. If the value of a particular data exceeds alarming threshold value, it will be stored while an emergency alert is generated to the Main AI module. The Main AI module decides if it is value of data for final emergency alert, or regular storage in EHRs. The proposed extensible big data architecture is capable of batch and stream processing and is based on available frameworks. For batch processing mode Hadoop allows distributed big data on a cluster of machines but may not be appropriate for stream processing. When stream processing is required, Apache Spark as an open-source unified engine for distributed data processing that includes higher-level libraries for supporting SQL queries (Spark SQL) that restore data from many sources and manipulate them using SQL, streaming data (Spark Streaming), iterative machine learning algorithms through library mechanism (MLlib), provides efficient algorithms with high speed, structured data analysis using Hive, and graph processing based on GraphX. Further information extraction and processing from EHRs requires specialized toolsets for Natural Language Processing, Image Analytics (Visualization Toolkit, GIMIAS, Elastix, MITK), Machine Learning (Tensorflow, Keras, Theano, Torch, Caffe...) and analytics for "Omics" data (SparkSeq, SAMQA, Distmap, Hydra,...).

[0015] Main AI module. Until the last few years, most of the techniques for analyzing rich EHRs data were based on traditional machine learning and statistical techniques such as logistic regression, support vector machines (SVM), and random forests. Recently, deep learning techniques have achieved great success in many domains through deep hierarchical feature construction and capturing long-range dependencies in data in an effective manner. Machine learning approaches can be broadly divided into two major categories: supervised and unsupervised learning. Supervised learning techniques involve inferring a mapping function y = f(x) from inputs x to outputs y. In contrast, the goal of unsupervised machine learning techniques is to learn interesting properties about the distribution of x itself. The representation of inputs is a fundamental issue spanning all types of machine learning frameworks. For each data point, sets of attributes known as features are extracted to be used as input to machine learning techniques. In traditional machine learning, these features are hand-crafted based on domain knowledge. One of the core principles of deep learning is automatic data-oriented feature extraction. The vast majority of deep learning algorithms and architectures are built upon the framework of the artificial neural network (ANN). ANNs are composed of a number of interconnected nodes (neurons), arranged in layers. The most common deep learning architectures for analyzing EHR data differ in terms of their node types and the connection structure (e.g. fully connected versus locally connected). Several open source tools exist for working with deep learning algorithms in a variety of programming languages, including TensorFlow, Theano, Keras, Torch, PyTorch, Caffe, CNTK.

[0016] While deep learning techniques produce state-of-the-art performance on a variety of tasks, one of its main criticisms is that the resulting models are difficult to naturally interpret. In this regard, many deep learning frameworks are often referred to as “black boxes”, where only the input and output predictions convey meaning to a human observer. Since correct clinical decision-making can be the difference between life and death, many practitioners must be able to understand and trust the predictions and recommendations made by deep learning systems. There are authors attempts and approaches [Shickel, B., Tighe, P. J.,Bihorac, A., & Rashidi, P. (2018). Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis. IEEE journal ofbiomedical and health informatics, 22(5), 2018., 1589–1604. https: / doi.org / 10.1109 / JBHI.2017.2767063.] to make clinical deep learning more interpretable include: Maximum Activation: A popular tactic in the image processing community is to examine the types of inputs that result in the maximum activation of each of a model’s hidden units. This represents an attempt to examine what exactly the model has learned and can be used to assign importance to the raw input features; Constraints: Others have imposed training constraints specifically aimed at increasing the interpretability of deep models. For example, Med2Vec framework [E. Choi, M. T. Bahadori, E. Searles, C. Coffey, and J. Sun, “Multi-layer Representation Learning for Medical Concepts”arXiv, pp. 1–20, 2016.] for learning concept and patient visit representations uses a non-negativity constraint enforced upon the learned code representations; Qualitative Clustering: In the case of EHR concept representation and phenotype studies, some studies point to a more indirect notion of interpretability by examining natural clusters of the resulting vectorized representations. This is most commonly performed using a visualization technique known as t-Distributed Stochastic Neighbor Embedding (t-SNE), a method for plotting pairwise similarities between high-dimensional data points in two dimensions [L. Van DerMaatenand G. Hinton, “Visualizing Data using t-SNE” Journal of Machine Learning Research, vol. 9, pp. 2579–2605, 2008.]; Mimic Learning [Z. Che, S.Purushotham, R. Khemani, and Y. Liu, “Interpretable deep models foricuoutcome prediction” in AMIA Annual Symposium Proceedings, vol. 2016. American Medical Informatics Association,2016, p.371.]: a deep neural network is first trained on raw patient data with associated class labels, which produces a vector of class probabilities for each sample. They train an additional gradient boosting tree (GBT) model on the raw patient data, but instead use the deep network’s probability prediction as the target label. Since GBTs are interpretable linear models, they are able to assign feature importance to the raw input features while harnessing the power of deep networks. The approaches given and known from the open references cannot be recognized as fully naturally interpretable and are not widely accepted.

[0017] Our approach to reach naturally interpretable AI is synergistic AI concept. The main idea is to reach explainable and causal AI by combining the benefits of the accuracy of deep-learning algorithms with visibility on the factors that are important to the algorithm’s conclusion, in a way that is accessible to physicians and other practitioners, as in rules-based systems, enabled by richer data and improved computing. Therefore, we propose AI in healthcare as a synergy of two concepts, where encoding clinical guidelines and / or existing clinical protocols provides a starting point, which then can be augmented by models that learn from data and demonstrate the distinctive properties of AI, the ability to perform tasks in complex environments without constant user guidance and improving performance by learning from experience. The proposed concept is realized through Main AI modulewith two threads: supervised and unsupervised thread. Supervised thread is based on interpretable rules-based systems upgraded with visualization techniques. Unsupervised thread is deep learning ANN with two layers. The first ANN layer, composed of more sub layers, analyses data to provide a mapping of types and features of disease, feeds Interpretable Rules-Based thread (IRB) allowing professionals to reach a clinical decision "independently" by supervised IRB thread. The second ANN layer, composed of more sub layers, analyses this map to present clinicians with a potential diagnosis and recommendation.

[0018] Actually, our approach is an attempt to follow the physicians thinking through the clinical decision procedure. Physicians first follow some rules and knowledge, then "call" experience (data) and make a decision. If we offer rules and knowledge in human understandable way physicians will believe and can check with their experience, but the deep learning ANN will extend that experience improving accuracy.

[0019] The system with all relevant data can enable clinical pathways and protocols to be redesigned towards intervening proactively in the highest risk cases, even if they are asymptomatic. This could force the working patterns of practitioners from reactive care to proactive care. Appropriate risk stratification of these patients could help us identify patients with the highest risk and to react in proper time. It could give us a chance to organize health programs and develop predictive modeling features. AI-based models could help reduce the significant numbers of timely unrecognized patients and avoid further complications in treatment and subsequent lengthy hospital stays.

[0020] This would imply systems to aggregate larger population datasets and aligning resources while continuing to improve AI models.

[0021] The AIHeal system, as software infrastructure that is also a medical device, should be designed, implemented, tested, and documented using generally recognized quality assurance methods for software development used in the medical domain [MahadevaiahG,RvP, Bermejo I, Jaffray D, Dekker A, Wee L. Artificial intelligence-based clinical decision support in modern medical physics: Selection, acceptance, commissioning, and quality assurance. Med Phys. 2020 Jun;47(5): e228-e235.doi: 10.1002 / mp.13562. PMID: 32418341; PMCID: PMC7318221. Jun 2020.]. The methodology is based on reference model for Clinical Decision Support Systems (CDSS) [Zikos, D., DeLellis, N. CDSS-RM: a clinical decision support system reference model. BMC Med Res Methodology 18, 137. https: / doi.org / 10.1186 / s12874-018-0587-6. 2018.]. The six elements / considerations of the reference model are: (1) Do CDSS mimic the cognitive process of clinical decision makers? (2) Do CDSS provide recommendations with longitudinal insight? (3) Is the model performance contextually realistic? (4) Is the ‘Historical Decision’ bias taken into consideration in CDSS design? (5) Do CDSS integrate established clinical standards and protocols? (6) Do CDSS utilize unstructured data?

[0022] The EU has published a wealth of reports on the topic since 2018. Following the European Council’s call to put forward a European approach to AI, the European Commission (EC) published its Communication on AI, and Digital Europe Programme for the period 2021-2027 [Digital EuropeProgramme: a proposed €9.2 billion of funding for 2021-2027, European Commission, June 26, 2019.]. In addition, there are a series of initiatives underway in areas such as health policy, digital transformation of health and care, R&D and innovation on health and ageing, plans on AI, data, and industrial policy implementation [“Europe and AI in healthcare – the EU’s strategy for AI”, 2019.]. Not just strategies, but investment in AI in healthcare is increasing across the world, particularly in countries as diverse as Finland, Germany, the UK, Israel, China, and the United States. The private sector also has a significant role, with venture capital funding for the top 50 firms in healthcare related AI reaching, since 2010, $8.5 billion [http: / pitchbook.com / research-process].

[0023] The project affects healthcare organizations and healthcare services, physicians and other practitioners and patients, especially senior citizens, and citizens with chronic conditions. The average life over the past century rises from less than 50 years to 78.9 years for the USA and to 80.9 years for EU. By 2050, 1 in 6 people will be over the age of 65, in Europe and North America, this will be 1 in 4. This demographic shift, combined with rapid urbanization, modernization, globalization and accompanying changes in risk factors and lifestyles, means chronic conditions will be more common, and an increasingly comorbid population’s demand for healthcare will increase.

[0024] In this context, financial sustainability is a core challenge for European healthcare systems. Healthcare spending as a share of GDP has been growing since 1990, outpacing average wage growth and the growth of GDP itself.

[0025] Without major structural and transformational change, healthcare systems will struggle to find the funding needed to address growing demand, whilst maintaining or improving standards of care, access, and patient experience. Also, according to the World Health Organization, projected staff shortages of 9.9 million physicians and inevitable skill gaps will be limiting healthcare systems’ ability to satisfy this increasing demands.

[0026] The proposed AIHeal system solves the crucial disadvantages of currently commercially available systems and has the potential to transform the way care is delivered and to help meet the challenges.

[0027] Accordingly, the target group and final beneficiaries are:Healthcare organizations and healthcare services, Health Insurance Agencies, which have to find the funding needed to address growing demand, staff shortages and skill gaps, whilst maintaining or improving standards of care, access, and patient experience.The final beneficiaries are also patients as healthcare customers. It’s time to reduce paternalism in healthcare. Other industries involve the customer as much as possible. Why should medicine be different?Software industry because the proposed healthcare infrastructure offers the opportunity for various medical use cases upgrades.

Claims

The architecture of the proposed AIHeal system [Fig.1]. The proposed Electronic Health Records (EHR) are medical records for patients with any information relating to the past, present or future physical / mental health or condition of an individual and connect longitudinal data and new types of data from wearables, mobiles, NL data, sensors and genomics and other omics data. Genomics data are becoming more accessible as the costs of sequencing and bioinformatics techniques have significantly reduced. Both the quality and consistency of data are improving as more data are machine generated. Focus is on consistent interconnected data infrastructure capable for data exchange and semantic interoperability of different medical fields and use cases. The proposed EHRs, unlike the traditional ones, may include structured (demographics, medications, diagnoses, laboratory tests, doctor's note, radiology documents, clinical information), semistructured (from various medical devices, implants, wearables...) and unstructured data (text, voice, biomedical images...). This data integration allows us to develop the “digital twin” of the patient and makes a prerequisite to design modular and flexible AI infrastructure capable to cover different use cases. The EHRs are connected with Big Data and AI Main modules. The process of extracting from big data can be broken down into five stages. These five stages form the two main sub-processes: data management and data analytics. Data management involves processes and supporting technologies to acquire and store data and to prepare and retrieve it for analysis: Acquisition and Recording; Extraction, Cleaning and Annotation; and Integration, Aggregation and Representation. Analytics, on the other hand, refers to techniques used to analyze and acquire intelligence from big data. The proposed extensible big data architecture consists of Data Acquisition, Data Semantic and AI Data Analytics modules [Fig.1]. The Data Acquisition module is responsible for collecting data from various sources. The Data Semantic module is mapping heterogeneous databases into common structure and semantics. The Web Ontology Language with XML syntax can be used as standard interchange format regarding ontology. The AI Data Analytics module is responsible for Extraction, Cleaning and Annotation, Integration, Aggregation and Representation, and is based on AI mechanisms proposed through the Main AI module proposal. The AI Data Analytics module compares every processed data to predefined user's threshold. If the value of a particular data exceeds alarming threshold value, it will be stored while an emergency alert is generated to the Main AI module. The Main AI module decides if it is value of data for final emergency alert, or regular storage in EHRs. The proposed extensible big data architecture is capable of batch and stream processing. The AI Main module [Fig.1] is responsible to reach naturally interpretable AI as synergistic AI concept. The main idea is to reach explainable and causal AI by combining the benefits of the accuracy of deep-learning algorithms with visibility on the factors that are important to the algorithm’s conclusion, in a way that is accessible to physicians and other practitioners, as in rules-based systems, enabled by richer data and improved computing. Therefore, we propose AI in healthcare as a synergy of two concepts, where encoding clinical guidelines and / or existing clinical protocols provides a starting point, which then can be augmented by models that learn from data and demonstrate the distinctive properties of AI, the ability to perform tasks in complex environments without constant user guidance and improving performance by learning from experience. The proposed concept is realized through Main AI module [Fig.1] with two threads: supervised and unsupervised thread. Supervised thread is based on interpretable rules-based systems upgraded with visualization techniques. Unsupervised thread is deep learning ANN with two layers. The first ANN layer, composed of more sub layers, analyses data to provide a mapping of types and features of disease, feeds Interpretable Rules-Based thread (IRB) allowing professionals to reach a clinical decision "independently" by supervised IRB thread. The second ANN layer, composed of more sub layers, analyses this map to present clinicians with a potential diagnosis and recommendation. Actually, our approach is an attempt to follow the physicians thinking through the clinical decision procedure. Physicians first follow some rules and knowledge, then "call" experience (data) and make a decision. If we offer rules and knowledge in human understandable way physicians will believe and can check with their experience, but the deep learning ANN will extend that experience improving accuracy.

Images