System and method for AI-assisted cardiac echocardiography

Abstract

A method for processing sparse data sources, comprising: (a) retrieving data from the sparse data source to form a basic data set, wherein the sparse data source includes a plurality of patient records containing patient mortality data, and each of the patient records includes at least one unpopulated data field corresponding to a medical measurement; (b) dividing the basic data set into two parts: a first part including a training data set at a predefined ratio of the basic data set, and a second part including a validation data set at a predefined ratio of the basic data set; (c) analyzing the training data set to co-model variable relationships using a non-linear function approximation algorithm repeatedly applied to the records of the training data set for the purpose of obtaining a trained model and a measurement prediction protocol for populating the unpopulated data fields within the training data set; (d) calculating predicted values of measurement data for the unpopulated data fields using the measurement prediction protocol; (e) populating the predicted values into the records within the training data set; (f) analyzing the training data set based on a predefined disease state in the known patient records of the basic data set to form a phenotype model associating the patient's phenotype data with the probability of the disease state in the patient records of the training data set; (g) populating the predicted values into the records within the validation data set; (h) validating the validation data set using the phenotype model, and determining a validation error including the probability of correctly predicting the patient's phenotype associated with the probability of the disease state in the records of the validation data set; (i) repeating steps (c) through (h) to minimize the validation error, and calculating and predicting a high-probability disease state phenotype for each patient record in the basic data set. Records within the validation data set include the patient's phenotype data associated with the probability of the disease state.

Description

Technical Field

[0001] The present invention relates to artificial intelligence, and more particularly to systems and methods for integrating artificial intelligence into the analysis of medical data and records.

[0002] The present invention has been developed primarily for use in systems and methods for AI-assisted echocardiography for identifying the phenotype of severe aortic stenosis, and will be described hereinafter with reference to this application. However, it will be understood that the present invention is not limited to this particular field of use.

Background Art

[0003] Discussing background art throughout this specification should not be taken as an admission that such background art is prior art, or that such background art is widely known in the relevant field in Australia or the world, or forms part of common general knowledge.

[0004] All references, including patents or patent applications cited in this specification, are incorporated herein by reference. No document is admitted to be prior art. The discussion of references states what their authors have asserted, and the applicant reserves the right to challenge the accuracy and validity of the cited documents. Many prior art documents are referred to in this specification, but it should be clearly understood that this reference does not admit that any of these documents forms part of the common general knowledge in the relevant technical field in Australia or any other country.

[0005] A phenotype refers to the observable characteristics of an organism as a multifactorial result of genetic traits and environmental influences. The phenotypes of an organism include morphological, biochemical, physiological, and behavioral characteristics. Thus, a phenotype is the total characteristics exhibited by an organism resulting from the expression of its genes, the influence of environmental factors experienced by the organism, and random mutations of the genes.

[0006] Echocardiography (or simply "echo") is a subspecialty of cardiology that uses a special ultrasound device to take diagnostic images of the heart. It is particularly valuable as a first-line diagnostic tool because it can non-invasively and cost-effectively evaluate the internal structure and function of the heart.

[0007] In a comprehensive echocardiogram, many features of the heart can be measured, and a total of about 150 unique variables can be measured, but it is rare to measure all features due to time constraints.

[0008] Echocardiogram data from a wide range of clinics across Australia are stored in the National Echocardiogram Database Australia (NEDA). In its current form, NEDA consists of measurements taken from echocardiography procedures and report text, which is the analysis from those procedures for many patients. Currently, this database has echocardiogram data on over one million patients.

[0009] NEDA includes echocardiogram measurements and report data from participating real-world clinical echocardiography laboratories. The measurements are performed as part of a standard clinical echocardiogram examination and are carried out for clinical indication under a standard echocardiogram imaging protocol. Although there are some differences among the testing facilities, image collection and measurement are standardized. Guidelines for performing a comprehensive standard transthoracic echocardiogram have been published (see https: / / www.asecho.org / wp content / uploads / 2018 / 10 / Guidelines for Performing a Comprehensive Transthoracic Echocardiography Examination in Adults.pdf). In the latest echocardiogram examinations, images are standardly saved in DICOM (Digital Imaging and Communications in Medicine) format, and the measured values are saved in SR (Structured Reporting) format together with the images. When the echocardiogram examination is completed, the images and SR files are transferred to the cardiology PACS (Picture Archive and Communications System). From there, the images can be viewed together with the accompanying measured values to create an echocardiogram report. In the standard workflow, a preliminary report by the echocardiogram technician who performed the examination and a final report by a cardiologist are created. The final report is sent to the medical record and referring physicians as the definitive interpretation of the echocardiogram examination. The final echocardiogram examination report usually includes the measured values transferred in the SR file, a text interpretation of the echocardiogram, and a conclusion section. See the following for recommendations regarding a standardized transthoracic echocardiogram examination report. https: / / www.asecho.org / wpcontent / uploads / 2013 / 05 / Standardized_Echo_Report_Rev1.pdf

[0010] For the purposes of this application, "echocardiogram report data" is defined as including all measurement and report information included in the final echocardiogram report. NEDA developed a unique system for collecting all retrospective echocardiogram examination report data from participating echocardiogram facilities, enabling all measured echocardiogram variables and corresponding interpretive text information to be collated in a single database containing records unique to each echocardiogram. Each database is remotely transferred to the master NEDA database by a "vendor-independent" automated data extraction process, and all measurements of each echocardiogram performed are converted to a standardized NEDA data format (compliant with the NEDA data dictionary). Each individual contributing to NEDA is assigned a unique identifier along with their demographic profile (date of birth and gender) and all data recorded on the echocardiogram. Using this method, NEDA has collected over 1,000,000 echocardiograms and linked this data to the Australian National Deaths Index (NDI) through the Data Linkage Unit of the Australian Institute of Health and Welfare (AIHW) in Canberra, Australia. NEDA has obtained ethical approval for mortality data linkage from public and private echocardiogram facilities across Australia and the HREC of the AIHW. NEDA is registered with the Australasian Clinical Trials Registry (http: / / www.anzctr.org.au / ACTRN12617001387314.aspx). The storage and analysis of all data are managed and conducted on an anonymized basis to protect the anonymity of participants. This unique resource provides a vast repository of echocardiogram report data that can be used for training artificial intelligence (AI) systems. The NEDA data used for training AI systems is inherently incomplete. Over 150 measurements are possible with a transthoracic echocardiogram, but it is rarely necessary to perform all measurements on an individual patient, and typically only about one-third of the measurements are taken.Since the individual measurements performed vary depending on the clinical indications for echocardiography and the findings revealed during the performance of echocardiography, there is no minimal dataset that exists for all echocardiograms. Therefore, although NEDA contains a large amount of echocardiogram report data, there may be cases where specific measurements are sparsely performed. Figure 1 shows some examples of measurements and missing measurements that exist in echocardiograms of a randomly selected small number of patients. Each row in the table is a typical example of sparse echo data that records the available echo measurements for one patient.

[0011] The NEDA database contains the measurements necessary to diagnose most heart diseases that can be identified by echocardiography. The report data includes additional information obtained by visual inspection of echocardiogram images. Since each heart disease identified by echocardiography has typical features ("phenotypes") included in the measurements and text information, NEDA contains a wealth of disease phenotypes. However, each disease phenotype is not labeled (or identified) within the NEDA database. Therefore, NEDA does not contain patient phenotype information for identifying traits or groups of traits that patients with common diseases have.

[0012] The workflow of a typical prior art echocardiography examination process is depicted in Figure 2. Briefly, a typical workflow consists of the following steps as described below.

[0013] Images are acquired by an echocardiographer using a special ultrasound examination device.

[0014] For example, the diameter of the left ventricle is commonly measured. (Note: Typically, many of the measurements are performed during the image acquisition process with the patient present. However, there are also some that may be measured from images acquired during a procedure after the patient has left.) In this workflow, the echocardiographer can fully and independently control whether sufficient images of the patient have been acquired or whether additional images are needed for a meaningful diagnosis of the patient's actual or suspected condition.

[0015] The image and measurement values are manually interpreted by a sonographer.

[0016] A preliminary report detailing the interpretation of the examination is created by a sonographer.

[0017] A cardiologist reads the preliminary report and examines the manual analysis result 109 and the measurement values.

[0018] The cardiologist creates the findings and conclusions obtained from the examination as the final report 111.

[0019] Importantly, because the series of images and measurements necessary to ensure that either a sonographer or cardiologist has sufficient data to diagnose the patient's condition is comprehensive, an echocardiogram is very time-consuming for a sonographer, and errors are likely to occur, such as a less experienced or inexperienced sonographer overlooking specific data during the scan. In practice, however, not all possible measurements are taken, and only a subset related to suspected conditions or diseases is recorded by the sonographer. Knowing which measurement values are important for subsequent analysis and diagnosis depends on the skill and experience of the sonographer.

[0020] Therefore, there is a need for a method to obtain a complete echocardiogram dataset for diagnosing a patient's abnormal or diseased condition within the time and cost constraints of a physical echocardiogram procedure. There is also a need for a method and system for determining meaningful data to provide a complete echocardiogram record of past echocardiogram patients.

[0021] Furthermore, there is a need to identify patients at increased risk due to specific diseases such as aortic valve stenosis. Preferably, the likelihood of a specific disease state in the patient should be identified during the echocardiogram examination. This would enable the sonographer to be advised of additional measurement values to record during the examination to assist the cardiologist in confirming or ruling out the identified disease state.

[0022] Severe aortic stenosis (AS) is the most common primary valvular disease leading to intervention. 1 Without timely intervention, AS leads to progressive myocardial hypertrophy and dysfunction 2 , left atrial dilation and pulmonary hypertension. 1,3 Echocardiography is extremely important in identifying AS and its associated adaptive responses. 1,3 However, the diagnosis of AS is highly operator-dependent 4 and requires expert interpretation. Furthermore, small errors in the measurement of left ventricular outflow tract dimensions and velocity-time integrals are multiplied when calculating the aortic valve area using the continuity equation, which is a pitfall pointed out in clinical guidelines 1 and results in a poor diagnostic outcome.

[0023] Severe AS 3 The transaortic gradient, a reliable indicator of severe AS, is affected by reduced left ventricular systolic function, which greatly influences left ventricular systolic function. A low transaortic gradient in the presence of reduced left ventricular systolic function affects the interpretation of AS severity. Due to such technical challenges, there is a need for improvement in automated and objective echocardiographic interpretation systems. 6 Artificial intelligence (AI) is a disruptive technology with great potential to improve the quality and consistency of echocardiography. However, interpreting the comprehensive set of measurements performed during echocardiography based on AI has not been explored. Therefore, we have initiated the development of a robust AI-based system using echocardiographic measurements to interpret the pathophysiology of AS and the phenotypes of patients presenting with AS for the first time. Specifically, the aim is to develop a highly reliable AI system that supports the clinical diagnosis of severe AS without requiring measurements of left ventricular outflow tract dimensions and velocities that may lack reliability, and to introduce a robust quality feedback system that can be applied daily in the clinical setting.

Summary of the Invention

Problems to be Solved by the Invention

[0024] The object of the present invention is to overcome or improve at least one of the drawbacks of the prior art or to provide a useful alternative means.

Means for Solving the Problems

[0025] The systems and methods disclosed herein provide an artificial intelligence (AI) system for predicting missing measurements from cardiac echo data recordings and providing disease risk assessments from incomplete data recordings using AI-input data.

[0026] One embodiment provides a computer program product for executing the methods described herein.

[0027] One embodiment provides a non-transitory storage medium storing computer-executable program code that causes a processor to execute the methods according to the present disclosure when executed by the processor.

[0028] One embodiment provides a system configured to execute the methods described herein.

[0029] According to a first aspect of the present invention, a method for processing sparse data sources is provided. This method includes (a) retrieving data from the sparse data source to form a basic data set, the sparse data source including a plurality of patient records including patient mortality data, each of the patient records including at least one unpopulated data field corresponding to a medical measurement; (b) dividing the basic data set into the following two parts, namely a first part including a predefined percentage X% of the basic data set as a training data set; a second part including a predefined percentage (100% - X%) of the basic data set as a validation data set; and (c) To obtain a trained model and a measurement prediction protocol for inputting into unentered data fields within the training data set, analyze the training data set to jointly model variable relationships using a non-linear function approximation algorithm that is repeatedly applied to the records of the training data set. (d) Calculate predicted values of the measurement data for the unentered data fields using the measurement prediction protocol. (e) Input the predicted values into the records within the training data set. (f) Analyze the training data set based on predefined disease states in the known patient records of the basic data set to form a phenotype model that associates the patient's phenotype data with the probability of the disease state in the patient records of the training data set. (g) Input the predicted values into the records within the validation data set. (h) Validate the validation data set using the phenotype model and determine a validation error that includes the probability of correctly predicting the patient's phenotype associated with the probability of the disease state in the records of the validation data set. (i) Repeat steps (c) through (h) to minimize the validation error and calculate and predict the disease state phenotypes with high probability for each patient record in the basic data set. The records within the validation data set include the patient's phenotype data associated with the probability of the disease state.

[0030] Optionally, the step of analyzing the training data set is performed using a machine learning system.

[0031] Optionally, the disease state is aortic stenosis.

[0032] Optionally, the sparse data source includes a plurality of medical records containing measurement data obtained from a medical research process.

[0033] Optionally, the medical research process includes an echocardiogram process.

[0034] Optionally, during the measurement process, it includes the step of predicting unentered measurement data based on the data collected by the measurement process operator using the measurement prediction protocol.

[0035] Optionally, during the measurement process, it includes the step of determining the patient's phenotype by the phenotype model based on the data collected by the measurement process operator and / or based on the measurement data predicted using the measurement prediction protocol.

[0036] Optionally, during the measurement process, it includes the step of calculating unentered measurement data and the patient's phenotype in real time.

[0037] Optionally, the machine learning system comprises a neural network, during the measurement process, the step of forming an updated dataset by incorporating the measurement values obtained by the measurement process operator into the training dataset and updating it; using the neural network to analyze the updated training dataset and calculate an updated measurement prediction protocol and / or an updated phenotype model; using the updated measurement prediction protocol and / or the updated phenotype model to analyze the measurement values obtained during the measurement process and predict the likely disease state of the patient undergoing the measurement process.

[0038] Optionally, as a result of the phenotype prediction related to a predefined disease state, it includes the step of instructing the measurement process operator to record relevant measurement data to enhance the reliability of the prediction of the patient's phenotype and the related disease state.

[0039] According to a second aspect of the present invention, an apparatus for performing a measurement process on a patient is provided. This apparatus includes a measurement tool related to the measurement process, recording means for recording measurement data from a patient during a measurement process, and transmission means for transmitting the measurement data to an analysis means. The analysis means includes input means for receiving the measurement data and phenotype data, association means for associating the measurement data with the phenotype data to determine the phenotype of a patient related to one or more disease states, a measurement prediction protocol for predicting measurement data for unentered measurement fields, and / or a phenotype model for associating patient data with a phenotype related to one or more disease states, thereby predicting the likely disease state of a patient undergoing a measurement process, warning means for warning a measurement operator of predicted measurement data and predicted disease states, and instruction means for instructing a measurement operator of related measurement data to be collected based on the predicted disease state.

[0040] Optionally, the disease state is aortic stenosis.

[0041] Optionally, it comprises a display surface for displaying to the measurement operator a notification including the predicted measurement data or the predicted disease state.

[0042] According to a third aspect of the present invention, there is provided a computer-implemented method for processing sparse data sources. The method comprises (a) retrieving data from the sparse data source to form a basic data set, wherein the sparse data source includes a plurality of patient records including patient mortality data, each of the patient records including at least one unentered data field corresponding to a medical measurement value, (b) dividing the basic data set into the following two parts, namely, a first part including a training data set of a predefined percentage X% of the basic data set, a second portion including a validation data set at a predefined percentage (100% - X%) of the basic data set, and dividing it; (c) Analyzing the training data set to jointly model variable relationships using a non - linear function approximation algorithm repeatedly applied to the records of the training data set for the purpose of obtaining a trained model and a measurement prediction protocol for inputting into un - filled data fields within the training data set; (d) Calculating predicted values of measurement data for the unfilled data fields using the measurement prediction protocol; (e) Inputting the predicted values into the records within the training data set; (f) Analyzing the training data set based on predefined disease states in the known patient records of the basic data set to form a phenotype model that associates a patient's phenotype data with the probability of a disease state in the patient records of the training data set; (g) Inputting the predicted values into the records within the validation data set; (h) Validating the validation data set using the phenotype model and determining a validation error including the probability of correctly predicting the phenotype of a patient associated with the probability of a disease state in the records of the validation data set; (i) Repeating steps (c) through (h) to minimize the validation error and calculating and predicting a high - probability disease state phenotype for each patient record in the basic data set. The records within the validation data set include a patient's phenotype data associated with the probability of the disease state.

[0043] According to a fourth aspect of the present invention, a computer system is provided. This computer system comprises one or more processors, for the one or more processors, (a) Retrieving data from sparse data sources to form a basic data set, wherein the sparse data sources include a plurality of patient records containing patient mortality data, and each of the patient records includes at least one unpopulated data field corresponding to a medical measurement value, characterized by the step of; (b) Dividing the basic data set into the following two parts, namely, A first part including a training data set of a predefined percentage X% of the basic data set, and A second part including a validation data set of a predefined percentage (100% - X%) of the basic data set, characterized by the step of dividing; (c) Analyzing the training data set to jointly model variable relationships using a non-linear function approximation algorithm repeatedly applied to the records of the training data set for the purpose of obtaining a trained model and a measurement prediction protocol for populating the unpopulated data fields within the training data set; (d) Calculating predicted values of measurement data for the unpopulated data fields using the measurement prediction protocol; (e) Populating the predicted values into the records within the training data set; (f) Analyzing the training data set based on predefined disease states in known patient records of the basic data set to form a phenotype model that associates patient phenotype data with the probability of disease states in the patient records of the training data set; (g) Populating the predicted values into the records within the validation data set; (h) Validating the validation data set using the phenotype model and determining a validation error including the probability of correctly predicting the patient phenotype associated with the probability of disease states in the records of the validation data set; (i) Repeating steps (c) through (h) to minimize the validation error, and calculating and predicting high-probability disease state phenotypes for each patient record in the basic data set. One or more memories for storing instructions to cause execution, and. The records in the verification data set include phenotypic data of patients associated with the probability of the disease state.

[0044] According to a fifth aspect of the present invention, there is provided a computer program product having a computer-readable medium on which a computer program for processing a sparse data source is recorded. This computer program product (a) Computer program code for retrieving data from the sparse data source to form a basic data set, wherein the sparse data source includes a plurality of patient records including patient mortality data, and each of the patient records includes at least one unpopulated data field corresponding to a medical measurement value. Computer program code characterized by (b) The basic data set into the following two parts, namely, A first part including a training data set of a predefined percentage X% of the basic data set; A second part including a verification data set of a predefined percentage (100% - X%) of the basic data set; Computer program code for dividing into; (c) Analyzing the training data set to jointly model variable relationships using a non-linear function approximation algorithm that is repeatedly applied to the records of the training data set for the purpose of obtaining a trained model and a measurement prediction protocol for inputting into the unpopulated data fields within the training data set. Computer program code (d) Computer program code for calculating predicted values of measurement data for the unpopulated data fields using a measurement prediction protocol; (e) Computer program code for inputting the predicted values into the records within the training data set; (f) computer program code for analyzing a training data set based on a predefined disease state in known patient records of the basic data set to form a phenotype model that associates the patient's phenotype data with the probability of the disease state in the patient records of the training data set; (g) computer program code for inputting the predicted values into the records in the validation data set; (h) computer program code for validating the validation data set using the phenotype model, and computer program code for determining a validation error including the probability of correctly predicting the patient's phenotype associated with the probability of the disease state in the records of the validation data set; (i) computer program code for repeating computer program code (c) to (h) to minimize the validation error, and for calculating and predicting a high-probability disease state phenotype for each patient record in the basic data set; one or more memories storing instructions for causing the above to be executed. The records in the validation data set include the patient's phenotype data associated with the probability of the disease state.

[0045] Optionally, the disease state is aortic valve stenosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Other optional forms included within the scope of the present invention and preferred embodiments of the present invention will be described by way of example only with reference to the accompanying drawings.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9A

Figure 9B

Figure 9C

Figure 9D

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Modes for Carrying Out the Invention

[0047] [Definitions] The following definitions are given as general definitions. The scope of the present invention is not limited only to these terms. The following definitions are for the purpose of assisting in the understanding of this specification.

[0048] Unless otherwise specifically defined, technical and scientific terms used in this specification have the same meaning as commonly understood by those skilled in the art. Furthermore, the terms used here have a meaning consistent with the context of this specification and the related technology. These terms do not have an idealized or overly public meaning unless otherwise defined. Additional terms for the present invention are defined below. Furthermore, all definitions in this specification are to be understood as taking precedence over dictionary definitions, definitions included in references, and / or ordinary meanings. However, in case of doubt regarding a specific term, the dictionary definition and / or the commonly used meaning take precedence.

[0049] For the present invention, terms are defined as follows.

[0050] The indefinite articles "a" and "an" are used for plural matters (i.e., meaning at least one). For example, "an element" means one or more elements.

[0051] As used herein, the term "about" means an amount having a width, for example, of 30% (preferably 20%, more preferably 10%) with respect to the amount being referred to. When "about" is used with a numerical value, it simply means that the numerical value is not exact.

[0052] Throughout this specification, unless the context requires otherwise, the terms "comprising" and "comprises" mean that an element or step belongs to a particular set of elements or steps, but do not exclude the possibility that another element or step belongs to that set.

[0053] The terms "including" and "includes" as used herein are also used in a non - limiting sense. That is, it means that an element / feature is included in a certain concept, but does not exclude the possibility that another element / set is included in that concept. Thus, "including" and "comprising" are synonyms.

[0054] In the claims and the specification, all transitional phrases (such as "comprising", "carrying", "having", "including", "encompassing", "possessing", "consisting of", etc.) should be understood to be open - ended (i.e., "including but not limited to"), except for the terms "consisting of" and "consisting essentially of", which are closed - ended and semi - closed - ended respectively.

[0055] The term "real - time", for example, "display real - time data", means to display data without intentional delay, taking into account the processing limitations of the system and the time required to accurately measure the data.

[0056] "Substantially real-time" means, for example, as in "acquire real-time or substantially real-time data", without intentional delay ("real-time"), or as close to real-time as is realistically possible (i.e., within the constraints of the system and processing limitations for acquiring, recording, or transmitting data, with a minimal but non-zero delay, whether intentional or not) to acquire data.

[0057] Any method or thing similar to those shown in this specification may be used in the practice or testing of the present invention. However, preferred methods or things may also be described. The methods, apparatuses, and systems described in this specification may be implemented in various ways and for various purposes. Such descriptions are for illustrative purposes only.

[0058] The methods and processes outlined in this specification may be coded as software executable on one or more processors using one of various operating systems or platforms. Further, such software may be described using an appropriate programming language and / or programming tools or scripting tools. Such software may be compiled as machine language code or intermediate language code executable on a framework or virtual machine.

[0059] In this regard, various inventive concepts may be embodied as a computer-readable media storage system (e.g., computer memory, one or more floppy (registered trademark) disks, compact disk (registered trademark), optical disk, magnetic tape, flash memory, circuit arrangements within a field-programmable gate array, or other non-transitory media or tangible computer storage media) encoded with one or more programs (which, when executed on one or more computers or other processors, perform various embodiments of the methods according to the present disclosure). The computer-readable media is storable and, for example, can load the stored program onto one or more different computers or other processors and execute various aspects according to the present disclosure.

[0060] As used herein, the terms "program" or "software" generally refer to any type of computer code or set of computer-executable instructions that can be used to program a computer or other processor to implement various embodiments of the present disclosure. Further, in some embodiments, one or more computer programs that execute the methods of the present invention need not be implemented on a single computer and may be distributed in modular form across multiple computers or processors.

[0061] Computer-executable instructions may be in many forms, such as, for example, program modules, and may be executed on one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. These perform particular tasks or implement particular abstract data types. Typically, the functionality of program modules is combined or distributed in the manner desired in various embodiments.

[0062] Also, data structures may be stored on a computer-readable medium in any suitable form. For simplicity of explanation, a data structure may be shown as having fields related to positions within the data structure. Similarly, such relationships may be achieved by applying storage to positions within a computer-readable medium that have relationships between fields and between the fields. However, any suitable mechanism may be used to establish relationships between information in fields within a data structure. Such mechanisms include mechanisms for establishing relationships between tags and other elements.

[0063] Moreover, the concept of the present invention may be implemented in one or more ways. An example thereof has already been described. The steps executed as part of this method may be ordered in any suitable manner. Accordingly, in embodiments, some steps may be executed in an order different from that described herein. For example, even if the embodiments described in the specification are sequential steps, some steps may be executed simultaneously.

[0064] As used herein and in the claims, the phrase "and / or" means "either, or both" of the combined elements. That is, the elements combined by this phrase may exist conjunctively or disjunctively. The same applies to multiple elements (i.e., "one or more" combined elements) enumerated with "and / or". Another element separate from the element specified by "and / or" may optionally exist (either in relation to or independent of the specified element). Thus, as a non-limiting example, when "A and / or B" is used with an open-ended phrase such as "comprising", it may mean only A (optionally including elements other than B), only B (optionally including elements other than A), or both A and B (optionally including other elements).

[0065] As used in this specification and the claims, the term "or" has the same meaning as "and / or" as defined above. For example, when separating listed items, "or" or "and / or" is understood to be inclusive. That is, it may include at least one of the listed items, or it may include two or more of them. Additionally, optionally, it may include other additional items not listed. On the other hand, only in explicit expressions such as "only one of...", "exactly one of...", or "consisting of..." within the claims, it includes only one element among the listed items. Generally, the term "or" used in this specification has an exclusive meaning only when following exclusive terms such as "either...", "one of...", "only one of...", or "exactly one of...". When the expression "substantially consisting of..." is used in the claims, it follows the ordinary meaning in patent law.

[0066] When referring to one or more of the listed elements, the phrase "at least one" as used in this specification and the claims means at least one of the listed elements, does not necessarily mean a particular one or all of the elements, and does not exclude any combination of the listed elements. Another element may optionally exist (either in relation to or independently of the element(s) specified by "at least one"). Thus, by way of non-limiting example, when referring to "at least one of A and B" (similarly "at least one of A or B" or "at least one of A and / or B"), it may mean that at least one A exists and B does not exist (optionally including elements other than B), or that at least one B exists and A does not exist (optionally including elements other than A), or that at least one A exists and at least one B exists (optionally including other elements).

[0067] In this specification, when the steps of a method are described sequentially, unless there is no other logical interpretation, the sequence need not be executed in that order in time series.

[0068] Furthermore, when the features or aspects of the present invention are described by a Markush group, the present invention can also be described using individual elements of the Markush group or subgroups of such elements.

[0069] [Detailed Description] In the following description, it should be noted that the same or identical reference numerals in different embodiments indicate the same or similar features.

[0070] As described above, the systems and methods disclosed herein predict missingness from cardiac echo data recordings, identify phenotypes of patients exhibiting aortic stenosis (AS) or similar disease states, and provide an artificial intelligence (AI) system and method for providing a disease risk assessment from incomplete data recordings using AI-input data.

[0071] Described below (particularly from paragraphs

[0120] to

[0165] of this document) is an implementation of an AI approach in which a machine learning system with a supervised neural network (in the form of a mixture density network) is trained to output a probabilistic imputation of missing data, and a secondary classification algorithm applies clinically determined thresholds to the imputation output to predict the presence of a disease phenotype. The neural network randomly holds training data and is internally validated by minimizing the imputation error. The validation of the classification algorithm is described in paragraphs

[0197] to

[0227] , but in summary, it is as follows

[0072] The classification algorithm is validated by optimizing performance metrics on a validation dataset. The performance of the classification algorithm in predicting the AS phenotype showed an AUROC of 0.9696 (see particularly paragraph

[0218] ). The AI diagnosis maintained a high predictability of death even after adjustment for age and gender (see particularly paragraph

[0220] ).

[0073] External validation was performed in the form of several independent clinical trials conducted in Australia and the United States, and consistently identified additional patients with severe aortic stenosis who deviated from the diagnostic guidelines but were at high risk of death from the disease. In one study, the past echocardiogram records of 9,189 patients were analyzed using the AI system and method disclosed above. In contrast to the diagnosis made by medical experts during echocardiography, the AI system described herein found a 72% increase in the number of cases of severe aortic stenosis. It is well known that AS is often undiagnosed in female patients, probably due to differences based on sexual dimorphism in the heart muscle. This study also found that women were 66% more likely to be misdiagnosed by human diagnosis alone than men. Systems and methods such as the AI system described above examine many times more data points in a completely objective manner, hold great promise in improving patient outcomes and correctly identifying diseases so that patients can receive appropriate treatment in a timely manner. It is clear that AI systems and methods such as those disclosed herein can be valuable tools to complement the expertise of medical professionals.

[0074] A specified copy of the NEDA database containing all measurements, including mortality data, is used to create the AI model described below. A random subset of 70% of the patients is used to train the modified mixture density network 10 to function as a multiple imputation model. This model is trained using missing data by augmenting the model with a boolean input indicating whether a measurement exists or not in order to predict severe AS.

[0075] In particular, for data entries from, for example, NEDA data sources, measurements were randomly excluded from the training input and used as the target output for regression. Backpropagation was applied only to the model output where the target output was present. Thus, the training examples approximately resemble a typical set of measurements encountered in the echo without requiring a complete set of measurements for the training process. As a result, the model is general-purpose and designed to perform inference using any set of available measurements. The non-standard backpropagation process utilized here can be regarded as training a family of models with shared weights (this is similar to the technique previously applied to restricted Boltzmann machines 15 ). The input holdout process is similar to a common technique known as dropout (however, in this case, instead of using the existing technique of discarding the "dropped-out" values, it is used to construct the sentinel vector and target output), and it is thought to have a secondary effect of regularizing the model. This promotes the learning of more generalizable patterns 11 . The Continuous Ranked Probability Score (CRPS) has a closed-form solution for mixtures of Gaussians 12 and is chosen as the loss function because it promotes convergence to sharp and well-calibrated predictions 13 . Intuitively, the CRPS loss function penalizes models that predict incorrect expected values and also penalizes overconfident or underconfident predictive distributions. Figure 10 shows the overall architecture 800 of the model and the learning process. The overall learning process consists of a random input holdout 801 followed by backpropagation from the target output. The input 803[x1...x n is the echo measurement containing missing values, and the output 805(μ i , σ i )(where i ∈ {1,...,n} is the mean μ i and the standard deviation σ ishows a Gaussian predictive density (however, the general approach can be applied to any choice of density function having a closed-form solution for CRPS that includes a mixture of Gaussians). The enlarging part 850 is the predicted value x n-2 ~N(μ n-2 , σ n-2 )'s cumulative density function (CDF), plotted for possible values of the measured value z and compared with the target value. CRPS is calculated by integrating the square of the height of the shaded part over all values of z.

[0076] This model was tested by applying it to the remaining 30% subset of test patients that were not used for training. As an initial diagnosis, a selected group of relevant measurements was withheld and the AI's predicted values were evaluated against the known measurements. From these results, it was shown that the predicted measurements had minimal bias and surprisingly low error considering the non-uniform nature of the data and that important information (such as left ventricular outflow tract data) had been removed from the study. Figure 11 is a plot of the error distribution when the trained model was applied to the test subset, that is, a plot of the prediction error of the measurements in the 30% test set.

[0077] In Figure 11, considering the non-uniform nature of the data and that important information (i.e., LVOT data) has been removed from the study, the results show minimal bias and a low error rate. The left panel shows the input measurements overlaid with the actual measurements. The right panel shows the imputation error (imputation vs actual measurement) calculated after predicting while holding each measurement and any direct dependent variables. The mean error (95% confidence interval) was as follows. LVOT dimension = 0.010 cm (-0.165 to 0.165), LVOT velocity time integral = 0.669 cm (-6.019 to 4.249), mean transaortic valve gradient = 0.068 mmHg (-5.639 to 3.133), aortic valve area = 0.056 cm 2 (-0.885 to 0.664), p = ns for each imputation value vs actual measurement.

[0078] Subsequently, the AI-predicted value of the aortic valve area was evaluated in the clinical context of severe AS classification. First, the data of the test set were filtered to consider only studies with a known aortic valve area calculated using the continuity equation, and this was used to label the studies as "severe AS" or "not severe AS". Next, all measurements of the left ventricular outflow tract (velocity, gradient, diameter), aortic valve area, and dimensions of the aortic root were excluded from this test set, and the model was used to predict the distribution of likely values of the aortic valve area. The cumulative density function of the predicted values was evaluated to calculate p(AVA < 1 cm 2 ), and the predicted probability of severe AS was derived. Next, ROC analysis was performed to quantify the performance of the classifier by comparing the probability predicted by the AI with the original classification label using the standard and continuously derived AVA. It will be readily understood by those skilled in the art that this method is suitable for applying this type of model to existing workflows, processes, and guidelines. [Severe Aortic Valve Stenosis]

[0079] AS is evaluated from echocardiogram data using measurements such as, for example, peak aortic jet velocity, mean aortic gradient, and aortic valve area defined by the continuity equation (CE) 14 : [Number] (CE) Here, CSA LVOT is the cross-sectional area of the left ventricular outflow tract LVOT = (πD 2 ) / 4 and D = LVOT are the dimensions (cm) VTI LVOT is the velocity-time integral of the LVOT velocity trace VTI AV is the velocity-time integral of the aortic valve velocity trace is

[0080] Severe AS is defined as AVA < 1.0 cm 2 (the highest measured VTI AV and mean VTI LVOT ). [Characteristics of patients identified as severe AS by AI]

[0081] Patients identified as severe AS by the AI system had the expected demographic and clinical characteristics (see Table 4 in Figure 12), with an increased aortic valve gradient, decreased left ventricular diastolic function, and increased indexed left ventricular mass, indexed left atrial volume, and right ventricular systolic pressure. Patients diagnosed as severe AS by the AI system but not by the continuity equation (CE) had similar characteristics to those diagnosed as severe AS by the continuity equation, except for lower transaortic gradient and stroke volume index, which was consistent with the interpretation of typical cardiac structural changes associated with aortic valve stenosis by AI.

[0082] Using the characteristics of patients diagnosed with AS, identify the characteristics of the AS-related phenotypes and apply the characteristics of that phenotype to new patient data. Mortality data from the NEDA database are also examined in conjunction with the characteristics of patients confirmed with AS to provide an improved set of phenotypic characteristics to provide a more accurate prediction of AS diagnosis. [Application to a limited dataset]

[0083] To determine which echocardiographic parameters most influence AI-based prediction, measurements typically representing the pressure-loaded left ventricle in severe AS were selected. Using these, a limited test set containing only the following variables as model inputs was created (if known). Gender, height, weight, basic two-dimensional dimensions (interventricular septum and posterior wall diastolic thickness, systolic and diastolic left ventricular internal dimensions), left ventricular ejection fraction measured using Simpson's biplane method, ascending aortic dimension, atrial dimensions (four-chamber and two-chamber left atrial area, four-chamber left atrial length, right atrial area), mitral valve length, left atrial area, right atrial area, right atrial area), mitral inflow pulsed wave Doppler data (E velocity, A velocity, E-wave pressure half-time, mitral inflow velocity time integral), left ventricular diastolic basal tissue Doppler velocity (E' septal velocity, E' lateral velocity), transaortic velocity (aortic peak velocity, aortic velocity time integral), pulmonary valve peak velocity. [(a) Complete set of measured variables]

[0084] The complete set of variables used in the training of artificial intelligence in the "included" examination was as follows.

[0085] General - height, weight, body mass index, body surface area, gender, age, heart rate.

[0086] Two - dimensional measurements - IVS end - diastolic thickness, LVPW end - diastolic thickness, LA area, LA length 4C, LA systolic LX diameter, LA volume, LA volume index, LV end - diastolic area 2C, LV end - diastolic area 4C, LV end - diastolic area PSAX, LV end - diastolic diameter PLAX, LV systolic diameter MM, LV systolic diameter PLAX, LV end - diastolic length 2C, LV end - diastolic length 4C, LV end - diastolic volume 2C, LV end - diastolic volume 2D Teich, LV end - diastolic volume 4C, LV end - diastolic volume BP, LV ejection fraction 2C, LV ejection fraction 4C, LV ejection fraction BP, LV stroke volume 2C AL, LV stroke volume 2D Teich, LV stroke volume 4C AL, LV stroke volume MOD 2C, LV stroke volume MOD 4C, LV stroke volume MOD BP, LV stroke volume SIM, LV systolic area 2C, LV systolic area 4C, LV systolic diameter 4C, LV systolic length 2C, LV systolic length 4C, LV systolic volume 2C AL, LV systolic volume 2D Teich, LV systolic volume 4C AL, LV systolic volume MOD 2C, LV systolic volume MOD BP, ejection fraction (regardless of method), aortic annulus diameter, aortic annulus diameter, ascending aorta diameter, distal transverse aorta diameter, IVC diameter expiratory, IVC diameter inspiratory, LV Epi end - diastolic area PSAX, LV mass AL, LV mass AL index, RA area, right atrial volume, right atrial volume index.

[0087] Doppler measurements - MV mean gradient, MV mean velocity, MV peak gradient, MV peak velocity, MV pressure half-time, MV velocity time integral, MV area PHT, MV deceleration gradient, MV deceleration time, mitral valve A-wave velocity, mitral valve E-wave velocity, mitral valve E / A ratio, MV E' septal velocity, LV E' lateral velocity, LV A' velocity, mitral valve E / E prime. Septal ratio, mitral valve E / E prime lateral ratio, mitral valve mean E / E' ratio, AR peak velocity, AR pressure half-time, atrioventricular mean gradient, atrioventricular mean velocity, atrioventricular peak gradient, atrioventricular peak velocity, atrioventricular velocity time integral, pulmonary vein atrial duration, pulmonary vein atrial reversal velocity, pulmonary vein A velocity, pulmonary vein diastolic velocity, pulmonary vein S / D ratio, pulmonary vein systolic velocity, TR peak gradient, TR peak velocity, right ventricular systolic pressure, PV mean gradient, PV mean velocity, PV peak gradient, PV peak velocity, PV velocity time integral, RVOT mean gradient, RVOT mean velocity, RVOT peak gradient, RVOT peak velocity, RVOT velocity time integral. [(b) Reduced / Limited Set of Measurement Variables]

[0088] The set of variables included in the "limited" test was as follows.

[0089] General - Gender, height, weight, body mass index, body surface area.

[0090] 2D measurements - Ejection fraction (by any method), LV systolic diameter-based PLAX, LV diastolic diameter-based PLAX, LV systolic diameter PLAX, LVPW diastolic thickness, IVS diastolic thickness, LA area, RA area, aortic root diameter, aortic root diameter, ascending aorta diameter.

[0091] Doppler measurements - AV mean gradient, AV mean velocity, AV peak gradient, AV peak velocity, AV velocity time integral, mitral valve E' septal velocity, mitral valve E' lateral velocity, mitral valve E to E prime ratio septal, mitral valve E to LV E prime ratio lateral, mitral valve E to MV E prime ratio mean, MV mean gradient, MV mean velocity, MV peak gradient, MV peak velocity, MV pressure half-time, MV velocity time integral, mitral valve A point velocity, mitral valve E point velocity, mitral valve E to A ratio, PV peak gradient, PV peak velocity.

[0092] This limited dataset takes approximately 10 minutes to acquire when performing a transthoracic echocardiogram. All other variables, including left ventricular outflow tract measurements, were excluded from the test set, and the AI was evaluated in the same manner as described above. This was defined as the "limited echocardiogram" system, and the output classification was the probability of severe AS as described above.

[0093] Since echocardiogram examinations are time-consuming, for efficiency, the echocardiogram examination focuses only on measurements that the cardiologist determines to be relevant. This means that the NEDA database contains an incomplete set of measurements for each patient, which is an obstacle to the use of many techniques from the fields of statistics and machine learning.

[0094] A naive solution to this problem is "complete case analysis", where a subset of measurements is selected and all patient data with incomplete sets of measurements are discarded. This approach is flawed for two reasons. · As a result of this approach, a large amount of useful information is discarded. · This approach results in sampling bias. This is because the fact that a certain patient population has a particular set of measurements is likely to be related to a particular disease group. In statistical terms, the measurements are MNAR (Missing Not At Random).

[0095] A better option is to "fill in the blanks", which in statistical terms is to impute the missing values.

[0096] The AI engine described above is also configured to determine a typical phenotype of a patient with aortic stenosis (AS) or a similar disease state by identifying a phenotype of increased risk having characteristics similar to those observed in aortic stenosis. Specifically, the AI engine is intended for applications that use echocardiogram data and mortality data to predict risk phenotypes that may be found in aortic stenosis and other diseases with similar characteristics.

[0097] The general characteristics of AS are as follows. There are abnormalities in the velocity across the aortic valve during systole, accompanied by high flow, normal flow, or low flow. There is aortic valve stenosis associated with a reduction in aortic valve area, aortic valve calcification, bicuspid aortic valve, unicuspid aortic valve, or other reasons for abnormal aortic valve opening. The flow rate through the left ventricular outflow tract is increased, normal, or low. The dimensions of the left ventricle are normal, enlarged, or small.

[0098] Left ventricular systolic function is measured using left ventricular ejection fraction, left ventricular fractional shortening, left ventricular systolic and diastolic volumes, or left ventricular systolic and diastolic dimensions, and is normal, hyperdynamic, or impaired.

[0099] The thickness of the left ventricular wall may be normal, increased, or decreased, and is related to changes in left ventricular mass (which may be normal, increased, or decreased). Left ventricular diastolic function is normal or abnormal and is related to the following measured values: mitral valve E-wave velocity, mitral valve A-wave velocity, mitral valve E / A ratio, septal E' velocity, lateral E' velocity, septal E:E' ratio, lateral E:E' ratio, global longitudinal strain, left atrial area, left atrial dimensions, left atrial width, left atrial volume, left atrial volume index, right ventricular systolic pressure, tricuspid valve regurgitation velocity, right atrial pressure, and right atrial area.

[0100] In severe aortic valve stenosis, there is a typical phenotype of abnormalities. The above-mentioned AI engine aims to identify this phenotype, noting that this phenotype may also be seen in other similar or related diseases such as amyloid cardiomyopathy, hypertensive heart disease, infiltrative cardiomyopathy, restrictive cardiomyopathy, left ventricular outflow tract obstruction, left ventricular outflow tract membrane, supravalvular aortic valve stenosis, and valvular hemodynamic disorders after aortic valve replacement. Artificial intelligence identifies the typical changes in aortic valve stenosis, but they may be seen in anyone among this series of diseases or in combinations of individual diseases.

[0101] Typical phenotypes are as follows, but as described above, there are many variations. In severe aortic stenosis, typically, there is an increase in the transvalvular aortic gradient associated with a small aortic valve area, and the left ventricular outflow tract velocity is normal. The size of the left ventricular cavity and left ventricular volume are normal, and the left ventricular ejection fraction and stroke volume are normal. The left ventricular wall thickness and left ventricular mass increase. There is a decrease in the left ventricular myocardial relaxation velocity (septal and lateral E' velocity), an increase in the mitral inflow E wave velocity, and a normal (pseudo-normal) E / A ratio. The A wave velocity may decrease. Signs of increased left ventricular filling pressure with an increase in the E:e' ratio are seen in the septum and laterally. The left atrial volume increases, accompanied by an increase in left atrial pressure, an increase in tricuspid regurgitation velocity, and an increase in right ventricular systolic pressure (signs of pulmonary hypertension).

[0102] An important variation in the phenotype of aortic stenosis is when the systolic function is decreased. Again, there are various variations, but the typical scenario is as follows. In severe low-flow low-gradient aortic stenosis, the aortic valve area is small and the left ventricular outflow tract velocity is low, so the transaortic velocity and transvalvular aortic gradient increase (or are normal). The size of the left ventricular cavity is normal or increased, accompanied by a decrease in the left ventricular ejection fraction and a decrease in stroke volume. The left ventricular wall thickness and left ventricular mass increase. The left ventricular myocardial relaxation velocity (septal and lateral E' velocity) decreases, the mitral inflow E wave velocity increases, and the E / A ratio is normal (pseudo-normal). The A wave velocity may decrease. Signs of increased left ventricular filling pressure with an increase in the E:e' ratio are seen in the septum and laterally. The left atrial volume increases, accompanied by an increase in left atrial pressure, an increase in tricuspid regurgitation velocity, and an increase in right ventricular systolic pressure (signs of pulmonary hypertension). [AI-assisted Report]

[0103] In the embodiments disclosed in this specification, an AI-assisted cardiac echocardiogram report assisting device is provided. In this embodiment, the patient's cardiac echocardiogram examination is performed as usual, and then the AI model is used to reinforce the examination by a set of predictions and determination of the patient's phenotypic characteristics. This process 300 is schematically shown in FIG. 3.

[0104] A major difference between the prior art process 200 of FIG. 2 and the AI-assisted reporting process 300 shown in FIG. 3 is that an AI model prediction 301 is added to impute missing measurement data into the patient's measurement record for the scan. This enables step 4 - "AI-assisted analysis" 303. The AI-assisted analysis 303 can use the actual measurements obtained by the sonographer in step 101 and optionally also include predictions by an AI model of missing measurement parameters from step 301, and provide the medical expert (e.g., sonographer or cardiologist) analyzing the examination with a calculated estimate of the patient's phenotypic characteristics and the risk of related possible disease states. Next, the sonographer creates a preliminary report 109 that includes the measurement data recorded during the scan and, if utilized, the measurement data input into the scan record by the AI system, and the report is transferred to the cardiologist for further analysis. In addition to the AI-assisted analysis performed by the sonographer in step 4 303, the cardiologist can optionally use an AI system (either the same one used by the sonographer or a different AI model) to analyze the scanned measurement data and / or the input measurement data 305 from the perspective of determining the risk of progression to the patient's general health or disease state.

[0105] Either the sonographer or the cardiologist can refer to the phenotypic characteristics determined by the AI model and compare them to phenotypes with known associations to AS or similar disease states.

[0106] An important efficiency improvement from process 300 of the prior art process 200 lies in shortening the time that both the sonographer and the cardiologist spend manually checking measurements for the presence or absence of anomalies in a patient's scan results, which leads to a specific diagnosis of the patient's health condition, by using AI-assisted analysis techniques as discussed herein. Process 300 also has the significant advantage of being configured, optionally, to identify other potential patient anomalies based on predictive data for the absence of measurement data entered into the patient's scan record. This enables healthcare providers to pick up on subtle and rare conditions that less experienced sonographers or cardiologists might miss. In such cases, the medical condition may progress untreated and only be discovered once more extreme symptoms appear. In such prior art processes, subtle or less common disease indicators are missed by healthcare providers, leading to a worse patient outcome and generally increased costs for healthcare providers. [AI in the Loop]

[0107] A more advanced application of the AI model disclosed herein is an "AI in the Loop" that integrates directly with the measurement process performed by the sonographer while taking patient measurements during the scan. This configuration of the AI model is adapted to provide real-time predictions of various echocardiogram measurements to the sonographer while the patient is being scanned. Figure 4 is a schematic diagram of the workflow of the AI in the Loop configuration.

[0108] The main advantage of this approach is that the system can predict 401 in real time while the scan is in progress, so that in some cases, it may not be necessary for the sonographer to perform specific measurements. The system can also update the predicted values of one or more possible diagnoses 403 of the conditions the patient may have, based on the predicted measurements 401 and the measurements obtained by the sonographer. If the measurements predicted by the AI output a sufficiently high "confidence level" from the system, they can be used as is, saving time. Alternatively, if the sonographer is not satisfied with the reliability of the predicted measurements, specific measurements can also be manually performed while the patient is present. The system can also optionally request specific measurements 405 that the sonographer obtains based on the predicted diagnosis 403, for example, manually obtain specific measurements useful for confirming or excluding a specific predicted diagnosis made based on existing acquired and predicted measurements.

[0109] The method 400 proposed in Figure 4 is a particularly persuasive proposal, especially considering the fact that new measurement techniques are constantly emerging in the literature, and echo specialists (especially sonographers) are increasingly required to prioritize which measurements to perform within the limited time available for the examination.

[0110] The second advantage of method 400 is that when there are subtle abnormalities in the patient's heart, the system is configured to propose additional measurements to the sonographer, and data can be acquired on-site while the patient is present. This significantly improves the method 300 of Figure 3, where the medical staff can only flag the abnormality after the patient has left the clinic. This is because the patient may need to return for an expensive and time-consuming second health check.

[0111] Furthermore, the characterization of the patient's phenotype during an echocardiogram is particularly advantageous when the identification of phenotypes associated with AS or similar diseases is being evaluated, as the AI model can notify the sonographer of a possible positive association during the echocardiogram, such that while the patient is available, the sonographer can be instructed to record one or more additional measurements of the patient, which can be collected on the spot without the need to recall the patient after the initial examination to perform further investigations and record the additional measurements in order to confirm or rule out a possible disease state. [System Architecture]

[0112] In recent practice at the European Patent Office, very detailed descriptions are required regarding the actual processes used for training and implementing AI systems. In the following description, as much specific detail as possible will be provided.

[0113] Figure 5 depicts the main components of a system 500 adapted for the AI-assisted echocardiogram disclosed herein, as well as the interconnections and interactions between the components of system 500. The details of the implementation of specific components of system 500 typically vary depending on the application and may be subject to change, and are thus considered outside the scope of this document.

[0114] System 500 is particularly adapted to connect to a database 501 of medical records. This medical record may be in any form of structured medical data, and may include data obtained from medical images, measurements taken during a procedure, or even the output of a natural language processing (NLP) algorithm by reading existing medical reports. In the embodiments disclosed herein, data source 501 includes a set of measurements taken from an echocardiogram, although it will be readily apparent to those skilled in the art that the methods and systems disclosed below may be applied when substituting missing measurement data in records of different types of datasets (medical or otherwise) that contain a relatively large number of possible measurement data per record.

[0115] A portion of the data records from the data source 501 is designated as training data 503 that is used to train the system 500. Due to the complex nature of the human body, usually a large number of examples are required. By having records of a large number of examples (on the order of hundreds of thousands of records), it becomes more possible for the AI model to distinguish the true patterns in the data from the random noise caused by various factors such as human errors. Approximately 60%, 80%, typically about 70% of the records from the data source 501 are selected to form the learning data. The remaining 20%, 40%, typically about 30% are designated as test data 505. This is used to verify the prediction ability of the AI model on an actual data set with known data values in the latter half of the process. As will be understood by those skilled in the art, since many AI models are prone to overfitting, it is important not to use the training data 503 for verifying the AI model. Because such data leads to the undesirable result of fitting the model to the patterns of the training data, which are not meaningful data but non-generalizable "noise".

[0116] This architecture of the system 500 enables the possibility of online training as more data is collected in the clinical setting and continuously added to the data source 501.

[0117] The AI model in the currently described arrangement of the system 500 is provided with an initial model state 507 that encodes the behavior of the AI. For example, an artificial neural network (ANN) model (a type of machine learning system) has a set of weight coefficients that are learned during the learning process. New models are usually initialized in a random state, but in the case of online learning, a previously learned model can also be used as the initial model state.

[0118] As will be appreciated, the system 500 presented herein enables any AI model to be used in the system 500, provided that the inputs and outputs meet a standard interface specification that is specific to the desired result of the system and takes into account the nature of the data records in the data source 501. The details of the learning process 509 depend heavily on the selected AI algorithm. Typical algorithms "learn" by repeatedly processing each example and using the "mistakes" made by the model to gradually change the state of the model. If the learning process is successful, more accurate predictions can be made over time.

[0119] When the learning process 509 is complete, the AI model reaches (converges to) the proposed trained model 511 for data analysis that encodes the state of the learned AI model. A preferred training process 509 is non-parametric in that the training process technique employed does not make explicit assumptions about the relationships between variables or data elements within the data source 501. A useful analogy for the training process 509 would be to train an AI system (such as system 500) to build a model of a cube across three dimensions of width, height, and depth (i.e., the available variables or features to be modeled).

[0120] This is in contrast to traditional modeling techniques that assume explicit parametric relationships or probabilistic models. In this case, human trial and error is required to find effective approximate relationships. As the required model fidelity increases and the number of variables grows and complex interdependencies exist, it becomes almost impossible for humans to trial and error. [MDN Model - Sparse Data Mixture Density Network (MDN) Imputation Model]

[0121] In an alternative specific embodiment of system 500, the AI model takes the form of a sparse data mixture density network (MDN) imputation model as discussed below (note that the previous mathematical definitions in this document, particularly the definitions related to the sparse data self-organizing map imputation model (A) discussed above, should be ignored in favor of the following definitions).

[0122] The formulation of the sparse data MDN discussed here can be applied to the selection of any parametric mixture model. For the purpose of showing a specific embodiment, this section provides additional details regarding the formulation of an algorithm using the selection of a Gaussian mixture model (GMM). [MDN.1 Definition]

[0123] m t ∈Z + Let it be the number of training examples.

[0124] n v ∈Z + Let it be the variables per training example.

[0125] Let N be the number of training epochs.

[0126] Define the "index matrix" as follows. [Number]

[0127] Define the search function for example i, variable j as follows. [Number] [Number]

[0128] Let the normalization constant of the z-score be given by the following equation. [Number] (B1)

Number

[0129] Let the z - score transformation function of variable j be given by the following equation.

Number

[0130] Let the normalized example vector be given by the following equation.

Number

[0131] Let p holdout be the probability that a variable is not used as a target output during the learning process.

[0132] Let the random input split vector of example i be given by the following equation.

Number

[0133] δ j Each of them is assumed to be drawn independently and identically distributed (i.i.d) from a binomial distribution of a single trial.

Number

[0134] Let the negated split vector be as follows.

Number

[0135] Let the binary operator be element - by - element vector multiplication.

Number

Number

[0136] Let the training input be given by the following equation.

Number

[0137] Let the target output be given by the following equation.

Number

[0138] Let the input sentinel vector of the neural network be given by the following equation.

Number

Number

[0139] Define the mini - batch size as B.

[0140] Define the Gaussian probability density function (PDF) as follows.

Number

[0141] Define the Gaussian cumulative density function (CDF) as follows.

Number

[0142] Define the softmax function as follows.

Number

Number

[0143] Let the input of the neural network for Example 1 be given by the following equation.

Number

[0144] Let the mini - batch of training data be as follows.

Number

[0145] Let the mini - batch of target outputs be as follows.

Number

[0146] Let the batch - splitting matrix and the negative batch - splitting matrix be as follows respectively.

Number

Number

[0147] c ∈ Z + be the number of model mixture components.

[0148] n p ∈ Z + be the number of parameters for each mixture component.

[0149] Let the overall feed - forward neural network function from input to output be as follows.

Number

[0150] f nnAlthough it is possible to define any feedforward neural network with given input and output dimensions, note that a nominal four - layer network with a width of 2048 and having the "leaky ReLU" activation function is sufficient.

[0151] Let the output logit of the neural network be given by the following equation.

Number

Number

[0152] Let the parametric probability density function representing the selected mixture model be as follows.

Number

[0153] Let the parametric probability density function representing the probabilistic prediction for batch i and variable j be given by the following equation.

Number

[0154] Let the cumulative density function of the predicted value for batch i and variable j be given by the following equation.

Number

[0155] Let the continuous ranked probability score (CRPS) of the probabilistic prediction of variable x with PDF ^ f(x) with respect to the true value v be given by the following equation.

Number

[0156] Let the CRPS of the prediction for batch i and variable j be given by the following equation.

Number

[0157] Let the observation fraction of the variable be given by the following equation.

Number

[0158] Let the observation fluctuation correction coefficient be given by the following equation.

Number

[0159] Let the batch loss function be given by the following equation.

Number

[0160] When the selected mixture model is a GMM, let n p = 3, and the output logit of the neural network is as follows.

Number

Number

[0161] When the selected mixture model is a GMM, to avoid problems due to limited numerical precision, let ε σ > 0 be a small constant that sets the minimum standard deviation of the mixture components.

[0162] When the selected mixture model is a GMM, the output predicted value is as follows.

Number

Number

Number

[0163] If the selected mixture model is a GMM, do the following.

Number

[0164] If the selected mixture model is a GMM, note the following.

Number

[0165] Note that a closed-form solution exists if the selected mixture model is a GMM.

Number

Number

Number

Number

[0166] For each batch x Nmt extracted from columns x1, x2, ..., x batch , x( batch_target ): 1. Calculate loss(x batch , x batch_target ). 2. Using automatic differentiation, calculate the loss gradient with respect to each neural network weight: ∂ / ∂w(loss(x batch , x batch_target ). 3. Update the neural network using backpropagation with the selected optimization algorithm. [MDN.2.4 Output] 1. Trained neural network. 2. Normalization constants μ1, ..., μ nv , σ1, ..., σ nv . [MDN.2.4 Imputation Algorithm]

[0167] The imputation algorithm disclosed below provides the system 500 with the ability to fill in missing or blank data measurement fields in records of the data source 501 during model validation 513, and also provides the system 500 with the ability to predict missing or additional data measurement values in the generated data source 519 while the system is in use. Generally speaking, imputation holds all cases by replacing missing data with estimates based on other available information. In the algorithms below, the estimates of the missing data are provided as a probability distribution of likely values. [MDN.3.1 Input] 1. Trained neural network 2. Normalization constants.

Number

[0150] (above), the normalized output of the search function for a series of measurement values:

Number

Number

[0163] . 3. Perform forward inference of the neural network to derive the probabilistic imputation ^f ij , ^F ij . [MDN.3.3 Output] 1. The input density functions: ^f ij , ^F ij

[0168] The classification algorithm disclosed below provides the system 500 with the ability to classify the likely severity of a disease in a patient based on the records in the data source 501 during model verification 513 and the measurements in the generated data source 519 during system use, based on the imputation density function for a given measurement value and the clinical threshold established for that measurement value. [MDN.4 Classification Algorithm] [MDN.4.1 Input] 1. The μ and σ of the input density functions ^f ij , ^F ij . 2. Selection of the threshold variable j. 3. A set of T - 1 clinical thresholds of the variable j that defines the T stages of the severity of the disease (e.g., normal, mild, moderate, severe):

Number

Number

[0169] For the predicted AVA, evaluate the cumulative distribution function (CDF) value with respect to the AVA value of 1.0. If the CDF value is greater than threshold_var_mid and less than threshold_var_high, the phenotype is moderate AS. If the CDF value is greater than threshold_var_high, the phenotype is severe AS. [MDN.4.3 Output] AS phenotype [Example]

[0170] An example of a dataset replicating the sparse data source 501 is shown in Table 1 below. Table 1 Example of a sparse data source

Table 1

[0171] Use the data source in Table 1 as the training set 503, and the system is configured to develop a model of a cube with width = height = depth constraints. Once the model is learned, predictions can be made for a similarly sparse dataset. For example, if the width is known, both the height and depth of the cube can be predicted. In this simple example, refer to Table 2 below, which consists of the original sparse dataset, i.e., the initial dataset (normal font typeface), and the predicted measurement values (bold typeface) output from the learned AI model, to show how the blank data fields of the cube records are filled. Table 2 Example of a sparse data source with the input data

Table 2

[0172] In the embodiments described herein, a cubic dataset (Table 1) is substituted with a NEDA database capable of constructing a heart model (phenotype) across a plurality of variables. This phenotypic model data describes the interrelationships of variables across many different heart configurations and diseases.

[0173] Model verification 513 uses test data 505 to evaluate the performance of the trained AI model 511 on new, previously unseen data. The output of the verification process 513 depends heavily on the particular application and data source 501 being used and is tested against specific performance evaluation criteria 515 to evaluate predictive power (e.g., the probability that predicted data measurements are within a defined tolerance compared to actual measurement data). An example of a useful performance evaluation criterion 515 is the Root Mean Square Error of the predicted measurements compared to the known data of the test data 505.

[0174] When it is verified that the learned model 511 can provide meaningful measurement predictions within a predefined tolerance range (defined according to the typical measurement error range obtained from the validation study plus a tolerance range of about 5%), the learned model 511 is imported into the primary prediction engine 517 of the system 500. Then, the prediction engine 517 is used to analyze new data during system operation, such as the generated data source 519, which can be directly derived from the measurements made by the patient's sonographer during the echocardiogram. For example, the generated data source 519 may be a database of measurements within an echo report software package, or it may be the measurements on an echo / ultrasound workstation obtained during the operation of the methods 200 or 300 disclosed above. The output from the AI prediction engine 517 consists of, for example, predicted measurement values 521 and / or predicted diagnostic values 523 for the generated data source 519, obtained during the operation of the methods 200 or 300 disclosed above. As can be easily understood, the output of the AI prediction engine 517 may be a set of predicted measurement values 521 that were not provided as input, or alternatively, it may be a prediction of a diagnosis based on a set of predefined risk factors for various diseases related to the nature of the data source. For example, if the data source consists of echocardiogram measurements, the AI prediction engine 517 can provide a prediction of the probability that a particular patient has, or will subsequently have, a heart-related disease such as arterial stenosis, ventricular or valve dysfunction.

[0175] The AI prediction engine 517 can also utilize an adapted implementation of the AI algorithm to generate predictions in the required operating (e.g., software) environment. For example, the implementation for an echo software package is likely to be implemented to run on Microsoft Windows®, while the implementation for an echo workstation needs to be adjusted to the vendor-specific hardware and operating system running on the workstation.

[0176] Further exemplary embodiments of the system disclosed above can be implemented, for example, as an application that enables medical experts to access through a generally accessible communication network such as an intranet network of a diagnostic clinic or the Internet. The network-accessible application can be provided as an interactive web application having a simple workflow, as depicted in the wireframe schematic representation example 600 shown in FIG. 6. In an exemplary workflow embodiment of the application 600 in use, the user inputs a set of echo measurements to the input interface 601 of the web interface 610 (where the application 600 is accessible via the web / Internet worldwide) (alternatively, connect the input to, for example, a database of echo measurements for multiple patients). The backend (not shown) of the application 600 is connected to an AI system such as the system 500 depicted in FIG. 5. The application 600 sends the input user data to the backend analysis system and returns an output consisting of predicted measurement data 620 to fill in the measurements of blank or missing data from the input data, and further outputs predicted disease risk factors 630. The predicted measurements and disease risk factors are then presented to the user via the interface 610 on the user display means 640, and this display means 640 can present the output predictions in any useful way for interpretation by the user. For example, the output can be graphically displayed for easy interpretation by the user.

[0177] A method of training and operating the AI-assisted cardiac echography system disclosed herein (e.g., methods 200, 300, 400, and 500 depicted in FIGS. 2, 3, 4, and 5 respectively) can be implemented by an exemplary computer system 700 as shown in FIG. 7 as software such as one or more application programs executable within the computer system 700. In particular, the steps of the method(s) 200, 300, 400, and 500 are realized by instructions within the software executed within the computer system 700. The instructions may be formed as one or more code modules each for performing one or more specific tasks. The software can also be divided into two separate parts, in which case the first part and the corresponding code modules execute the described method, and the second part and the corresponding code modules manage the user interface between the first part and the user. The software can be stored, for example, on a computer-readable medium including a storage device described below. The software is loaded from the computer-readable medium into the computer system 700 and executed by the computer system 700. A computer-readable medium on which such software or a computer program is recorded is a computer program product. The use of the computer program product in the computer system 700 preferably results in an advantageous apparatus for AI-assisted cardiac echography.

[0178] In the example of FIG. 7, an exemplary computer system 700 and instructions for implementing the disclosed technology in hardware, software, or a combination of hardware and software are schematically represented, for example, as boxes and circles, at the same level of detail commonly used by those of ordinary skill in the art of the technology related to this disclosure for communicating about computer architecture and the implementation of computer systems.

[0179] Exemplary computer system 700 can include, but is not limited to, one or more central processing units (CPUs) 701 having one or more processors 702, a system memory 703, and a system bus 704 that couples various system components including the system memory 703 to the processing unit 701. The system bus 704 can be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. Computer system 700 can also typically include computer-readable media which can include any available media accessible by computer system 700, including both volatile and non-volatile media, and removable and non-removable media. By way of example and not limitation, computer-readable media can include computer storage media and communication media. Computer storage media includes media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CDROM (registered trademark), digital versatile disk (DVD (registered trademark)) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other media which can be used to store the desired information and which can be accessed by computer system 700. Communication media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. By way of example and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer-readable media.

[0180] The system memory 703 includes computer storage media in the form of volatile and / or non-volatile memory such as read only memory (ROM) 705 and random access memory (RAM) 706. The basic input / output system 707 (BIOS), which contains basic routines that help transfer information between elements within the computer system 700, such as during startup, is typically stored in the ROM 705. The RAM 706 is typically immediately accessible to and / or contains data and / or program modules that are currently being operated on by the processing unit 701. By way of example and not limitation, FIG. 7 shows an operating system 708, other program modules 709, and program data 710.

[0181] Computer-readable instructions stored in the memory 703, ROM 705, RAM 706, or hard disk drive 711 may be composed of one or more sets of instructions organized as modules, methods, objects, functions, routines, or calls. The instructions may be configured as an application program including one or more computer programs, operating system services, or mobile apps. The instructions may constitute data protocol instructions or stacks for implementing one or more libraries, TCP / IP, HTTP (registered trademark), or other communication protocols to support operating systems and / or system software, multimedia, programming, or other functions, or file format processing instructions for parsing or rendering files encoded using HTML (registered trademark), XML (registered trademark), JPEG (registered trademark), MPEG (registered trademark), or PNG (registered trademark). The same applies to application software such as user interface instructions for rendering or interpreting commands of a graphical user interface (GUI), command line interface, or text user interface, office suites, Internet access applications, design and manufacturing applications, graphic applications, audio applications, software engineering applications, educational applications, games, or miscellaneous applications. The instructions may implement a web server, web application server, or web client. The instructions may be organized as a presentation layer, application layer, and data storage layer, such as a relational database system using Structured Query Language (SQL (registered trademark)), or a relational database system not using SQL, object store, graph database, flat file system, or other data storage.

[0182] In addition, computer system 700 can include other removable / non-removable, volatile / non-volatile computer storage media. By way of example only, FIG. 7 shows a hard disk drive 711 that reads from and writes to a non-removable, non-volatile magnetic medium. Other removable / non-removable, volatile / non-volatile computer storage media that can be used with the exemplary computing device include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tapes, solid state RAM, solid state ROM, and the like. Hard disk drive 711 is typically connected to system bus 704 via a non-removable memory interface such as interface 712.

[0183] The drives and associated computer storage media described above and illustrated in FIG. 7 provide storage of computer readable instructions, data structures, program modules, and other data for computer system 700. In FIG. 7, for example, hard disk drive 711 is illustrated as storing operating system 713, other program modules 714, and program data 715. Note that these components may be the same as or different from operating system 708, other program modules 709, and program data 710. Operating system 3013, other program modules 714, and program data 715 are given different reference numerals here in order to illustrate, at a minimum, that they are different copies.

[0184] The computing device also includes one or more input / output (I / O) interfaces 730 connected to the system bus 704, which include an audio / video interface coupled to an output device that includes one or more of the video display 734 and the speaker 735. The input / output interface 730 is also coupled to one or more input devices that include, for example, the mouse 731, the keyboard 732, or a touch-sensitive device 733 such as a smartphone or a tablet device. In an embodiment disclosed herein, the input interface 730 can also constitute an echo / ultrasound handpiece, and the computer system 700 can constitute an echo / ultrasound workstation or can be integrated with an echo / ultrasound workstation.

[0185] In connection with the following description, computer system 700 can operate in a network environment using logical connections to one or more remote computers. For purposes of simplicity of explanation, computer system 700 is shown in FIG. 7 as being connected to a network 720 that can include, for example, Ethernet®, Bluetooth®, or IEEE 802.X wireless protocols, although it is not limited to any particular network or network protocol. The logical connections depicted in FIG. 7 are a common network connection 721 that can be a local area network (LAN), a wide area network (WAN), or other networks such as, for example, the Internet. Computer system 700 is connected to the common network connection 721 via a network interface or adapter 722, which in turn is connected to system bus 704. In a networked environment, program modules depicted in connection with computer system 700, or portions thereof, or peripheral devices, may be stored in the memory of one or more other computing devices that are communicatively coupled to computer system 700 via the common network connection 721. The illustrated network connection is exemplary, and it will be appreciated that other means of establishing a communications link between computing devices may be used.

[0186] All analyses discussed in this specification were derived from echocardiograms performed between April 11, 2000 and June 13, 2017. They involve 530,871 echocardiograms of 331,344 adults over 18 years of age (52% male) with follow-up mortality data up to October 30, 2017 (study consensus). [Example 1 - Aortic Stenosis Diagnosis]

[0187] The following examples demonstrate the utility of the AI-assisted cardiac echo method and system disclosed above. This method is used to predict the incidence of aortic valve stenosis while completely eliminating the need for left ventricular outflow tract measurements through the use of artificial intelligence. In addition to significantly saving the scanning time of sonographers performing echocardiograms, the disclosed system and method are observed to improve echo consistency.

[0188] Comprehensive evaluation of the aortic valve is a standard part of all echocardiograms and requires measurements performed from multiple echo windows, as well as the use of two-dimensional measurements and spectral Doppler. Measurement of aortic velocity using continuous wave Doppler is accurate and reproducible, but the same measurement of the LVOT is prone to error. Errors in 2D measurements of the LVOT are amplified by multiplying or squaring the measured values as part of the continuity equation (CE). The time required to calculate the aortic valve area is approximately 7 minutes per patient.

[0189] The goal of the model is to create a comprehensive echo interpretation system using artificial intelligence to provide an efficient, rapid, and reproducible echocardiogram with accurate and reliable interpretation. In this large-scale project, it was evaluated whether AI could input the aortic valve area from other echo data with the aim of creating a system that is as accurate as conventional aortic valve area calculations, more reproducible, and faster with fewer images and measurements. [Guidelines]

[0190] For the diagnosis of severe aortic stenosis (AS) in patients, the currently clinically accepted guidelines recommend using the following three main criteria. AS jet velocity ≧ 4.0 m / s Mean aortic gradient ≧ 40 mmHg Aortic valve area ≦ 1 cm 2

[0191] Due to the complex shape of the aortic valve, instead of directly measuring its cross-sectional area, an estimated value of the effective orifice area is calculated using the following formula. LVOT area = π × (1 / 2 × VLOT diameter) 2 (3)

[0192] Here, the VLOT diameter is a circular approximation of the cross-sectional area measured by an ultrasound technician during the patient's echocardiogram used for the evaluation of the above continuity equation (CE).

[0193] Since the AS jet velocity and the AV mean gradient are flow-dependent, it is important to note that in patients with low flow, the possibility of aortic valve stenosis is not excluded even if these measured values are low.

[0194] The estimated AV area is relatively flow-independent and thus becomes an important parameter in patients with low flow. However, since it depends on the estimated VLOT area, it is not without difficulty. Citing current guidelines, "[...] the measurement variability of the VLOT diameter is 5 - 8%. When the VLOT diameter is squared for the calculation of the CSA, it becomes the most significant potential cause of measurement error in the continuity equation.

[0195] For example, in the case of a patient with a VLOT diameter of 2 cm, π*(2 / 2) 2 = π = 3.14 cm 2 , π*2.1*2.1 / 4 = 1.1π = 3.45 cm 2 and when VLOT VTI = 20 cm and AV VTI = 40 cm. The cross-sectional area of the aortic valve (AV) is given by: AV area = VLOT area * VLOT VTI / AV VTI (4) resulting in A1 = 3.14*20 / 40 = 1.57 cm 2 , a2 = 3.45*20 / 40 = 1.72 cm 2 And here, let VLOT VTI = 15 cm and AV VTI = 45 cm: A1 = 3.14 / 3 = 1.05 moderate; and A2 = 3.45 / 3 = 1.15 mild [Example 1A - AS Prediction in the General Population] Process 1. A snapshot of the NEDA database (about 650,000 patients), including patient mortality data, was obtained for a wide range of heart disease states of the types normally associated with diagnosis by echocardiogram diagnosis, and divided into a 70% training set and a 30% test set. 2. The training set was used to train an imputation model using the MDN algorithm disclosed above. 3. The training set was also used to train an AI engine that associates one or more related disease states, including aortic stenosis, with phenotypic characteristics. 4. The test set was subsampled to include only patients with a complete set of the following measurements: AS jet velocity (or AV peak velocity) AV mean gradient VLOT diameter VLOT VTI AV VTI 5. The AV area was calculated for each patient in the subsampled test set using the above formula (4), and a "ground truth" binary label labeled as "severe AS" or "not severe AS" was generated for each patient in the subsampled test set. Only when the AV area < 1 cm 2 was the patient labeled "severe AS". 6. The measurements of "VLOT diameter" and "VLOT VTI" were removed from each patient. Furthermore, the values of "VLOT mean velocity" and "VLOT peak velocity" were removed because they were highly correlated with VLOT VTI. 7. The imputation model disclosed above was used to predict the values of "VLOT diameter" and "VLOT VTI" in place of the removed values. 8. The predicted AV area was calculated for each patient using the following inputs: the predicted VLOT diameter, the predicted VLOT VT, and the measured AV VTI. 9. Patient records are associated and classified by the phenotypic characteristics of the patient's relevant measurement data. 10. To investigate the effectiveness of the predicted AV area as a predictor of severe AS at different cut-off values, ROC (Receiver Operating Characteristic) plots and Precision Recall plots were created. The "ground truth" labels from Step 4 were used. 11. Next, the classified phenotypic characteristics of the patient are associated with one or more disease states to predict the likelihood of the patient's disease state.

[0196] The measured values included in the imputation model are as follows. AR peak velocity; AR pressure half-time; AV mean gradient; AV mean velocity; AV peak velocity; AV velocity time integral; aortic root diameter; aortic root diameter; aortic root diameter; aortic root diameter; aortic root diameter; aortic root diameter; aortic root diameter; aortic root diameter; aortic root diameter; aortic root diameter; aortic root diameter; aortic root diameter; aortic root diameter; aortic base diameter; aortic base diameter MM; ascending aorta diameter; body weight; diastolic gradient; distal transverse aorta diameter; EFtext; heart rate; IVC diameter at completion; IVS diastolic thickness; IVS diastolic thickness MM; IVS systolic thickness MM; LA area in 4C view; LA length in 4C; LA systolic diameter LX; LA systolic diameter MM; LA systolic diameter transverse; LV diameter; LV diastolic area in 2C; LV diastolic area in 4C; LV diastolic area in PSAX; LV diastolic diameter in 4C; LV diastolic diameter MM; LV diastolic diameter in PLAX; LV diastolic length in 2C; LV diastolic length in 4C; LV E' lateral velocity; LV diastolic area in PSAX; LV mass (C)D; LV relative wall thickness; LV systolic area in 2C; LV systolic area in 4C; LV systolic diameter in 4C; LV systolic diameter base LX; LV systolic diameter MM; LV systolic diameter in PLAX; LV systolic length in 2C; LV systolic length in 4C; VLOT diameter; VLOT mean velocity; VLOT peak velocity; VLOT velocity time integral; LVPW diastolic thickness; LVPW diastolic thickness MM; LVPW systolic thickness MM; MV A' velocity; MV deceleration time; MV E' velocity; MV mean velocity; MV peak velocity; MV pressure half-time; MV velocity time integral; mitral valve A point velocity; mitral valve E point velocity; PV peak velocity; PV velocity time integral; patient age; pulmonary vein atrial duration; pulmonary vein atrial reversal velocity; pulmonary vein A velocity; pulmonary vein diastolic velocity; pulmonary vein systolic velocity; RA area; RA systolic diameter LX; RA systolic diameter transverse; RA pressure text; RVOT peak velocity; right atrial pressure; TR peak velocity; thoracic aorta diameter. [Results]

[0197] Figure 8 shows the Receiver Operating Characteristic and Precision Recall curve areas for this example. The total number of patients in the sample was 24,748, and the diagnosis was made based on the following measurements: AV area (VTI), holdout: [VLOT mean velocity, VLOT peak velocity, VLOT velocity time integral, VLOT diameter]. The number of patients predicted to have severe AS was 1,834 (7.410700%), and the number of patients predicted not to have severe AS was 22,914 (92.589300%). [AS prediction in patients with reduced ejection fraction in Example 1B]

[0198] Since there are many patients with normal LV systolic function, the results of the general population may be affected. Also, since the performance when the LV function is reduced may be masked, the above experiment was repeated for patients with an EF of less than 50% and less than 30% as described below. [Process] 1. In step 3, follow all the instructions of Experiment 1 except to create a subsample that includes only patients with an ejection fraction of less than 50%. 2. Repeat step 1 for ejection fractions of 40%, 35%, and 30%. [Results]

[0199] Figure 9A shows the results of the model using values of ejection fraction ≤ 50%, and the diagnosis is made based on the following. Exclude AV area (VTI), holdout: [VLOT mean velocity, VLOT peak velocity, VLOT velocity time integral, VLOT diameter], and exclude patients with an EF exceeding 50.000000. The total number of patients in the sample was 1,391. The number of patients predicted to have severe AS was 143 (10.280374%), and the number of patients predicted not to have severe AS was 1,248 (89.719626%).

[0200] Figure 9B shows the results of the model diagnosed based on the following using an ejection fraction value ≤ 40%. AV area (VTI), holdout: diagnosed based on [VLOT average velocity, VLOT peak velocity, VLOT velocity time integral, VLOT diameter], excluding patients with an EF exceeding 40.000000. The total number of patients in the sample was 861. The number of patients predicted to have severe AS was 96 (11.149826%), and the number of patients predicted not to have severe AS was 765 (88.850174%).

[0201] Figure 9C shows the results of the model diagnosed based on the following using an ejection fraction value ≤ 35%. AV area (VTI), holdout: diagnosed based on [VLOT average velocity, VLOT peak velocity, VLOT velocity time integral, VLOT diameter], excluding patients with an EF exceeding 35.000000. The number of patients predicted to have severe AS was 58 (11.026616%), and the number of patients predicted not to have severe AS was 468 (88.973384%).

[0202] Figure 9D shows the results of the model diagnosed based on the following using an ejection fraction value ≤ 30%. AV area (VTI), holdout: diagnosed based on [VLOT average velocity, VLOT peak velocity, VLOT velocity time integral, VLOT diameter), excluding patients with an EF exceeding 30.000000. The total number of patients in the sample was 426. The number of patients predicted to have severe AS was 45 (10.563380%), and the number of patients predicted not to have severe AS was 381 (89.436620%). [Summary of Example 1 - Results]

[0203] The following Table 3 summarizes the results of the above Example 1A and 1B. Table 3 Summary of Results - Prediction of Aortic Valve Stenosis [Table 3] [Example 1 - Discussion]

[0204] Average precision is generally a more useful metric than AUC for evaluating the performance of binary classifiers on imbalanced data. With this in mind, the results are interpreted as follows.

[0205] From Table 3, it can be seen that severe AS can be predicted in the general population with an AUC = 0.96 and an average precision of 73% without measuring the VLOT diameter or VLOT VTI. In the group of patients with an ejection fraction of less than 50%, the average precision improved to 80%, and similar results were obtained in other patient groups with a decreased ejection fraction. This is particularly important considering that in these subpopulations, the imputation model cannot necessarily rely solely on the simple association with an increase in AV mean gradient / AV peak velocity. Future studies are planned to investigate which variables are the main factors contributing to the surprising performance of the imputation model in this setting.

[0206] There are several important points to note. · The sample size of the reduced EF subpopulation is an order of magnitude smaller than that of the general population. Consequently, the resulting AUC and average precision statistics are associated with more uncertainty. However, a sample size of 1391 patients is still reasonably large. · The prevalence of severe AS is higher in the subpopulation with a decreased EF. Since the precision-recall is not relatively sensitive to changes in class distribution, the average precision should be meaningfully comparable. However, further investigation is needed to determine the exact reasons for the improved performance in this subset. For example, it is possible that the "decision boundary" of this subset is more prominent due to more advanced disease progression.

[0207] The assessment of AS severity in severe LV dysfunction is well known to be difficult due to low flow and the resulting low gradients. In patients with an EF < 50%, the ROC area was 0.96 and the PR area was 0.80 (in a study of 1391, 10% had severe AS). In patients with an EF < 30%, the ROC area was 0.85 and the PR area was 0.77 (in a study of 426, 10% had severe AS).

[0208] From this, it was confirmed that our model can identify severe AS not only when LV function is normal but also when LV function is reduced. [Example 1 - Comparison of ROC Curve and Precision Recall Curve in Imbalanced Data]

[0209] The Receiver Operating Characteristic (ROC) curve plots the True Positive Rate (TPR) against the False Positive Rate (FPR). The Precision Recall curve plots precision against recall. The relevant equations are as follows. TPR = Recall = TP / (TP + FN) FPR = FP / (FP + TN) Precision = TP / (TP + FP) Here, TP = True Positive FP = False Positive TN = True Negative FN = False Negative That is.

[0210] In the setting of the above example, the measurement criteria can be described as follows. · Recall / TPR is the proportion of severe AS patients correctly identified by the system. · FPR is the proportion of patients without severe AS misdiagnosed as having severe AS by the system. · Precision is the proportion of positive diagnoses of severe AS generated by the system that are correct.

[0211] Therefore, Precision Recall is a better metric for imbalanced test sets for the following reasons: · The FPR metric is a relative value with respect to the number of patients without severe AS (the majority of patients in this setting). Therefore, even if the FPR appears low, there are actually many false positive diagnoses. · The precision metric is a relative value with respect to the number of positive diagnoses generated by the algorithm. This allows for a better understanding of how the system is perceived in actual use. For example, 80% precision means that when the system diagnoses a patient with severe AS, there is an 80% probability of being correct. [Example 2 - Severe aortic valve stenosis]

[0212] Taking into account the size and scope of the available data, a formal analysis of statistical power was not performed. After training the AI system, two AS severity predictions (complete echocardiogram and limited echocardiogram) from 30% of the test dataset, along with the identified copy of the NEDA database, were imported into Statistical Package for Social Sciences (SPSS) version 25.0 (IBM) for comparison with the aortic valve area derived from the continuity equation. Statistical significance was set at a P-value < 0.05. For both the complete echocardiogram examination and the limited echocardiogram examination, a probability cut-off for severe AS prediction using AI was selected and optimized to have approximately equal sensitivity and specificity and used for the remaining analysis. Discrete variables were presented as numbers and percentages. Intergroup comparisons were evaluated using Student's t-test, Mann Whitney U-test (for continuous data not normally distributed), and chi-square test (for categorical data).

[0213] Using big data from the Australian National Echocardiogram Database, a modified mixture density network AI system was created. 530,000 echocardiograms related to 331,344 adult individuals with follow-up mortality data were randomly split into 363,887 (70%) used for AI training and 155,967 (30%) used to validate the performance of the trained model. Two models were trained to predict severe AS without the need for data on the left ventricular outflow tract, one using the remaining measurements of the echocardiogram examination and the other using a limited dataset that was thought to vary with aortic valve stenosis.

[0214] The performance of the AI system was evaluated using the receiver operating characteristic (ROC) curve in the prediction of severe AS by AI for the aortic valve area calculated using the continuity equation. The area under the ROC curve (AUC) was calculated for all patients and for patients with reduced left ventricular function. The last echocardiogram of all patients in whom all lethal events were identified and follow-up was completed was used to calculate the mathematical 5-year mortality survival curve. The Cox proportional hazard ratio adjusted for age and sex was calculated and further adjusted for aortic valve area and mean aortic gradient. To clarify the potential differences in survival rates between patients diagnosed with severe AS by AI and those diagnosed with severe AS by calculation of the aortic valve area, the mathematical 5-year survival curve was examined. The phenotypic diagnosis of severe AS was compared with severe AS derived from the continuity equation using the AI system.

[0215] To evaluate whether this result was mainly for patients with classical high-gradient, normal ejection fraction severe AS, three patient groups (the full system and the limited dataset) were compared. 1) All patients in the test set, 2) patients with an ejection fraction of less than 50%, 3) patients with an ejection fraction of less than 30%. Patients without all measurements available for the calculation of the continuity equation were excluded from the statistical analysis (n = 361,067 patients without a valid left ventricular outflow tract measurement). The prediction of severe AS by AI used all available data (excluding left ventricular outflow tract data), and missing measurements were ignored by the AI. In the limited dataset, only the above measurements were displayed to the AI (only if measured). The aortic valve area was estimated by the AI system based on the remaining available echocardiogram measurements, and then the probability of severe AS was calculated as described above.

[0216] In the test set, 2,382 out of 32,574 patients (7.38%) had severe aortic valve stenosis, and the AI system predicted with an area under the receiver operating characteristic curve (AUC) of 0.97. The AUC was 0.95 for patients with a left ventricular ejection fraction of less than 50% and 0.92 for patients with an ejection fraction of less than 30%. The performance of the AI was maintained using the limited dataset (AUC 0.97, 0.94, 0.93 respectively).

[0217] Figure 13 shows the analysis flow 900 performed on data obtained from a cohort of 1,715 men and 1,584 women (age 61.5 ± 17.6 years), with a median follow-up period of 4.1 years (IQR 2.2 to 7.1 years). There were no differences in baseline characteristics between the 70% test set and the 30% training set, nor among those with a complete dataset available for calculating aortic valve area using the continuity equation (Table 4). Among the 30% test set, 2,382 / 32,574 people (7.38%, 95% CI 7.10 to 7.67%) in whom the aortic valve area was determined from the continuity equation had severe AS. With an output probability cutoff of 0.065425, the sensitivity and specificity of AI-diagnosed severe AS were 91.4% when compared with the calculated aortic valve area. [Degree of agreement of severe AS obtained from AI and continuity equation]

[0218] The ROC curve 951 for AI diagnosis of severe AS was 0.9696 (see Figure 14) compared with diagnosis by the continuity equation, and the positive likelihood ratio was 45.9%. This model showed results almost equivalent to AUC 0.9462 (2,308 cases, 11% severe AS) in patients 953 with an ejection fraction of less than 50% (Figure 14). Also, in patients with an ejection rate of less than 30% (491 cases, 13% with low-gradient, low-output severe AS), AI showed an AUC of 0.9200, indicating very good results (Figure 14).

[0219] In the limited echo cohort, the prediction of severe AS by AI was robust. When applied to the test dataset, the AUC of the ROC curve 952 was 0.9648 (Figure 14). Consistent with the comprehensive dataset, AI showed almost similar results in patients with an ejection rate of less than 50% 954 and less than 30% 956 (AUCs were 0.9450 and 0.9269 respectively - Figure 14). [Long-term survival rate of severe AS diagnosed by AI and continuity equation]

[0220] The mathematical 5-year mean survival period (mean ± standard error) for non-severe AS diagnosis by AI was 1536.0 ± 8.8 days, while for severe AS it was 1072.5 ± 23.3 days, with p < 0.00001, showing a difference in mean survival period of 463.5 days (Figure 15). Patients were matched in a 30% test cohort. The upper line 961 in panels (A) and (B) indicates the number of people at risk of not having severe aortic valve stenosis at each time point. The lower line 963 in each panel of (A) and (B) represents the number of people at risk of being diagnosed with severe aortic valve stenosis at each period. In the continuous group of severe AS, the mean survival period was 1489.0 ± 8.9 days, while it was 1086.0 ± 31.6 days, showing a difference of 403 days. The non-severe AS group identified by AI lived 47 days longer (95% confidence interval 23.3 - 71.7, p = 0.02) than the non-severe AS group identified by the continuous equation. Also, in the group identified as severe AS by AI, the lifespan was similar to that of the group identified as severe AS by the continuous equation (13.5 days, 95% CI 90.4 - 63.4 days, p = 0.7). The AI diagnosis maintained a high predictability of death even after adjustment for age and gender (adjusted HR = 1.37, 95% CI 1.22 - 1.54, p < 0.0001), while the continuous equation diagnosis remained weakly predictive even after adjustment for age and gender (HR = 1.19, 95% CI 1.04 - 1.37, p = 0.025). After adjustment, AI continued to predict future risk of death (HR = 1.34, 95% CI = 1.13 - 1.59, p = 0.001), consistent with an approach using multiple parameters for AI diagnosis in addition to the transaortic valve gradient. Even after adjusting for clinical outcomes such as aortic valve replacement, there was no significant change in the predicted values of the above AI or continuous equation (CE). [Example 2 - Discussion]

[0221] As demonstrated in Example 2 above, AI can strongly reinforce the diagnosis of severe AS by interpreting the entire echocardiogram phenotype without relying on measurements of the left ventricular outflow tract in a very large cohort that has undergone long-term follow-up. Furthermore, consistent with its multi-parameter approach to interpretation, AI-enhanced diagnosis of severe AS remained a significant predictor of long-term mortality even after adjustment for conventional AS severity metrics. Therefore, the purpose-specific AI systems disclosed herein also introduce the first potential quality system for echocardiography by providing automated measurements and disease prediction in real time. These disclosed systems can provide known statistical results for a defined set of measurements. Importantly, a fully trained AI system requires minimal computational power to operate and can be installed on both echocardiography devices and image-reading software to improve diagnostic consistency in the absence of expert review.

[0222] If its effectiveness and reliability are proven, AI will be an extremely useful clinical tool. Currently, the complex interactions seen in AS require evaluation by echocardiographers without specialized training. Furthermore, even after expert evaluation, underdiagnosis of severe AS can occur in some patients. 165 High-quality guidelines for the diagnosis of AS 1、3 are well established, but their strict application may not be routinely performed 16、18 and errors may not be identified. The AI evaluated in this study consistently examines the entire echocardiogram phenotype, taking into account known pathophysiological changes such as left ventricular diastolic and systolic dysfunction, left atrial enlargement, and pulmonary hypertension. 23 Known pathophysiological changes 19-22 such as 24-26which may lead to misclassification of AS severity and affect the follow-up of echocardiography and the timing of intervention. In contrast to current clinical practice, AI is consistent and completely eliminates the need to measure the dimensions and velocities of the left ventricular outflow tract, which is relevant to both the consistency and timing of AS diagnosis and follow-up 16 。

[0223] Outside of guidelines, there are no generally accepted quality metrics for clinical echocardiography27. The rigor applied to echocardiogram interpretation in clinical trials28,29 may not be consistently applied in real-world settings and can affect outcomes and intervention decisions30,31. Quality improvement programs reduce error rates17, but automation and real-time feedback are required for universal application6. AI is ideally suited to this task because its analysis is consistent and phenotype-based. Importantly, the AI systems disclosed herein function similarly whether on a comprehensive echocardiogram or a limited dataset that can be acquired in just 10 minutes. When applied to specific scenarios (such as follow-up echocardiograms for known AS), it affects efficiency, consistency, and cost, but further evaluation is required

[0224] The AI systems disclosed herein are trained on data from NEDA, a very large echocardiography database linked to mortality. Due to the nature and scope of NEDA, quality control for individual images is not feasible, and the data obtained from individual laboratories are assumed to be correct. However, since NEDA is not a single source of information and is provided by various tertiary hospital laboratories across Australia, systematic bias is unlikely. The AI accurately identified severe AS, but this model needs to be validated across the full range of aortic valve diseases. The replacement error for aortic valve area was small (see Table 4), there was no systematic bias, and the mean replacement error (95% CI) was 0.056 (-0.885, 0.664) cm 2It was the case. As a realistic nature of NEDA data, there were missing data points, and a complete set of echocardiogram measurements was not obtained for all patients. Some AS patients may not have obtained sufficient measurements for inclusion in the AI evaluation, and testing of the AI system in clinical and reference echocardiography laboratories is necessary.

[0225] AI can identify patients who do not have severe AS but have similar changes in cardiac phenotypes. However, these patients follow the same mortality trajectories as conventional severe AS patients, highlighting the ability of AI to identify high-risk patients. Australia is a multicultural country that widely represents the world's population, with over 300 different ancestries and 28% of the resident population born overseas. Also, the clinical linkages related to the dataset used could not be included in the above system validation examples. It may also be necessary to consider an individual's clinical background regarding factors that may contribute to phenotypic changes, such as hypertension and valvular disease.

[0226] In summary, a prototype of an AI echocardiogram measurement interpretation system has been developed to reinforce the diagnosis of severe AS by evaluating the entire phenotype. This process completely eliminates the need for left ventricular outflow tract measurements. This AI functions similarly whether the systolic function is normal or reduced, including in cases of severe left ventricular dysfunction, and accurately predicts future mortality risk regardless of the AS gradient. The system disclosed in this specification also provides real-time feedback during echocardiography, bringing advantages to efficiency, consistency, and follow-up recommendations. It also has an important impact on the examination period, cost, and the risk of injury to echocardiographers. Overall, to consistently diagnose the severity of AS, a shift from the current completely manual interpretation to a more automated and objective process is necessary. The role of AI in these goals requires rigorous clinical evaluation in different patient groups and different diseases identified by echocardiography.

[0227] The above verification examples strongly indicate that by analyzing the entire echocardiogram phenotype and without the need for left ventricular outflow tract measurements, AI can enhance and improve the diagnosis of severe AS and the prediction of related risk mortality. In particular, the AI system disclosed herein was able to predict patients at high risk of death due to AS, independent of the aortic valve gradient. The decision for intervention in severe aortic stenosis (AS) depends on the reliable interpretation of echocardiograms. The artificial intelligence (AI) system disclosed in this specification is specifically designed to enhance the diagnosis of severe AS using echocardiogram measurement data, and as a result, provides important input for the development of effective patient care outcomes by medical experts. [Interpretation] [Bus]

[0228] In the context of this specification, the term "bus" and its derivatives broadly refer to any system for data communication. However, in a preferred embodiment, it may refer to parallel connections including bus subsystems for interconnecting various devices, such as Industry Standard Architecture (ISA (registered trademark)), conventional Peripheral Component Interconnect (PCI (registered trademark)), etc., or serial connections such as PCI Express (PCIe (registered trademark)), Serial Advanced Technology Attachment (Serial ATA (registered trademark)). [According to]

[0229] In this specification, "according to" may also mean "as a function of" and is not limited to the specific integers involved. [Component]

[0230] In this specification, "computer-executed method" does not imply being executed on a single computer device. That is, the steps of such a method may be jointly executed by multiple computer devices.

[0231] Similarly, when the terms "Web server", "server", "client computer device", "computer-readable medium" are used in this specification, they are not limited to a single object and may be executed as multiple objects. For example, a Web server may be a group of multiple Web servers in a server farm jointly achieving one purpose, and a computer-readable medium may be distributed in multiple forms. For example, the program code in a compact disc activated by a license key may be downloadable from a computer network. [Database]

[0232] In the context of this specification, the term "database" and its derivatives are used to represent a single database, a set of databases, a database system, etc. A database system can consist of a set of databases, and the set of databases can be stored on a single implementation or span multiple implementations. Also, the term "database" is not limited to referring to a specific database format, but rather may refer to any database format. For example, database formats include MySQL (registered trademark), MySQLi (registered trademark), XML (registered trademark), etc. [Wireless]

[0233] The present invention can be embodied for other applications including other WLAN standards and other wireless standards, using devices compliant with other network standards. Applicable applications include IEEE802.11 wireless LANs and links, and wireless Ethernet.

[0234] In the context of this specification, the term "wireless" and its derivatives may be used to describe circuits, devices, systems, methods, techniques, communication channels, etc. that can communicate data through the use of modulated electromagnetic radiation via a non-solid medium. This term does not mean, in some embodiments it may not be so, that the associated devices do not include wires. In the context of this specification, the term "wired" and its derivatives may be used to describe circuits, devices, systems, methods, techniques, communication channels, etc. that can communicate data through the use of modulated electromagnetic radiation via a solid medium. This term does not mean that the associated devices are coupled by conductive wires. [Process]

[0235] Unless otherwise specified, throughout this specification, terms such as "process", "computing", "calculating", "determining", "analyzing", etc. refer to the execution of and / or processes by a computer, computer system, or other electrical computing device. This involves processing data represented as a physical quantity (e.g., electricity) and / or converting it into data similarly represented as a physical quantity. [Processor]

[0236] Similarly, the term "processor" refers to any device or part of a device that processes electrical data. Such processing includes, for example, converting data from registers and / or memory into other electrical data and storing it in registers and / or memory. "Computer", "computer device", "calculator", "computing platform" may include one or more processors.

[0237] The method herein can be executed by one or more processors that, in one embodiment, accept computer-readable (or also referred to as machine-readable) code that includes an instruction set that, when executed by one or more processors, performs at least one method according to this specification. Any processor capable of executing the instruction set (sequentially or otherwise) that can identify the actions to be taken is included. Thus, by way of example, a typical processing system including a processor is also included by way of example. The processing system may further include a memory subsystem including main RAM, static RAM, and / or ROM. [Computer-readable medium]

[0238] Furthermore, the computer-readable medium may form a computer program product or may be included in a computer program product. The computer program product may be stored on a computer-usable medium. A computer program product with a computer-readable medium may activate a processor to perform the method herein. [Network or multiprocessor]

[0239] In an alternative embodiment, one or more processors operate as a stand-alone device. Alternatively, one or more processors may be connected within a network arrangement, for example, to form a network with other processors. One or more processors may operate within the scope of a server or client machine in a server-client system environment, or may operate as a peer machine in a peer-to-peer or distributed network environment. One or more processors may be any machine capable of executing an instruction set (sequentially or otherwise) that can identify the actions to be taken, such as a web appliance, a network router, a switch or bridge, or any other.

[0240] Some of the drawings show only a single processor or a single memory that executes computer-readable code, but may include more than one of the above components (although they may not be explicitly shown so as not to obscure the inventive aspects). For example, even when only a single machine is shown, the term "machine" may refer to a set of machines (which, individually or jointly, execute an instruction set to implement any one or more of the methods herein). [Additional Embodiments]

[0241] Some embodiments of each method herein take the form of a computer-readable medium storing an instruction set (e.g., a computer program executed by one or more processors). Thus, embodiments of the present invention may be a method, an apparatus for a particular purpose, an apparatus with a data processing system, a computer-readable medium, etc. The computer-readable medium stores computer-readable code (which includes an instruction set that, when executed on one or more computers, causes a processor to execute the method). Thus, aspects of the present invention may take the form of a method, hardware in its entirety, software in its entirety, or a combination of hardware and software. Further, the present invention may take the form of a storage medium (e.g., a computer program product stored on a computer-readable medium). [Implementation]

[0242] In some embodiments, the steps of the above method are executed by a suitable processor of a processing system (i.e., a computer) that executes instructions (computer-executable code) stored in storage. The present invention is not limited to a particular implementation or programming technique and may be executed using any appropriate technique for performing the above functions. The present invention is not limited to a particular programming language or operating system. [Means for Realizing a Method or Function]

[0243] Some further embodiments are described as a processor, a processor device, a computer system or any subject capable of implementing the present function, or a combination of method elements of a method. Thus, a processor with the instructions necessary to execute such a method (or method element) is a means for executing the method (or method element). Further, the elements of the apparatus in this specification are examples of means for realizing the functions of the elements for realizing the present invention. [Connected]

[0244] Similarly, when the term "connected" is used in the claims, it is not construed as being limited to only indicating a connection. Thus, the scope of the expression "device A connected to device B" is not limited to an apparatus or system in which the output of device A is directly connected to the input of device B. This expression means that there is some path between the output of A and the output of B. This path may include other devices or means. "Connected" may mean that two or more elements are "physically" or "electrically" in direct contact. Further, "connected" may mean that two or more elements do not directly contact each other but cooperate or interact with each other. [Embodiment]

[0245] Throughout this specification, when terms such as "embodiment", "an embodiment", "configuration", "a configuration" are used, it means that the specific features, structures or characteristics of the embodiment / configuration described in the specification are included in at least one embodiment / configuration of the present invention. Thus, when "in an embodiment" is used in this specification, it does not necessarily mean all of the same embodiment / configuration. Further, in one or more embodiments / configurations, as will be apparent to those skilled in the art, certain specific features, structures or characteristics may be combined in any suitable manner.

[0246] Similarly, in the above embodiments of the present invention, in one embodiment / configuration, drawing, or detailed description, various features of the present invention are often grouped. This is for the purpose of making this disclosure reasonable and assisting in the understanding of one or more various inventive aspects. However, this method of disclosure is not intended for the invention recited in the claims to claim features beyond what is explicitly written in the claims. Rather, the invention recited in the following claims may sometimes have only a part of all the features of the embodiment / configuration described in the above detailed description. Accordingly, the following claims are a part of the detailed description, and the claims themselves are an independent embodiment / configuration of the present invention.

[0247] Some of the embodiments / configurations described in this specification non - limitatively include features included in other embodiments / configurations, different embodiments / configurations. On the other hand, combinations of different embodiments / configurations are also included in the scope of the present invention and form different embodiments / configurations. This is as understood by those skilled in the art. For example, any of the embodiments / configurations in the following claims can be used in combination. [Specific details]

[0248] Some specific details are described in this specification. However, the present invention can be implemented without these specific details. For known methods, structures, and technologies, in order not to obscure the understanding of the present invention, they may not be described in detail. [Terms]

[0249] For clarity, specific terms may be used with respect to the illustrated preferred embodiments of the present invention. However, the present invention is not limited to these selected specific terms. Each of these specific terms includes equivalent technical content used in the same way to solve similar technical problems. Terms such as "forward", "backward", "radially", "circumferentially", "upward", and "downward" are used as convenient words to provide a reference point but are not used to limit the invention. [Different examples of the object]

[0250] Unless otherwise specified, terms such as "first", "second", and "third" used herein with respect to ordinary objects simply mean different examples of similar objects. These terms do not refer to a predetermined order with respect to time, space, hierarchy, etc. [comprising and including]

[0251] In the following claims and the preceding description of the invention, unless the context clearly requires otherwise, or is otherwise required by explicit statement or necessary implication, the words "comprise", "comprises", "comprising", and the like are used in an inclusive sense, that is, they are used to specify the presence of the stated features but not to preclude the presence or addition of further features in various embodiments of the invention.

[0252] Any one of the terms: "including" or "which includes" or "that includes" as used herein also means "including at least" the element / feature that follows the term, but is an open term that does not exclude others. Thus, "comprising" is synonymous with "including".

[0253] [Scope of the Invention] Although the preferred embodiments of the present invention have been described, it is understood by those skilled in the art that further modifications can be made without departing from the spirit of the present invention. Such modifications and variations are also included within the scope of the present invention. Some functions may be omitted or added to the illustrated functional blocks, and the operations between the functional blocks may be exchanged. Within the scope of the present invention, some steps may be added or omitted from the methods described herein.

[0254] The present invention has been described with reference to specific examples, but the present invention can take many other forms. [Industrial Applicability]

[0255] It is clear from the above description that the above embodiments are applicable to the mobile device industry, particularly to methods and systems for providing digital media using mobile devices.

[0256] The systems and methods for AI-assisted cardiac echo examination described and / or illustrated herein substantially provide an eye examination application using a mobile device, particularly an application for self-determining an eyeglass prescription using a mobile computer device.

[0257] The systems and methods for AI-assisted cardiac echo examination described and / or illustrated herein are shown by way of example only and do not limit the scope of the present invention. Unless otherwise specified, the individual aspects and components of the systems and methods herein may be modified, may be replaced with known equivalent components, or may be replaced with unknown components to be developed in the future. With respect to various applications, the systems and methods described herein may be modified within the scope and spirit of the present invention. This is because the scope of potential applications is wide and the systems and scope of the present invention are applicable in various fields. [References] 1. Baumgartner H, Falk V, Bax JJ, et al. 2017 ESC / EACTS Guidelines for the management of valvular heart disease. European Heart Journal 2017;38:2739 91. 2. Badiani S, van Zalen J, Treibel TA, Bhattacharyya S, Moon JC, Lloyd G. Aortic Stenosis, a Left Ventricular Disease: Insights from Advanced Imaging. Current Cardiology Reports 2016;18:80. 3. Nishimura RA, Otto CM, Bonow RO, et al. 2014 AHA / ACC Guideline for the Management of Patients With Valvular Heart Disease: A Report of the American College of Cardiology / American Heart Association Task Force on Practice Guidelines. Journal of the American College of Cardiology 2014;63:e57 e185. 4. Mo Y, Penicka M, Di Gioia G, et al. Resolving Apparent Inconsistencies Between Area, Flow, and Gradient Measurements in Patients With Aortic Valve Stenosis and Preserved Left Ventricular Ejection Fraction. The American Journal Of Cardiology 2018;121:751 7. 5. Bennet CS, Abeya FC, Hoffman A, et al. Performance and Interpretation Training of Transthoracic Echocardiography in Resource Limited Settings. Journal Of The American College Of Cardiology 2017;70:1940 1. 6. Pellikka PA, Douglas PS, Miller JG, et al. American Society of Echocardiography Cardiovascular Technology and Research Summit: a roadmap for 2020. Journal Of The American Society Of Echocardiography: Official Publication Of The American Society Of Echocardiography 2013;26:325 38. 7. Gandhi S, Mosleh W, Shen J, Chow C M. Automation, machine learning, and artificial intelligence in echocardiography: A brave new world. Echocardiography (Mount Kisco, NY) 2018. 8. Narula S, Shameer K, Salem Omar AM, Dudley JT, Sengupta PP. Machine Learning Algorithms to Automate Morphological and Functional Assessments in 2D Echocardiography. Journal Of The American College Of Cardiology 2016;68:2287 95. 9. Zhang J, Gajjala S, Agrawal P, et al. Fully Automated Echocardiogram Interpretation in Clinical Practice. Circulation 2018;138:1623 35. 10. Bishop C. Mixture density networks. Technical Report. Birmingham, UK: Aston University; 1994. 11. Srivastava NH, G; Krizhevsky, A; Sutskever, I; Salakhutdinov, R. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research 2014;15:1929 58. 12. Grimit EP, Gneiting T, Berrocal VJ, Johnson NA. The continuous ranked probability score for circular variables and its application to mesoscale forecast ensemble verification. Quarterly Journal of the Royal Meteorological Society 2007;132:2925 42. 13. D’Isanto A. Uncertain Photometric Redshifts with Deep Learning Methods. Proceedings of the International Astronomical Union 2016;12:209 12. 14. Baumgartner H, Hung J, Bermejo J, et al. Recommendations on the Echocardiographic Assessment of Aortic Valve Stenosis: A Focused Update from the European Association of Cardiovascular Imaging and the American Society of Echocardiography. Journal of the American Society of Echocardiography 2017;30:372 92. 15. Malouf J, Le Tourneau T, Pellikka P, et al. Aortic valve stenosis in community medical practice: determinants of outcome and implications for aortic valve replacement. The Journal Of Thoracic And Cardiovascular Surgery 2012;144:1421 7. 16. Chan RH, Shaw JL, Hauser TH, Markson LJ, Manning WJ. Guideline Adherence for Echocardiographic Follow Up in Outpatients with at Least Moderate Valvular Disease. Journal Of The American Society Of Echocardiography: Official Publication Of The American Society Of Echocardiography 2015;28:795 801. 17. Fanari Z, Choudhry UI, Reddy VK, et al. The Value of Quality Improvement Process in the Detection and Correction of Common Errors in Echocardiographic Hemodynamic Parameters in a Busy Echocardiography Laboratory. Echocardiography (Mount Kisco, NY) 2015;32:1778 89. 18. Nagueh SF, Farrell MB, Bremer ML, Dunsiger SI, Gorman BL, Tilkemeier PL. Predictors of Delayed Accreditation of Echocardiography Laboratories: An Analysis of the Intersocietal Accreditation Commission Database. Journal Of The American Society Of Echocardiography: Official Publication Of The American Society Of Echocardiography 2015;28:1062 9.e7. 19. Delaye J, Chevalier P, Delahaye F, Didier B. Valvular aortic stenosis and coronary atherosclerosis: pathophysiology and clinical consequences. European Heart Journal 1988;9 Suppl E:83 6. 20. Pibarot P, Dumesnil JG. New concepts in valvular hemodynamics: implications for diagnosis and treatment of aortic stenosis. The Canadian Journal Of Cardiology 2007;23 Suppl B:40B 7B. 21. Dweck MR, Boon NA, Newby DE. Calcific aortic stenosis: a disease of the valve and the myocardium. Journal Of The American College Of Cardiology 2012;60:1854 63. 22. Mutlak D, Aronson D, Carasso S, Lessick J, Reisner SA, Agmon Y. Frequency, determinants and outcome of pulmonary hypertension in patients with aortic valve stenosis. The American Journal Of The Medical Sciences 2012;343:397 401. 23. Bartel T, Muller S. Preserved ejection fraction can accompany low gradient severe aortic stenosis: impact of pathophysiology on diagnostic imaging. European Heart Journal 2013;34:1862 3. 24. Michelena HI, Margaryan E, Miller FA, et al. Inconsistent echocardiographic grading of aortic stenosis: is the left ventricular outflow tract important? Heart (British Cardiac Society) 2013;99:921 31. 25. Oh JK, Kane GC. The Echo Manual: Lippincott Williams & Wilkins, Lippincott Williams & Wilkins, Attn: Sharon Kimmel, 14700 Citicorp Dr, Bldg 3, Hagerstown, MD, 21742; 2018. 26. Otto CM, Otto CM. The practice of clinical echocardiography: Elsevier Science Health Science, Elsevier Science Health Science, Attn Customer Service, 11830 Westline Industrial Dr, Saint Louis, MO, 63146; 2016. 27. Crowley AL, Yow E, Barnhart HX, et al. Critical Review of Current Approaches for Echocardiographic Reproducibility and Reliability Assessment in Clinical Research. Journal of the American Society of Echocardiography 2016;29:1144 54.e7. 28. Douglas PS, DeCara JM, Devereux RB, et al. Echocardiographic Imaging in Clinical Trials: American Society of Echocardiography Standards for Echocardiography Core Laboratories. Journal of the American Society of Echocardiography 2009;22:755 65. 29. Khouri MG, Ky B, Dunn G, et al. Echocardiography Core Laboratory Reproducibility of Cardiac Safety Assessments in Cardio Oncology. Journal Of The American Society Of Echocardiography: Official Publication Of The American Society Of Echocardiography 2018;31:361 71.e3. 30. Khouri MG, Ky B, Dunn G, et al. Echocardiography Core Laboratory Reproducibility of Cardiac Safety Assessments in Cardio Oncology. Journal of the American Society of Echocardiography 2018;31:361 71.e3. 31. Rigolin VH. Quality Matters. Journal of the American Society of Echocardiography 2017;30:A17 A8.

Claims

1. A method for processing a sparse data source, comprising: (a) retrieving data from the sparse data source to form a basic data set, wherein the sparse data source includes a plurality of patient records including patient mortality data, and each of the patient records includes at least one unpopulated data field corresponding to a medical measurement; (b) dividing the basic data set into the following two parts: a first part including a training data set of a predefined percentage X% of the basic data set; a second part including a validation data set of a predefined percentage (100% - X%) of the basic data set; (c) analyzing the training data set using a non-linear function approximation algorithm repeatedly applied to the records of the training data set to jointly model variable relationships for the purpose of obtaining a trained model and a measurement prediction protocol for populating the unpopulated data fields within the training data set; (d) calculating predicted values of measurement data for the unpopulated data fields using the measurement prediction protocol; (e) populating the predicted values into the records within the training data set; (f) analyzing the training data set based on predefined disease states in the known patient records of the basic data set to form a phenotype model associating patient phenotype data with the probability of disease states in the patient records of the training data set; (g) populating the predicted values into the records within the validation data set; (h) validating the validation data set using the phenotype model and determining a validation error including the probability of correctly predicting the patient phenotype associated with the probability of disease states in the records of the validation data set; (i) repeating steps (c) through (h) to minimize the validation error and calculating and predicting a high-probability disease state phenotype for each patient record of the basic data set. The method according to claim 1, wherein the records within the validation data set include patient phenotype data associated with the probability of the disease state.

2. The method according to claim 1, wherein the step of analyzing the training data set is performed using a machine learning system.

3. ​ ​ The method according to claim 1 or 2, characterized in that the disease state is aortic stenosis.

4. The method according to claim 1 or 2, characterized in that the sparse data source includes a plurality of medical records containing measurement data obtained from a medical research process.

5. The method according to any one of claims 1 to 3, characterized in that the medical research process includes an echocardiogram examination process.

6. The method according to any one of claims 1 to 5, characterized by including, during the measurement process, a step of predicting unentered measurement data based on data collected by a measurement process operator using the measurement prediction protocol.

7. The method according to any one of claims 1 to 6, characterized by including, during the measurement process, a step of determining a patient's phenotype by the phenotype model based on data collected by a measurement process operator and / or based on measurement data predicted using the measurement prediction protocol.

8. The method according to any one of claims 1 to 7, characterized by including, during the measurement process, a step of calculating unentered measurement data and a patient's phenotype in real time.

9. The machine learning system comprises a neural network, During the measurement process, a step of forming an updated data set by incorporating measurement values obtained by a measurement process operator into the training data set; Analyzing the updated training data set using the neural network and calculating an updated measurement prediction protocol and / or an updated phenotype model; Analyzing measurement values obtained during the measurement process using the updated measurement prediction protocol and / or the updated phenotype model and predicting a likely disease state of a patient undergoing the measurement process The method according to any one of claims 1 to 8, characterized by including.

10. The method according to claim 7, characterized by including a step of instructing the measurement process operator to record relevant measurement data in order to enhance the reliability of the prediction of the patient's phenotype and the related disease state as a result of the phenotype prediction related to a predefined disease state.

11. An apparatus for performing a measurement process on a patient, A measurement tool related to the measurement process; Recording means for recording measurement data from the patient during the measurement process; Transmission means for transmitting the measurement data to analysis means, comprising, wherein the analysis means has input means for receiving the measurement data and phenotype data, association means for associating the measurement data with the phenotype data in order to determine the phenotype of a patient related to one or more disease states, a measurement prediction protocol for predicting measurement data of unentered measurement fields, and / or a phenotype model for associating patient data with phenotypes related to one or more disease states, thereby predicting the disease states with high likelihood of the patients undergoing the measurement process, warning means for warning the measurement operator of the predicted measurement data and the predicted disease states, instruction means for instructing the measurement operator of the relevant measurement data to be collected based on the predicted disease states characterized in that the device comprises.

12. The device according to claim 11, characterized in that the disease state is aortic valve stenosis.

13. The device according to claim 11 or 12, characterized in that it comprises a display surface for displaying to the measurement operator a notification including the predicted measurement data or the predicted disease state.

14. A computer-implemented method for processing sparse data sources, comprising: (a) retrieving data from the sparse data source to form a basic data set, wherein the sparse data source includes a plurality of patient records including patient mortality data, and each of the patient records includes at least one unentered data field corresponding to a medical measurement value, (b) dividing the basic data set into the following two parts, namely, a first part including a training data set of a predefined percentage X% of the basic data set, and a second part including a validation data set of a predefined percentage (100% - X%) of the basic data set, and dividing; (c) analyzing the training data set using a non-linear function approximation algorithm repeatedly applied to the records of the training data set to jointly model variable relationships in order to obtain a trained model and a measurement prediction protocol for inputting into the unentered data fields within the training data set, (d) calculating predicted values of the measurement data of the unentered data fields using the measurement prediction protocol. (e) inputting the predicted values into records in the training data set; (f) analyzing the training data set based on a predefined disease state in known patient records of the basic data set to form a phenotype model that associates the patient's phenotype data with the probability of the disease state in the patient records of the training data set; (g) inputting the predicted values into records in the validation data set; (h) validating the validation data set using the phenotype model and determining a validation error including the probability of correctly predicting the phenotype of the patient associated with the probability of the disease state in the records of the validation data set; (i) repeating steps (c) to (h) to minimize the validation error, and calculating and predicting a high-probability disease state phenotype for each patient record in the basic data set; comprising wherein records in the validation data set include patient phenotype data associated with the probability of the disease state.

15. A computer system, comprising one or more processors; for the one or more processors, (a) retrieving data from a sparse data source to form a basic data set, the sparse data source including a plurality of patient records including patient mortality data, each of the patient records including at least one unpopulated data field corresponding to a medical measurement; (b) dividing the basic data set into the following two parts: a first part including a predefined percentage X% of the training data set of the basic data set; and a second part including a predefined percentage (100% - X%) of the validation data set of the basic data set; and dividing; (c) analyzing the training data set to jointly model variable relationships using a non-linear function approximation algorithm repeatedly applied to records of the training data set for the purpose of obtaining a trained model and a measurement prediction protocol for inputting into unpopulated data fields in the training data set; (d) calculating predicted values of measurement data for the unpopulated data fields using the measurement prediction protocol; (e) inputting the predicted values into records in the training data set; (f) Analyzing a training dataset based on predefined disease states in known patient records of the basic dataset to form a phenotypic model that associates a patient's phenotypic data with the probability of a disease state in the patient records of the training dataset; (g) Inputting the predicted values into the records in the validation dataset; (h) Validating the validation dataset using the phenotypic model and determining a validation error including the probability of correctly predicting the patient's phenotype associated with the probability of a disease state in the records of the validation dataset; (i) Repeating steps (c) through (h) to minimize the validation error and calculating and predicting a high-probability disease state phenotype for each patient record in the basic dataset; One or more memories storing instructions to cause the above to be executed; comprising; A computer system, wherein the records in the validation dataset include a patient's phenotypic data associated with the probability of the disease state.

16. A computer program product having a computer-readable medium storing a computer program for processing sparse data sources, the computer program comprising: (a) Computer program code for retrieving data from the sparse data source to form a basic dataset, the sparse data source including a plurality of patient records including patient mortality data, each of the patient records including at least one unpopulated data field corresponding to a medical measurement; (b) Dividing the basic dataset into two parts as follows: A first part including a predefined percentage X% of the training dataset of the basic dataset; A second part including a predefined percentage (100% - X%) of the validation dataset of the basic dataset; Computer program code for dividing; (c) Computer program code for analyzing the training dataset to jointly model variable relationships using a non-linear function approximation algorithm repeatedly applied to the records of the training dataset for the purpose of obtaining a trained model and a measurement prediction protocol for populating the unpopulated data fields in the training dataset; (d) computer program code for calculating a predicted value of measurement data for the unentered data field using a measurement prediction protocol; (e) computer program code for entering the predicted value into a record in the training data set; (f) computer program code for analyzing the training data set based on a predefined disease state in the known patient records of the basic data set to form a phenotype model that associates the patient's phenotype data with the probability of the disease state in the patient records of the training data set; (g) computer program code for entering the predicted value into a record in the validation data set; (h) computer program code for validating the validation data set using the phenotype model and computer program code for determining a validation error including the probability of correctly predicting the patient's phenotype associated with the probability of the disease state in the records of the validation data set; (i) computer program code for repeatedly performing (c) to (h) to minimize the validation error, and for calculating and predicting a high-probability disease state phenotype for each patient record in the basic data set; one or more memories storing instructions for causing the execution; comprising; A computer program product, wherein the records in the validation data set include patient phenotype data associated with the probability of the disease state.

17. The method according to claim 14, wherein the disease state is aortic valve stenosis.

Images