System and method for determining, using an artificial intelligence (AI) model, whether to modify a diagnostic interpretation result of a rule-based ECG analysis model

Abstract

A system and method for determining, using an Al model, whether to modify a diagnostic interpretation result of a rule-based ECG analysis model are provided. Information identifying a set of features of an ECG determined by the rule-based ECG analysis model may be received. Information identifying a diagnostic interpretation result of the ECG determined by the rule-based ECG analysis model may be received. The set of features of the ECG and the diagnostic interpretation result may be input to the Al model. Whether the diagnostic interpretation result should be modified may be determined based on an output of the Al model. The diagnostic interpretation result of the ECG may be transmitted based on determining whether the diagnostic interpretation result should be modified.

Description

SYSTEM AND METHOD FOR DETERMINING, USING AN ARTIFICIAL INTELLIGENCE (Al) MODEL, WHETHER TO MODIFY A DIAGNOSTIC INTERPRETATION RESULT OF A RULE-BASED ECG ANALYSIS MODELBACKGROUND

[0001] An electrocardiogram (ECG) is a graphic representation of electrical activity of the heart, and is generally represented as a waveform. A rule-based ECG analysis model may receive an ECG, determine a set of features of the ECG, and generate a diagnostic interpretation result of the ECG based on the set of features. A physician may review the ECG and / or the diagnostic interpretation result to assess the cardiac activity of a patient.SUMMARY

[0002] This summary introduces concepts that are described in more detail in the detailed description. It should not be used to identify essential features of the claimed subject matter, nor to limit the scope of the claimed subject matter.

[0003] In an aspect, a method may include receiving information identifying a set of features of an electrocardiogram (ECG) determined by a rule-based ECG analysis model; receiving information identifying a diagnostic interpretation result of the ECG determined by the rulebased ECG analysis model; inputting, into an artificial intelligence (Al) model, the set of features of the ECG and the diagnostic interpretation result; determining, based on an output of the Al model, whether the diagnostic interpretation result should be modified; and transmitting the diagnostic interpretation result of the ECG based on determining whether the diagnostic interpretation result should be modified.

[0004] In another aspect, a device may include a memory configured to store instructions; and one or more processors configured to execute the instructions to: receive information identifying a set of features of an electrocardiogram (ECG) determined by a rule-based ECG analysis model; receive information identifying a diagnostic interpretation result of the ECG determined by the rale-based ECG analysis model; input, into an artificial intelligence (Al) model, the set of features of the ECG and the diagnostic interpretation result; determine, based on an output of the Al model, whether the diagnostic interpretation result should be modified; and transmit the diagnostic interpretation result of the ECG based on determining whether the diagnostic interpretation result should be modified.

[0005] In yet another aspect, a non-transitory computer-readable medium may store instructions that, when executed by one or more processors of a device, cause the one or more processors to: receive information identifying a set of features of an electrocardiogram (ECG) determined by a rule-based ECG analysis model; receive information identifying a diagnostic interpretation result of the ECG determined by the rule-based ECG analysis model; input, into an artificial intelligence (Al) model, the set of features of the ECG and the diagnostic interpretation result; determine, based on an output of the Al model, whether the diagnostic interpretation result should be modified; and transmit the diagnostic interpretation result of the ECG based on determining whether the diagnostic interpretation result should be modified.

[0006] Some embodiments of the present disclosure provide a technical improvement to patient diagnosis using a rale-based ECG analysis model by utilizing an Al model that is trained to determine, using a set of features of an ECG and a diagnostic interpretation result of the ECG determined by the rule-based ECG analysis model, whether a diagnostic interpretation result of the ECG should be modified. In this way, some embodiments herein may improve the accuracy of diagnostic interpretation results of ECGs by generating diagnostic interpretation results that more accurately align with final diagnoses determined by physicians. Further, some embodiments herein may improve patient safety and the accuracy and efficacy of diagnoses.BRIEF DESCRIPTION OF DRAWINGS

[0007] FIG. 1 is a diagram of an example system for determining, using an Al model, whether to modify a diagnostic interpretation result of a rale-based ECG analysis model.

[0008] FIG. 2 is a diagram of example components of a device of the example system shown in FIG. 1.

[0009] FIG. 3 is a flowchart of an example process for training an Al model.

[0010] FIG. 4 is a diagram of an example process for determining, using a rale-based ECG analysis model, a set of features of an ECG and a diagnostic interpretation result of the ECG.

[0011] FIGS. 5A-5E are diagrams of an example process for determining modification information.

[0012] FIG. 6A and 6B are diagrams of example decision trees for determining whether to modify a diagnostic interpretation result of a rule-based ECG analysis model.

[0013] FIG. 7 is a diagram of an example decision tree for determining whether to modify a diagnostic interpretation result of a rule-based ECG analysis model.

[0014] FIG. 8 is a flowchart of an example process for determining whether a diagnostic interpretation result satisfies a modification criteria.

[0015] FIG. 9 is a flowchart of an example process for determining, using an Al model, whether to modify a diagnostic interpretation result of a rule-based ECG analysis model.

[0016] FIG. 10 is a flowchart of an example process for determining whether to modify a diagnostic interpretation result based on a confidence score.

[0017] FIGS. 11A-1 ID are diagrams of an example process for selectively modifying diagnostic interpretation results.

[0018] FIG. 12 is a diagram of an example process for determining, using an Al model, whether to modify a diagnostic interpretation result of a rule-based ECG analysis model.DETAILED DESCRIPTION

[0019] As addressed above, a rule-based ECG analysis model may determine a set of features of an ECG of a patient and generate a diagnostic interpretation result based on the set of features of the ECG. A physician may review the ECG and the diagnostic interpretation result to assess cardiac activity of the patient. In some cases, the physician might correct the diagnostic interpretation result, such as by removing a diagnosis, replacing a diagnosis, adding a diagnosis, or the like. Updating the rule-based ECG analysis model to reflect the physician feedback might be impractical, computationally-expensive, or impossible.

[0020] The embodiments of the present disclosure provide an Al model that is trained to determine, using a set of features of an ECG and a diagnostic interpretation result of the ECG determined by a rule-based ECG analysis model, whether a diagnostic interpretation result of the ECG should be modified. That is, the Al model may receive the set of features of the ECG and the diagnostic interpretation result of the ECG as an input, and generate information identifying whether the diagnostic interpretation result should be modified as an output.

[0021] The Al model may be trained using training data that includes modification information identifying whether a diagnostic interpretation result determined by the rule-based ECG analysis model was modified by a physician (or another reviewing entity). In this way,some embodiments herein may improve the accuracy of diagnostic interpretation results of ECGs by generating diagnostic interpretation results that more accurately align with final diagnoses determined by physicians.

[0022] The Al model may be trained based on training data that reflects the diagnoses that are modified the most, or most frequently, by physicians. Accordingly, some embodiments herein may reduce the amount of training of the Al model and reduce the amount of training data needed to train the Al model by focusing on the diagnoses that are modified the most, or most frequently, by physicians.

[0023] The Al model may be fully-differential from the rule-based ECG analysis model. In this way, the Al model may be trained, and updated, to modify diagnostic interpretation results of the rule-based ECG analysis model in a manner that does not affect the underlying rule-based ECG analysis. Further, when implemented, the Al model might not affect the performance or operation of the rule-based ECG analysis model.

[0024] The Al model may operate in a transparent manner. For instance, the Al model may utilize features of the ECG and thresholds that are transparent to a reviewing entity. In this way, some embodiments herein improve transparency of the decision-making of the Al model, and might improve regulatory compliance.

[0025] FIG. 1 is a diagram of an example system 100 for determining, using an Al model, whether to modify a diagnostic interpretation result of a rule-based ECG analysis model. As shown in FIG. 1, the system 100 may include an ECG device 110, electrodes 120, an ECG analysis device 130, a rule-based ECG analysis model 140, a platform 150, an Al model 160, a user device 170, a database 180, and a network 190.

[0026] The ECG device 110 may be configured to generate an ECG of a patient. For example, the ECG device 110 may be a stand-alone ECG device, a portable ECG device, a multi vital sign monitoring device, or the like. The ECG device 110 may receive cardiac electrical signals via the electrodes 120, and generate the ECG based on the cardiac electrical signals. The ECG may be a 3 -lead ECG, a 5-lead ECG, a 6-lead ECG, a 12-lead ECG, or the like. The ECG device 110 may include any number of electrodes 120. For example, the ECG device 110 may include ten electrodes for generating a 12-lead ECG.

[0027] The ECG analysis device 130 may be configured to receive the ECG from the ECG device 110, determine a set of features of the ECG using the rule-based ECG analysis model 140, and determine a diagnostic interpretation result of the ECG based on the set of features and using the rule-based ECG analysis model 140. For example, the ECG analysis device may be a server, a cloud computing system, a standalone medical device, or the like.

[0028] The rule-based ECG analysis model 140 may be configured to receive the ECG from the ECG device 110, determine a set of features of the ECG, and determine a diagnostic interpretation result of the ECG. For example, the rule-based ECG analysis model 140 may be the Marquette™ 12SL ECG analysis program, or the like.

[0029] The platform 150 may be configured to receive information identifying the set of features of the ECG determined by the rule-based ECG analysis model 140, and receive information identifying the diagnostic interpretation result of the ECG determined by the ralebased ECG analysis model 140. Further, the platform 150 may be configured to input, into the Al model 160, the set of features of the ECG and the diagnostic interpretation result, and determine, based on an output of the Al model 160, whether the diagnostic interpretation result should be modified. Further, the platform 150 may transmit the diagnostic interpretation result of the ECG based on determining whether the diagnostic interpretation result should be modified. For example, the platform 150 may be a server, a cloud computing system, or the like.

[0030] The Al model 160 may be configured to receive a set of features of an ECG and a diagnostic interpretation result of the ECG, and determine whether the diagnostic interpretation result should be modified. For example, the Al model 160 may be a decision tree, a linear regression model, a neural network (e.g., a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), or the like), a logistic regression model, a support vector machine, or the like.

[0031] The user device 170 may be configured to receive the diagnostic interpretation result from the platform 150, and provide the diagnostic interpretation result for display. For example, the user device 170 may be a smartphone, a laptop computer, a desktop computer, a wearable device, a medical device, or the like.

[0032] The database 180 may be configured to store an ECG, a set of features of the ECG, a diagnostic interpretation result of the ECG, modification information that identifies amodification to the diagnostic interpretation result of the ECG, or the like. For example, the database 180 may be a hierarchical database, a network database, a relational database, or the like. According to an embodiment, the database 180 may store information associated with a patient. For example, the information associated with the patient may identify whether a specific diagnosis was present in a previous diagnostic interpretation result of the patient, identify whether a physician had previously modified a diagnosis interpretation result, identify demographic information of the patient, identify health conditions of the patient, identify prescriptions of the patient, identify previous procedures of the patient, or the like.

[0033] The network 190 may be configured to permit communication between the devices of the system 100. For example, the network 190 may be a cellular network (e.g., a fifth generation (5G) network, a long-term evolution (LTE) network, a third generation (3G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., the Public Switched Telephone Network (PSTN)), a private network, an ad hoc network, an intranet, the Internet, a fiber optic-based network, or the like, and / or a combination of these or other types of networks

[0034] The number and arrangement of the devices of the system 100 shown in FIG. 1 are provided as an example. In practice, the system 100 may include additional devices, fewer devices, different devices, or differently arranged devices than those shown in FIG. 1.Additionally, or alternatively, a set of devices (e.g., one or more devices) of the system 100 may perform one or more functions described as being performed by another set of devices of the system 100.

[0035] FIG. 2 is a diagram of example components of a device 200 of the example system shown in FIG. 1. The device 200 may correspond to the ECG device 110, the ECG analysis device 130, the platform 150, the user device 170, and / or the database 180. As shown in FIG. 2, the device 200 may include a bus 210, a processor 220, a memory 230, a storage component 240, an input component 250, an output component 260, and a communication interface 270.

[0036] The bus 210 includes a component that permits communication among the components of the device 200. The processor 220 may be implemented in hardware, firmware, or a combination of hardware and software. The processor 220 may be a central processing unit(CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or another type of processing component. The processor 220 may include one or more processors capable of being programmed to perform a function. The processor 220 may include one or more processors 220 configured to perform the operations described herein. For example, a single processor 220 may be configured to perform all of the operations described herein. Alternatively, multiple processors 220, collectively, may be configured to perform all of the operations described herein, and each of the multiple processors 220 may be configured to perform a subset of the operations descried herein. For example, a first processor 220 may perform a first subset of the operations described herein, a second processor 220 may be configured to perform a second subset of the operations described herein, etc.

[0037] The memory 230 may include a random access memory (RAM), a read only memory (ROM), and / or another type of dynamic or static storage device (e.g., a flash memory, a magnetic memory, and / or an optical memory) that stores information and / or instructions for use by the processor 220.

[0038] The storage component 240 may store information and / or software related to the operation and use of the device 200. For example, the storage component 240 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optic disk, and / or a solid state disk), a compact disc (CD), a digital versatile disc (DVD), a floppy disk, a cartridge, a magnetic tape, and / or another type of non-transitory computer-readable medium, along with a corresponding drive.

[0039] The input component 250 may include a component that permits the device 200 to receive information, such as via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, a button, a switch, a camera, and / or a microphone for receiving the reference audio input and / or visual input). Additionally, or alternatively, the input component 250 may include a sensor for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, and / or an actuator). The output component 260 may include a component that provides output information from the device 200 (e.g., a display, a speaker for outputting sound at the output sound level, and / or one or more light-emitting diodes (LEDs)).

[0040] The communication interface 270 may include a transceiver-like component (e.g., a transceiver and / or a separate receiver and transmitter) that enables the device 200 to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wireless connections. The communication interface 270 may permit the device 200 to receive information from another device and / or transmit information to another device. For example, the communication interface 270 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, or the like.

[0041] The device 200 may perform one or more processes described herein. The device 200 may perform these processes based on the processor 220 executing software instructions stored by a non-transitory computer-readable medium, such as the memory 230 and / or the storage component 240. A computer-readable medium may be defined herein as a non-transitory memory device. A memory device may include memory space within a single physical storage device or memory space spread across multiple physical storage devices.

[0042] The software instructions may be read into the memory 230 and / or the storage component 240 from another computer-readable medium or from another device via the communication interface 270. When executed, the software instructions stored in the memory 230 and / or the storage component 240 may cause the processor 220 to perform one or more processes described herein. Additionally, or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, implementations described herein are not limited to any specific combination of hardware circuitry and software.

[0043] The number and arrangement of the components shown in FIG. 2 are provided as an example. In practice, the device 200 may include additional components, fewer components, different components, or differently arranged components than those shown in FIG. 2.Additionally, or alternatively, a set of components (e.g., one or more components) of the device 200 may perform one or more functions described as being performed by another set of components of the device 200.

[0044] FIG. 3 is a flowchart of an example process 300 for training an Al model. According to an embodiment, the platform 150 may perform the operations of the process 300. FIG. 4 is adiagram of an example process 400 for determining, using a rule-based ECG analysis model, a set of features of an ECG and a diagnostic interpretation result of the ECG. FIGS. 5A-5E are diagrams of an example process 500 for determining modification information.

[0045] As shown in FIG. 3, the process 300 may include receiving training data including a set of features of an ECG determined by the rule-based ECG analysis model 140, a diagnostic interpretation result of the ECG determined by the rule-based ECG analysis model 140, and modification information identifying whether the diagnostic interpretation result was modified (operation 310).

[0046] The set of features of the ECG determined by the rule-based ECG analysis model 140 may include an amplitude of a wave of the ECG, a duration of a wave of the ECG, or the like. For example, the set of features may include an amplitude of a P wave, a duration of the P wave, an amplitude of a Q wave, a duration of the Q wave, an amplitude of an R wave, a duration of the R wave, a PR interval, an amplitude of an S wave, a duration of the S wave, a duration of the QRS complex, an amplitude of a T wave, a duration of the T wave, a QT interval, or the like. For example, as shown in FIG. 4, the rule-based ECG analysis model 140 may receive an ECG 410, and determine a set of features 420 of the ECG 410 (e.g., feature 1, feature 2, . . ., feature n, etc.).

[0047] The diagnostic interpretation result of the ECG determined by the rule-based ECG analysis model 140 may include a diagnosis of the ECG. For example, the diagnostic interpretation result may include a diagnosis, such as atrial-paced rhythm, ventricular-paced rhythm, atrial flutter, ectopic atrial tachycardia, sinus bradycardia, sinus tachycardia, junctional bradycardia, atrial fibrillation, left bundle branch block, septal infarct, or the like. For example, as shown in FIG. 4, the rule-based ECG analysis model 140 may determine a diagnostic interpretation result 430 including a diagnosis based on the set of features 420.

[0048] The modification information may identify whether the diagnostic interpretation result was modified. For example, the modification information may identify whether a diagnosis of the diagnostic interpretation result determined by the rale-based ECG analysis model 140 was accepted, removed, or replaced. As another example, the modification information may identify whether a diagnosis that was not determined by the rule-based ECG analysis model 140 was added. The modification information may identify a replacement diagnosis in the event that the diagnosis was replaced.

[0049] According to an embodiment, the platform 150 may determine the modification information based on a diagnostic interpretation result determined by the rule-based ECG analysis model 140 and a final diagnostic interpretation result determined by a physician after the physician reviews the diagnostic interpretation result determined by the rule-based ECG analysis model 140. For example, the platform 150 may compare the diagnostic interpretation result determined by the rule-based ECG analysis model 140 and the final diagnostic interpretation result determined by a physician, and determine the modification information based on the comparison.

[0050] As an example, and as shown in FIG. 5A, a physician may review a diagnostic interpretation result 502 including a diagnosis (“diagnosis A”) determined by the rale-based ECG analysis model 140, determine that the diagnosis is acceptable for the ECG, and generate a final diagnostic interpretation result 504 including the diagnosis. The platform 150 may compare the diagnostic interpretation result 502 and the final diagnostic interpretation result 504, and determine modification information 506 based on the comparison. In this case, the modification information 506 may identify that the diagnostic interpretation result 502 was not modified.

[0051] As another example, and as shown in FIG. 5B, a physician may review a diagnostic interpretation result 508 including a diagnosis (“diagnosis A”) determined by the rale-based ECG analysis model 140, determine that the diagnosis is not acceptable for the ECG, and generate a final diagnostic interpretation result 510 that does not include the diagnosis. The platform 150 may compare the diagnostic interpretation result 508 and the final diagnostic interpretation result 510, and determine modification information 512 based on the comparison In this case, the modification information 512 may identify that the diagnostic interpretation result 508 was modified by removing a diagnosis.

[0052] As another example, and as shown in FIG. 5C, a physician may review a diagnostic interpretation result 514 including a diagnosis (“diagnosis A”) determined by the rale-based ECG analysis model 140, determine that the diagnosis is not acceptable for the ECG and should be replaced, and generate a final diagnostic interpretation result 516 including a different diagnosis (“diagnosis B”) for the ECG. The platform 150 may compare the diagnostic interpretation result 514 and the final diagnostic interpretation result 516, and determine modification information 518 based on the comparison. In this case, the modificationinformation 518 may identify that the diagnostic interpretation result 514 was modified by replacing a diagnosis.

[0053] As another example, and as shown in FIG. 5D, a physician may review a diagnostic interpretation result 520 that does not include a diagnosis determined by the rule-based ECG analysis model 140, determine that a diagnosis should be added for the ECG, and generate a final diagnostic interpretation result 522 including the diagnosis. The platform 150 may compare the diagnostic interpretation result 520 and the final diagnostic interpretation result 522, and determine modification information 524 based on the comparison In this case, the modification information 524 may identify that the diagnostic interpretation result 520 was modified by adding a diagnosis.

[0054] According to an embodiment, the platform 150 may determine the modification information based on a diagnostic interpretation result determined by the rule-based ECG analysis model 140 and a diagnostic interpretation result determined by a physician in a situation where the physician has not reviewed the diagnostic interpretation result determined by the ralebased ECG analysis model 140. For example, the platform 150 may compare the diagnostic interpretation result determined by the rale-based ECG analysis model 140 and the diagnostic interpretation result determined by a physician, and determine the modification information based on the comparison.

[0055] As an example, and as shown in FIG. 5E, the rale-based ECG analysis model 140 may receive an ECG 526, and determine a diagnostic interpretation result 528 based on a set of features of the ECG 526. Further, a physician may review the ECG 526 and determine a diagnostic interpretation result 530 based on reviewing the ECG 526. The platform 150 may compare the diagnostic interpretation result 528 and the diagnostic interpretation result 530, and determine modification information 532 based on the comparison. In this case, the modification information 532 may be indicative of whether the physician would have accepted, removed, replaced, or added a diagnosis of the rale-based ECG analysis model 140 in the event that the physician had reviewed the diagnostic interpretation result 528 of the rale-based ECG analysis model 140.

[0056] As further shown in FIG. 3, the process 300 may include training the Al model based on the training data (operation 320), and storing the trained Al model (operation 330). Theplatform 150 may train the Al model 160 using the training data and a training technique. For example, the platform 150 may train the Al model 160 using unsupervised learning, supervised learning, reinforcement learning, or the like.

[0057] According to an embodiment, the Al model 160 may be a decision tree. In this case, the platform 150 may generate the Al model 160 to include a set of nodes. For example, the set of nodes may include a root node, one or more intermediate nodes, and leaf nodes. The platform 150 may generate the Al model 160 using an attribute selection measure. For example, the attribute selection measure may be information gain, a gain ratio, a Gini index, or the like. The platform 150 may generate the decision tree, and prune the decision tree using a pruning technique. For example, the pruning technique may be cost complexity pruning, reduced error pruning, or the like.

[0058] The platform 150 may generate, store, train, and / or use the Al model 160. According to an embodiment, the platform 150 may include the Al model 160 and / or instructions associated with the Al model 160. For example, the platform 150 may include instructions for generating the Al model 160, training the Al model 160, using the Al model 160, etc. According to another embodiment, a system or device other than the platform 150 may be used to generate and / or train the Al model 160. For example, a system or device may include instructions for generating the Al model 160, and / or instructions for training the Al model 160. The system or device may provide a resulting trained Al model 160 to the platform 150 for use.

[0059] According to an embodiment, the Al model 160 may include a training phase, a deployment phase, and a monitoring phase. In the training phase, the platform 150 may receive and process training data to generate the trained Al model for determining whether a diagnostic interpretation result should be modified. The training data may include a plurality of training datasets respectively including a set of features of an ECG determined by the rule-based ECG analysis model 140, a diagnostic interpretation result of the ECG determined by the rule-based ECG analysis model 140, and modification information identifying whether the diagnostic interpretation result was modified. Each training dataset of the plurality of training datasets may be associated with a particular ECG and a particular diagnostic interpretation result. An example training dataset may include a set of features of an ECG determined by the rule-based ECG analysis model 140, a diagnostic interpretation result of the ECG determined by the rule-basedECG analysis model 140, and modification information identifying whether the diagnostic interpretation result was modified that may be provided as inputs to train the Al model 160.

[0060] The training data may be generated, received, or otherwise obtained from internal and / or external resources. For example, the training data may be generated, received, or otherwise obtained from the ECG device 110, the ECG analysis device 130, the user device 170, and / or the database 180.

[0061] Generally, the Al model 160 may include a set of variables (e.g., nodes, neurons, filters, or the like) that are tuned (e.g., weighted, biased, or the like) to different values via the application of the training data. According to an embodiment, the training process may employ supervised, unsupervised, semi-supervised, and / or reinforcement learning processes to train the Al model 160. According to an embodiment, a portion of the training data may be withheld during training and / or used to validate the trained Al model 160.

[0062] For supervised learning processes, the training data may include labels or scores that may facilitate the training process by providing a ground truth. For example, the labels or scores may indicate whether a diagnostic interpretation result was modified. Training may proceed by feeding a training dataset including a set of features of an ECG determined by the rale-based ECG analysis model 140, a diagnostic interpretation result of the ECG determined by the ralebased ECG analysis model 140, and modification information identifying whether the diagnostic interpretation result was modified from the training data into the Al model 160. The Al model 160 may have variables set at initialized values (e.g., at random, based on Gaussian noise, based on pre-trained values, or the like). The Al model 160 may output information identifying whether a diagnostic interpretation result should be modified. The output may be compared with the corresponding label or score (e.g., the ground truth) indicating whether the diagnostic interpretation result was modified, which may then be back-propagated through the Al model 160 to adjust the values of the variables. This process may be repeated for a plurality of samples at least until a determined loss or error is below a predefined threshold. According to an embodiment, some of the training data may be withheld and used to further validate or test the trained Al model 160.

[0063] For unsupervised learning processes, the training data may not include pre-assigned labels or scores to aid the learning process. Instead, unsupervised learning processes may includeclustering, classification, or the like, to identify naturally occurring patterns in the training data. As an example, training data may be clustered into groups based on identified similarities and / or patterns. K-means clustering or K-Nearest Neighbors may also be used, which may be supervised or unsupervised. Combinations of K-Nearest Neighbors and an unsupervised cluster technique may also be used. For semi-supervised learning, a combination of training data with pre-assigned labels or scores and training data without pre-assigned labels or scores may be used to train the Al model 160.

[0064] When reinforcement learning is employed, an agent (e.g., an algorithm) may be trained to make a decision regarding whether a diagnostic interpretation should be modified from the training data through trial and error. For example, based on making a decision, the agent may then receive feedback (e.g., a positive reward if the prediction was above a predetermined threshold), adjust its next decision to maximize the reward, and repeat until a loss function is optimized.

[0065] After being trained, the trained Al model 160 may be stored and subsequently applied by the platform 150 during the deployment phase. For example, during the deployment phase, the trained Al model 160 executed by the platform 150 may receive input data. The input data may include a set of features of an ECG determined by the rule-based ECG analysis model 140 and a diagnostic interpretation result of the ECG determined by the rule-based ECG analysis model 140. The Al model 160 may generate output data identifying whether the diagnostic interpretation result should be modified.

[0066] The monitoring data may include data that identifies whether the diagnostic interpretation result was modified by a physician. During the monitoring phase, the monitoring data may be analyzed along with the output data and input data to determine an accuracy of the trained Al model 160. According to an embodiment, based on the analysis, the platform 150 may return to the training phase, where values of one or more variables of the Al model 160 may be adjusted to improve the accuracy of the Al model 160.

[0067] FIG. 6A and 6B are diagrams of example decision trees 600 for determining whether to modify a diagnostic interpretation result of a rule-based ECG analysis model. As shown in FIG. 6A, a decision tree 600 may correspond to a particular diagnosis 610 (“diagnosis A”) being present in a diagnostic interpretation result. The decision tree 600 may include a root node 620that corresponds to a comparison of a feature of an ECG to a threshold. Alternatively, the root node may correspond to a determination of whether another diagnosis (“diagnosis B”) is present in the diagnostic interpretation result. As another alternative, the root node may correspond to a determination of whether the other diagnosis is not present in the diagnostic interpretation result. The decision tree 600 may include no intermediate nodes, or may include one or more intermediate nodes. The intermediate node, or nodes, may be similar to the root node 620. The decision tree 600 may include leaf nodes that correspond to modification classifications. A modification classification may identify whether a diagnostic interpretation result should be accepted, removed, or replaced. For example, as shown, the leaf node 630 may correspond to a modification classification that identifies that the diagnosis 610 should be accepted, the leaf node 640 may correspond to a modification classification that identifies that the diagnosis 610 should be removed, and the leaf node 650 may correspond to a modification classification that identifies that the diagnosis 610 should be replaced.

[0068] As shown in FIG. 6B, a decision tree 600 may correspond to a particular diagnosis 660 (“diagnosis A”) not being present in a diagnostic interpretation result. The decision tree 600 may include a root node 670 that corresponds to a comparison of a feature of an ECG to a threshold. Alternatively, the root node may correspond to a determination of whether another diagnosis (“diagnosis B”) is present in the diagnostic interpretation result. As another alternative, the root node may correspond to a determination of whether the other diagnosis is not present in the diagnostic interpretation result. The decision tree 600 may include no intermediate nodes, or may include one or more intermediate nodes. The intermediate node, or nodes, may be similar to the root node 670. The decision tree 600 may include leaf nodes that correspond to modification classifications. A modification classification may identify whether a diagnostic interpretation result should not be added or should be added. For example, as shown, the leaf node 680 may correspond to a modification classification that identifies that the diagnosis 660 should not be added, and the leaf node 690 may correspond to a modification classification that identifies that the diagnosis 660 should be added.

[0069] FIG. 7 is a diagram of an example decision tree 700 for determining whether to modify a diagnostic interpretation result of a rule-based ECG analysis model. The decision tree 700 may correspond to a particular diagnosis of left bundle branch block (LBBB). The decisiontree 700 may include a root node 710, a leaf node 720, an intermediate node 730, a leaf node 740, an intermediate node 750, a leaf node 760, and a leaf node 770.

[0070] The root node 710 corresponds to a comparison of a feature of R-prime area in lead aVL and a threshold of 3564.5. The Gini index of the root node 710 is 0.232. From the training data, 745 data points included an acceptance, by a physician, of the diagnosis of LBBB determined by the rule-based ECG analysis model 140 where the R-prime area in lead aVL was less than or equal to 3564.5. Further, from the training data, 115 data points included a removal, by a physician, of the diagnosis of LBBB determined by the rule-based ECG analysis model 140 where the R-prime area in lead aVL was less than or equal to 3564.5. The root node 710 includes a modification classification that identifies that the diagnosis of LBBB should be accepted in the event that the feature of R-prime area in lead aVL is less than or equal to the threshold of 3564.5.

[0071] The leaf node 720 includes a Gini index of 0.219. From the training data, 7 data points included a removal, by a physician, of the diagnosis of LBBB determined by the rulebased ECG analysis model 140 where the R-prime area in lead aVL was greater than 3564.5. Further, from the training data, 1 data point included an acceptance, by a physician, of the diagnosis of LBBB determined by the rule-based ECG analysis model 140 where the R-prime area in lead aVL was greater than 3564.5. The leaf node 720 includes a modification classification that identifies that the diagnosis of LBBB should be removed in the event that the feature of R-prime area in lead aVL is greater than the threshold of 3564.5.

[0072] The intermediate node 730 corresponds to a determination of whether a diagnosis of left ventricular hypertrophy, that corresponds to a diagnostic statement 541 , is present in the diagnostic interpretation result. The intermediate node 730 includes a Gini index of 0.221. From the training data, 744 data points included an acceptance, by a physician, of the diagnosis of LBBB determined by the rale-based ECG analysis model 140 where the diagnosis of left ventricular hypertrophy was not present in the diagnostic interpretation result. Further, from the training data, 108 data points included a removal, by a physician, of the diagnosis of LBBB determined by the rule-based ECG analysis model 140 where the diagnosis of left ventricular hypertrophy was not present in the diagnostic interpretation result. The intermediate node 730 includes a modification classification that identifies that the diagnosis of LBBB should beaccepted in the event that the diagnosis of left ventricular hypertrophy is not present in the diagnostic interpretation result.

[0073] The leaf node 740 includes a Gini index of 0.0. From the training data, 7 data points included a removal, by a physician, of a diagnosis of LBBB determined by the rule-based ECG analysis model 140 where the diagnosis of left ventricular hypertrophy was present in the diagnostic interpretation result. Further, from the training data, 0 data points included an acceptance of a diagnosis of LBBB determined by the rule-based ECG analysis model 140 where the diagnosis of left ventricular hypertrophy was present in the diagnostic interpretation result. The leaf node 740 includes a modification classification that identifies that the diagnosis of LBBB should be removed in the event that the diagnosis of left ventricular hypertrophy is present in the diagnostic interpretation result.

[0074] The intermediate node 750 corresponds to a comparison of a feature of a duration of an R wave in lead V2 and a threshold of 35. The intermediate node 750 includes a Gini index of 0.21. From the training data, 744 data points included an acceptance, by a physician, of a diagnosis of LBBB determined by the rule-based ECG analysis model 140 where the duration of the R wave in lead V2 was less than or equal to 35. Further, from the training data, 101 data points included a removal, by a physician, of a diagnosis of LBBB determined by the rule-based ECG analysis model 140 where the duration of the R wave in lead V2 was greater than 35. The intermediate node 750 includes a modification classification that identifies that the diagnosis of LBBB should be accepted in the event that the duration of the R wave in lead V2 is less than or equal to 35.

[0075] The leaf node 760 includes a Gini index of 0.489. From the training data, 19 data points included a removal, by a physician, of a diagnosis of LBBB determined by the rule-based ECG analysis model 140 where the duration of the R wave in lead V2 was greater than 35. Further, from the training data, 14 data points included an acceptance of a diagnosis of LBBB determined by the rule-based ECG analysis model 140 where the duration of the R wave in lead V2 was greater than 35. The leaf node 760 includes a modification classification that identifies that the diagnosis of LBBB should be removed in the event that the duration of the R wave in lead V2 was greater than 35.

[0076] The leaf node 770 includes a Gini index of 0.182. From the training data, 730 data points included an acceptance, by a physician, of a diagnosis of LBBB determined by the rulebased ECG analysis model 140 where the feature of R-prime area in lead aVL is less than or equal to the threshold of 3564.5, the diagnosis of left ventricular hypertrophy was not present in the diagnostic interpretation result, and the duration of the R wave in lead V2 was less than or equal to 35. Further, from the training data, 82 data points included a removal of a diagnosis of LBBB determined by the rale-based ECG analysis model 140 the feature of R-prime area in lead aVL is less than or equal to the threshold of 3564.5, the diagnosis of left ventricular hypertrophy was not present in the diagnostic interpretation result, and the duration of the R wave in lead V2 was less than or equal to 35. The leaf node 770 includes a modification classification that identifies that the diagnosis of LBBB should be accepted in the event that the feature of R-prime area in lead aVL is less than or equal to the threshold of 3564.5, the diagnosis of left ventricular hypertrophy was not present in the diagnostic interpretation result, and the duration of the R wave in lead V2 was less than or equal to 35.

[0077] Although FIG. 3 shows example operations of the process 300, in some embodiments, the process 300 may include additional operations, fewer operations, different operations, or differently arranged operations than those depicted in FIG. 3. Additionally, or alternatively, two or more of the operations of the process 300 may be performed in parallel.

[0078] FIG. 8 is a flowchart of an example process 800 for determining whether a diagnostic interpretation result satisfies a modification criteria. The platform 150 may be configured to perform the operations of the process 800.

[0079] As shown in FIG. 8, the process 800 may include receiving information identifying a diagnostic interpretation result of an electrocardiogram (ECG) determined by a rule-based ECG analysis model (operation 810), and determining whether the diagnostic interpretation result satisfies a modification criteria (operation 820).

[0080] According to an embodiment, the modification criteria may identify whether a diagnostic interpretation result includes a diagnosis that has been modified a threshold number of times. For example, if the training data indicates that a particular diagnosis determined by the rule-based ECG analysis model 140 is never modified, or is modified relatively infrequently, then a diagnosis interpretation result that includes the particular diagnosis might not satisfy themodification criteria. Alternatively, if the training data indicates that the particular diagnosis determined by the rale-based ECG analysis model 140 is modified a large number of times, or is modified frequently, then a diagnosis interpretation result that includes the particular diagnosis might satisfy the modification criteria.

[0081] According to an embodiment, the modification criteria may identify whether the platform 150 stores a trained Al model 160 for a diagnosis of a diagnostic interpretation result. For example, an Al model 160 may correspond to a particular diagnosis. For instance, and referring to FIG. 7, the decision tree 700 corresponds to a particular diagnosis of LBBB. If the platform 150 includes a trained Al model 160 for a particular diagnosis included in a diagnostic interpretation result, then the diagnosis interpretation result might satisfy the modification criteria. Alternatively, if the platform 150 does not store a trained Al model 160 for a particular diagnosis included in a diagnostic interpretation result, then the diagnostic interpretation result might not satisfy the modification criteria.

[0082] According to an embodiment, the modification criteria may identify whether a confidence score of a diagnosis included in a diagnostic interpretation result satisfies a threshold. For example, the rale-based ECG analysis model 140 may determine a diagnosis included in a diagnostic interpretation result, and the determination may be associated with a confidence score. If the confidence score is greater than a threshold, then the diagnostic interpretation result might not satisfy the modification criteria. Alternatively, if the confidence score is less than the threshold, then the diagnostic interpretation result might satisfy the modification criteria.

[0083] As further shown in FIG. 8, if the diagnostic interpretation result satisfies the modification criteria (operation 820 - YES), then the process 800 may include executing an Al model (operation 830). The platform 150 may execute the Al model 160 based on the diagnostic interpretation result satisfying the modification criteria. For example, the platform 150 may perform the operations of FIG. 9.

[0084] As further shown in FIG. 8, if the diagnostic interpretation result does not satisfy the modification criteria (operation 820 - NO), then the process 800 may include preventing execution of the Al model (operation 840). The platform 150 may prevent execution of the Al model 160 based on the diagnostic interpretation result not satisfying the modification criteria. For example, the platform 150 may prevent the performance of the operations of FIG. 9.

[0085] Although FIG. 8 shows example operations of the process 800, in some embodiments, the process 800 may include additional operations, fewer operations, different operations, or differently arranged operations than those depicted in FIG. 8. Additionally, or alternatively, two or more of the operations of the process 800 may be performed in parallel.

[0086] FIG. 9 is a flowchart of an example process 900 for determining, using an Al model, whether to modify a diagnostic interpretation result of a rule-based ECG analysis model. The platform 150 may be configured to perform the operations of the process 900.

[0087] As shown in FIG. 9, the process 900 may include receiving information identifying a set of features of an electrocardiogram (ECG) determined by a rule-based ECG analysis model (operation 910).

[0088] The set of features of the ECG determined by the rule-based ECG analysis model 140 may include an amplitude of a wave of the ECG, a duration of a wave of the ECG, or the like. For example, the set of features may include an amplitude of a P wave, a duration of the P wave, an amplitude of a Q wave, a duration of the Q wave, an amplitude of an R wave, a duration of the R wave, a PR interval, an amplitude of an S wave, a duration of the S wave, a duration of the QRS complex, an amplitude of a T wave, a duration of the T wave, a QT interval, or the like.

[0089] As further shown in FIG. 9, the process 900 may include receiving information identifying a diagnostic interpretation result of the ECG determined by the rule-based ECG analysis model (operation 920).

[0090] The diagnostic interpretation result of the ECG determined by the rale-based ECG analysis model 140 may include a diagnosis of the ECG. For example, the diagnostic interpretation result may include a diagnosis, such as atrial-paced rhythm, ventricular-paced rhythm, atrial flutter, ectopic atrial tachycardia, sinus bradycardia, sinus tachycardia, junctional bradycardia, atrial fibrillation, left bundle branch block, septal infarct, or the like.

[0091] As further shown in FIG. 9, the process 900 may include inputting, into an artificial intelligence (Al) model, the set of features of the ECG and the diagnostic interpretation result (operation 930), and determining, based on an output of the Al model, whether the diagnostic interpretation result should be modified (operation 940).

[0092] According to an embodiment, the input to the Al model 160 may be the set of features of the ECG determined by the rule-based ECG analysis model 140, and the diagnostic interpretation result of the ECG determined by the rule-based ECG analysis model 140.According to an embodiment, the input to the Al model 160 may be information associated with a patient. For example, the information associated with the patient may identify whether a specific diagnosis was present in a previous diagnostic interpretation result of the patient, identify whether a physician had previously modified a diagnosis interpretation result, identify demographic information of the patient, identify health conditions of the patient, identify prescriptions of the patient, identify previous procedures of the patient, or the like. The platform 150 may receive a patient identifier of the patient, and receive the information associated with the patient from the database 180 using the patient identifier.

[0093] The output of the Al model 160 may be information identifying whether the diagnostic interpretation result should be modified. For example, the output may identify whether a diagnosis included in the diagnostic interpretation result should be accepted. As another example, the output may identify whether a diagnosis included in the diagnostic interpretation result should be removed. As another example, the output may identify whether a diagnosis included in the diagnostic interpretation result should be replaced with a different diagnosis. As another example, the output may identify whether a diagnosis that was not included in the diagnostic interpretation result should be added.

[0094] According to an embodiment, the Al model 160 may be a decision tree. In this case, the Al model 160 may determine a classification of the diagnostic interpretation result. The classification may identify whether the diagnostic interpretation result should be modified.Further, the classification may identify whether a diagnosis included in the diagnostic interpretation result should be accepted, removed, or replaced. As another example, the classification may identify whether a diagnosis that was not included in the diagnostic interpretation result should be added. The Al model 160 may use the nodes of the decision tree to classify the diagnostic interpretation result. For example, the Al model 160 may compare features of the ECG with respective thresholds corresponding to respective nodes. As another example, the Al model 160 may determine whether the diagnostic interpretation result includes particular diagnoses corresponding to respective nodes.

[0095] FIG. 10 is a flowchart of an example process 1000 for determining whether to modify a diagnostic interpretation result based on a confidence score. The platform 150 may be configured to perform the operations of the process 1000.

[0096] As shown in FIG. 10, the process 1000 may include determining, based on an output of the Al model, that the diagnostic interpretation result should be modified (operation 1010), and determining whether a confidence score satisfies a threshold (operation 1020).

[0097] The confidence score may identify a confidence associated with the determination that the diagnostic interpretation result should be modified. According to an embodiment, the platform 150 may determine a confidence score based on an amount of training data that was used to train the Al model 160. As another example, the platform 150 may determine the confidence score based on an attribute selection measure (e.g., information gain, a gain ratio, a Gini index, or the like). As another example, the platform 150 may determine the confidence score based on feedback information received in association with previous modifications of diagnostic interpretation results.

[0098] As further shown in FIG. 10, if the confidence score satisfies the threshold (operation 1020 - YES), then the process 1000 may include modifying the diagnostic interpretation result (operation 1030).

[0099] As further shown in FIG. 10, if the confidence score does not satisfy the threshold (operation 1040 - NO), then the process 1000 may include preventing modification of the diagnostic interpretation result (operation 1040).

[0100] Although FIG. 10 shows example operations of the process 1000, in some embodiments, the process 1000 may include additional operations, fewer operations, different operations, or differently arranged operations than those depicted in FIG. 10. Additionally, or alternatively, two or more of the operations of the process 1000 may be performed in parallel.

[0101] As further shown in FIG. 9, the process 900 may include transmitting the diagnostic interpretation result of the ECG based on determining whether the diagnostic interpretation result should be modified (operation 950).

[0102] According to an embodiment, the platform 150 may transmit the diagnostic interpretation result to the user device 170 without modifying the diagnostic interpretation result.For example, the platform 150 may determine, based on an output of the Al model 160, that the diagnostic interpretation result should not be modified, and may transmit the diagnostic interpretation result to the user device 170 without modifying the diagnostic interpretation result based on the output of the Al model 160 identifying that the diagnostic interpretation result should not be modified. As another example, the platform 150 may determine, based on an output of the Al model 160, that the diagnostic interpretation result should be modified, and may also determine that a confidence score does not satisfy a threshold. In this case, the platform may transmit the diagnostic interpretation result to the user device 170 without modifying the diagnostic interpretation result based on the confidence score not satisfying the threshold.

[0103] FIGS. 11 A-l ID are diagrams of an example process 1100 for selectively modifying diagnostic interpretation results. As shown in FIG. 11 A, the input to the Al model 160 may be the set of features 1102 of the ECG determined by the rule-based ECG analysis model 140, and the diagnostic interpretation result 1104 of the ECG determined by the rale-based ECG analysis model 140. The output of the Al model 160 may be information 1106 identifying that the diagnostic interpretation result 1104 should not be modified. That is, the output may identify that a diagnosis (“diagnosis A”) included in the diagnostic interpretation result 1104 should be accepted. The platform 150 may transmit the diagnostic interpretation result 1108 that includes the diagnosis included in the diagnostic interpretation result 1104.

[0104] According to an embodiment, the platform 150 may modify the diagnostic interpretation result, and transmit the diagnostic interpretation result to the user device 170 based on modifying the diagnostic interpretation result. For example, the platform 150 may determine, based on an output of the Al model 160, that the diagnostic interpretation result should be modified, may modify the diagnostic interpretation result, and may transmit the diagnostic interpretation result to the user device 170 based on modifying the diagnostic interpretation result.

[0105] As shown in FIG. 1 IB, the input to the Al model 160 may be the set of features 1110 of the ECG determined by the rule-based ECG analysis model 140, and the diagnostic interpretation result 1112 of the ECG determined by the rale-based ECG analysis model 140.The output of the Al model 160 may be information 1114 identifying that the diagnostic interpretation result 1112 should be modified. That is, the output may identify that a diagnosis(“diagnosis A”) included in the diagnostic interpretation result 1112 should be removed. The platform 150 may remove the diagnosis from the diagnostic interpretation result 1112, and transmit the diagnostic interpretation result 1116 that does not include the diagnosis included in the diagnostic interpretation result 1112 based on removing the diagnosis from the diagnostic interpretation result 1112.

[0106] As shown in FIG. 1 1C, the input to the Al model 160 may be the set of features 1118 of the ECG determined by the rale-based ECG analysis model 140, and the diagnostic interpretation result 1120 of the ECG determined by the rale-based ECG analysis model 140. The output of the Al model 160 may be information 1122 identifying that the diagnostic interpretation result 1120 should be modified. That is, the output may identify that a diagnosis (“diagnosis A”) included in the diagnostic interpretation result 1120 should be replaced. The platform 150 may replace the diagnosis from the diagnostic interpretation result 1120 with a different diagnosis (“diagnosis B”), and transmit the diagnostic interpretation result 1124 that includes the replacement diagnosis based on replacing the diagnosis from the diagnostic interpretation result 1120.

[0107] As shown in FIG. 1 ID, the input to the Al model 160 may be the set of features 1126 of the ECG determined by the rale-based ECG analysis model 140, and the diagnostic interpretation result 1128 of the ECG determined by the rule-based ECG analysis model 140. The output of the Al model 160 may be information 1130 identifying that the diagnostic interpretation result 1128 should be modified. That is, the output may identify that a diagnosis (“diagnosis A”) that was not included in the diagnostic interpretation result 1128 should be added. The platform 150 may add the diagnosis to the diagnostic interpretation result 1 128, and transmit the diagnostic interpretation result 1132 that includes the diagnosis.

[0108] According to an embodiment, the platform 150 may modify the diagnostic interpretation result by adding information identifying that a diagnosis should be further- analyzed by a physician. For example, the platform 150 may transmit information identifying that a diagnosis determined by the rule-based ECG analysis model 140 should be analyzed by the physician. For instance, the platform 150 may transmit the foregoing information based on determining that the diagnosis should be modified and also determining that a confidence score does not satisfy a threshold. The platform 150 may transmit information identifying anotherdiagnosis that could replace the diagnosis, may transmit information identifying another diagnosis that could be added to the diagnostic interpretation result, or the like.

[0109] According to an embodiment, the platform 150 may transmit information identifying a modification of the diagnostic interpretation result by the platform 150. For example, the platform 150 may transmit information identifying that the platform 150 modified a diagnosis of the diagnostic interpretation result. According to an embodiment, the platform 150 may transmit information identifying the set of features and / or diagnoses that the platform 150 utilized in modifying the diagnostic interpretation result. Further, the platform 150 may transmit information identifying the respective thresholds that the platform 150 utilized in modifying the diagnostic interpretation result.

[0110] According to an embodiment, the platform 150 may display the diagnostic interpretation result of the ECG based on determining whether the diagnostic interpretation result should be modified. For example, a processor of the platform 150 may transmit the diagnostic interpretation result to a display of the platform 150 to display the diagnostic interpretation result. According to another embodiment, the platform 150 may cause the diagnostic interpretation result to be displayed based on determining whether the diagnostic interpretation result should be modified. For example, the platform 150 may transmit the diagnostic interpretation result to the user device 170 to cause the user device 170 to display the diagnostic interpretation result of the ECG based on determining whether the diagnostic interpretation result should be modified.

[0111] Although FIG. 9 shows example operations of the process 800, in some embodiments, the process 900 may include additional operations, fewer operations, different operations, or differently arranged operations than those depicted in FIG. 9. Additionally, or alternatively, two or more of the operations of the process 900 may be performed in parallel.

[0112] FIG. 12 is a diagram of an example process 1200 for determining, using an Al model, whether to modify a diagnostic interpretation result of a rule-based ECG analysis model. As shown in FIG. 12, the rule-based ECG analysis model 140 may receive an ECG 1210, and determine a set of features 1220 of the ECG and a diagnostic interpretation result 1230 of the ECG. The platform 150 may input the set of features 1220 of the ECG and the diagnostic interpretation result 1230 of the ECG into the Al model 160, and determine modificationinformation 1240 that identifies whether the diagnostic interpretation result 1230 should be modified based on an output of the Al model 160. The platform 150 may transmit the diagnostic interpretation result 1230 to the user device 170 to permit the user device 170 to display a diagnosis 1250 included in the diagnostic interpretation result.

[0113] Some embodiments of the present disclosure provide a technical improvement to patient diagnosis using the rule-based ECG analysis model 140 by utilizing an Al model 160 that is trained to determine, using a set of features of an ECG and a diagnostic interpretation result of the ECG determined by the rule-based ECG analysis model 140, whether a diagnostic interpretation result of the ECG should be modified. In this way, some embodiments herein may improve the accuracy of diagnostic interpretation results of ECGs by determining diagnostic interpretation results that more accurately align with final diagnoses determined by physicians. Further, some embodiments herein may improve patient safety and the accuracy and efficacy of diagnoses.

[0114] Although the embodiments herein are associated with ECGs and a rule-based ECG analysis model, it should be understood that the embodiments herein are applicable to other types of biosignals and other types of rule-based biosignal analysis models. For example, the embodiments herein are applicable to electroencephalograms (EEGs), electromyograms (EMGs), electrooculograms (EOGs), electroretinograms (ERGs), electrogastrograms (EGGs), or the like, and respective rule-based analysis models that are associated with the foregoing biosignals.

[0115] Embodiments of the present disclosure shown in the drawings and described above are example embodiments only and are not intended to limit the scope of the appended claims, including any equivalents as included within the scope of the claims. Various modifications are possible and will be readily apparent to the skilled person in the art. It is intended that any combination of non-mutually exclusive features described herein are within the scope of the present invention. That is, features of the described embodiments can be combined with any appropriate aspect described above and optional features of any one aspect can be combined with any other appropriate aspect. Similarly, features set forth in dependent claims can be combined with non-mutually exclusive features of other dependent claims, particularly where the dependent claims depend on the same independent claim. Single claim dependencies may havebeen used as practice in some jurisdictions require them, but this should not be taken to mean that the features in the dependent claims are mutually exclusive.

Claims

CLAIMSWhat is claimed is:

1. A method comprising: receiving information identifying a set of features of an electrocardiogram (ECG) determined by a rule-based ECG analysis model; receiving information identifying a diagnostic interpretation result of the ECG determined by the rale-based ECG analysis model; inputting, into an artificial intelligence (Al) model, the set of features of the ECG and the diagnostic interpretation result; determining, based on an output of the Al model, whether the diagnostic interpretation result should be modified; and transmitting the diagnostic interpretation result of the ECG based on determining whether the diagnostic interpretation result should be modified.

2. The method of claim 1, wherein: the determining whether the diagnostic interpretation result should be modified comprises determining that the diagnostic interpretation result should not be modified, the method further comprises accepting a diagnosis included in the diagnostic interpretation result based on determining that the diagnostic interpretation result should not be modified, and the transmitting the diagnostic interpretation result of the ECG comprises transmitting the diagnostic interpretation result including the diagnosis.

3. The method of claim 1, wherein: the determining whether the diagnostic interpretation result should be modified comprises determining that the diagnostic interpretation result should be modified, the method further comprises removing a diagnosis included in the diagnostic interpretation result based on determining that the diagnostic interpretation result should be modified, and the transmitting the diagnostic interpretation result of the ECG comprises transmitting the diagnostic interpretation result that does not include the diagnosis based on removing the diagnosis from the diagnostic interpretation result.

4. The method of claim 1, wherein: the determining whether the diagnostic interpretation result should be modified comprises determining that the diagnostic interpretation result should be modified, the method further comprises replacing a first diagnosis included in the diagnostic interpretation result with a second diagnosis based on determining that the diagnostic interpretation result should be modified, and the transmitting the diagnostic interpretation result of the ECG comprises transmitting the diagnostic interpretation result that does not include the first diagnosis and that does include the second diagnosis based on replacing the first diagnosis with the second diagnosis.

5. The method of claim 1, wherein: the determining whether the diagnostic interpretation result should be modified comprises determining that the diagnostic interpretation result should be modified,the method further comprises adding a diagnosis that was not included in the diagnostic interpretation result based on determining that the diagnostic interpretation result should be modified, and the transmitting the diagnostic interpretation result of the ECG comprises transmitting the diagnostic interpretation result that includes the diagnosis based on adding the diagnosis to the diagnostic interpretation result.

6. The method of claim 1, further comprising: determining that the diagnostic interpretation result satisfies a modification criteria, wherein the inputting, into the Al model, the set of features of the ECG and the diagnostic interpretation result comprises inputting, into the Al model, the set of features of the ECG and the diagnostic interpretation result based on determining that the diagnostic interpretation result satisfies the modification criteria.

7. The method of claim 1, further comprising: determining whether a confidence score satisfies a threshold; and selectively modifying the diagnostic interpretation result based on determining whether the confidence score satisfies the threshold.

8. A device comprising: a memory configured to store instractions; and one or more processors configured to:receive information identifying a set of features of an electrocardiogram (ECG) determined by a rule-based ECG analysis model; receive information identifying a diagnostic interpretation result of the ECG determined by the rule-based ECG analysis model; input, into an artificial intelligence (Al) model, the set of features of the ECG and the diagnostic interpretation result; determine, based on an output of the Al model, whether the diagnostic interpretation result should be modified; and transmit the diagnostic interpretation result of the ECG based on determining whether the diagnostic interpretation result should be modified.

9. The device of claim 8, wherein: the one or more processors, when determining whether the diagnostic interpretation result should be modified, are configured to determine that the diagnostic interpretation result should not be modified, the one or more processors are further configured to accept a diagnosis included in the diagnostic interpretation result based on determining that the diagnostic interpretation result should not be modified, and the one or more processors, when transmitting the diagnostic interpretation result of the ECG, are configure to transmit the diagnostic interpretation result including the diagnosis.

10. The device of claim 8, wherein:the one or more processors, when determining whether the diagnostic interpretation result should be modified, are configured to determine that the diagnostic interpretation result should be modified, the one or more processors are further configured to remove a diagnosis included in the diagnostic interpretation result based on determining that the diagnostic interpretation result should be modified, and the one or more processors, when transmitting the diagnostic interpretation result of the ECG, are configured to transmit the diagnostic interpretation result that does not include the diagnosis based on removing the diagnosis from the diagnostic interpretation result.

11. The device of claim 8, wherein: the one or more processors, when determining whether the diagnostic interpretation result should be modified, are configured to determine that the diagnostic interpretation result should be modified, the one or more processors are further configured to replace a first diagnosis included in the diagnostic interpretation result with a second diagnosis based on determining that the diagnostic interpretation result should be modified, and the one or more processors, when transmitting the diagnostic interpretation result of the ECG, are configured to transmit the diagnostic interpretation result that does not include the first diagnosis and that does include the second diagnosis based on replacing the first diagnosis with the second diagnosis.

12. The device of claim 8, wherein:the one or more processors, when determining whether the diagnostic interpretation result should be modified, are configured to determine that the diagnostic interpretation result should be modified, the one or more processors are further configured to add a diagnosis that was not included in the diagnostic interpretation result based on determining that the diagnostic interpretation result should be modified, and the one or more processors, when transmitting the diagnostic interpretation result of the ECG, are configured to transmit the diagnostic interpretation result that includes the diagnosis based on adding the diagnosis to the diagnostic interpretation result.

13. The device of claim 8, wherein the one or more processors are further configured to: determine that the diagnostic interpretation result satisfies a modification criteria, wherein the one or more processors, when inputting, into the Al model, the set of features of the ECG and the diagnostic interpretation result, are configured to input, into the Al model, the set of features of the ECG and the diagnostic interpretation result based on determining that the diagnostic interpretation result satisfies the modification criteria.

14. The device of claim 8, wherein the one or more processors are further configured to: determine whether a confidence score satisfies a threshold; and selectively modify the diagnostic interpretation result based on determining whether the confidence score satisfies the threshold.

15. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a device, cause the one or more processors to: receive information identifying a set of features of an electrocardiogram (ECG) determined by a rale-based ECG analysis model; receive information identifying a diagnostic interpretation result of the ECG determined by the rule-based ECG analysis model; input, into an artificial intelligence (Al) model, the set of features of the ECG and the diagnostic interpretation result; determine, based on an output of the Al model, whether the diagnostic interpretation result should be modified; and transmit the diagnostic interpretation result of the ECG based on determining whether the diagnostic interpretation result should be modified.

16. The non-transitory computer-readable medium of claim 15, wherein: the one or more instructions, that cause the one or more processors to determine whether the diagnostic interpretation result should be modified, cause the one or more processors to determine that the diagnostic interpretation result should not be modified, the one or more instructions further cause the one or more processors to accept a diagnosis included in the diagnostic interpretation result based on determining that the diagnostic interpretation result should not be modified, andthe one or more instructions, that cause the one or more processors to transmit the diagnostic interpretation result of the ECG, cause the one or more processors to transmit the diagnostic interpretation result including the diagnosis.

17. The non-transitory computer-readable medium of claim 15, wherein: the one or more instructions, that cause the one or more processors to determine whether the diagnostic interpretation result should be modified, cause the one or more processors to determine that the diagnostic interpretation result should be modified, the one or more instructions further cause the one or more processors to remove a diagnosis included in the diagnostic interpretation result based on determining that the diagnostic interpretation result should be modified, and the one or more instructions, that cause the one or more processors to transmit the diagnostic interpretation result of the ECG, cause the one or more processors to transmit the diagnostic interpretation result that does not include the diagnosis based on removing the diagnosis from the diagnostic interpretation result.

18. The non-transitory computer-readable medium of claim 15, wherein: the one or more instructions, that cause the one or more processors to determine whether the diagnostic interpretation result should be modified, cause the one or more processors to determine that the diagnostic interpretation result should be modified, the one or more instructions further cause the one or more processors to replace a first diagnosis included in the diagnostic interpretation result with a second diagnosis based on determining that the diagnostic interpretation result should be modified, andthe one or more instructions, that cause the one or more processors to transmit the diagnostic interpretation result of the ECG, cause the one or more processors to transmit the diagnostic interpretation result that does not include the first diagnosis and that does include the second diagnosis based on replacing the first diagnosis with the second diagnosis.

19. The non-transitory computer-readable medium of claim 15, wherein: the one or more instructions, that cause the one or more processors to determine whether the diagnostic interpretation result should be modified, cause the one or more processors to determine that the diagnostic interpretation result should be modified, the one or more instructions further cause the one or more processors to add a diagnosis that was not included in the diagnostic interpretation result based on determining that the diagnostic interpretation result should be modified, and the one or more instructions, that cause the one or more processors to transmit the diagnostic interpretation result of the ECG, cause the one or more processors to transmit the diagnostic interpretation result that includes the diagnosis based on adding the diagnosis to the diagnostic interpretation result.

20. The non-transitory computer-readable medium of claim 15, wherein the one or more instructions further cause the one or more processors to: determine that the diagnostic interpretation result satisfies a modification criteria; wherein the one or more instructions, that cause the one or more processors to input, into the Al model, the set of features of the ECG and the diagnostic interpretation result, cause the one or more processors to input, into the Al model, the set of features of the ECG and thediagnostic interpretation result based on determining that the diagnostic interpretation result satisfies the modification criteria.

Images