Medical device setup and prescription assistance methods, apparatuses, and systems

Abstract

Methods, apparatuses, and systems for medical device setup and prescription assistance are disclosed herein. An example dialysis machine (100) includes a display screen (102), an imaging device (220), and a network connection (210). The display screen displays a graphical user interface (110) configured to receive at least one of programming setup information (400) and treatment input data (440). The imaging device is communicatively coupled to the dialysis machine and is configured to capture setup image data. The network connection communicatively couples the dialysis machine to a cloud computing environment. The dialysis machine is configured to send the captured setup image data to the cloud computing environment and receive an error correction suggestion. The error correction suggestion is based on an error identified in the setup image data when comparing the setup image data to preloaded image data in a training dataset.

Description

MEDICAL DEVICE SETUP AND PRESCRIPTION ASSISTANCE METHODS, APPARATUSES, AND SYSTEMSBACKGROUND

[0001] Renal replacement therapy (“RRT”) is a therapy that replaces the normal bloodfiltering function of the kidneys. It is used when the kidneys are not working well, which is known as kidney failure and includes acute kidney injury (“AKI”) and chronic kidney disease (“CKD”). RRT involves removal of water from the body of a patient suffering from kidney failure, as well as exchange of solutes with the body. One example of RRT is an extracorporeal blood therapy, in which blood is circulated outside of a patient and interfaced with one or more medical fluids. Modalities of extracorporeal blood therapy include hemodialysis (“HD”), hemofiltration (“HF”) and hemodiafiltration (“HDF”). Another example of RRT is peritoneal dialysis (“PD”), in which a medical fluid is infused into a peritoneal cavity of a patient to interface with the blood of the patient through a peritoneal membrane.

[0002] RRT is performed by a dialysis system which is formed by arranging one or more disposable components on a dialysis machine. Medical fluids used in HD and PD are commonly known as dialysis fluids. In HF, the medical fluid is known as replacement fluid, since it is infused into the blood of a patient to replace fluid removed during therapy. In HDF, both dialysis fluid and replacement fluid are used.

[0003] Extracorporeal blood therapy by HD, HF, or HDF is performed differently for treatment of patients with AKI compared to patients with CKD, by use of a different type of dialysis machine. Generally, compared to CKD patients, AKI patients are treated continuously over a longer period of time and at lower fluid flow rates. Such continuous treatment is commonly known as CRRT (“Continuous Renal Replacement Therapy”). To ensure precise and consistent monitoring and control of fluid removal, known as ultrafiltration, AKI machines are typically provided with scales that are used for measuring the weight of fresh treatment fluid and the weight of spent treatment fluid during therapy. CKD machines instead use flow meters or volumetric pumping to control ultrafiltration.

[0004] PD machines, also known as cyclers, may include at least one scale to measure the weight of fresh treatment fluid infused into the peritoneal cavity and the weight of spent treatmentfluid withdrawn from the peritoneal cavity. Alternatively, cyclers may use volumetric pumps to control ultrafiltration.

[0005] In any of the above modalities, an automated dialysis machine is located in a medical center or a patient’s home. To program or change a treatment, a patient or a clinician currently has to setup the treatment or therapy by entering patient and treatment specific information, select and load a proper disposable set, and make various connections before priming the setup and connecting to the patient. The various connections can include several different fluid, electrical and communication connections between the medical device, disposable set, and other components and accessories. Treatment setup can be complex with many different connections and configurations, which can be difficult and prone to setup errors. Furthermore, many different therapy and set configurations may be applicable for various ailments.

[0006] A need accordingly exists for assistance with setup and prescription selection to provide optimal treatment plans and reduce setup errors.SUMMARY

[0007] Example systems, methods, and apparatuses are disclosed herein for setup assistance and / or prescription assistance when preparing a medical device. An example dialysis machine may include a display screen, an imaging device, and a network connection. The display screen may be configured to display a graphical user interface. The graphical user interface may be configured to receive programming setup information, treatment input data or both. The imaging device may be communicatively coupled to the dialysis machine, and the imaging device may be configured to capture setup image data. In an example, the network connection communicatively couples the dialysis machine to a cloud computing environment. In one or more examples, the dialysis machine may be configured to, via the network connection, send the captured setup image data to the cloud computing environment, and receive an error correction suggestion from the cloud computing environment. The error correction suggestion may be based on an error identified in the setup image data when comparing the setup image data to preloaded image data in a training dataset.

[0008] The preloaded image data may include images of a specified therapy setup. Additional setup image data may be saved to the preloaded image data to enhance and improve the existing preloaded image data. The setup image data may include images, photos, videos,screengrabs, snapshots or the like of (i) therapy inputs provided in a graphical user interface of the dialysis machine and / or (ii) all or portions (e.g., areas of interest) of the physical components and connections of the dialysis machine such as a disposable set, fluid connections between the disposable set and other treatment components, fluid connections between the dialysis machine, the patient and other treatment components, etc.

[0009] Received setup image data may be analyzed against the preloaded image data, which may be stored in a training dataset, to identify setup errors. The analysis or image data comparison may include calculating an absolute difference between the images or any other suitable image analysis / comparison technique.

[0010] In light of the disclosure herein and without limiting the disclosure in any way, in a first aspect of the present disclosure, which may be combined with any other aspect listed herein, a dialysis machine includes a display screen, an imaging device, and a network connection. The display screen is configured to display a graphical user interface, and the graphical user interface is configured to receive at least one of programming setup information and treatment input data. The imaging device is communicatively coupled to the dialysis machine, and the imaging device is configured to capture setup image data. The network connection communicatively couples the dialysis machine to a cloud computing environment. The dialysis machine is configured to, via the network connection, send the captured setup image data to the cloud computing environment, and receive an error correction suggestion from the cloud computing environment. The error correction suggestion is based on an error identified in the setup image data when comparing the setup image data to preloaded image data in a training dataset.

[0011] In a second aspect of the present disclosure, which may be combined with any other aspect listed herein, the preloaded image data includes images of a specified therapy setup.

[0012] In a third aspect of the present disclosure, which may be combined with any other aspect listed herein, the setup image data includes at least one image, photo, video, screengrab or snapshot of therapy inputs provided in a graphical user interface of the dialysis machine.

[0013] In a fourth aspect of the present disclosure, which may be combined with any other aspect listed herein, the setup image data includes at least one image, photo, video, screengrab or snapshot of an area of interest on the dialysis machine. The area of interest includes at least one of a disposable set, fluid connections between the disposable set and other treatment components, and fluid connections between the dialysis machine and a patient.

[0014] In a fifth aspect of the present disclosure, which may be combined with any other aspect listed herein, the imaging device is a camera.

[0015] In a sixth aspect of the present disclosure, which may be combined with any other aspect listed herein, the imaging device is a mobile user device.

[0016] In a seventh aspect of the present disclosure, which may be combined with any other aspect listed herein, a method includes capturing, by an imaging device, setup image data for a therapy setup on a dialysis machine. The dialysis machine is communicatively coupled to a cloud computing environment and the imaging device. The method also includes sending, by at least one of the dialysis machine and the imaging device, the captured setup image data to the cloud computing environment and receiving, by the cloud computing environment, the setup image data. Additionally, the method includes analyzing, by the cloud computing environment, the received setup image data. Responsive to identifying a setup error based on the analysis, the method includes providing, by the cloud computing environment, an error correction suggestion to at least one of the dialysis machine and / or the imaging device.

[0017] In an eighth aspect of the present disclosure, which may be combined with any other aspect listed herein, analyzing the received setup image data includes comparing the setup image data to preloaded image data in a training dataset. The preloaded image data includes images of a specified therapy setup.

[0018] In a nineth aspect of the present disclosure, which may be combined with any other aspect listed herein, comparing the setup image data to preloaded image data includes calculating an absolute difference between a first image from the setup image data to a second image from the preloaded image data.

[0019] In a tenth aspect of the present disclosure, which may be combined with any other aspect listed herein, the method further includes saving at least a portion of the setup image data to the training dataset.

[0020] In an eleventh aspect of the present disclosure, which may be combined with any other aspect listed herein, the setup image data includes at least one image, photo, video, screengrab or snapshot of therapy inputs provided in a graphical user interface of the dialysis machine.

[0021] In a twelfth aspect of the present disclosure, which may be combined with any other aspect listed herein, the setup image data includes at least one image, photo, video, screengrab orsnapshot of an area of interest on the dialysis machine. The area of interest includes at least one of a disposable set, fluid connections between the disposable set and other treatment components, and fluid connections between the dialysis machine and a patient.

[0022] In a thirteenth aspect of the present disclosure, which may be combined with any other aspect listed herein, the imaging device is a camera.

[0023] In a fourteenth aspect of the present disclosure, which may be combined with any other aspect listed herein, the imaging device is a mobile user device.

[0024] In a fifteenth aspect of the present disclosure, which may be combined with any other aspect listed herein, a dialysis therapy system includes a dialysis machine, a cloud computing environment, and a network connection. The dialysis machine includes a display screen configured to display a graphical user interface, and the graphical user interface is configured to receive treatment input data. The network connection communicatively couples the dialysis machine to the cloud computing environment. The dialysis machine is configured to send the treatment input data to the cloud computing environment through the network connection. The cloud computing environment is configured to receive the treatment input data, analyze the treatment input data, identify at least one of an error or an empty field in the treatment input data, and provide an optimal prescription plan. The optimal prescription plan includes at least one entry to correct the error or fill an empty field.

[0025] In a sixteenth aspect of the present disclosure, which may be combined with any other aspect listed herein, treatment input data includes patient information, therapy information, and prescription information.

[0026] In a seventeenth aspect of the present disclosure, which may be combined with any other aspect listed herein, treatment input data includes urine volume information, creatinine level information, and patient condition information.

[0027] In an eighteenth aspect of the present disclosure, which may be combined with any other aspect listed herein, the optimal prescription plan includes a proposed therapy modality, a proposed disposable set, and proposed flow rates associated with the proposed therapy and proposed disposable set.

[0028] In a nineteenth aspect of the present disclosure, which may be combined with any other aspect listed herein, a method includes receiving, by a dialysis machine, treatment input data. The dialysis machine is communicatively coupled to a cloud computing environment. The methodalso includes sending, by the dialysis machine, the received treatment input data and receiving, by the cloud computing environment, the treatment input data. Additionally, the method includes analyzing the received treatment input data. Analyzing the received treatment input data includes comparing the treatment input data to preloaded data in a training dataset. Responsive to identifying at least one of an input error or an empty field associated with the treatment input data, the method includes providing, by the cloud computing environment, an optimal prescription plan.

[0029] In a twentieth aspect of the present disclosure, which may be combined with any other aspect listed herein, the treatment plan input data includes patient information, therapy information, and prescription information.

[0030] In a twenty-first aspect of the present disclosure, which may be combined with any other aspect listed herein, the patient information includes a patient identifier and at least one of a patient weight, a patient hematocrit percent, and patient condition information.

[0031] In a twenty-second aspect of the present disclosure, which may be combined with any other aspect listed herein, the therapy information includes at least one of a therapy type, a therapy modality, a set type, an anticoagulation method, and an identification of an associated treatment accessory.

[0032] In a twenty -third aspect of the present disclosure, which may be combined with any other aspect listed herein, the prescription information includes at least one of a set flow rate, a blood flow rate, a resulting treatment parameter, and dose information.

[0033] In a twenty-fourth aspect of the present disclosure, which may be combined with any other aspect listed herein, the treatment input data includes urine volume information, creatinine level information, and patient condition information.

[0034] In a twenty-fifth aspect of the present disclosure, which may be combined with any other aspect listed herein, the optimal prescription plan includes a proposed therapy modality, a proposed disposable set, and proposed flow rates associated with the proposed therapy and proposed disposable set.

[0035] In a twenty-sixth aspect of the present disclosure, which may be combined with any other aspect listed herein, the method further includes receiving, by a dialysis machine, treatment outcome information and sending the received treatment outcome information. The method also includes receiving, by the cloud computing environment, the treatment outcome information. Additionally, the method includes saving, by the cloud computing environment, at least a portionof the treatment outcome information and / or at least a portion of the treatment input data to the training dataset.

[0036] In a twenty-seventh aspect of the present disclosure, which may be combined with any other aspect listed herein, the method further includes, prior to saving at least a portion of the treatment outcome information and / or the treatment input data, classifying the treatment outcome information and / or the treatment input data.

[0037] In a twenty-eighth aspect of the present disclosure, which may be combined with any other aspect listed herein, the method further includes prior to saving at least a portion of the treatment outcome information and / or the treatment input data, organizing the treatment outcome information and / or the treatment input data.

[0038] In a twenty-nineth aspect of the present disclosure, which may be combined with any other aspect listed herein, the treatment outcome information includes at least one of feedback on treatment outcome status, patient lab report parameters, prescription parameters, type of alarms triggered, quantity of alarms triggered, and fluid removal information.

[0039] In a thirtieth aspect of the present disclosure, which may be combined with any other aspect listed herein, the treatment outcome status is one of condition improved, condition worsened or deteriorated, and condition remained unchanged.

[0040] In a thirty-first aspect of the present disclosure, any of the structure and functionality disclosed in connection with Figs. 1 to 10C may be combined with any of the other structure and functionality disclosed in connection with Figs. 1 to 10C.

[0041] In light of the present disclosure and the above aspects, it is therefore an advantage of the present disclosure to provide setup assistance and prescription assistance when setting up a medical device, such as a dialysis machine, to deliver a treatment to a patient.

[0042] It is another advantage of the present disclosure to use an artificial intelligence analysis and assistance module to detect setup errors and provide corrective suggestions to prevent injuries, equipment damage, or improper treatments due to improper setup (e.g., improper therapy selections or connections).

[0043] It is another advantage of the present disclosure to use an artificial intelligence analysis and assistance module to provide prescription recommendations, including set flow rats and other flow rates based on patient data, training data and / or therapy and set information to ensure optimal therapy selection and treatment for patients. Traditionally, setting personalizedprescription for a patient can otherwise be cumbersome due to the various set flow rate and other flow rate constrains based on a selected dialysis machine therapy and set.

[0044] Additional features and advantages are described in, and will be apparent from, the following Detailed Description and the Figures. The features and advantages described herein are not all-inclusive and, in particular, many additional features and advantages will be apparent to one of ordinary skill in the art in view of the figures and description. Also, any particular embodiment does not have to have all of the advantages listed herein and it is expressly contemplated to claim individual advantageous embodiments separately. Moreover, it should be noted that the language used in the specification has been selected principally for readability and instructional purposes, and not to limit the scope of the inventive subject matter.BRIEF DESCRIPTION OF THE FIGURES

[0045] Fig. 1 is a diagram of a dialysis machine, according to an example embodiment of the present disclosure.

[0046] Fig. 2 is a diagram of the dialysis machine of Fig. 1 located in a medical system that includes an imaging device and a cloud computing environment, according to an example embodiment of the present disclosure.

[0047] Fig. 3A is a diagram of an example process for providing setup assistance via artificial intelligence hosted in the cloud computing environment, according to an example embodiment of the present disclosure.

[0048] Fig. 3B is a diagram of an example process for providing therapy prescription assistance via artificial intelligence hosted in the cloud computing environment, according to an example embodiment of the present disclosure.

[0049] Fig. 3C is a diagram of an example process for improving and / or fine-tuning the artificial intelligence hosted in the cloud computing environment, according to an example embodiment of the present disclosure.

[0050] Fig. 4A is a diagram of programming setup information that is entered into a dialysis machine when setting up a treatment, according to an example embodiment of the present disclosure.

[0051] Fig. 4B is a diagram of treatment input data, according to an example embodiment of the present disclosure.

[0052] Fig. 4C is a diagram of treatment outcome information that is either entered into a dialysis machine or uploaded to a cloud computing environment after therapy completion, according to an example embodiment of the present disclosure.

[0053] Fig. 5 A is a diagram of an artificial intelligence analysis and assistance module that is hosted in a cloud computing environment, according to an example embodiment of the present disclosure.

[0054] Fig. 5B is a diagram of an artificial intelligence analysis and assistance module that is hosted in a cloud computing environment, according to an example embodiment of the present disclosure.

[0055] Fig. 6A is a diagram of a neural network receiving inputs, providing outputs and updating via a therapy feedback loop, according to an example embodiment of the present disclosure.

[0056] Fig. 6B is a diagram of node calculations within a neural network, according to an example embodiment of the present disclosure.

[0057] Fig. 7A shows an example training image from a training dataset, according to an example embodiment of the present disclosure.

[0058] Fig. 7B shows an example training image from a training dataset, according to an example embodiment of the present disclosure.

[0059] Fig. 8 shows an example setup image, which is provided to compare against training dataset to determine whether any setup errors exist, according to an example embodiment of the present disclosure.

[0060] Fig. 9 shows an example output image after comparing the setup image of Fig. 8 against the training dataset, the output image indicating areas that need correction, according to an example embodiment of the present disclosure.

[0061] Fig. 10A is a flow diagram of an example process for detecting setup errors and providing corrective suggestions via Al hosted in a cloud computing environment, according to an example embodiment of the present disclosure.

[0062] Fig. 10B is a flow diagram of an example process for providing prescription assistance via Al hosted in a cloud computing environment, according to an example embodiment of the present disclosure.

[0063] Fig. 10C is a flow diagram of an example process for saving at least a portion of treatment outcome information in a training dataset to improve Al capabilities, according to an example embodiment of the present disclosure.DETAILED DESCRIPTION

[0064] Methods, systems, and apparatuses are disclosed for medical device, and in particular dialysis machine, setup and prescription assistance. The methods, systems, and apparatuses are configured to provide setup assistance and / or prescription assistance via artificial intelligence to ensure proper device programming and physical connections (e.g., fluid connections) are made and to ensure optimal or best-available therapies are provided to the patient. The methods, systems, and apparatuses use an artificial intelligence analysis and assistance module, hosted in a cloud computing environment, to detect setup errors and provide corrective setup suggestions. Furthermore, the methods, systems and apparatuses use an artificial intelligence analysis and assistance module, hosted in a cloud computing environment to provide optimal therapy recommendations specific to a particular patient’s needs. The artificial intelligence analysis and assistance module may comprise a neural network architecture.

[0065] Fig. 1 is a diagram of a medical device, such as a dialysis machine 100. The dialysis machine 100 is a medical device that is configured to accept one or more parameters specifying a treatment or prescription (i.e., treatment programming information). The dialysis machine 100 performs one or more dialysis routines, such as priming patient tubing, disinfecting tubing, and providing one or more dialysis treatments.

[0066] In the illustrated example, the medical device (e.g., dialysis machine 100) is the PrisMax CRRT machine manufactured by the assignee of the subject application. It should be appreciated that in other embodiments, the dialysis machine 100 may include any other renal failure therapy machine. Further, the dialysis machine 100 may instead be a medical device that includes an infusion pump, a physiological sensor, etc. For example, the medical device may include, for example, an infusion pump (e.g., a syringe pump, a linear peristaltic pump, a large volume pump (“LVP”), an ambulatory pump, multi-channel pump), a nutritional compounding machine, an oxygen sensor, a respiratory monitor, a glucose meter, a blood pressure monitor, an electrocardiogram (“ECG”) monitor, a weight scale, and / or a heart rate monitor. It should be appreciated that multimodality inputs may be used virtually on any medical device.

[0067] The example dialysis machine 100 may include a display screen 102 for displaying instructions and receiving control inputs from a user. The display screen 102 may include buttons, a control panel, or a touchscreen. The display screen 102 may also be configured to enable a user to navigate to a certain window (e.g., control window, application tab, etc.) or graphical user interface. The display screen 102 may also display instructions for operating or controlling the dialysis machine 100 and / or a status of the dialysis machine 100.

[0068] The example dialysis machine 100 also includes the control engine 104. The example control engine 104 operates according to one or more instructions or software routines 108 for performing a treatment on a patient. The instructions or software routines 108 may be stored on a memory device 106 of the dialysis machine 100. The memory device 106 may also store one or more graphical user interfaces 110 that are displayed by the display screen 102.

[0069] Inputs for operating the dialysis machine 100 may be received by the display screen 102 via the control engine 104. In an example, the control engine 104 receives the inputs as dialysis machine command signals. The control engine 104 is configured to use the inputs in conjunction with the instructions or software routines 108 to control operation of the dialysis machine 100. The control engine 104 may also be configured to monitor device components for issues, which are documented as diagnostic information. Additionally, the control engine 104 may create medical device data in conjunction with operating one or more pumps or other components to administer a treatment.

[0070] In the illustrated example, the dialysis machine 100 also includes a processor 112, which may include a microcontroller, an application specific integrated circuit, a logic controller, etc. The processor 112 is configured to determine commands from various user inputs or selections on a graphical user interface (“GUI”) displayed on display screen 102 (e.g., user inputs or selections when setting up treatment or therapy). The processor 112 may use one or more data structures 114 in the memory device to determine an association between a command and command signal(s) transmitted to the control engine 104.

[0071] The processor 112 is also configured to render the graphical user interface(s) 110 for display on the display screen 102 of the dialysis machine 100. The graphical user interface(s) 110 may include links between each other to provide a menu structure (e.g., selectable menu options, tabs, windows, etc.). Further, the graphical user interface(s) 110 may specify conditions when certain of the interfaces are to be displayed, such as which of the graphical user interface(s)110 are displayed at startup. In an example, the processor 112 determines which of the graphical user interfaces 110 is to be displayed based on received commands and / or a current operating status of the dialysis machine 100. The processor 112 then renders the determined graphical user interface 110 for display on the display screen 102.

[0072] Display 102 may be a touchscreen and / or one or more selectable buttons, icons, menus or the like. When a touchscreen is included with the dialysis machine 100, the touchscreen may be part of a display screen that includes a capacitive touchscreen that transmits messages indicative of screen coordinates of a user’s touch. The messages are transmitted to a microcontroller and / or a processor. The microcontroller and / or a processor identify which graphical user interface is displayed by the display screen 102 and coordinates of graphical buttons shown within the graphical user interface. Each graphical button is associated with a touch command for operating the dialysis machine 100. The microcontroller and / or processor matches the received touch coordinates to coordinates of one graphical button.

[0073] As shown in Fig. 1, the processor 112 is communicatively coupled to the control engine 104, which may comprise another processor of the dialysis machine 100. The processor 112 may be separated from the control engine 104 to prevent any issues specific to processor 112 from affecting the operation of the dialysis machine 100. Alternatively, the processor 112 may include or be integrally formed with the control engine 104.

[0074] The dialysis machine 100 is a medical device that is configured to accept one or more parameters specifying a treatment or prescription (i.e., treatment programming information). Additionally, the dialysis machine 100 performs one or more dialysis routines, such as priming patient tubing, disinfecting tubing, and providing one or more dialysis treatments. Medical device(s), such as dialysis machine 100, and more specifically the PrisMax CRRT machine, may perform CRRT therapy, which is a non-stop, 24-hour dialysis therapy that is used to treat patients with AKI and fluid overload. The setup for a CRRT treatment may include entering prescription information (e.g., set selection, flow rates, etc.), loading tubing sets, connecting bags (dialysate, effluent, auto effluent, replacement, pre-blood-pump “PBP” solution), priming patient connections, connecting lines to external devices (extracorporeal membrane oxygenation “ECMO”, warmer, eccor2, etc.).

[0075] As used herein, “sets” refer to disposable sets, filter sets, dialysis sets (e.g., CRRT filter sets, such as the PrisMax ST sets), or the like, which are typically pre-assembled, closed andsterile systems that are safer and more effective than piecing together separate tubing and components. The “sets” may be specific to a therapy type, medical device (e.g., dialysis machine 100), etc.

[0076] When setting up the dialysis machine 100 for a treatment therapy, a nurse may start to program a new therapy or treatment by using the GUI displayed to start a new treatment by selecting either “new patient” or “same patient.” In an example, the “same patient” selection may be provided if settings from a previous treatment were saved and stored. After selecting the type of patient, the GUI may enter a “setup mode” where programming setup information (e.g., programming setup information 400, as illustrated in Fig. 4A) is provided. It should be appreciated that “setting up a treatment” or “setting up a therapy” may include any of the steps associated with programming the medical device (e.g., dialysis machine 100), inputting information or making selections regarding the treatment or therapy, positioning and connecting a set (e.g., disposable set), connecting other devices and equipment required for the treatment or therapy, priming the setup, and making patient connections in order to provide a personalized prescribed therapy or treatment to a patient.

[0077] Referring briefly to Fig. 4A, the programming setup information 400 may include patient information 410, which may include information about a patient identification (“ID”) 412, patient weight 414, and patient hematocrit percent 416. In some examples, patient information 410 may also include patient lab / condition information 418, which may include lab report information, such as glomerular filtration rate (“GFR”), creatinine level, blood pressure (“BP”), sugar levels, complete blood count (“CBC”), blood urea nitrogen (“BUN”), electrolyte levels (including calcium and phosphate). Additionally, the patient lab / condition information 418 may include information about the patient’s condition, such as diabetic ketoacidosis, gastrointestinal bleeding, drug overdose, respiratory failure, sepsis, stroke, cancer, etc. It should be appreciated that provided examples of the patient information 410 are non-limiting and non-exhaustive and the patient information 410 may include other patient information 411. Some other patient information 411 may include fluid status information, such as gain / loss limits [ml / 3h], current gain / loss [ml / 3h], and current makeup value [ml],

[0078] Additionally, therapy information 420 may be provided including information regarding the selection or settings related to: therapy type or modality 422, set type 424, anticoagulation 426 (e.g., anticoagulation information), and accessories 428 (e.g., accessoryinformation) such as an Auto Effluent (“AE”) accessory, blood warmer, etc. Furthermore, prescription information may also be entered including: set flow rates, blood flow rate, resulting treatment parameters, dose information, and the like. The therapy type or modality 422 may include selections from various types of CRRT, such as: slow continuous ultrafiltration (“SCUF”), continuous veno-venous hemofiltration (“CWH”), continuous veno-venous hemodialysis (“CVVHD”), and continuous veno-venous hemodiafiltration (“CVVHDF”). The therapy type or modality 422 may also include therapeutic plasma exchange (“TPE”). It should be appreciated that the list of therapy types and modalities 422 is non-limiting and non-exhaustive and depends on the type of medical device and the device’s capabilities.

[0079] The set type 424 may include information regarding the specific set(s) (e.g., disposable set) selected, the number of sets for treatment, set usage time, etc. In some instances, set type 424 and other set information may be provided by scanning an identifier, such as a barcode associated with the disposable set or accessory set. In other instances, the set type 424 may be entered manually or by making a selection from a menu (e.g., drop-down list). Set types 424 may include low-flow sets, high-flow sets, CRRT disposable sets,

[0080] In an example, a CRRT disposable set may include various components, including one or more sample sites, pressure pods, deaeration chambers, PBP lines, replacement lines, dialysate lines, effluent lines, effluent bags, access lines, chamber monitor lines, cartridges, filters, pump segments, electrostatic discharge rings, return lines, syringe lines, warmer connections, and rinsing accessories (e.g., a y-line with bag spike and connections for priming operations). Sample sites may include ports that allow needle entry to the set to obtain fluid or blood samples. Pressure pods and associated sensors allow non-invasive pressure monitoring, and the pressure pods may include an access pod, a filter pod, and an effluent pod. Deaeration chambers may be positioned on a return line and is configured to allow the medical device to manage air and to add post-filter replacement solution to the return line.

[0081] In another example, a TPE disposable set may include various components, including one or more sample sites, pressure pods, deaeration chambers, PBP lines, replacement lines, effluent bag fill connectors, effluent lines, effluent bags, access lines, chamber monitor lines, cartridges, filters, pump segments, electrostatic discharge rings, return lines, syringe lines, warmer connections, and rinsing accessories (e.g., a y-line with bag spike and connections for priming operations). Some example disposable sets that may be used in conjunction with the medicaldevice include M60, Ml 00, Ml 50, oXiris, HF 1000, HF 1400, TPE2000. It should be appreciated that disposable sets may include a subset of components listed above. Moreover, components other than those listed above may be included in a disposable set.

[0082] Referring now to Fig. 4B, similar to the programming setup information 400 of Fig. 4A, a user (e.g., nurse) may also provide and enter treatment input data 440 when initially setting up a treatment or therapy. The treatment input data 440 may include partial data or empty fields in the programming setup information 400. Additionally, the treatment input data 440 may include urine volume information 442, creatinine level information 444, and patient condition information 446. The patient condition information 446 may include information regarding the patient’ s current ailment or condition, such as diabetic ketoacidosis, gastrointestinal bleeding, drug overdose, respiratory failure, sepsis, stroke, cancer, etc. This additional information may assist an Al agent or model hosted in the cloud (e.g., AAAM 550 of Fig. 5B or neural network model 605 of Figs. 6A and 6B) in providing an optimal (e.g., best available) prescription including modality, set and flow rates.

[0083] Referring back to Fig. 1, after programming the medical device (e.g., dialysis machine 100), the nurse may load the set (e.g., disposable set, filter set, etc.), connect set components, and install accessories. Then, the nurse may connect priming solution(s), fluid bag(s), and syringe(s) if used. After connecting the set components and installing any accessories, the nurse may verify prime readiness, start prime, and view the progress of the prime. After prime is complete, the nurse may view and confirm the prescription before connection the patient and starting treatment. Upon confirming prescription, the nurse may connect the patient (e.g., access and return lines) and start the treatment.

[0084] Due to the complexity of the setup, nurses and other medical practitioners may make mistakes during (i) the initial programming setup when entering patient information, therapy information and / or prescription information and (ii) physical connection setup when loading a set, connecting set components, fluid components and bags, priming solutions, accessories, etc. Furthermore, additional complications may exist if the medical device (e.g., dialysis machine 100, such as a CRRT machine) is used along with other external devices. For example, some of the external devices, such as an ECMO, may have multiple different wiring combinations, which further augments the complexity of the treatment setup. By integrating he medical device (e.g., dialysis machine 100, such as a CRRT machine) with artificial intelligence (“Al”), such as an Alagent or model hosted in the cloud (described in more detail with respect Fig. 2 and Figs. 3A-3C), the dialysis machine 100 may assist users in identifying mistakes (e.g., setup, configuration or connection mistakes) and also provide suggested solutions, setup routines, or adjustments to correct any of the identified mistakes. This feature may also be used to verify the connections before treatment as well as making therapy suggestions.

[0085] Fig. 2 is a diagram of the medical device (e.g., dialysis machine 100 of Fig. 1) located in a medical system 200 that includes server(s) 202, database(s) 204 associated with cloud206. It should be appreciated that cloud 206 and associated components (e.g., server(s) 202 and database(s) 204) may be generally referred to herein as a cloud computing environment 207. The cloud computing environment provides on demand availability of computer system resources like processing and storage as well as computing services, like software, analysis, and analytics. In the illustrated example, server(s) 202 may be cloud servers and database(s) 204 may be cloud databases that are deployed, delivered, and accessed in cloud 206. Cloud server(s), such as server(s) 202 may be virtual (e.g., non-physical) servers running in a cloud computing environment207. Cloud server(s) operate like physical servers and perform similar functions, such as storing data and running applications. Cloud database(s), such as database(s) 204, may organize and store structured, unstructured, and semi-structured data like a traditional on-premises database. Additionally, medical system 200 may include a user device 208, according to an example embodiment of the present disclosure.

[0086] The medical device 100 located in the medical system 200 may include an imaging device 220, such as a wireless camera, that is paired with the dialysis machine 100. In an example, the imaging device 220 may be remote from, but communicatively paired with dialysis machine 100. In another example, imaging device 220 may be physically connected to the dialysis machine 100. The imaging device 220 (e.g., wireless camera) may be configured to take photo(s) or video(s) of the physical setup (e.g., connections between sets, fluid components and accessories (priming solutions, fluid bags, syringes, disposable set connectors, etc.). The photo(s) or video(s) may be uploaded for analysis along with any other setup image data. As used herein, “setup image data” may include one or more photo(s) or video(s) captured by imaging device 220 along with any snapshot(s) or screengrab(s) from the programming information provided through the GUI, including patient information, therapy information, and prescription information. It should be appreciated that the “setup image data” may include data representations of the one or moreimage(s), photo(s), video(s), snapshot(s) or screengrab(s), such that the image data may be transformed or reformatted before being uploaded for analysis.

[0087] As illustrated in Fig. 2, processor 112 of the dialysis machine 100 is communicatively coupled to the server(s) 202, database(s) 204 and cloud 206, or more generally the cloud computing environment 207 via a network 210. The example network 210 may include a local area network, a wide area network (e.g., the Internet), a cellular network, or combinations thereof. The network 210 is configured to enable communication between devices and systems, such as communication between dialysis machine 100, imaging device 220, user device 208 and the cloud 206. For example, the network 210 may enable communication between dialysis machine 100 and user device 208 as well as enable communication between the dialysis machine 100 and cloud 206 (including server(s) 202 and database(s) 204). Similarly, network 210 enables communication between user device 208 and cloud 206 (including server(s) 202 and database(s) 204). As illustrated in Fig. 2, network 210 is depicted as dashed-line(s) illustrating some of the various communication channels enabled by the network 210.

[0088] The server(s) 202 may include a processor, a workstation, or a distributed cloud computing system. It should be appreciated that the server(s) may include physical or virtual components such as a virtual processor, virtual workstations, etc. Server(s) 202 is configured to provide management and updates for control provided by the processor 112. The server 202 may also update the data structures 114 based on input commands or selections made at the dialysis machine 100.

[0089] In some embodiments, the system 200 of Fig. 2 includes the user device 208, which may include a smartphone, a tablet computer, a desktop computer, a workstation, a smartwatch, smart eyewear, etc. The user device 208 is communicatively coupled to the processor 112 of the dialysis machine 100 via the network 210. In other instances, the user device 208 is communicatively coupled to the dialysis machine 100 via a local connection, such as Bluetooth®, Zigbee®, and / or near field communication (“NFC”).

[0090] In clinical environments, the user device 208 enables a user 55 (e.g., clinician) to remotely change settings of the dialysis machine 100 using various inputs. This may be beneficial when a patient is located in a containment room that isolates a patient’s disease. Instead of donning protective equipment, the clinician can provide inputs into the user device 208 from a remotelocation (e.g., outside the patient’s room). In an example, user device 208 is configured to transmit signals, commands, instructions, or the like to the dialysis machine 100 for processing.

[0091] In home environments, the user device 208 may be a remote control for the dialysis machine 100. Instead of having to move close to the dialysis machine 100 to provide an input, a user 55 can transmits text, a unique message, or signal of the input to the dialysis machine 100 for processing. Such a configuration may enable a user 55 to provide inputs while laying down when they are not within range to manually input changes or instructions of the dialysis machine 100.

[0092] The user device 208 includes a processor and a memory device storing machine- readable instructions. Execution of the machine-readable instructions by the processor causes an application to be executed. The application may be configured to enable a user to remotely enter inputs to the dialysis machine 100. When connected to the dialysis machine 100 after an authentication process, the application may display the same graphical user interfaces 110 that are displayed on the display screen 102. A user may select graphical buttons via the application as though they were interfacing directly with the GUI displayed on display screen 102 of the dialysis machine 100. Additionally, the user device 208 may be configured to take and upload image(s), photo(s), video(s), snapshot(s) or screengrab(s) to the cloud computing environment 207, similar to imaging device 220.Artificial Intelligence Setup Mistake / Error Detection and Error Correction

[0093] Fig. 3A illustrates an example process for detecting treatment setup mistakes or errors and providing suggestions to correct the detected setup error(s). As used herein, a setup error includes any errors, mistakes and otherwise improper or irregular selections or connections when (i) providing inputs or making selections when programming a treatment or therapy (e.g., patient information, therapy information, and prescription information) and (ii) when connecting wires, fluid lines and connectors between devices (e.g., CRRT machine and external device), other treatment components (e.g., fluid bags such as dialysate, effluent, etc.), and a patient. For example, setup errors may include errors when entering prescription data (e.g., set selection, flow rates, etc.). Additionally, setup errors may include errors when loading a set, connecting bags (e.g., dialysate, effluent, auto effluent, replacement solution, PBP solution, etc.), connecting patient lines or other patient connectors, connections to external devices (e.g., ECMO).

[0094] As illustrated in Fig. 3A, at (S301) a user 55 (e.g., nurse practitioner or clinician) selects therapy inputs, chooses sets and flow rates, etc. Specifically, the user 55 may enter patient information (e.g., patient ID, patient weight, patient hematocrit percent), therapy information (e.g., selected therapy, set information, accessories, etc.), and prescription information (set flow rates, blood flow rate, resulting treatment parameters, dose information, etc.). Additionally, at (S301) user 55 may perform wiring and fluid line connections at the dialysis machine 100. Wiring and fluid connections may include loading the set (e.g., disposable set, filter set, etc.), connecting set components, and installing accessories. Additionally, wiring and connections may include connections from dialysis machine 100 to the loaded disposable set, fluid bags, priming solutions, patient connectors, and the like. Once setup, images and / or video of the wiring and fluid line connections (i.e., setup image data) may be captured by imaging device 220.

[0095] At (S302) setup image data (e.g., images, photos, videos, etc. captured by imaging device 220 and / or screen captures from GUI or display screen 102) may be uploaded, transmitted, sent or otherwise communicated to cloud computing environment 207. Prior to capturing setup image data, the user 55 may first make a selection on GUI or the imaging device 220 indicating that assistance is needed. For example, selecting a “need assistance” icon on the GUI or pressing an “assistance” button on the imaging device 220 may then prompt the user 55 or prepare the system 300 for capturing and storing setup image data. Alternatively, the user 55 may determine assistance is needed during or after setup and may make a selection indicating that assistance is needed. Setup image data may be captured for complete setups and / or partial setups. For example, setup image data may be provided for various stages of the setup process, e.g., during any stage of programming the medical device (e.g., entering patient information, therapy information, prescription information) or during any stage of the physical setup (e.g., after loading the disposable set, during or after making connections and prior to priming, during or after priming, or connection to patient).

[0096] In an example, the GUI or imaging device may be programmed to provide instructions or guide a user when taking or capturing setup images. For example, the GUI or imaging device may enter a “capture setup image” mode where a display associated with the imaging device indicates a field of view, boundaries, or guides of where the medical device, disposable set, and certain connections should be positioned. For example, the guides may be edge or boundary guides, similar to a watermark, on an image capture program or application on theimaging device 220 or user device 208. The guides may also be provided with an instruction, “position dialysis machine within boundary provided”, “position syringe within boundary A and deaeration chamber within boundary B”, or “point camera towards dialysis machine until reference markers are scanned.” As described in more detail below, reference markers may be included on the dialysis machine 100, disposable set and other connections and components, which may also the image capture program or application to properly orient and size the captured image. In some examples, the user or nurse may be instructed to capture multiple setup images (e.g., an image of the dialysis machine 100, and image of disposable set and connections, and image of patient connections, etc.) and each capture session may have its own boundaries, guides and instructions for the user. The assistance and guidance may help ensure that appropriate setup images are captured (e.g., that the images include the components and connections associated with training areas of interest or training regions, see Figs. 7A and 7B) and that the setup images are oriented and sized appropriately for comparison against a training dataset.

[0097] At (S303), the cloud computing environment 207 analyzes the received setup image data and analyzes the data to detect any setup errors. Setup errors may include programming errors (e.g., input errors for therapy or treatment programming when entering patient information, therapy information and / or prescription information). Additionally, setup errors may include physical setup or connection errors (e.g., errors associated with loading or connecting sets, connecting fluid lines and wires, connecting fluid bags, priming solutions, patient connectors, etc.).

[0098] At (S304), if a setup error is detected, an alert may be sent to dialysis machine 100. The alert may include text, audio, image and / or video information alerting to the one or more setup error(s) detected from the uploaded setup image data. Additionally, the alert may include assistance information regarding how to correct the setup error. Specifically, text suggestions may be provided such as “connection between disposable set and blood warmer is missing” or an assistance image may be provided that highlights areas, regions or specific connections of the setup that are improper. In another example, text suggestions may be provided in conjunction with an assistance image. In other examples, assistance may be provided in the way of audio instructions, animated video instructions, annotated image instructions (See Fig. 9), or the like.

[0099] Similar to (S302), at (S305) setup image data (e.g., photos, videos, etc. captured by user device 208 and / or screen captures from GUI, display screen 102 or user device 208) may be uploaded, transmitted or communicated to cloud computing environment 207. Specifically, a user55 (e.g., nurse practitioner or clinician) may take image(s), photo(s), video(s), snapshot(s) or screengrab(s) of the setup in a mobile app running on user device 208 to upload, transmit, send or otherwise communicate the setup image data to the cloud computing environment 207 for analysis.

[0100] As noted above, the user may be assisted or guided while providing setup image data from the user device. The assistance and guidance may help ensure that appropriate setup images are captured (e.g., that the images include the components and connections associated with training areas of interest or training regions, see Figs. 7A and 7B) and that the setup images are oriented and sized appropriately for comparison against a training dataset.

[0101] Similar to (S304), at (S306), if a setup error is detected, an alert may be sent to user device 208. The alert may include text, audio, image and / or video information alerting to the one or more setup error(s) detected from the uploaded setup image data. Additionally, the alert may include assistance information regarding how to correct the setup error.Artificial Intelligence Prescription Assistance

[0102] Setting up a dialysis machine 100 for a personalized prescribed therapy or treatment can be challenging for a nurse. For example, a personalized prescription may include several different prescription elements, quantities, and the like. Some decisions involve choosing the best therapy modality, for example, choosing between a host of CRRT modalities (e.g., CWH, CVVHD, CVVHDF, SCUF) and a TPE modality. Other decisions involve selecting the proper set to use, which may be based on targeted removal and patient weight. Furthermore, anticoagulation method or strategy decisions and decisions involving flow rates are required when setting up a personalized prescribed therapy for a patient. Flow rates included in the programming for a CRRT therapy include blood flow rate, patient fluid removal (“PFR”) rate, pre and post replacement split rate, syringe flow rate, pre blood pump (“PBP”) flow rate, dialysate flow rate, replacement flow rates, calcium compensation, etc. Flow rates included in the programming for a TPE therapy include blood flow rate, patient plasma loss (“PPL”), post replacement flow rate, and citrate dose.

[0103] In some examples, a medical device, such as dialysis machine 100 may include a flow constraint calculator (“FCC”) that determines if a given flow rate falls within the limits defined for safe and accurate therapies. For example, the FCC may be configured to provide a safety check for each flow rate to ensure that each of the programmed flow rates for the selected therapy fall within safety limits. If a flow rate is initiated (e.g., flow rate is programmed orselected), adjusted or changed, the newly initiated or adjusted flow rate values are compared against newly generated safety limits because flow rate values are interdependent on other programmed flow rates as well as safety limits for each of the flow rates (e.g., flow rates and safety limits are interdependent). The interdependence of flow rates and safety limits imposes several restrictions to setting allowable prescribed flow rates for a patient’s therapy and can be quite cumbersome for nurses programming therapies. Furthermore, flow rates may be limited by the type of set selected for therapy and the therapy modality, adding to the complexity and cumbersome nature of therapy programming and setup.

[0104] In some instances, a nurse 55 may provide treatment feedback after a treatment is completed. The treatment feedback may include information regarding the patient’s condition after treatment. Additional treatment feedback may include the patient’s lab report or a system readout of the patient’s electronic medical records (“EMR”), e.g., from EMR dialysis software. The information provided may be used to enhance training datasets and improve the capabilities of the AAAM (e.g., AAAM 510 of Fig. 5A and / or AAAM 550 of Fig. 5B).

[0105] As illustrated in Fig. 3B, at (S321) a user 55 (e.g., nurse practitioner or clinician) selects therapy inputs, chooses sets and flow rates, etc. Specifically, the user 55 may enter patient information (e.g., patient ID, patient weight, patient hematocrit percent), therapy information (e.g., selected therapy, set information, accessories, etc.), and prescription information (set flow rates, blood flow rate, resulting treatment parameters, dose information, etc.). However, due to the complexities described above regarding the interdependence of both flow rates and safety limits imposed on those flow rates, along with flow rates being further limited based on set selection and therapy modality selection, the user may make errors when entering flow rates. The errors may be entering or choosing a flow rate that is outside of the safety limits.

[0106] Upon detecting a predetermined threshold of programming errors (e.g., flow rate entry or selection errors), the system 320 may inform the user 55 that assistance may be provided through the use of artificial intelligence. Specifically, an artificial intelligence agent or model, such as analysis and assistance module 550 of Fig. 5B, may be used to provide assistance in entering and selecting flow rates when programming the dialysis machine 100.

[0107] For example, at (S321) the user 55 may receive a message or alert on display 102 of the dialysis machine 100 that Al assistance is available. Alternatively, at (S321), the user 55 may make a selection (e.g., selecting a “need assistance” icon on the GUI displayed on display 1screen 102) to request help or assistance in entering prescription information, such as flow rate information. Then, the user 55 may be prompted to enter patient information 410 (e.g., weight, age, patient vitals) along with other patient lab or condition information 418 (e.g., lab report parameters, such as serum creatinine, BUN, GFR, urine report, patient urine output, etc.). As noted above, prior to entering prescription information or patient information, the user 55 may first make a selection on the GUI indicating that assistance is needed. For example, selecting a “need assistance” button or icon on the GUI may then prompt the user 55 or prepare the system 320 for receiving and storing prescription information or patient information. Alternatively, the user 55 may determine assistance is needed during or after setup and may make a selection indicating that assistance is needed at various stages of the treatment or therapy programming process.

[0108] At (S322) the patient details (e.g., patient information 410) collected at (S321) may be uploaded, transmitted, sent or otherwise communicated to cloud computing environment 207. The patient details may be encoded, encrypted, or reformatted before being uploaded to the cloud computing environment 207.

[0109] Then, at (S323), the cloud computing environment 207 analyzes the received patient details and analyzes the data to determine an optimal (e.g., best available) prescription including modality, set and flow rates. For example, an AAAM 550 of Fig. 5B may be hosted in the cloud computing environment 207 and may be configured to perform the analysis at (S323).

[0110] At (S324), optimal prescription information including modality, set and flow rates is sent from cloud computing environment 207 to the dialysis machine 100. In an example, the information may be provided as a text file or a programming assistance image or audio file. In another example, the optimal prescription information may be provided and automatically programmed into the dialysis machine 100 for the user 55 to review and confirm.

[0111] Along with the optimal prescription information, the cloud computing environment 207 (e.g., AAAM 550 hosted in the cloud) may also provide possible outcomes of the optimal prescription information, including clearance, which may also be displayed on the GUI once received at (S324). After confirming the new prescription (e.g., user 55 reviewing and confirming proposed prescription), the new prescription may be applied to the system 320, and more specifically dialysis machine 100.

[0112] In some examples, the provided optimal prescription information may include indications on the GUI on display 102 for any portions of the prescription information that is changed from the user’s initial inputs (e.g., entry fields highlighted or annotated). Furthermore, additional details and explanations regarding the provided optimal prescription information may also be included. For example, the cloud computing environment 207, and more specifically the AAAM 550 executing in the cloud may provide annotations or notes for selections (e.g., “flow rate of “X” is ideal for “Y” treatment modality and “Z” set for patient’s weighting “AB to CD” lbs.). Specifically, the optimal prescription information may include additional information regarding the acceptable limits or ranges for each of the flow rates, etc.Artificial Intelligence Prescription Assistance Model Improvement and Training

[0113] Referring now to Fig. 3C, after the treatment is completed, at (S331) the user 55 may enter treatment outcome information, such as user feedback (e.g., nurse feedback on the treatment outcome), patient information, other patient details (e.g., lab report parameters), and prescription parameters that were entered or selected in the GUI of display 102 when programming the treatment or therapy. The treatment outcome information may include each of the items described in Fig. 4C corresponding to treatment outcome information 460. The user feedback may include information regarding how the treatment went, the type and quantity of alarms triggered or alerts received, information regarding the patient’s condition (e.g., whether the patient’s condition improved, condition worsened or deteriorated, or condition remained unchanged or stayed the same), and details and information regarding fluid removal.

[0114] Referring briefly to Fig. 4C, the treatment outcome information 460 may include a treatment outcome status 470. Treatment outcome status 470 may be a condition improved status 572, a condition worsened status 474, or a condition unchanged status 476. Additionally, the treatment outcome information 460 may include lab report parameters 478. Uab report parameters 478 may include information regarding glomerular filtration rate (“GFR”), creatinine level, blood pressure (“BP”), sugar levels, complete blood count (“CBC”), blood urea nitrogen (“BUN”), electrolyte levels (including calcium and phosphate). Additionally, the lab report parameters 478 may include information about the patient’s condition, such as diabetic ketoacidosis, gastrointestinal bleeding, drug overdose, respiratory failure, sepsis, stroke, cancer, etc.

[0115] As illustrated in Fig. 4C, treatment outcome information 460 may also include alarm information 480, such as information about the type of alarm triggered during treatment or therapy (e.g., alarm type 482) and the quantity of alarms triggered during treatment or therapy (e.g., alarm quantity 484). The treatment outcome information 460 may also include prescription parameters 490 and fluid removal information 494. The prescription parameters 490 may include parameters that were entered or selected in the GUI of display 102 when programming the treatment or therapy and may be similar to or the same as the prescription information 430 of Fig. 4A. The fluid removal information 494 may include information regarding the patient fluid removal (“PFR”) rate or volume of fluid removed during therapy.

[0116] Referring back to Fig. 3C, at (S332), the treatment outcome information collected at (S331) may be uploaded, transmitted, sent or otherwise communicated to cloud computing environment 207. The treatment outcome information may be encoded, encrypted, or reformatted before being uploaded to the cloud computing environment 207.

[0117] Then, at (S333), the cloud computing environment 207 receives the treatment outcome information. Additionally, at (S333) the cloud computing environment 207 (e.g., AAAM 550) may classify, organize, and save at least some of the treatment outcome information to enhance training datasets used by AAAM 550 and to improve the capabilities of the AAAM 550. For example, by using the treatment outcome information, the Al assistance may be further trained to provide more accurate recommendations and to further fine-tune the model. Some example Al architectures and models are described in more detail below.Artificial Intelligence Analysis and Assistance Modules

[0118] Referring now to Fig. 5 A, an artificial intelligence (“Al”) analysis and assistance module 510 may be hosted in cloud computing environment 207 to perform various steps illustrated in Fig. 3 A and described in more detail above. For example, the AAAM 510 may be configured to compare received image(s), photo(s), video(s), snapshot(s) or screengrab(s) against images contained within training datasets to detect setup mistakes and errors and to provide error correction suggestions.

[0119] The AAAM 510 may include one or more modules and may execute one or more Al algorithms, models, services or agents. In the illustrated example, the AAAM 510 may include an association module 512, a data normalization module 514, an image alignment module516 and an image resizing module 518. The association module 512 may be configured to identify the proper training dataset for the provided input data. Additionally, the association module may identify the types of input data received and what portions of the training dataset to compare the input data to. The data normalization module 514 may be configured to normalize or reformat the input data such that it matches the format of the related data in the training dataset. The data normalization module 514 may also be configured to convert input images or videos to grayscale or adjust other image properties (e.g., brightness, contrast, sharpness, pixel density, etc.). In other examples, the input data may be normalized or reformatted at the device level (e.g., imaging device 220, medical device 100, user device 208) prior to being uploaded to the cloud computing environment 207 and / or the AAAM 510.

[0120] The AAAM 510 may also include an image alignment module 516 and an image resizing module 518. The image alignment module 516 is configured to align features, markers and / or regions of interest from input images with corresponding features, markers and / or regions of interest of images in the training dataset. The image resizing module 518 is configured to resize an image such that the input image is appropriately sized according to an associated training dataset image. In an example, the image resizing module 518 may similarly use the alignment features, markers or regions of interest to determine howto resize an image (e.g., enlarge an image or shrink an image such that alignment features of the input image and alignment features of the training dataset image are appropriately sized and spaced), thereby allowing the right-sized input image to be aligned with a training dataset image.

[0121] As mentioned above, image data obtained by the imaging device 220 or user device 208 may include reference data, such as one or more reference features and / or reference markers. A reference feature may be a distinct feature on the medical device, set or connector. For example, a reference feature may be one or more corners or edges of a syringe holder, a filter pressure pod, a deaeration chamber or any other features that are easily identifiable and in a relatively fixed position with respect to at least one other reference feature. A reference marker may be a barcode or other marker that is adapted to assist with alignment and positioning of an input image with a training dataset image. The reference markers may be visual objects with data encoding ability, such as barcodes or square matrix codes (e.g., ID, 2D or the 3D barcodes or square matrix codes), for example a QR code and may function similar to a finder marker, alignment marker, finder pattern and alignment pattern of a QR code. For example, the medicaldevice (e.g., dialysis machine 100), set, connectors and / or accessories may include reference markers. The reference markers may be positioned in predetermined locations about the medical device, set, connectors and / or accessory equipment. The reference markers or information regarding their locations and / or positions are also included in the training dataset images such that input images can be aligned and compared against images of an associated training dataset (e.g., a training dataset image).

[0122] The AAAM 510 may also include an image analysis module 520, which is configured to analyze and compare input data from image(s), photo(s), video(s), snapshot(s) or screengrab(s) to training set data (e.g., image data for a “correct setup”). The image analysis module 520 may utilize a library, such as OpenCV in Python, which allows image processing tasks, such as image comparison. The Absolute Difference method in OpenCV is an image comparison technique that calculates the absolute pixel-wise difference between two images. Specifically, the Absolute Difference method measures the discrepancy between corresponding pixels in images without considering their context or structural information. For example, the image analysis module 520 may analyze and compare input images / data to image data for a “correct setup” from the appropriate training dataset. To perform image analysis and comparison, it is important to ensure that the images have the same dimensions, such that resizing an image may be necessary (e.g., via the image resizing module 518).

[0123] The absolute difference between the images may be calculated using the “diff’ function (diff = cv2.absdiff(imagel, image2)). The diff function subtracts the pixel values of the second image from the pixel values of the first image, resulting in an image the represents the absolute difference between the first and second images. Optionally, the difference image may be converted into grayscale (e.g., via data normalization module 514) for better visualization. Further examination or inspection of the difference image (e.g., via image analysis module 520) may indicate areas where the two images differ. It is important to note that the Absolute Difference model does not consider the context or structural information of the images and thus may not be suitable for all image comparison tasks. For larger data sets, a machine learning mode may be built and trained to detect faults using a neural network (e.g., neural network 530, described in more detail below).

[0124] The AAAM 510 may also include or comprise a neural network 530. In another example, AAAM 510 may be built and trained as a neural network regression model (e.g.,neural network multiple regression model 605 of Figs. 6A and 6B). Building and training the model may include data collection, data processing, model design, model training, model evaluation and model optimization. Data collection may include collecting data (e.g., one or more datasets) containing relevant information about medical devices associated with the model. For example, the dataset may include input features (e.g., patient specifications, therapy data, etc.) and corresponding target values (e.g., device prescribed flowrates). Data processing may include processing the collected data (e.g., one or more datasets) to prepare the data (e.g., training dataset) for training the neural network 530. For example, the data processing may include normalizing and / or standardizing the data (e.g., standardizing features), handling missing data, encoding categorical variables, and splitting the data into training and / or testing sets.

[0125] Referring now to Fig. 5B, an artificial intelligence (“Al”) analysis and assistance module 550 may be hosted in cloud computing environment 207 to perform various steps illustrated in Fig. 3B, which is described in more detail above. For example, the AAAM 550 may be configured to compare received treatment input data (e.g., treatment input data 440 of Fig. 4B) against training datasets to detect entry errors or empty fields in treatment input data and to provide an optimal prescription plan or other therapy suggestions. The optimal (e.g., best available) prescription plan may include a selected modality, set, flow rates, etc.

[0126] The AAAM 550 may include one or more modules and may execute one or more Al algorithms, models, services or agents. In the illustrated example, the AAAM 550 may include an association module 552, a data normalization module 554, and a programming analysis module 560. The association module 552 may be configured to identify the proper training dataset for the provided input data (e.g., treatment input data 440). Additionally, the association module 552 may identify the types of input data received and what portions of the training dataset to compare the input data to. The data normalization module 554 may be configured to normalize or reformat the input data (e.g., treatment input data 440) such that it matches the format of the related data in the training dataset. In other examples, the input data may be normalized or reformatted at the device level (e.g., medical device 100 or user device 208) prior to being uploaded to the cloud computing environment 207 and / or the AAAM 550.

[0127] The AAAM 510 may also include a programming analysis module 560, which is configured to analyze and compare input data (e.g., treatment input data 440) to training set data (e.g., input data for a “correct therapy, treatment or prescription”).

[0128] The AAAM 550 may also include or comprise a neural network 570. In another example, AAAM 550 may be built and trained as a neural network regression model (e.g., neural network multiple regression model 605 of Figs. 6A and 6B). Building and training the model may include data collection, data processing, model design, model training, model evaluation and model optimization. Data collection may include collecting data (e.g., one or more datasets) containing relevant information about medical devices associated with the model. For example, the dataset may include input features (e.g., patient specifications, therapy data, etc.) and corresponding target values (e.g., device prescribed flowrates). Data processing may include processing the collected data (e.g., one or more datasets) to prepare the data (e.g., training dataset) for training the neural network 570. For example, the data processing may include normalizing and / or standardizing the data (e.g., standardizing features), handling missing data, encoding categorical variables, and splitting the data into training and / or testing sets.Artificial Intelligence Architecture and Models

[0129] Artificial intelligence model design may include choosing an architecture and / or design of the Al or neural network. For multiple regression, one type of architecture and / or design is a feed-forward neural network model (“NNM”), as illustrated in Fig. 6A. The NNM 605, or more generally model 605, illustrated in Fig. 6A includes multiple input nodes 614 (e.g., 614a, 614b and 614 c in the input layer 620), at least one hidden layer 630 with a plurality of nodes 616, and output nodes 618 (e.g., 618a and 618b in the output layer 640) for the predicted target variable. Once the model design is determined, the NNM 605 may be trained. Specifically, model training may include training the neural network using the prepared training dataset. During training, the NNM 605 learns the relationships between the input features and the target values. Relationships may be learned and developed by optimizing the model’s weights and biases using a suitable optimization technique (e.g., gradient descent) and a defined loss function (e.g., mean squared error).

[0130] Each of the nodes 616 may represent a calculation or determination based on various node inputs 662 (e.g., inputs 662a-c of Fig. 6B), weights 664 (e.g., weights 664a-c of Fig. 6B), and function(s) 670 (e.g., activation function 670a of Fig. 6B) to determine a node output 668 (e.g., node output 668a of Fig. 6B).

[0131] It should be appreciated that the architectures and models described in Figs.6A and 6B may be implemented as part of AAAM’s 510 and 550 described in Figs. 5A and 5B. Additionally, it should be appreciated that the architectures and models described in this section and with reference to Figs. 6A and 6B may be hosted in cloud computing environment 207 and provide many of the features and functions described with respect to the cloud computing environment 207, AAAM 510, 550 or any other Al assistance algorithms, models, services or agents described herein.

[0132] Specifically, input(s) 610 may be provided to the model 605. The input(s) may include patient information 410 and more specifically patient lab / condition information 418 (e.g., pathology report, patient condition), patient weight 414 and patient hematocrit percent 416. For example, inputs 610 may refer to all or some of the information received at step (S302) in Fig. 3A, step (322) in Fig. 3B, or step (332) in Fig. 3C.

[0133] The input layer 620 is the first layer of the NNM 605 and the input nodes 614 represent the features or attributes of the input data from input 610. For example, each node 614 in the input layer 620 may correspond to a specific feature, and the values at the input nodes 614 are associated with the input data. For example, input node 614a may include values associated with the patient’s pathology report, while input node 614b has values associated with the patient’s condition and input node 614c has values associated with the patient’s weight 414. It should be appreciated that NNM 605 may include additional input nodes than those illustrated in Fig. 6A (e.g., an input node 614 with values associated with the patient’s hematocrit percent 416, etc.).

[0134] Depending on the input(s) 610, weights associated with each of the input(s) 610, and other training information, the NNM 605 provides an output(s) 650. The output(s) 650 may include suggested therapy types, set types, accessories, and flow rates. To update the model, the output(s) 650 may be passed back though the model via a therapy feedback loop 652 to update, fine-tune and further train NNM 605. For example, the NNM 605 may provide output(s) 650 similar to those provided at (S324) in Fig. 3B. Additionally, the feedback loop 652 may correspond to the fine-tuning and updates described at (S333) in Fig. 3C.

[0135] Model evaluation includes evaluating the trained model’s performance using the testing dataset. In an example, evaluation may include calculating relevant metrics, such as mean squared error, mean absolute error, or R-squared value to assess the model’s accuracy inpredicting the target variable. Model optimization includes fine-tuning the model by adjusting hyper-parameters and / or modifying the network architecture.

[0136] Referring to Fig. 6B, each node (e.g., nodes 614, 616, 618 of Fig. 6A) may be a neural node and may perform calculations within the node. For example, in the first layer or input later, a node may perform a calculation based on various inputs 662a-c, weights 664a-c and function parameters (e.g., parameters of activation function 670a) to provide a node output 668a. inputs may be referred to generally as inputs 662, weights may be referred to generally as weights 664, activation functions may be referred to generally as activation function 670, and outputs may be referred to generally as output 668. The node output 668 is passed on to the next node and becomes one of the inputs 662 for that node. Specifically, as illustrated in Fig. 6B, output 668a may become input 662e. In another example, output 668a may form all or part of inputs 662d-f in conjunction with other outputs from nodes within the hidden layer 630. In a subsequent calculation layer, node inputs 662d-f may be associated with weights 664d-f, which may be passed to function 670b to generate node output 668b.

[0137] Additionally, referring back to Fig. 6A, node 614a may provide a node output 668 that is passed on to node 616a in the second layer or one of the hidden layers 630. The node output 668 from node 614a may be treated as a node input 662 by node 616a. Furthermore, the node output 668 from node 614a may be provided to each node in the next layer such that the node output 668 from node 616a is passed to node 616a and the three other nodes illustrated below node 616a in the first hidden layer 630. Specifically, each node in a subsequent layer (e.g., second layer) may receive a node input 662 from each node in previous layer (e.g., first layer), and the node input 662 may be the node output 668 of the previous layer. The node calculations continue until a final output (e.g., output 650) is reached. In the illustrated example of Fig. 6B, the node outputs 668 from nodes 618a and 618b in the output layer 640 may be combined or used to form output 650.

[0138] Node calculations (e.g., calculations performed within a node) may consist of two parts. A first part of the calculation may include multiplication of all of the node inputs 662 by their respective weights 664 (e.g., “input_l” 662a associated with “weight_l” 664a, “input_2” 662b associated with “weight_2” 664b, “input_3” 662c associated with “weight_3” 664c and so on). In the first part of the calculation, each input 662 may be multiplied with a corresponding weight 664 and the outcomes may be summed. In a second part of the calculation, the node maybe associated with an activation function 670 that allows the network or model (e.g., model 605, such as AAAM 510, 550) to perform more complex calculations and solve more advanced problems.

[0139] After applying both parts of the node calculation, the node output 668 is passed on to multiple nodes in the next layer. In a neural network architecture, like the one illustrated in Fig. 6A, the hidden layers 630 may be fully connected (e.g., each node in the first hidden layer 630 is connected to each node in the second hidden layer 630, which is connected to each node in the third hidden layer, etc.).

[0140] In an example, the model 605 illustrated in Fig. 6A, which may make up all or part of the AAAM 510, 550 or neural network 530, may be built from various inputs 662. The inputs 662 may include various input parameters, which may include patient information 410 and patient lab / condition information 418, which as discussed above may include lab report information, such as glomerular filtration rate (“GFR”), creatinine level, blood pressure (“BP”), sugar levels, complete blood count (“CBC”), blood urea nitrogen (“BUN”), electrolyte levels (including calcium and phosphate). Additionally, the patient lab / condition information 418 may include information about the patient’s condition, such as diabetic ketoacidosis, gastrointestinal bleeding, drug overdose, respiratory failure, sepsis, stroke, cancer, etc.Al Model for Prescription Assistance

[0141] This section describes the features and capabilities of a model 605 (described in Figs. 6A and 6B) and may correspond to the features and capabilities of AAAM 550 illustrated in Fig. 5B and the cloud computing environment 207 illustrated in Figs. 3B and 3C. Based on the input selected, the model 605 (e.g., AAAM 550) may be configured to determine and suggest an optimal (e.g., best available) prescription or therapy including modality, set and flow rates based on the therapy data the model 605 (e.g., AAAM 550) is trained on. In an example, the model 605 (e.g., AAAM 550) may provide an output 650 or suggestion that contains therapy information such as suggested therapy, modality, set, and an appropriate anticoagulation accessory. The model 605 (e.g., AAAM 550) may also be configured to provide a suggestion for ideal flowrates based on the patient condition. For example, the model 605 (e.g., AAAM 550) may suggest the following flowrates and parameters, “[PBP -100, BFR -180, Dia - 1000, Replacement, Syringe].” To improve performance of the model 605 (e.g., AAAM 550), the model 605 (e.g.,AAAM 550) may be continuously or incrementally adjusted (e.g., improved) with a feedback loop based on the input received from the feedback loop 652.

[0142] The table below provides an example input (e.g., input parameters) and output (e.g., AAAM suggestion) using the model 605 (e.g., AAAM 550) described above. In the example illustrated in Table 1 below, the pathological information, patient condition, patient weight and hematocrit value assist the model with understanding therapy requirements. Based on the model’s 605 (e.g., AAAM 550) understanding of the therapy requirements, an output 650 or suggestion of the optimal (e.g., best available) treatment or therapy is provided, which may include therapy parameters. The suggested therapy and / or therapy parameters may be based on clinical data (e.g., past clinical data) the model 605 (e.g., AAAM 550) is trained on. In the case below, the patient has increased creatinine levels and decreased urine output, but the patient condition is stable. Taking these parameters into consideration a suggested therapy, therapy modality, set type, flowrates, etc. are provided by the model 605 (e.g., AAAM 550). The flowrates may be optimized for the specific therapy, modality and based on patient weight.

[0143] Table 1 - AAAM Therapy Suggestion

[0144] As illustrated above, the model 605 (e.g., AAAM 550) provides a suggested therapy of CRRT, a suggested modality of CVVHDF, a suggested set (e.g., M100), a suggested anticoagulation method (e.g., systemic), suggested or required accessories based on therapy andmodality (e.g., Thermax) as well as various suggested flow rates, which are shown in Table 1 in units of mL / h. In the example illustrated in Table 1 above, the syringe is used to infuse heparin to the blood circuit.

[0145] Table 2, provided below is another example input 610 (e.g., input parameters) and output 650 (e.g., AAAM suggestion) using model 605 (e.g., AAAM 550). In the example illustrated in Table 2 below, the pathological information, patient condition, patient weight and hematocrit value are input and the model 605 (e.g., AAAM 550) provides an output 650 based on the inputs, clinical data, etc.

[0146] Table 2 - AAAM Therapy Suggestion

[0147] In the case illustrated in Table 2 above, the patient has increased creatinine levels, but the urine output is in an acceptable range (e.g., urine output is not at an alarming level). In the above example, the patient is experiencing respiratory failure and sepsis. Taking these parameters into consideration a suggested therapy, therapy modality, set type, flowrates, etc. are provided by the model 605 (e.g., AAAM 550). Specifically, the model 605 (e.g., AAAM 550) suggests a CRRT with extracorporeal carbon dioxide removal (“ECOO2R”) therapy along with an Oxiris set. ECOO2R assists with respiratory problems and Oxiris sets are typically used for sepsis treatments. In the example illustrated in Table 2 shows a Regional Calcium Anticoagulation(“RCA”) anticoagulation strategy where Calcium is infused using a syringe. The percentage (e.g., 100%) next to the syringe (“Syr”) output parameter indicates how much of computed Calcium is to be infused and here the setting is set to 100%, which may be a default setting.Al Model for Setup Mistake / Error Detection and Error Correction

[0148] This section describes the features and capabilities of a model 605 (described in Figs. 6A and 6B) and may correspond to the features and capabilities of AAAM 510 illustrated in Fig. 5A and the cloud computing environment 207 illustrated in Fig. 3 A. Setup errors may be identified through a rule-based approach associated with a training dataset. The training dataset may include data from image(s), photo(s), video(s), snapshot(s) or screengrab(s) or other image data for a “correct setup”. As used herein, a “correct setup” is a treatment or therapy setup that uses proper inputs (e.g., proper patient information 410, therapy information 420, prescription information 430), proper equipment and accessories (e.g., proper set, proper connectors, proper accessory devices), and proper connections (e.g., fluid connections, electrical connections, data connections, patient connections). There may be a training dataset associated with each specific “correct setup”, which may differ by therapy selected, set selection, accessory selection, etc. For example, a plurality of training datasets may be associated with various setups for different therapies, different sets and / or different accessories. In some examples, some setup configurations may share all or part of a training data set with another setup configuration. Furthermore, training datasets may be built from a combination of training data sub-sets, such that a training data set comprises a plurality of training data sub-sets and the training dataset for one therapy configuration may include one or more of the same training data sub-sets as a different therapy configuration.

[0149] The training dataset(s) are built from “correct setup” data, which may include data from: screengrabs of proper therapy and prescription information inputs, images of the medical device, disposable set, and any associated accessories properly connected. As additional images are uploaded for analysis and review, the training dataset may be adjusted (e.g., fine-tuned) based on the additional information. For example, referring back to Fig. 3A, additional data from image(s), photo(s), video(s), snapshot(s) or screengrab(s) or other image data provided by a user during (S302) or (S305) may be used to further train or fine-tune the training dataset.

[0150] After analysis, the model 605 or more specifically the analysis module 520 may provide an output that identifies a fault based on the rules established by the training dataset.The rules may be created and based upon treatment parameters or setup conditions, such as the therapy selected, cartridge or set used, accessories (e.g., blood warmer) that are part of the therapy, etc. The rules may be wiring, connection, orientation, placement, or value rules that indicate when a setting or piece of equipment is violating a rule. Once analyzed, in the case of an improper setup, the model 605 (e.g., image analysis module 520 of AAAM 510 running in the cloud computing environment 207 may output an image describing the errors or faults in the initial setup and may provide the image with indicators (e.g., highlighted boundaries) to further draw the user’s attention to the error or fault. An example of the input images (see Fig. 8) and the output image with issues highlighted (see Fig. 9) are illustrated in Figs. 8 and 9.

[0151] Example rule may be a check that: (i) fluid lines need to be visible and exiting from both sides of a blood leak detector, (ii) the filter pressure pod is attached, (iii) syringe is placed in syringe holder (e.g., for therapy that has anticoagulation with syringe selected), etc. For example, referring briefly to Fig. 9, error indication (E901) may be triggered because rule (iii) above is broken (e.g., a syringe is missing and not positioned in the syringe holder). There may be hundreds of rules based on the therapy selected and accessories used. Typically, the rules are applied after extracting features and other information from the input image illustrated in Fig. 8 before generating the output image illustrated in Fig. 9, which may include error indications for rules that are broken.Training Images for Training Dataset

[0152] Fig. 7A illustrates an example training image 700A used in a training dataset, and more specifically an example training image for CRRT with systemic anticoagulation. In the illustrated example, a proper setup may include various connections, such as connections of an effluent line, an effluent pump, a syringe, a syringe holder, an auto effluent set, an affluent line (e.g., an affluent line from a filter), an effluent pressure pod, a return pod, a deaeration chamber, an access line from a patient, a blood pump assembly, a fluid line from the filter, a filter pressure pod, a return line and associated retaining clamp, and an access pressure pod.

[0153] In the illustrated example, the training image 700A includes various training areas of interest or training regions with proper setup. As shown in Fig. 7A, training region (T701) is used to verify the effluent line is placed properly within the effluent pump setup, training region (T702) is used to verify the syringe is placed properly in the syringe holder, training region (T703)is used to verify the auto effluent set is loaded and the effluent line is connected, training region (T704) is used to verify the effluent line from the filter is connected to the effluent pressure pod, training region (T705) is used to verify the return pod is attached, training region (T706) is used to verify the deaeration chamber is attached, training region (T707) is used to verify the access line from the patient is properly placed with the blood pump assembly, training region (T708) is used to verify the fluid line from the filter is connected to the filter pressure pod, training region (T709) is used to verify the return line is positioned inside the retaining clamp, training region (T710) is used to verify the access line from the patient is connected to the access pressure pod.

[0154] Fig. 5B illustrates an example training image 700B used in a training dataset, and more specifically an example training image for CRRT with systemic anticoagulation. In the illustrated example, a proper setup may include various connections, such as connections of an auto drain line, an effluent line, an effluent bag, a pre-blood-pump (“PBP”) bag, a PBP scale, a first colored-coded fluid line (e.g., white color-coded line) on a PBP pump, a dialysate bag, a dialysate scale, a second colored-coded fluid line (e.g., green color-coded line), a dialysate pump, a fluid bag, a replacement scale, a color-coded fluid line (e.g., a purple color-coded line), a replacement pump, a replacement bag, a replacement line, a dialysate line and associated clamping status, and a PBP line and associated clamping status.

[0155] In the illustrated example, the training image 700B includes various training areas of interest or training regions with proper setup. As shown in Fig. 7B, training region (T721) is used to verify the auto drain line is connected, training region (T722) is used to verify the effluent line is connected to the effluent bag, training region (T723) is used to verify the proper PBP bag is supported or attached (e.g., hung from machine), training region (T724) is used to verify the PBP bag on the PBP scale is connected to the white color-coded line going to the PBP pump, training region (T725) is used to verify the PBP line is not clamped, training region (T726) is used to verify the proper dialysate bag is supported or attached (e.g., hung from machine), training region (T727) is used to verify the proper replacement bag is supported or attached (e.g., hung from machine), training region (T728) is used to verify the bag on the dialysate scale is connected to the green color-coded line going to the dialysate pump, training region (T729) is used to verify the replacement bag on the replacement scale is connected to the purple color-coded line going to the replacement pump, training region (T730) is used to verify the replacement line is not clamped, and training region (T731) is used to verify the dialysate line is not clamped.

[0156] In an example, the image analysis may utilize a library, such as OpenCV in Python, which allows image processing tasks, such as image comparison. The Absolute Difference method in OpenCV is an image comparison technique that calculates the absolute pixel-wise difference between two images. Specifically, the Absolute Difference method measures the discrepancy between corresponding pixels in images without considering their context or structural information. To perform image comparison, the images to be compared are loaded into the database. It is important to ensure that the images have the same dimensions, such that resizing an image may be necessary.Example Input Setup Images and Output Correction Images

[0157] Fig. 8 illustrates an example setup image 800 that may make up all or part of the setup image data. For example, setup image 800 may be the image captured by imaging device 220 or captured by user device 208 and then uploaded to the cloud computing environment at steps (S302) and (S305) in Fig. 3A. After the example setup image 800 is uploaded to the cloud computing environment 207 for analysis, the model 605 (or more specifically image analysis module 520 of AAAM 510) may provide an output image 900, as illustrated in Fig. 9. In the illustrated example, the output image 900 may include error indications (e.g., annotations, highlights, color-coding, framed boarders, etc.) of any issues, mistakes or otherwise improper setup items determined from the analysis.

[0158] In the illustrated example in Fig. 9, the output image 900 includes error indications (E901) highlighting that a syringe is missing and not positioned in the syringe holder, error indication (E902) highlighting that an extracorporeal CO2 removal (“ECCO2R”) cartridge is missing, error indication (E903) highlighting that the line connecting to the ECCO2R cartridge is not attached, and error indication (E904) highlighting that there is a fluid wiring error at the connection to the deration chamber. The error indications may be accompanied by other annotations or text instructions regarding how to correct errors or provide additional details about the possible errors. For example, an annotation may include an instruction that states “syringe missing, place Syringe model Y into the syringe holder and connect to syringe line.”Al Assistance Methods and Process Flows

[0159] Fig. 10A is a diagram of an example process 1000 performed by one or more components of Fig. 3 A and optionally AAAM 510 of Fig. 5A or neural network model 605 of Figs. 6A and 6B, to identify an error in a setup (e.g., error with wiring, fluid connections, etc.) and providing an error correction suggestion to rectify the identified setup error, according to an example embodiment of the present disclosure. In the illustrated example, the process 1000 begins when the imaging device 220 captures setup image data for a treatment or therapy setup on a dialysis machine (block 1002). In an example, the setup image data may be captured as one or more images, photos, videos, snapshots or screengrabs (e.g., a series of photos of various fluid connections between the dialysis machine, disposable set, and patient). The imaging device 220 may be a camera that is communicatively coupled to the dialysis machine, such as a wireless camera in communication with the dialysis machine 100 and / or a cloud computing environment through a network connection 210. The imaging device 220 may also be physically coupled (e.g., via a data cable) to the dialysis machine 100 or may be an integrated component of the dialysis machine 100. In other examples, the imaging device 220 may be a mobile user device 208, such as a smartphone with an integrated camera.

[0160] Next, the captured setup image data is sent to a cloud computing environment 207 (block 1004). For example, the imaging device 220 or the dialysis machine 100 may send the setup image data captured at block 1002 to the cloud computing environment 207, and more specifically artificial intelligence hosted in the cloud computing environment 207. Sending the setup image data may include uploading the image data, transmitting the image data, or otherwise communication the image data to the cloud computing environment 207. In some examples, the setup image data may be reformatted prior to being sent. For example, the setup image data may be resized, have properties adjusted (e.g., contrast, color, etc.), encoded or encrypted before the imaging device 220 or dialysis machine 100 sends the setup image data to the cloud computing environment 207. Then, the cloud computing environment receives the setup image data (block 1006). For example, the cloud computing environment 207, or more specifically an AAAM 510 hosted by the cloud computing environment 207, may receive the setup image data sent at block 1004.

[0161] Next, the cloud computing environment 207 analyzes the setup image data (block 1008). For example, the cloud computing environment 207, or more specifically an AAAM 510 hosted by the cloud computing environment 207, may analyze the setup image data receivedat block 1006. Analysis may be performed by an image analysis module 520. In some instances, the AAAM 510 may first associate the setup image data with appropriate data from a training dataset before analysis takes place. Additionally, AAAM 510 may normalize the setup image data (e.g., reformat, standardize, etc. the setup image data via data normalization module 514), resize the setup image data (e.g., adjust the size of the setup image data to match that of corresponding image data from the training dataset via an image resizing module 518), and align the setup image data (e.g., align the setup image data with corresponding image data from the training dataset via an image alignment module 516). Analyzing the image data may include creating a difference image, comparing image pixels in various regions of interest (e.g., training regions illustrated in Figs. 7A and 7B), etc. The difference image may be created using an absolute difference method.

[0162] The example process 1000 continues with identifying an error in the setup image data (block 1010). For example, the cloud computing environment 207, or more specifically an AAAM 510 hosted by the cloud computing environment 207, may identify an error in the setup image data received at block 1006 and analyzed at block 1008. An error may be determined if a difference image of grayscale versions of the setup image data and corresponding image data from the training dataset exceeds a predetermined threshold. Specifically, an error may be detected if pixel values of the difference image for a specific region of interest (e.g., training region, such as training region T701 of Fig. 7A) exceed a predetermined threshold.

[0163] Then, an error correction suggestion is provided to at least one of the imaging device 220 and / or the dialysis machine 100 (block 1012). For example, the cloud computing environment 207, or more specifically an AAAM 510 hosted by the cloud computing environment 207, may send an error correction suggestion, which may be displayed on the imaging device 220 or the display 102 of the dialysis machine 100. The error correction suggestion may include an image with annotations, text instructions, or audio instructions on suggested corrective actions to rectify the identified error in setting up the dialysis machine 100 for treatment. For example, the error correction suggestion may be an annotated image, like the one illustrated in Fig. 9 that highlights areas of improper connections.

[0164] In some embodiments, block 1002 may be performed solely by an imaging device 220, solely by a mobile user device 208, or a combination thereof. For example, the imaging device 220, such as a wireless camera, may be used to capture images or videos of fluid, electrical and data connections between the dialysis machine 100, disposable set and other fluid lines, bagsand connectors while the mobile user device 208 is used to capture snapshots or screengrabs of programming setup information 400. In an example, the screengrabs may be of the inputs and selected menu options from the graphical user interface displayed on display 102 of the dialysis machine 100. It should be appreciated that screengrabs of the inputs and selected menu options (e.g., programming setup information 400) may be obtained directly on dialysis machine 100, e.g., the dialysis machine 100 may include a selectable icon on the GUI or a button on the dialysis machine 100 to capture a snapshot or otherwise obtain the programming setup information 400.

[0165] Fig. 10B is a diagram of an example process 1020 performed by one or more components of Fig. 3B and optionally AAAM 550 of Fig. 5B or neural network model 605 of Figs. 6A and 6B, to identify an error or empty field in treatment input data 440 and providing an optimal prescription plan that corrects errors and suggests inputs for empty fields, according to an example embodiment of the present disclosure. In the illustrated example, the process 1020 begins when the dialysis machine 100 receives treatment input data 440 (block 1022). The treatment input data 440 may be the same or similar to the programming setup information 400 of Fig. 4A. For example, the treatment input data 440 may include patient information 410, therapy information 420 and prescription information 430. The treatment input data may also include the inputs and values associated with the inputs illustrated in Table 1 and Table 2 above, such as urine volume information 442, creatinine level information 444, and / or patient condition information 446.

[0166] Next, the treatment input data 440 is sent to a cloud computing environment 207 (block 1024). For example, the dialysis machine 100 may send the treatment input data 440 received at block 10202 to the cloud computing environment 207, and more specifically artificial intelligence hosted in the cloud computing environment 207. Sending the treatment input data 440 may include uploading the input data, transmitting the input data, or otherwise communicating the input data to the cloud computing environment 207. In some examples, the input data may be reformatted or standardized prior to being sent. Additionally, the treatment input data 440 may be encoded or encrypted before the dialysis machine 100 sends the treatment input data 440 to the cloud computing environment 207. Then, the cloud computing environment receives the treatment input data 440 (block 1026). For example, the cloud computing environment 207, or more specifically an AAAM 550 hosted by the cloud computing environment 207, may receive the treatment input data 440 sent at block 1024.

[0167] Next, the cloud computing environment 207 analyzes the treatment input data 440 (block 1028). For example, the cloud computing environment 207, or more specifically an AAAM 550 hosted by the cloud computing environment 207, may analyze the treatment input data 440 received at block 1026. Analysis may be performed by programming analysis module 560 of Fig. 5B. In some instances, the AAAM 550 may first associate the treatment input data 440 with appropriate data from a training dataset before analysis takes place. Additionally, AAAM 550 may normalize or standardize the treatment input data 440.

[0168] The example process 1020 continues with identifying at least one error or empty field in the treatment input data (block 1030). For example, the cloud computing environment 207, or more specifically an AAAM 550 hosted by the cloud computing environment 207, may identify an error or empty field in the treatment setup data received at block 1026 and analyzed at block 1028.

[0169] Then, an optimal prescription plan is provided (block 1032). The optimal prescription plan may be provided by the cloud computing environment 207, or more specifically an AAAM 550 hosted by the cloud computing environment 207, and sent to at least one of the dialysis machine 100 or user device 208. The optimal prescription plan may include entries that fix errors or complete empty fields identified in block 1030.

[0170] Fig. 10C is a diagram of an example process 1040 performed by one or more components of Fig. 3C and optionally AAAM 550 of Fig. 5B or neural network model 605 of Figs. 6A and 6B, to update a training dataset used by a cloud computing environment to enhance Al capabilities of the cloud computing environment, according to an example embodiment of the present disclosure. In the illustrated example, the process 1040 begins when the dialysis machine 100 receives treatment outcome information 460 (block 1042). The treatment outcome information 460 may be the same or similar to the programming setup information 400 of Fig. 4A. For example, the treatment input data 440 may include patient information 410, therapy information 420 and prescription information 430. The treatment input data may also include the inputs and values associated with the inputs illustrated in Table 1 and Table 2 above.

[0171] Then, the treatment outcome information 460 is sent to a cloud computing environment 207 (block 1044). For example, the dialysis machine 100 may send the treatment outcome information 460 received at block 1042 to the cloud computing environment 207, and more specifically artificial intelligence hosted in the cloud computing environment 207. Sendingthe treatment outcome information 460 may include uploading the outcome information, transmitting the outcome information, or otherwise communicating the outcome information to the cloud computing environment 207. In some examples, the treatment outcome information 460 may be reformatted or standardized prior to being sent. Additionally, the treatment outcome information 460 may be encoded or encrypted before the dialysis machine 100 sends the information to the cloud computing environment 207. Then, the cloud computing environment receives the treatment outcome information 460 (block 1046). For example, the cloud computing environment 207, or more specifically an AAAM 550 hosted by the cloud computing environment 207, may receive the treatment outcome information 460 sent at block 1044.

[0172] Next, the cloud computing environment 207 may optionally classify the treatment outcome information and / or organize the treatment outcome information (blocks 1048 and 1050). For example, the cloud computing environment 207, or more specifically an AAAM 550 hosted by the cloud computing environment 207, may classify the treatment outcome information 460 and organize the treatment outcome information 460 into various categories, such that the treatment outcome information can be readily used to update a training dataset. Classifying and organizing the treatment outcome information may include normalizing the information, standardizing the information, or grouping the information.

[0173] Then, example process 1040 includes saving at least a portion of the treatment outcome information 460 in a training dataset (block 1052). Specifically, some of the treatment outcome information 460 may be saved to a training dataset, to update the training dataset and improve the capabilities of providing optimal prescription plans described in Fig. 3B and Fig. 10B.

[0174] It should be appreciated that similar to the process described in Fig. 10C, at least a portion of setup image data may be similarly classified, organized and saved to a training dataset to update and improve the capabilities of identifying setup errors and providing error correction suggestions as described in Fig. 3 A and Fig. 10A.Conclusion

[0175] It should be understood that various changes and modifications to the presently preferred embodiments described herein will be apparent to those skilled in the art. Such changes and modifications can be made without departing from the spirit and scope of the presentsubject matter and without diminishing its intended advantages. It is therefore intended that such changes and modifications be covered by the appended claims.

Claims

CLAIMSThe invention is claimed as follows:

1. A dialysis machine (100) comprising: a display screen (102) configured to display a graphical user interface (110), the graphical user interface configured to receive at least one of programming setup information (400) and treatment input data (440); an imaging device (220) communicatively coupled to the dialysis machine, the imaging device configured to capture setup image data; and a network connection (210) communicatively coupling the dialysis machine to a cloud computing environment (207), wherein the dialysis machine, via the network connection is configured to: send the captured setup image data to the cloud computing environment, and receive an error correction suggestion from the cloud computing environment, wherein the error correction suggestion is based on an error identified in the setup image data when the setup image data is compared to preloaded image data in a training dataset.

2. The dialysis machine of Claim 1 , wherein the preloaded image data includes images of a specified therapy setup.

3. The dialysis machine of Claim 1, wherein the setup image data includes at least one image, photo, video, screengrab or snapshot of therapy inputs provided in the graphical user interface of the dialysis machine.

4. The dialysis machine of Claim 1, wherein the setup image data includes at least one image, photo, video, screengrab or snapshot of an area of interest on the dialysis machine, wherein the area of interest includes at least one of a disposable set, fluid connections between the disposable set and other treatment components, and fluid connections between the dialysis machine and a patient.

5. The dialysis machine of Claim 1, wherein the imaging device is a camera.

6. The dialysis machine of Claim 1, wherein the imaging device is a mobile user device.

7. A method comprising: capturing, by an imaging device (220), setup image data for a therapy setup on a dialysis machine (100), wherein the dialysis machine is communicatively coupled to a cloud computing environment (107) and the imaging device; sending, by at least one of the dialysis machine and the imaging device, the captured setup image data to the cloud computing environment; receiving, by the cloud computing environment, the setup image data; analyzing, by the cloud computing environment, the received setup image data; and responsive to identifying a setup error based on the analysis, providing, by the cloud computing environment, an error correction suggestion to at least one of the dialysis machine and / or the imaging device.

8. The method of Claim 7, wherein analyzing the received setup image data includes comparing the setup image data to preloaded image data in a training dataset, and wherein the preloaded image data includes images of a proper therapy setup.

9. The method of Claim 8, wherein comparing the setup image data to preloaded image data includes calculating an absolute difference between a first image from the setup image data to a second image from the preloaded image data.

10. The method of Claim 8, further comprising saving at least a portion of the setup image data to the training dataset.

11. The method of Claim 7, wherein the setup image data includes at least one image, photo, video, screengrab or snapshot of therapy inputs provided in a graphical user interface of the dialysis machine.

12. The method of Claim 7, wherein the setup image data includes at least one image, photo, video, screengrab or snapshot of an area of interest on the dialysis machine, wherein the area of interest includes at least one of a disposable set, fluid connections between the disposable set and other treatment components, and fluid connections between the dialysis machine and a patient.

13. The method of Claim 7, wherein the imaging device is a camera.

14. The method of Claim 7, wherein the imaging device is a mobile user device.

15. A dialysis therapy system (200) comprising: a dialysis machine (100), the dialysis machine including a display screen (102) configured to display a graphical user interface (110), the graphical user interface configured to receive treatment input data (440); a cloud computing environment (207); and a network connection (210) communicatively coupling the dialysis machine to the cloud computing environment, wherein the dialysis machine is configured to: send the treatment input data to the cloud computing environment through the network connection, and wherein the cloud computing environment is configured to: receive the treatment input data, analyze the treatment input data, identify at least one of an error or an empty field in the treatment input data, and provide an optimal prescription plan, wherein the optimal prescription plan includes at least one entry to correct the error or fill an empty field.

16. The dialysis therapy system of Claim 15, wherein the treatment input data includes patient information, therapy information, and prescription information.

17. The dialysis therapy system of Claim 15, wherein the treatment input data includes urine volume information, creatinine level information, and patient condition information.

18. The dialysis therapy system of Claim 15, wherein the optimal prescription plan includes a proposed therapy modality, a proposed disposable set, and proposed flow rates associated with the proposed therapy and proposed disposable set.

19. A method comprising: receiving, by a dialysis machine (100), treatment input data (440), wherein the dialysis machine is communicatively coupled to a cloud computing environment (207); sending, by the dialysis machine, the received treatment input data; receiving, by the cloud computing environment, the treatment input data; analyzing the received treatment input data, wherein analyzing the received treatment input data includes comparing the treatment input data to preloaded data in a training dataset; and responsive to identifying at least one of an input error or an empty field associated with the treatment input data, providing, by the cloud computing environment, an optimal prescription plan.

20. The method of Claim 19, wherein the treatment plan input data includes patient information, therapy information, and prescription information.

21. The method of Claim 20, wherein the patient information includes a patient identifier and at least one of a patient weight, a patient hematocrit percent, and patient condition information.

22. The method of Claim 20, wherein the therapy information includes at least one of a therapy type, a therapy modality, a set type, an anticoagulation method, and an identification of an associated treatment accessory.

23. The method of Claim 20, wherein the prescription information includes at least one of a set flow rate, a blood flow rate, a resulting treatment parameter, and dose information.

24. The method of Claim 19, wherein the treatment input data includes urine volume information, creatinine level information, and patient condition information.

25. The method of Claim 19, wherein the optimal prescription plan includes a proposed therapy modality, a proposed disposable set, and proposed flow rates associated with the proposed therapy and proposed disposable set.

26. The method of Claim 19, further comprising: receiving, by the dialysis machine, treatment outcome information; sending, by the dialysis machine, the received treatment outcome information; receiving, by the cloud computing environment, the treatment outcome information; and saving, by the cloud computing environment, at least a portion of the treatment outcome information and / or at least a portion of the treatment input data to the training dataset.

27. The method of Claim 26, further comprising: prior to saving at least a portion of the treatment outcome information and / or the treatment input data, classifying the treatment outcome information and / or the treatment input data.

28. The method of Claim 26, further comprising: prior to saving at least a portion of the treatment outcome information and / or the treatment input data, organizing the treatment outcome information and / or the treatment input data.

29. The method of Claim 26, wherein the treatment outcome information includes at least one of feedback on treatment outcome status, patient lab report parameters, prescription parameters, type of alarms triggered, quantity of alarms triggered, and fluid removal information.

30. The method of Claim 30, wherein the treatment outcome status is one of condition improved, condition worsened or deteriorated, and condition remained unchanged.

Images