Methods and systems for determining operational states

Abstract

Provided herein are methods and systems for monitoring a medication delivery process such as a process for delivering a drug, a therapeutic, or medication. Provided herein are also methods and systems for determining operational states of a medication delivery device.

Description

PATENT APPLICATIONMETHODS AND SYSTEMS FOR DETERMINING OPERATIONAL STATES  Inventor(s):Matthew J. HuddlestonLoveland, OH Aaron SwickCovington, KY Daniel WaitesWilmington, OH James MarousSouth Vienna, OH   Assignee:Enable Injections, Inc.2863 East Sharon RoadCincinnati, OH 45241   Entity:Small Business Concern   650 Page Mill RoadPalo Alto, CA 94304(650) 493-9300 (Main)(650) 493-6811 (Facsimile)  METHODS AND SYSTEMS FOR DETERMINING OPERATIONAL STATESCROSS-REFERENCE[1] This application claims the benefit of U.S. Provisional Application No. 63 / 615,429, filed December 28, 2023, which is entirely incorporated herein by reference. [2] The subject matter of this patent application is related to the subject matter in Patent Cooperation Treaty Application No. PCT / US2019 / 069142, U.S. Patent No. 11,109,800, U.S. Patent No. 11,571,164, and U.S. Patent No. 11,786,173, each of which is entirely incorporated herein by reference.BACKGROUND[3] Self-administering injection devices have been the subject of continuing development to better offer advantages such as ease of use while providing effective subcutaneous injection. However, it may be difficult to determine the status of the injection and ensure compliance and adherence to an injection schedule or protocol. Improvements are therefore desired.SUMMARY[4] Provided herein are systems and methods for guiding and monitoring a medication delivery process to ensure successful delivery of medication to a subject. [5] One aspect provided herein is a method of determining an operational state of a medication delivery device, comprising: a) instructing a user to capture one or more images of at least a portion of the medication delivery device using an image sensor; and b) determining the operational state of the medication delivery device by analyzing the one or more images, wherein the determining comprises assessing a position of the at least the portion of the medication delivery device. In some embodiments, the method further comprises transferring the one or more images to a cloud-based server. [6] In some embodiments, a medication is delivered by advancing a movable component inside the medication delivery device. In some embodiments, the movable component advances in a trajectory that is not straight. In some embodiments, an amount of medication delivered is in a linear relationship with a distance that the movable component advances. In some embodiments, an amount of medication delivered is in a non-linear relationship with a distance that the movable component advances. [7] In some embodiments, the determining comprises determining a position of a button cap of the medication delivery device. In some embodiments, the position of the button cap comprises a ready to inject position, an inject position, a pause position, or a lockout position. In some embodiments, the determining comprises determining a position of a needle of the medication delivery device. In some embodiments, the position of the needle comprises a retracted position or an extended position. In some embodiments, the determining comprises determining a position of a gas gauge of the medication delivery device. In some embodiments, the position of the gas gauge comprises a ready to inject position, an inject position, a pause position, a lockout position, or a complete position.[8] In some embodiments, the method further comprises assigning a first operational state at a time a first image is captured, assigning a second operational state at a time a second image is captured, and determining a volume delivered between the time the first image is captured and the time the second image is captured. In some embodiments, the operational state comprises an injection started state, an injection paused state, an injection in progress state, or an injection completed state. In some embodiments, the operational state comprises a delivered volume state, a remaining volume state, and / or a remaining time until completion state. In some embodiments, the image sensor is a digital camera or a digital video camera. [9] In some embodiments, the analyzing is performed by comparing the one or more images with a pre-determined image. In some embodiments, the analyzing is performed with a mobile phone. In some embodiments, the analyzing is performed with a software application.

[10] In some embodiments, the method further comprises displaying the operational state. In some embodiments, the method further comprises receiving input information from the user and calculating a medication dosage amount to be administered in a subsequent delivery. In some embodiments, the image sensor comprises instructions for the user to guide positioning of the image sensor relative to the portion of the medication delivery device while capturing the one or more images. In some embodiments, the method further comprises instructing the user to match a device outline displayed on the image sensor with the portion of the medication delivery device while capturing the one or more images. In some embodiments, the method further comprises storing the one or more images. In some embodiments, the method further comprises storing information of the operational state. In some embodiments, the method further comprises alerting the user if the operational state is deviant from a pre-determined state.

[11] Another aspect provided herein is a system for determining an operational state of a medication delivery device, comprising: an image sensor for capturing one or more images of at least a portion of the medication delivery device; a data transmission interface for sending the one or more images to an electronic device; and a processor in communication with the electronic device for analyzing the one or more images and determining the operational state.

[12] In some embodiments, the image sensor comprises a digital camera or a digital video camera. In some embodiments, the medication delivery device comprises a patch and an injector. In some embodiments, the patch and the injector are removably coupled. In some embodiments, the at least the portion of the medication delivery device comprises a button. In some embodiments, the at least the portion of the medication delivery device comprises a needle. In some embodiments, the at least the portion of the medication delivery device comprises a gas gauge. In some embodiments, the at least the portion of the medication delivery device is configured to alter a position indicative of the operational state. In some embodiments, the processor is configured to operate a software application.

[13] In some embodiments, the system further comprises a non-transitory computer-readable storage media encoded with a computer program including instructions executable by the processor. In some embodiments, the system further comprises a data reporting accessory to provide information to a user. In some embodiments, the system further comprises a data display accessory to display the operational state. In some embodiments, the image sensor, the data transmission interface, and the processor are in a common housing. In some embodiments, the common housing is a smartphone.

[14] Another aspect provided herein is a computer-implemented on-boarding method for monitoring a medication delivery, the method comprising: a) receiving an account information from a user to create a new user account; b) displaying an overview of an application, a safety information, a training video, or any combination thereof; c) receiving a data consent, a confirmation from the user to accept notifications, a symptom tracking information, or any combination thereof; d) generating a patient profile and an injection schedule; and e) establishing connection with a medication delivery device of the user.

[15] In some embodiments, displaying the overview of the application comprises displaying an infographic regarding how to record and track injections, set reminders to take medications, create a schedule and reminder for treatment appointments, track and identify symptoms to the user’s healthcare team, access a support program, or any combination thereof. In some embodiments, displaying the safety information comprises displaying a drug prescription information. In some embodiments, generating the patient profile comprises receiving a user date of birth, a user gender, a user name, a user age, a user weight, a user height, a user ethnicity, a user nationality, a user residential address, a user zip code, a user e-mail, a user phone number, a user contact information, or any combination thereof. In some embodiments, generating the injection schedule comprises receiving a dose frequency, an infusion date, an infusion time, or any combination thereof. In some embodiments, establishing connection with the medication delivery device of the user is performed via a wireless communication protocol, a machine-readable barcode, or both. In some embodiments, the wireless communication protocol comprises near-field communication (NFC), Bluetooth, Bluetooth LE, Wi-Fi, radio frequency identification (RFID), iBeacon, ZigBee, Z-Wave, or any combination thereof.

[16] In some embodiments, establishing connection with the medication delivery device of the user comprises instructing the user to: a) activate a wireless communication protocol for the medication delivery device; b) position a mobile computing device of the user in proximity to the medication delivery device, the packaging of the medication delivery device, or both, wherein the mobile computing device is positioned in sufficient proximity to activate wireless communication between the mobile computing device and the medication delivery device; c) remove a battery isolation tab to activate the medication delivery device; or d) any combination thereof.

[17] In some embodiments, the method further comprises bonding the medication delivery device with the application. In some embodiments, the bonding is performed when the application is in a background of a mobile computing device of the user. In some embodiments, the bonding is performed in the absence of opening the application on a mobile computing device. In some embodiments, the medication delivery device or the packaging comprises a bonding stimulus. In some embodiments, the bonding stimulus is provided by a wireless communication protocol, a machine-readable barcode, or both.

[18] In some embodiments, the method further comprises providing a notification to the user when the medication delivery device has established a connection with the application.

[19] In some embodiments, the medication delivery device comprises a syringe, a wearable autoinjector, a wearable infusion pump, a device directly attached for subcutaneous delivery, a device indirectly attached for subcutaneous delivery, or an autoinjector pen.

[20] Another aspect provided herein is a non-transitory computer-readable storage media coupled to a processor and having instructions stored thereon which, when executed by the processor, cause the processor to implement an application comprising: a) an infusion module instructing a user to gather supplies, scan a drug, warm the drug, transfer the drug to a transfer base and press down, select an injection site, remove an injection device from the transfer base, attach the injection device to the user with a view window facing up, or any combination thereof; and b) a progress monitor module displaying an injection status, an injector on / off body status, an injection progress, a remaining injection time, or any combination thereof.

[21] In some embodiments, the application further comprises a home module displaying views of next infusion times, or views of missed infusion notifications, or any combination thereof. In some embodiments, the home module displays a notification if a scanned drug is determined to be expired, incorrect, or both. In some embodiments, the infusion module is configured to receive a stomach quadrant selection representing a location in which the injection device is attached. In some embodiments, the application further comprises a record module displaying an overview of the user’s adherence to an injection schedule and / or indications of symptom progression. In some embodiments, the record module is configured to receive an indication of a complete injection. In some embodiments, the record module is configured to determine the user’s adherence to the injection schedule based on the indication of complete injection. In some embodiments, the application further comprises a settings module displaying the user’s patient profile, infusion schedule, notification settings, or any combination thereof. In some embodiments, the settings module is configured to receive an edit from the user to the user’s patient profile, the infusion schedule, the notification settings, or any combination thereof. In some embodiments, the user’s patient profile comprises a user name, a user date-of-birth, a user name, a user age, a user weight, a user height, a user ethnicity, a user nationality, a user residential address, a user zip code, a user e-mail, a user phone number, or any combination thereof. In some embodiments, the application further comprises a login module receiving a symptom notification and an injection notification, wherein the login module is accessible via the home module. In some embodiments, the application further comprises an injection complete module displaying a notification that the injection is complete. In some embodiments, the progress monitor module comprises the injection complete module. In some embodiments, the application further comprises a schedule module receiving a user response to a symptom questionnaire. In some embodiments, the application further comprises a resources module displaying instructions for use document, a demo video, a terms and conditions document, a privacy policy document, or any combination thereof.INCORPORATION BY REFERENCE

[22] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. To the extent publications and patents or patent applications incorporated by reference contradict the disclosure contained in the specification, the specification is intended to supersede and / or take precedence over any such contradictory material.BRIEF DESCRIPTION OF THE DRAWINGS

[23] The novel features of the present disclosure are set forth with particularity in the appended claims. A better understanding of the features and advantages of the present disclosure will be obtained by reference to the following detailed description that sets forth illustrative embodiments, in which the principles of the disclosure are utilized, and the accompanying drawings of which:

[24] FIG. 1 shows an exemplary flow chart of a method of guiding and monitoring an injection of medication, in accordance with some embodiments;

[25] FIG. 2shows exemplary screenshots of pre-validation instructions provided on a mobile application, in accordance with some embodiments;

[26] FIG. 3shows additional exemplary screenshots of pre-validation instructions provided on a mobile application, in accordance with some embodiments;

[27] FIG. 4 shows schematically a plurality of devices or systems which may be integrated with the mobile application(s) descried herein and training videos for using a plurality of medication delivery devices or systems described herein, in accordance with some embodiments;

[28] FIG. 5shows exemplary displays and instructions provided on a mobile application, in accordance with some embodiments;

[29] FIG. 6A shows an exemplary step of placing / connecting a syringe into a syringe transfer or filing base of an injector, and FIG. 6B shows an exemplary step of pressing on the syringe plunger to transfer the medication from the syringe to the injector, in accordance with some embodiments of the methods described herein;

[30] FIG. 7A shows an exemplary step of placing or connecting a vial into a vial transfer or filing base of an injector, and FIG. 7B shows an exemplary step of transferring the medication from the vial to the injector, in accordance with some embodiments of the methods described herein;

[31] FIG. 8 shows an exemplary status of an injector, in accordance with some embodiments;

[32] FIG. 9shows an exemplary step of removing retaining straps over a medication delivery device, in accordance with some embodiments of the methods descried herein;

[33] FIG. 10A shows an exemplary graphical instruction of removing the injector from the transfer or filing base and placing on body, FIG. 10B shows an exemplary step of removing the injector from a vial transfer or filing base, and FIG. 10C shows an exemplary step of removing the injector from a syringe transfer or filing base, in accordance with some embodiments of the methods described herein;

[34] FIG. 11A shows an exemplary target area of a user for attaching an injector, FIG. 11B shows an exemplary step of attaching an injector to the body of the user, FIG. 11C shows an exemplary side view of attached injector, FIG. 11D shows an exemplary graphical instruction to remove a safety strip, and FIG. 11E shows an exemplary step of removing the safety strip from the injector, in accordance with some embodiments;

[35] FIG. 12A shows an exemplary graphical instruction of pressing a button to begin injection, and FIG. 12B shows an exemplary side view of pressing the button, in accordance with some embodiments;

[36] FIG. 13shows an exemplary flow chart of determining operational states of a medication delivery device, in accordance with some embodiments;

[37] FIG. 14A shows an exemplary button position in an injector, and FIG. 14B shows another exemplary button position in an injector, in accordance with some embodiments;

[38] FIG. 15shows exemplary needle positions in an injector, in accordance with some embodiments;

[39] FIG. 16A shows an exemplary status indicator for an injector that the medication delivery device is ready to inject, FIG. 16B shows an exemplary status indicator for the injector that medication is being delivered, and FIG. 16C shows an exemplary status indicator for the injector that medication delivery is complete, in accordance with some embodiments;

[40] FIG. 17shows exemplary operational states of a medication delivery device, in accordance with some embodiments;

[41] FIG. 18A shows an exemplary display of an injection progress over time, and FIG. 18B shows an exemplary display of an injection progress on-body over time, in accordance with some embodiments;

[42] FIG. 19shows a computer system that is programmed or otherwise configured to implement methods provided herein;

[43] FIG. 20 shows components and data flows in an exemplary digital ecosystem for the application(s) and medication delivery devices described herein, in accordance with some embodiments;

[44] FIG. 21shows a flow chart of interacting digital components in an exemplary medication delivery device ecosystem, in accordance with some embodiments;

[45] FIG. 22 shows a flow chart of an exemplary digital offerings evolution, in accordance with some embodiments;

[46] FIG. 23 shows a flow chart of an exemplary digital ecosystem for the application(s) and medication delivery devices described herein, in accordance with some embodiments;

[47] FIG. 24 shows an image of an exemplary injector, in accordance with some embodiments;

[48] FIG. 25 shows a diagram of an exemplary an on-boarding method for monitoring a medication delivery, in accordance with some embodiments;

[49] FIG. 26 shows a diagram of an exemplary application for repeated monitoring of a medication delivery, in accordance with some embodiments;

[50] FIG. 27A shows a graphical user interface of an exemplary login module, FIG. 27B shows a graphical user interface of an exemplary module for instructing a user to select guidance mode, FIG. 27C shows a graphical user interface of an exemplary module for instructing a user to gather supplies, FIG. 27D shows a graphical user interface of an exemplary module for instructing a user to open packaging, FIG. 27E shows a graphical user interface of an exemplary module for instructing a user to tap to pair, FIG. 27F shows a graphical user interface of an exemplary module for instructing a user to hold the mobile device within a distance from the injection device to scan the injection device, FIG. 27G shows an exemplary screen when the injection device is recognized and the injection device and the application are paired, FIG. 27H shows a graphical user interface of an exemplary module for instructing a user to confirm the drug (medication), FIG. 27I shows a graphical user interface of an exemplary module for instructing a user to scan a barcode on a medication container (e.g., vial), FIG. 27J shows an exemplary screen showing the medication is confirmed, FIG. 27K shows a graphical user interface of an exemplary module for instructing a user to select an injection site, FIG. 27L shows an exemplary screen displaying the selected injection site, and FIG. 27M shows an exemplary screen displaying an alert or warning if the user selected a region of the last injection, in accordance with some embodiments;

[51] FIG. 28A shows a graphical user interface of an exemplary instruction with a re-playable animation for the user to prepare their drug vial, FIG. 28B shows a graphical user interface of an exemplary instruction with a re-playable animation for the user to remove the injector and surrounding fill base from packaging and place the injector on a clean and flat surface, FIG. 28C shows a graphical user interface of an exemplary instruction with a re-playable animation for the user to insert the vial to the injector, and FIG. 28D shows a first graphical user interface of an exemplary instruction for the user to prepare the injection site, in accordance with some embodiments;

[52] FIG. 29A shows a graphical user interface of an exemplary instruction with a re-playable animation for the user to prepare the injection site, FIG. 29B shows a graphical user interface of an exemplary instruction with a re-playable animation to remove the injector from the filling base, FIG. 29C shows a graphical user interface of an exemplary instruction with a re-playable animation to attach the injector, and FIG. 29D shows a graphical user interface of an exemplary instruction with a re-playable animation to remove the safety tab on the injector, in accordance with some embodiments;

[53] FIG. 30A shows a graphical user interface of an exemplary instruction with a re-playable animation for the user to start the delivery, FIG. 30B shows an exemplary screen displaying a delivery status, FIG. 30C shows an exemplary Instructions for Use (IFU) for the delivery, FIG. 30D shows an exemplary screen displaying a delivery is paused, FIG. 30E shows an exemplary screen displaying a delivery is completed, FIG. 30F shows a graphical user interface of an exemplary instruction with a re-playable animation for the user to remove the injector, FIG. 30G shows an exemplary screen displaying the monitoring patch, and FIG. 30H shows a graphical user interface of an exemplary instruction with a re-playable animation for the user to dispose the injector,in accordance with some embodiments;

[54] FIG. 31A shows a graphical user interface of an exemplary expanded notification on a lock screen, FIG. 31B shows a graphical user interface of an exemplary collapsed notification on the lock screen, FIG. 31C shows a graphical user interface of an exemplary expanded notification on a home screen, and FIG. 31D shows a graphical user interface of an exemplary collapsed notification on the home screen, in accordance with some embodiments;

[55] FIG. 32A shows a graphical user interface of an exemplary iconographic notification on the home screen that the device has been recognized, FIG. 32B shows a graphical user interface of an exemplary iconographic notification on the home screen that the device is being paired, FIG. 32C shows a graphical user interface of an exemplary iconographic notification on the home screen that the device has been paired, and FIG. 32D shows a graphical user interface of an exemplary iconographic notification on the home screen that infusion has begun, in accordance with some embodiments;

[56] FIG. 33A shows a graphical user interface of an exemplary iconographic notification on the home screen that infusion has paused, FIG. 33B shows a graphical user interface of an exemplary iconographic notification on the home screen of an infusion error, FIG. 33C shows a graphical user interface of an exemplary iconographic notification on the home screen that the infusion is complete, and FIG. 33D shows a graphical user interface of an exemplary iconographic notification on the lock screen that the device has been recognized, in accordance with some embodiments;

[57] FIG. 34A shows a graphical user interface of an exemplary iconographic notification on the lock screen that the device is being paired, FIG. 34B shows a graphical user interface of an exemplary iconographic notification on the lock screen that infusion has started, FIG. 34C shows a graphical user interface of an exemplary iconographic notification on the lock screen that infusion is in progress, and FIG. 34D shows a graphical user interface of an exemplary iconographic notification on the lock screen that infusion has paused, in accordance with some embodiments;

[58] FIG. 35A shows a graphical user interface of an exemplary iconographic notification on the lock screen of an infusion error, FIG. 35B shows a graphical user interface of an exemplary iconographic notification on the lock screen that the infusion is complete, FIG. 35C shows a graphical user interface of another exemplary iconographic notification on the lock screen that the infusion is delivering, FIG. 35D shows a graphical user interface of another exemplary iconographic notification on the lock screen that the infusion is paused, FIG. 35E shows a graphical user interface of another exemplary iconographic notification on the lock screen that the infusion is complete, FIGS. 35F, 35G, and 35H show iconographic notifications at the compact dynamic island for infusion status of delivering, paused, and complete, FIGS. 35I, 35J, and35K show iconographic notifications at the expanded dynamic island for infusion status of delivering, paused, and complete, FIGS. 35L,35M, and35N show iconographic notifications at the minimal dynamic island for infusion status of delivering, paused, and complete, FIG. 35O shows a graphical user interface of an exemplary iconographic notification on the lock screen of skin temperature, FIG. 35P shows a graphical user interface of an exemplary iconographic notification on the lock screen of steps,FIG. 35Q shows iconographic notifications at the expanded dynamic island for monitoring patch data, and FIG. 35R shows a graphical user interface of an exemplary iconographic notification on the lock screen of lost connectivity to the patch, in accordance with some embodiments;

[59] FIG. 36 showsgraphical user interface of an exemplary tap-to-bond pairing process, in accordance with some embodiments;

[60] FIG. 37A shows an exemplary graphic user interface for user to select “Delete Account” from Settings of the application, FIG. 37B shows an exemplary graphic user interface for user to confirm “Delete Account” and / or select to export data from the application, FIG. 37C shows an exemplary graphic user interface for user to confirm “Delete Account” after the data has been exported, FIG. 37D shows an exemplary graphic user interface for user to confirm account deletion after the user taps the button “Delete Account”, and FIG. 37E shows an exemplary graphic user interface for user to create a new account or signing into an existing account, in accordance with some embodiments;

[61] FIG. 38A shows an exemplary graphic user interface for user to manage medicine schedules, FIG. 38B shows an exemplary graphic user interface for user to add the medicine name,FIG. 38C shows an exemplary graphic user interface for user to add the frequency, FIG. 38D shows an exemplary graphic user interface for user to select the different information to add, FIG. 38E shows an exemplary screen displaying a medicine (e.g., Aleve) is scheduled to be administered every day at 8:00 am and 8:00 pm, FIG. 38F shows an exemplary graphic user interface for the user to choose an icon and color for a medicine,FIG. 38G shows an exemplary graphic user interface for the user to set up the starter dose schedule and the administration time for the medicine (e.g., Enfusimab), and FIG. 38H shows an exemplary screen displaying the medicine (e.g., Enfusimab) is scheduled to be administrated by an injector (e.g., enFuse injector), every 2 weeks at 9:00 am, with a dosage of 150 mg / mL, and from 2 / 1 / 2023 ongoing, in accordance with some embodiments;

[62] FIG. 39A shows an exemplary screen showing a medication schedule. The screen shows the medicine name, dosage, and administration schedule, FIG. 39B shows an exemplary graphic user interface for user to select skip or complete the administration, FIG. 39C shows an exemplary screen that the medicine (e.g., Enfusimab) administration is completed and Aleve is scheduled at 11:00 am, FIG. 39D shows an exemplary screen that the medicine (e.g., Enfusimab) administration is skipped and Aleve is scheduled at 11:00 am, FIG. 39E shows an exemplary screen of the home screen of the application during an injection of the medicine (e.g., Enfusimab), FIG. 39F shows an exemplary screen of the home screen of the application when an injection is paused, FIG. 39G shows an exemplary screen of the home screen of the application when an injection is completed and the monitoring patch is still in use, and FIG. 39H shows an exemplary screen of the home screen of the application when the monitoring patch is removed, in accordance with some embodiments;

[63] FIG. 40A shows an exemplary screen of the statistics module, FIG. 40B shows an exemplary screen displaying averaged scores of quality of life and disease symptoms based on user’s input in each survey, FIG. 40C shows an exemplary screen displaying the distribution and number of the injection sites for a selected duration of time, FIG. 40D shows an exemplary screen displaying injection regimen adherence, FIG. 40E shows an exemplary screen displaying monitoring patch data from a recent injection, FIG. 40F shows an exemplary screen displaying monitoring patch data from a recent injection where an elevated temperature has been detected, and FIGS. 40G-40H show an exemplary screen (a single figure broke to two figures for presentation purpose) displaying a summary of monitoring patch data for a selected duration of time, in accordance with some embodiments.DETAILED DESCRIPTION

[64] Recognized herein are needs for new and / or improved methods and systems for initiating, guiding, and monitoring a medication delivery process and determining operational states of a medication delivery device. Such methods or systems may provide consistent, accurate, and reliable ways for monitoring the medication delivery schedule and / or protocol, alerting the user of any non-compliance of use, and ensuring adherence to the medication delivery schedule and / or protocol.

[65] Whenever the term “at least,” “greater than,” or “greater than or equal to” precedes the first numerical value in a series of two or more numerical values, the term “at least,” “greater than” or “greater than or equal to” applies to each of the numerical values in that series of numerical values. For example, greater than or equal to 1, 2, or 3 is equivalent to greater than or equal to 1, greater than or equal to 2, or greater than or equal to 3.

[66] Whenever the term “no more than,” “less than,” or “less than or equal to” precedes the first numerical value in a series of two or more numerical values, the term “no more than,” “less than,” or “less than or equal to” applies to each of the numerical values in that series of numerical values. For example, less than or equal to 3, 2, or 1 is equivalent to less than or equal to 3, less than or equal to 2, or less than or equal to 1.

[67] The term “subject,” as used herein, generally refers to a user of a device, system, or method of the present disclosure, or an individual on which a device, system, or method of the present disclosure is being used. The subject may be a patient (e.g., a patient that is being treated or monitored by a physician or healthcare provider). As an alternative, the subject may not be a patient. The subject may have or be suspected of having a disease or disorder. As an alternative, the subject may be asymptomatic with respect to a disease or disorder. The subject may be a vertebrate, a mammal (e.g., human or animal), a non-human primate, etc. The subject may be an animal, such as a rodent (e.g., rat or mouse), a canine (e.g., dog), a feline (e.g., cat), a bovine, or other animal.

[68] The term “medicament,” “medicine,” “drug,” or “medication,” as used interchangeably herein, generally refers to a substance that is used for treating a health or physiological state or condition of a subject (e.g., medical treatment). The medicament may be a drug or therapeutic agent. The medicament may be a solid, liquid, gas, or combinations thereof. The medicament may be an aerosol, pill, tablet, capsule, pastille, elixir, emulsion, effervescent powder, solution, suspension, tincture, liquid, gel, dry powder, vapor, droplet, ointment, or a combination or variation thereof. A medicament may be used to treat an illness, ailment, or disease, or may be used as a health supplement (e.g., vitamins, minerals, probiotics, etc.).

[69] The term “medication delivery device,” “delivery device,” “injector,” or “infusion device,” as used interchangeably herein, generally refers to a device that is used to deliver, inject, or infuse a medication to a subject.

[70] The term “injection,” “delivery,” or “infusion,” as used interchangeably herein, generally refers to a process to inject a medication to a subject.Pre-Injection Validation and Operationof an Injector

[71] To use on-body medication delivery devices, e.g., injectors, users may require feedback as to the operation of an injector such as confirmation of the successful initiation of an injection. Users may also need to learn how to use the injector, and be able to successfully transfer medication, place the injector, and initiate an injection.

[72] In an aspect, the present disclosure provides a system and a method that may guide a user through the steps of use of an injector and its companion transfer systems and provide the user feedback as to the pre-injection validation and transfer process.

[73] FIG. 1 shows an exemplary flow chart of method 100 for guiding and monitoring an injection, in accordance with some embodiments. In some embodiments, the method 100 is performed by an injection system. In some embodiments, the system comprises a mobile device. In some embodiments, the mobile device comprises a mobile application. In some embodiments, the mobile application comprises a user interface to provide written instructions, images, graphics, animations, audio, and the like. In some embodiments, the user interface comprises screens that guide a user through the following steps: pre-injection validation, medication transfer, device placement and injection initiation, and injection progress.

[74] The method 100, at operation 101, may comprise instructing a user to perform pre-injection validation. Referring to FIG. 2, upon selection of a function 201 (e.g., starting a new injection / infusion) from the menu screen 200of a mobile application,screen 202may appear. In some embodiments, a barcode, a quick response code (QR code), or near field targets may be embedded into a drug vial, syringe packaging or labeling. In some embodiments, the mobile application may instruct the user to scan the barcode or QR code on a vial (see 203) or on a delivery device (see 204). The mobile application may be integrated with another application on the mobile device such as a camera. Screen 205 shows an example screen of a scanned QR code, which may present information on the medication or injector. In some embodiments, the mobile application may instruct the user to bring the mobile device encompassing the mobile application into close proximity of the vial, syringe, or injector. Information on the medication or delivery device may appear on the screen of the mobile application.

[75] The mobile application may subsequently verify the medication and injector compatibility and / or other parameters of the medication / injector, e.g., expiration date, dosage, etc. In cases where the drug or device is inappropriate for the user (e.g., expired drug), screens 301or 302(see FIG. 3) may appear, which notify the user of the inappropriateness of the medication or injector. In cases where the medication or injector is appropriate for the user, screen 303 may appear, which may provide guidance, instructions, or directions to the user. Instructions may be presented in a continuous scroll format, as exemplified in screen 303. The mobile application may then be paired or registered with the injector. For example, the injector may comprise a unique identifier or other indicia the mobile application can read and register to a user, the user profile, or both.

[76] After the pre-injection validation, the method 100, at operation 102, may comprise instructing the user to transfer a medication to an injector. In some embodiments, the mobile application may instruct the user to follow the demonstration on the screen of the mobile application. In some embodiments, the mobile application may display on the screen a specific volume to be drawn to the syringe. Referring to FIGS. 6Aand 6B, the mobile application may display on the screen the steps of placing / connecting a syringe 601 into a syringe transfer / filing base 602 of an injector 603 and pressing on the syringe plunger 611 to transfer the medication from the syringe 601 to the injector 603. In some embodiments, the mobile application may instruct the user to capture one or more images of at least a portion of the syringe and the injector after connecting. The mobile application may analyze the one or more images (for example, with a computer vision program or module) and determine whether the syringe is placed / connected correctly. In the cases that the syringe is placed / connected correctly, the mobile application may display on the screen that the user may proceed to the next step. In the cases that the syringe is placed / connected incorrectly, the mobile application may display on the screen that the user needs to adjust the placement / connection of the syringe.

[77] Referring to FIGS. 7A and 7B, the mobile application may display on the screen the steps of placing / connecting a vial 701 into a vial transfer / filing base 702 of an injector 703 and pressing on the vial 702 to transfer the medication from the vial to the injector 703. In some embodiments, the mobile application may instruct the user to capture one or more images of at least a portion of the vial and the injector after connecting. The mobile application may analyze the one or more images and determine whether the vial is placed or connected correctly. In the cases that the vial is placed / connected correctly, the mobile application may display on the screen that the user may proceed to the next step. In the cases that the vial is placed or connected incorrectly, the mobile application may display on the screen that the user needs to adjust the placement / connection of the vial. In some embodiments, the mobile application may display on the screen the volume of medication to be transferred to the injector.

[78] Referring to FIG. 8, after the transfer of the medication to the injector, the mobile application may display on the screen an exemplary image of the injector 801 with an illustrative position of gas gauge 802, indicating the filing state of the injector.

[79] Referring to FIG. 9,after the transfer of the medication to the injector, the mobile application may display on the screen the step of removing retaining straps 902 from the injector 901. In some embodiments, the mobile application may instruct the user to capture one or more images of at least a portion of the injector. In some embodiments, the mobile application may analyze the one or more images (for example, with a computer vision program or module and determine whether the retaining straps are removed correctly. In the cases that retaining straps are removed correctly, the mobile application may display on the screen that the user may proceed to the next step. In the cases that the retaining straps are removed incorrectly, the mobile application may display on the screen that the user needs to make corrections.

[80] After the retaining straps are removed, the method 100, at operation 103, may comprise instructing the user to place the injector on body and initiate an injection.

[81] Referring toFIGS. 10A-10C, after the retaining straps 1002 are removed, the mobile application may display on the screen the steps of removing the injector 1001 from the transfer or filing base 1003 and placing the injector on body.FIG. 10B shows an exemplary method of removing the injector 703 from a vial transfer or filing base 702, and FIG. 10C shows an exemplary method removing injector 603 from a syringe transfer or filing base 602.

[82] FIG. 11A shows an exemplary target area 1101 of a user for attaching an injector. The target areas may comprise a trunk, a torso, or an arm of a subject. FIG. 11B shows an exemplary method of attaching an injector 1102 to the target area 1101 of the body of the subject. In some embodiments, the undersurface of the injector may carry an adhesive tape for securing the injector 1102temporarily to the body (e.g., the skin) of a subject until the injection is complete. After the attachment of the injector to a subject, referring to FIG. 11C, the mobile application may display on the screen an exemplary side view of attached injector 1102. In some embodiments, the mobile application may instruct the user to capture one or more images of at least a portion of the injector. In some embodiments, the mobile application may analyze the one or more images and determine whether the injector is attached to the subject correctly. In the cases that the injector is attached to the subject correctly, the mobile application may display on the screen that the user may proceed to the next step. In the cases that the injector is attached to the subject incorrectly, the mobile application may display on the screen that the user needs to make corrections and steps of attaching the injector.

[83] After the injector is attached to the subject correctly, referring to FIG. 11D, the mobile application may display on the screen instructions to remove safety strip 1103 and press the button 1104 to initiate injection. FIG. 11E shows an exemplary method of removing the safety strip 1103 from the injector 1102 by pulling it sideway and substantially parallel to the surface of the injector 1102. In some embodiments, the mobile application may instruct the user to capture one or more images of at least a portionthe injector. In some embodiments, the mobile application may analyze the one or more images and determine whether the safety strip is removed correctly. In the cases that the safety strip is removed correctly, the mobile application may display on the screen that the user may proceed to the next step. In the cases that the safety strip is removed incorrectly, the mobile application may display on the screen that the user needs to make corrections and steps of removing the safety strip.

[84] After the safety strip is removed correctly, referring to FIG. 12A, the mobile application may display on the screen instructions to press the button 1201 to initiate the injection. FIG. 12B shows an exemplary side view of pressing the button 1201 of the injector 1202. Button position may be ready to inject, inject, pause, and lockout. FIG. 14A shows an exemplary button position where the button is at the ready to inject position 1401. FIG. 14B shows another exemplary button position where the button is at the inject position 1402.In some embodiments, the mobile application may instruct the user to capture one or more images of at least a portion of the injector. In some embodiments, the mobile application may analyze the one or more images (for example, with a computer vision program or module) and determine whether the button is at the correct position. In the cases that the button is not at the correct position, the mobile application may display on the screen that the user needs to make corrections and steps of pressing the button to the correct position.

[85] Referring to FIG. 15, the mobile application may display on the screen pictures or animations showing the different positions of the needle, for example, paused 1501, delivering 1502, or complete 1503.

[86] In some embodiments, the mobile application may provideadditional guidelines to the user. Referring to FIG. 2, upon selection of a function 211 (e.g., training videos) from the menu screen 200,screen 400 may appear (see FIG. 4). The mobile application may comprise a variety of tutorials or training information for the subject. FIG. 4demonstrates schematically a plurality of devices or systems which may be integrated with the mobile application. Upon selection of a device or system (e.g., syringe transfer system 401, handheld system 402, vial transfer system 403, reconstitution system 404), screens 401, 402, 403, or404may appear, which may comprise a video demonstrating a tutorial or method of use of the device or system. FIG. 5illustrates schematically an example workflow of a mobile application, which may be used in conjunction with one or more workflows of the mobile application. Upon selection of a function 221 (e.g., additional information, see FIG. 2) from the menu screen 200 (see FIG. 2), screen 500may appear, which may comprise a menu displaying one or more health of physiological parameters or one or more device parameters. Additional information may be available to the subject (e.g., prescription information, device information, etc.). Upon selection of a function in the menu, screen 502 (e.g., prescription information) or 504 (e.g., device information) may appear, which may further comprise options to display additional information, e.g., safety information (e.g., screen 502 or505) or questions and answers (e.g., screen 503or 506). The mobile application may include options for the subject to view and / or input patient information (e.g., age, gender, height, weight, activity level). Additional settings may be implemented in the mobile application, such as alarms, alerts, emails, notifications, etc.

[87] Safety features may be included in the mobile application, e.g., if a safety measure (e.g., safety tab) has not been performed by the user, the mobile application may notify the user. The mobile application may display on the screen one or more injector parameters (e.g., injection status, injection of the cannula into the body of the subject, etc.). Incomplete injection may present on the screen which may indicate the status of the injection and may include other indications of the device parameters (e.g., “injection paused”).

[88] In some embodiments, the mobile applicationmay present the user with options to rank the injection experience. Multiple steps in the process may also comprise communication steps(e.g., via Bluetooth, Wi-Fi) to a separate device, cloud computing, clinician server, etc.

[89] The method 100, at operation 104, may comprise instructing the user to monitor the injection process. In some embodiments, monitoring the injection process comprises determining operational states of an injector.Method for Monitoring aMedication Delivery Process

[90] In another aspect, the present disclosure provides methods for monitoring the medication delivery process and providing the user feedback as to the operational state of the injector.

[91] FIG. 13shows an exemplary flow chart of a method 1300 for determining operational states of an injector.

[92] Referring toFIG. 13, at an operation 1301, the method 1300 comprises instructing a user to capture one or more images of at least a portion of a medication delivery device. In some embodiments, the medication delivery device may be a syringe, a wearable autoinjector, or an autoinjector pen. In some embodiments, the medication delivery device may be a wearable infusion pump. In some embodiments, the medication delivery device may be a device directly attached for subcutaneous delivery of a medicinal product (or a medication). In some embodiments, the medication delivery device may be a device indirectly attached, e.g., through a cannula or a tube set, for subcutaneous delivery of a medicinal product (or a medication). In some embodiments, the medication delivery device may be any injector as disclosed in PCT / US2019 / 069142, PCT / US2021 / 039545, or PCT / US2018 / 034486, all of which are incorporated by reference herein in their entirety.

[93] The medication delivery device comprises a reservoir to hold the medication. In some embodiments, a medication is delivered by advancing a movable component inside the medication delivery device. As the movable component advances inside the medication delivery device, a medication is pushed out of the reservoir and delivered to a subject through a needle or a cannula. In some embodiments, the movable component advances in a trajectory that is not straight. In some embodiments, the movable component advances in a trajectory that is straight. The movable component may be a plunger, i.e., of a syringe. The movable component may be a gas gauge, i.e., of an injector as described in PCT / US2019 / 069142, PCT / US2021 / 039545, or PCT / US2018 / 034486.

[94] In some embodiments, an amount of medication delivered is in a linear relationship with a distance that the movable component advances. In some embodiments, an amount of medication delivered is in a non-linear relationship with a distance that the movable component advances.

[95] In some embodiments, the one or more images may be taken from at least a portion of the medication delivery device. In some embodiments, the at least the portion may comprise a needle. In some embodiments, the at least the portion may comprise a button. In some embodiments, the at least the portion may comprise a movable component, e.g., a plunger or a gas gauge. In some embodiments, the at least the portion of the medication delivery device may be the whole medication delivery device.

[96] In some embodiments, the one or more images may be taken using an image sensor. Non-limiting examples of image sensor comprise a camera, e.g., a digital camera, a mobile phone, a video camera, e.g., a digital video camera, or a video recorder. The method may instruct the user to take images at different time or stage during the delivery process. In some embodiments, the method may instruct the user to take images before the delivery starts, which may be considered as the initial state. In some embodiments, the method may calculate the length of the delivery process and instruct the user to periodically take images at various intervals spreading from the initial state to completion. For instance, if the delivery takes 1 hour to complete, the method may instruct the user to take images at 1 min, 2 min, 5 min, 10 min, 20 min, 30 min, 40 min, 50 min and 60 min. In some embodiments, the method may instruct to take images at shorter intervals at earlier stage of delivery. In some embodiments, the method may instruct to take images at shorter intervals at later stage of delivery. The method may allow the user to set up the time schedule before the delivery starts. In some embodiments, the method may allow the user to modify the schedule at any point of the delivery process.

[97] In some embodiments, the image sensor may comprise instructions to guide positioning of the image sensor relative to at least a portion of the delivery device while capturing the one or more images. In some embodiments, the instructions may comprise placement distance and angle of the image sensor relative to the at least the portion of the delivery device. The image sensor may automate the image capture if the image sensor is at the right distance and in the right orientation.

[98] In some embodiments, the image sensor may display a device outline on the screen and the mobile application may instruct the user to match the device outline with at least a portion of the medication delivery device while capturing one or more images. In some embodiments, the method may comprise storing the one or more images. In some embodiments, the one or more images may be stored in the image sensor. In some embodiments, the one or more images may be transmitted to another mobile device.

[99] Referring to FIG. 13, at an operation 1302, the method 1300 may comprise analyzing the one or more images. In some embodiments, the analyzing may be performed with a software application. In some embodiments, the analyzing may be performed with an algorithm embedded in the software application. In some embodiments, the analyzing may be performed with a mobile phone. In some embodiments, the one or more images may be transferred to a cloud-based server and analyzed by a program in the server.

[100] In some embodiments, the analyzing may comprise assigning a first operational state at a time a first image is captured, assigning a second operational state at a time a second image is captured, and determining a volume delivered between the time the first image is captured and the time the second image is captured.

[101] In some embodiments, the analyzing can be performed by comparing the one or more images with a pre-determined image, for example, with a computer vision program or module. In some embodiments, the analyzing can be performed by comparing the one or more images to a baseline. In some embodiments, the analyzing can be performed by comparing a point array within a given relationship to account for scale or image capture variation. In some embodiments, an algorithm can be used in the analyzing. In some embodiments, machine learning can be used in the analyzing. Any suitable algorithm and / or AI / ML training disclosed in the present disclosure can be used in the analyzing.

[102] In some embodiments, the analyzing may be performed by a trained model such as a computer vision model. In some embodiments, the trained model may comprise a machine learning algorithm.

[103] Referring to FIG. 13, at an operation 1303, a method 1300 may comprise determining the operational state of the medication delivery device. In some embodiments, the determining may comprise assessing a position of at least a portion of the medication delivery device.

[104] In some embodiments, the determining may comprise determining a button cap position. Referring to FIG. 17, the button cap position may comprise ready to inject 1702, inject 1703, pause 1704, or lockout / complete 1701.

[105] In some embodiments, the determining may comprise determining a needle position. Referring to FIG. 15, the needle position may comprise a retracted position (1501 and 1503) or an extended position (1502).

[106] In some embodiments, the determining may comprise determining a gas gauge position. The gas gauge position may comprise a ready to inject position, an inject position, a pause position, or a lockout or complete position. FIG. 16Ashows exemplary position of the gas gauge 1601 for operational status of “ready to inject”. FIG. 16B shows exemplary position of the gas gauge 1611 for operational status of “inject”. FIG. 16A shows exemplary position of the gas gauge 1621 for operational status of “complete”. The injector may have lid markings 1602 on the reservoir window.

[107] In some embodiments, the operational state may comprise an injection started state, an injection paused state, an injection in progress state, or an injection completed state. In some embodiments, the operational state may comprise or provide information for delivered volume, remaining volume, and / or remaining time until completion.

[108] In some embodiments, the method may compare the operational state with a pre-determined state. In the cases that the operational state is deviant from a pre-determined state, the method may comprise alerting a user of the deviation. In some embodiments, the method may comprise steps for troubleshooting.

[109] After the operational state is determined, the operational state may be displayed on a display device. The display device may be a mobile phone or a cloud-based server. The operational state may be stored on a mobile phone or a cloud-based server.

[110] In some embodiments, the method 1300 may comprise receiving input information from a user and calculating a medicament dosage amount to be administered in a subsequent delivery.

[111] FIG. 25 shows a diagram of an exemplary on-boarding method for monitoring a medication delivery. In some embodiments, the method may comprise a step 2501 of downloading an application for monitoring a medication delivery, a step 2502 of creating an account, a step 2503 of performing an application overview, a step 2504 of receiving a data consent, a step 2505 of displaying a safety information, a step 2506 of generating a patient profile, a step 2507 of setting an injection schedule, a step 2508 of receiving a confirmation from a user to accept notifications, a step 2509 of receiving a symptom tracking information, a step 2510 of establishing connection with the user’s device, and a step 2512 of displaying a training video. In some embodiments, the step of creating the account 2502 may comprise receiving a user email. In some embodiments, the step of performing the application overview 2503 may comprise showing the user how to record and track injections, set reminders to take medications, create a schedule and reminder for treatment appointments, track and identify symptoms to the user’s healthcare team, access a support program, or any combination thereof. In some embodiments, the step of displaying the safety information 2505 may comprise displaying a drug prescription information. In some embodiments, the patient profile may be generated by receiving a user date of birth, a user gender, a user name, a user age, a user weight, a user height, a user ethnicity, a user nationality, a user residential address, a user address / zip code, a user e-mail, a user phone number, a user contact information, or any combination thereof. In some embodiments, the step of setting the injection schedule 2507 may comprise receiving a dose frequency, an infusion date, an infusion time, or any combination thereof. In some embodiments, the step of establishing connection with the user’s device 2510 may be performed via a wireless communication protocol, a machine-readable barcode, or both. In some embodiments, the wireless communication protocol comprises near-field communication (NFC), Bluetooth, Bluetooth LE, Wi-Fi, radio frequency identification (RFID), iBeacon, ZigBee, Z-Wave, or any combination thereof.

[112] In some embodiments, the step of establishing connection with the user’s device 2510 may be performed by instructing the user to turn on Bluetooth, Wi-Fi, RFID, or any combination thereof; tap their mobile device against the medication delivery device (e.g., an injection device), its packaging, or both; remove a battery isolation tab to turn the injection device on; or any combination thereof.

[113] In some embodiments, establishing connection with the medication delivery device may comprise instructing the user to: activate a wireless communication protocol for the medication delivery device; and position a mobile device (or a mobile computing device) of the user in proximity to the medication delivery device, the packaging of the medication delivery device, or both. In some embodiments, the mobile computing device may be positioned in sufficient proximity to activate wireless communication between the mobile computing device and the medication delivery device. In some embodiments, the method may further comprise instructing the user to remove a battery isolation tab to activate the medication delivery device.

[114] In some embodiments, the method may further comprise providing a notification to the user when the medication delivery device has established a connection with the mobile device (e.g., an application on the mobile device).

[115] FIGS. 27A-30H show graphical user interfaces (GUIs) of an exemplary application for setting up, performing, and monitoring a medication delivery. In some embodiments, the graphical user interfaces may be presented to a user in the order shown. In some embodiments, two or more consecutive graphical user interfaces may be presented to a user in the order shown. FIG. 27A shows a graphical user interface of an exemplary log-in module, wherein a user may (i) provide a log-in email address and password to log into the application for monitoring of a medication delivery or (ii) create an account for the application for monitoring of a medication delivery. FIG. 27B shows a graphical user interface of an exemplary module for instructing a user to select a guidance mode. The guidance mode may comprise a fully guided mode that comprises step-by-step instructions. The guidance mode may comprise a quick guided mode that comprises summarized instructions. In some embodiments, the application may further comprise notifications to the user to record their symptoms, perform an infusion, and / or refill their prescription. The user may tap on Home to go back to the home module. The user may tap on stats to go to the statistics module. In some embodiments, the graphical user interface may separate notifications to be addressed today and notifications to be addressed in the future. FIG. 27C shows a graphical user interface of an exemplary module for instructing a user to gather supplies, which in this example, comprise a medication vial, a delivery device, and an alcohol wipe. FIGS. 27D-27K show graphical user interfaces of exemplary modules for instructing the user to set up an injection. FIG. 27D shows a graphical user interface of an exemplary module for instructing a user to open packaging. The module may display a step-by-step animation to navigate the user to open the packaging. The instruction may comprise removing the injector and surrounding fill base from the packaging and placing the injector on a clean and flat surface. FIG. 27E shows a graphical user interface of an exemplary module for instructing a user to tap to pair. The module may comprise a re-playable animation and may instruct the user to bring a mobile device close to the box of the injection device (or injector) to start a pairing process. The box of the injector may remain closed at this step. The user may tap Start Pairing to pair the mobile device to the injection device. FIG. 27F shows a graphical user interface of an exemplary module for instructing a user to hold the mobile device within a distance (e.g., 2 inches) from the injection device to scan the injection device. When a successful scan is detected, the application may automatically direct to the next screen. The user may tap Cancel on the modal to return to the previous screen. FIG. 27G shows an exemplary screen when the injection device is recognized and the injection device and the application are paired. The screen may comprise a re-playable animation. FIG. 27H shows a graphical user interface of an exemplary module for instructing a user to confirm the drug (medication). The application may display a re-playable animation of the instruction. The user may tap Next button to open the camera. In some embodiments, the application may request an access to the camera to the mobile device. FIG. 27I shows a graphical user interface of an exemplary module for instructing a user to scan a barcode (or QR-code) on a medication container (e.g., vial). The module may instruct the user to center the container’s barcode inside the frame on the screen. FIG. 27J shows an exemplary screen displaying the medication is confirmed (e.g., the medication is correct and not expired). If the medication is not correct or is expired, the application may send an alert to the user to replace with a correct one.FIG. 27K shows a graphical user interface of an exemplary module for instructing a user to select an injection site. The application may instruct the user to determine where to place the injection device. The instructions may include choosing injection site at belly only. The instructions may include choosing injection site at least 1 inch from the belly button. The instructions may include avoiding site that is tender, bruised, red, hard, irritated, scarred, tattooed, or has stretch marks. The instructions may include not applying injector along belt line. The screen may display four regions corresponding to four regions at the belly. For the first time injection, the application may instruct the user to choose any of the four regions, following the instructions. The application can record the rejection site for the injection. For a subsequent rejection, the screen can show the injection site of the previous injection in a different notation (e.g., different color) and suggest the user to choose a different region. FIG. 27L shows an exemplary screen displaying the selected injection site. In some embodiments, the application may instruct the user to record information regarding symptom and feeling (e.g., if any discomfort, pain, irritation) during an injection process. In some embodiments, the application may process the user input information and sensed information (e.g., health or physiological data from one or more sensors) to determine if a region is not suitable for injection or may be avoided. FIG. 27M shows an exemplary screen displaying an alert or warning if the user selected a region of the last injection. The display may include a recommendation for the user to rotate the injection site to avoid bruising, pain, and / or change in skin properties that could affect future injections. The user may continue selecting the same injection site or select a different injection site.

[116] FIGS. 28A-29D show graphical user interfaces of exemplary instructions for the user to prepare the injection. FIG. 28A shows a graphical user interface of an exemplary instruction with a re-playable animation for the user to prepare their drug vial with an image of a cap being removed from a vial and with instructions to: Remove plastic cap from vial; Clean rubber stopper on vial with alcohol wipe; Let Dry. FIG. 28B shows a graphical user interface of an exemplary instruction with a re-playable animation for the user to remove the injector and surrounding fill base from packaging and place the injector on a clean and flat surface. FIG. 28C shows a graphical user interface of an exemplary instruction with a re-playable animation for the user to insert the vial to the injector. The instructions may include: Insert vial into port on Filling Base; Do Not pick up or move device while filling; Firmly push vial down until a constant stream of bubbles are seen indicting filling has begun; Gauge will move as the device fills; Filling may take up to 10 minutes, filling is complete when retainer strap automatically opens; and Leave on flat surface until filling is complete. FIG. 28D shows a first graphical user interface of an exemplary instruction for the user to prepare the injection site, which may comprise cleaning the injection site prior to placing the device. The screen may display an image of the location that has been selected for the injection. FIG. 29A shows a graphical user interface of an exemplary instruction with a re-playable animation for the user to prepare the injection site with instructions to: Clean site with alcohol wipe; Let dry, DO NOT let your clothes touch the clean site. FIG. 29B shows a graphical user interface of an exemplary instruction with a re-playable animation to remove the injector from the filling base, with instructions that: Filling is complete when Retainer Strap automatically opens; Lift the infuser from Filling Base, the clear Adhesive liner will stay attached to the Filling Base; DO NOT touch adhesive on the bottom of infuser; DO NOT remove Safety tab until the infuser is attached to the body. In some embodiments, the module can instruct the user to capture an image of the medication delivery device after the placement (or attachment) of the medication delivery device on the body of the user. In some embodiments, the module can analyze the image to determine the location of the placement (or attachment) on the body of the user. In some embodiments, the module can record and store the location of the attachment. In some embodiments, the image can be captured by a camera or a digital device.FIG. 29C shows a graphical user interface of an exemplary instruction with a re-playable animation to attach the injector. The instruction may comprise positioning the injector so the fill window is pointed up toward the user’s face, and firmly pressing on the injector to attach it to the belly. FIG. 29D shows a graphical user interface of an exemplary instruction with a re-playable animation to remove the safety tab on the injector. The instruction may comprise holding injector with one hand and using the other hand to pull the safety tab off. Delivery may not start until the safety tab is removed.

[117] FIGS. 30A-30H show graphical user interfaces of exemplary instructions for the user to perform the injection. FIG. 30A shows a graphical user interface of an exemplary instruction with a re-playable animation for the user to start the delivery. The instruction may comprise starting the delivery by firmly pushing the button until it stays in place. After the button is pressed, the injection may start and the status of the injection may automatically display on screen. FIG. 30B shows an exemplary screen displaying a delivery status. The screen may display the delivery is in progress. An injection status window and a monitoring patch data window may display on the screen. The injection status window may display the start time, and estimated infusion time. The patch data may comprise skin temperature, activity level, step count, and start date. The screen may comprise a View important IFU button for the user to view the instructions. FIG. 30C shows an exemplary IFU for the delivery. The IFU may comprise: Delivery continues as long as button is in; To estimate progress, watch Fill Gauge move across Fill Window toward empty; Gauge moves slowly and may take 15 minutes to start moving; Gauge is only an estimate of progress; Do not remove the injector until button pops out; When button pops out, delivery is done; Gauge is only estimate of progress; When the button pops out, the needle is pulled out of the skin, back into the injector, and will not stick to you. FIG. 30D shows an exemplary screen displaying a delivery is paused. FIG. 30E shows an exemplary screen displaying a delivery is completed. FIG. 30F shows a graphical user interface of an exemplary instruction with a re-playable animation for the user to remove the injector. The instruction may comprise using thumb to lift adhesive tab, holding tab against the injector, and slowly peeling the injector from the skin. FIG. 30G shows an exemplary screen displaying the monitoring patch. After removal of the injector, the monitoring patch may remain on the body to continue tracking the activity levels and skin temperature. The monitoring patch may function for up to 5 days and the application may notify the user if it is time to remove the monitoring patch from the belly. The re-playable animation and instruction may comprise using thumb to lift adhesive tab, holding tab against the injector, and slowly peeling the injector from the skin. FIG. 30H shows a graphical user interface of an exemplary instruction with a re-playable animation for the user to dispose the injector. The instruction may comprise putting the used filling base and injector in an FDA cleared sharps disposal container after use and not disposing in household trash.

[118] In some embodiments, the user may select the quick guided mode. The quick guided mode may comprise pairing with device (FIGS. 27E and27E), scanning to confirm drug (FIGS. 27I and 27J), recording injection site (FIG. 27L), and performing injection (FIGS. 30A-30H).

[119] In some embodiments, the system provided herein can enable bonding of an application (e.g., a mobile application) with a medication delivery device and / or a packaging of a medication delivery device. In some embodiments, the bonding can be performed while the application is in the background of a mobile device and its operating system. In some embodiments, the bonding can be performed without opening the application on the mobile device, i.e., having the application as an active window or screen. In some embodiments, the medication delivery device can comprise any medication delivery device disclosed herein. In some embodiments, the medication delivery device may be a syringe, a wearable autoinjector, or an autoinjector pen. In some embodiments, the medication delivery device may be a wearable infusion pump. In some embodiments, the medication delivery device may be a device directly attached for subcutaneous delivery of a medicinal product (or a medication). In some embodiments, the medication delivery device may be a device indirectly attached, e.g., through a cannula or a tube set, for subcutaneous delivery of a medicinal product (or a medication). In some embodiments, the packaging of the medication delivery device can be any suitable packaging that can contain the medication delivery device. In some embodiments, the medication delivery device or the packaging can comprise a bonding, pairing, or registration stimulus or mechanism to initiate and implement the bonding with the application of the mobile device. In some embodiments, the bonding stimulus or mechanism can comprise a wireless communication protocol or the reading of a computer readable barcode, for example, near-field communication (NFC), radio frequency identification (RFID), Bluetooth, Bluetooth LE, Wi-Fi, iBeacon, ZigBee, Z-Wave, QR codes, or combinations thereof. In some embodiments, when the medication delivery device or the packaging is brought to proximate of the mobile device (such as in sufficient proximity to establish wireless communication and / or accurately read a barcode), the application of the mobile device can be activated and pop up a notification on the screen of the mobile device, thereby implementing the bonding process. Following the bonding process, the application can navigate the user through the pairing process to pair the mobile device with the medication delivery device for subsequent delivery process. In some embodiments, the pairing process can be completed through the application notification functionality or live activity functionality. In some embodiments, the pairing process can be completed by opening of the application. This automated bonding process, e.g., without the need to open the application on the mobile device, can simplify the pairing process.

[120] FIG. 36 showsgraphical user interface of an exemplary tap-to-bond pairing process. As shown, in one example, the pairing process can comprise instructing the user to “Bring your phone close to the device’s box to start a pairing process.” Once the user performs this action, a notification can be shown that the infusion devices is ready to scan and an instruction is presented to the user to “hold your smartphone near the package label to begin pairing,” wherein the user’s mobile device recognizes a tag or a code, e.g., an NFC tag or a QR code, in the packaging and / or infusion device, and a notification can be provided that the device is recognized. In some embodiments, the user can be further instructed to activate the infusion device. In some embodiments, the user can be further instructed to activate the infusion device by removing a battery tab to allow the battery to provide power to the infusion device. In some embodiments, the user can be further instructed to turn on the infusion device by removing the infusion device from its base. In some embodiments, the user can be further instructed to turn on the infusion device by inserting a vial and / or syringe. In some embodiments, tapping the mobile device against the packaging can activate the pairing application. In some embodiments, pairing enables communication between the infusion device and the mobile device.System for Monitoring aMedicationDelivery Process

[121] In yet another aspect, the present disclosure provides a system for monitoring the medication delivery process and providing the user feedback as to the operational state of the medication delivery device.

[122] The system may comprise an image sensor for capturing one or more images of at least a portion of the medication delivery device, a data transmission interface for sending the one or more images to an electronic device, and a processor for analyzing the one or more images and determining an operational state.

[123] In some embodiments, the system may comprise the medication delivery device. In some embodiments, the medication delivery device may be any injector as disclosed in PCT / US2019 / 069142, PCT / US2021 / 039545, or PCT / US2018 / 034486. In some embodiments, the medication delivery may comprise a patch and an injection unit. In some embodiments, the patch and the injection unit can be removably coupled.

[124] In some embodiments, the at least the portion of the medication delivery device can comprise a button, a needle, or a gas gauge. In some embodiments, the at least the portion of the medication delivery device can be the whole medication delivery device. In some embodiments, the at least the portion of the medication delivery device can be configured to alter a position, indicative of an operational state or a change of an operational state.

[125] The image sensor may be any suitable image sensor that may capture an image. In some embodiments, the image sensor can be a digital camera or a digital video camera. In some embodiments, the image sensor can be a digital camera integrated in a mobile device. In some embodiments, a mobile device may be a laptop, a tablet, a phone (e.g., smartphone), or other electronic device (e.g., portable electronic device).

[126] In some embodiments, the processor may comprise a software application. In some embodiments, the software application may be integrated in a mobile device. In some embodiments, the mobile device may comprise a mobile phone. In some embodiments, the system may comprise a non-transitory computer-readable storage media encoded with a computer program including instructions executable by the processor. In some embodiments, the system may comprise a data reporting accessory to provide information to a user.

[127] In some embodiments, the system may comprise a data display accessory to display the operational state. In some embodiments, the operational state may comprise injection started, injection paused, injection in progress, or injection completed. In some embodiments, the operational state may comprise delivered volume, remaining volume, and / or remaining time until completion. The display may be in text, image, graph, or animation.

[128] FIG. 18A shows an exemplary display of an injection progress over time. During injection, gas gauge 1801 may advance in a trajectory inside the injector 1802 as demonstrated schematically by arrow 1803. The trajectory may be non-straight. FIG. 18B shows an exemplary display of an injection progress of the injector 1812 on-body over time. A gas gauge may advance in a trajectory inside the injector 1812 as demonstrated schematically by arrow 1813.

[129] FIG. 16A shows an exemplary display that medication delivery is ready to inject, FIG. 16B shows an exemplary display that medication delivery is being delivered, and FIG. 16C shows an exemplary display that medication delivery is complete.

[130] In some embodiments, the image sensor, data transmission interface, and processor can be in a common housing. In some embodiments, the common housing can be a mobile phone.

[131] FIG. 20 shows components and data flows in an exemplary digital ecosystem. As shown, an on-body delivery device 2010 transmits injection data to a mobile application 2020. In some embodiments, the on-body delivery device 2010 transmits injection data to the mobile application 2020 via Bluetooth. In some embodiments, the mobile application 2020 provides a digital interface for training, reminders, instructions and resources, and transmits data to a cloud 2030. In some embodiments, the cloud 2030 serves as a database and performs analysis to determine meaningful information from the injection data and stored patent data. In some embodiments, the cloud 2030 transmits data to a dashboard 2040, which acts as an interface for remove stakeholders to utilize the meaningful information determined by the cloud 2030.

[132] FIG. 21shows an exemplary flow chart of digital components in a medication delivery device ecosystem. As shown, in some embodiments, a drug product information 2111, a device information 2112, and a patient / caregiver information 2113 is provided to a mobile application 2020. In some embodiments, the drug product information 2111 comprises a drug type and a drug expiration. In some embodiments, the device information 2112 comprises an infusion date, an infusion start time, an infusion stop time, an infusion duration, a dose volume, an on / off body status, a patient biometric, or any combination thereof. In some embodiments, the patient or caregiver information 2113 comprises a patient reported outcome. In some embodiments, the mobile application 2020 further provides use guidance, infusion reminders, prescription reminders, symptom guidance and adherence tracking as user feedback 2114 to the patient or caregiver. Further, as shown, in some embodiments, the mobile application 2020 is communicably coupled to a cloud storage 2030. In some embodiments, the cloud storage 2030 provides a clinical trial dashboard 2041 and a provider dashboard 2042 comprising remote monitoring related to adherence, endpoints, symptoms, or any combination thereof. In some embodiments, the cloud storage 2030 further provides a patient dashboard 2043 based on remote device performance data.

[133] FIG. 22 shows a flow chart of an exemplary digital offerings evolution. As shown, the mobile application 2020 receives a manual input 2050 from a user, performance and use data and biometric data 2011 from the on-body delivery device 2010. In some embodiments, the mobile application 2020 transmits data to the cloud 2030, which is communicably coupled to the dashboard 2040. Further, as shown, the manual input 2050, the mobile application 2020, the cloud 2030, and the dashboard exist in a digital ecosystem offering 2210, wherein the on-body delivery device 2010 is in a device monitoring offering 2220, and wherein the biometric data 2011 is received in a patient monitoring offering 2230. FIG. 24 shows an image of an exemplary on-body delivery device 2010.

[134] FIG. 23 shows a flow chart of four tiers of an exemplary digital ecosystem for collaboration with a pharma companion. In a first tier 2310, the security, analytics and user experience may be managed by a pharma companion, which employs a pharma-companion application 2021, a pharma-companion cloud 2031, and a pharma-companion dashboard 2041. In a second tier 2320, the pharma companion may employ the mobile application 2020 herein, wherein the cloud 2030 comprises a pharma-companion cloud and the dashboard 2040 comprises a pharma-companion dashboard. In a third tier 2330, the pharma companion may employ the mobile application 2020 and cloud 2030 herein, wherein the cloud 2030 communicates, wherein the dashboard 2040 comprises a pharma-companion dashboard.Application for Monitoring ofa Medication Delivery

[135] Provided herein is a non-transitory computer-readable storage media coupled to a processor and having instructions stored thereon which, when executed by the processor, cause the processor to provide an application.

[136] In some embodiments, the application may instruct the user to create an injection schedule and receive reminders. In some embodiments, the application may instruct the user to use step-by-step injection guidance. In some embodiments, the application may monitor the injection progress. In some embodiments, the application may record the injections.

[137] In some embodiments, the application may comprise a home module. In some embodiments, the home module may display views of next infusion times, or views of missed infusion notifications, or any combination thereof. In some embodiments, the application may comprise a login module. In some embodiments, the login module may receive a symptom notification and an injection notification. In some embodiments, the login module may comprise a function to create account or sign in. In some embodiments, the login module may be accessible via the home module. In some embodiments, the application may comprise an infusion module. In some embodiments, the infusion module may instruct a user to gather supplies, scan a drug, warm the drug, transfer the drug to a transfer base and press down, select an injection site, remove an injection device from the transfer base, or attach the injection device to the user with a view window facing up, or any combination thereof. In some embodiments, the application may comprise a progress monitor module. In some embodiments, the progress monitor module may display an injection status, an injector on / off body status, an injection progress, or a remaining injection time, or any combination thereof. In some embodiments, the application may comprise an injection complete module. In some embodiments, the injection complete module may display a notification that the injection is complete. In some embodiments, the progress monitor module may comprise the injection complete module. In some embodiments, the application may comprise a schedule module. In some embodiments, the schedule module may receive a user response to a symptom questionnaire or survey. In some embodiments, the application may comprise a record module. In some embodiments, the record module may display an overview of the user’s adherence to an injection schedule and / or indications of symptom progression. In some embodiments, the application may comprise a settings module. In some embodiments, the settings module may display the user’s patient profile, infusion schedule, notification settings, or any combination thereof. In some embodiments, the application may comprise a resources module. In some embodiments, the resources module may display instructions for use document, a demo video, a terms and conditions document, or a privacy policy document, or any combination thereof.

[138] FIG. 26 shows a diagram of an exemplary application for repeated monitoring of a medication delivery. A login module 2620 may receive a symptom notification 2611 and an injection notification 2612, and may be communicably coupled to a home module 2610. The home module may allow a user to navigate the modules herein, view next infusion times, view missed infusion notifications, or any combination thereof. In some embodiments, from the home module 2610, a user may select to enter a module for performing infusions 2630, wherein the user may be instructed to gather supplies 2631, scan their drug 2632, warm the drug 2633, transfer the drug 2634 to the transfer base and press down, select an injection site 2635, remove the injection device from a transfer base 2636, and place the device on the selected injection site (e.g., stomach) and initiate the injection (2637). Once injection is initiated, in some embodiments, a progress monitor module 2638 may display an injection status, an injector on / off body status, an injection progress, a remaining injection time, or any combination thereof. In some embodiments, when the injection is complete, an injection complete module 2639 may provide a notification. In some embodiments, a notification may be provided to the user if the scanned drug 2632 is determined to be expired and / or incorrect. In some embodiments, a notification may be provided to the user if the scanned drug 2632 is determined to correct or in compliance. In some embodiments, upon instructing the user to place the device at the selected injection site (e.g., on their stomach with the window facing up) 2637, a quadrant selection (e.g., stomach quadrant selection) may be received representing a location in which the device was placed. In some embodiments, from the home module 2610, a user may select to enter a schedule module 2640 to receive user responses to a symptom questionnaire or survey. In some embodiments, from the home module 2610, a user may select to enter a record module 2650, which displays an overview of the user’s adherence 2651 to their injection schedule and / or indications of symptom progression 2652. In some embodiments, the record module 2650 may further receive an indication for every complete injection, wherein the adherence 2651 may be based on the indication. In some embodiments, from the home module 2610, a user may select to enter a settings module 2660, where the user may display and / or edit the patient profile 2661 (e.g., name, date-of-birth, email address, address, password, and / or zip code), display and / or edit their infusion schedule 2662, and / or display and / or edit their notifications 2663 related to injection reminders and symptom tracking. In some embodiments, from the home module 2610, a user may select to enter a resources module 2670, wherein the user may view the instructions for use (IFU) 2671 (e.g., a pdf copy of the IFU), a demo video 2672, the terms and conditions 2673, and the privacy policy 2674.

[139] FIG. 31A shows a graphical user interface of an exemplary expanded notification on a lock screen, wherein four notification icons are arrayed such that each is visible. In some cases, at least one, at least two, at least three, at least four, at least five, or more notification icons may be displayed. FIG. 31B shows a graphical user interface of an exemplary collapsed notification on the lock screen, wherein one or more notifications overlap, wherein dismissing one notification shows the subsequent notification beneath. In some embodiments, the notification may comprise the injection progress (e.g., initiated, in progress, paused, completed, injector removed, and / or monitoring patch removed). In some embodiments, the notification may comprise a warning. In some embodiments, the warning may comprise an elevated skin temperature warning (detected by the monitoring patch). In some embodiments, the warning may comprise a high or vigorous activity level. In some embodiments, the warning may comprise a static activity level (e.g., activity level is close to 0).FIG. 31C shows a graphical user interface of an exemplary expanded notification on a home screen, wherein four notification icons are arrayed such that each is visible. FIG. 31D shows a graphical user interface of an exemplary collapsed notification on the home screen, wherein one or more notifications overlap, wherein dismissing one notification shows the subsequent notification beneath.

[140] FIG. 32A shows a graphical user interface of an exemplary iconographic notification on the home screen that the device has been recognized and provides an estimated infusion time. FIG. 32B shows a graphical user interface of an exemplary iconographic notification on the home screen that the device is being paired and provides an estimated infusion time. FIG. 32C shows a graphical user interface of an exemplary iconographic notification on the home screen that the device has been paired and provides an estimated infusion time. FIG. 32D shows a graphical user interface of an exemplary iconographic notification on the home screen that infusion has begun with a different color (e.g., green) than the pairing notification and with an estimated infusion time. FIG. 33A shows a graphical user interface of an exemplary iconographic notification on the home screen that infusion has paused with a different color (e.g., yellow) than the infusion beginning notification and with an estimated infusion time. FIG. 33B shows a graphical user interface of an exemplary iconographic notification on the home screen of an infusion error with a different color (e.g., red) than the pairing notification and / or the infusion beginning notification and with an estimated infusion time. FIG. 33C shows a graphical user interface of an exemplary iconographic notification on the home screen that the infusion is complete with a different color (e.g., green) than the error notification and / or the pause notification. FIG. 33D shows a graphical user interface of an exemplary iconographic notification on the lock screen that the device has been recognized and an estimated infusion time.

[141] FIG. 34A shows a graphical user interface of an exemplary iconographic notification on the lock screen that the device is being paired and an estimated infusion time. FIG. 34B shows a graphical user interface of an exemplary iconographic notification on the lock screen that infusion has started and an estimated infusion time. FIG. 34C shows a graphical user interface of an exemplary iconographic notification on the lock screen that infusion is in progress, with a progress bar and an estimated infusion time. FIG. 34D shows a graphical user interface of an exemplary iconographic notification on the lock screen that infusion has paused with a progress bar having a different color (i.e., yellow) than the color of the infusing progress bar and an estimated infusion time. FIG. 35A shows a graphical user interface of an exemplary iconographic notification on the lock screen of an infusion error, with a progress bar having a different color (i.e., red) than the color of the infusing progress bar and an estimated infusion time. FIG. 35B shows a graphical user interface of an exemplary iconographic notification on the lock screen that the infusion is complete.FIG. 35C shows a graphical user interface of another exemplary iconographic notification on the lock screen that the infusion is delivering. FIG. 35D shows a graphical user interface of another exemplary iconographic notification on the lock screen that the infusion is paused. FIG. 35E shows a graphical user interface of another exemplary iconographic notification on the lock screen that the infusion is complete. In some embodiments, iconographic notifications may be at the compact dynamic island or expanded dynamic island of the unlocked home screen. FIGS. 35F, 35G, and 35H show iconographic notifications at the compact dynamic island for infusion status of delivering (FIG. 35F), paused (FIG. 35G), and complete (FIG. 35H). FIGS. 35I, 35J, and35K show iconographic notifications at the expanded dynamic island for infusion status of delivering (FIG. 35I), paused (FIG. 35J), and complete (FIG. 35K). FIGS. 35L,35M, and35N show iconographic notifications at the minimal dynamic island for infusion status of delivering (FIG. 35L), paused (FIG. 35M), and complete (FIG. 35N). In some embodiments, the monitoring patch data may be displayed in the lock screen. FIG. 35O shows a graphical user interface of an exemplary iconographic notification on the lock screen of skin temperature. FIG. 35P shows a graphical user interface of an exemplary iconographic notification on the lock screen of steps. In some embodiments, the monitoring patch data may be displayed at the dynamic island of the unlocked screen, in expanded, compact, or a minimal mode.FIG. 35Q shows iconographic notifications at the expanded dynamic island for monitoring patch data. In some embodiments, additional notification may display in the lock screen. FIG. 35R shows a graphical user interface of an exemplary iconographic notification on the lock screen of lost connectivity to the patch. In some embodiments, additional notification may display at the dynamic island of the unlocked screen, in expanded, compact, or a minimal mode.

[142] In some embodiments, the settings module may comprise a sub-module for a user to delete an account. In some embodiments, deletion of the account may occur after the user has completed the prescribed treatment. In some embodiments, deletion of the account may occur if the user decides to stop the treatment (e.g., no more injection is needed or desired).

[143] FIGS. 37A-37E show exemplary graphic user interfaces for deletion of an account. FIG. 37A shows an exemplary graphic user interface for user to select “Delete Account” from Settings of the application. FIG. 37B shows an exemplary graphic user interface for user to confirm “Delete Account” and / or select to export data from the application. The user may export data by tapping “Export Data” and the data may be exported to an email address associated with the account or any email address the user provides. FIG. 37C shows an exemplary graphic user interface for user to confirm “Delete Account” after the data has been exported. The screen may display notification, for example, “data successfully exported” before the user proceeds to delete account. The user may check the email to ensure the exported data has been successfully received. FIG. 37D shows an exemplary graphic user interface for user to confirm account deletion after the user taps the button “Delete Account”. The application may ask if the user is sure that he / she wants to delete the account. The user may tap cancel to close dialogue and return to previous screen. If confirmed to delete the account, the user may tap delete to delete the account. After the account is deleted, the application jumps to the First Time Welcome Screen, where the user can set up a new account. FIG. 37E shows an exemplary graphic user interface for user to create a new account or signing into an existing account.

[144] In some embodiments, application may comprise a sub-module for a user to manage medicine schedules. In some embodiments, the user may manage schedules for one or more medicines. In some embodiments, the application can serve an integrated medicine management system for the user to manage schedules for one or more medicines that the user or another person may take, perform injections, and monitor symptoms. In some embodiments, the one or more medicines may comprise the medication that is injected by the medication delivery device. In some embodiments, the one or more medicines may comprise a medication that is orally administrated. In some embodiments, the one or more medicines may comprise a medication that is topically administrated.

[145] FIG. 38A shows an exemplary graphic user interface for user to manage medicine schedules. The screen shows the user may add Medicine A, Medicine B, and / or Medicine C in the Medicine Cabinet setting. The user may tap on the Add A medicine button to add medicine schedules. In some embodiments, the user may tap on the Add Medicine (e.g., Enfusimab) button to add medicine Enfusimab (a medication subcutaneously administrated). The user may select the administration method. FIG. 38B shows an exemplary graphic user interface for user to add the medicine name.FIG. 38C shows an exemplary graphic user interface for user to add the frequency (e.g., as needed or how many times a day, at which time each administration is scheduled, on which specific days the administration is scheduled, e.g., every Monday or every 5 days) to administer the medicine. FIG. 38D shows an exemplary graphic user interface for user to select the different information to add (e.g., medicine name, frequency, dosage, duration, and notes & photos). The user may tap on the respective button to fill in the information. Once finished, the user may save the information by tap on Save Medicine. The user may activate reminder function such that the application sends notification and / or reminder to the user when an administration is due. FIG. 38E shows an exemplary screen showing medicine Aleve is scheduled to be administered every day at 8:00 am and 8:00 pm. In duration, the user may add the start date and the end date. The user may edit the default icon and color for the added medicine by tapping on the edit icon. FIG. 38F shows an exemplary graphic user interface for the user to choose an icon and color for a medicine. In “Notes & Photos”, the user may add notes and add a photo of the medicine. The application may send an alert if the reminders are turned off. The application may send a reminder that the user should take medicines as prescribed and not solely rely on the application for a reminder.FIG. 38G shows an exemplary graphic user interface for the user to set up the starter dose schedule and the administration time for a medicine (e.g., Enfusimab). FIG. 38H shows an exemplary screen showing a medicine (e.g., Enfusimab) is scheduled to be administrated by an injector (e.g., enFuse injector), every 2 weeks at 9:00 am, with a dosage of 150 mg / mL, and from 2 / 1 / 2023 ongoing.

[146] FIG. 39A shows an exemplary screen showing a medication schedule. The screen shows the medicine name, dosage, and administration schedule. The user may tap on dosage button to edit the dosage. The user may tap on quality of life survey to provide feedback on feelings. The quality of life survey may comprise questions on 1) lack of energy, 2) having a pain, 3) having nausea, 4) worrying about condition may get worse, 5) sleeping well, 6) able to enjoy life, and 7) content with the quality of life, with evaluation of not at all, a little bit, somewhat, quite a bit, very much. After the answers are provided, the application may calculate a quality of life score (e.g., in the range of 0-28, the higher the score, the better the quality of life). The questions listed herein are for demonstration purposes only and are not limited by those disclosed. Other questions may be added. The user may tap on disease symptoms survey to provide feedback on symptoms. The symptoms survey may comprise questions on 1) having certain parts of the body experiencing pain, 2) feeling weak all over, 3) getting tired easily, 4) having trouble concentrating, 5) worrying about getting infections, 6) feeling discouraged about the illness, 7) having difficulty planning for the future, 8) worrying about getting new symptoms, 9) having emotional ups and downs, 10) having bone pain, 11) needing help doing the usual activities, 12) having trouble walking because of pain, 13) feeling fatigued, and 14) having gained weight, with evaluation of not at all, a little bit, somewhat, quite a bit, very much. After the answers are provided, the application may calculate a disease symptom score (e.g., in the range of 0-56, the higher the score, the less disease symptoms experienced). The questions listed herein are for demonstration purposes only and are not limited by those disclosed. Other questions may be added.

[147] When an administration is scheduled, the user may select to perform the administration or skip the administration. FIG. 39B shows an exemplary graphic user interface for user to select skip or complete the administration. The user may select skip to skip an administration. The user may select complete to perform the administration and after the administration is done, to record a completed administration. FIG. 39C shows an exemplary screen that the medicine (e.g., Enfusimab) administration is completed and Aleve is scheduled at 11:00 am. FIG. 39D shows an exemplary screen that the medicine (e.g., Enfusimab) administration is skipped and Aleve is scheduled at 11:00 am.

[148] FIG. 39E shows an exemplary screen of the home screen of the application during an injection of the medicine (e.g., Enfusimab). The screen shows the weekly schedule with the medicine (e.g., Enfusimab) injection in progress and Advil scheduled. When injector and monitoring patch are in use, the injection status and monitoring patch data may surface at the top of the screen. The patch data may comprise skin temperature, activity level, and step count. The patch data may remain at the top of the screen for as long as the patch is in use. The injection status may show the start time and the estimated infusion time. When injector is in use but no monitoring patch is in use, only injection status without the patch data may surface at the top of the screen. FIG. 39F shows an exemplary screen of the home screen of the application when an injection is paused. The injection status and monitoring patch show at the top of the screen. The injection status is paused with a notification that injection is paused while button is depressed and release button to continue. FIG. 39G shows an exemplary screen of the home screen of the application when an injection is completed and the monitoring patch is still in use. The injection status window disappears and the medicine (e.g., Enfusimab) schedule changes to “completed”. The patch data window may remain until the patch is removed. FIG. 39H shows an exemplary screen of the home screen of the application when the monitoring patch is removed. The user may tap the patch removed banner to review patch data in statistics.

[149] In some embodiments, the application may further comprise a statistics or history module. FIG. 40A shows an exemplary screen of the statistics module. In some embodiments, the statistics module may comprise data on user’s feeling. FIG. 40B shows an exemplary screen displaying averaged scores of quality of life and disease symptoms based on user’s input in each survey. The higher the score, the better the quality of life and less disease symptoms experienced. The module may display statistics for a selected duration of time. In some embodiments, the statistics module may comprise data of injection site tracking. FIG. 40C shows an exemplary screen displaying the distribution and number of the injection sites for a selected duration of time. FIG. 40C shows in 2023, the injection was done 30% at the upper right region, 30% at the upper left region, 10% at the lower right region, and 30% at the lower left region. FIG. 40C also shows that July had the most injections followed by March. In some embodiments, the statistics module may comprise data of injection adherence. FIG. 40D shows an exemplary screen displaying injection regimen adherence comprising injection on time, early, late, skipped, and unrecorded. FIG. 40D also shows the detailed adherence event in each month. In some embodiments, the statistics module may comprise monitoring patch data. FIG. 40E shows an exemplary screen displaying monitoring patch data from a recent injection. The screen may display the wear time, injection data, and removal date. The screen may display skin temperature, including average temperature and max temperature. The screen may display a notice if an elevated temperature is detected (e.g., a temperature above 100 degrees F). The screen may display activity levels in a pie chart showing the percentage of time that the user is static, has light, moderate, high, or vigorous activity level during the patch wear. FIG. 40F shows an exemplary screen displaying monitoring patch data from a recent injection where an elevated temperature has been detected. FIGS. 40G-40H show an exemplary screen (a single figure broke to two figures for presentation purpose) displaying a summary of monitoring patch data for a selected duration of time (e.g., 3 months, 6 months, 1 year). The screen may display total patches worn, total steps, minimum temperature, maximum temperature, and average temperature. The screen may comprise bar graphs to visualize steps for each patch worn. The screen may comprise a graph for skin temperature trend showing average temperature and maximum temperature for each patch worn. The screen may comprise graphs showing percentage of time of activity levels (static, light, moderate, high, or vigorous) during each patch wear. In some embodiments, the statistics data may display how the user adhered to the injection schedule and the presence or likelihood of any abnormal event the user may have experienced (e.g., low score for quality of life and disease symptoms, elevated temperatures). In some embodiments, the statistics data may be used to extract a relationship between user’s activity and the quality of life and disease symptoms. In some embodiments, the statistics data may be used to provide injection recommendations to the user. The injection recommendations may comprise injection time, injection site, and / or injection frequency. The injection recommendations may comprise means to improve adherence (e.g., set up additional reminders, form an injection pattern). The injection recommendations may comprise means to adjust activity levels during the injection. Computer Systems

[150] The present disclosure provides computer systems that are programmed to implement methods of the disclosure. FIG. 19 shows a computer system 1901 that is programmed or otherwise configured to transmit and / or receive data, and process data. The computer system 1901 may regulate various aspects of the present disclosure, such as, for example, methods for data analysis, subject monitoring and measurement of operational states, and providing an output of the operational states. The computer system 1901 may be an electronic device of a user or a computer system that is remotely located with respect to the electronic device. The electronic device may be a mobile electronic device.

[151] The computer system 1901 includes a central processing unit (CPU, also “processor” and “computer processor” herein) 1905, which may be a single core or multi core processor, or a plurality of processors for parallel processing. The computer system 1901 also includes memory or memory location 1910 (e.g., random-access memory, read-only memory, flash memory), electronic storage unit 1915 (e.g., hard disk), communication interface 1920 (e.g., network adapter) for communicating with one or more other systems, and peripheral devices 1925, such as cache, other memory, data storage and / or electronic display adapters. The memory 1910, storage unit 1915, interface 1920 and peripheral devices 1925 are in communication with the CPU 1905 through a communication bus (solid lines), such as a motherboard. The storage unit 1915 may be a data storage unit (or data repository) for storing data. The computer system 1901 may be operatively coupled to a computer network (“network”) 1930 with the aid of the communication interface 1920. The network 1930 may be the Internet, an internet and / or extranet, or an intranet and / or extranet that is in communication with the Internet. The network 1930 in some cases is a telecommunication and / or data network. The network 1930 may include one or more computer servers, which may enable distributed computing, such as cloud computing. The network 1930, in some cases with the aid of the computer system 1901, may implement a peer-to-peer network, which may enable devices coupled to the computer system 1901 to behave as a client or a server.

[152] The CPU 1905 may execute a sequence of machine-readable instructions, which may be embodied in a program or software. The instructions may be stored in a memory location, such as the memory 1910. The instructions may be directed to the CPU 1905, which may subsequently program or otherwise configure the CPU 1905 to implement methods of the present disclosure. Examples of operations performed by the CPU 1905 may include fetch, decode, execute, and writeback.

[153] The CPU 1905 may be part of a circuit, such as an integrated circuit. One or more other components of the system 1901 may be included in the circuit. In some cases, the circuit is an application specific integrated circuit (ASIC).

[154] The storage unit 1915 may store files, such as drivers, libraries and saved programs. The storage unit 1915 may store user data, e.g., user preferences and user programs. The computer system 1901 in some cases may include one or more additional data storage units that are external to the computer system 1901, such as located on a remote server that is in communication with the computer system 1901 through an intranet or the Internet.

[155] The computer system 1901 may communicate with one or more remote computer systems through the network 1930. For instance, the computer system 1901 may communicate with a remote computer system of a user (e.g., Located at a physician’s office or a physician’s mobile device). Examples of remote computer systems include personal computers (e.g., portable PC), slate or tablet PC’s (e.g., Apple® iPad, Samsung® Galaxy Tab), telephones, Smart phones (e.g., Apple® iPhone, Android-enabled device, Blackberry®), or personal digital assistants. The user may access the computer system 1901 via the network 1930.

[156] Methods as described herein may be implemented by way of machine (e.g., computer processor) executable code stored on an electronic storage location of the computer system 1901, such as, for example, on the memory 1910 or electronic storage unit 1915. The machine executable or machine-readable code may be provided in the form of software. During use, the code may be executed by the processor 1905. In some cases, the code may be retrieved from the storage unit 1915 and stored on the memory 1910 for ready access by the processor 1905. In some situations, the electronic storage unit 1915 may be precluded, and machine-executable instructions are stored on memory 1910.

[157] The code may be pre-compiled and configured for use with a machine having a processer adapted to execute the code or may be compiled during runtime. The code may be supplied in a programming language that may be selected to enable the code to execute in a pre-compiled or as-compiled fashion.

[158] Aspects of the systems and methods provided herein, such as the computer system 1901, may be embodied in programming. Various aspects of the technology may be thought of as “products” or “articles of manufacture” typically in the form of machine (or processor) executable code and / or associated data that is carried on or embodied in a type of machine readable medium. Machine-executable code may be stored on an electronic storage unit, such as memory (e.g., read-only memory, random-access memory, flash memory) or a hard disk. “Storage” type media may include any or all of the tangible memory of the computers, processors or the like, or associated modules thereof, such as various semiconductor memories, tape drives, disk drives and the like, which may provide non-transitory storage at any time for the software programming. All or portions of the software may at times be communicated through the Internet or various other telecommunication networks. Such communications, for example, may enable loading of the software from one computer or processor into another, for example, from a management server or host computer into the computer platform of an application server. Thus, another type of media that may bear the software elements includes optical, electrical and electromagnetic waves, such as used across physical interfaces between local devices, through wired and optical landline networks and over various air-links. The physical elements that carry such waves, such as wired or wireless links, optical links or the like, also may be considered as media bearing the software. As used herein, unless restricted to non-transitory, tangible “storage” media, terms such as computer or machine “readable medium” refer to any medium that participates in providing instructions to a processor for execution.

[159] Hence, a machine readable medium, such as computer-executable code, may take many forms, including but not limited to, a tangible storage medium, a carrier wave medium or physical transmission medium. Non-volatile storage media include, for example, optical or magnetic disks, such as any of the storage devices in any computer(s) or the like, such as may be used to implement the databases, etc. shown in the drawings. Volatile storage media include dynamic memory, such as main memory of such a computer platform. Tangible transmission media include coaxial cables; copper wire and fiber optics, including the wires that comprise a bus within a computer system. Carrier-wave transmission media may take the form of electric or electromagnetic signals, or acoustic or light waves such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media therefore include for example: a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD or DVD-ROM, any other optical medium, punch cards paper tape, any other physical storage medium with patterns of holes, a RAM, a ROM, a PROM and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave transporting data or instructions, cables or links transporting such a carrier wave, or any other medium from which a computer may read programming code and / or data. Many of these forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to a processor for execution.

[160] The computer system 1901 may include or be in communication with an electronic display 1935 that comprises a user interface (UI) 1940.Examples of UI’s include, without limitation, a graphical user interface (GUI) and web-based user interface.

[161] Methods and systems of the present disclosure may be implemented by way of one or more algorithms. An algorithm may be implemented by way of software upon execution by the central processing unit 1905. The algorithm can, for example, process data, perform statistical analyses, plot or graphically represent data, and provide feedback for one or more systems disclosed herein (e.g., the patch and / or injector).

[162] Methods and systems of the present disclosure may be implemented by way of one or more digital processing devices. Non-limiting examples of digital processing devices may comprise server computers, desktop computers, laptop computers, notebook computers, sub-notebook computers, netbook computers, netpad computers, set-top computers, media streaming devices, handheld computers, internet appliances, mobile smartphones, tablet computers, personal digital assistants, video game consoles, and vehicles.

[163] In some embodiments, the digital processing device may comprise an operating system configured to perform executable instructions. The operating system is, for example, software, including programs and data, which manages the device’s hardware and provides services for execution of applications. Non-limiting examples of operating systems may comprise FreeBSD, OpenBSD, NetBSD®, Linux, Apple® Mac OS X Server®, Oracle® Solaris®, Windows Server®, and Novell® NetWare®. Non-limiting examples of mobile phone operating systems may comprise include Nokia® Symbian® OS, Apple® iOS®, Research In Motion® BlackBerry OS®, Google® Android®, Microsoft® Windows Phone® OS, Microsoft® Windows Mobile® OS, Linux®, and Palm® WebOS®.

[164] In some embodiments, a computer program may comprise a mobile application provided to a mobile digital processing device. In some embodiments, the mobile application is provided to a mobile digital processing device at the time it is manufactured. In other embodiments, the mobile application is provided to a mobile digital processing device via the computer network described herein. The mobile applications may be written in several programming languages. Non-limiting examples of programming languages may comprise C, C++, C#, Objective-C, Java™, JavaScript, Pascal, Object Pascal, Python™, Ruby, VB.NET, WML, and XHTML / HTML with or without CSS, or combinations thereof.Example Machine Learning Techniques

[165] In some cases, the systems, the methods, the computer-readable media, and the techniques disclosed herein may use various machine learning (ML) techniques. In some cases, ML may generally involve identifying and recognizing patterns in existing data in order to facilitate making predictions for subsequent data. ML may include a ML model (which may include, for example, a ML algorithm). Machine learning, whether analytical or statistical in nature, may provide deductive or abductive inference based on real or simulated data. The ML model may be a trained model. ML techniques may comprise one or more supervised, semi-supervised, self-supervised, or unsupervised ML techniques. For example, an ML model may be a trained model that is trained through supervised learning (e.g., various parameters are determined as weights or scaling factors). ML may comprise one or more of regression analysis, regularization, classification, dimensionality reduction, ensemble learning, meta learning, association rule learning, cluster analysis, anomaly detection, deep learning, or ultra-deep learning. ML may comprise, but is not limited to: k-means, k-means clustering, k-nearest neighbors, learning vector quantization, linear regression, non-linear regression, least squares regression, partial least squares regression, logistic regression, stepwise regression, multivariate adaptive regression splines, ridge regression, principal component regression, least absolute shrinkage and selection operation (LASSO), least angle regression, canonical correlation analysis, factor analysis, independent component analysis, linear discriminant analysis, multidimensional scaling, non-negative matrix factorization, principal components analysis, principal coordinates analysis, projection pursuit, Sammon mapping, t-distributed stochastic neighbor embedding, AdaBoosting, boosting, gradient boosting, bootstrap aggregation, ensemble averaging, decision trees, conditional decision trees, boosted decision trees, gradient boosted decision trees, random forests, stacked generalization, Bayesian networks, Bayesian belief networks, naïve Bayes, Gaussian naïve Bayes, multinomial naïve Bayes, hidden Markov models, hierarchical hidden Markov models, support vector machines, encoders, decoders, auto-encoders, stacked auto-encoders, perceptrons, multi-layer perceptrons, artificial neural networks, feedforward neural networks, convolutional neural networks, recurrent neural networks, long short-term memory, deep belief networks, deep Boltzmann machines, deep convolutional neural networks, deep recurrent neural networks, generative adversarial networks, vision transformers, long short-term memory networks (LSTM), masked autoencoders, etc.

[166] Training the ML model may include, in some cases, selecting one or more untrained data models to train using a training data set. The selected untrained data models may include any type of untrained ML models for supervised, semi-supervised, self-supervised, or unsupervised machine learning. The selected untrained data models be specified based upon input (e.g., user input) specifying relevant parameters to use as predicted variables or other variables to use as potential explanatory variables. For example, the selected untrained data models may be specified to generate an output (e.g., a prediction) based upon the input. Conditions for training the ML model from the selected untrained data models may likewise be selected, such as limits on the ML model complexity or limits on the ML model refinement past a certain point. The ML model may be trained (e.g., via a computer system such as a server) using the training data set. In some cases, a first subset of the training data set may be selected to train the ML model. The selected untrained data models may then be trained on the first subset of training data set using appropriate ML techniques, based upon the type of ML model selected and any conditions specified for training the ML model. In some cases, due to the processing power requirements of training the ML model, the selected untrained data models may be trained using additional computing resources (e.g., cloud computing resources). Such training may continue, in some cases, until at least one aspect of the ML model is validated and meets selection criteria to be used as a predictive model.

[167] In some cases, one or more aspects of the ML model may be validated using a second subset of the training data set (e.g., distinct from the first subset of the training data set) to determine accuracy and robustness of the ML model. Such validation may include applying the ML model to the second subset of the training data set to make predictions derived from the second subset of the training data. The ML model may then be evaluated to determine whether performance is sufficient based upon the derived predictions. The sufficiency criteria applied to the ML model may vary depending upon the size of the training data set available for training, the performance of previous iterations of trained models, or user-specified performance requirements. If the ML model does not achieve sufficient performance, additional training may be performed. Additional training may include refinement of the ML model or retraining on a different first subset of the training dataset, after which the new ML model may again be validated and assessed. When the ML model has achieved sufficient performance, in some cases, the ML may be stored for present or future use. The ML model may be stored as sets of parameter values or weights for analysis of further input (e.g., further relevant parameters to use as further predicted variables, further explanatory variables, further user interaction data, etc.), which may also include analysis logic or indications of model validity in some instances. In some cases, a plurality of ML models may be stored for generating predictions under different sets of input data conditions. In some embodiments, the ML model may be stored in a database (e.g., associated with a server).

[168] The systems, the methods, the computer-readable media, and the techniques disclosed herein may implement one or more computer vision techniques. Computer vision is a field of artificial intelligence that uses computers to interpret and understand the visual world at least in part by processing one or more digital images from cameras and videos. In some instances, computer vision may use deep learning models (e.g., convolutional neural networks). Bounding boxes and tracking techniques may be used in object detection techniques within computer vision.

[169] The systems, the methods, the computer-readable media, and the techniques disclosed herein may implement one or more deep-learning techniques. Deep learning is an example of ML that may be based on a set of algorithms that attempt to model high-level abstractions in data by using multiple processing layers, with complex structures or otherwise, composed of multiple non-linear transformations. In some cases, a drop out method can be used to reduce overfitting. At each training stage, individual nodes are either “dropped out” of the net (e.g., ignored) with probability 1−p or kept with probability p, so that a reduced network is left; incoming and outgoing edges to a dropped-out node may also be removed. In some cases, the reduced network is trained on the data in that stage. The removed nodes may then be reinserted into the network with their original weights.

[170] The systems, the methods, the computer-readable media, and the techniques disclosed herein may implement one or more vision transformer (ViT) techniques. A ViT is a transformer-like model that handles vision processing tasks. While CNNs use convolution, a “local” operation bounded to a small neighborhood of an image, ViTs use self-attention, a “global” operation, since the ViT draws information from the whole image. This allows the ViT to capture distant semantic relevance in an image effectively. Advantageously, ViTs may be well-suited catching long-term dependencies. In some cases, ViTs may be a competitive alternative to convolutional neural networks as ViTs may outperform the current state-of-the-art CNNs by almost four times in terms of computational efficiency and accuracy. ViTs may be well-suited to object detection, image segmentation, image classification, and action recognition. Moreover, ViTs may be applied in generative modeling and multi-model tasks, including visual grounding, visual-question answering, and visual reasoning. In some cases, ViTs may represent images as sequences, and class labels for the image are predicted, which allows models to learn image structure independently. Input images may be treated as a sequence of patches where every patch is flattened into a single vector by concatenating the channels of all pixels in a patch and then linearly projecting it to the desired input dimension. For example, a ViT architecture may include the following operations: (A) split an image into patches; (B) flatten the patches; (C) generate lower-dimensional linear embeddings from the flattened patches; (D) add positional embeddings; (E) provide the sequence as an input to a standard transformer encoder; (F) pretrain a model with image labels (e.g., fully supervised on a huge dataset); and (G) finetune on the downstream dataset for image classification. In some cases, there may be multiple blocks in a ViT encoder, with each block comprising three major processing elements: (1) Layer Norm; (2) Multi-head Attention Network; and (3) Multi-Layer Perceptrons. The Layer Norm may keep the training process on track and enable the model to adapt to the variations among the training images. The Multi-head Attention Network may be a network responsible for generating attention maps from the given embedded visual tokens. These attention maps may help the network focus on the most critical regions in the image, such as object(s). The Multi-Layer Perceptrons may be a two-layer classification network with a Gaussian Error Linear Unit at the end. The final Multi-Layer Perceptrons block may be used as an output of the transformer. An application of softmax on this output can provide classification labels (e.g., if the application is image classification).

[171] The systems, the methods, the computer-readable media, and the techniques disclosed herein may implement one or more masked autoencoder (MAE) techniques. MAEs are scalable self-supervised learners for computer vision. The MAE leverages the success of autoencoders for various imaging and natural language processing tasks. Some computer vision models may be trained using supervised learning, such as using humans to look at images and created labels for the images, so that the model could learn the patterns of those labels (e.g., a human annotator would assign a class label to an image or draw bounding boxes around objects in the image). In contrast, self-supervised learning may not use any human-created labels. One technique for self-supervised image processing training using an MAE is for before an image is input into an encoder transformer, a certain set of masks are applied to the image. Due to the masks, pixels are removed from the image and therefore the model is provided an incomplete image. At a high level, the model’s task is to now learn what the full, original image looked like before the mask was applied.

[172] In other words, MAE may include masking random patches of an input image and reconstructing the missing pixels. The MAE may be based on two core designs. First, an asymmetric encoder-decoder architecture, with an encoder that operates on the visible subset of patches (without mask tokens), along with a lightweight decoder that reconstructs the original image from the latent representation and mask tokens. Second, masking a high proportion of the input image, e.g., 75%, may yield a nontrivial and meaningful self-supervisory task. Coupling these two core designs enables training large models efficiently and effectively, thereby accelerating training (e.g., by 3× or more) and improving accuracy. MAE techniques may be scalable, enabling learning of high-capacity models that generalize well, e.g., a vanilla ViT-Huge model. As mentioned, the MAE may be effective in pre-training ViTs for natural image analysis. In some cases, the MAE uses the characteristic of redundancy of image information to observe partial images to reconstruct original images as a proxy task, and the encoder of the MAE may have the capability of deducing the content of the masked image area by aggregating context information. This contextual aggregation capability may be important in the field of image processing and analysis.

[173] The systems, the methods, the computer-readable media, and the techniques disclosed herein may implement one or more decision tree or random forest techniques. A decision tree may be a supervised ML algorithm that can be applied to both regression and classification problems. Decision trees may mimic the decision-making process of a human brain. For example, a decision tree may grow from a root (base condition), and when it meets a condition (internal node / feature), it may split into multiple branches. The end of the branch that does not split anymore may be an outcome (leaf). A decision tree can be generated using a training data set according to the following operations: (1) Starting from a root node (the entire dataset), the algorithm may split the dataset in two branches using a decision rule or branching criterion; (2) each of these two branches may generate a new child node; (3) for each new child node, the branching process may be repeated until the dataset cannot be split any further; (4) each branching criterion may be chosen to maximize information gain (e.g., a quantification of how much a branching criterion reduces a quantification of how mixed the labels are in the children nodes). The labels may be the data or the classification that is predicted by the decision tree.

[174] A random forest regression is an extension of the decision tree model that tends to yield more robust predictions by stretching the use of the training data partition. Whereas a decision tree may make a single pass through the data, a random forest regression may bootstrap 50% of the data (e.g., with replacement) and build many trees. Rather than using all explanatory variables as candidates for splitting, a random subset of candidate variables may be used for splitting, which may enable trees that have completely different data and different variables (hence the term random). The predictions from the trees, collectively referred to as the “forest,” may be then averaged together to produce the final prediction. Many trees (e.g., one hundred trees) may be included in a random forest model, with a number (e.g., 3, 6, 10, etc.) of terms sampled per split, a minimum of number (e.g., 1, 2, 4, 10, etc.) of splits per tree, and a minimum split size (e.g., 16, 32, 64, 128, 256, etc.). Random forests may be trained in a similar way as decision trees. Specifically, training a random forest may include the following operations: (1) select randomly k features from the total number of features; (2) create a decision tree from these k features using the same operations as for generating a decision tree; and (3) repeat the previous two operations until a target number of trees is created.

[175] The systems, the methods, the computer-readable media, and the techniques disclosed herein may implement one or more long short-term memory (LSTM) techniques. LSTM may be an artificial neural network used in the fields of artificial intelligence and deep learning. Unlike standard feedforward neural networks, LSTM may use feedback connections. The LSTM architecture may provide a short-term memory for a recurrent neural network (RNN). Such RNN can process not only single data points (such as images), but also entire sequences of data (such as speech or video). This characteristic may mean that LSTM networks are well-suited for processing and predicting data. The name of LSTM may refer to the analogy that a standard RNN has both “long-term memory” and “short-term memory.” The connection weights and biases in the RNN may change once per episode of training, analogous to how physiological changes in synaptic strengths store long-term memories; the activation patterns in the network may change once per time-step, analogous to how the moment-to-moment change in electric firing patterns in the brain store short-term memories. The LSTM architecture may provide a short-term memory for an RNN that can last many (e.g., thousands) timesteps.

[176] In some cases, a LSTM unit may comprise a cell, an input gate, an output gate, and a forget gate. The cell may remember values over arbitrary time intervals and the input gate, the output gate, and the forget gate may regulate the flow of information into and out of the cell. Forget gates may be used to decide what information to discard from a previous state by assigning a previous state, compared to a current input, a value between 0 and 1 (e.g., a (rounded) value of 1 may mean to keep the information, and a value of 0 means to discard it). The input gate may decide which pieces of new information to store in the current state, using the same system as the forget gates. The output gate may control which pieces of information in the current state to output (e.g., by assigning a value from 0 to 1 to the information, considering the previous and current states). Selectively outputting relevant information from the current state may allow the LSTM network to maintain useful, long-term dependencies to make predictions, both in current and future time-steps. LSTM networks may be well-suited to classifying, processing and making predictions based on time series data, since there can be lags of unknown duration between important events in a time series. LSTMs may resolve the vanishing gradient problem that can be encountered when training traditional RNNs. Relative insensitivity to gap length may be an advantage of LSTM over RNNs, hidden Markov models and other sequence learning methods in numerous applications.

[177] In some cases, LSTMs may be used with one or more various types of neural networks (e.g., convolutional neural networks (CNNs), deep neural network (DNNs), recurrent neural networks (RNNs), etc.). In some cases, CNNs, LSTM, and DNNs are complementary in their modeling capabilities and may be combined a unified architecture. For example, in such unified architecture, CNNs may be well-suited at reducing frequency variations, LSTMs may be well-suited at temporal modeling, and DNNs may be well-suited for mapping features to a more separable space. For example, input features to a ML model using LSTM techniques in the unified architecture may include segment features for each of a plurality of segments. To process the input features for each of the plurality of segments, the segment features for the segment may be processed using one or more CNN layers to generate first features for the segment; the first features may be processed using one or more LSTM layers to generate second features for the segment; and the second features may be processed using one or more fully connected neural network layers to generate third features for the segments, where the third features may be used for classification operations. In some examples, to process the first features using the one or more LSTM layers to generate the second features, the first features may be processed using a linear layer to generate reduced features having a reduced dimension from a dimension of the first features; and the reduced features may be processed using the one or more LSTM layers to generate the second features. Short-term features having a first number of contextual frames may be generated based on the input features, where features generated using the one or more CNN layers may include long-term features having a second number of contextual frames that are more than the first number of contextual frames of the short-term features. In some cases, the one or more CNN layers, the one or more LSTM layers, and the one or more fully connected neural network layers may have been jointly trained to determine trained values of parameters of the one or more CNN layers, the one or more LSTM layers, and the one or more fully connected neural network layers. In some cases, the input features may include log-mel features having multiple dimensions. The input features may include one or more contextual frames indicating a temporal context of a signal (e.g., input data). Advantageously, implementations for such unified architecture may leverage complementary advantages associated with each of a CNN, LSTM, and DNN. For example, convolutional layers may reduce spectral variation in input, which may help the modeling of LSTM layers. Having DNN layers after LSTM layers may help reduce variation in the hidden states of the LSTM layers. Training the unified architecture jointly may provide a better overall performance. Training in the unified architecture may also remove the need to have separate CNN, LSTM and DNN architectures, which may be expensive (e.g., in computational resource, in network traffic, in financial resources, in energy consumption, etc.). By adding multi-scale information into the unified architecture, information may be captured at different time scales.

[178] The systems, the methods, the computer-readable media, and the techniques disclosed herein may implement one or more support vector machine learning techniques. In machine learning, support vector machines (SVMs) may be supervised learning models with associated learning algorithms that analyze data for classification and regression analysis. SVMs may be a robust prediction method, being based on statistical learning. SVMs may be well-suited for domains characterized by the existence of large amounts of data, noisy patterns, or the absence of general theories.

[179] In general terms, SVMs may map input vectors into high dimensional feature space through non-linear mapping function, chosen a priori. In this high dimensional feature space, an optimal separating hyperplane may be constructed. The optimal hyperplane may then be used to determine things such as class separations, regression fit, or accuracy in density estimation. More formally, a SVM constructs a hyperplane or set of hyperplanes in a high or infinite-dimensional space, which can be used for classification, regression, or other tasks like outlier detection.

[180] Support vectors may be defined as the data points that lie closest to the decision surface (or hyperplane). Support vectors may therefore be the data points that are most difficult to classify and may have direct bearing on the optimum location of the decision surface. Given a set of training examples, each marked as belonging to one of two categories, an SVM training algorithm may build a model that assigns new examples to one category or the other, making it a non-probabilistic binary linear classifier (although methods such as Platt scaling exist to use SVM in a probabilistic classification setting). SVM may map training examples to points in space so as to maximize the width of the gap between the two categories. New examples may then be mapped into that same space and predicted to belong to a category based on which side of the gap they fall. In addition to performing linear classification, SVMs can efficiently perform a non-linear classification using what is called the kernel trick, implicitly mapping their inputs into high-dimensional feature spaces.

[181] Within a support vector machine, the dimensionally of the feature space may be large. For example, a fourth-degree polynomial mapping function may cause a 200-dimensional input space to be mapped into a 1.6 billionth dimensional feature space. The kernel trick and the Vapnik-Chervonenkis dimension may allow the SVM to thwart the “curse of dimensionality” limiting other methods and effectively derive generalizable answers from this very high dimensional feature space. Accordingly, SVMs may assist in discovering knowledge from vast amounts of input data.

[182] The systems, the methods, the computer-readable media, and the techniques disclosed herein may implement one or more gradient boosting techniques. Gradient boosting is a machine learning technique used in regression and classification tasks, among others. It gives a prediction model in the form of an ensemble of weak prediction models, which are typically decision trees. When a decision tree is the weak learner, the resulting algorithm is called gradient-boosted trees; it usually outperforms random forest. A gradient-boosted trees model is built in a stage-wise fashion as in other boosting methods, but it generalizes the other methods by allowing optimization of an arbitrary differentiable loss function.

[183] The systems, the methods, the computer-readable media, and the techniques disclosed herein may implement one or more K-nearest neighbors (KNN) techniques. KNN is a non-parametric classification method. In KNN classification, the output is a class membership. An object is classified by a plurality vote of its neighbors, with the object being assigned to the class most common among its k nearest neighbors (k is a positive integer, typically small). If k=1, then the object is assigned to the class of that single nearest neighbor. In KNN regression, the output is the property value for the object. This value is the average of the values of k nearest neighbors. KNN is a type of classification where the function is approximated locally, and computation is deferred until function evaluation. Since this algorithm relies on distance for classification, if the features represent different physical units or come in vastly different scales then normalizing the training data can improve its accuracy dramatically.

[184] The systems, the methods, the computer-readable media, and the techniques disclosed herein may implement one or more Monte Carlo techniques. Monte Carlo is a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results. The underlying concept is to use randomness to solve problems that might be deterministic in principle.

[185] The systems, the methods, the computer-readable media, and the techniques disclosed herein may implement one or more one-hot encoding techniques. One-hot encoding can be used to deal with categorical data. For example, an ML model can use input variables that are numeric. In one example, the categorical variables can be transformed in a pre-processing part. Categorical data can be either nominal or ordinal. Ordinal data can have a ranked order for its values and can therefore be converted to numerical data through ordinal encoding.

[186] The systems, the methods, the computer-readable media, and the techniques disclosed herein may implement one or more quadratic discriminant analysis (QDA) techniques. In some cases, QDA may assume that the measurements from each class are normally distributed. In QDA there is no assumption that the covariance of each of the classes is identical. When the normality assumption is true, the best possible test for the hypothesis that a given measurement is from a given class is the likelihood ratio test.

[187] The systems, the methods, the computer-readable media, and the techniques disclosed herein may implement one or more response surface methodology (RSM) techniques. RSM may explore the relationships between several explanatory variables and one or more response variables. RSM can use a sequence of designed experiments to obtain an optimal response. RSM can use a factorial experiment or a fractional factorial design. This may be sufficient to determine which explanatory variables affect the response variable(s) of interest. Once it is suspected that only significant explanatory variables are left, then a more complicated design, such as a central composite design can be implemented to estimate a second-degree polynomial model, which is still only an approximation at best. However, the second-degree model can be used to optimize (e.g., maximize, minimize, or attain a specific target for) the response variable(s) of interest.

[188] The systems, the methods, the computer-readable media, and the techniques disclosed herein may implement one or more Synthetic Minority Oversampling Techniques (SMOTE). SMOTE is type of data augmentation for a minority class. SMOTE can select examples that are close in the feature space, drawing a line between the examples in the feature space and drawing a new sample at a point along that line.***

[189] While preferred embodiments of the present disclosure have been shown and described herein, such embodiments are provided by way of example only. It is not intended that the present disclosure be limited by the specific examples provided within the specification. While the present disclosure has been described with reference to the aforementioned specification, the descriptions and illustrations of the embodiments herein are not meant to be construed in a limiting sense. Numerous variations, changes, and substitutions will now occur without departing from the present disclosure. Furthermore, it shall be understood that all aspects of the present disclosure are not limited to the specific depictions, configurations or relative proportions set forth herein which depend upon a variety of conditions and variables. It should be understood that various alternatives to the embodiments of the present disclosure described herein may be employed in practicing the present disclosure. It is therefore contemplated that the present disclosure shall also cover any such alternatives, modifications, variations or equivalents. It is intended that the following claims define the scope of the present disclosure and that methods and structures within the scope of these claims and their equivalents be covered thereby. 

Claims

1. A method of determining an operational state of a medication delivery device, said method comprising:a) instructing a user to capture one or more images of at least a portion of said medication delivery device using an image sensor; andb) determining said operational state of said medication delivery device by analyzing said one or more images,wherein said determining comprises assessing a position of said at least said portion of said medication delivery device.

2. The method of claim 1, further comprising transferring said one or more images to a cloud-based server.

3. The method of claim 1 or 2, wherein a medication is delivered by advancing a movable component inside said medication delivery device.

4. The method of claim 3, wherein said movable component advances in a trajectory that is not straight.

5. The method of claim 3, wherein an amount of medication delivered is in a linear relationship with a distance that said movable component advances.

6. The method of claim 3, wherein an amount of medication delivered is in a non-linear relationship with a distance that said movable component advances.

7. The method of any one of claims 1-6, wherein said determining comprises determining a position of a button cap of said medication delivery device.

8. The method of claim 7, wherein said position of said button cap comprises a ready to inject position, an inject position, a pause position, or a lockout position.

9. The method of any one of claims 1-8, wherein said determining comprises determining a position of a needle of said medication delivery device.

10. The method of claim 9, wherein said position of said needle comprises a retracted position or an extended position.

11. The method of any one of claims 1-10, wherein said determining comprises determining a position of a gas gauge of said medication delivery device.

12. The method of claim 11, wherein said position of said gas gauge comprises a ready to inject position, an inject position, a pause position, a lockout position, or a complete position.

13. The method of any one of claims 1-12, further comprising assigning a first operational state at a time a first image is captured, assigning a second operational state at a time a second image is captured, and determining a volume delivered between said time said first image is captured and said time said second image is captured.

14. The method of any one of claims 1-13, wherein said operational state comprises an injection started state, an injection paused state, an injection in progress state, or an injection completed state.

15. The method of any one of claims 1-14, wherein said operational state comprises a delivered volume state, a remaining volume state, and / or a remaining time until completion state.

16. The method of any one of claims 1-15, wherein said image sensor is a digital camera or a digital video camera.

17. The method of any one of claims 1-16, wherein said analyzing is performed by comparing said one or more images with a pre-determined image.

18. The method of any one of claims 1-17, wherein said analyzing is performed with a mobile phone.

19. The method of any one of claims 1-18, wherein said analyzing is performed with a software application.

20. The method of any one of claims 1-19, further comprising displaying said operational state.

21. The method of any one of claims 1-20, further comprising receiving input information from said user and calculating a medication dosage amount to be administered in a subsequent delivery.

22. The method of any one of claims 1-21, wherein said image sensor comprises instructions for said user to guide positioning of said image sensor relative to said at least said portion of said medication delivery device while capturing said one or more images.

23. The method of claim 22, further comprising instructing said user to match a device outline displayed on said image sensor with said at least said portion of said medication delivery device while capturing said one or more images.

24. The method of any one of claims 1-23, further comprising storing said one or more images.

25. The method of any one of claims 1-24, further comprising storing information of said operational state.

26. The method of any one of claims 1-25, further comprising alerting said user if said operational state is deviant from a pre-determined state.

27. A system for determining an operational state of a medication delivery device, said system comprising:(a) an image sensor for capturing one or more images of at least a portion of said medication delivery device;(b) a data transmission interface for sending said one or more images to an electronic device; and(c) a processor in communication with said electronic device for analyzing said one or more images and determining said operational state.

28. The system of claim 27, wherein said image sensor comprises a digital camera or a digital video camera.

29. The system of claim 27 or 28, wherein said medication delivery device comprises a patch and an injector, wherein said patch and said injector are removably coupled.

30. The system of any one of claims 27-29, wherein said at least said portion of said medication delivery device comprises a button.

31. The system of any one of claims 27-30, wherein said at least said portion of said medication delivery device comprises a needle.

32. The system of any one of claims 27-31, wherein said at least said portion of said medication delivery device comprises a gas gauge.

33. The system of any one of claims 27-32, wherein said at least said portion of said medication delivery device is configured to alter a position indicative of said operational state.

34. The system of any one of claims 27-33, wherein said processor is configured to operate a software application.

35. The system of any one of claims 27-34, further comprising a non-transitory computer-readable storage media encoded with a computer program including instructions executable by said processor.

36. The system of any one of claims 27-35, further comprising a data reporting accessory to provide information to a user.

37. The system of any one of claims 27-36, further comprising a data display accessory to display said operational state.

38. The system of any one of claims 27-37, wherein said image sensor, said data transmission interface, and said processor are in a common housing.

39. The system of claim 38, wherein said common housing is a smartphone.

40. A computer-implemented on-boarding method for monitoring a medication delivery, said method comprising:%2) receiving an account information from a user to create a new user account; %2) displaying an overview of an application, a safety information, a training video, or any combination thereof;%2) receiving a data consent, a confirmation from said user to accept notifications, a symptom tracking information, or any combination thereof;%2) generating a patient profile and an injection schedule; and%2) establishing connection with a medication delivery device of said user.

41. The method of claim 40, wherein displaying said overview of said application comprises displaying an infographic regarding how to record and track injections, set reminders to take medications, create a schedule and reminder for treatment appointments, track and identify symptoms to the user’s healthcare team, access a support program, or any combination thereof.

42. The method of claim 40 or 41, wherein displaying said safety information comprises displaying a drug prescription information.

43. The method of any one of claims 40-42, wherein generating said patient profile comprises receiving a user date of birth, a user gender, a user name, a user age, a user weight, a user height, a user ethnicity, a user nationality, a user residential address, a user zip code, a user e-mail, a user phone number, a user contact information, or any combination thereof.

44. The method of any one of claims 40-43, wherein generating said injection schedule comprises receiving a dose frequency, an infusion date, an infusion time, or any combination thereof.

45. The method of any one of claims 40-44, wherein establishing connection with said medication delivery device of said user is performed via a wireless communication protocol, a machine-readable barcode, or both.

46. The method of claim 45, wherein the wireless communication protocol comprises near-field communication (NFC), Bluetooth, Bluetooth LE, Wi-Fi, radio frequency identification (RFID), iBeacon, ZigBee, Z-Wave, or any combination thereof.

47. The method of any one of claims 40-46, wherein establishing connection with said medication delivery device of said user comprises instructing said user to:a) activate a wireless communication protocol for said medication delivery device;b) position a mobile computing device of the user in proximity to said medication delivery device, the packaging of said medication delivery device, or both, wherein the mobile computing device is positioned in sufficient proximity to activate wireless communication between the mobile computing device and said medication delivery device;c) remove a battery isolation tab to activate said medication delivery device; or d) any combination thereof.

48. The method of claim 47, further comprising bonding said medication delivery device with said application.

49. The method of claim 48, wherein said bonding is performed when said application is in a background of a mobile computing device of the user.

50. The method of claim 48 or 49, wherein said bonding is performed in the absence of opening said application on a mobile computing device.

51. The method of any one of claims 48-50, wherein said medication delivery device or said packaging comprises a bonding stimulus.

52. The method of any one of claims 48-51, wherein said bonding stimulus is provided by a wireless communication protocol, a machine-readable barcode, or both.

53. The method of any one of claims 47-52, further comprising providing a notification to the user when the medication delivery device has established a connection with said application.

54. The method of any one of claims 47-53, wherein said medication delivery device comprises a syringe, a wearable autoinjector, a wearable infusion pump, a device directly attached for subcutaneous delivery, a device indirectly attached for subcutaneous delivery, or an autoinjector pen.

55. A non-transitory computer-readable storage media coupled to a processor and having instructions stored thereon which, when executed by said processor, cause said processor to implement an application comprising:%2) an infusion module instructing a user to gather supplies, scan a drug, warm said drug, transfer said drug to a transfer base and press down, select an injection site, remove an injection device from said transfer base, attach said injection device to said user with a view window facing up, or any combination thereof; and%2) a progress monitor module displaying an injection status, an injector on / off body status, an injection progress, a remaining injection time, or any combination thereof.

56. The media of claim 55, wherein said application further comprises a home module displaying views of next infusion times, or views of missed infusion notifications, or any combination thereof.

57. The media of claim 56, wherein said home module displays a notification if a scanned drug is determined to be expired, incorrect, or both.

58. The media of any one of claims 55-57, wherein said infusion module is configured to receive a stomach quadrant selection representing a location in which said injection device is attached.

59. The media of any one of claims 55-58, wherein said application further comprises a record module displaying an overview of said user’s adherence to an injection schedule and / or indications of symptom progression.

60. The media of claim 59, wherein said record module is configured to receive an indication of a complete injection.

61. The media of claim 59, wherein said record module is configured to determine said user’s adherence to said injection schedule based on said indication of complete injection.

62. The media of any one of claims 55-61, wherein said application further comprises a settings module displaying said user’s patient profile, infusion schedule, notification settings, or any combination thereof.

63. The media of claim 62, wherein said settings module is configured to receive an edit from said user to said user’s patient profile, said infusion schedule, said notification settings, or any combination thereof.

64. The media of claim 63, wherein said user’s patient profile comprises a user name, a user date-of-birth, a user name, a user age, a user weight, a user height, a user ethnicity, a user nationality, a user residential address, a user zip code, a user e-mail, a user phone number, or any combination thereof.

65. The media of any one of claims 55-64, wherein said application further comprises a login module receiving a symptom notification and an injection notification, wherein said login module is accessible via said home module.

66. The media of any one of claims 55-65, wherein said application further comprises an injection complete module displaying a notification that the injection is complete.

67. The media of claim 66, wherein said progress monitor module comprises said injection complete module.

68. The media of any one of claims 55-67, wherein said application further comprises a schedule module receiving a user response to a symptom questionnaire.

69. The media of any one of claims 55-68, wherein said application further comprises a resources module displaying instructions for use document, a demo video, a terms and conditions document, a privacy policy document, or any combination thereof. 

Images