Systems, devices, and methods for capturing images of medical condition management events, and associated devices with smartphones and associated apps for processing images to reduce medical errors

Abstract

A device and method including a camera and a medical event image capture app are provided for capturing images of a medical condition management event involving a product bearing an indicator, performing image processing and analysis of the captured image to identify product characteristics, artifacts, and / or information from the indicators and other image elements in the captured image, and performing human-machine interaction (HMI) operations or other logical operations to alert a user about selected information, prompt for input, or educate a user about aspects of medical condition management. The medical event image capture app provides one or more of correct or compatible product verification, dosage verification, and defective product or medication or misuse detection. The medical event image capture app provides automated recording of medical event data to a patient's electronic record to support healthcare management, including billing, medical product inventory and reordering, care plan compliance, and clinical effectiveness.

Description

[Technical Field]

[0001] Exemplary embodiments generally relate to the use of portable, handheld devices such as smartphones to capture images of medical condition management events involving medical equipment, and associated smartphone apps that process the images and interact with a user. Exemplary embodiments generally relate to medical event image capture apps that process images of medical condition management events to reduce medical errors that can occur due to using incompatible injection devices and medications, drawing up the wrong amount of medication into a syringe or injection pen before administration, using contaminated or incorrect medications or damaged or incorrect injection supplies, etc. [Background technology]

[0002] Description of Related Art Medication non-adherence is a significant problem worldwide, particularly with regard to diabetes treatment. Fifty percent (50%) of all patients do not take their medications as prescribed. Non-adherence directly contributes to hundreds of thousands of deaths and billions of dollars in avoidable medical and related costs.

[0003] There are smartphone apps that use photos of prescription labels to help patients reorder prescription medications when they are low on stock, but these apps do not directly verify the medication or dosage before the patient takes it, and they do not work with syringes or pens.

[0004] There are smartphone apps that assist users in recording medical events such as injections, and there are smart injection devices that can assist users by automatically recording the amount dialed for administration and / or the administered amount of medication.

[0005] Nevertheless, there remains a continuing need for methods and devices to assist users (e.g., patients, their caregivers, their healthcare providers, and other medical condition management stakeholders, such as payers / insurance companies, pharmacies, or medical product suppliers or distributors) in obtaining and utilizing information related to medical condition management events to prevent medical errors, such as medication errors, as well as to improve related processes, such as refilling medical supplies, tracking compliance with medical condition management protocols or regimens, and sharing information among medical condition management stakeholders for the purposes of optimal patient treatment planning, billing, and insurance coverage. Summary of the Invention

[0006] The above-referenced and other problems are overcome, and additional advantages are realized, by the illustrative embodiments.

[0007] According to an aspect of an exemplary embodiment, a portable device for capturing images of a medical event to reduce medical errors includes an imaging device for imaging at least one medical product used during the medical event, a memory storing images captured by the imaging device and program instructions for processing the captured images, a user interface configured to generate output to a user, and a processor. The processor is adapted to execute the program instructions to analyze the captured images associated with the medical event to detect characteristics of the medical product selected from a group consisting of indicia on the medical product and specified attributes of the medical product, and analyze the detected characteristics to determine when a medical error occurs. A medical error refers to a medical product being incompatible with the medical event, a user error, or an anomaly. The processor is configured to generate output to the user via the user interface including an alert related to the medical error.

[0008] In one aspect of an exemplary embodiment, there is provided a portable device having at least one captured image in its memory corresponding to a medical event involving at least two medical products used together, a processor configured to analyze the at least one captured image to detect respective indicators on the at least two medical products, and the portable device analyzing the respective indicators on the at least two medical products using previously stored medical product data locally or remotely accessible by the processor, the previously stored medical product data including, for each medical product of the plurality of different medical products, indicators for one or more other medical products indicated to be compatible with the medical product, and the processor generating an output to a user if it determines that the at least two medical products are incompatible according to the previously stored medical product data.

[0009] In one aspect of the exemplary embodiment, a portable device is provided, wherein the medical device is a drug administration device having an indicator, and the processor is configured to analyze a captured image of the drug administration device to detect the indicator, analyze the captured image of the drug on the drug administration device or other captured images of the drug on the drug administration device, and detect an amount of the drug indicated for administration by the drug administration device, and further use previously stored medical product data locally or remotely accessible by the processor, the previously stored medical product data including a plurality of different drug administration devices and their respective indicators and specifications of an amount of the specified drug that can be administered via the drug administration device for each of the plurality of different drug administration devices, the processor determines an amount of the specified drug corresponding to the drug administration device associated with the indicator detected from the captured image, and generates an alert via a user interface if the detected amount of the drug indicated for administration is determined to be different from the specified amount of the drug.

[0010] For example, the detected amount of medication indicated for administration corresponds to a marking in the captured image associated with at least one of a dosage entered into the injection pen or a level indicator of the syringe barrel adjacent the fluid level in the syringe. As a further example, the processor analyzes the captured image or other captured images using an algorithm selected from a two-dimensional image processing algorithm and a three-dimensional image processing algorithm to detect the amount of medication indicated for administration by the medication administration device.

[0011] In one aspect of the exemplary embodiment, a portable device is provided, wherein the medical device is a drug administration device having an indicator, and the processor is configured to analyze a captured image of the drug administration device to detect the indicator, and further, using previously stored medical product data locally and / or remotely accessible by the processor, the previously stored medical product data including a plurality of different drug administration devices and their respective indicators and, for each drug administration device of the plurality of different drug administration devices, a specification of an amount of a specified drug that can be administered via the drug administration device, the processor determines an amount of the specified drug corresponding to the drug administration device associated with the indicator detected from the captured image, and generates an alert via a user interface if the amount of the drug indicated for prescribed administration is determined to differ from the specified drug amount.

[0012] The processor is configured to, for example, analyze the captured image of the drug of the drug administration device or the captured image of the drug of another drug administration device to detect an amount of the drug indicated for administration by the drug administration device, the detected amount of the drug indicated for administration corresponding to markings on the captured image associated with at least one of the dose entered into the injection pen and a level indicator on the syringe barrel adjacent the fluid level in the syringe, and further configured to generate an alert via the user interface if it is determined that the prescribed amount of the drug indicated for administration differs from the detected amount of the drug indicated for administration.

[0013] In one aspect of the exemplary embodiment, the medical device is a medication administration device, and the processor is configured to analyze a captured image of the medication administration device, detect an amount of medication indicated for administration by the medication administration device, and store the detected amount of medication indicated for administration in the memory device, e.g., the detected amount of medication indicated for administration corresponds to markings on the captured image associated with at least one of a dosage entered into the injection pen and a level indicator on the syringe barrel adjacent a fluid level in the syringe.

[0014] In one aspect of the exemplary embodiment, a portable device is provided in which the program instructions include at least one of a two-dimensional image processing algorithm and a three-dimensional image processing algorithm used by the processor to analyze the captured image.

[0015] In one aspect of an exemplary embodiment, a portable device is provided that is configured such that a processor analyzes at least one captured image to detect features of the medical product, including at least one specified attribute of the medical product selected from the group consisting of a selected color of the medical product, a selected dimension of the medical product, a selected form factor of the medical product, the presence of a safety feature on the medical product, and the absence of a safety feature on the medical product compared to stored images of medical products with the safety feature; and analyzes the detected features to determine whether a medical error has occurred using previously stored medical product data that is at least one of locally and remotely accessible by the processor, the previously stored medical product data including specified specifications of image characteristics of the medical product that correspond to the at least one specified attribute.

[0016] For example, the medical product may be a liquid medication being drawn into a syringe, and the at least one specified attribute of the liquid medication may be selected from the group consisting of opacity of the liquid medication, presence of air bubbles in the liquid medication, and presence of particles in the liquid medication. As a further example, the previously stored medical product data may include specified specifications of image characteristics of the at least one specified attribute of the liquid medication.

[0017] In one aspect of an exemplary embodiment, a portable device is provided, the portable device being at least one of a mobile phone and a computing device with a wireless communication interface, the memory configured to store at least one integrated disease management (IDM) app, the IDM app including an IDM personal app operated by a user and / or an IDM professional app operated by a healthcare professional, the processor is further adapted to execute instructions in accordance with the IDM app to operate the portable device in a cloud configuration with a remote IDM system, whereby the IDM app transfers data to and receives data from the IDM system during an app session.

[0018] In one aspect of the exemplary embodiment, a portable device is provided that operates in accordance with the IDM personal app and transfers and stores information from captured images to the IDM system, the information being selected from the group consisting of a dosage determined from at least one captured image, a date and / or timestamp of a medical event determined from at least one captured image, and / or a medical product identified from at least one captured image.

[0019] In one aspect of an exemplary embodiment, a portable device operating in accordance with an IDM Professional app is provided to determine patient information from information stored in an IDM system, the patient information including at least one of compliance data to a prescribed regimen based on dosage and information related to a date and / or time of a medical event, medical product prescription update data based on information related to the medical products identified from captured images corresponding to use of those medical products and the date and / or time of a medical event, and / or claims data corresponding to medical products identified from captured images corresponding to use of those medical products and the date and / or time of a medical event.

[0020] In one aspect of the exemplary embodiment, a mobile device capable of wirelessly connecting to at least one other medical condition management device and obtaining medical event information therefrom is provided, wherein the processor is further adapted to execute instructions according to an IDM app and to forward the medical event information to an IDM system.

[0021] In one aspect of an exemplary embodiment, a cloud configuration includes a private cloud and a public cloud, and a mobile device operating according to an IDM app is provided that determines whether at least one piece of information or other data related to a user stored in memory is proprietary or non-proprietary data, and selectively transfers the proprietary data via the private cloud and the non-proprietary data via the public cloud.

[0022] In one aspect of an exemplary embodiment, there is provided a portable device for capturing images of a medical event, the portable device including: an imaging device for imaging at least one medical product used during the medical event; a memory for storing images captured by the imaging device and program instructions for processing the captured images; a user interface configured to generate output to a user; and a processor adapted to execute the program instructions to analyze the captured images associated with the medical event to detect characteristics of the medical product selected from the group consisting of indicia on the medical product and specified attributes of the medical product, store data associated with the detected characteristics in the memory, and use the data associated with the detected characteristics to generate output to a user via the user interface.

[0023] In one aspect of the exemplary embodiment, a portable device is provided that is a monitor for a selected medical condition, and the detected characteristic is a monitored parameter that is detected by the monitor and indicated via a user interface associated with the monitor.

[0024] In one aspect of the exemplary embodiment, a portable device is provided that includes a process further adapted to execute program instructions for recording a date and / or time associated with the detected characteristic.

[0025] This is an aspect of an exemplary embodiment, where the monitor is selected from the group consisting of a pulse oximeter, a thermometer, a blood pressure monitor, and a blood glucose monitor.

[0026] Additional and / or other aspects and advantages of the exemplary embodiments will be set forth in, or will be obvious from, the description that follows, or may be learned by practice of, the exemplary embodiments. The exemplary embodiments may include apparatuses and methods for similar operation having one or more of the above-described aspects and / or one or more of the features or combinations thereof. The exemplary embodiments may include, for example, one or more features and / or combinations of the above-described aspects as set forth in the appended claims. [Brief explanation of the drawings]

[0027] The above and / or other aspects and advantages of embodiments of the present invention will be more readily understood from the following detailed description taken in conjunction with the accompanying drawings.

[0028] [Figure 1] FIG. 1 is a diagram illustrating a device with a medical event image capture app and various types of exemplary medication administration products, according to an exemplary embodiment. [Figure 2] FIG. 2 is a block diagram of a device with the medical event image capture app of FIG. 1, according to an exemplary embodiment. [Figure 3A] FIG. 3A illustrates a device with the medical event image capture app of FIG. 1 capturing an image of an exemplary medication administration product according to an exemplary embodiment. [Figure 3B] FIG. 3B illustrates a device with the medical event image capture app of FIG. 1 capturing an image of an exemplary medication administration product according to an exemplary embodiment. [Figure 3C] FIG. 3C illustrates a device with the medical event image capture app of FIG. 1 capturing an image of an exemplary medication administration product according to an exemplary embodiment. [Figure 3D] FIG. 3D illustrates a device with the medical event image capture app of FIG. 1 capturing an image of an exemplary medication administration product according to an exemplary embodiment. [Figure 3E] FIG. 3E illustrates a device with the medical event image capture app of FIG. 1 capturing an image of an exemplary medication administration product according to an exemplary embodiment. [Figure 4A] FIG. 4A illustrates a device with the medical event image capture app of FIG. 1 capturing an image of an exemplary medication administration product according to an exemplary embodiment. [Figure 4B] FIG. 4B illustrates a device with the medical event image capture app of FIG. 1 capturing an image of an exemplary medication administration product according to an exemplary embodiment. [Figure 5] FIG. 5 illustrates a device with the medical event image capture app of FIG. 1 capturing an image of an exemplary medication administration product according to an exemplary embodiment. [Figure 6] FIG. 6 is a flowchart including example operations performed by the device with the medical event image capture app of FIG. 1 according to an example embodiment. [Figure 7-1] 7A, 7B, and 7C are diagrams illustrating example GUI screens generated by a device with the medical event image capture app of FIG. 1 in accordance with an example embodiment. [Figure 7-2] 7D, 7E, and 7F illustrate example GUI screens generated by a device with the medical event image capture app of FIG. 1 in accordance with an example embodiment. [Figure 7-3] 7G and 7H illustrate exemplary GUI screens generated by a device with the medical event image capture app of FIG. 1 in accordance with an exemplary embodiment. [Figure 8-1] 8A, 8B, and 8C are diagrams illustrating example GUI screens generated by a device with the medical event image capture app of FIG. 1 according to an example embodiment. [Figure 8-2] 8D and 8E illustrate exemplary GUI screens generated by a device with the medical event image capture app of FIG. 1 in accordance with an exemplary embodiment. [Figure 9-1] FIG. 9A illustrates an exemplary GUI screen generated by a device with the medical event image capture app of FIG. 1 in accordance with an exemplary embodiment. [Figure 9-2] 9B and 9C illustrate exemplary GUI screens generated by a device with the medical event image capture app of FIG. 1 in accordance with an exemplary embodiment. [Figure 10] FIG. 10 illustrates an example integrated disease management system using devices with the medical event image capture app of FIG. 1, according to an example embodiment.

[0029] Throughout the drawings, like reference numerals will be understood to refer to like elements, features, and structures. DETAILED DESCRIPTION OF THE INVENTION

[0030] Reference will now be made in detail to the exemplary embodiments, which are illustrated in the accompanying drawings, and the embodiments described herein illustrate, by way of example only, exemplary embodiments.

[0031] 1 and 2 , according to an exemplary embodiment, a medical condition management event image capture app 40 is described herein as being able to be a standalone app on a smartphone 20 or other portable device with a camera (e.g., an iPad®) or as being able to be provided as an extension to a digital health (DH) app for a smartphone or other smart, connected device. The medical event image capture app 40 uses images of indicia 24 on a medical product 22, or images of the medical product 22 (e.g., a camera image of the product 22 with indicia 24, as shown at 25 in FIG. 7B ), to automate accessing and / or recording additional information to assist in medical condition management. The additional information from the captured medical product image is used by various functions of the app 40 to reduce medical errors (e.g., injection of an incorrect amount of medication due to misuse or defects in the medical product).

[0032] FIG. 1 illustrates an exemplary smartphone 20 equipped with a medical event image capture app 40 and several exemplary medical products 22, such as, but not limited to, injection pen needle assemblies and related injection pen products, syringe products, and safety syringe products (e.g., BDAutoShieldDuo® pen needles and BDSafetyFlide® 6 mm insulin syringes, which are designed to help prevent inadvertent needlestick injuries during injections in clinical settings, among other things). The products 22 can include various types of indicia 24, for example, printed or engraved directly thereon or indirectly applied to the products 22 using a label that includes the indicia. The indicia 24 can be one or more of alphanumeric text, symbols, different colors, and optically recognized codes such as barcodes, quick response (QR) codes, and universal product code (UPC) codes, among other types of indicia, an image of which can be captured via a camera and, according to one aspect of the exemplary embodiment, processed to decode relevant information about the item to which the indicia is applied or the associated event involving the use of the item.

[0033] Exemplary embodiments described herein relate, for example, to diabetes management and insulin injections. In exemplary embodiments, it can be appreciated that app processing for obtaining metrics, image capture, and medical condition management event information, as well as human-machine interactions based on that information, can be used to reduce errors associated with the management or treatment of other medical conditions that require the use of various devices and medical condition management procedures, such as surgical instruments, blood collection and transfusion products, and the administration of long- and short-acting medications other than insulin (e.g., medications related to hormone therapy, GLP-1s, treatments for rheumatoid arthritis or Crohn's disease, and other medications that require a dosing regimen and a level of control and monitoring). For example, exemplary embodiments can be used to reduce medical errors associated with self-injection with other types of medications, the correct use of surgical instruments for selected medical procedures, the correct use of equipment for intravenous administration of medical fluids to patients, and the like. Additionally, exemplary embodiments described herein are advantageous for a variety of different injection applications, such as veterinary treatments that use injection therapies other than human patient injection events.

[0034] Exemplary embodiments are described herein with respect to diabetes management and related injection products and events. However, it should be understood that these exemplary embodiments can be implemented with respect to other types of human and non-human animal medical conditions, medical events, and related condition management products. Furthermore, medical events need not be related to medication administration (e.g., they could instead be related to surgical instrument preparation). Furthermore, any medication administration application need not be limited to injections. For example, app 40 can be used to capture and manage oral and / or topical medication regimens. With respect to diabetes management, diabetes treatment companies manufacture numerous insulin administration or injection products that are essential for diabetes treatment for diabetic patients worldwide. These injection products are utilized by self-injecting patients and caregivers of diabetic patients and can include, but are not limited to, injection pens, pen needle assemblies, syringes, various needle sizes, and various types of insulin in various form factors (e.g., vials, pre-field syringes, injection pen cartridges). For example, a patient's injection regimen may require a selected syringe type, needle, and insulin type, so that use of the wrong type of insulin or syringe can affect the accuracy of the intended dose.

[0035] According to an exemplary embodiment, device 20 with medical event image capture app 40 (1) provides image capture and image processing of medical event devices to determine one or more of medical equipment correctness, dosage accuracy, and medication status, and (2) generates alerts and user guidance via a graphical user interface on device 20 to reduce medical errors. FIG. 2 is a block diagram illustrating exemplary device 20. While device 20 is referred to as a smartphone, it is understood that device 20 can be a dedicated medical management device or other portable, handheld device (e.g., an iPad®) with a device 28 for capturing or reading index images, such as a camera. Device 20 includes a processor 26 and a memory 36 that can store medical event image capture app 40 according to an exemplary embodiment, along with other device data, images, and apps. Device 20 can have one or more wireless communication interfaces 38, such as, for example, a Bluetooth®-enabled wireless communication interface and a cellular communication interface. The device 20 may also have one or more of a variety of user interfaces, such as a microphone 32, a touchscreen 30, or other display device for generating a graphical user interface (GUI) screen, such as a medical event image capture app 40, an optional keypad or other user input device (not shown), and an audio signal output device (e.g., a speaker or buzzer) 34.

[0036] The medical event image capture app 40 is program code that provides operations for capturing images of the indicia and / or injection products, as well as operations for processing the captured images. The captured image processing operations can (1) decode or otherwise identify artifacts and related information from the indicia and other image elements in the captured image, and (2) perform human-machine interaction (HMI) operations or other logical operations that alert the user regarding selected information and prompt or otherwise educate the user about the associated medical event. For example, the image capture operation captures an image from the device camera 28. The captured image processing operations can implement two-dimensional (2D) and / or three-dimensional (3D) image processing algorithms to detect selected artifacts from the captured image. The captured image processing operations can optionally include recognition operations, such as a QR code reader, a barcode or UPC code reader, or an optical character recognition (OCR) operation within the app 40. The HMI or other operation of the medical event image capture app 40 determines how the detected artifact affects the medical condition management event and generates a GUI screen or other HMI output (e.g., an audio query or message output to the user via a speaker) to educate the user or request user input.

[0037] According to an exemplary embodiment, at least three uses of medical event image capture app 40 are described with reference to Figures 3A-3E, 4A-4B, and 5, respectively: (1) correct injection device verification, (2) dosage verification, and (3) defective medication, device, or improper use detection. App 40 may provide only one of these uses, any two subsets of these uses, or all of these uses.

[0038] Regarding the first use of the medical event image capture app 40 (i.e., confirming the correct injection device 22), referring to FIGS. 3A through 3E, a smartphone 20 having the medical event image capture app 40 is depicted with one or more devices 22 (e.g., insulin vials and syringes) having indicia 24 in the image field 42 of the smartphone camera 28. The app 40 can recognize the correct injection device by virtue of image recognition, a QR code or other machine-readable code, color markings on the syringe or vial, or other distinguishing features. For syringes in particular, there may be various needle sizes, barrel capacities, and graduation marks specific to certain types of medications. The following example is provided with an insulin syringe, but is representative of most treatments using various types of medications. Some injection products, such as those commercially available from Becton, Dickinson and Company or "BD," already have unique markings indicating that they should be used with certain types of medications. These unique markings or indicia (e.g., QR codes) can be made readable by the app 40 via the app's image processing algorithms, thereby ensuring that the patient or caregiver is using the correct type of syringe for the correct insulin, thereby reducing medication errors.

[0039] For example, the app 40 can be programmed to reference locally stored or remotely accessed information, including a table or other data memory structure, for these unique markings 24 associated with specific injection products 22 for comparison or other analysis to identify the items 22 in the captured image. Alternatively, these unique markings or indicia 24 can be detected via the app 40 to automatically navigate the user to online educational materials (e.g., videos) regarding injections or other medical condition management techniques. For example, if the indicia 24 is a QR code, the app 40 can have a QR code scanner that converts the indicia 24 into some useful form (such as a standard URL for a website). The form can be a symbol or character that classifies the device (e.g., as belonging to a class of related products, as described below), or the decoded QR code can direct the smartphone 20 via the smartphone's browser to a URL for a web-based table to perform a lookup operation regarding the related product. Thus, the camera 28 of a smartphone 20 equipped with an app 40 having an integrated indicia reader can scan an image of a QR code or other indicia 24 on an item 22 to display text (e.g., contact information or instructions via a GUI screen 30), connect to a wireless network (e.g., to an HCP repository), or open a web page in the smartphone's browser. The app 40 can also generate a GUI screen or other alert if the medical device or product 22 appears to be a mismatch, as shown in FIG. 7F.

[0040] In the example shown in FIG. 3A , the medical event image capture app 40 can be programmed to detect an alphanumeric indicator 24 (e.g., “U-500”) in a captured image of an insulin vial and syringe held in front of the camera 28. The app 40 can then be programmed to reference locally stored or remotely accessed information including codes or indicators corresponding to respective families of medical products that are compatible when used together for injecting a precise dose (e.g., various sizes of syringes and corresponding types of insulin), thereby enabling the app 40 to verify whether the user is using compatible devices from the same family of medical devices for effective or precise dosing.

[0041] Continuing with reference to FIG. 3A , manufacturers of diabetes insulin and / or injection supplies typically provide indicia (e.g., stock keeping unit (SKU) numbers or other product identification indicia) on their respective products (e.g., syringes, insulin pens, needle assemblies, insulin vials and cartridges, and injection safety products, etc.). According to one aspect of the exemplary embodiment, manufacturers and other injection supply companies can generate a table of injection products matching selected product product codes linked to product families useful for administering injections, where the product families are given the selected codes (e.g., alphanumeric nomenclature or other machine-readable indicia, such as QR code 24 shown in FIG. 3B ). When a user operates smartphone 20 equipped with medical event image capture app 40 to capture an image of an item 22 the user is using to administer a self-injection or an injection to a patient, items 22 within field of view 42 of smartphone camera 28 are captured in the image, and processor 26 processes the image according to app 40.

[0042] The captured image may include, for example, image pixels representing items 22 of FIG. 3A , i.e., the vial and syringe. It will be appreciated that images of each item 22 of FIG. 3A used for injection may be captured separately, at the expense of the user, and / or processed separately via medical event image capture app 40, as described below, to identify indicia 24 or other event information, such as dose capture or the presence of air bubbles. Processor 26, under control of app 40, may process the image pixels representing the captured images of items 22 using 2D and / or 3D image analysis algorithms configured to identify one or more indicia 24 on items 22, such as a product code and / or product family code for each item 22. Once the indicia 24 are parsed from the image pixel data, the indicia 24 are decoded or otherwise identified.

[0043] For example, referring to FIG. 3B, a U40 syringe 22 is shown with an example QR code 24 that may be applied as part of the manufacturing process. The QR code indicates a 0.3 ml x 12.7 mm U40. When a user scans the code prior to injection, the app 40 recognizes the device 22 and either verifies that the correct device 22 is being used or alerts the user if not. For example, using a U40 syringe with U100 or U500 insulin will result in an incorrect dose of insulin being administered.

[0044] The indicia 24 can be a particular color or alphanumeric product name on a label (e.g., U-500), a product code (e.g., QR code 24 on syringe 22 in FIG. 3B), a product family identified in a QR code or other machine-readable code (i.e., with or without a product code), or a combination of indicia. The product family code can be, for example, a single character or multiple characters, and the characters can be alphanumeric, symbols, or other indicia. In one example, a table of compatible injection products can include product families “A, …, N” and can be stored locally or remotely with respect to device 20. If all of the captured images of item 22 have the same product family code “A,” processor 26 determines that the item 22 is compatible and optimized to deliver an accurate dosage. On the other hand, if processor 26 identifies two or more different product family codes in the captured image of item 22 (e.g., a QR code on a vial indicating product family "A" and a QR code on a syringe indicating product family "B"), processor 26, operated via medical event image capture app 40, can generate an alert to the user. For example, processor 26 can generate a GUI screen to be displayed on touchscreen 30 notifying the user of the detected item 22 non-conformity and, optionally, recommending a different size syringe of product family "A" instead of the image-captured syringe of family "B" for use with the detected vial from family "A."

[0045] Referring to Figure 3C, a BD AutoShield Pen Needle 22 is shown with an example QR code 24 that can be used to help promote better usage. In this case, the QR code 24 indicates "Injection Best Practices" and triggers functionality within the medical event image capture app 40 to play local content or take the user via the smartphone 20 browser to a website demonstrating good injection practices. Similar QR codes can also be used to enable patient education on other medical condition management topics.

[0046] The example product families stored locally or remotely in tables or other data memory structures accessed via the app 40 may also be defined according to the healthcare environment, i.e., a clinical environment where a healthcare provider (HCP) injects a patient, or a home healthcare environment where the patient self-injects or is injected by a home healthcare provider or family member. With further reference to FIG. 3C , the medical event image capture app 40 can optionally be configured to generate an alert to the HCP if a captured image of an injection product lacks a selected product family code 24 designated for an injection safety product. The alert can remind the HCP to use injection safety products (e.g., BD AutoShield Pen Needles or BD SafetyGlide 6mm insulin syringes) designed to help prevent inadvertent needlestick injuries during injections that can occur in clinical environments.

[0047] According to one aspect of the exemplary embodiment, the captured image of item 22 can be of a pen needle 22 with indicia 24, as shown in FIG. 3D , versus a syringe injection product, or an insulin pen injection product, such as a package of pen needles of a selected size with indicia 24 on the package 22, as shown in FIG. 3E . For example, a QR code 24 on a pen injector 22 can be utilized to visualize a treatment in which the syringe and pen needle are used together rather than separately. Programmed in accordance with the medical event image capture app 40, processor 26 can capture an image of item 22 and process the image to determine the type of product based on the code 24 and / or color and alphanumeric information on the product label, or other physical characteristics of the image-captured item 22. Processor 26 can perform a lookup operation in a local or remote table of compatible injection products and alert the user (e.g., via a GUI display on touchscreen 30) if the product and / or product family code 24 is incompatible with the type of injection pen or its insulin, as identified in the configuration profile of app 40. Additionally, processor 26 may control smartphone 20 to direct the user to a specific URL identified based on item code 24 (e.g., encoded in QR code 24) to display an informational video or website to better educate the user on optimal injection technique or usage of product 22.

[0048] With regard to the second use of the medical event image capture app 40 (i.e., dosage confirmation) and FIGS. 4A and 4B , according to an exemplary embodiment, the medical event image capture app 40 is configured to process captured images to determine the amount of medication administered. For people who inject themselves or others, dose measurement, dosage confirmation, and tracking can be challenges that can be alleviated by the functionality of the medical event image capture app 40. For example, the app 40 can also be used to ensure that patients are drawing the correct dose, particularly in the case of syringes, but also in the case of injection pens. The combination of the app 40's functionality to determine the correct device and the correct dosage, in addition to dosage confirmation, can be expected to result in improved compliance with treatment, a reduced likelihood of medication errors, and better patient outcomes.

[0049] 4A and 4B, the item 22 to be used for injection can be positioned within the field of view 42 of the camera 28. An image is captured that includes pixels indicative of an injection level or dosage indicator. The medical event image capture app 40 can provide 2D and / or 3D image processing algorithms configured to determine, for example, a plunger position 44 (FIG. 4B) corresponding to the aspirated amount (i.e., the amount to be injected via the syringe 22) or an injection pen dial indicator 44 (FIG. 4A) corresponding to the dialed amount of medication to be administered via the injection pen.

[0050] Apps exist that enable smartphones to wirelessly communicate with wirelessly enabled injection pens and wirelessly receive dialed and / or administered dose information. Therefore, these injection pens require a wireless communication interface to make the injection pen a wirelessly enabled device, adding complexity and therefore additional cost. In contrast, the medical event image capture app 40 with camera image processing in the exemplary embodiment enables automatic dose capture and confirmation of correct dose without requiring wireless communication between the device 22 and the smartphone 20, and therefore without adding complexity and cost to the pen-like injection device.

[0051] According to a third application of the exemplary embodiment (i.e., determining device or medication malfunction), the medical event image capture app 40 is configured to process captured images to determine whether the device 22 or medication is defective. For example, the app 40 can be used to identify the presence of air bubbles and notify the patient to ensure that the patient is aspirating the correct dose. Correctly aspirating a dose of medication into the syringe 22 is a critical step that involves visually detecting air bubbles in the syringe barrel and subsequently removing the bubbles from the syringe barrel. Often, poor visual acuity on the part of the user makes detecting air bubbles difficult. The app 40 advantageously incorporates image recognition capabilities and can both detect and quantify the volume of air bubbles in the syringe through processing of the captured images. For example, this processing can be performed using 3D image analysis or typical 2D projected surface area image analysis traditionally used for feature counting in various scientific fields. 2D image analysis algorithms can also be used to quantify bubble size. As described below in connection with Figure 7C, the app 40 can generate an alert to notify the user that an air bubble has been detected and needs to be removed from the syringe barrel prior to administration. The alert can be audible, which is particularly helpful for visually impaired or visually impaired users.

[0052] One example is shown in FIG. 5, in which a projected 2D image / photograph of a syringe 22 with an air bubble 46 is acquired via the image capture operation of the app 40 and analyzed by the app's image processing and analysis software to recognize the shape and size (and therefore volume) of the air bubble 46 using the projected surface area of ​​the 2D image. Such image processing can have multiple utilities, such as monitoring the effectiveness of patient usage, providing better education or training tools, and potentially tracking the accuracy of aspirated doses. The medical event image capture app 40 can also use capture image processing algorithms to detect other attributes of the item 22, such as determining whether particles 48 are present in the medication, whether the medication is opaque and not sufficiently transparent (e.g., meaning the medication is expired or contaminated), or whether the medical device 22 is missing a safety cap or has a bent or broken needle, among other undesirable attributes. In either case, the app 40 can generate an alert notifying the user that an undesirable attribute of the device 22 or medication has been detected, allowing the user an opportunity to resolve the issue before an incorrect dose is administered.

[0053] This third use of the medical event image capture app 40 is particularly beneficial in clinical settings. For example, the smartphone 20 equipped with the app 40 can detect visible contamination (e.g., a pen with expired medication that is cloudy or indicates floating particles). The app 40's capture image processing algorithms can be configured to detect, for example, opacity of the medication or whether the device is leaking. The capture image processing algorithms can also be configured to determine from captured images of the injection item 22 whether safety features, such as a sterile cap or other needlestick protection device, are missing and generate an alert to the HCP. These safety features may be required by hospital safety procedures, and the app 40 can ensure compliance as well as assist with inventory management and replenishment. For example, the app 40 can detect devices and other supplies used in an injection event whose images are captured and processed. The processor 26, programmed via the app 40, can advise individuals and clinical environment personnel regarding supply levels based on the amount of captured images of used supplies 22 to assist with automatic reordering of supplies. Indeed, many clinical environments provide HCPs with iPhone® or mobile devices for alerts and messages about when to arrive at the patient's bedside. App 40 can be provided to HCP devices 20 to enable them to capture images of medical condition management events and, through processing of the images, gather information to support the automatic recording of injection data into the patient's electronic record, as well as information to support billing, inventory control and reordering, and administrative recording in the clinical environment regarding care plan compliance and clinical effectiveness.

[0054] Exemplary image processing algorithms for processing captured images according to exemplary embodiments include, but are not limited to, any of the following image processing and / or image analysis algorithms: image segmentation (e.g., for recognition of the correct location of boundaries), image representation (e.g., for image clarity in the form of a pixel map), detection and recognition (e.g., for recognition of pre-quantified features), motion estimation (e.g., when using dynamic images), tracking (e.g., for tracking features identified during the detection step of exemplary embodiments), surface and shape estimation (e.g., for bubble detection and volume quantification according to exemplary embodiments), enhancement (e.g., for contrast stretching, noise filtering, histogram correction), restoration (e.g., for boundary editing, contrast adjustment, exposure correction), analysis (e.g., for identifying, classifying, and / or counting features), reconstruction, and data compression. Exemplary platforms that support these exemplary image processing and / or image analysis algorithms include, but are not limited to, commercially available and open source platforms such as Matlab, Image J, Icy, ENVI, FIJI, Image Tool, and Image-Pro Plus, among others.

[0055] 6 is a flowchart of an example operation of the medical event image capture app 40, according to an example embodiment. It is understood that the app 40 can provide all three uses (i.e., (1) correct or compatible device, (2) dosage confirmation, and (3) medical device 22 and / or medication anomalies), any two of these three uses, or only one of these uses. A user uses the camera 28 functionality of the smartphone 20 to take an image of the device 22 to be used to administer a medical management event, such as an injection (block 50). The captured image processing operation of the app 40 determines the indicia 24 or, optionally, other attributes of the device 22 (blocks 52 and 54). If the indicia 24 or a particular attribute is determined to be present from the captured image, the app 40 is configured to control the processor 26 to determine relevant equipment information (e.g., a table or other data structure stored in a local or remote computer memory device), such as whether the medical device 22 detected in the captured image belongs to the same product family or is otherwise compatible, indicates an amount to be administered, or indicates an anomaly (block 56). If the device 22 in the captured image is incompatible (e.g., part of a different product family as described above), the processor 26 will generate an alert or GUI screen to inform the user of the device 22 having an incompatible medication, a user error in inhaling other medication, or a possible incorrect dosage due to misuse of the medical device 22 (block 60). The indicia 24 detected by the image processing functionality of the app 40 can also determine whether a QR code or other indicia indicates that the user should receive playback of educational information, as described above (blocks 62 and 64).

[0056] Continuing with reference to FIG. 6, processor 26, controlled by medical event image capture app 40, determines whether the correct dose was aspirated and, if not, generates an alert or GUI screen to the user (blocks 66 and 68). For example, as described in connection with FIG. 5, if processor 26 detects a device or drug malfunction, processor 26 generates an alert or GUI screen to notify the user of the problem (blocks 70 and 72). Once the correct dose has been confirmed to have been administered per block 74 (e.g., via user input on a GUI screen generated by app 40 on touchscreen 30), medical event data or information (e.g., one or more of the confirmed dose, detected abnormality, product 22 code, among other data obtained via the captured image processing operations of app 40) may be stored locally or remotely for access and use by the patient and / or other medical condition management stakeholders (block 76). For example, medical event data or information can be automatically uploaded to a repository for inclusion in the patient's electronic record for medical billing, automatic refilling of medical supplies 22, and / or tracking care plan compliance, among other uses, such as incorporation into an integrated disease management system described below in connection with FIG. 10. The app 40 can optionally be used with an injection site rotation algorithm, as described in commonly owned U.S. Patent No. 10,173,015 (Block 78). For example, the injection site rotation algorithm can recommend the next body site injection location, and the injection app 40 can capture an image of the injection event and record the injection by verifying the amount administered. The injection site rotation algorithm can also record the body site injected.

[0057] 7A, 7B, 7C, 7D, 7E, 7F, 7G, and 7H are example GUI screens generated on device 20 by medical event image capture app 40 to guide the user to aspirate the correct dose (e.g., using a syringe) and take an image of the QR code or other indicator 24 on syringe 22, as depicted in FIG. 7A. The captured image is shown in screen 92 of FIG. 7B. In screen 94 of FIG. 7C, the user is alerted to the detection of the presence of an air bubble in the captured image, as determined using an image processing algorithm. After the air bubble is removed and the correct medication amount is aspirated (FIGS. 7D and 7E), app 40 generates screen 100 (FIG. 7F) requesting confirmation that the correct syringe is being used with the insulin type (e.g., as detected using indicator 24 on the vial). FIGS. 7G and 7H are administration confirmation screens.

[0058] FIGS. 8A, 8B, 8C, 8D, and 8E show example GUI screens generated by the app 40 on the device 20 to guide a user through aspirating a dose (e.g., with a syringe) without detecting any air bubbles, unlike the screens of FIGS. 7B-7D. In addition to using the app 40 to capture information from components such as injection devices and medication vials, the app 40 also serves to record information from other medical devices, such as monitors (e.g., blood glucose monitors (BGMs), pulse oximeters, thermometers, and blood pressure monitors, among other monitors that may not have wireless communication interfaces). FIGS. 9A, 9B, and 9C show example GUI screens generated by the app 40 on the device 20 for capturing information 24 from the screen of a BGM 22 using photo capture, as described herein. FIG. 9A shows a device (e.g., a smartphone 20) capturing information 24 from the display of a BGM 22 within the image field 42 of the smartphone camera 28. A camera image of information 24 (e.g., a glucose value of 206 mg / dL at 10:43 AM) is shown at 25 in FIG. 9B. Referring to FIG. 9C, app 40 records the information from the camera image in memory on smartphone 20 and displays it to the user on smartphone screen 30. Such passive information capture and recording using app 40 makes devices such as monitors without wireless communication interfaces more versatile and cost-effective for some users who, for example, do not have affordable access to continuous glucose monitoring systems that automatically and wirelessly record glucose readings on a separate device. App 40 provides an affordable solution to the known challenges users face of tracking and transcribing blood glucose levels by giving these users the option to electronically record data from the monitor screen, reducing the likelihood of human error.

[0059] The illustrative embodiments disclose multiple ways to better engage with users and enhance the user experience by leveraging strengths from both the injection product 22 and the medical event image capture app 40. The app 40 is used to identify both specific functions and activities, verify they are as intended, provide confirmation to the user, and enable recording and tracking of information for posterity. Overall, the integrated use of the device 22 and app 40 is expected to promote better compliance and improved patient outcomes. Furthermore, the combination of detecting the correct device 22 and monitoring and recording administered dose information is expected to provide more accurate data, enabling better clinical decision-making and reducing the likelihood of medication errors. While primarily targeted at a self-injecting patient base, the app 40 (and the combined features of the device 22) can be equally easily utilized in other environments (e.g., institutional and alternate sites) and by caregivers (e.g., nurses, family members, etc.).

[0060] In addition to these various insulin delivery or injection products, diabetes care companies can offer digital health (DH) apps that enable patients to maintain improved control of their diabetes treatment regimens, such as the BD Diabetes Care app available from Becton Dickinson and Company. For example, the BD Diabetes Care app assists patients and / or their caregivers with recording injections, recording or monitoring blood glucose levels, recording carbohydrate intake, and recording exercise, all of which affect a patient's need for injected insulin.

[0061] The medical event image capture app 40 can also be integrated into digital health apps (e.g., the BD Diabetes Care app). For example, the medical event image capture app 40 and its generated information can be automatically combined with content from other digital health apps, such as injections, exercise, carbohydrate intake, and blood glucose levels, to assist patients and disease management stakeholders with patient compliance with prescribed disease management regimes (e.g., how well a patient is maintaining target blood glucose levels), reordering supplies (e.g., home medical supplies such as self-injection devices and medications, and pharmacy inventory), automatic shipping of prescribed medications and medical supplies to patients or commercial facilities, inventory tracking, and billing for medical events captured in clinical settings. Thus, the illustrated embodiments herein provide convenience and other benefits to various categories of users (e.g., self-injectors and caregiver-injectors) in various categories of settings (e.g., home environments or other alternative sites such as nursing homes, long-term care and rehabilitation facilities, and clinical / hospital settings).

[0062] The medical event image capture app 40 can be a standalone app that communicates with the user (e.g., patient) or other stakeholders on the user's medical condition management team, such as caregivers (e.g., parents, spouse, school nurse), healthcare providers, clinical environment managers, pharmacies, payers (e.g., insurance companies), and medical device suppliers and distributors.

[0063] The medical event image capture app 40 can also be integrated into an integrated disease management (IDM) system 150, as shown in FIG. 10 according to an exemplary embodiment. It is understood that the IDM system 150 is useful for managing other types of diseases, including the collection, analysis, and dissemination of information to assist disease stakeholders (e.g., patients, caregivers, healthcare providers, disease management companies, pharmacies, suppliers of disease management-related products, insurance companies, and other payers). The IDM system 150 can be used by many types of people, including, but not limited to, people with diabetes, people without diabetes, caregivers, and healthcare professionals or organizations, such as disease management companies, pharmacies, suppliers of disease management-related products, insurance companies, and other payers. For ease of explanation, this disclosure will describe the IDM system with reference to a user. Reference to a “user” is intended to encompass all types of users without limitation. Furthermore, in some examples, this disclosure will refer to a patient or a diabetic. This is done in the context of a non-limiting example and is not intended to be limiting. Thus, reference to a patient or a diabetic is intended to refer to all types of users without limitation. The IDM system 150 can include an interactive interface that is simple, attractive, and provides a scalable means for users to seek information and support when needed, allowing them to feel more in control of their condition.

[0064] IDM system 150 may also include or have access to a user database and a content database (not shown). For example, a healthcare professional or related organization may develop recommended disease management protocols and recommended lifestyle choices to optimize patient outcomes and store this diabetes information content in the content database. IDM system 150 may be configured to securely (e.g., encrypted) transmit data to a remote server, such as a cloud storage server, perform analysis of the received data (e.g., disease management data), provide feedback to the user (e.g., customized feedback with curated content based on the user's data and interface interactions), and transmit all or a portion of the data and / or curated content to another user device or a remote health management access point (e.g., as cloud storage) from which healthcare stakeholders, such as the patient's physician or other HCP, family or other caregivers, pharmacists, disease management companies, medical supply suppliers, or payers, can access the information. Conversely, alerts, reminders, and interventions may be securely provided to the user by the user's network, e.g., HCPs, via IDM system 150.

[0065] User access to the IDM system 150 is via a user device 20 equipped with an interactive interface that can be accessed via a web browser or software application (e.g., a smartphone app or computer application). The user device 20 can be, but is not limited to, a smartphone, a smartwatch, a tablet, a laptop, a computer, a personal digital assistant ("PDA"), or the like. In some examples, the user device 20 is a portable device, such as, but not limited to, a smartphone, a tablet computer, or any communication device with computing capabilities, a mobile device connectivity module, and any portable device known in the art, including an applicable user interface, preferably, but not limited to, a touchscreen. Users typically use such mobile devices for various functions, such as making and receiving phone calls, sending and receiving text messages, and / or browsing the Internet. The user device 20 communicates with the IDM system via wireless and / or wired networks.

[0066] 10 , the IDM system operates in conjunction with an IDM personal app 140 installed on a user device 20 operated by a patient and an IDM professional app downloaded or otherwise installed on user devices 20 operated by professionals, such as clinicians 124, pharmacists 126, payers 130, and pharmaceutical companies 132. The IDM app (e.g., IDM personal app 140) can operate in a cloud-dependent configuration, where a mobile device equipped with the app transfers data to and receives data from the cloud (e.g., the IDM system), for example, during an app session or periodically or continuously in the background, or in a distributed configuration, where the app functions in a standalone mode and then selectively connects to the cloud (e.g., the IDM system) as needed.

[0067] For example, the IDM Personal app 140 can be displayed as a single icon on the patient's device 20. The IDM Personal app 140 provides the patient or patient's caregiver with an interface to the IDM system for functions and experiences such as viewing dosage data, texting with clinicians, adding dietary data to the patient's stored data, and importing BG data. The IDM Personal app 140 incorporates the operations of the Medical Event Image Capture app 40 and can retrieve and store information from captured images such as dosage amounts, date / time stamps of events associated with the captured images, and products 22 identified via the captured images, among other data. The IDM Professional app provides an interface to the IDM system for other users, such as clinicians 124, pharmacists 126, payers 130, pharmaceutical companies 132, or other healthcare companies, for functions such as viewing patient or patient population data, sending texts to patients, and performing dosage titrations, among other functions. The IDM Personal (Patient, Caregiver) software 140 can include, for example, one or more apps. IDM Professional (clinician, pharmacist, etc.) software can be web-based to suit various user types, providing personalized experiences for patient care team providers, payers, and pharmacists. For example, the IDM Professional app can be programmed to pull data from the patient's IDM Personal app 140.

[0068] Continuing with reference to FIG. 10 , the user device 20 can connect (e.g., via Bluetooth®) with other devices such as one or more medication delivery devices (MDDs) 22 (e.g., insulin syringes 22 and / or pumps, indicated at 120) and other devices such as glucose or lifestyle monitoring devices, generally indicated at 122. For example, the other devices may include, but are not limited to, one or more carbohydrate input devices or apps running on the user's mobile phone that allow the patient to input their food and drink intake, devices or apps with oral medication input elements that allow the user to track oral medications taken, background music and / or CGM, and devices or apps for inputting health data such as user activity levels. Once downloaded, the IDM personal app 140 allows the user to selectively activate additional features associated with each smart device, such as an MDD (e.g., an injection pen 22 app, a pump 120 app, or other dose capture apps). The MDD app can provide device connectivity and data offloading, dosage data storage and access, dosage data transfer to the cloud, user profile creation and authentication, connected third-party experiences (e.g., interactions between users and third parties, such as BG data tracking vendors), and output and analysis of dosage and blood glucose monitoring (BGM) data. Referring to Figure 10, some device data can be sent to the user device 20 using the IDM personal app 140 for storage on a private cloud, while other data (e.g., non-proprietary or unregulated medical device data from the device) can be sent by the device or its vendor 138 to the public cloud 136 for access by the user device 20.

[0069] Similarly, an IDM Professional app can be selectively configured with various functionality by various stakeholders, including, for example, a patient population management sub-app, a patient outcomes sub-app, data and communication protocols, application programming interfaces (APIs) that enable data communication between users and systems, etc. Some examples are a proprietary cloud or "closed API" that allows users to create an account and have direct access to data and functionality through the app view, a commercial cloud or "open API" where data is passed to another entity (e.g., Glooko) to facilitate use by the end user (e.g., via an open API), or a hybrid model that simultaneously offers both the open and closed API options described above, utilizing data from devices with open APIs in addition to proprietary data generated from devices with closed APIs.

[0070] According to one aspect of the embodiment illustrated in FIG. 10 , one or more devices 22, 120, and 122 are connected to devices capable of communicating data (e.g., administered amounts of medication and blood glucose measurements) directly to the IDM system. Examples of connected medication delivery devices (MDDs) are described in commonly owned US20160074587, which is incorporated herein by reference. A platform of connected devices communicating with the IDM system is described below. The IDM system and connected devices (e.g., MDDs 12, 120 and other devices 122) advantageously provide an end-to-end IDM solution for people with diabetes (PWDs) and their care networks (e.g., healthcare providers, caregivers, pharmacists, and insurance companies), easing the burden of diabetes management for PWDs as well as other disease management stakeholders. The IDM solution transforms data to enhance end-user experience and improve outcomes. The IDM solution can be implemented as a broad family of products addressing needs related to a specific medical condition, such as diabetes, although the IDM solution can also be configured to manage different medical conditions. A product can be hardware and / or software that provides value to a defined group of people, such as patients or caregivers, or professional disease management personnel, such as healthcare providers, pharmacists, and insurance companies. A software product (e.g., a phone app or computer application) described herein can include one or more modules, where a module is understood to be a group of functions that provides a sequence of experiences, such as discrete events, tasks, and actions.

[0071] Those skilled in the art will understand that the present disclosure is not limited in its application to the details of construction and the arrangement of components set forth below or illustrated in the drawings. The embodiments herein are capable of other embodiments and of being practiced or carried out in various ways. It is also understood that the phraseology and terminology used herein are for descriptive purposes and should not be construed as limiting. The use of "including," "comprising," or "having," and variations thereof, herein means the inclusion of additional items in addition to the items listed thereafter and their equivalents. Unless otherwise limited, the terms "connected," "coupled," and "mounted," and variations thereof, herein are used broadly to encompass both direct and indirect connections, couplings, and attachments. Furthermore, the terms "connected" and "coupled," and variations thereof, are not limited to physical or mechanical connections or couplings. Furthermore, terms such as top, bottom, and upper surface are relative and are used to aid in illustration, but are not limiting.

[0072] Components of the illustrated devices, systems, and methods employed in accordance with the illustrated embodiments may be implemented at least in part in digital electronic circuitry, analog electronic circuitry, or computer hardware, firmware, software, or combinations thereof. These components may be implemented as a computer program product, such as a computer program, program code, or computer instructions tangibly embodied in an information medium or machine-readable storage device for execution by, or to control the operation of, a data processing apparatus such as a programmable processor, a computer, or multiple computers.

[0073] The computer program may be written in any type of programming language, including compiled or interpreted languages, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. The computer program may be deployed to be executed on one computer or multiple computers at one site, or distributed across multiple sites and interconnected by a communication network. Furthermore, functional programs, codes, and code segments for achieving the exemplary embodiments can be readily interpreted by programmers skilled in the art to which the illustrated embodiments pertain to be within the scope of the claims illustrated by the exemplary embodiments. Method steps associated with the exemplary embodiments of the present invention may be performed by one or more programmable processors that execute computer programs, codes, or instructions to perform functions (e.g., by manipulating input data and / or generating output). For example, the method steps may also be performed by special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit), and the apparatus of the exemplary embodiments may be implemented in this manner.

[0074] The various illustrative logic blocks, modules, and circuits described in connection with the embodiments disclosed herein may be implemented or performed by a general-purpose processor, a digital signal processor (DSP), an ASIC, an FPGA, or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in combination with a DSP core, or any other such configuration.

[0075] Processors suitable for executing a computer program include, by way of example, both general-purpose and special-purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, a processor receives instructions and data from a read-only memory and / or a random-access memory. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer also includes, or is operatively coupled to transfer data from, one or more mass storage devices for storing data, such as magnetic, magneto-optical, or optical disks. Information media suitable for embodying computer program instructions and data include, by way of example, semiconductor memory devices such as electrically programmable read-only memory (EPROM), electrically erasable programmable ROM (EEPROM), flash memory devices, and all forms of non-volatile memory, including data storage disks (such as magnetic disks, internal or removable disks, magneto-optical disks, CD-ROM, and DVD-ROM disks). The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0076] Those skilled in the art will understand that information and signals may be represented using any of a variety of different technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0077] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the claims as exemplified by the illustrated embodiments. A software module may reside in random access memory (RAM), flash memory, ROM, EPROM, EEPROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. Alternatively, the storage medium may be integral to the processor. In other words, the processor and the storage medium may reside within an integrated circuit or may be implemented as discrete components.

[0078] A computer-readable non-transitory medium includes any type of computer-readable medium, including magnetic storage media, optical storage media, flash media, and solid-state storage media. It should be understood that the software can be sold for installation on a central processing unit (CPU) device. Alternatively, the software can be obtained and loaded onto a CPU device, including obtaining the software through physical media or a distribution system, such as from a server owned by the software creator or from a server used but not owned by the software creator. The software can be stored on a server for distribution, for example, via the Internet.

[0079] The above description and illustrations are intended to be illustrative only and are not intended to limit the invention in any way, except as set forth in the following claims. It is particularly noted that those skilled in the art can readily combine the various technical aspects of the various elements of the various exemplary embodiments described above in many other ways, all of which are considered to be within the scope of the present invention.

Claims

1. 1. A portable device for capturing images of a medical event to reduce medical errors, comprising: an imaging device for imaging at least one medical product used during a medical event; a memory for storing images captured by the imaging device and program instructions for processing the captured images; a user interface configured to generate an output to a user; a processor adapted to execute said program instructions; When the program instructions are executed by the processor, the processor analyzing the captured image associated with the medical event to detect a characteristic of the medical product selected from the group consisting of indicia on the medical product and designated attributes of the medical product; analyzing the detected characteristics to determine whether a medical error has occurred, the medical error corresponding to a mismatch of the medical product with the medical event, a user error, or an anomaly; a handheld device that generates an output to the user via the user interface including an alert related to the medical error; at least one captured image in the memory corresponds to a medical event involving at least two medical products used together; When the program instructions are executed by the processor, the processor: analyzing the at least one captured image to detect respective indicia on the at least two medical products; analyzing each indicator on the at least two medical products using previously stored medical product data locally or remotely accessible by the processor, the previously stored medical product data including indicators for each of a plurality of different medical products and, for each medical product of the plurality of different medical products, a product family code that is a corresponding indicator of one or more other medical products that have been shown to be compatible with the medical product; generating an output to a user if the processor determines that the at least two medical products are incompatible with each other according to a product family code of the previously stored medical product data; Portable devices.

2. 10. The portable device of claim 1, wherein the program instructions include an algorithm selected from two-dimensional and three-dimensional image processing algorithms used by the processor to analyze the captured images.

3. When the program instructions are executed by the processor, the processor: analyzing at least one captured image to detect characteristics of the medical product including at least one specified attribute of the medical product selected from the group consisting of a selected color of the medical product, a selected dimension of the medical product, a selected form factor of the medical product, the presence of a safety feature on the medical product, and the absence of a safety feature on the medical product compared to stored images of the medical product with the safety feature; analyzing the detected features to determine whether a medical error has occurred using previously stored medical product data locally or remotely accessible by the processor, the previously stored medical product data including specified specifications of image characteristics of the medical product corresponding to the at least one specified attribute; The portable device of claim 1 .

4. at least one of the medical products is a liquid medication drawn into a syringe, and the at least one specified attribute of the liquid medication is selected from the group consisting of opacity of the liquid medication, the presence of air bubbles in the liquid medication, and the presence of particles in the liquid medication; the previously stored medical product data includes a specified specification of image characteristics of the at least one specified attribute of the liquid medication; 4. The portable device of claim 3.

5. the portable device is a mobile phone or a computing device equipped with a wireless communication interface; the memory is configured to store integrated disease management (IDM) apps, the IDM apps including an IDM personal app operated by a patient user and / or an IDM professional app operated by a healthcare professional; The processor executes instructions according to an IDM app to operate the portable device in a cloud configuration with a remote IDM system, whereby the IDM app transmits data to and receives data from the IDM system during an app session. The portable device of claim 1 .

6. The portable device operates according to the IDM personal app to transfer and store information from the captured images to the IDM system, the information being selected from the group consisting of a dosage determined from at least one of the captured images, a date and / or timestamp of a medical event determined from at least one of the captured images, and a medical product identified from at least one of the captured images.

6. A portable device according to claim 5.

7. 10. The portable device of claim 6, wherein the portable device operates in accordance with the IDM Professional app to determine patient information from information stored in the IDM system, the patient information including at least one of compliance data with prescribed regimens based on dosage and the information related to the date and / or time of a medical event, medical product prescription update data based on the information related to the medical products identified from the captured images corresponding to the use of those medical products and the date and / or time of a medical event, and / or billing data corresponding to medical products identified from the captured images corresponding to the use of those medical products and the date and / or time of a medical event.

8. 8. The portable device of claim 7, wherein the portable device is capable of wirelessly connecting to at least one other medical condition management device to obtain medical event information therefrom, and the processor executes instructions in accordance with the IDM app to transfer the medical event information to the IDM system.

9. 6. The portable device of claim 5, wherein the cloud configuration includes a private cloud and a public cloud, and the portable device operates in accordance with the IDM app to determine whether at least one piece of information or other data related to the user stored in the memory is proprietary or non-proprietary data, and selectively transfers the proprietary data via the private cloud and the non-proprietary data via the public cloud.

10. 1. A portable device for capturing images of a medical event to reduce medical errors, comprising: an imaging device for imaging at least one medical product used during a medical event; a memory for storing images captured by the imaging device and program instructions for processing the captured images; a user interface configured to generate an output to a user; a processor adapted to execute said program instructions; When the program instructions are executed by the processor, the processor analyzing the captured image associated with the medical event to detect a characteristic of the medical product selected from the group consisting of indicia on the medical product and designated attributes of the medical product; analyzing the detected characteristics to determine whether a medical error has occurred, the medical error corresponding to a mismatch of the medical product with the medical event, a user error, or an anomaly; a handheld device that generates an output to the user via the user interface including an alert related to the medical error; at least one of the medical products is a drug administration device having an indicia; When the program instructions are executed by the processor, the processor: analyzing a captured image of the medication administration device and detecting the indicator; analyzing the captured image of the drug on the drug administration device or the captured image of the drug on another drug administration device and detecting the amount of drug indicated for administration by the drug administration device; using previously stored medical product data locally or remotely accessible by the processor, the previously stored medical product data including a plurality of different drug administration devices and their respective indicia, and for each drug administration device of the plurality of different drug administration devices, drug administration device specifications including an amount of a specified drug that can be administered via the drug administration device, to determine the amount of the specified drug based on the specifications corresponding to the drug administration device associated with the indicia detected from the captured image; generating an alert via the user interface if the detected amount of medication indicated for administration is determined to be different from the specified amount of medication; Portable devices.

11. 11. The handheld device of claim 10, wherein the detected amount of medication indicated for administration corresponds to a dose entered into an injection pen and markings in the captured image relating to at least one of a level indicator of a syringe barrel adjacent a fluid level in the syringe.

12. 11. The portable device of claim 10, wherein when the program instructions are executed by the processor, the processor analyzes the captured image or other captured images using an algorithm selected from a two-dimensional image processing algorithm and a three-dimensional image processing algorithm and detects the amount of medication indicated for administration by the medication administration device.

13. at least one of the medical products is a drug administration device having an indicia; When the program instructions are executed by the processor, the processor:

11. The portable device of claim 10, wherein the device generates an alert via the user interface if the amount of medication indicated for prescribed administration is determined to differ from the specified amount of medication.

14. When the program instructions are executed by the processor, the processor: analyzing the captured image of the drug from the drug administration device or other captured images of the drug from the drug administration device and detecting an amount of the drug indicated for administration by the drug administration device, the detected amount of the drug indicated for administration corresponding to a dose entered into an injection pen and / or a marking on the captured image associated with a level indicator on a syringe barrel adjacent a fluid level in the syringe; 14. The portable device of claim 13, wherein the device generates an alert via the user interface if the amount of medication indicated for the prescribed administration is determined to differ from the amount of medication indicated for the detected administration.

15. 11. The handheld device of claim 10, wherein the detected amount of medication indicated for administration corresponds to markings on the captured image associated with at least one of a dosage entered into an injection pen and a level indicator on the syringe barrel adjacent the fluid level in the syringe.

Images