Systems and methods for automated optical analyte measurement via wearable smart devices - Patents.com

Abstract

The system and method for measuring an analyte includes an instrument configured to perform an analyte testing operation. The instrument includes a wearable electronics and a remote instrument operatively connected to each other, each having a processor, which cooperate with each other in executing program instructions to measure the analyte. The wearable electronics includes a camera configured to generate a video stream, which is analyzed to identify a missing test component and an application of a bodily fluid on a test strip where the sample undergoes a change in one or more optical properties, the image of which is analyzed to determine a level of the analyte. The wearable electronics further includes a heads-up display (HUD) for providing output messages to a user regarding the performance and status of the analyte testing operation.

Description

[Technical field]

[0001] Technical Field The present disclosure relates generally to the field of analyte measurement systems, more particularly to optical measurement systems that assist a user in performing an analyte measurement process, and even more particularly to the use of wearable smart devices to perform automated optical analyte measurements. [Background technology]

[0002] background Analyte measurement systems known in the art allow for the analysis of a bodily fluid dose provided by a user to identify the level of one or more analytes in the user's body using electronic devices and one or more electrochemical reactions. These analyte measurement systems provide significant advantages in the accurate measurement of analytes in an individual user's fluid sample (i.e., biological or environmental). Some analyte measurement systems use test strips that carry a chemical reagent. Upon receiving a fluid dose containing an analyte, a chemical reaction between the reagent and the analyte changes the color of the reagent, and the color change varies based on the concentration of the analyte, providing a measurement of the analyte. While many analytes are measured in this manner, one specific example of an analyte measured in a fluid dose is glucose measured in a bodily fluid dose as part of the monitoring and treatment of diabetes mellitus.

[0003] Older test strip systems that measure analyte levels in a sample based on detectable color changes have relied on a human observer to determine the analyte measurement by observing the color change of the reagent, often with the aid of a printed color matching guide. Such manual systems can present problems with reduced accuracy and inconsistent measurements based on the perception of different human observers. More recently, automated analyte measurement devices have been developed that use a camera to observe the color change of the reagent to help improve the accuracy of the analyte measurement. For example, widely available smartphones include optical sensors, such as a camera, and digital image processing hardware that enable the smartphone to generate measurements of the analyte in the test strip when the smartphone runs a specially configured analyte measurement software application.

[0004] Examples of systems and configurations are known, for example, from PCT / EP 2019 / 080154 (published as WO 2020 / 094594), which describes methods, computer programs and devices that enable a mobile device to perform analytical measurements with increased measurement accuracy and improved reliability while ensuring relatively convenient handling for the user. Such configurations include a method in which an optical test strip is provided with a test field to which no sample has been applied (i.e., a blank test field), using a camera of the mobile device (e.g., a smartphone) to capture at least one image of part or all of the blank test field, applying a sample of bodily fluid to the test field (i.e., an administered test field), waiting a predetermined time until a color change reaction occurs, using the camera of the mobile device to capture a second image of part or all of the administered test field (ensuring that the same image acquisition settings are in place as when the image of the blank test field was captured), and using the captured images to determine an analytical measurement result value.

[0005] To further increase the accuracy and reliability of the analyte measurements performed in this way, further developments are underway, including the use of standard color standards, and the camera of the mobile device may be further calibrated based on captured images of such standard standards. For example, European Patent Application Publication No. 20173917.4 (unpublished) discloses a color reference card as part of a kit for performing analyte measurements. The disclosed method and arrangement in this case relies on the color standard to normalize the camera system of the mobile device according to the perceived difference between the measured reference color value and the known (predetermined) color value of the color standard.

[0006] In other configurations, with or without a standard color standard, the method and device may be configured to address other technical challenges of analytical measurements using mobile devices, increase measurement accuracy and improve reliability while allowing convenient handling for users. For example, a method is known for assessing the suitability of a mobile device for performing analytical measurements based on a color reaction, the mobile device having at least one camera, such method comprising a step of assessing the suitability of the mobile device for performing analytical measurements based on a color reaction. A further method comprises calibrating the mobile device for performing analytical measurements. Furthermore, a computer program is provided, in which the suitability assessment, calibration and measurement steps are performed by the mobile device. Such is known, for example, from PCT / EP 2019 / 079332 (published as WO 2020 / 094436).

[0007] Such system embodiments rely heavily on programming the mobile device to perform steps related to measurement, calibration, conformity assessment, and color referencing, among others. This is typically accomplished using a software application or app installed on the mobile device, such as a smartphone or tablet device. App functionality typically includes displaying analyte measurement results on the device's display. The app may further enable access to web or cloud-based data management tools, health care records, etc., to enhance disease management functions. Optionally, the app may include programming resident in the mobile device's memory that causes the execution of steps that lead to accurate and reliable measurement results. Alternatively, the app programming may direct the mobile device to interface with a memory location remote from the mobile device, such as a remote computer server, to download programming instructions for performing the measurement steps. In the latter case, the mobile device must be reliably connected to the Internet.

[0008] While the use of specially configured optical measurement devices improves analyte measurement, challenges remain in ensuring ease of use and accuracy of the measurement process. One such challenge arises with respect to the proper angle, positioning, and / or distance of the smartphone optical sensor relative to the dispensed test strip when an image of the dispensed test strip should be captured to ensure robust digital image processing techniques. A smartphone camera that is held too close or too far away from the dispensed strip during image capture may not provide adequate image resolution. Similarly, pointing the smartphone camera at too great an angle toward the dispensed test strip can distort the image, resulting in unbalanced or inconsistent color saturation, specular reflections, depth of field issues (which can occur with or without the use of a color reference card if all areas of the card are not in focus simultaneously), and spatial resolution issues such as compression of one axis due to an improper angle of the camera relative to the measurement area of ​​the dispensed test strip. Any of these situations can lead to inaccurate analyte measurement results. Another such challenge arises in connection with positioning the dispensed test strip on a surface for the user to manage holding the smartphone in the proper position for image capture. When a test strip dosed with a bodily fluid such as blood is placed on a flat surface, there is a risk of contamination of the surface. Another such challenge is the discontinuity of the analyte measurement process. This process requires continuous confirmation from the user on their smartphone to transmit the current status to the software. Furthermore, the equipment used for the process is remembered at each status check, which interrupts the natural flow of the measurement process. Therefore, improvements in optical analyte measurement systems that overcome these challenges would be beneficial. Summary of the Invention

[0009] overview Embodiments of the disclosure relate to systems and methods for measuring analytes, where a user wears a wearable electronics device and a remote device, the wearable electronics device comprising a camera configured to generate a video stream, a first communication transceiver configured to send and receive communications to and from a second communication transceiver of the remote device, a heads-up display (HUD), a first memory configured to store program instructions, and a first processor operably connected to the first communication transceiver, the camera, and the first memory, the first processor configured to execute the program instructions. The remote device comprises a second memory configured to store program instructions, and a second processor operably connected to the second transceiver and the second memory, the second transceiver configured to send and receive communications to and from the first communication transceiver. The wearable electronics and the remote device are operatively connected to each other, and the first processor and the second processor cooperate to execute program instructions to identify a test component in the video stream, provide guidance to a user via the HUD of any required test components not detected in the video stream, identify application of a bodily fluid sample to an application site on the test strip based on the video stream, start a timer in response to confirmation of application of the bodily fluid sample, generate at least one image of the test strip including at least a portion of the measurement site via the camera after a minimum period from starting the timer and before a maximum period, the at least one image being captured after guidance to the user via the HUD to position the test strip at a predetermined distance and angular orientation relative to the camera, analyze optical properties of at least a portion of the measurement site in the image to determine a level of an analyte, and display the level of the analyte on the HUD.

[0010] In at least one embodiment, the program instructions proceed to perform an analyte testing operation based on one or more inputs from a user upon completion of at least one of the steps of applying a bodily fluid sample to a deposition site on the test strip and generating at least one image.

[0011] In at least one embodiment, the program instructions proceed based on automatic tracking of user activity and automatic identification of components required for the analyte testing operation, and at least one of the first processor and the second processor is configured to perform the automatic tracking and automatic identification via a video stream generated by the camera.

[0012] In at least one embodiment, at least one of the first processor and the second processor is further configured to identify a vial in the video stream generated by the camera based on at least one registration mark located on the vial depicted in the video stream, identify an opening of the vial in the video stream based on at least one registration mark located on the lid of the vial, and identify an extraction of a test strip from the vial after identifying the opening of the vial in the video stream based on at least one registration mark associated with the test strip depicted in the video stream.

[0013] In at least one embodiment, the registration mark associated with the vial or lid or test strip comprises an indicator formed on a surface, such as the label of the vial, the inner surface of the lid, or the surface of the test strip or a color card that holds the test strip.

[0014] In at least one embodiment, at least one of the first processor and the second processor is further configured to identify in the video stream that the back side of the test strip is exposed based on the absence of an indicator formed on the surface of the test strip, and to generate an output message on the HUD indicating that the test strip should be rotated to expose the surface of the test strip carrying the indicator.

[0015] In at least one embodiment, at least one of the first processor and the second processor is further configured to identify errors in the extraction and processing of the test strip depicted in the video stream and provide guidance to the user via the HUD based on the identified errors.

[0016] In at least one embodiment, the step of identifying the application of the bodily fluid sample occurs after guidance to the user via the HUD for correct application of the bodily fluid to the test strip.

[0017] In at least one embodiment, at least one of the first processor and the second processor is further configured to identify a user's finger in the video stream, generate guidance on the HUD for generating a bodily fluid sample following identification of the finger in the video stream, identify contact of the finger with the application site in the video stream, and identify application of the fluid sample in response to a change in optical properties of the application site in the video stream after contact of the finger with the application site.

[0018] In at least one embodiment, at least one of the first processor and the second processor is further configured to identify application of the fluid sample in response to a change in an optical property of the deposition site in the video stream.

[0019] In at least one embodiment, at least one of the first processor and the second processor is further configured to generate an output message or image on the HUD informing a user that the measurement of the analyte in the fluid sample cannot be completed in response to an optical measurement of the measurement site not being generated after a predetermined minimum period of time has elapsed and before a predetermined maximum period of time has elapsed.

[0020] In at least one embodiment, the remote device further comprises a display operatively connected to the second processor, and the operation further comprises displaying the level of the analyte on the display. The remote device in at least one embodiment comprises a mobile electronic device. Further, the mobile electronic device in at least one embodiment comprises a smartphone, and the program instructions include a software application stored in a second memory and executed by the second processor in communication with the first processor to instruct the wearable electronic device in performing the analyte testing operation. [Brief description of the drawings]

[0021] Further advantages, benefits, features and objects will become more readily apparent in light of the following detailed description and with reference to the following drawings.

[0022] [Figure 1] FIG. 1 illustrates components of an analyte measurement system that uses wearable and mobile electronics to identify various aspects of a measurement kit and the performance of various steps during the analyte measurement process. [Diagram 2] FIG. 2 is a schematic diagram illustrating components of the wearable and mobile electronic devices of FIG. 1. [Diagram 3] FIG. 2 is a block diagram of a process for operation of the analyte measurement system. [Figure 4] 1A-1C are a series of views of a vial holding a test strip as the vial is opened and the test strip is removed from the vial. [Diagram 5] 1 is a series of diagrams of a test strip receiving a fluid dose. [Figure 6] 1 is a series of diagrams of test strips placed on a color card to receive a fluid dose. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Description of the Preferred Embodiments of the Invention In the description, reference is made to the accompanying drawings which form a part hereof, and in which is shown by way of example, and not by way of limitation, embodiments of the inventive concepts. Corresponding reference numerals indicate corresponding parts throughout the several views of the drawings.

[0024] While the inventive concept is susceptible to various modifications and alternative forms, exemplary embodiments thereof are shown by way of example in the drawings and described in detail herein. However, it should be understood that the following description of exemplary embodiments is not intended to limit the inventive concept to the particular form disclosed, but rather, the intention is to cover all advantages, effects, and features that fall within the spirit and scope defined by the embodiments described herein and the following embodiments. Therefore, reference should be made to the embodiments described herein and the following embodiments to interpret the scope of the inventive concept. Therefore, it should be noted that the embodiments described herein may have advantages, effects, and features that are useful in solving other problems.

[0025] The devices, systems, and methods will now be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the inventive concepts are shown. Indeed, the devices, systems, and methods may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, but rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements.

[0026] Similarly, many modifications and other embodiments of the devices, systems, and methods described herein will come to mind to one skilled in the art to which this disclosure pertains having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. It is therefore to be understood that the devices, systems, and methods are not limited to the particular embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the embodiments. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this disclosure pertains. Although any methods and materials similar or equivalent to those described herein can be used in the practice or testing of the methods, the preferred methods and materials are described herein.

[0028] Moreover, the reference to an element by the indefinite article "a" or "an" does not exclude the possibility that more than one element is present, unless the context clearly requires that only one element is present. Thus, the indefinite article "a" or "an" usually means "at least one". Similarly, the terms "have", "comprise", or "include", or any grammatical variants thereof, are used in a non-exclusive manner. Thus, these terms can refer both to the situation in which, apart from the features introduced by these terms, no further features are present in the entity described in this context, and to the situation in which one or more further features are present. For example, the expressions "A has B", "A comprises B", and "A includes B" can both refer to the situation in which, apart from B, no other elements are present in A (i.e., A consists solely and exclusively of B), or to the situation in which, apart from B, one or more further elements are present in A, such as element C, elements C and D, or further elements.

[0029] As used herein, the term "mobile electronic device" refers to a portable computing device that provides a user with one or more of the following components: output devices, input devices, memory, and wireless communication devices controlled by one or more processors within the mobile electronic device. As used herein, the term "wearable electronic device" refers to a type of mobile electronic device that is further adapted to be worn by a human user in a manner similar to eyeglasses, clothing, watches, or jewelry. Examples of output devices include, but are not limited to, liquid crystal display (LCD) displays, organic or inorganic light emitting diode (LED) displays, and other forms of graphical display devices, audio speakers, and haptic feedback devices. Examples of input devices include, but are not limited to, buttons, keyboards, touch screens, acute vibration sensors, still and video cameras, and audio microphones. Examples of memory include, but are not limited to, both volatile data storage devices, such as random access memory (RAM), and non-volatile data storage devices, such as magnetic disks, optical disks, and solid-state storage devices, including EEPROM, NAND flash, or other forms of solid-state data storage devices. Examples of wireless communication devices include, but are not limited to, wireless transceivers operating with Near Field Communication (NFC) protocols, the Bluetooth® protocol family including Bluetooth® Low Energy (BLE), the IEEE 802.11 protocol family ("Wi-Fi"), and cellular data transmission standards ("4G", "5G", etc.). Examples of processors include one or more central processing units (CPUs), graphic processing units (GPUs), neural network processors (NPUs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and any other suitable digital logic devices within an integrated device or as a combination of devices operating together to implement a processor. Common examples of mobile electronic devices include, but are not limited to, smartphones, tablet computing devices, and notebook computers.Common examples of wearable electronic devices include, but are not limited to, smart watches and smart glasses.

[0030] FIG. 1 illustrates an analyte measurement system 100 including a wearable electronic device 104 and a remote device 140. Throughout this disclosure, the embodiments described will typically refer to an operably connected arrangement of the wearable electronic device 104 and the remote device 140, which may typically take the form of a mobile electronic device such as a smartphone or tablet device. However, as will become apparent from this description, it should be noted that the remote device 140 may be effectively replaced by a remote Internet-based device (e.g., a remote server accessed via an Internet connection) such that the wearable device 104 directly interacts with such remote device in the same or substantially similar manner as the remote device 140, provided that the wearable device 104 has a suitable connection to the Internet via a suitable wireless communication device as described above. Any software applications described herein relating to the functionality of the remote device 140 may also be fully executed by a remote Internet-based device to which the wearable device 104 is operably connected.

[0031] With reference to the schematic diagrams of FIGS. 1 and 2, in one embodiment of the system 100, the wearable electronics 104 are embodied as a pair of glasses, also referred to as "smart glasses," although other forms of wearable electronics, including smart watches, may be used in alternative configurations. The wearable electronics 104 include a frame and optional lenses similar to traditional glasses. The wearable electronics 104 further include a camera 108, a position sensor 112, and a head-up display (HUD) 116, each operatively connected to an electronic control unit 120. The camera 108 is, for example, a CMOS or other suitable digital imaging device that generates images and video streams of an area in front of the wearable electronics 104 corresponding to the field of view of a person wearing the wearable electronics 104. In some embodiments, a single monochrome or color camera generates the video stream as a two-dimensional video stream. In other embodiments, the camera 108 is further configured to generate a video stream that provides three-dimensional object data. For example, in one configuration, the camera 108 further comprises two or more cameras providing stereoscopic video, or the camera 108 includes a depth sensor that provides three-dimensional depth information corresponding to objects in the video stream. The position sensor 112 includes, for example, a microelectromechanical (MEM) three-axis gyroscope and one or more accelerometers that provide data for identifying the spatial orientation of the wearable electronics 104 during operation. The HUD 116 provides visual output to the wearer without requiring the wearer to change their gaze to a particular display device. While FIG. 1 shows the HUD 116 separate from a glass lens in the wearable electronics 104, alternative configurations provide one or more visual display devices that are integrated into the lens or project graphical output onto the lens. Although not shown in further detail, the wearable electronics 104 also optionally includes audio input and output devices that may be located, for example, in an earpiece, so that sounds and audible communications may be communicated to the user and / or the user may provide voice commands for input to and interaction with the wearable electronics 104.In such an embodiment, the wearable electronics 104 includes a microphone operably connected to the processor 204, which can execute voice recognition programming to convert a user's audible commands, such as initiating an analyte measurement operation or indicating the completion of various actions associated with such an operation, into actionable input.

[0032] 2, the electronic control unit 120 houses at least one wearable electronics processor 204 operably connected to the camera 108, the position sensor 112, and the HUD 116. The electronic control unit 120 further houses a memory 208 and a communication transceiver 228 operably connected to the processor 204. In the embodiment of FIG. 2, the memory 208 stores firmware instructions 212 that control the operation of the wearable electronics 104. The communication transceiver 228 includes a transmitter that enables transmission of data including a video stream and one or more images to the remote device 140 or a corresponding communication transceiver 258 in a wireless connection device configured to bidirectionally communicate with the remote device (e.g., a local WiFi communication signal via a signal router operably connected to the Internet). The communication transceiver 228 further includes a receiver that enables the wearable electronics 104 to receive data originating from the remote device 140, and in particular, to receive messages from the remote device 140 for display to the user via the HUD 116. In the illustrative example of FIG. 2, communications transceiver 228 is a Bluetooth® or Bluetooth® Low Energy wireless data communications transceiver, although alternative configurations may use different wireless communication standards or may employ a wired connection interface such as Universal Serial Bus (USB).

[0033] 1 and 2, in one embodiment of the system 100, the remote device 140 further includes a processor 224 operably connected to a timer 226, a memory 232, a communications transceiver 258, and one or more display and user input / output (I / O) devices 146. The remote device 140 is operably connected to the wearable electronics 104. As will be appreciated, in other embodiments, the wearable processor 204 is comprised of similar components as the processor 224 of the remote device 140 in order for the wearable electronics 104 to perform the functions described herein related to performing analyte measurements.

[0034] The remote device 140 may include an optical sensor 142 operably connected to the processor 224 as part of its standard features as typically provided on a smart phone or tablet device. However, it is understood that an optical sensor on the remote device 140 is not required in accordance with embodiments of the present disclosure. Nevertheless, for illustrative purposes, in an alternative embodiment of the invention described in a related but separate patent application filed by the inventor concurrently herewith, the optical sensor 142 provided on the remote device 140 comprises a digital camera that generates still images or a video stream.

[0035] According to an embodiment of the present disclosure, the camera 108 is configured to generate images and video streams related to the performance of analytical measurements according to a sequence of steps that may be initiated directly by the wearable electronics 104 or by user operation of an app on a remote device 140 operatively connected to the wearable electronics 104. For example, once a sequence is initiated, the camera 108 may generate a video stream of a user acquiring a vial 160 of test strips 170, removing such test strips 170, and optionally placing a color card 180 to generate at least one optical measurement for analysis to measure an analyte level in a dose of bodily fluid that is subsequently applied to the test strip 170. In the configuration of FIG. 1, the camera 108 is configured to generate a video stream capturing the entire scene including the vial 160, the test strips 170 and attachment site 172, optionally the color card 180, and a finger 190 to identify the moment the test strip 170 receives a dose of fluid. The camera 108 is further configured to generate one or more images of the measurement site 178 on the back side of the test strip 170' at an appropriate time after the moment of application of the bodily fluid dose to provide input to the analyte measurement process due to the image having optically measurable properties that allow optical measurement. The optical measurement can be identified, for example, from a digital photograph that includes the measurement site 178 on the test strip 170'. Thus, the camera 108 provides a wider view of the elements used in the analyte testing process, as well as more detailed digital images or videos of the reagents on the test strip 170 and, optionally, calibration data provided on the color card 180. For example, the camera 108 may be configurable to generate a video stream with reduced resolution for identification and tracking of the vial 160, the test strip 170, the color card 180, and the finger 190 to enable the remote device 140 to identify the moment the test strip 170 receives the fluid dose. The camera 108 may then operate at a higher resolution to capture one or more high fidelity images of the test strip 170', either alone or in combination with the color card 180, to provide input for the analyte measurement process.

[0036] In one embodiment of the remote device 140, the user input / output (I / O) devices 146 include a touch screen display device that receives touch input to provide graphical output to the user and to control the operation of the remote device 140, more specifically to provide input to the analyte measurement process. Other examples of I / O devices include a microphone for audio input and a speaker for audio output, mechanical buttons, and the like. In some configurations, the wearable electronics 104 implements user I / O devices 146, such as an audio input device or a gesture tracking input device that uses the camera 108 to record input from the user that the camera 108 transmits to the remote device 140. The wearable electronics 104 may further receive output data from the remote device 140 for display to the user via the HUD 116.

[0037] In the remote device 140, the timer 226 allows the processor 224 to keep count of elapsed time during operation, including counting the elapsed time beginning from the moment the test strip 170 receives a dose of bodily fluid to ensure that an optical measurement of the reagent for analyte measurement occurs after a predetermined minimum time has elapsed and before a predetermined maximum time has elapsed. Although the timer 226 is shown as a separate component for illustrative purposes, in many practical embodiments the timer 226 will be integrated into the processor 204, 224 as a timer circuit or implemented as a software timer.

[0038] In FIG. 1 , the test strip 170 includes an attachment site 172 where a user provides a bodily fluid sample, such as liquid blood, which may be produced using a lancing device 192. The test strip 170 may also include a registration mark 174, shown as a printed mark in the form of an arrow on the surface of the test strip 170 at a predetermined location relative to the attachment site 172. A hole 176 is also formed through one end of the test strip 170. The registration mark 174 allows for efficient identification and tracking of the test strip 170, including identification of which side of the test strip 170 faces the camera in the video stream. Although in the illustrative example of FIG. 1 , only one side of the test strip 170 is configured to accept a blood sample and the registration mark 174 is formed only on this side of the test strip 170, in alternative embodiments the test strip may be configured to accept a volume of blood on both sides of the test strip. In FIG. 1, test strip 170' is the same test strip 170, but with the backside visible, including measurement site 178 and hole 176, but without arrow indicator 174, and in some embodiments, system 100 can detect that the backside of test strip 170' is exposed to camera 108. In test strip 170 / 170', attachment site 172 provides a fluid inlet that allows a fluid dose to permeate one or more internal layers in the test strip to allow chemical reaction with one or more reagents in the test strip. Examples of internal layers include, for example, different layers of filters and chemical reagents that react to one or more analytes in the fluid dose. Measurement site 178 is an optically exposed area formed on the backside 170' of the test strip, which changes color depending on the level of analyte in the fluid dose that permeates the test strip from attachment site 172. In one configuration, the reagent is directly exposed at the measurement site 178, while in another configuration, an optically transparent layer, such as a film, covers the reagent while providing an optical aperture to allow for the generation of an optical measurement of the reagent to detect a color change in the reagent due to exposure by the analyte in the fluid volume.

[0039] In the remote device 140, the memory 232 includes one or more non-volatile and volatile data storage devices. In the configuration of FIG. 2, the memory 232 stores application software 250 and operating system software 254, both of which include instructions for execution by the remote device processor 224. The application software 250 includes instructions implementing a user interface and an analyte analysis program to perform an analyte measurement process based on image analysis of one or more optical measurements of a reagent on the test strip 170. The application software 250 also stores predetermined minimum and maximum elapsed time thresholds to ensure that an optical measurement is generated after the fluid dose has had sufficient time to react with the reagent in the test strip 170, but before a maximum effective time period for measuring an analyte has elapsed. As described in more detail below, part of the analyte measurement process includes identifying the vial 160, removing the test strip 170, and identifying contact between the finger 190 and the test strip 170 to apply a fluid dose to the attachment site 172. The application software 250 further includes object recognition data 252 that enables the processor 224 to perform automatic object identification and tracking of the vial 160, test strip 170, color card 180, and lancing device 192 in the video stream received from the wearable electronics 104. The object recognition data 252 is generated by a training process that occurs prior to distribution of the application software 250. In particular, the training process utilizes the predetermined shapes, colors, and patterns of registration marks formed on the vial 160, test strip 170, and color card 180 to enable automatic identification and tracking of these components in the video stream. The object recognition data 252 may include image classifiers such as neural networks, particularly convolutional neural networks, support vector machines, hidden Markov models, one-dimensional and two-dimensional barcode scanning engines, and the like.Additionally, object recognition data 252 may include filters for color detection and edge detection, along with other image processing data necessary for tasks such as object detection and image segmentation to enable tracking of objects such as vials 160, test strips 170, and color cards 180 in the video stream. Operating system (OS) software 254 includes a software kernel, drivers, libraries, and other system software associated with a standard commercial operating system. The OS software 254 provides standardized services such as network and graphics stacks, a file system for data storage and management, software access to display and I / O devices 146, timers 226, communications transceiver 258, and other components within remote device 140.

[0040] In the remote device 140, the communications transceiver 258 includes a transmitter that enables transmission of data, including command data and output message data, to a corresponding transceiver 228 in the wearable electronics 104. The communications transceiver 258 further includes a receiver that enables the remote device 140 to receive data from the mobile electronics 140, and in particular to receive a video stream from the camera 108 in the wearable electronics 104. In the illustrative example of FIG. 2, the communications transceiver 258 is a Bluetooth® or Bluetooth® Low Energy wireless data communications transceiver, although alternative configurations may use different wireless communication standards or employ a wired connection interface, such as USB.

[0041] FIG. 1 further illustrates a vial 160 and a color card 180. The vial 160 houses one or more of the test strips 170. In addition to providing storage, the vial 160 protects the test strips from contamination in the environment, including preventing the reagents in the test strips 170 from absorbing excessive amounts of moisture from the surrounding environment. The vial 160 includes a printed label 162 that further carries one or more registration marks, shown as dashed indicators 163 printed along one or more edges of the label 162 in FIG. 1. The registration marks form a simple visual indicator that allows efficient identification and tracking of the vial 160 in the video stream that the wearable electronics 104 generates and transmits to the mobile electronics 140. The registration marks 163 are widely distributed on the exterior of the vial 160, allowing identification and tracking of the vial 160 from a wide range of viewing angles and when the user's hand is holding the vial 160. Alternative embodiments of the registration mark of the vial 160 include alternative printed pattern indicators including, for example, a bar code formed on the exterior of the vial 160, or engraved or embossed geometric shapes that aid in automatic identification and tracking of the vial 160. In the embodiment of FIG. 1, the lid 164 provides access to the interior of the vial 160. The lid 164 may be fully removable or may remain attached to the body of the vial 160 while open. In either configuration, a second registration mark 168 is formed on the interior surface of the vial lid 164. The second registration mark 168 is, for example, a circle or other geometric shape formed in a predetermined color that contrasts with the color of the vial 160 to clearly indicate that the vial 160 has been opened in a video stream generated by the wearable electronics 104 during use of the vial 160. In alternative embodiments, the registration mark 168 is a one- or two-dimensional bar code or other registration mark that is identifiable to automated vision algorithms. Registration marks 163 and 168 allow for accurate identification of vial 160 and for determining when vial 160 is closed and opened.

[0042] According to an embodiment of the present disclosure, during operation, the wearable electronics 104 generates a video stream that allows tracking of various activities and actions related to the analytical measurement process. For example, at the start of a measurement sequence (which may be performed using an open app or software program running on the wearable electronics 104 or the remote device 140), the wearable electronics may track test components for the analytical measurement process, including the vial 160, color card 180, and lancing device 192. The wearable electronics 104 may then utilize the (HUD) 116 to generate instructions to the user to test the components that are not being tracked. Additionally, the wearable electronics 104 may also utilize the HUD 116 to identify light conditions that may interfere with the analytical measurement and generate messages to the user regarding the unfavorable conditions.

[0043] In at least one embodiment, the wearable electronics 104 may track the vial 160 holding one or more test strips 170, arranged in relation to a color reference card, to identify when the test strip 170 is removed from the vial and receives a volume of fluid, such as a volume of blood from a human subject's finger 190. The wearable electronics 104 may then utilize the (HUD) 116 to provide instructions to the user for a step-by-step process of performing an analyte measurement. For example, the HUD 116 may instruct the user to prepare for the measurement and then scan an area within the field of view of the camera 108. As described in more detail below, the system 100 starts a timer upon detecting the application of a fluid dose to the deposition site 172 on the test strip 170, and after a predetermined minimum time has elapsed but before a predetermined maximum time has elapsed, instructs the user via the HUD 116 to capture an image using the camera 108 to generate an optical measurement of the deposition site 172 on the test strip 170, which is transmitted to the remote device 140.

[0044] During operation, the camera 108 captures images to detect a color change of a reagent visible at the measurement site 178 on the test strip 170' in response to one or more chemical reactions with an analyte in the fluid dose. In the illustrative example of FIG. 1, the color change of the reagent located at the measurement site 178 indicates the level of glucose analyte in the blood sample. As described above, the system 100 identifies when the deposition site 172 receives a fluid dose and uses a timer to determine when the optical sensor should generate a subsequent optical measurement of the measurement site 178 to ensure an accurate measurement of the blood glucose level. While the system 100 shows a test strip including separate deposition sites 172 and measurement sites 178 for illustrative purposes, one of ordinary skill in the art will recognize that alternative test strips provide a single deposition site and measurement site with the reagent co-located on the same region of the test strip. Thus, in some embodiments, the deposition site and the reagent occupy separate locations on the test strip, while in other embodiments, the deposition site and the reagent refer to a single location on the test strip.

[0045] In FIG. 1, color card 180 is an optional component with a back side shown in FIG. 1 that holds test strip 170 in place prior to administration. Color card 180 also has a front side (not shown) that includes a predetermined arrangement of color and other fiducial markings that help calibrate the image from camera 108 for accurate color measurement of exposed measurement site 178. An opening 186 in color card 180 allows measurement site 178 on the opposite side of test strip 170' to be measured by an optical sensor within the color pattern of color card 180. The back side of color card 180 includes registration marks 182 and 184, shown as arrow indicators printed on the back side of color card 180 in the exemplary embodiment of FIG. 1. Registration marks 182 and 184 are associated with test strip 170 and further assist in identifying and tracking test strip 170 in a video stream to detect when test strip 170 receives a fluid dose. The color card 180 is optional, and the system 100 is configured to generate optical measurements of a reagent at the measurement site 178 of the test strip 170' to measure a glucose analyte or another type of analyte in a blood sample using the test strip 170' alone or in conjunction with the color card 180.

[0046] FIG. 3 illustrates a process 300 for the operation of the system 100 to perform an automated analyte testing operation. The automatic detection of different aspects of the operation relieves the user from any requirement to interface, input, or operate the devices in the system 100, and further allows for more accurate analyte measurements. For example, by automatically detecting when a fluid dose is applied to the deposition site of the test strip, it is possible to initiate automatic timing of when the system 100 should generate one or more images for optical measurement of the measurement site to measure the level of the analyte in the fluid dose. In the description of the process 300, references to a process performing a function or action refer to the operation of one or more digital processors, such as the processors 204, 224 of the wearable electronic device 104 and the remote device 140, respectively, to cooperate to execute stored program instructions to perform the function or action in conjunction with other components in the system 100.

[0047] It will be apparent to one of ordinary skill in the art how independent processors may cooperate in the execution of stored program instructions, including how and where the respective processors may store the program instructions in the memory of each device in cooperation with each other. For example, the remote device processor 224 executes locally stored program instructions to trigger wireless communication of the program instructions to the wearable electronics via the transceivers 228, 258, and the wearable electronics memory 208 stores the received program instructions for immediate or delayed execution according to instructions received from the remote device 140. Similarly, the remote device processor 224 may deliver instructions to the wearable electronics processor 204 to directly trigger the execution of various steps of the analyte measurement operation by components of the wearable electronics 104. In other embodiments, the remote device 140 controls all aspects of the execution of the analyte measurement operation, and the processor 224 delivers the program instructions to the wearable electronics 104 for immediate execution by the processor 204. In yet other embodiments, memory 208 stores program instructions complementary or auxiliary to the execution of program instructions that may be received for immediate execution from processor 224, and the complementary or auxiliary program instructions are configured such that execution by processor 224 of program instructions received from processor 204 may be interrupted based on real-time input or sensory observations by components of the wearable electronics 104 that indicate interruption and independent action by the wearable electronics.

[0048] Cooperation between the processors 204, 224 in executing stored program instructions to perform the analyte measurement operation may further include data, images, video streams or other information being transmitted or communicated by the processor 204 via exchanges between the transceivers 228, 258 to the remote device processor 224, which executes locally stored program instructions relating to further processing of the information related to the analyte measurement operation. In this manner, the wearable electronics 104 operates in a so-called servant capacity relative to the remote device 140 operating in a so-called master capacity.

[0049] Process 300 begins after a user begins execution of application software 250 on system 100, and remote device 140 sends a command to wearable electronics 104 to activate camera 108 to generate a video stream of the scene in front of the user at the start of the analyte testing process (block 304). In the embodiment of FIG. 1, wearable electronics 104 transmits the video stream from camera 108 to remote device 140 using transceiver 228, so that remote device processor 224 can receive the video stream for further processing using corresponding transceiver 258. As is commonly known in the art, the video stream includes a series of frames of image data depicting a view from camera 108 over time during the analyte testing process.

[0050] The process 300 continues by the processor 224 automatically identifying the test components (vial 160, color card 180, and lancing device 192) in the video stream generated by the wearable electronics 104 and providing guidance to the user via the HUD 116 of the unidentified test components (block 306). The guidance may take the form of a message or image designating the missing test components. For example, the image may be an image of an outline of the missing component or a designated image representing the missing component. Optionally, block 306 may be repeated until all test components have been identified.

[0051] The process 300 continues when the processor 224 automatically identifies the vial 160 in the video stream generated by the wearable electronics 104 (block 308). Numerous digital image processing techniques may be used to identify objects such as the vial 160, or other objects detected in the video stream during the process 300, although non-limiting examples of preferred techniques are described in further detail herein. The vial 160 identification process further includes an object tracking operation that segments different portions of frames in the video stream that include the object, and an object identification operation that uses an image classifier to identify the tracked object.

[0052] In an object tracking operation, the processor 224 of the remote device 140 identifies and tracks one or more objects depicted in the video stream. To track the objects, the processor 224 performs a contour detection operation to identify boundaries of various objects in the video stream that have similar image intensity values, including the boundaries of the vial 160. In some configurations, the processor 224 performs image pre-processing operations such as converting the color video stream to grayscale, thresholding the grayscale pixels, and performing an edge detection process to improve the accuracy of the contour detection process. The processor 224 segments the original image, for example, using a rectangular bounding box that encloses the detected contour region, and the processor 224 performs the contour detection process over a series of video frames to track the movement of the object, such as when the user moves the vial 160. For example, as shown in view 404 of FIG. 4, the video stream depicts the vial 160, and the processor 224 generates a rectangular bounding box 406 segment in the frames of the video stream that includes the contour of the detected vial 160. Although view 404 shows vial 160 in isolation, some frames in the video stream contain more than one object, and the contour detection process described above allows for tracking of multiple objects in the video stream.

[0053] Once the tracking operation is complete, the processor 224 may have access to one or more image segments that contain the object, but the processor 224 has not yet determined the identity of the particular object, e.g., the processor 224 has tracked an object in the image segment 406, but has not yet identified the object as a vial 160 or some other object. The object tracking process generates multiple image segments that can improve the accuracy of an image classifier for detection of multiple related objects that may occupy different portions of a frame in a video stream. To complete the object identification process, the processor 224 provides the segmented portion of the image that contains the tracked object as an input to a trained image classifier that is stored in the memory 232 along with the object recognition data 252. The image classifier may be, for example, a trained convolutional neural network (CNN), or other suitable image classifier, trained to identify a predetermined set of objects, such as test strips 170, 171', color cards 180, or the interior of the vial 160 and vial lid 164 on either side of the finger 190. A training process for the image classifier occurs prior to process 300, in particular the image classifier is trained to explicitly or implicitly recognize registration mark features formed on the vial 160, the interior of the vial lid 164, both sides of the test strip 170, and on the color card 180 to improve accuracy in identifying a given object. Additionally, the training process may include training examples that occur when the registration mark is only partially visible to the camera 108, such as when a user is holding the vial 160 in their hand that may occlude a portion of the registration mark 163. The processor 224 optionally performs additional pre-processing of the image data, which may include resizing the image data to a predetermined resolution or performing a rotational transformation of the image based on metadata received from the position sensor 112 in the wearable electronics 104 that identifies the angular orientation of the camera 108 when each frame of the video stream is generated to improve accuracy of the image classifier.In some configurations, the image classifier is trained using black and white image data, while in other configurations, color images are preferred, including configurations where registration marks are formed using a predefined color that aids in image classification to identify the object. Unrelated objects that may be present in the video stream may also be rejected as unrelated by the image classifier. Furthermore, because the video stream includes a series of frames, the system 100 may be configured to recognize the vial 160 in one or more frames of the video stream even if the tracking and identification process is not successful in some of the video stream frames. One example of a software framework that enables the image processing operations described above in the application software 250 is the Open Computer Vision (OpenCV) project, available at https: / / opencv.org / . The above process for identifying the vial 160 is substantially the same as the process described below for identifying other objects in the video stream during process 300.

[0054] During the vial identification process, the remote device 140 optionally sends a graphical indicator, such as an icon, to the wearable electronics 104 to assist the user in identifying the next step in the process for performing the test analysis. For example, the remote device 140 sends a graphical icon corresponding to the shape of the vial 160 to the wearable electronics 104, and the wearable electronics processor 204 generates a graphical display of the icon using the HUD 116 to alert the user to remove the vial 160 and place it within the field of view of the camera 108 until the vial 160 is identified in the video stream. In FIG. 4, view 404 depicts an exemplary icon 408 that the HUD 116 overlays on the scene recorded by the camera 108 to prompt the user to remove the vial 160.

[0055] Referring again to Figure 3, the process 300 continues by the remote device processor 224 identifying that the vial 160 has been opened in response to detecting a registration mark 168 formed on the interior of the lid 164 in the vial 160 (block 312). Referring to Figure 4, diagram 412 illustrates the lid 164 removed from the vial 160 with the registration mark 168 visible on the interior of the lid 164. The remote device processor 224 tracks and identifies the lid 164 in the image segment 416 in the same manner as described above with respect to the vial 160. Additionally, the remote device 140 optionally transmits an icon of the lid to the wearable electronics 104, which displays an icon 420 on the HUD 116 to provide guidance to the user.

[0056] Referring again to FIG. 3, the process 300 continues by the remote device processor 224 identifying that the test strip 170 has been removed from the opened vial 160 by the user (block 316). The remote device processor 224 tracks and identifies the test strip 170 in the image segment 428 in the same manner as described above with respect to the vial 160 and the lid 164. Additionally, the remote device 140 optionally transmits an icon of the test strip to the wearable electronics 104, which displays an icon 432 on the HUD 116 to provide guidance to the user. In some cases, the HUD 116 may also display a text or graphic warning to the user regarding a potential error in the strip processing detected in the video stream. Examples of such errors may include leaving the vial open after strip extraction or removing more than one strip. As shown in FIG. 4, in some cases, the user removes a test strip with the back surface 170' visible to the camera 108. The remote device processor 224 tracks and identifies the back side 170' of the test strip within the region 430, and optionally generates an output message to the user via the HUD 116 or another output device 146 to rotate the test strip so that the side of the test strip 170 having the registration marks 174 and the attachment sites 172 is visible in the video stream. The absence of the registration marks 174 on the back side 170', and optionally other distinct features of the back side 170', provide sufficient difference for an image classifier to distinguish between the sides 170, 170' of the test strip. This operation is also performed at a subsequent stage of the process 300 when the test strip 170 is turned over to expose the back side 170' before the test strip 170 receives a fluid dose. Although FIG. 424 shows the vial 160, the inside of the lid 164, and the test strip 170 simultaneously for illustrative purposes, detecting the removal of the test strip 170 from the vial 160 only requires a sequence that detects the vial 160, the inside of the lid 164, and the test strip 170 in the same video stream within a relatively short period of time, such as a 10 second, 30 second, or 60 second time window.Thus, the vial 160 , the lid 164 , and the test strip 170 do not need to be identified simultaneously in the video stream for the process 300 to identify that the test strip 170 has been removed from the vial 160 .

[0057] During process 300, if the remote device processor 224 fails to identify a sequence of vials 160, the interior of the lid 164 indicating that the vial 160 has been opened, or the removal of a test strip 170 within a predetermined period of time (block 320), the process 300 returns to the processing described above with reference to block 308 to allow the user to repeat the process. Upon successfully identifying that a test strip 170 has been removed from a vial 160 (block 320), the process 300 continues by the remote device processor 224 continuing to track the identified test strip 170 in the video stream (block 328). Alternatively, the user may opt out of the automated process and begin manually performing the operational activities throughout process 300.

[0058] In one configuration, when the remote device processor 224 successfully identifies that the test strip 170 has been removed from the vial 160, it starts a second timer to count down a predetermined sample application time during which the user should apply a bodily fluid sample to the deposition site on the test strip. The predetermined sample application time is determined based on an optimized time after the test strip 170 is removed from the vial 160 and exposed to the environment outside the vial before the bodily fluid sample is applied. If the user waits too long to apply the sample, the test strip may be exposed to the environment and contaminate the test strip in a manner undesirable for accurate and valid analyte measurement. The countdown that begins with the start of the second timer continues until the remote device processor 224 confirms the application of the sample. If confirmation of the application of the sample to the test strip occurs before the countdown is completed, the second timer is canceled. If confirmation of the sample application does not occur and the countdown ends, one or both of an audible and visual alarms are triggered to inform the user that the test strip has been exposed for too long and should be discarded. In some embodiments, the countdown may be displayed on the HUD 116 and / or the display 146 of the remote device 140.

[0059] Similarly, another configuration of the remote device processor 224 tracks the vial 160 and lid 164 to monitor and track whether the user has closed the lid. Closing the lid 164 serves to protect the remaining test strips 170 in the vial 160 from exposure to the environment. If the remote device processor 224 does not detect that the lid 164 of the vial 160 has been closed after the test strip 170 has been removed from the vial, the remote device processor 224 may be configured to generate a text or graphic alert to the user via the HUD 116 and / or the display 146 of the remote device 140 to remind the user to close the lid.

[0060] In one configuration, the remote device processor 224 tracks the test strip 170 alone, as shown in diagram 504 of FIG. 5. In another configuration utilizing the color card 180, the remote device processor 224 identifies the color card 180 based at least in part on the registration mark arrows 182 and 184, and tracks the insertion of the test strip 170 into the color card 180 by the user, as shown in diagram 604 of FIG. 6. After identification of the test strip 170 and before the test strip 170 receives a fluid dose, the remote device processor 224 instructs the wearable electronics 104 to capture at least one image of the test strip including the attachment site 172 and store the at least one image in memory 232. The instructions may include messaging and / or graphics provided via the HUD 116 to enable the user to manipulate the color card 180 with the test strip 170 to properly position the color card 180 with the test strip 170 at an optimal distance and position relative to the angular orientation of the image of the blank attachment site 172 relative to the camera 108. As described in further detail below, at least one optical characteristic of the attachment site 172 changes after the attachment site 172 receives a fluid dose, and the change in optical characteristic enables detection of the time when the test strip 170 has received a fluid dose.

[0061] The process 300 continues by the remote device processor 224 instructing the user via the HUD 116 to apply a liquid blood sample to the attachment site 172 of the test strip 170. Such instructions displayed on the HUD 116 may indicate instructions for changing the lancet, preparing a lancing aid, and / or a tip for lancing (i.e., lancing site, depth setting, or optimal size of blood drop). The user may then take steps to procure a liquid blood sample, such as by lancing the user's skin and using an appropriate lancing device to procure a blood source. The user may then grasp the test strip 170 or the color card 180 with the test strip disposed therein with one hand and proceed with application of a fluid dose of blood from the punctured skin to the attachment site. The remote device processor 224 then, in one embodiment, identifies application of a fluid dose to the attachment site on the test strip based on the video stream and starts a timer 226 upon identification of the fluid dose application (block 332). In at least one embodiment, block 332 may also include the remote device processor 224 detecting any errors in the fluid dose application in the video stream (i.e., application of blood with pressure where the blood is not fully attached to the application site) and displaying such errors and guidance on the HUD 116 of the possibility of repeating the fluid dose application.

[0062] In one configuration, the remote device processor 224 identifies the user's finger 190 in the video stream using the same procedures described above for identifying the vial 160, lid 164, test strip 170, and color card 180. The remote device processor 224 further identifies contact of the finger with the deposition site 172 in the video stream. For example, as shown in diagram 508 of FIG. 5 and diagram 608 of FIG. 6, contact is identified in response to the finger 190 obstructing the test strip 170 in the video stream. After identifying contact, the remote device processor 224 identifies that a fluid dose has been applied to the deposition site 172 based on a change in at least one optical property of the deposition site 172 in the video stream relative to a previously recorded image of the deposition site 172. Examples of optical properties of the deposition site 172 that change after the deposition site 172 receives a fluid dose include a change in one or more of color, contrast, and brightness of the deposition site 172 caused by the application of the fluid dose.

[0063] In the simplified configuration, the remote device processor 224 omits identification of the finger 190 in the video stream and contact of the finger 190 with the test strip 170. In this simplified configuration, the remote device processor 224 continues to track the test strip 170 until detection of a change in at least one optical property of the deposition site 172 to identify that the test strip 170 has received a fluid dose. In both configurations, the remote device processor 224 starts a timer 226 upon detecting application of a fluid dose to the deposition site 172 on the test strip 170.

[0064] The process 300 continues by the timer 226 reaching a predetermined minimum time and the remote device processor 224 optionally generating an output signal to the user that the camera 108 should be used to generate one or more optical measurements of the measurement site 178 on the back side of the test strip 170' (block 336). The remote device processor 224 generates an output on the display touch screen 146 of the remote device 140 or via the HUD 116 of the wearable computing device 104 to indicate that the optical measurement of the test strip should proceed when the camera 108 generates one or more optical measurements of the measurement site 178 (block 340). The user may then hold the test strip 170 or color card 180 with one or both hands and position it relative to the camera 108 for optimal image capture conducive to analyte measurement. In one embodiment, the HUD 116 may provide graphical aids such as contours for an augmented reality based method for positioning the test strip or color card. Thus, the user moves the test strip or color card (or moves the orientation of the wearable electronic device) so that it is within a graphically provided outline indicating the optimal position in front of the camera 108. In a further embodiment, the processor 204 or 224 automatically detects that the test strip or color card is properly positioned within the outline and further causes the camera 108 to capture images for the optical measurement. In other embodiments, once the user has determined the proper position of the test strip or color card within the graphically provided outline, the user manually activates the camera 108 to capture images for the optical measurement.

[0065] The remote device processor 224 optionally generates an output including a countdown timer to indicate the amount of time remaining within a predefined time window after the expiration of the minimum time period and before the expiration of the predefined maximum time period for generating optical measurements. In one configuration, the remote device processor 224 activates the camera 108 only after the timer 226 indicates that the predefined minimum time period has expired, and in another configuration, the remote device processor 224 only accepts optical measurements from the camera 108 having a timestamp that falls within the predefined time window. While the exact predefined minimum and maximum time periods for generating optical measurements may vary between embodiments, in one configuration, the minimum time period after fluid administration is 13 seconds and the maximum time period is 45 seconds, thereby providing a 32 second time window for the camera 108 to generate one or more optical measurements of the measurement site 178.

[0066] After the timer 226 reaches the expiration of the predetermined maximum time period, the remote device processor 224 generates an output indicating that the maximum time period has expired (block 344). If a sufficient number of images for optical measurements have been generated before the expiration of the predetermined maximum time period (block 348), the remote device processor 224 continues to execute the analyte measurement process based on processing the optical measurements (block 352). In another configuration, if the camera 108 generates a sufficient number of optical measurements before the expiration of the predetermined maximum time period, the remote device processor 224 optionally initiates the measurement process of block 352 without waiting for the expiration of the timer 226. Although not described in further detail herein, the analyte measurement process analyzes the color and optionally other optical properties of the reagent at the measurement site 178 on the test strip to determine the level of an analyte in the fluid sample, such as the level of glucose in a blood sample. In a configuration using a color card 180, the remote device processor 224 uses additional optical data from the color card 180 to assist in the analyte measurement process. The remote device 140 may provide a display of the analyte level measurements to a user via a display device 146, the HUD 116 of the wearable electronic device 104, or another output device.

[0067] The system 100 and process 300 improve the reliability of the analyte measurement process because the user can more easily achieve an optimized position of the test strip or color card in view of the camera 108 because the user can manually place them on the wearable electronics 104 rather than awkwardly manipulating the remote device 140 to capture an image of the test strip or color card set on a surface. Reliability of the process in other embodiments is also improved because the optical measurements are generated during a predetermined window of time, ensuring that the reagents in the test strip 170 have sufficient time to complete a chemical reaction before the optical measurements are generated, but also do not experience dehydration or bleaching before the optical measurement process is completed.

[0068] During process 300, if a predetermined maximum period of time expires prior to the generation of a sufficient number of optical measurements (block 348), the remote device processor 224 does not continue the analyte measurement process and the remote device 140 generates an output message via the display device 146, the HUD 116 of the wearable electronics 104, or another output device indicating that the analyte measurement cannot be completed and instructing the user to begin the analyte testing process again using a new test strip (block 356).

[0069] As described above, the process 300 performs object identification in the video stream starting with the identification of the test strip vial 160 and the opening of the lid 164, which allows the system 100 to verify that the test strip 170 has been extracted from the vial 160, rather than a careless test strip that may have been outside the vial 160 for an extended period of time. Some test strips may become contaminated if left outside the vial for an extended period of time. However, in a simplified configuration of the process 300, the system 100 omits the identification of the test strip vial 160, the lid 164, and the extraction of the test strip 170 from the vial 160. The simplified configuration begins with the generation of the video stream and the identification of the test strip 170. In this configuration, the process 300 does not verify that the test strip 170 has been extracted from the vial, which may not be necessary for some analyte testing systems. This simplified configuration of the process 300 is otherwise identical to the processing described above.

[0070] Those skilled in the art will appreciate that the tracking and identification steps of process 300 are not required for the system 100 to be able to operate to perform an analyte testing operation. The programming and instructions provided by the remote device 140 via the display device 146, HUD 116, or other output device for a testing operation may include sufficient information for a user to manually perform the entire operation, rather than requiring the processor 204, 224 of the wearable electronics 104 or remote device 140 to track and identify specific actions related to the testing operation. For example, initiation of a testing operation may be completed manually by a user via the wearable electronics 108 or remote device 140, after which the user may complete steps, e.g., steps 312, 316, 332, and 340, without generating a video stream of the operation actions. Instead, execution of the program for performing a testing operation may be configured such that a user manually indicates completion of key steps, e.g., administering a test strip (which may start the timer 226), and manually initiating other system-based actions, such as image capture using the camera 108.

[0071] When performing an analyte testing operation, either automatically as in process 300, or manually by the user primarily using the wearable electronics 104 of the system 100 to initiate and complete operational actions for the analyte testing, the user is relieved from handling and using the handheld remote device 140 to capture an image of the administered test strip 170' for its optical measurement. In this way, the user can focus on handling the test strip and optionally the color card, obtaining a source of bodily fluid sample, administering at the attachment site, positioning the test strip with optional color reference, and image capture. User benefits include an easier way for a user holding the test strip (or the color card on which the test strip is placed) to properly and optimally position the distance and angle of the test strip / color reference in front of the camera 108 for accurate image capture, compared to when the user holds the test strip (or the color card on which the test strip is placed) and manipulates the remote device 140 to the appropriate position on the test strip / color card set on a surface. An additional benefit of this approach is that proper positioning of the test strip / color reference for image capture allows for greater accuracy of the analyte measurement, since optical measurements based on images of the administered test strip are more robust.

[0072] Another advantage of using primarily the wearable electronics 104 to complete the operational actions of the measurement process described herein is that the administered test strip / color card is held by the user rather than placed on a surface that may be contaminated by the bodily fluid sample. Similarly, if the user's skin comes into contact with the administered bodily fluid sample on the test strip, by completing the process using only the wearable electronics 104, the user does not need to manually handle the remote device 140 and potentially spread contamination to that device.

[0073] Another advantage of using wearable electronics 104 to complete the operational actions of the measurement process described herein is that the user can receive feedback to accurately complete the measurement process without interrupting the process.

[0074] Although the embodiments disclosed herein use separate wearable electronics 104 and remote device 140 for illustrative purposes, those skilled in the art will recognize that a single electronic device may be configured to perform the operations described herein. In particular, state-of-the-art wearable computing devices typically interface with a remote device for complex operations, while more capable wearable computing devices may implement all of the functions described herein. Alternatively, the remote device 140 may be configured to perform all of the functions described herein to generate a video stream using the optical sensor 142 as a camera and perform other processing as described above. Thus, specific references to the operation of the wearable electronics processor 204 and the remote device processor 224 in the above description should be understood to alternatively refer to the operation of a single processor in an alternative configuration using a single electronic device. Notwithstanding the above, as is typical in the state-of-the-art of medical diagnostic analysis systems, the results of the analyte measurements may be transmitted, uploaded, or otherwise provided to the remote device 140, such as an internet-based or internet-connected data storage and processing location, for purposes of data management, data backup, data processing, and engagement and interfacing with a healthcare provider or healthcare system using purpose-built software applications and the like.

[0075] The present disclosure will be described in connection with what are believed to be the most practical and preferred embodiments. However, these embodiments are presented as examples and are not intended to be limited to the disclosed embodiments. Accordingly, those skilled in the art will understand that the present disclosure encompasses all modifications and alternative configurations within the spirit and scope of the present disclosure and as set forth in the following claims.

[0076] Embodiments include systems for measuring an analyte, as well as methods for measuring an analyte.

[0077] In one embodiment, a method is provided for measuring an analyte using wearable electronics and a remote device having processors operatively connected to each other and configured to cooperatively execute program instructions, the processors identifying a test component in the video stream, providing guidance to a user via a HUD to any required test components not detected in the video stream, identifying application of a bodily fluid sample to a deposition site on the test strip based on the video stream, starting a timer in response to confirmation of application of the bodily fluid sample, generating at least one image of the test strip including at least a portion of the measurement site via the camera after a minimum period from starting the timer and before a maximum period, the at least one image being captured after guidance to the user via the HUD to position the test strip at a predetermined distance and angular orientation relative to the camera, analyzing optical properties of at least a portion of the measurement site in the image to determine a level of the analyte, and displaying the level of the analyte on the HUD.

[0078] In another embodiment, the method includes identifying a vial in a video stream generated by the camera based on at least one registration mark located on the vial depicted in the video stream, identifying an opening of the vial in the video stream based on at least one registration mark located on a lid of the vial, and identifying an extraction of a test strip from the vial in the video stream after identifying the opening of the vial based on at least one registration mark associated with a test strip depicted in the video stream.

[0079] In another embodiment, the method includes identifying in the video stream that the back side of the test strip is exposed based on the absence of an indicator formed on the surface of the test strip, and generating an output message on the HUD indicating that the test strip should be rotated to expose the surface of the test strip carrying the indicator.

[0080] In a further embodiment, the method identifies a user's finger in the video stream, generates guidance on the HUD for generating a bodily fluid sample following identification of the finger in the video stream, identifies contact of the finger with the deposition site in the video stream, and identifies application of the fluid sample in response to a change in optical properties of the deposition site in the video stream after contact of the finger with the deposition site.

[0081] In another embodiment, the program instructions proceed based on automatic tracking of user activity and automatic identification of components required for the analyte testing operation, and at least one of the first processor and the second processor is configured to perform the automatic tracking and automatic identification via a video stream generated by the camera.

[0082] In another embodiment, the method identifies errors in the extraction and handling of the test strip depicted in the video stream and provides guidance to the user via the HUD based on the errors.

[0083] In a further embodiment, the step of identifying the application of the bodily fluid sample occurs after guidance to the user via the HUD for correct application of the bodily fluid to the test strip.

[0084] In another embodiment, the method includes initiating a second timer in response to a user removing the test strip from the vial, the second timer configured to count down a predetermined sample application time during which the user applies the bodily fluid sample to the application site of the test strip. In one aspect, the method includes displaying a dynamic countdown of the predetermined sample application time on the HUD and / or a display of the remote device to visually indicate to the user the time remaining to apply the bodily fluid sample to the application site of the test strip. In another aspect, the method includes canceling the second timer in response to confirmation of application of the bodily fluid sample or sounding an audible and / or visual alarm in response to failure to confirm application of the bodily fluid sample.

[0085] In a further embodiment, the method includes using the wearable electronics or remote device to initiate program instructions and provide guidance to the user regarding the performance of further steps of the method.

[0086] In yet another embodiment, the method includes generating a video stream using a camera of the wearable electronics, optionally identifying one or more components required for the analyte testing operation including a vial, one or more test strips, a color card, and the generated bodily fluid sample, and tracking user activity with respect to the one or more components.

[0087] In a further embodiment, the method includes generating an output message on the HUD informing a user that the measurement of the analyte in the fluid dose cannot be completed in response to the optical measurement of the measurement site not being generated after a predetermined minimum period of time has elapsed and before a predetermined maximum period of time has elapsed.

[0088] In another embodiment, the method includes displaying the level of the analyte on a display of a remote device.

[0089] In yet another embodiment, the wearable electronics and / or remote device are configured to receive and respond to voice communications from a user, and the method further includes the user providing verbal commands instructing one or more of: initiating execution of program instructions by the device to perform an analyte testing operation; confirming extraction of the test strip; confirming placement of the test strip within a color card configured to hold the test strip; confirming generation of the bodily fluid sample; confirming application of the bodily fluid sample at an administration site of the test strip; generating a video stream; and generating one or more images indicative of analyzable optical properties of the measurement site of the test strip to which the bodily fluid sample has been applied.

Claims

1. 1. A system for measuring an analyte, comprising a wearable electronic device and a remote device, The wearable electronic device comprises: a camera configured to generate a video stream; a first communications transceiver configured to transmit and receive communications to and from a second communications transceiver of said remote device; - a head-up display (HUD), a first memory configured to store program instructions; a first processor operatively connected to said first communications transceiver, said camera, and said first memory, said first processor configured to execute said program instructions; Equipped with The remote device: a second memory configured to store program instructions; a second processor operatively connected to the second transceiver and the second memory, the second processor configured to transmit and receive communications to and from the first communications transceiver; Furthermore, the wearable electronic device and the remote device are operatively connected to each other, and the first processor and the second processor cooperate to execute the program instructions; - Identifying test components within the video stream and guiding the user via the HUD to any required test components not detected in the video stream; - identifying the application of a bodily fluid sample to a deposition site on a test strip based on said video stream; - initiating a timer in response to confirming said application of said body fluid sample; - generating at least one image of the test strip via the camera, including at least a portion of a measurement site, after a minimum period of time from the starting of the timer and before a maximum period of time, the at least one image being captured after guiding the user via the HUD to position the test strip at a predetermined distance and angular orientation relative to the camera; - analyzing optical properties of at least a portion of the measurement site in the image to determine the level of an analyte; - displaying the level of the analyte on the HUD; system.

2. 10. The system of claim 1, wherein the program instructions proceed based on one or more inputs from the user upon completion of at least one of the steps of applying the bodily fluid sample to the deposition site on the test strip and generating the at least one image.

3. 2. The system of claim 1, wherein the program instructions proceed based on automatic tracking of user activity and automatic identification of components required for analyte testing operations, and at least one of the first processor and the second processor is configured to perform the automatic tracking and the automatic identification via a video stream generated by the camera.

4. At least one of the first processor and the second processor - identifying the vial in the video stream produced by the camera based on at least one registration mark located on the vial depicted in the video stream; - identifying the opening of the vial in the video stream based on at least one registration mark located on the lid of the vial; - identifying the removal of the test strip from the vial after the identification of the opening of the vial in the video stream based on at least one registration mark associated with the test strip depicted in the video stream. The system of claim 3 further configured to:

5. 5. The system of claim 4, wherein the at least one registration mark associated with the vial further comprises an indicator formed on a label of the vial.

6. The system of claim 4 , wherein the at least one registration mark associated with the test strip further comprises an indicator formed on the surface of the test strip at a predetermined location relative to the attachment site.

7. At least one of the first processor and the second processor - identifying in the video stream that the backside of the test strip is exposed based on the absence of the indicator formed on the front side of the test strip; - generating an output message on the HUD indicating that the test strip should be rotated to expose the surface of the test strip that carries the indicator; The system of claim 6, further configured to:

8. 4. The system of claim 3, wherein at least one of the first processor and the second processor is further configured to identify errors in the extraction and handling of the test strip depicted in the video stream and provide guidance to the user via the HUD based on the errors.

9. 10. The system of claim 1, wherein the step of identifying the application of a bodily fluid sample occurs after guidance to the user via the HUD for correct application of bodily fluid to the test strip.

10. At least one of the first processor and the second processor - identifying a user's finger in said video stream; - generating instructions on the HUD for generating a bodily fluid sample following identification of the finger in the video stream; - identifying contact between the finger and the attachment site in the video stream; - identifying said application of said fluid sample in response to a change in optical properties of said application site in said video stream after said contact of said finger with said application site; The system of claim 3 further configured to:

11. 4. The system of claim 3, wherein at least one of the first processor and the second processor is further configured to identify the application of the fluid sample in response to a change in optical properties of the deposition site in the video stream.

12. 4. The system of claim 3, wherein at least one of the first processor and the second processor is further configured to generate an output message on the HUD informing the user that the measurement of the analyte in the fluid dose cannot be completed in response to no optical measurement of the measurement site being generated after the predetermined minimum period of time has elapsed and before the predetermined maximum period of time has elapsed.

13. 13. The system of claim 1, wherein the remote device further comprises a display operably connected to the second processor, and the action further comprises displaying the level of the analyte on the display.

14. The system of claim 13 , wherein the remote device comprises a mobile electronic device.

15. 15. The system of claim 14, wherein the mobile electronic device comprises a smartphone, and the program instructions comprise a software application stored in the second memory and executed by the second processor in communication with the first processor to instruct the wearable electronic device in performing an analyte testing operation.

Images