Systems and methods for automated optical measurement of analytes with wearable smart devices - Patents.com

Abstract

A system and method for measuring an analyte includes an instrument configured to perform an analyte test operation. The instrument includes a wearable electronics and a remote instrument operatively connected to each other and each having a processor, which cooperate with each other in executing program instructions configured to direct the instrument in performing the analyte test operation. The wearable electronics includes a camera configured to generate a video stream and one or more images of a user removing a test strip from a vial, generating a bodily fluid sample, and applying the sample to a deposition site on the test strip, the sample undergoing a change in one or more optical properties, and the images are analyzed to determine the level of the analyte. The wearable electronics further includes a heads-up display (HUD) for providing output messages to a user regarding the execution and status of the analyte test operation.

Description

[Technical field]

[0001] The present disclosure relates generally to the field of test substance measurement systems, more specifically to optical measurement systems that assist a user in performing a test substance measurement process, and even more specifically to the use of wearable smart devices to perform automated optical measurements of test substances. [Background technology]

[0002] Analyte measurement systems known in the art allow for the analysis of a bodily fluid application provided by a user to identify the level of one or more analytes in the user's body using electronics and one or more electrochemical reactions. These analyte measurement systems provide important benefits to individual users with respect to accurate measurement of an analyte (i.e., biological or environmental) in a fluid sample. Some analyte measurement systems use a test strip that carries a chemical reagent. Upon receiving a fluid application containing an analyte, a chemical reaction between the reagent and the analyte causes the reagent to change color, and the color change varies based on the concentration of the analyte, which provides a measurement of the analyte. While many analytes are measured in this manner, one specific example of an analyte measured in a fluid application is glucose, which is measured in a bodily fluid application as part of the monitoring and treatment of diabetes mellitus.

[0003] Older test strip systems that measure the level of an analyte in a sample based on a detectable color change have relied on a human observer, often with the aid of a printed color matching guide, to determine the analyte measurement by observing the color change of the reagent. Such manual systems can exhibit problems with low accuracy and inconsistent measurements based on the perception of different human observers. More recently, automated analyte measurement devices have been developed that use a camera to observe the color change of the reagent to help improve the accuracy of the analyte measurement. For example, widely available smartphones include optical sensors, such as a camera, and digital image processing hardware that allows the smartphone to generate a measurement of the analyte in the test strip when the smartphone runs a specially configured analyte measurement software application.

[0004] Examples of systems and configurations are known, for example, from PCT / EP2019 / 080154 (published as WO 2020 / 094594), which describes methods, computer programs and devices that allow a mobile device to perform analytical measurements with higher measurement accuracy and greater reliability while ensuring relatively convenient handling for the user. Such configurations include a method of providing an optical test strip with a test field to which no sample has been applied (i.e., a blank test field), acquiring at least one image of part or the whole of the blank test field using a camera of the mobile device (e.g., a smartphone), applying a sample of body fluid to the test field (i.e., the post-application test field), waiting a predefined time for a color change reaction to occur, acquiring a second image of part or the whole of the post-application test field using the camera of the mobile device (ensuring that the same image acquisition settings are used as when the image of the blank test field was acquired), and using the acquired images to determine an analytical measurement result value.

[0005] To further increase the accuracy and reliability of the measurement of the test material carried out in this manner, further developments have been made, including the use of a standard color standard, and an image of such a standard can be acquired, and the camera of the mobile device can be further calibrated based on the acquired image.For example, European Application No. 20173917.4 (unpublished) discloses a color reference card as part of a kit for carrying out the measurement of the test material.The method and arrangement disclosed therein rely on the color standard and normalize the camera system of the mobile device according to the perceived difference between the measured reference color value and the known (predetermined) color value of the color standard.

[0006] In other configurations, with or without standard color standards, the method and device can be configured to address other technical challenges of analytical measurements using mobile devices, increase measurement accuracy and improve reliability while still allowing convenient handling for users. For example, a method is known for evaluating the suitability of a mobile device having at least one camera for performing an analytical measurement based on a color reaction, such a method comprising a step for evaluating the suitability of the mobile device for performing an analytical measurement based on a color reaction. A further method comprises calibrating the mobile device for performing the analytical measurement. Furthermore, a computer program is provided for carrying out the steps of suitability evaluation, calibration and measurement by the mobile device. They are known, for example, from PCT / EP2019 / 079332 (published as WO2020 / 094436).

[0007] Such system embodiments rely heavily on programming the mobile device to perform steps related to, among other things, measurement, calibration, suitability assessment, and color referencing. This is typically accomplished using a software application or app that is installed on the mobile device, such as a smartphone or tablet device. The app's functionality typically includes displaying analyte measurement results on the device's display. Additionally, the app may enable access to web or cloud-based data management tools, health care records, and the like, to enhance disease management functions. If desired, the app may include programming residing in the memory of the mobile device that causes the execution of steps that result in accurate and reliable measurement results. Alternatively, the app's programming may direct the mobile device to interact with a memory location remote from the mobile device, such as a remote computer server, to download programming instructions for performing the measurement steps. In the latter case, a reliable connection of the mobile device to the Internet is required.

[0008] Although the use of specially configured optical measurement devices improves the measurement of test materials, challenges remain in ensuring ease of use and accuracy of the measurement process. One such challenge arises with respect to the proper angle, position, and / or distance of the smartphone's optical sensor relative to the applied test specimen when an image of the applied test specimen should be captured to ensure robust digital image processing techniques. If the smartphone camera is too close to the applied test specimen or too far away from the applied test specimen when the image is captured, it may not provide proper image resolution. Similarly, if the smartphone camera is pointed at too large an angle toward the applied test specimen, it may result in distorted images that cause unstable or inconsistent color saturation, specular reflections, depth of field issues (which can occur with or without a color reference card, when not all areas of the card are in focus at the same time), and spatial resolution issues such as compression of one axis due to an improper angle of the camera relative to the measurement area of ​​the applied test specimen. Any of these situations may result in inaccurate test material measurement results. Another such challenge arises in connection with placing the test strip on a surface after application, in order for the user to hold and operate the smartphone in the proper position for image acquisition. Placing a test strip on a flat surface with a bodily fluid such as blood applied thereto risks contamination of the surface. Therefore, improvements in optical analyte measurement systems that overcome these challenges would be beneficial. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] PCT / EP2019 / 080154 (International Publication No. 2020 / 094594) [Patent Document 2] European Application No. 20173917.4 [Patent Document 3] PCT / EP2019 / 079332 (International Publication No. 2020 / 094436) Summary of the Invention

[0010] The present invention relates to a system and method for the measurement of an analyte, the system and method including a wearable electronic device and a remote device, the wearable electronic device including a camera configured to generate a video stream and at least one image having an optically measurable characteristic, a first communication transceiver configured to transmit and receive communications with a second communication transceiver of the remote device, a head-up display (HUD), a first memory configured to store program instructions, and a first processor operatively connected to the first communication transceiver, the camera, and the first memory and configured to execute the program instructions. The remote device includes a second memory configured to store program instructions, the second transceiver, and a second processor operatively connected to the second memory, the second transceiver configured to transmit and receive communications with the first communication transceiver. The wearable electronics and the remote device are operatively connected to each other, and the first processor and the second processor cooperatively execute program instructions configured to operate the system to perform a analyte test operation on a bodily fluid sample, the operations including a user removing a test strip from a vial, generating a bodily fluid sample, applying the bodily fluid sample to an application site on the test strip, activating a timer in response to confirmation of application of the bodily fluid sample, generating at least one image by the camera of the test strip including at least a portion of the measurement site after a minimum time and before a maximum time from activation of the timer, the at least one image being obtained after guidance to the user via the HUD to position the test strip at a predetermined distance and angular orientation relative to the camera, analyzing optical properties of at least a portion of the measurement site in the image to determine a level of the analyte, and displaying the level of the analyte on the HUD.

[0011] In one aspect, the program instructions proceed to execute the test specimen testing operation based on one or more inputs from a user upon completion of at least one of the steps of removing a test strip, generating a bodily fluid sample, applying the bodily fluid sample to the attachment site on the test strip, and generating at least one image.

[0012] In another aspect, the program instructions proceed with the execution of the test substance testing operation based on automatic tracking of user activity and automatic identification of components required for the test substance testing operation, and at least one of the first processor and the second processor is configured to perform the automatic tracking and automatic identification via a video stream generated by the camera.

[0013] In yet another embodiment, at least one of the first processor and the second processor is further configured to identify a vial in the video stream generated by the camera based on at least one registration mark located on the vial depicted in the video stream, identify an opening of the vial in the video stream based on at least one registration mark located on a lid of the vial, and identify an extraction of a test strip from the vial following identification of the opening of the vial in the video stream based on at least one registration mark associated with the test strip depicted in the video stream.

[0014] In yet another embodiment, the registration mark associated with the vial or lid or test strip comprises an indicator formed on a surface such as a label on the vial, an inner surface of the lid, or a surface of the test strip or a color card that holds the test strip.

[0015] In yet another aspect, at least one of the first processor and the second processor is further configured to identify that the back surface of the test strip is appearing in the video stream based on the absence of an indicator formed on the front surface of the test strip and to generate an output message on the HUD indicating that the test strip should be rotated to expose the front surface of the test strip having the indicator.

[0016] In yet another embodiment, at least one of the first processor and the second processor is further configured to identify a user's finger in the video stream, identify contact between the finger and the deposition site in the video stream, and identify application of the dosage in response to a change in optical properties of the deposition site in the video stream following contact between the finger and the deposition site.

[0017] In yet another embodiment, at least one of the first processor and the second processor is further configured to identify application of the dosage in response to a change in an optical property of the deposition site within the video stream.

[0018] In yet another embodiment, at least one of the first processor and the second processor is further configured to generate an output message on the HUD informing a user that the measurement of the test substance in the fluid application volume cannot be completed in response to optical measurement of the measurement site not occurring after a predetermined minimum time has elapsed and before a predetermined maximum time has elapsed.

[0019] In further aspects of the present systems and methods, the remote device further comprises a display device operably connected to the second processor, and the operation further comprises displaying the level of the analyte on the display device. The remote device may further include a mobile electronic device, such as a smart phone, and the program instructions include a software application stored in a second memory, the software application executed by the second processor in communication with the first processor to instruct the wearable electronic device in performing an analyte testing operation. [Brief description of the drawings]

[0020] Further advantages, benefits, features and objects will become more readily apparent from a consideration of the following detailed description and by reference to the following drawings. [Figure 1]FIG. 1 illustrates components of an analyte measurement system using wearable and mobile electronics to clarify various aspects of the measurement kit and the execution of various steps in the analyte measurement process. [Diagram 2] FIG. 2 is a schematic diagram illustrating components of the wearable and mobile electronic devices of FIG. 1. [Diagram 3] FIG. 2 is a block diagram of a process for operation of the analyte measurement system. [Figure 4] 1 is a series of views of a vial holding a test strip as the vial is opened and a test strip is removed from the vial. [Diagram 5] 1 is a series of images of a test strip receiving a fluid dose. [Figure 6] 1 is a series of images of a test strip placed within a color card to receive a fluid dose. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Reference is now made to the accompanying drawings which form a part hereof and which illustrate, by way of example, and not by way of limitation, embodiments of the inventive concepts. Corresponding reference numerals indicate corresponding parts throughout the several views of the drawings.

[0022] While the inventive concept is susceptible to various modifications and alternative forms, exemplary embodiments thereof are shown by way of example in the drawings and described in detail herein. However, it should be understood that the following description of exemplary embodiments is not intended to limit the inventive concept to the specific form disclosed, but rather, the intention is to embrace all advantages, effects, and features included in the spirit and scope of the invention as defined by the embodiments described herein and the following embodiments. Therefore, reference should be made to the embodiments described herein and the following embodiments to interpret the scope of the inventive concept. Therefore, it should be noted that the embodiments described herein may have advantages, effects, and features that are useful in solving other problems.

[0023] The devices, systems, and methods will now be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the inventive concepts are shown. Indeed, the devices, systems, and methods may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, but rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements.

[0024] Similarly, numerous modifications and other embodiments of the devices, systems, and methods described herein will come to mind to one skilled in the art to which this disclosure pertains having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. It is therefore to be understood that the devices, systems, and methods are not limited to the particular embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the embodiments. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this disclosure pertains. Although any methods and materials similar or equivalent to those described herein can be used in the practice or testing of the present method, the preferred methods and materials are described herein.

[0026] Furthermore, the reference of an element by the indefinite article "a" or "an" does not exclude the possibility that more than one element is present, unless the context clearly requires that only one element is present. Thus, the indefinite article "a" or "an" normally means "at least one". Similarly, the terms "having", "comprising" or "including" or any grammatical variants thereof are used in a non-exclusive manner. Thus, these terms can refer both to the situation where, apart from the features introduced by these terms, no further features are present in the entity described in this context, and to the situation where one or more further features are present. For example, the expressions "A has B", "A comprises B" and "A includes B" can refer both to the situation where no other elements are present in A apart from B (i.e., the situation where A is exclusively composed of B), or to the situation where, apart from B, one or more further elements are present in A, such as element C, elements C and D, or even further elements.

[0027] As used herein, the term "mobile electronic device" refers to a portable computing device that provides a user with one or more of the following components: output devices, input devices, memory, and wireless communication devices each controlled by one or more processors within the mobile electronic device. As used herein, the term "wearable electronic device" refers to a type of mobile electronic device that is further configured to be worn by a human user in a manner similar to eyeglasses, clothing, watches, or jewelry. Examples of output devices include, but are not limited to, liquid crystal display (LCD) displays, organic or inorganic light emitting diode (LED) displays, other forms of graphical display devices, audio speakers, and haptic feedback devices. Examples of input devices include, but are not limited to, buttons, keyboards, touch screens, acute vibration sensors, still and video cameras, and audio microphones. Examples of memory include, but are not limited to, both volatile data storage devices, such as random access memory (RAM), and non-volatile data storage devices, such as magnetic disks, optical disks, and solid-state storage devices, including EEPROM, NAND flash, or other forms of solid-state data storage devices. Examples of wireless communication devices include, but are not limited to, wireless transceivers that operate with the Near Field Communication (NFC) protocol, the Bluetooth® protocol suite including Bluetooth® Low Energy (BLE), the IEEE 802.11 protocol suite ("Wi-Fi"), and cellular data transmission standards ("4G", "5G", etc.). Examples of processors include digital logic devices implementing one or more central processing units (CPUs), graphics processing units (GPUs), neural network processors (NPUs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), and application specific integrated circuits (ASICs), as well as any other suitable digital logic devices in an integrated device that cooperate to implement a processor, or as a combination of devices that cooperate to implement a processor.Common examples of mobile electronic devices include, but are not limited to, smart phones, tablet computing devices, and notebook computers. Common examples of wearable electronic devices include, but are not limited to, smart watches and smart glasses.

[0028] FIG. 1 illustrates an analyte measurement system 100 including a wearable electronic device 104 and a remote device 140. Throughout this disclosure, the embodiments described typically refer to an operably connected configuration of the wearable electronic device 104 and the remote device 140, which may typically take the form of a mobile electronic device such as a smartphone or tablet device. However, as will become apparent from this description, it should be noted that the remote device 140 may be effectively replaced by an Internet-based remote device (e.g., a remote server accessed via an Internet connection), provided that the wearable device 104 has a suitable connection to the Internet via a suitable wireless communication device, such as the wireless communication device described above, and the wearable device 104 interacts directly with such a remote device in the same or substantially similar manner as if the mobile electronic device were the remote device 140. Also, any software applications described herein for the functionality of the remote device 140 may be fully executed by the Internet-based remote device to which the wearable device 104 is operably connected.

[0029] With reference to the schematic diagrams of FIGS. 1 and 2, in one embodiment of the system 100, the wearable electronics 104 are embodied as glasses, sometimes referred to as "smart glasses," although in alternative configurations, other forms of wearable electronics may be used, including smart watches. The wearable electronics 104 include a frame and optional lenses similar to traditional glasses. The wearable electronics 104 further include a camera 108, a position sensor 112, and a head-up display (HUD) 116, each of which is operatively connected to an electronic control unit 120. The camera 108 is, for example, a CMOS or other suitable digital imaging device that generates images and video streams of the area in front of the wearable electronics 104 corresponding to the field of view of a person wearing the wearable electronics 104. In some embodiments, a single monochrome or color camera generates the video stream as a two-dimensional video stream. In other embodiments, the camera 108 is further configured to generate a video stream that provides three-dimensional object data. For example, in one configuration, the camera 108 further comprises two or more cameras providing stereoscopic vision, or the camera 108 includes a depth sensor that provides three-dimensional depth information corresponding to objects in the video stream. The position sensor 112 includes, for example, a microelectromechanical (MEMs) three-axis gyroscope and one or more accelerometers that provide data to identify the spatial orientation of the wearable electronics 104 during operation. The HUD 116 provides visual output to the wearer without the wearer having to change his or her line of sight to a particular display device. Although FIG. 1 shows the HUD 116 separate from the glass lens in the wearable electronics 104, alternative configurations provide one or more visual displays that are integrated into the lens or that project graphic output onto the lens. Although not shown in further detail, the wearable electronics 104 also optionally includes audio input and output devices that may be located, for example, in the earpiece, to communicate sounds and audible communications to the user and / or allow the user to provide voice commands for input to and interaction with the wearable electronics 104.In such an embodiment, the wearable electronics 104 includes a microphone operably connected to the processor 204, which can execute voice recognition programming to convert a user's audible commands into actionable inputs, such as initiating a test substance measurement operation or indicating the completion of various activities associated with such an operation.

[0030] 2, the electronic control unit 120 houses at least one wearable electronics processor 204 operably connected to the camera 108, the position sensor 112, and the HUD 116. Furthermore, the electronic control unit 120 houses a memory 208 and a communication transceiver 228 operably connected to the processor 204. In the embodiment of FIG. 2, the memory 208 stores firmware instructions 212 that control the operation of the wearable electronics 104. The communication transceiver 228 includes a transmitter that allows data including a video stream and one or more images to be transmitted to a corresponding communication transceiver 258 in the remote device 140 or in a wireless connection device configured to communicate bidirectionally with the remote device (e.g., a local WiFi communication signal via a signal router operably connected to the Internet). Furthermore, the communication transceiver 228 includes a receiver that allows the wearable electronics 104 to receive data originating from the remote device 140, and in particular to receive messages from the remote device 140 for display to the user via the HUD 116. In the illustrative example of FIG. 2, communications transceiver 228 is a Bluetooth® or Bluetooth® Low Energy wireless data communications transceiver, although alternative configurations may use different wireless communication standards or may employ a wired connection interface such as Universal Serial Bus (USB).

[0031] 1 and 2, in one embodiment of the system 100, the remote device 140 further includes a processor 224 operably connected to a timer 226, a memory 232, a communications transceiver 258, and one or more display and user input / output (I / O) devices 146. The remote device 140 is operably connected to the wearable electronics 104. As will be appreciated, in other embodiments, the wearable processor 204 is comprised of similar components as the processor 224 of the remote device 140 for performing the functions described herein related to performing analyte measurements by the wearable electronics 104.

[0032] The remote device 140 may include an optical sensor 142 operably connected to the processor 224 as part of standard features such as are typically provided on a smart phone or tablet device. However, it will be understood that an optical sensor on the remote device 140 is not required in accordance with embodiments of the present invention. Nevertheless, for purposes of illustration, in an alternative embodiment of the invention described in a related but separate patent application filed by the inventor of the present invention concurrently herewith, the optical sensor 142 provided on the remote device 140 comprises a digital camera that produces still images or a video stream.

[0033] According to an embodiment of the invention, the camera 108 is configured to generate images and video streams related to the performance of analytical measurements according to a series of steps that may be initiated directly by the wearable electronics 104 or by a user's operation of an app on a remote device 140 operatively connected to the wearable electronics 104. For example, as the series of steps is initiated, the camera 108 may generate a video stream of a user acquiring a vial 160 of test strips 170, removing such test strips 170, and optionally positioning a color card 180 to generate at least one optical measurement for analysis to measure the level of an analyte in a bodily fluid application that is subsequently applied to the test strip 170. In the configuration of FIG. 1, the camera 108 is configured to generate a video stream capturing the entire scene, including the vial 160 as well as the test strip 170 and application site 172, optionally including the color card 180, and further including the finger 190 to identify the moment when the test strip 170 receives the fluid application. The camera 108 is further configured to generate one or more images of the measurement site 178 on the back side of the test strip 170' at an appropriate time after the moment of application of the bodily fluid dose, due to the image having optically measurable properties that allow for optical measurements. The optical measurements can be identified, for example, from a digital photograph that includes the measurement site 178 on the test strip 170'. The camera 108 thus provides a wider view of the elements used in the analyte testing process, as well as more detailed digital images or videos of the reagents on the test strip 170 and, optionally, the calibration data provided on the color card 180. For example, the camera 108 may be configurable to generate a lower resolution video stream for identification and tracking of the vial 160, the test strip 170, the color card 180, and the finger 190, so that the moment the test strip 170 receives the fluid dose can be identified by the remote device 140. The camera 108 may then operate at a higher resolution to capture one or more high fidelity images of the test strip 170', either alone or in conjunction with the color card 180, to provide input for the test substance measurement process.

[0034] In one embodiment of the remote device 140, the user input / output (I / O) device 146 includes a touch screen display device that provides graphical output to the user and receives touch input to control the operation of the remote device 140 and, more particularly, to provide input to the analyte measurement process. Other examples of I / O devices include a microphone for audio input and a speaker for audio output, mechanical buttons, and the like. In some configurations, the wearable electronics 104 implements user I / O devices 146 such as an audio input device or a gesture tracking input device that uses the camera 108 to record input from the user which the camera 108 transmits to the remote device 140. The wearable electronics 104 may further receive output data from the remote device 140 for display to the user via the HUD 116.

[0035] To ensure that the optical measurement of the analyte measurement reagent occurs after a predetermined minimum time and before a predetermined maximum time, in the remote device 140, a timer 226 enables the processor 224 to keep count of elapsed time during operation, such as a count of elapsed time beginning the moment the test strip 170 receives the fluid dose. Although the timer 226 is shown as a separate component for illustrative purposes, in many practical embodiments the timer 226 will be integrated into the processor 204, 224 as a timer circuit or implemented as a software timer.

[0036] In FIG. 1 , test strip 170 includes an attachment site 172 where a user provides a bodily fluid sample, such as blood. Additionally, test strip 170 may include a registration mark 174, shown as a printed mark in the form of an arrow on a surface of test strip 170 at a predetermined location relative to attachment site 172. Additionally, a hole 176 is formed through one end of test strip 170. Registration mark 174 allows for efficient identification and tracking of test strip 170, such as identifying which side of test strip 170 faces a camera in a video stream. Although in the illustrative example of FIG. 1 , only one side of test strip 170 is configured to receive a blood sample and registration mark 174 is formed only on this side of test strip 170, in alternative embodiments, the test strip may be configured to receive a blood application on both sides of the test strip. In FIG. 1, test strip 170' shows the same test strip 170 but with the measurement site 178 and hole 176, but without the arrow indicator 174, and in some embodiments, system 100 can detect that the back side of test strip 170' is revealed to camera 108. In test strip 170 / 170', attachment site 172 provides a fluid inlet that allows the fluid dose to permeate through one or more interior layers of the test strip to allow a chemical reaction with one or more reagents in the test strip. Examples of interior layers include, for example, filters as well as various layers of chemical reagents that react to one or more analytes in the fluid dose. Measurement site 178 is an optically revealed area formed on the back side 170' of the test strip that changes color in response to the level of analyte in the fluid dose that permeates through the test strip from attachment site 172. In one configuration, the reagent is presented directly at the measurement site 178, while in another configuration, an optically transparent layer, such as a film, covers the reagent while providing an optical aperture to allow for the generation of an optical measurement of the reagent to detect a color change of the reagent due to exposure to the analyte in the fluid dose.

[0037] In the remote device 140, the memory 232 includes one or more non-volatile and volatile data storage devices. In the configuration of FIG. 2, the memory 232 stores application software 250 and operating system software 254, both of which include instructions for execution by the processor 224 of the remote device. The application software 250 includes instructions implementing a user interface and an analyte analysis program that performs an analyte measurement process based on image analysis of one or more optical measurements of a reagent on the test strip 170. Additionally, the application software 250 stores predetermined minimum and maximum elapsed time thresholds to ensure that an optical measurement is generated after the fluid application has had sufficient time to react with the reagent in the test strip 170 and before a maximum valid time period for the measurement of the analyte has elapsed. As described in more detail below, part of the analyte measurement process includes identifying the vial 160, removing the test strip 170, and identifying contact between the finger 190 and the test strip 170 to apply a fluid application to the deposition site 172. The application software 250 further includes object recognition data 252 that enables the processor 224 to perform automatic object identification and tracking of the vials 160 and test strips 170 in the video stream received from the wearable electronics 104. The object recognition data 252 is generated by a training process that occurs prior to distribution of the application software 250. In particular, the training process utilizes predefined shapes, colors, and patterns of registration marks formed on the vials 160, test strips 170, and color cards 180 to enable automatic identification and tracking of these components in the video stream. Examples of the object recognition data 252 include image classifiers such as neural networks, in particular convolutional neural networks, support vector machines, hidden Markov models, one-dimensional and two-dimensional barcode scanning engines, and the like.Additionally, object recognition data 252 may include filters for color detection and edge detection, along with other image processing data necessary for tasks such as object detection and image segmentation to enable tracking of objects, such as in vials 160, test strips 170, and color cards 180, within the video stream. Operating system (OS) software 254 includes the software kernel, drivers, libraries, and other system software associated with a standard commercially available operating system. The OS software 254 provides standardized services such as network and graphics stacks, a file system for data storage and management, software access to display and I / O devices 146, timers 226, communications transceiver 258, and other components within remote instrument 140.

[0038] In the remote device 140, the communication transceiver 258 includes a transmitter that enables data, including command data and output message data, to be transmitted to the corresponding transceiver 228 in the wearable electronics 104. The communication transceiver 258 further includes a receiver that enables the remote device 140 to receive data from the mobile electronics 140, and in particular to receive a video stream from the camera 108 of the wearable electronics 104. In the illustrative example of FIG. 2, the communication transceiver 258 is a Bluetooth® or Bluetooth® Low Energy wireless data communications transceiver, although alternative configurations may use different wireless communication standards or may employ a wired connection interface, such as USB.

[0039] FIG. 1 further illustrates a vial 160 and a color card 180. The vial 160 contains one or more test strips 170. In addition to providing containment, the vial 160 protects the test strips from contamination in the environment, such as preventing the reagents in the test strips 170 from absorbing too much moisture from the surrounding environment. The vial 160 further includes a printed label 162 having one or more registration marks, shown in FIG. 1 as dashed indicators 163 printed along one or more edges of the label 162. The registration marks form a simple visual indicator that allows efficient identification and tracking of the vial 160 in the video stream generated by the wearable electronic device 104 and transmitted to the mobile electronic device 140. The registration marks 163 are widely distributed on the exterior of the vial 160 to allow identification and tracking of the vial 160 from a wide range of viewing angles and when the vial 160 is held in the user's hand. Alternative embodiments of the registration mark of the vial 160 include alternative printed pattern indicators, including, for example, a bar code, or engraved or embossed geometric shapes formed on the exterior of the vial 160 that aid in automatic identification and tracking of the vial 160. In the embodiment of FIG. 1, a lid 164 provides access to the interior of the vial 160. The lid 164 may be fully removable or may remain attached to the body of the vial 160 when open. In either configuration, a second registration mark 168 is formed on the interior surface of the vial lid 164. The second registration mark 168 may be, for example, a circle or other geometric shape formed in a predetermined color that contrasts with the color of the vial 160 to clearly indicate that the vial 160 has been opened in a video stream generated by the wearable electronic device 104 when the vial 160 is in use. In alternative embodiments, the registration mark 168 is a one- or two-dimensional bar code, or other registration mark identifiable by an automated vision algorithm. Registration marks 163 and 168 allow for precise identification of both vial 160 and determination of when vial 160 has been opened or closed.

[0040] According to an embodiment of the present invention, during operation, the wearable electronics 104 generates a video stream that allows tracking of various activities and actions related to the analytical measurement process. For example, once a measurement sequence is initiated (which may be performed using an open app or software program running on the wearable electronics 104 or on the remote device 140), the wearable electronics 104 may track the vial 160 holding one or more test strips 170 to identify when the test strip 170 is removed from the vial, positioned with respect to a color reference card, and receives a fluid application, such as a blood application, from a human subject's finger 190. The wearable electronics 104 may then utilize the (HUD) 116 to provide instructions to the user on the step-by-step process of performing the analyte measurement. For example, the HUD 116 may instruct the user to prepare for the measurement and then scan an area within the field of view of the camera 108. As described in more detail below, upon detecting application of an applied amount of fluid to the deposition site 172 on the test strip 170, the system 100 starts a timer and, after a predetermined minimum time has elapsed and before a predetermined maximum time has elapsed, instructs the user via the HUD 116 to use the camera 108 to capture an image that is transmitted to the remote device 140 to generate an optical measurement of the deposition site 172 on the test strip 170.

[0041] In operation, the camera 108 acquires images from which a color change in a reagent found at the measurement site 178 on the test strip 170' is detected in response to one or more chemical reactions with the analyte in the fluid application. In the illustrative example of FIG. 1, the color change in the reagent located at the measurement site 178 represents the level of the glucose analyte in the blood sample. As discussed above, the system 100 identifies when the deposition site 172 has received a fluid application and uses a timer to determine when the optical sensor should generate a subsequent optical measurement of the measurement site 178 to ensure an accurate measurement of the blood glucose level. Although the system 100 depicts a test strip in which the deposition site 172 and the measurement site 178 are separate for purposes of illustration, one of ordinary skill in the art will recognize that alternative test strips provide a single deposition site and measurement site with the reagent located together in the same region of the test strip. Thus, in some embodiments, the deposition site and the reagent occupy separate locations on the test strip, while in other embodiments, the deposition site and the reagent refer to a single location on the test strip.

[0042] In FIG. 1, color card 180 is an optional component having a back surface shown in FIG. 1 that holds test strip 170 in place prior to application. Additionally, color card 180 has a front surface (not shown) that includes a predetermined arrangement of color and other fiducial markings that aid in calibration of the image from camera 108 for accurate color measurement of the presented measurement site 178. An opening 186 in color card 180 allows measurement site 178 on the opposite side of test strip 170' to be measured by an optical sensor within the color pattern of color card 180. The back surface of color card 180 includes registration marks 182 and 184, shown as arrow indicators printed on the back surface of color card 180 in the exemplary embodiment of FIG. 1. Registration marks 182 and 184 are associated with test strip 170 to further aid in identification and tracking of test strip 170 within a video stream for detecting when test strip 170 receives a fluid application. Color card 180 is optional, and system 100 is configured to use test strip 170' alone or in conjunction with color card 180 to generate optical measurements of a reagent at measurement site 178 of test strip 170' to measure a glucose analyte or another type of analyte in a blood sample.

[0043] 3 illustrates a process 300 for the operation of the system 100 to perform an automated analyte testing operation. Automatically detecting various aspects of the operation relieves the user from the need to interact with, input from, or operate devices in the system 100, and also allows for more accurate analyte measurements. For example, automatically detecting when a fluid application is applied to a deposition site on a test strip can initiate automatic timing of when the system 100 should generate one or more images for optical measurement of the measurement site to measure the level of analyte in the fluid application. In describing the process 300, references to a process of performing a function or operation refer to the operation of one or more digital processors, such as the processors 204, 224 of the wearable electronics 104 and the remote device 140, respectively, to cooperate to execute stored program instructions to perform the function or operation in conjunction with other components in the system 100.

[0044] It will be clear to one skilled in the art how independent processors may cooperate in the execution of stored program instructions, including how and where each processor may store program instructions in the memory of each device that cooperates with each other. For example, the processor 224 of the remote device executes locally stored program instructions to trigger wireless communication of the program instructions to the wearable electronics via the transceivers 228, 258, and the memory 208 of the wearable electronics stores the received program instructions for immediate or delayed execution according to instructions received from the remote device 140. Similarly, the processor 224 of the remote device may deliver instructions to the processor 204 of the wearable electronics to directly trigger the execution of various steps of the analyte measurement operation by components of the wearable electronics 104. In other embodiments, the remote device 140 controls all aspects of the execution of the analyte measurement operation, and the processor 224 delivers the program instructions to the wearable electronics 104 for immediate execution by the processor 204. In yet other embodiments, the memory 208 stores complementary or auxiliary program instructions with respect to the execution of program instructions that may be received for immediate execution from the processor 224, and the complementary or auxiliary program instructions are configured to enable execution by the processor 204 of the program instructions received from the processor 224 to be interrupted based on real-time input or sensory observations by components of the wearable electronics 104 that direct interruption and independent operation by the wearable electronics.

[0045] Cooperation between the processors 204, 224 in executing stored program instructions to perform the analyte measurement operation may further include the processor 204 causing transmission or communication of data, images, video streams, or other information to the processor 224 of the remote device via exchanges between the transceivers 228, 258, which in turn executes locally stored program instructions relating to further processing of the information related to the analyte measurement operation. In this manner, the wearable electronics 104 functions in a so-called servant capacity relative to the remote device 140 functioning in a so-called master capacity.

[0046] Process 300 begins after a user initiates execution of application software 250 on system 100, with remote device 140 sending a command to wearable electronics 104 to activate camera 108 to generate a video stream of a scene in front of the user at the start of a specimen testing process (block 304). In the embodiment of FIG. 1, wearable electronics 104 transmits a video stream from camera 108 to remote device 140 using transceiver 228, which enables processor 224 of the remote device to receive the video stream for further processing using corresponding transceiver 258. As is commonly known in the art, the video stream includes a series of frames of image data over time depicting a view from camera 108 during a specimen testing process.

[0047] The process 300 continues when the processor 224 automatically identifies the vial 160 in the video stream generated by the wearable electronics 104 (block 308). Numerous digital image processing techniques may be used to identify objects such as the vial 160, or other objects detected in the video stream during the process 300, but non-limiting examples of preferred techniques are described in further detail herein. The vial 160 identification process further includes an object tracking operation that segments different portions of frames in the video stream that contain the object, and an object identification operation that uses an image classifier to identify the tracked object.

[0048] In an object tracking operation, the processor 224 of the remote device 140 identifies and tracks one or more objects depicted in the video stream. To track the objects, the processor 224 performs a contour detection operation to identify boundaries of various objects in the video stream that have similar image intensity values, such as the boundaries of the vial 160. In some configurations, the processor 224 performs image pre-processing operations such as converting the color video stream to grayscale, thresholding the grayscale pixels, and performing an edge detection process to improve the accuracy of the contour detection process. The processor 224 segments the original image, for example, using a rectangular bounding box that encloses the detected contour area, and the processor 224 performs the contour detection process over a series of video frames to track the movement of the object, for example, as the user moves the vial 160. For example, as shown in the field of view 404 of FIG. 4, the video stream depicts the vial 160, and the processor 224 generates a segment of a rectangular bounding box 406 in the frames of the video stream that includes the detected contour of the vial 160. Although the field of view 404 shows the vial 160 in isolation, some frames in the video stream contain multiple objects, and the contour detection process described above allows for tracking of multiple objects within the video stream.

[0049] Once the tracking operation is complete, the processor 224 may have access to one or more image segments that contain the object, but the processor 224 has not yet determined the identity of the individual object, e.g., the processor 224 has tracked an object in the image segment 406, but has not yet identified whether the object is a vial 160 or some other object. The object tracking process generates multiple image segments that can improve the accuracy of an image classifier for detection of multiple related objects that may occupy different portions of a frame in a video stream. To complete the object identification process, the processor 224 provides the segmented portion of the image that contains the tracked object as an input to a trained image classifier stored with the object recognition data 252 in the memory 232. The image classifier may be, for example, a trained convolutional neural network (CNN) or other suitable image classifier trained to identify a predetermined set of objects, such as the interior of the vial 160 and vial lid 164, both sides of the test strips 170, 171', the color card 180, or the finger 190. A training process for the image classifier occurs prior to process 300, in particular the image classifier is trained to explicitly or implicitly recognize registration mark features formed on the vial 160, the interior of the vial lid 164, both sides of the test strip 170, and the color card 180 to improve accuracy of identification of a given object. Additionally, the training process may include training examples that occur when only a portion of the registration marks are visible to the camera 108, such as when a user holds the vial 160 in their hand and some of the registration marks 163 may be occluded. The processor 224 optionally performs additional pre-processing of the image data, which may include resizing the image data to a predetermined resolution or performing a rotational transformation of the image based on metadata received from the position sensor 112 in the wearable electronics 104 that identifies the angular orientation of the camera 108 when each frame of the video stream is generated to improve accuracy of the image classifier.In some configurations, the image classifier is trained using monochrome image data, while in other configurations, color images are preferred, such as configurations in which registration marks are formed using a predefined color that aids image classification to identify the object. Unrelated objects that may be present in the video stream may also be rejected as unrelated by the image classifier. Furthermore, since the video stream comprises a series of frames, the system 100 may be configured to recognize the vial 160 in one or more frames of the video stream, even if the tracking and identification process is not successful in some of the frames of the video stream. One example of a software framework that enables the image processing operations described above in the application software 250 is the Open Computer Vision (OpenCV) project, available at https: / / opencv.org / . The above process for identifying the vial 160 is substantially the same as the process described below for identifying other objects in the video stream in process 300.

[0050] In the vial identification process, the remote device 140 optionally sends a graphical indicator, such as an icon, to the wearable electronics 104 to assist the user in identifying the next step in the process for performing the test analysis. For example, the remote device 140 sends a graphical icon corresponding to the shape of the vial 160 to the wearable electronics 104, and the processor 204 of the wearable electronics generates a graphical display of the icon using the HUD 116 to alert the user to pick up the vial 160 and place it within the field of view of the camera 108 until the vial 160 is identified in the video stream. In FIG. 4, the field of view 404 shows an exemplary icon 408 that the HUD 116 overlays on the scene recorded by the camera 108 to prompt the user to pick up the vial 160.

[0051] Referring again to Figure 3, the process 300 continues when the remote device's processor 224 identifies the vial 160 as having been opened (block 312) in response to detecting a registration mark 168 formed on the interior of the lid 164 of the vial 160. Referring to Figure 4, a field of view 412 shows the lid 164 removed from the vial 160, where the registration mark 168 is visible on the interior of the lid 164. The remote device's processor 224 tracks and identifies the lid 164 within the image segment 416 in the same manner as described above with respect to the vial 160. Additionally, the remote device 140 optionally transmits an icon of the lid to the wearable electronics 104, which displays the icon 420 on the HUD 116 to provide guidance to the user.

[0052] 3, the process 300 continues when the remote device processor 224 identifies that the test strip 170 has been removed by the user from the opened vial 160 (block 316). The remote device processor 224 tracks and identifies the test strip 170 within the image segment 428 in the same manner as described above with respect to the vial 160 and lid 164. Additionally, the remote device 140 optionally transmits an icon of the test strip to the wearable electronics 104, which displays an icon 432 on the HUD 116 to provide guidance to the user. As shown in FIG. 4, in some cases, the user removes the test strip with the back surface 170' visible to the camera 108. The remote instrument processor 224 tracks and identifies the back side 170' of the test strip within the region 430 and optionally generates an output message to the user via the HUD 116 or other output device 146 to rotate the test strip so that the side of the test strip 170 having the registration marks 174 and the attachment sites 172 is visible in the video stream. If the registration marks 174 are not present on the back side 170', then optionally various other features of the back side 170' provide sufficient differentiation for the image classifier to distinguish between the sides 170, 170' of the test strip. This operation is also performed in subsequent stages of the process 300 when the test strip 170 is inverted to reveal the back side 170' before the test strip 170 receives the fluid dose. Although field of view 424 shows vial 160, the interior of lid 164, and test strip 170 simultaneously for illustrative purposes, detection of the removal of test strip 170 from vial 160 only requires sequential detection of vial 160, interior of lid 164, and test strip 170 in the same video stream over a relatively short period of time, such as a 10, 30, or 60 second time window. Thus, vial 160, lid 164, and test strip 170 do not need to be identified simultaneously in the video stream to identify that test strip 170 has been removed from vial 160 in process 300.

[0053] In the process 300, if the processor 224 of the remote device is unable to achieve sequential identification of the vial 160, the interior of the lid 164 indicating that the vial 160 has been opened, or the removed test strip 170 within a predetermined period of time (block 320), the process 300 returns to the processing described above with reference to block 308 to allow the user to repeat the process. If the test strip 170 is successfully identified as having been removed from the vial 160 (block 320), the process 300 continues as the processor 224 of the remote device continues to track the identified test strip 170 within the video stream (block 328). Alternatively, the user may choose to abort the automated process and begin manual execution of the entire operational activities of the process 300.

[0054] In one configuration, once the processor 224 of the remote device successfully identifies that the test strip 170 has been removed from the vial 160, a second timer is initiated to count down a predetermined sample application time during which the user must apply a bodily fluid sample to the attachment site on the test strip. The predetermined sample application time is determined based on the optimal time between the test strip 170 being removed from the vial 160 and exposed to the environment outside the vial and the bodily fluid sample being applied. If the user waits too long to apply the sample, the exposure of the test strip to the environment may contaminate the test strip in a manner detrimental to accurate and valid analyte measurement. The countdown initiated by the activation of the second timer continues until the processor 224 of the remote device confirms the application of the sample. If the application of the sample to the test strip is confirmed before the countdown is completed, the second timer is cancelled. If the application of the sample is not confirmed and the countdown ends, one or both of an audible and visual alarms are triggered to inform the user that the test strip has appeared too long and should be discarded. In some embodiments, the countdown may be displayed on the HUD 116 and / or the display 146 of the remote device 140.

[0055] In a similar manner, the processor 224 of the remote device in another configuration tracks the vial 160 and lid 164 to monitor and track whether the user has closed the lid. Closing the lid 164 helps protect the remaining test strip 170 in the vial 160 from exposure to the environment. If the processor 224 of the remote device does not detect that the lid 164 of the vial 160 has been closed after the test strip 170 has been removed from the vial, the processor 224 of the remote device may be configured to generate a text or graphic alert to the user via the HUD 116 and / or the display 146 of the remote device 140 to remind the user to close the lid.

[0056] In one configuration, the remote device processor 224 tracks a single test strip 170, as shown in field of view 504 of FIG. 5. In another configuration utilizing a color card 180, the remote device processor 224 identifies the color card 180 based at least in part on the registration mark arrows 182 and 184, and tracks the user's insertion of the test strip 170 into the color card 180, as shown in field of view 604 of FIG. 6. After identifying the test strip 170 and before the test strip 170 receives a fluid application, the remote device processor 224 instructs the wearable electronics 104 to acquire at least one image of the test strip including the application site 172 and store the at least one image in memory 232. The instructions may include messages and / or graphics provided via the HUD 116 to enable the user to manipulate the color card 180 in a manner to optimally position the color card 180 with the test strip 170 at the proper distance and location with respect to the angular orientation of the image of the unapplied application site 172 relative to the camera 108. As described in further detail below, after the attachment site 172 receives an application of fluid, at least one optical characteristic of the attachment site 172 changes, and this change in optical characteristic allows detection of the point at which the test strip 170 has received an application of fluid.

[0057] The process 300 continues by the processor 224 of the remote device prompting the user via the HUD 116 to apply a blood sample to the application site 172 of the test strip 170. The user may then perform the step of obtaining a blood sample, such as by using an appropriate lancing device to lance the user's skin to obtain a source of blood. The user may then grasp the test strip 170 or the color card 180 on which the test strip is located in one hand and begin applying a fluid dab of blood from the lanced skin to the application site. The processor 224 of the remote device, in one embodiment, then identifies the application of a fluid dab to the application site on the test strip based on the video stream and starts a timer 226 upon identifying the application of the fluid dab (block 332).

[0058] In one configuration, the remote device processor 224 identifies the user's finger 190 in the video stream using the same procedures described above for identifying the vial 160, the lid 164, the test strip 170, and the color card 180. The remote device processor 224 further identifies contact between the finger and the deposition site 172 in the video stream. For example, contact is identified in response to the finger 190 obstructing the test strip 170 in the video stream, as shown in field of view 508 of FIG. 5 and field of view 608 of FIG. 6. After identifying contact, the remote device processor 224 identifies that a fluid load has been applied to the deposition site 172 based on a change in at least one optical property of the deposition site 172 in the video stream relative to a previously recorded image of the deposition site 172. Examples of optical properties of the deposition site 172 that change after the deposition site 172 receives a fluid load include a change in one or more of color, contrast, and brightness of the deposition site 172 resulting from the application of the fluid load.

[0059] In a simplified configuration, the remote device processor 224 omits identification of the finger 190 in the video stream and contact between the finger 190 and the test strip 170. In this simplified configuration, the remote device processor 224 continues to track the test strip 170 until a change in at least one optical property of the application site 172 is detected to identify that the test strip 170 has received a fluid dose. In both configurations, the remote device processor 224 starts a timer 226 upon detecting the application of a fluid dose to the application site 172 on the test strip 170.

[0060] The process 300 continues when the timer 226 reaches a predetermined minimum time, and the processor 224 of the remote device optionally generates an output signal to the user that the camera 108 should be used to generate one or more optical measurements of the measurement site 178 on the back side of the test strip 170' (block 336). The processor 224 of the remote device generates an output on the display touch screen 146 of the remote device 140 or via the HUD 116 of the wearable computing device 104 to indicate that the optical measurement of the test strip should proceed when the camera 108 generates one or more optical measurements of the measurement site 178 (block 340). The user may then hold the test strip 170 or color card 180 in one or both hands and position it relative to the camera 108 for optimal image acquisition to aid in the measurement of the analyte. In one embodiment, the HUD 116 may provide graphical aids such as contours for an augmented reality based method for positioning the test strip or color card. Thus, the user moves the test strip or color card (or orients the wearable electronics) so that it is located within the outline provided in the graphic that indicates optimal positioning in front of the camera 108. In further embodiments, the processor 204 or 224 automatically detects that the test strip or color card is properly positioned within the outline, and further automatically causes the camera 108 to capture an image for the optical measurement. In other embodiments, once the user determines that the test strip or color card is properly positioned within the outline provided in the graphic, the user manually activates the camera 108 to capture an image for the optical measurement.

[0061] The remote device's processor 224 optionally generates an output including a countdown timer to indicate the amount of time remaining in a predetermined time window after the expiration of the minimum time and before the expiration of the predetermined maximum time for generating optical measurements. In one configuration, the remote device's processor 224 activates the camera 108 only after the timer 226 indicates that the predetermined minimum time has expired, while in another configuration, the remote device's processor 224 only accepts optical measurements from the camera 108 having a timestamp that falls within the predetermined time window. While the exact predetermined minimum and maximum times for generating optical measurements may vary between embodiments, in one configuration, the minimum time following application of the fluid is 13 seconds and the maximum period is 45 seconds, providing the camera 108 with a 32 second window of time for generating one or more optical measurements of the measurement site 178.

[0062] After the timer 226 reaches the expiration of the predetermined maximum time, the processor 224 of the remote device generates an output indicating that the maximum time has expired (block 344). If a sufficient number of images for optical measurements have been generated before the expiration of the predetermined maximum time (block 348), the processor 224 of the remote device continues to execute the analyte measurement process based on processing the optical measurements (block 352). In another configuration, if the camera 108 generates a sufficient number of optical measurements before the expiration of the predetermined maximum time, the processor 224 of the remote device optionally initiates the measurement process of block 352 without waiting for the expiration of the timer 226. Although not described in further detail herein, the analyte measurement process analyzes the color and optionally other optical characteristics of the reagent at the measurement site 178 on the test strip to determine the level of the analyte in the fluid sample, such as the level of glucose in a blood sample. In a configuration using a color card 180, the processor 224 of the remote device uses additional optical data from the color card 180 to assist in the analyte measurement process. The remote device 140 causes a display of the analyte level measurement to the user via a display device 146, the HUD 116 of the wearable electronic device 104, or another output device.

[0063] System 100 and process 300 improve the reliability of the analyte measurement process because the user can more easily achieve optimal positioning of the test strip or color card within the field of view of camera 108 because the user can manually place the test strip or color card into the wearable electronics 104 rather than awkwardly manipulating remote device 140 to acquire an image of the test strip or color card setting on the surface. Reliability of the process in other embodiments is also improved because the optical measurements are generated during a predetermined window of time to ensure that the reagents in the test strip 170 have sufficient time for chemical reactions to complete before the generation of the optical measurements, but do not suffer drying or fading before the optical measurement process is completed.

[0064] In process 300, if the predetermined maximum time expires before a sufficient number of optical measurements are generated (block 348), the processor 224 of the remote device does not continue the test substance measurement process and the remote device 140 generates an output message via the display device 146, the HUD 116 of the wearable electronic device 104, or another output device indicating that the test substance measurement cannot be completed and instructing the user to begin the test substance testing process again with a new test strip (block 356).

[0065] As described above, the process 300 performs object identification starting with the identification of the test strip vial 160 and the opening of the lid 164 in the video stream, which allows the system 100 to verify that the test strip 170 has been removed from the vial 160, rather than a deceptive test strip that may have been outside the vial 160 for an extended period of time. Some test strips may become contaminated if left outside the vial for an extended period of time. However, in a simplified configuration of the process 300, the system 100 omits the identification of the test strip vial 160, the lid 164, and the extraction of the test strip 170 from the vial 160. The simplified configuration begins with the generation of the video stream and the identification of the test strip 170. In this configuration, the process 300 does not verify that the test strip 170 has been removed from the vial, which may not be necessary for some analyte testing systems. This simplified configuration of the process 300 is otherwise identical to the process described above.

[0066] Those skilled in the art will appreciate that the tracking and identification steps of the process 300 are not necessary for the effective operation of the system 100 to perform a test operation. The programming and instructions provided by the remote device 140 via the display device 146, HUD 116, or other output device for a test operation may contain sufficient information for a user to manually perform the entire operation, rather than requiring the processor 204, 224 of the wearable electronics 104 or remote device 140 to track and identify specific actions related to the test operation. For example, initiation of a test operation may be completed manually by a user via the wearable electronics 108 or remote device 140, after which the user may complete steps, such as steps 312, 316, 332, and 340, without generating a video stream of the action. Instead, execution of the program for performing a test operation may be configured such that the user manually indicates completion of key steps, such as application to a test strip (which may initiate the timer 226), and manually initiates other system-based actions, such as capturing images using the camera 108.

[0067] When the analyte testing operation is performed automatically as in process 300, or manually by a user who primarily uses the wearable electronics 104 of the system 100 to initiate and complete the actions of the analyte testing operation, the user is freed from handling and using a handheld remote device 140 to capture images of the test strip 170' after application for optical measurement. In this way, the user can focus on handling the test strip and optional color card, obtaining a source of bodily fluid sample, applying to the deposition site, positioning the test strip with the optional color reference, and capturing images. The advantage to the user is that it is easier for the user to hold the test strip (or color card with the test strip placed on it) and properly and optimally position the test strip / color reference at a distance and angle in front of the camera 108 so that an accurate image can be captured, as compared to maneuvering the remote device 140 into the proper position above the test strip / color card placed on a surface. An additional advantage of this approach is that proper positioning of the test strip / color standard for image acquisition results in more stable optical measurements based on the image of the test strip after application, allowing for greater accuracy of analyte measurement.

[0068] Another advantage of using primarily the wearable electronics 104 to complete the operational acts of the measurement process described herein is that the test strip / color card after application is held by the user rather than placed on a surface that may be contaminated by the bodily fluid sample. Similarly, by completing the process using only the wearable electronics 104, if the user's skin comes into contact with the bodily fluid sample applied onto the test strip, the user is spared the need to manually handle the remote device 140 and potentially spread contamination to that device.

[0069] Although the embodiments disclosed herein use separate wearable electronics 104 and remote device 140 for illustrative purposes, one skilled in the art will appreciate that a single electronic device may be configured to perform the operations described herein. In particular, state-of-the-art wearable computing devices typically interface with a remote device for complex operations, but a more capable wearable computing device may implement all of the functions described herein. Alternatively, the remote device 140 may be configured to perform all of the functions described herein using the optical sensor 142 as a camera to generate a video stream and perform other processing as described above. Thus, specific references in the above description to the operation of the wearable electronics processor 204 and the remote device processor 224 should be understood to also refer to the operation of the single processor in an alternative configuration using a single electronic device. Notwithstanding the above, as is typical in the state of the art medical diagnostic analytical systems, the results of the analyte measurements may be transmitted, uploaded, or otherwise provided to a remote device 140, such as an internet-based or internet-connected data storage and processing location, for purposes of data management, data back-up, data processing using purpose-built software applications, etc., and engagement and interaction with a healthcare provider or healthcare system.

[0070] The present disclosure has been described in connection with what are believed to be the most practical and preferred embodiments. However, these embodiments are presented as examples and are not limited to the disclosed embodiments. Accordingly, those skilled in the art will understand that the present disclosure encompasses all modifications and alternative configurations that fall within the spirit and scope of the present disclosure and are set forth in the following claims.

[0071] Embodiments include systems for measuring an analyte, as well as methods for measuring an analyte.

[0072] In one embodiment, a method is provided for measuring a analyte using a wearable electronics and a remote device operatively connected to each other, the wearable electronics and the remote device having processors that cooperatively execute program instructions for performing an analyte testing operation, the method steps including a user removing a test strip from a vial, generating a bodily fluid sample, and applying the bodily fluid sample to an application site on the test strip; activating a timer in response to confirmation of application of the bodily fluid sample; providing guidance to the user via a heads-up display (HUD) of the wearable device to position the test strip at a predetermined distance and angular orientation relative to a camera of the wearable electronics; operating the camera to generate at least one image of the test strip including at least a portion of a measurement site on the test strip after a minimum time and before a maximum time from activation of the timer; analyzing optical properties of at least a portion of the measurement site in the at least one image to determine a level of the analyte; and displaying the level of the analyte on the HUD.

[0073] In another embodiment, the method includes initiating a second timer in response to removal of the test strip from the vial by the user, the second timer configured to count down a predetermined sample application time during which the user should apply the bodily fluid sample to the application site of the test strip. In one aspect, the method includes displaying a dynamic countdown of the predetermined sample application time on the HUD and / or remote device display to visually indicate to the user the time remaining to apply the bodily fluid sample to the application site of the test strip. In another aspect, the method includes canceling the second timer in response to confirmation of application of the bodily fluid sample or announcing an audible and / or visual alarm in response to failure to confirm application of the bodily fluid sample.

[0074] In a further embodiment, the method includes using the wearable electronics or the remote device to initiate program instructions and provide guidance to the user regarding the performance of further steps of the method.

[0075] In yet another embodiment, the method includes generating a video stream using a camera of the wearable electronics, identifying a vial in the video stream based on at least one registration mark located on the vial depicted in the video stream, identifying an opening of the vial in the video stream based on at least one registration mark located on a lid of the vial, and identifying an extraction of a test strip from the vial after identifying the opening of the vial based on at least one registration mark associated with the test strip depicted in the video stream, where execution of the program instructions continues based on one or more of the opening of the vial and the extraction of the test strip.

[0076] In yet another embodiment, the method includes generating a video stream using a camera of the wearable electronic device, optionally identifying one or more components required for a test specimen testing operation including a vial, one or more test strips, a color card, and a generated bodily fluid sample, and tracking a user's activity with respect to the one or more components.

[0077] In yet another embodiment, the method includes identifying a vial in a video stream generated by a camera of the wearable electronic device based on at least one registration mark located on the vial depicted in the video stream, identifying an opening of the vial in the video stream based on at least one registration mark located on a lid of the vial, and identifying an extraction of a test strip from the vial after identifying the opening of the vial. In further embodiments, the at least one registration mark associated with the vial includes an indicator formed on a label of the vial, and / or the at least one registration mark located on the lid of the vial includes a color marking formed on an inner surface of the lid, and / or the registration mark associated with the test strip includes an indicator formed on a surface of the test strip at a predetermined location relative to the application site or measurement site, and / or the registration mark associated with the test strip includes an indicator formed on a surface of a color card that holds the test strip.

[0078] In a further embodiment, the method includes identifying that the back side of the test strip is appearing in the video stream based on the absence of an indicator formed on a front side of the test strip or color card holding the test strip, and generating an output message on the HUD indicating that the test strip should be rotated so that the front side of the test strip or color card having the indicator is revealed.

[0079] In yet a further embodiment, the method includes generating an output message on the HUD informing a user that the measurement of the test substance in the fluid dose cannot be completed in response to optical measurement of the measurement site not occurring after a predetermined minimum time has elapsed and before a predetermined maximum time has elapsed.

[0080] In another embodiment, the method includes displaying the level of the analyte on a display of the remote device.

[0081] In yet another embodiment, the wearable electronics and / or remote device are configured to receive and respond to voice communications from a user, and the method further includes the user providing verbal commands for one or more of: initiating execution of program instructions by the device to perform a analyte testing operation, confirming extraction of the test strip, confirming placement of the test strip on a color card configured to hold the test strip, confirming generation of the bodily fluid sample, confirming application of the bodily fluid sample at an application site on the test strip, generating a video stream, and generating one or more images indicative of analyzable optical properties of the measurement site on the test strip to which the bodily fluid sample was applied.

Claims

1. 1. A system for measuring an analyte, comprising: Wearable and Remote Electronic Devices It is equipped with The wearable electronic device includes: a camera configured to generate a video stream and at least one image having optically measurable characteristics; a first communication transceiver configured to transmit and receive communications with a second communication transceiver of the remote device; A head-up display (HUD) a first memory configured to store program instructions; a first processor operatively connected to the first communications transceiver, the camera, and the first memory, the first processor configured to execute the program instructions; Equipped with The remote device is a second memory configured to store program instructions; a second transceiver configured to transmit and receive communications with the first communications transceiver, and a second processor operatively connected to the second memory; Further preparation, the wearable electronic device and the remote device are operatively connected to one another, and the first processor and the second processor cooperatively execute the program instructions, the program instructions configured to operate the system to perform an analyte test operation on a bodily fluid sample; The test substance testing procedure includes: a user removing a test strip from a vial; Producing a body fluid sample; applying the bodily fluid sample to the attachment site of the test strip; activating a timer in response to confirming application of the bodily fluid sample; generating, by the camera, at least one image of the test strip including at least a portion of a measurement site after a minimum time and before a maximum time from the activation of the timer, the at least one image being acquired after guiding the user via the HUD to position the test strip at a predetermined distance and angular orientation relative to the camera; analyzing optical properties of at least a portion of the measurement site in the image to determine a level of the analyte; displaying the level of the analyte on the HUD; and Including, the system.

2. 10. The system of claim 1, wherein the program instructions proceed with the execution of the test material testing operation based on one or more inputs from the user upon completion of at least one of the steps of removing the test strip, generating the bodily fluid sample, applying the bodily fluid sample to the deposition site on the test strip, and generating at least one image.

3. 2. The system of claim 1, wherein the program instructions proceed with the execution of the test substance testing operation based on automatic tracking of user activity and automatic identification of components required for the test substance testing operation, and at least one of the first processor and the second processor is configured to perform the automatic tracking and the automatic identification via a video stream generated by the camera.

4. At least one of the first processor and the second processor identifying a vial in the video stream produced by the camera based on at least one registration mark located on the vial depicted in the video stream; identifying the opening of the vial in the video stream based on at least one registration mark located on the lid of the vial; identifying an extraction of the test strip from the vial after the identification of the opening of the vial in the video stream based on at least one registration mark associated with the test strip depicted in the video stream. The system of claim 3 further configured to:

5. The system of claim 4 , wherein the at least one registration mark associated with the vial further comprises an indicator formed on a label of the vial.

6. 5. The system of claim 4, wherein the at least one registration mark located on the lid of the vial further comprises a color marking formed on an inner surface of the lid.

7. The system of claim 4 , wherein the at least one registration mark associated with the test strip further comprises an indicator formed on a surface of the test strip at a predetermined location relative to the deposition site.

8. At least one of the first processor and the second processor identifying that the back surface of the test strip is present in the video stream based on the absence of the indicator formed on the front surface of the test strip; generating an output message on the HUD indicating that the test specimen should be rotated to expose the surface of the test specimen having the indicator. The system of claim 7 further configured to:

9. The system of claim 4 , wherein the at least one registration mark associated with the test strip further comprises an indicator formed on a back surface of a color card that holds the test strip.

10. At least one of the first processor and the second processor Identifying a user's finger in the video stream; Identifying contact between the finger and the attachment site in the video stream; and identifying application of a dose in response to a change in an optical property of the deposition site in the video stream after the contact between the finger and the deposition site. The system of claim 4 further configured to:

11. 5. The system of claim 4, wherein at least one of the first processor and the second processor is further configured to identify application of a dosage in response to a change in an optical property of the deposition site in the video stream.

12. 5. The system of claim 4, wherein at least one of the first processor and the second processor is further configured to generate an output message on the HUD informing the user that the measurement of the test substance in the fluid dose cannot be completed in response to optical measurement of the measurement site not occurring after the predetermined minimum time has elapsed and before the predetermined maximum time has elapsed.

13. 13. The system of claim 1, wherein the remote device further comprises a display device operatively connected to the second processor, and the analyte testing operation further comprises displaying the level of the analyte on the display device.

14. The system of claim 13 , wherein the remote device comprises a mobile electronic device.

15. 15. The system of claim 14, wherein the mobile electronic device comprises a smartphone, and the program instructions comprise a software application stored in the second memory and executed by the second processor in communication with the first processor to instruct the wearable electronic device in the performance of the test material testing operation.

Images