How to perform analytical measurements
Patent Information
- Application Number
- JP2022528314
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2019-11-26
- Filing Date
- 2020-11-25
- Publication Date
- 2025-04-14
- Estimated Expiration
- 2040-11-25
AI Technical Summary
Existing methods for analytical measurements using mobile devices face challenges in ensuring reliability and accuracy due to variations in environmental conditions and positioning, particularly when capturing multiple images of optical test strips before and after sample application.
A method involving a mobile device with a camera that captures a first image of an optical test strip without a sample, applies a sample, captures a second image, and determines an analytical measurement result by aligning both images with similar local positions relative to environmental features using augmented reality guidance.
This approach enhances measurement reliability and accuracy by ensuring consistent imaging conditions, reducing variability caused by environmental factors and device positioning.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[Technical Field]
[0001] The present application relates to a method for performing analytical measurements based on a color-forming reaction in an optical test strip by using a mobile device with a camera. The invention further relates to a computer program and a computer-readable storage medium having program means for performing the method according to the invention. Furthermore, the invention relates to a mobile device and a kit for performing analytical measurements. The method, computer program, mobile device and kit according to the invention can be used in medical diagnostics for qualitatively or quantitatively detecting one or more analytes in one or more body fluids. However, other fields of application of the invention are possible. [Background technology]
[0002] In the field of medical diagnostics, it is often necessary to detect one or more analytes from a sample of bodily fluid, such as blood, interstitial fluid, urine, saliva, or other types of bodily fluid. Examples of analytes to be detected are glucose, triglycerides, lactate, cholesterol, or other types of analytes typically present in these bodily fluids. Depending on the concentration and / or presence of the analyte, appropriate processing can be selected as needed. Without narrowing the scope, the present invention can be specifically described with respect to blood glucose measurement. However, it should be noted that the present invention can also be used for other types of analytical measurements using test elements.
[0003] Generally, devices and methods known to those skilled in the art utilize test elements containing one or more test chemicals that can undergo one or more detectable detection reactions, such as optically detectable detection reactions, in the presence of an analyte to be detected. For example, European Patent Application Publication No. 0 821 234 describes a diagnostic test carrier for determining an analyte from whole blood using a reagent system contained in the carrier, and a method for determining an analyte from whole blood using the diagnostic test carrier. The diagnostic test carrier contains a color-forming reagent. The test field has a sample application side to which a blood sample is delivered and a detection side where an optically detectable change occurs as a result of a reaction between the analyte and the reagent system. Furthermore, the test field is designed to prevent red blood cells contained in the sample from reaching the detection side. Furthermore, the test field includes a transparent film and first and second overlapping film layers applied to the transparent film, with the first layer on the transparent film scattering light in a wet state substantially less than the second layer above it.
[0004] Regarding the test chemicals contained in the test elements, reference can be made, for example, to J. Hoenes et al.: The Technology Behind Glucose Meters: Test Strips, Diabetes Technology & Therapeutics, Volume 10, Supplement 1, 2008, S-10 to S-26. Other types of test chemicals are possible and can be used to practice the present invention.
[0005] In analytical measurements, particularly those based on color-forming reactions, one technical challenge lies in evaluating the color change resulting from the detection reaction. In addition to using dedicated analytical devices such as handheld blood glucose meters, the use of commonly available electronic devices such as smartphones and portable computers has become increasingly common in recent years. International Publication No. 2012 / 131386 discloses a test device for performing an assay, comprising a container containing a reagent that reacts to an applied test sample by exhibiting a color or pattern change, and a portable device, such as a mobile phone or laptop, equipped with a processor and an image capture device, the processor configured to process data captured by the image capture device and output a test result for the applied test sample. Furthermore, software applications for use with smartphones are available for download, such as the ACCU-CHEK® SugarView App by Roche Diabetes Care GmbH, Germany, available at https: / / www.accu-chek-sugarview.com.
[0006] In contrast to laboratory measurements and measurements performed using dedicated analytical measurement equipment, various influences must be taken into account when using mobile computing devices such as smartphones. For example, lighting conditions, positioning, vibrations, or other more or less uncontrollable conditions must be taken into account. In the field of mobile computing device technology, various technological approaches have been developed in recent years to improve image recognition and, for example, to obtain additional information about unknown geometric parameters of the setup.
[0007] Thus, by way of example, U.S. Pat. No. 9,691,152 discloses an approach for minimizing variations in the height of a computing device's camera when estimating the distance to an object represented in image data captured by the camera. For example, a front-facing camera of the computing device can be used to capture a live camera view of a user. An application can analyze the image data to locate the user's facial features for the purpose of aligning the user with the computing device. As the position and / or orientation of the device changes relative to the user, the image data can be analyzed to detect whether the location of a representation of the user's features aligns with an alignment element. Once the features align with the alignment element, a rear-facing camera or other camera can capture second image data of the object. The second image data can be analyzed to determine a geometric relationship between the rear-facing camera and the object, which can be used to determine the distance to the object relative to the computing device.
[0008] Furthermore, U.S. Patent No. 9,934,612 discloses a method for determining a camera pose relative to an object in a real environment for use in authoring / augmented reality applications, the method including: generating a first image by a camera capturing a real object in the real environment; generating first orientation data from at least one orientation sensor associated with the camera or from an algorithm that analyzes the first image to find and determine features indicative of the camera's orientation; assigning a distance to the real object of the camera; generating distance data indicative of the assigned distance; and determining the camera pose relative to a coordinate system associated with the real object in the real environment using the distance data and the first orientation data. The method can be performed with reduced processing requirements and / or increased processing speed on a mobile device such as a mobile phone having a display, a camera, and an orientation sensor.
[0009] U.S. Patent No. 9,667,860 discloses a method, computer-readable medium, and apparatus for providing composition and position guidance for a camera. One method includes detecting an image received on a capture device. The image is analyzed to identify objects and determine object attributes. Based on the analysis of the image, suggestions are provided for adjusting at least one of the position or settings of the capture device according to rules. Adjustments of the capture device based on the suggestions are detected and used to receive an adjusted image. The adjusted image is captured by the capture device.
[0010] Furthermore, U.S. Patent No. 20110254860 discloses a mobile device including a visual input device for capturing external visual information with real visual background information and a processing device. The processing device is for associating a selected application with the external visual information and executing the selected application based on the external visual information and user-related input information. The processing device for generating a visual output signal associated with at least one virtual visual object in response to the application is further configured to provide the visual output signal to a projector device included within the mobile device, such that the projector device is configured to project the visual output signal associated with the at least one virtual visual object onto a visual background, thereby modifying the external visual information.
[0011] International Publication No. WO2013103912 discloses a transactional visual capture device, method, and system ("TVC") that converts mobile device location coordinate information transmission, real-time reality visual capture, and mixed gesture capture via a TVC component into real-time behavior-sensitive product purchase-related information, shopping purchase transaction notifications, and electronic receipts. In one implementation, the TVC acquires user check-in information from the user's mobile device when the user enters a store. The TVC extracts a user identifier based on the user check-in information and accesses a database of user profiles. The TVC determines the user's prior behavioral patterns from the accessed user profile and acquires the user's real-time in-store behavior data from the user's mobile device.
[0012] U.S. Patent Application Publication No. 2018 / 024049 describes a method and a colorimetric device for performing colorimetric analysis of a test fluid to evaluate associated physiological parameters. Images of test strips at different heights are captured by a calorimetric device, and multiple geometric parameters associated with the test strips are determined based on an analysis of the captured images. Image resizing factors are determined based on the multiple geometric parameters, and resized images are generated based on the image resizing factors. Upon generating the resized images, calorimetric measurements associated with the resized images are determined based on which physiological parameters associated with the test fluid are evaluated.
[0013] EP 1 801 568 A1 describes a method that involves placing a camera on a test strip to graphically detect a color indicator and a reference color area. A measurement of the relative position between the camera and the test strip is determined and compared to a desired value area. During a deflection between the measured value and the desired value, the camera is moved to reduce the deflection relative to the strip. The image area assigned to the indicator is localized in the color image detected by the camera. The concentration of the analyte in the sample is determined by the comparison value.
[0014] Despite the advantages associated with using mobile computing devices to perform analytical measurements, several technical challenges remain. Specifically, the reliability and accuracy of measurements need to be improved and guaranteed. A key challenge is the presence and influence of various environmental conditions, such as lighting conditions. Therefore, measurement results can be strongly dependent on the environment and / or background lighting and, therefore, may vary from measurement to measurement, even under identical chemical or biochemical conditions. Furthermore, measurement results can depend on the relative positioning of the lighting device and the mobile device's camera, which can vary for different types or models of mobile devices due to the vast number of different mobile devices available on the market. These technical challenges are also accentuated by the fact that, typically, when performing analytical measurements using optical test strips, at least two images must be captured and analyzed: one image shows at least a portion of the test field where no sample has been applied, while at least one second image is acquired where sample has been applied, and the application of the second image typically occurs after waiting a certain period of time for the color-forming reaction to occur. In this case, at least two images must be compared, and the test strip is typically processed and repositioned between capturing these two images, further increasing the measurement uncertainty.
[0015] Issues to be resolved Therefore, it is desirable to provide a method and apparatus that addresses the above-mentioned technical challenges of analytical measurements using mobile devices such as mobile devices for consumer electronics, particularly multi-purpose mobile devices that are not dedicated to analytical measurements, such as smartphones and tablet computers. Specifically, it is necessary to propose a method and apparatus that ensures the reliability and accuracy of measurements. Summary of the Invention
[0016] This problem is addressed by a method, a computer program, a computer-readable storage medium, and an apparatus having the features of the independent claims. Advantageous embodiments, which can be realized alone or in any combination, are set forth in the dependent claims.
[0017] When used below, the terms "have," "comprise," or "include," or any grammatical variations thereof, are used in a non-exclusive manner. Thus, these terms may refer both to the situation in which, in addition to the features introduced by these terms, no further features are present in the entity described in this context, and to the situation in which one or more additional features are present. For example, the expressions "A has B," "A comprises B," and "A includes B" may both refer to the situation in which, apart from B, no other elements are present in A (i.e., the situation in which A consists solely and exclusively of B), and to the situation in which, apart from B, one or more further elements are present in entity A, such as element C, elements C and D, and even further elements.
[0018] Furthermore, it should be noted that the terms "at least one," "one or more," or similar expressions indicating that a feature or element may be present one or more times are typically used only once when introducing each feature or element. In the following, in most cases, when referring to each feature or element, the expressions "at least one" or "one or more" will not be repeated, despite the fact that each feature or element may be present one or more times.
[0019] Furthermore, when used hereinafter, the terms "preferably," "more preferably," "particularly," "more particularly," "particularly," "more particularly," or similar terms are used in conjunction with any feature without limiting the possibility of substitution. Therefore, features introduced by these terms are optional features and are not intended to limit the scope of the claims in any way. The present invention may also be implemented by using alternative features, as recognized by those skilled in the art. Similarly, features introduced by "in an embodiment of the present invention" or similar expressions are intended to be optional features without limitations on alternative embodiments of the invention, without limitations on the scope of the invention, and without limitations on the possibility of combining the feature introduced in such a way with other optional or non-optional features of the invention.
[0020] In a first aspect, a method for performing an analytical measurement based on a color-forming reaction in an optical test strip by using a mobile device with a camera is disclosed. The method includes, by way of example, the following steps, which may be performed in a given order. However, it should be noted that a different order is also possible. Furthermore, one or more of the method steps may be performed only once or may be repeated. Furthermore, two or more of the method steps may be performed simultaneously or with overlapping timing. The method may include additional method steps not described.
[0021] Generally, the method comprises the following steps: a) providing an optical test strip having a test field without applying a sample to the test field; b) capturing at least one first image of at least a portion of a test field of the optical test strip using a camera without applying a sample to the test field; c) applying a sample of the bodily fluid to a test field of the optical test strip; d) capturing, using a camera, at least one second image of at least a portion of the test field of the optical test strip having the sample of bodily fluid applied thereto; e) determining an analytical measurement result value by using the first and second images of the optical test field of the optical test strip; Including, f) Providing, by the mobile device, an indication of where to capture the second image such that the first image and the second image are captured at similar local locations with respect to at least one environmental feature.
[0022] Therein, in particular, method step f) may be performed before or during the execution of method step d), and the remaining method steps may in particular be performed in a given order, although, as outlined above, different orders are also possible, in particular for method steps a) to e).
[0023] The term "analytical measurement" as used herein is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to any special or customized meaning. This term can specifically, but is not limited to, refer to the quantitative and / or qualitative determination of at least one analyte in any sample. Samples include bodily fluids such as blood, interstitial fluid, urine, saliva, or other types of bodily fluids. By way of example, the result of the analytical measurement can be the concentration of the analyte and / or the presence or absence of the analyte to be determined. Specifically, by way of example, the analytical measurement can be a blood glucose measurement, and thus the result of the analytical measurement can be, for example, a blood glucose concentration. In particular, an analytical measurement value can be determined by the analytical measurement. The term "analytical measurement value" as used herein is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to any special or customized meaning. This term can specifically, but is not limited to, refer to a numerical representation of the analyte concentration in a sample.
[0024] For example, the at least one analyte can be or include one or more specific chemical compounds and / or other parameters. For example, one or more analytes involved in metabolism, such as blood glucose, can be determined. Additionally or alternatively, other types of analytes or parameters can be determined, such as pH values. The at least one sample can specifically be or include at least one bodily fluid, such as blood, interstitial fluid, urine, saliva, etc. However, additionally or alternatively, other types of samples, such as water, can also be used.
[0025] The analytical measurement may specifically be an analytical measurement involving a change in at least one optical property of the optical test strip, which may be visually measured or determined using a camera. Specifically, the analytical measurement may be or may include a color-forming reaction in the presence of at least one analyte to be determined. The term "color-forming reaction" as used herein is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. Specifically, the term may refer to, but is not limited to, a chemical, biological, or physical reaction in which the color, specifically the reflectance, of at least one component involved in the reaction changes as the reaction progresses.
[0026] The term "optical test strip" as used herein is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to any special or customized meaning. This term can specifically, but not exclusively, refer to any element or device including at least one strip-shaped carrier having at least one test strip applied thereto or incorporated therein, the element configured to perform a color change detection reaction. Optical test strips may also be referred to as test strips or test elements. Optical test strips can have, among other things, a test field containing at least one test chemical for detecting at least one analyte. For example, an optical test strip can include at least one substrate, such as at least one carrier, having at least one test field applied thereto or incorporated therein. In particular, an optical test strip can further include at least one white region, such as a white field, particularly adjacent to the test field, e.g., surrounding or encircling the test field. The white region can be a separate field independently disposed on the substrate or carrier. Additionally or alternatively, however, the substrate or carrier itself can be or include a white region. These test strips are widely available and in common use. A test strip can carry a single test field or multiple test fields with the same or different test chemicals contained therein.
[0027] As further used herein, the term "test field" is a broad term and should be given its ordinary and customary meaning to one of ordinary skill in the art and should not be limited to any special or customized meaning. This term can specifically refer to, but is not limited to, a coherent amount of test chemical relative to a field, such as a circular, polygonal, or rectangular shaped field, having one or more layers of material, with at least one layer of the test field containing the test chemical.
[0028] The term "mobile device," as used herein, is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to any special or customized meaning. The term may specifically refer to, but is not limited to, a portable electronic device, more specifically a portable communication device such as a mobile phone or smartphone. Additionally or alternatively, as outlined in more detail below, a mobile device may also refer to a tablet computer or another type of portable computer having at least one camera.
[0029] The term "camera," as used herein, is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to any special or customized meaning. The term may specifically, but is not limited to, refer to a device having at least one imaging element configured to record or capture spatially resolved one-, two-, or three-dimensional optical data or information. By way of example, a camera may include at least one camera chip, such as at least one CCD chip and / or at least one CMOS chip, configured to record an image. As used herein, but is not limited to, the term "image" may specifically relate to data recorded by using a camera, such as multiple electronic readings from an imaging element, such as pixels of a camera chip.
[0030] In addition to at least one camera chip or imaging chip, the camera may include one or more optical elements, such as one or more lenses. For example, the camera may be a fixed-focus camera with at least one lens that is fixedly adjusted relative to the camera. Alternatively, however, the camera may also include one or more variable lenses that can be adjusted automatically or manually. The present invention should be particularly applicable to cameras typically used in mobile applications, such as notebook computers, tablets, or mobile phones, particularly smartphones. Thus, in particular, the camera may be part of a mobile device that includes, in addition to at least one camera, one or more data processing devices, such as one or more data processors. However, other cameras may also be used.
[0031] The camera may be a color camera. Therefore, color information such as three color values of R, G, and B may be provided or generated for each pixel. More color values are also possible, such as four color values for each pixel, such as R, G, G, and B. Color cameras are generally known to those skilled in the art. Thus, for example, a camera chip may each comprise three or more different color sensors, such as color recording pixels, one pixel for red (R), one pixel for green (G), and one pixel for blue (B). For each pixel, such as R, G, and B, a value may be recorded by the pixel, such as a digital value ranging from 0 to 255, depending on the intensity of each color. For example, instead of using a triple color such as R, G, and B, a quadruple such as R, G, G, and B may be used. The color sensitivity of a pixel may be generated by a color filter or by the appropriate inherent sensitivity of the sensor element used in the camera pixel. These techniques are generally known to those skilled in the art.
[0032] Steps b) and d) each involve capturing at least one image using a camera. The term "capturing at least one image" can refer to one or more of imaging, image recording, image acquisition, and image capture. The term "capturing at least one image" can include capturing a single image and / or multiple images, such as a series of images. For example, capturing images can include continuously recording a series of images, such as a video or motion picture. The capture of the at least one image can be initiated by a user action or can be initiated automatically, for example, upon automatic detection of the presence of at least one object within the field of view of the camera and / or within a predetermined sector of the field of view. These automatic image capture techniques are known, for example, in the field of automatic barcode readers, such as automated barcode reading apps. The image capture can be performed, for example, by capturing a stream or "lifestream" of images with a camera, and one or more of the images can be stored and used as at least one first image or at least one second image, respectively, automatically or through user interaction, such as pressing a button. Image capture can be supported by a processor in the mobile device, and image storage can occur in a data storage device of the mobile device.
[0033] In each of steps b) and d), at least one image of at least a portion of the test field is captured using a camera. These images are referred to as "at least one first image" and "at least one second image," and the terms "first" and "second" are used for naming purposes only, without ranking or numbering these images or imparting any preference. The terms "at least a portion of the test field" or "at least a portion of the test field" both refer to the fact that at least a portion of at least one test field should be visible in each image; different portions of the at least one test field may be visible in the first and second images. In addition to at least one portion of the test field, in each case, further portions of the optical test strip may be visible, such as at least a portion of the test strip substrate.
[0034] In step b), at least one first image is captured without applying a sample to the test field. This at least one first image is typically also referred to as a "blank image." In typical evaluation methods, the image is used for reference purposes to account for variations in the color or other optical properties of the test field that are not due to the sample or analyte itself. The sample application in step c) can be performed directly or indirectly, for example, via at least one capillary element. The at least one second image captured after sample application is typically also referred to as a "wet image," even if the sample may be dry when the image is actually captured. The second image is typically captured after waiting at least a predetermined waiting time, for example, 5 seconds or more, to allow the detection reaction to occur.
[0035] Thus, by way of example, the method may include waiting at least a predetermined minimum amount of time between steps c) and d). This predetermined minimum amount of time may be specifically sufficient for a detection reaction to occur within the test strip. By way of example, the minimum amount of waiting time may be at least 5 seconds.
[0036] In step e), an analytical measurement result value is determined using the first and second images of the optical test field of the optical test strip. The analytical measurement result value can be, for example, a numerical indicator of the result of the analytical measurement, such as indicating the concentration of at least one analyte in the sample, such as a blood glucose concentration. To determine the analytical measurement result value from the at least one first image and the at least one second image, for example, a predetermined or determinable relationship between information derived from the first and second images, such as color information or color change information, and the at least one analytical measurement result value can be used. This predetermined or determinable relationship can, for example, be stored in a data storage device of the mobile device and / or a processor of the mobile device. For example, the processor can be configured by software programming to derive at least one item of information, such as at least one color coordinate, from the first and second images and apply the predetermined or determinable relationship to the at least one item of information. For example, the correlation, which can be a conversion function, a conversion table, or a look-up table, can be determined empirically and stored in at least one data storage device of the mobile device by software, for example, an app downloaded from an app store. As an example of deriving the information for the at least one item, the processor may be programmed to automatically recognize the test field or at least a portion of the test field in the image, for example, by pattern recognition and / or other algorithms. Consequently, the processor may be programmed to determine the information for the at least one item, such as one or more color coordinates. The information for each at least one item derived from the blank image may be used for normalization, for example, by dividing the information for the at least one item derived from the wet image by the information for the at least one item derived from the corresponding blank image, or by subtracting the information for the at least one item derived from the wet image from the information for the at least one item derived from the blank image, or vice versa. Other normalization methods are also possible.The correlation, which may be, for example, a conversion function, a conversion table, or a look-up table, may be determined, for example, empirically, and may be stored in at least one data store of the mobile device, for example, by software, in particular an app downloaded from an app store or the like.
[0037] The method may further include displaying the analytical measurement value on a display, such as the mobile device. One or more numerical values indicative of the analytical measurement value may be displayed, for example, on the display. Additionally or alternatively, a range indication for the analytical measurement value may be provided, such as by indicating whether the analytical measurement value falls within one or more predetermined ranges. For example, a range indication, such as a high range, a target range, or a low range, may be provided without necessarily displaying the numerical value of the analytical measurement value. Thus, the display need not necessarily be a numerical indication. Other means for displaying the analytical measurement value are feasible, and the display may be one or more of a visual display, an audible display, or a tactile display. Additionally or alternatively, the method may include storing the analytical measurement value in at least one data storage device of the mobile device. Additionally and alternatively, the method may further include transmitting the analytical measurement value to another computer, for example, via at least one interface and / or via at least one data transmission network, for further evaluation.
[0038] As described above, the method is performed by the mobile device to provide instructions of where to capture the second image such that the first and second images are captured at similar local locations with respect to at least one environmental feature.
[0039] Therein, the term "environmental feature" may generally refer to any reference element and / or reference characteristic within the field of view of the camera, as described in more detail below. The environmental feature may specifically be suitable for defining a position and / or a coordinate system in space and / or may be used as a position marker within the field of view of the camera. The environmental feature may specifically be a feature having a fixed and / or unchanging position between capturing the first and second images. The fixed and / or unchanging position of the environmental feature may particularly refer to a fixed and / or unchanging absolute position in space. The environmental feature may specifically be a feature whose position is likely to remain unchanged between capturing the first and second images. The environmental feature may be or include an article or part thereof, such as a table on which a test strip is placed during measurement, or a part of a table, such as the surface structure of the table. The at least one environmental feature may specifically include at least one of an article within the field of view of the camera or a structural feature of an article within the field of view of the camera. The environmental feature or the article may be tangible. The environmental characteristics can be different from the test strip or part of the test strip and the mobile device having the camera or part of the camera.
[0040] To detect the at least one environmental feature, the method may include detecting the at least one environmental feature in one or both of the first and second images. To this end, by way of example, the method may utilize image recognition, such as software-based automatic image recognition and / or image recognition by a machine learning process.
[0041] Thus, the sub-feature of feature f) indicating that the first and second images were captured at similar local locations relative to at least one environmental feature may generally refer to the fact that the at least one environmental feature provides a coordinate system that defines the first and / or second local locations. Thus, the local location of the capture of the first image may be detected and / or monitored with respect to the at least one environmental feature, or the local location of the capture of the second image may be detected and / or monitored with respect to the at least one environmental feature, or both. Further exemplary embodiments are described in detail below.
[0042] Providing instructions is generally indicated by method step f), which can specifically be performed before or during the execution of method step d). Indeed, method step f) can be part of method step d), such as by capturing or recording a life image or image stream by a camera of the mobile device and displaying this image stream or life image on a display of the mobile device, and providing instructions by the mobile device of where to take the second image, such that the first and second images are taken at similar local positions with respect to at least one environmental feature. By way of example, augmented reality can be used, such as by adding instructions regarding where to take the second image to the image stream or life image displayed on the display, as will be outlined in more detail below.
[0043] As outlined in more detail below, the term "local location," as used herein, particularly in the context of capturing the first and second images, hereinafter also referred to simply as "location," is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to any special or customized meaning. This term may specifically refer to, but is not limited to, at least one item of spatial information relating to one or more of the camera or test field at the time of image capture, such as at and / or during image capture, where the at least one spatial information takes into account at least one environmental feature, such as by determining the location of the at least one environmental feature within the camera's field of view. The at least one item of spatial information may refer to at least one of a spatial coordinate and / or a spatial direction, such as at least one spatial coordinate and / or at least one spatial direction in at least one coordinate system defined by the at least one environmental feature. The at least one item of spatial information may refer to an absolute item of spatial information or at least one relative item of spatial information, such as a relative item of spatial information relating to the spatial relationship between the camera and the test field. Thus, by way of example, absolute items of spatial information may be determined relative to the camera or the test field within a coordinate system defined by at least one environmental feature, and relative items of spatial information may be determined relative to the other of the camera or the test field, or absolute items of spatial information may be determined relative to both the camera and the test field within a coordinate system defined by at least one environmental feature.
[0044] By way of example, the local position may include at least one of: an absolute position of the camera in space relative to at least one environmental feature; an absolute position of the test field in space relative to at least one environmental feature; a relative position between the camera and the test field in a coordinate system defined by at least one environmental feature in the field of view of the camera; an orientation of the camera in space relative to at least one environmental feature; an orientation of the test field in space relative to at least one environmental feature; a relative angular orientation between the camera and the test field in a coordinate system defined by at least one environmental feature in the field of view of the camera; a relative angular orientation between the camera and at least one environmental feature in the field of view of the camera; a relative angular orientation between the test field and at least one environmental feature in the field of view of the camera; and a relative angular orientation between the camera and the test field in a coordinate system defined by at least one environmental feature in the field of view of the camera.
[0045] As outlined in further exemplary detail below, the term "similar" as used herein, and particularly in the context of the local locations of first and second images, is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to any special or customized meaning. Specifically, the term may refer, without limitation, to the fact that the first and second images are captured at local locations that satisfy at least one predetermined or determinable similarity criterion. Thus, by way of example, the local locations of the first and second images may be identical within at least a predetermined tolerance, such as a predetermined tolerance stored in at least one data storage device of the mobile device.
[0046] Thus, step f), i.e., providing instructions for where to capture the second image such that the first and second images are captured at similar local locations with respect to at least one environmental feature, may comprise providing at least one similarity criterion with respect to the local locations when capturing the first and second images. By way of example, the similarity criterion may be provided by a data storage of the mobile device having stored therein at least one similarity criterion.
[0047] The method may further include evaluating whether a similarity criterion is met. Thus, by way of example, the processor may be configured to evaluate local locations at which the first and second images were taken, where the location of the second image may be a current location in the process of capturing the second image, such as by taking an image sequence or lifestream of images, and the processor may be configured to determine whether at least one similarity criterion is met.
[0048] The at least one similarity criterion may specifically include at least one of an illumination similarity criterion and a local similarity criterion. Thus, by way of example, positional similarity may refer to illumination-related similarity and / or local similarity-related similarity.
[0049] Evaluating whether the at least one similarity criterion is satisfied may specifically include comparing local locations when capturing the first and second images, specifically comparing a deviation between the local locations to at least one tolerance threshold. Thus, by way of example, the processor may be configured to determine a deviation between the second local location and the first local location, and may be configured to compare the deviation to at least one tolerance threshold, such as at least one tolerance threshold stored in at least one data store of the mobile device.
[0050] The method can include determining a first local location when capturing a first image and determining at least one second local location before or when capturing a second image, wherein one or both of the first and second local locations can be determined relative to at least one environmental feature, such as in at least one coordinate system defined by the at least one environmental feature, as outlined above. The method can include comparing the first and second local locations to determine whether the first and second images were captured at similar local locations relative to the at least one environmental feature. Thus, as outlined above, the processor can be configured to determine the first and second local locations and compare the first and second local locations and / or their deviation to at least one tolerance threshold to determine whether a similarity criterion is met.
[0051] The instructions provided in step f), i.e., instructions provided by the mobile device of where to capture the second image so that the first and second images are captured at similar local positions relative to at least one environmental feature, can specifically include visual instructions, more specifically, visual instructions on the display of the mobile device. The instructions may specifically be provided at least partially as augmented reality on the display of the mobile device. The term "augmented reality," as used herein, is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. The term may specifically, but is not limited to, refer to a method of overlaying a current image, a life image, or an image stream of a scene on a display with one or more additional information, such as one or more visual instructions. Thus, by way of example, the augmented reality in step f) can be provided by providing one or more arrows, frames, or lines on the display indicating preferred positioning of the camera and test strip. Additionally or alternatively, text may be displayed indicating in which direction the camera and / or test strip should be moved. Other augmented reality methods are possible.
[0052] The method can specifically include determining and optionally storing at least one first item of position information relating to one or both of the camera position and the test field position when the first image is captured. Thus, the position of the camera and / or test strip at at least one time point when the at least one first image is captured can be determined and optionally stored. Various means, described in more detail below, are feasible for determining the position. The at least one first item of local information can specifically include at least one of the absolute position of the test field and the absolute position of the camera. In particular, the at least one first item of position information, such as one or more of the absolute positions listed above, can be determined with respect to at least one environmental feature, such as at least one coordinate system defined by at least one environmental feature.
[0053] Determining the at least one first item of location information may include image analysis of the at least one first image to detect at least one environmental feature in the at least one first image. However, additionally or alternatively, other means of determining the at least one first item of location information may be applied. Thus, by way of example, determining the at least one first item of location information may include using at least one sensor of the mobile device, specifically at least one of a position sensor, a gyro sensor, an optical sensor, and a tilt sensor. Thus, one or more sensors of the mobile device itself may be used to determine the at least one first item of location information.
[0054] As outlined above, visual means, such as image recognition means, may be used to determine the at least one first item of location information. Thus, determining the at least one first item of location information may involve detecting at least one environmental feature of the local environment instead of, or in addition to, using at least one sensor of the mobile device.
[0055] The at least one first item of position information may include position information of one or both of the camera and the test field relative to at least one environmental feature. Thus, by way of example, the at least one environmental feature may be used by the processor to define a coordinate system in space, and the position information of the camera and / or the test field may be defined in this coordinate system. Additionally or alternatively, the at least one first item of position information may simply refer to a spatial distance between one or more of the camera and the test field and the environmental feature.
[0056] The detection of the at least one environmental feature may specifically include image recognition in at least one image provided by the camera, more specifically, a still image at the time the at least one first image was captured, and / or a life image and / or an image stream. Thus, by way of example, the first image itself may be used and may undergo image recognition to thereby recognize the at least one environmental feature. Thus, by way of example, the processor may be configured to perform a histogram analysis of an image, such as the first image, to determine salient environmental features that may be used as spatial criteria.
[0057] The instructions provided in step f), i.e., instructions provided by the mobile device of where to capture the second image so that the first and second images are captured at similar local positions with respect to at least one environmental feature, may be provided specifically based on the first item of location information. Thus, the mobile device may be configured to record the position of the camera and / or the test field when capturing the first image, specifically with respect to the at least one environmental feature, to instruct the user where to capture the second image. The instructions may include visual instructions, specifically visual instructions on a display of the mobile device, more specifically visual instructions at least partially using augmented reality on a display of the mobile device.
[0058] To be able to provide the instructions in step f), the method can include determining, in particular live determining, and optionally storing in a data storage device on the mobile device, at least one second item of location information relating to one or both of the camera position and the test field position, and comparing the second item of location information with the first item of location information. The instructions in step f), i.e., providing instructions for where to capture the second image such that the first and second images are captured at similar local positions with respect to at least one environmental feature, can include providing instructions such that the second item of location information when capturing the second image is identical to the first item of location information, at least within a predetermined tolerance.
[0059] Similar to the at least one first item of location information, the at least one second item of local information may include at least one of the absolute position of the test field, the absolute position of the camera, and the relative position between the camera and the test field, specifically at the time the second image was captured and / or before the second image was captured. Thus, the second item of location information may specifically be determined as an information stream prior to capturing the second image to provide continuous or repeated instructions so that a user can adjust the position of the test field and / or camera accordingly prior to capturing the second image. Again, the at least one second item of location information may be determined with respect to at least one environmental feature, such as at least one coordinate system defined by the at least one environmental feature.
[0060] For determining the at least one second item of location information, essentially the same options exist as for determining the at least one first item of location information. Thus, by way of example, at least one internal sensor and / or image recognition means can be used, as described above with respect to the at least one first item of location information. Thus, determining the at least one second item of location information can include image analysis of the at least one second image to detect at least one environmental feature in the at least one second image. However, additionally or alternatively, other means for determining the at least one second item of location information can be applied. Thus, by way of example, determining the at least one second item of location information can include using at least one sensor of the mobile device, specifically at least one of a position sensor, a gyro sensor, an optical sensor, and a tilt sensor. Additionally or alternatively, as outlined above, determining the at least one second item of location information can include detecting at least one environmental feature of the local environment, such as the same environmental feature used to determine the at least one first item of location information. The at least one second item of location information can include location information of one or both of the camera and the test field relative to at least one environmental feature. Detection of the at least one environmental feature can include image recognition in images provided by the camera, more specifically, in life images. Thus, as outlined above, the at least one second item of location information can be continuously and / or repeatedly determined before and / or during capture of the at least one second image to allow a user to adjust the local position of one or both of the camera and the test field. As outlined above, the at least one environmental feature can include at least one of an article in the field of view of the camera or a structural feature of an article in the field of view of the camera. Other options are possible.
[0061] If at least one environmental feature is not within the camera's field of view when preparing to capture at least one second image, the mobile device can provide the user with instructions, such as visual instructions, on how to move and / or reorient the mobile device to bring the at least one environmental feature into view, particularly via augmented reality. Thus, by way of example, when capturing the first image, the mobile device can determine at least one first local location relative to at least one environmental feature visible within the camera's field of view. When the mobile phone detects that the environmental feature is not visible to capture the at least one second image, it can provide the user with instructions indicating where to move and / or how to reorient the mobile device to bring the same at least one environmental feature into view of the camera again. In this case, because the environmental feature is not available for first determining the second local location, one or more internal and / or external sensors can be used to provide the instructions. Thus, by way of example, one or more internal or external sensors can be used to detect whether the mobile device has been moved and / or reoriented after capturing the first image to provide an indication, such as by indicating to the user that the mobile device needs to move back and / or reorient to bring the environmental feature into view. By way of example, once at least one environmental feature is in view, image analysis can be used to determine a second local location for the at least one environmental feature.
[0062] The capture of the at least one second image may be user-initiated and / or automatically initiated. Specifically, the capture may be initiated automatically as soon as the mobile device recognizes that the condition that the second image is captured at a similar local location as the previous first image is met. Thus, step d) may be initiated automatically when at least one of the camera and the test field is in a predetermined position such that the first and second images are taken at similar local locations with respect to at least one environmental feature.
[0063] For capturing at least one first image, essentially two options can be considered. Thus, the first image can essentially be captured at any non-predefined local location. As outlined above, the local location when capturing the first image can be determined and / or stored in a data storage device. For example, the capture of the first image can be initiated by the user as soon as the user places the test strip in a suitable location, such as on a table. As a second option, the local location of one or both of the test field and the camera when capturing the at least one first image can be determined by the mobile device, and instructions can be provided to the user, similar to step f), regarding where to capture the first image. For possible options for these instructions, see the description of step f). Thus, instructions regarding where to capture the first image can be provided by the mobile device before performing step b). For example, the mobile device can determine suitable environmental features in an image, such as an image stream and / or a life image, before capturing the first image, and, if a suitable feature is detected, can suggest a location near the feature to the user by providing corresponding instructions. The instructions include visual instructions, particularly visual instructions on a display of the mobile device, and more particularly visual instructions at least in part through the use of augmented reality on a display of the mobile device.
[0064] The above-mentioned options can also be used as possible alternatives at the user's discretion. Thus, for example, the user can override the suggestions provided by the mobile device and use different local settings to capture the first image.
[0065] The mobile device may be further configured to provide guidance to the user regarding one or more of the treatment steps. Thus, by way of example, instructions for performing one or more actions may be provided on the display. By way of example, user guidance for one or more of steps a) through d) may be provided in a visual format and / or by other means, such as audio guidance.
[0066] For example, user guidance may be provided in sample application in step c), such that step c) may be as follows: - the mobile device prompts the user to apply a sample, specifically a drop of a bodily fluid, to the test field of the optical test strip; - the mobile device prompting the user to confirm application of the bodily fluid sample to the test field of the optical test strip; It may include at least one of:
[0067] As outlined above, the prompting may be done in a visual form by the mobile device providing corresponding guidance to the user, and / or by other means such as audio means. By way of example, user guidance may be provided on a display.
[0068] Similarly, user guidance may be provided by the mobile device to perform step a). Thus, step a) may comprise: - the mobile device prompting the user to provide an optical test strip having a test field without applying a sample to the test field; - the mobile device prompts the user to confirm that they have provided an optical test strip with a test field without applying a sample to the test field; - the mobile device automatically detects whether an optical test strip has been presented, in particular by image recognition; It may include at least one of:
[0069] For prompting options, see the explanation above. Thus, the steps of providing the optical test strip and even applying the sample to the test strip can be at least partially supported by the mobile device.
[0070] As outlined above, the methods in one or more of the disclosed embodiments may be fully or partially computer-implemented. Thus, in a further aspect, a computer program is proposed comprising instructions that, when the program is executed by a mobile device having a camera, in particular by a processor of the mobile device, cause the mobile device to perform the method described herein, more in particular at least steps b), d), e) and f) of the method, wherein steps a) and c) may also be at least in part computer-implemented or at least computer-supported.
[0071] A computer program may in particular be designed as an application, which may for example be downloaded from a download server.
[0072] As generally used herein, a "computer" can refer to a device having at least one processor and, optionally, additional elements such as one or more interfaces, one or more data storage devices, one or more user interfaces, etc. The term "processor" as used herein is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to any special or customized meaning. The term can specifically, but not exclusively, refer to any logic circuitry configured to perform the basic operations of a computer or system, and / or generally to a device configured to perform calculations or logical operations. In particular, a processor can be configured to process the basic instructions that drive a computer or system. By way of example, a processor can include at least one arithmetic logic unit (ALU), at least one floating-point unit (FPU), such as a math coprocessor or numeric coprocessor, multiple registers, specifically registers configured to supply operands to the ALU and store calculation results, and memory, such as L1 and L2 cache memories. In particular, a processor can be a multi-core processor. Specifically, a processor can be or include a central processing unit (CPU). Additionally or alternatively, a processor may be or include a microprocessor, and thus, in particular, elements of a processor may be included on one single integrated circuit (IC) chip. Additionally or alternatively, a processor may be or include one or more application specific integrated circuits (ASICs) and / or one or more field programmable gate arrays (FPGAs), etc.
[0073] The computer program may further comprise instructions that, when the program is executed by the mobile device, may further prompt the user to perform steps a) and / or c) or to confirm that steps a) and / or c) have been performed. For possible options, reference may be made to the method description above.
[0074] In a further aspect, a computer-readable storage medium, particularly a non-transitory storage medium, is disclosed that includes instructions that, when executed by a mobile device having a camera, particularly by a processor of the mobile device, cause the mobile device to perform a method according to the present invention, such as according to any one of the embodiments disclosed above and / or any one of the embodiments disclosed in more detail below. Specifically, at least steps b), d), e), and f) may be performed as outlined above, and one or both of steps a) and c) may also be at least partially computer-implemented or at least computer-supported, as outlined above. Thus, as noted above, the computer-readable storage medium may further include instructions that, when executed by the mobile device, further prompt a user to perform steps a) and / or c), or to confirm that steps a) and / or c) have been performed.
[0075] As used herein, the terms "computer-readable data carrier" and "computer-readable storage medium" may specifically refer to non-transitory data storage means, such as a hardware storage medium on which computer-executable instructions are stored. A computer-readable data carrier or storage medium may specifically be or include a storage medium, such as a random access memory (RAM) and / or a read-only memory (ROM).
[0076] A computer program may also be embodied as a computer program product. As used herein, a computer program product may refer to a program as a tradeable product. The product generally exists in any format, such as a paper format, or on a computer-readable data carrier and / or a computer-readable storage medium. In particular, the computer program product may be distributed over a data network.
[0077] In a further aspect, a mobile device for performing analytical measurements is disclosed. For definitions and options of the mobile device, reference may be made to the method description above. The mobile device may include at least one camera and one or more processors. The mobile device is configured to perform at least steps b), d), e), and f) of the method for performing analytical measurements according to the present invention, such as according to any one of the embodiments disclosed above and / or any one of the embodiments disclosed in more detail below. Thus, the processor of the mobile device may be software-configured to perform and / or control the execution of at least steps b), d), e), and f) of the method, and steps a) and / or c) may also be at least partially controlled and / or supported by the processor, as outlined above.
[0078] The mobile device may comprise further elements. Thus, by way of example, the mobile device may further comprise at least one illumination source configured to illuminate the test field. By way of example, the mobile device may comprise at least one light-emitting diode. The processor may also be configured to control the illumination source, such as during image capture steps b) and / or d).
[0079] As outlined above, the mobile device may comprise at least one processor programmed to control at least one of steps b), d), e) and f), for definitions and options regarding the design of the processor reference may be made to the above description.
[0080] In a further aspect, a kit for performing an analytical measurement is disclosed. The kit comprises: - at least one mobile device according to any one of the preceding claims relating to mobile devices, - at least one optical test strip having at least one test field; Equipped with.
[0081] The term "kit," as used herein, is a broad term and should be given its ordinary and accustomed meaning to those skilled in the art and should not be limited to any special or customized meaning. The term can specifically, but is not limited to, refer to an assembly of multiple components, where the components are each capable of functioning and can be treated independently of one another, and where the components of the kit can interact to perform a common function.
[0082] The present invention in any one of the aspects described herein can provide many advantages over known methods and devices of this type. Thus, specifically, the present invention can address the above-mentioned technical problems. As outlined above, smartphone-based methods typically require capturing at least two images, at least one image taken before sample application and at least one image taken after, which may be referred to as a "wet" or final image. The present invention can enable the use of augmented reality to improve positioning during capturing the first and second images. Thus, the same or at least similar positioning of the test strip can be permitted during capturing the first image or blank image after sample application and at least one second image or final image capture.
[0083] In general, the present invention can significantly improve the measurement performance of analytical measurements. Therefore, the measurement performance of smartphone-based optical analysis of test strips can typically be strongly dependent on the conditions under which images are taken before and after sample application. Ideally, the conditions are the same for both images. Augmented reality can be used to guide the user to take both images under very similar conditions. For positioning, the mobile phone's position sensor and / or image recognition technology can be used to determine the conditions to improve measurement performance.
[0084] Among these, several options exist: predetermined locations for capturing both the blank image and the final image can be used; additionally or alternatively, the blank image can be captured at essentially any first local location, and the mobile device can determine and / or mark the first local location and later use the first local location to provide user guidance so that the user is directed to the correct location for capturing the second image.
[0085] In summary, without excluding further possible embodiments, the following embodiments can be envisaged:
[0086] Embodiment 1: A method of performing an analytical measurement based on a color-forming reaction in an optical test strip by using a mobile device having a camera, comprising: a) providing an optical test strip having a test field without applying a sample of bodily fluid to the test field; b) capturing at least one first image of at least a portion of a test field of the optical test strip using a camera without applying a sample of bodily fluid to the test field; c) applying a sample of the bodily fluid to a test field of the optical test strip; d) capturing, using a camera, at least one second image of at least a portion of the test field of the optical test strip having the sample of bodily fluid applied thereto; e) determining an analytical measurement result value by using the first image (214) and the second image (222) of the optical test field of the optical test strip; Including, f) providing, by the mobile device, an indication of where to capture the second image such that the first and second images are captured at similar local locations relative to at least one environmental feature.
[0087] Embodiment 2: The method of embodiment 1, wherein the method further comprises waiting at least a predetermined minimum time between steps c) and d).
[0088] Embodiment 3: The method of embodiment 2, wherein the minimum amount of waiting time is at least 5 seconds.
[0089] Embodiment 4: A method according to any one of embodiments 1 to 3, wherein the local position includes at least one of: an absolute position of the camera in space relative to at least one environmental feature; an absolute position of the test field in space relative to at least one environmental feature; a relative position between the camera and the test field in a coordinate system defined by at least one environmental feature in the field of view of the camera; an orientation of the camera in space relative to at least one environmental feature; an orientation of the test field in space relative to at least one environmental feature; a relative angular orientation between the camera and the test field in a coordinate system defined by at least one environmental feature in the field of view of the camera; a relative angular orientation between the camera and at least one environmental feature in the field of view of the camera; a relative angular orientation between the test field and at least one environmental feature in the field of view of the camera; and a relative angular orientation between the camera and the test field in a coordinate system defined by at least one environmental feature in the field of view of the camera.
[0090] Embodiment 5: A method according to any one of embodiments 1 to 4, wherein the instructions for where to capture the second image include providing at least one similarity criterion for local locations when capturing the first and second images, so that the first and second images are captured at similar local locations with respect to at least one environmental feature.
[0091] Embodiment 6: The method of embodiment 5, wherein the method comprises evaluating whether a similarity criterion is met.
[0092] Embodiment 7: The method of embodiment 5 or 6, wherein the at least one similarity criterion includes at least one of an illumination similarity criterion and a local similarity criterion.
[0093] Embodiment 8: A method according to any one of embodiments 5 to 7, wherein at least one similarity criterion comprises comparing local positions when capturing the first and second images, in particular comparing the deviation between the local positions with at least one tolerance threshold.
[0094] Embodiment 9: The method of any one of embodiments 1 to 8, wherein the method includes determining a first local location when capturing a first image, and determining at least one second local location before or when capturing a second image, and the method includes comparing the first and second local locations to determine whether the first and second images were captured at similar local locations with respect to at least one environmental feature.
[0095] Embodiment 10: A method according to any one of embodiments 1 to 9, wherein the indication provided by the mobile device of where to capture the second image comprises a visual indication, specifically a visual indication on a display of the mobile device, so that the first and second images are captured at similar local positions with respect to at least one environmental feature.
[0096] Embodiment 11: A method according to any one of embodiments 1 to 10, wherein the indication provided by the mobile device of where to capture the second image is provided at least in part as augmented reality on the display of the mobile device so that the first and second images are captured at similar local positions relative to at least one environmental feature.
[0097] Embodiment 12: A method according to any one of embodiments 1 to 11, wherein the method includes determining and optionally storing at least one first item of position information relating to one or both of the position of the camera and the position of the test field when capturing the first image.
[0098] Embodiment 13: The method of embodiment 12, wherein the at least one first item of local information includes at least one of the absolute position of the test field, the absolute position of the camera.
[0099] Embodiment 14: The method of any one of embodiments 12 to 13, wherein determining at least one of the at least one first items of location information includes using at least one sensor of the mobile device, in particular at least one of a position sensor, a gyro sensor, an optical sensor, and a tilt sensor.
[0100] Embodiment 15: A method according to any one of embodiments 12 to 14, wherein determining at least one first item of location information includes detecting at least one environmental feature of the local environment, and the at least one first item of location information includes location information of one or both of the camera and the test field relative to the at least one environmental feature.
[0101] Embodiment 16: The method of embodiment 15, wherein the detection of at least one environmental feature comprises image recognition in an image provided by a camera, more particularly in a life image.
[0102] Embodiment 17: The method of any one of embodiments 15 to 16, wherein the at least one environmental feature includes at least one of an item within the field of view of the camera or a structural feature of an item within the field of view of the camera.
[0103] Embodiment 18: A method according to any one of embodiments 12 to 17, wherein the instructions provided by the mobile device of where to capture the second image are provided based on the first item of location information so that the first and second images are captured at similar local locations with respect to at least one environmental feature.
[0104] Embodiment 19: The method of embodiment 18, wherein the instruction includes a visual instruction, specifically a visual instruction on a display of the mobile device, more specifically a visual instruction at least partially using augmented reality on a display of the mobile device.
[0105] Embodiment 20: A method as described in embodiment 18 or 19, wherein the method includes determining, particularly live determining, and optionally storing at least one second item of location information regarding one or both of the camera position and the test field position, and comparing the second item of location information with the first item of location information.
[0106] Embodiment 21: The method of embodiment 20, wherein providing instructions for where to capture the second image so that the first and second images are captured at similar local locations with respect to at least one environmental feature includes providing instructions so that the second item of location information when capturing the second image is identical to the first item of location information, at least within a predetermined tolerance.
[0107] Embodiment 22: A method according to any one of embodiments 20 to 21, wherein at least one second item of local information includes at least one of the absolute position of the test field, the absolute position of the camera.
[0108] Embodiment 23: A method according to any one of embodiments 20 to 22, wherein determining at least one second item of location information includes using at least one sensor of the mobile device, in particular at least one of a position sensor, a gyro sensor, an optical sensor, and a tilt sensor.
[0109] Embodiment 24: A method according to any one of embodiments 20 to 23, wherein determining at least one second item of location information includes detecting at least one environmental feature of the local environment, specifically the same environmental feature used to determine at least one first item of location information, and wherein the at least one second item of location information includes position information of one or both of the camera and the test field relative to the at least one environmental feature.
[0110] Embodiment 25: The method of embodiment 24, wherein the detection of at least one environmental feature comprises image recognition in an image provided by a camera, more particularly in a life image.
[0111] Embodiment 26: The method of any one of embodiments 24 to 25, wherein the at least one environmental feature includes at least one of an item within the field of view of the camera or a structural feature of an item within the field of view of the camera.
[0112] Embodiment 27: A method according to any one of embodiments 20 to 25, wherein step d) is initiated automatically when at least one of the camera and the test field is in a predetermined position so that the first and second images are captured at similar local positions with respect to at least one environmental feature.
[0113] Embodiment 28: A method according to any one of embodiments 1 to 27, wherein the method further comprises, before performing step b), providing by the mobile device an indication of where to capture the first image.
[0114] Embodiment 29: The method of embodiment 28, wherein the instruction includes a visual instruction, specifically a visual instruction on a display of the mobile device, more specifically a visual instruction at least partially using augmented reality on a display of the mobile device.
[0115] Embodiment 30: Step c) - the mobile device prompts the user to apply a sample, specifically a drop of a bodily fluid, to the test field of the optical test strip; - the mobile device prompting the user to confirm application of the bodily fluid sample to the test field of the optical test strip; 30. The method of any one of embodiments 1 to 29, comprising at least one of:
[0116] Embodiment 31: Step a) comprises: - the mobile device prompting the user to provide an optical test strip having a test field without applying a sample to the test field; - the mobile device prompts the user to confirm that they have provided an optical test strip with a test field without applying a sample to the test field; - the mobile device automatically detects whether an optical test strip has been presented, in particular by image recognition; 31. The method of any one of embodiments 1 to 30, comprising at least one of:
[0117] Embodiment 32: A computer program comprising instructions that, when executed by a mobile device having a camera, specifically by a processor of the mobile device, cause the mobile device to perform the method according to any one of embodiments 1 to 31, more specifically at least steps b), d), e) and f) of the method according to any one of embodiments 1 to 31.
[0118] Embodiment 33: The computer program of embodiment 32, further comprising instructions that, when the program is executed by the mobile device, further prompt the user to perform steps a) and / or c) or to confirm that steps a) and / or c) have been performed.
[0119] Embodiment 34: A computer-readable storage medium, particularly a non-transitory storage medium, comprising instructions that, when executed by a mobile device having a camera, particularly by a processor of the mobile device, cause the mobile device to perform the method according to any one of method embodiments 1 to 31, more particularly at least steps b), d), e) and f) of the method according to any one of method embodiments 1 to 31.
[0120] Embodiment 35: A computer-readable storage medium as described in embodiment 34, wherein the storage medium further includes instructions that, when executed by the mobile device, further prompt the user to perform steps a) and / or c) or to confirm that steps a) and / or c) have been performed.
[0121] Embodiment 36: A mobile device for performing analytical measurements, the mobile device having at least one camera, the mobile device configured to perform at least steps b), d), e), and f) of the method for performing analytical measurements described in any one of the preceding embodiments relating to the method for performing analytical measurements.
[0122] Embodiment 37: A mobile device as described in embodiment 36, wherein the mobile device further comprises at least one illumination source configured to illuminate the test field.
[0123] Embodiment 38: A mobile device as described in embodiment 36 or 37, wherein the mobile device comprises at least one processor programmed to control at least one of steps b), d), e), and f).
[0124] Embodiment 39: A kit for performing an analytical measurement, comprising: - at least one mobile device according to any one of the preceding embodiments relating to a mobile device; - at least one optical test strip having at least one test field; A kit comprising: [Brief explanation of the drawings]
[0125] Further optional features and embodiments are disclosed in more detail in the subsequent description of the embodiments, preferably in conjunction with the dependent claims. Therein, each optional feature may be realized independently as well as in any possible combination, as understood by a person skilled in the art. The scope of the present invention is not limited by the preferred embodiments. The embodiments are illustrated diagrammatically in the figures, in which the same reference numerals in these figures refer to identical or functionally comparable elements.
[0126] The diagram is as follows:
[0127] [Figure 1] 1 shows a perspective view of an embodiment of a kit and a mobile device for performing analytical measurements. [Figure 2] 1 shows a flowchart of an embodiment of a method for performing analytical measurements. [Figure 3] Illustratively, first and second images taken before and after sample application are shown. [Figure 4] Illustratively, first and second images taken before and after sample application are shown. [Figure 5] Comparative blood glucose measurements are shown. DETAILED DESCRIPTION OF THE INVENTION
[0128] 1 shows an exemplary embodiment of a kit 110 for performing analytical measurements in perspective view. The kit 110 comprises a mobile device 112, such as a smartphone, and further comprises at least one optical test strip 114 in this configuration, positioned within the field of view 116 of a camera 118 of the mobile device 112.
[0129] In addition to at least one camera 118, the mobile device 112 may include at least one processor 120 and at least one data storage device 122. The mobile device 112 may further include at least one display 124, such as for displaying life images captured by the camera 118 and / or for displaying information to a user. The mobile device 112 may further include at least one illumination source 123, such as an LED.
[0130] The optical test strip 114 can include at least one substrate 126, such as a flexible strip-like substrate. The optical test strip 114 further includes at least one test field 128 applied to the substrate, the test field 128 containing at least one test chemical for performing a detection reaction with at least one analyte contained in the sample 130. The sample 130 can be applied directly or indirectly to the test field 128, for example, by applying a droplet of bodily fluid to the test field 128 and / or to a capillary element for directing the sample 130 to the test field 128.
[0131] The mobile device 112 is configured, by suitable programming of the processor 120, to execute and / or support the method according to the present invention as described with reference to the exemplary embodiment shown in the flowchart of FIG.
[0132] In a first step (step a)), indicated by reference numeral 210, an optical test strip 114 is provided. By way of example, a user may take an optical test strip from a test strip container. Step 210 may be supported by the mobile device 112, such as by prompting the user to provide the optical test strip 114, for example by displaying a corresponding message on the display 124. Additionally or alternatively, the mobile device 112 may also prompt the user to ensure that the optical test strip 114 has been provided, for example, within the field of view 116 of the camera. Again, additionally or alternatively, the mobile device 112 may also be configured to automatically detect that the optical test strip 114 has been provided, for example, by image recognition means.
[0133] In a further step (step b)) indicated by reference numeral 212, the mobile device 112 captures at least one first image 214 of at least a portion of the test field 128 of the optical test strip 114 by using the camera 118 without applying a sample 130 to the test field 128, as symbolically shown in FIG. 2 . The first image 214 can be stored in the data storage 122 of the mobile device 112. The first image 214 may also be referred to as a “blank image,” and the first image 214 can comprise a single image or a series of images, such as an image stream. The capture of the first image 214 may be triggered by a user and / or supported fully or partially by the mobile device 112, for example, by automatically triggering image capture upon recognition of the optical test strip 114 and / or the test field 128. To capture first image 214, optical test strip 114 can be placed by a user at any position relative to camera 118 within field of view 116, shown symbolically in FIG. 1 , and / or relative to at least one environmental feature 132. In this case, mobile device 112 may be specifically configured to determine a first local location when capturing first image 214 and, optionally, store the first local location in data storage device 122. Additionally or alternatively, mobile device 112 may also be configured to provide user guidance as to where to capture first image 214, for example, by indicating an appropriate location on display 124.
[0134] In a further step (step c)) indicated by reference numeral 216, a sample of bodily fluid 130 is applied directly or indirectly to the test field 128 of the optical test strip 114. Again, similar to step 210, this sample application can optionally be supported by the mobile device 112, such as by prompting the user to apply the sample 130 and / or by prompting the user to confirm that the sample 130 has been applied. Again, by way of example, corresponding instructions can be displayed on the display 124. Additionally or alternatively, the mobile device 112 may also be configured to automatically detect sample application.
[0135] After sample application in step 216, the method may provide an appropriate waiting period. By way of example, the waiting period may be at least 5 seconds to allow the detection reaction to occur. The waiting period may be displayed on the display 124, and / or the mobile device 112 may be configured to prevent further steps until the waiting period has expired.
[0136] In a further step (step f), indicated by reference numeral 218, the mobile device 112 captures, or provides instructions to the user on where to capture, a second image of at least a portion of the test field 128 of the optical test strip 114 having the bodily fluid sample 130 applied thereto, such that the first and second images are captured at similar local positions relative to the at least one environmental feature 132. Here, by way of example, the local position may be determined using the coordinate system 134 defined by the at least one environmental feature 132 of FIG. 1 . The local position may include one or more of an absolute position or a relative position of the optical test strip 114, the camera 118, or both. As outlined above, the term “position” may also include an orientation, such as an angular direction. To perform step f), the mobile device 112 may, by way of example, record and / or display a lifestream of images of the field of view 116, may also provide augmented reality on the display 124, and may provide instructions to the user on where to position the test field 128 and / or the camera 118. Below, exemplary embodiments are presented.
[0137] 2, at least one second image is captured, indicated by reference numeral 222, of at least a portion of the test field 128 of the optical test strip 114 having the bodily fluid sample 130 applied thereto. Again, this capture may be triggered automatically, such as when the mobile device 112 recognizes that the camera 118 and / or the test field 128 have been correctly positioned, at least within a predetermined tolerance, such that the first and second images 214, 222 are taken at the same location, i.e., at a similar local location with respect to at least one environmental feature 132 that satisfies at least one predetermined similarity criterion.
[0138] In a further step (step e) indicated by reference numeral 224, the mobile device 112 is configured to determine at least one analytical measurement value by using the first and second images 214 and 222 of the optical test field 128 of the optical test strip 114. One or more evaluation algorithms can be applied to the images 214 and 222 by the processor 120. For example, the processor 120 can derive at least one color coordinate from each of the images 214 and 222 to determine a color change between the blank image 214 and the final image 222 induced by a detection reaction. For example, by using a predetermined correlation or calibration function or lookup table, an analytical measurement value, such as the concentration of at least one analyte of interest, can be determined from the color change. For example, the glucose concentration in blood or interstitial fluid can be determined as the analytical measurement value. Other possibilities exist.
[0139] 3 and 4 show live images displayed on the display 124 of the mobile device 112, including augmented reality. For example, FIG. 3 shows the situation when capturing a first image 214, and FIG. 4 shows the live image before capturing a second image 222. As can be seen, multiple environmental features 132, such as a computer keyboard on a table, a cube, a computer mouse, or simply a pattern on the table, such as a wooden texture, are detected within the field of view 116 of the camera 118. These environmental features 132 can be used to determine the local position of the test field 128 and / or the camera 118, for example, by image recognition methods, as in FIG. 1. Additionally or alternatively, one or more sensors 136 of the mobile device 112 may be used to determine the first and / or second local positions.
[0140] 4, augmented reality may be used to provide instructions on where to capture the second image 222 so that the first and second images 214, 222 are captured at a similar local location relative to at least one environmental feature 132. The augmented reality may include symbols such as "no parking" signs and arrows, as well as text such as text containing instructions regarding any necessary positional corrections of the test field 128 and / or camera 118.
[0141] Generally, as outlined above, two options can be considered. Thus, as a first option, the first image 214 may be captured at any first local location. The mobile device 112 may be configured to determine the first local location and store the first local location, for example, in the data storage device 122. When capturing the second image 222, instructions are provided by the mobile device 112, for example, by augmented reality as shown in FIG. 4, as to where to capture the second image 222, so that the first and second images 214, 222 are captured at similar local locations relative to the at least one environmental feature 132. As a second option, the mobile device 112 may already provide instructions, for example, by augmented reality as shown in FIG. 4, as to where to capture the first image 214, as well as for capturing the second image 222.
[0142] FIG. 5 shows measurement results demonstrating the effect of controlling the local location for capturing the blank and final images. For these experiments, blood glucose measurements were performed using optical test strips 114 and samples 130. Two different setups were used: in a first setup, designated by reference numeral 310, the blank or first image 214 and the final or second image 222 were captured at the same local location. In a second setup, designated by reference numeral 312, the blank or first image 214 was captured at a common first local location, and the final or second image 222 was captured at a common second local location, where the second local location was different from the first local location. For each setup, 10 measurements were performed; for the blank images, a new optical test strip 114 was used (no sample was applied), while for the final images, an optical test strip 114 was used three days after sample application (the test field of this strip had consistent optical properties different from a new optical test strip).
[0143] The measurement results are shown in FIG. 5, where the horizontal axis shows the two different settings 310, 312. The vertical axis shows the determined analytical measurement result, in this case the blood glucose concentration c in mg / dL. The results are shown as box plots for both settings 310, 312. As can be seen, a significant difference occurs between the correct or controlled setting 310 and the uncontrolled setting 312. This difference is primarily attributed to the different lighting conditions at the first and second local locations. However, other effects may also occur. This difference clearly demonstrates the advantage of the present invention, since taking the first and second images 214, 222 at similar local locations with respect to at least one environmental feature increases the reproducibility of the measurements. [Explanation of symbols]
[0144] 110 Kits for performing analytical measurements 112 Mobile Devices 114 Optical Test Strips 116 Field of view 118 Camera 120 processors 122 Data storage device 123 Illumination source 124 display 126 Base material 128 Test Field 130 samples 132 Environmental Characteristics 134 Coordinate Systems 136 Sensors 210 Provide optical test strips 212. Capture a first image of the test field without applying a sample to the test field. 214 First Image 216 Apply a sample of body fluid to the test field of the optical test strip 218 Providing, by the mobile device, an indication of where to capture the second image such that the first and second images are captured at similar local locations with respect to at least one environmental feature. 220 Capture a second image of the test field onto which the body fluid sample has been applied. 222 Second Image 224 Determine analytical measurement results 310 First setting: Blank image and final image taken at the same local position 312 Second setting: Blank image and final image captured at different local positions
Claims
1. 1. A method for performing an analytical measurement based on a color-forming reaction in an optical test strip (114) by using a mobile device (112) having a camera (118), comprising: a) providing an optical test strip (114) having a test field (128) without applying a sample of bodily fluid (130) to said test field (128); b) capturing at least one first image (214) of at least a portion of the test field (128) of the optical test strip (114) without applying a sample (130) of bodily fluid to the test field (128) by using the camera (118); c) applying a sample (130) of bodily fluid to the test field (128) of the optical test strip (114); d) capturing at least one second image (222) of at least a portion of the test field (128) of the optical test strip (114) having the sample (130) applied thereto by using the camera (118); e) determining an analytical measurement result value by using the first image (214) and the second image (222) of the test field (128) of the optical test strip (114); Including, f) instructions are provided by the mobile device (112) of where to capture the second image (222) such that the first image and the second image (214, 222) are captured in a similar local spatial arrangement relative to at least one environmental feature (132), the environmental feature (132) being a feature having a fixed and / or unchanging spatial arrangement between capture of the first image (214) and the second image (222), the environmental feature (132) being different from the optical test strip (114) or a portion of the optical test strip (114) and the mobile device (112) or a portion of the mobile device (112) having the camera (118), and the spatial arrangement of the optical test strip (114) is changeable relative to the environmental feature (132).
2. The method of claim 1 , further comprising waiting at least a predetermined minimum time between steps c) and d).
3. The local spatial arrangement may include an absolute position of the camera (118) in space relative to the at least one environmental feature (132), an absolute position of the test field (128) in space relative to the at least one environmental feature (132), a relative position between the camera (118) and the test field (128) in a coordinate system defined by at least one environmental feature (132) within a field of view of the camera (118), an orientation of the camera (118) in space relative to the at least one environmental feature (132), an orientation ... at least one environmental feature (132) and the at least one environmental feature (132). a relative angular orientation between the camera (118) and the test field (128) in a coordinate system defined by (a)(b), a relative angular orientation between the camera (118) and at least one environmental feature (132) in the field of view of the camera (118), a relative angular orientation between the test field (128) and at least one environmental feature (132) in the field of view of the camera (118), a relative angular orientation between the camera (118) and the test field (128) in a coordinate system defined by (a)(b),
4. 4. The method of claim 1, further comprising: determining a first local spatial arrangement when capturing the first image (214) and determining at least one second local spatial arrangement before or when capturing the second image (222), and comparing the first and second local spatial arrangements to determine whether the first and second images (214, 222) were captured at similar local spatial arrangements with respect to at least one environmental feature.
5. 5. The method of claim 1, wherein an indication of where to capture the second image (222) provided by the mobile device (112) is provided at least in part as augmented reality on a display of the mobile device (112) such that the first image and the second image (214, 222) are captured in a similar local spatial arrangement with respect to the at least one environmental feature.
6. 6. The method of claim 1, further comprising determining at least one first item of spatial arrangement information relating to one or both of a spatial arrangement of the camera (118) and a spatial arrangement of the test field (128) when capturing the first image (214).
7. Determining at least one first item of spatial arrangement information includes: using at least one sensor (136) of the mobile device (112); detecting the at least one environmental feature (132) of a local environment, wherein at least one first item of spatial arrangement information includes spatial arrangement information of one or both of the camera (118) and the test field (128) relative to the at least one environmental feature (132); The method of claim 6, comprising at least one of:
8. 8. The method of claim 6 or 7, wherein the method includes determining at least a second item of spatial arrangement information relating to one or both of a spatial arrangement of the camera (118) and a spatial arrangement of the test field (128) and comparing the second item of spatial arrangement information with the first item of spatial arrangement information, and wherein providing instructions of where to capture the second image (222) such that the first and second images (214, 222) are captured in a similar local spatial arrangement with respect to the at least one environmental feature includes providing the instructions such that the second item of spatial arrangement information when capturing the second image (222) is identical to the first item of spatial arrangement information, at least within a predetermined tolerance.
9. The method of any one of claims 1 to 8, further comprising, prior to performing step b), providing, by the mobile device (112), an indication of where to take the first image (214).
10. Step c) is the mobile device (112) prompting a user to apply a sample of the bodily fluid (130) to the test field (128) of the optical test strip (114); the mobile device (112) prompting the user to confirm application of the sample of bodily fluid (130) to the test field (128) of the optical test strip (114); The method according to claim 1 , comprising at least one of the following:
11. The method of any one of claims 1 to 10, wherein the environmental feature (132) comprises an item or a portion of the item.
12. Step a) is the mobile device (112) prompting a user to provide the optical test strip (114) having the test field (128) without applying the sample (130) to the test field (128); the mobile device (112) prompting the user to confirm that they have provided the optical test strip (114) having the test field (128) without applying the sample (130) to the test field (128); the mobile device (112) automatically detecting whether the optical test strip (114) has been provided; The method according to claim 1 , comprising at least one of the following:
13. A computer program comprising instructions which, when executed by a mobile device (112) having a camera (118), causes the mobile device (112) to perform at least steps b), d), e), and f) of the method of any one of claims 1 to 12.
14. A computer-readable storage medium comprising instructions that, when executed by a mobile device (112) having a camera (118), cause the mobile device (112) to perform at least steps b), d), e), and f) of the method of any one of claims 1 to 12.
15. A mobile device (112) for performing analytical measurements, the mobile device (112) having at least one camera (118) and configured to perform at least steps b), d), e), and f) of the method according to any one of claims 1 to 12.
16. A kit (110) for carrying out an analytical measurement, comprising: At least one mobile device (112) according to claim 15; at least one optical test strip (114) having at least one test field (128); A kit (110).