Method and apparatus for controlling automatic white balance settings of a mobile device for color-based measurements using a color reference card - Patents.com

Abstract

11. A method for controlling an auto white balance setting of a mobile device having a camera and a preset auto white balance mode for performing color-based measurements, comprising: capturing an image of at least a portion of one or more gray fields of a color reference card; determining, for a region of interest in the image, a number of gray fields for which color information for a first color and for a second color indicates overexposure as a first overexposure number and a second overexposure number, respectively, where the first color is associated with a smaller wavelength than the second color; and recapturing the image in a hotter white balance mode if the first overexposure number is greater than the second overexposure number, or recapturing the image in a cooler white balance mode if the first overexposure number is less than the second overexposure number.

Description

[Technical field]

[0001] The present invention generally relates to a method for controlling an automatic white balance setting of a mobile device having a color camera and a plurality of pre-set white balance modes, each associated with a different color temperature, for performing color-based measurements using a color reference card including one or more gray fields, each with a known dedicated gray value. The present invention further relates to a mobile device having at least one camera, a kit for determining the concentration of an analyte in a sample of a body fluid, as well as a computer program and a computer-readable storage medium. The method, device, computer program and storage medium may be used in medical diagnostics, in particular for qualitatively and / or quantitatively detecting one or more analytes in one or more body fluids, such as for example for detecting glucose in blood and / or interstitial fluid. However, other fields of application of the present invention are also possible. [Background technology]

[0002] In the field of medical diagnostics, it is often necessary to detect one or more analytes from a sample of a body fluid, such as blood, interstitial fluid, urine, saliva, or other types of body fluids. Examples of analytes to be detected are glucose, triglycerides, lactate, cholesterol, or other types of analytes that are normally present in these body fluids. Depending on the concentration and / or presence of the analytes, appropriate treatments may be selected as required. Without narrowing the scope, the present invention may be specifically described with respect to blood glucose measurement. However, it should be noted that the present invention may also be used for other types of analytical measurements using test strips.

[0003] Generally, the devices and methods known to those skilled in the art utilize test strips that contain one or more test chemicals, which can carry out one or more detectable detection reactions, such as optically detectable detection reactions, in the presence of the analyte to be detected.For these test chemicals, reference can be made, for example, to J.Hoenes et al.:The Technology Behind Glucose Meters:Test Strips,Diabetes Technology&Therapeutics,Volume 10,Supplement 1,2008,S-10 to S-26.Other types of test chemicals are possible and can be used to implement the present invention.

[0004] Typically, one or more optically detectable changes in the test chemistry are monitored in order to extract from these changes the concentration of at least one analyte to be detected. Various types of detectors (with various types of light sources for illuminating the test field) are known for detecting at least one change in the optical properties of the test field of the test strip (also called dedicated meters or analytical measurement devices).

[0005] Furthermore, in addition to using specially developed customized detectors to optically detect changes in the test chemicals contained in the corresponding test strips, recent developments aim to use widely available devices such as smartphones. However, when a consumer electronics device with a camera, such as a smartphone, is used instead of a dedicated analytical measuring device to determine the analyte concentration, various influences must be taken into account. As an example, lighting conditions, positioning, or other more or less uncontrollable conditions must be taken into account.

[0006] In order to improve the reliability of photometric analysis using mobile devices, EP 3 575 781 A1 discloses an analyte test strip for detecting properties of an analyte in a analyte sample which has an additional color calibration area in addition to a reaction area for receiving the analyte sample.

[0007] A method of analyzing a colorimetric assay to identify a value for an assay parameter is disclosed in US Patent Application Publication No. 2015 / 0055134, in which intensity data for at least one of the color channels is converted to a first data point and compared to a standardized curve.

[0008] US 2020 / 204718 describes, for example, an automatic white balance method in which the white balance settings are automatically changed to capture an aesthetically pleasing photograph based on the captured color values ​​and / or the colors emitted by the light source and / or the subject of the photograph. Other automatic white balance procedures are described in US 9,628,704 or EP 2 498 498.

[0009] WO 2019 / 056242 describes a method for selecting a white balance mode to achieve a particular aesthetic effect, particularly when capturing images at night, i.e. when there is no or low ambient light.

[0010] WO 2021 / 010974 describes an auto white balance procedure in which an image is automatically segmented and different auto white balances are applied to different segments.

[0011] US Patent Application Publication No. 2019 / 0007589 describes a method for initializing a camera that includes different pre-configured capture settings, such as auto white balance. For initialization, the camera starts with a default auto white balance setting, captures one or more image frames, analyzes the image frames by measuring the color balance of at least some of the captured image frames against one or more thresholds, which may include blue / green color ratios and / or red / green color ratios against one or more thresholds, and enters into a recursive process of adjusting the auto white balance setting based on the analysis. The capture, measurement, and adjustment are repeated until the color balance of the image capture is within an acceptable range.

[0012] A basic white balance method for digital camera devices using a reference sample is described in WO 2014 / 025415. The method includes identifying a reference sample that is white under standard lighting conditions in a captured image, determining a correction factor based at least in part on an average color of the white reference sample, and applying the correction factor to the captured image and / or further captured images.

[0013] Despite the advantages associated with the use of mobile devices and color reference cards to perform analytical measurements, some technical challenges remain. In particular, the use of mobile devices to determine analyte concentrations using test strips may require accurate determination of the color change of the test strip and therefore often remains challenging. Proper image brightness and color balance need to be ensured. The observed brightness and color of light of a captured image may depend on various influencing factors, such as, for example, the settings that determine the exposure or white balance when capturing the image, as well as post-processing steps applied to the image after it is captured. The determination of some settings, such as the exposure or white balance settings, by the mobile device is often a device-specific process that is performed in automatic mode. However, typically, the automatic white balance of a mobile device, such as a mobile phone or tablet, is aimed at producing an aesthetically pleasing image and not at reproducing colors as faithfully as possible. In particular, the automatic white balance settings automatically selected by the mobile device may result in oversaturation and / or clipping of one or more of the color channels, which impedes further processing and determination of the blood glucose level based on the respective image.

[0014] Issues to be resolved It is therefore desirable to provide a method and apparatus that addresses the above-mentioned technical challenges of analytical measurements using mobile devices, such as consumer electronics mobile devices, in particular multi-purpose mobile devices that are not dedicated to analytical measurements, such as smartphones and tablet computers. In particular, a method and apparatus is proposed that is broadly applicable to available mobile devices and suitable for enhancing measurement accuracy and convenience for users. Summary of the Invention

[0015] overview This problem is addressed by a method and an arrangement having the features of the independent claims. Advantageous embodiments, which may be realised alone or in any combination, are set out in the dependent claims.

[0016] When used below, the terms "have", "comprise" or "include" or any grammatical variants thereof are used in a non-exclusive manner. These terms may therefore refer both to the situation in which, in addition to the features introduced by these terms, no further features are present in the entity described in this context, and to the situation in which one or more additional features are present. As an example, the expressions "A has B", "A comprises B" and "A includes B" may both refer to the situation in which, apart from B, no other elements are present in A (i.e., A consists solely and exclusively of B), and to the situation in which, apart from B, one or more further elements are present in the entity A, such as element C, elements C and D, as well as further elements.

[0017] Furthermore, it should be noted that the terms "at least one" or "one or more," or similar expressions, indicating that a feature or element may be present one or more times, are typically used only once when introducing each feature or element. Hereinafter, in most cases, when referring to each feature or element, the expressions "at least one" or "one or more" will not be repeated, despite the fact that each feature or element may be present one or more times.

[0018] Furthermore, when used hereinafter, the terms "preferably", "more preferably", "particularly", "more particularly", "particularly" or "more particularly" or similar terms are used in relation to any feature without limiting the possibility of substitution. Thus, features introduced by these terms are optional features and are not intended to constrain the scope of the claims in any manner. The invention may be implemented by using alternative features, as recognized by those skilled in the art. Similarly, features introduced by "in an embodiment of the invention" or similar expressions are intended to be optional features and do not entail any limitations on alternative embodiments of the invention, any limitations on the scope of the invention, or any limitations on the possibility of combining the features introduced in such a manner with other optional or non-optional features of the invention.

[0019] In a first aspect, a method is disclosed for controlling an automatic white balance setting of a mobile device for performing color-based measurements using the mobile device and a color reference card, the mobile device comprising a color camera and a plurality of pre-set white balance modes, each white balance mode being associated with a different color temperature, and the color reference card including one or more gray fields, each gray field having a known dedicated gray value.

[0020] The method comprises the steps of: a. capturing an image of at least a portion of one or more gray fields of a color reference card using a camera and one of a white balance mode; capturing an image of at least a portion of one or more gray fields of a color reference card, the image including a plurality of color pixels and color information for at least three different colors of the plurality of pixels; b. determining for a region of interest in the image a number of gray fields for which color information for a first color indicates overexposure as a first overexposure number; c. determining, for a region of interest in the image, a number of gray fields for which color information for a second color indicates overexposure as a second overexposure number; determining as a second overexposure number the number of grey fields in which the first colour is associated with a smaller wavelength or a smaller range of wavelengths than the second colour; d. recapturing the image of step a) in a hotter white balance mode if one of the first overexposure number and / or the second overexposure number, or the sum of the first overexposure number and the second overexposure number, exceeds a respective pre-set overexposure threshold, and the first overexposure number is greater than the second overexposure number; e. recapturing the image of step a) in a cooler white balance mode if one of the first overexposure number and / or the second overexposure number, or the sum of the first overexposure number and the second overexposure number, exceeds a respective pre-set overexposure threshold and the first overexposure number is less than the second overexposure number; Includes.

[0021] The method may include further steps not listed.

[0022] The term "control" generally refers to a process for influencing or adjusting a setting or for determining a setting in a defined or definable manner.

[0023] The term "automatic white balance setting" as used herein is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. The term may specifically, but is not limited to, refer to an automated process that aims to match the camera settings to the prevailing conditions, especially the scene lighting, so that neutral colors such as gray or white colors in the scene also appear neutral in the image captured by the camera. The term may further, but is not limited to, refer to an algorithm for selecting a pre-set white balance setting with the correct color temperature to match the scene lighting.

[0024] As used herein, the term "color temperature" of light, e.g., from a light source or ambient light, is a broad term and should be given its ordinary and customary meaning to one of ordinary skill in the art and should not be limited to any special or customized meaning. The term may specifically, but is not limited to, refer to the temperature of an ideal black body radiator that would radiate light of a color comparable to the respective light.

[0025] The term "preset white balance mode" as used herein is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. The term may specifically refer to a set of camera parameters for color balancing or color matching to a particular color temperature, i.e., a particular light condition, such as daylight or flashlight or other artificial or natural lighting. The camera parameters may relate to, but are not limited to, scaling of relative luminance by introducing relative scaling factors for different color channels of the camera. Examples of typical preset white balance modes are, but are not limited to, auto, daylight, incandescent, fluorescent, cloudy, speedlight, i.e., flash, and / or preset white balance categories.

[0026] As used herein, the term "mobile device" is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to any special or customized meaning. The term may specifically refer to, but is not limited to, a portable electronic device, more specifically a portable communication device such as a mobile phone or a smartphone. Additionally or alternatively, as outlined in more detail below, a mobile device may also refer to a tablet computer or another type of portable computer having at least one color camera.

[0027] The term "color reference card" as used herein is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. The term may specifically, but not exclusively, refer to any article having, disposed in or on at least one surface, such as having one or more gray fields with a predetermined gray level, i.e., gray value, and may further refer to a plurality of different color reference fields having known color or optical characteristics, such as having a plurality of colored fields with known reference color values. By way of example, the color reference card may be a flat card including at least one substrate having, disposed on and / or within at least one surface, a plurality of color reference fields with known color coordinates and one or more gray fields with known gray levels. The color reference card may also include one or more additional gray fields, as will be further described below.

[0028] The substrate may in particular have a flat surface including one or more gray fields, for example, a plurality of color reference fields and / or further gray fields. By way of example, the substrate may be or include one or more of a paper substrate, a cardboard substrate, a plastic substrate, a ceramic substrate or a metal substrate. Laminate substrates are also possible. The substrate may be, by way of example, sheet-like or flexible. It should be noted, however, that the substrate may be implemented in an article of use, such as a wall of a box, a vial, a container, a medical consumable such as a test strip. Thus, the color reference card may also be fully or partially integrated into the test strip. Thus, at least one image of at least a portion of the color reference card may fully or partially include an image of at least a portion of a test strip having at least one test field.

[0029] Furthermore, the color reference card may include at least one position marker. The at least one position marker may be or include, as an example, an ArUco code, among others. In particular, the at least one position marker may be located at at least one corner of the color reference card. The mobile device may therefore be configured to detect and / or read the marker, in particular by optically detecting the marker (e.g., on at least one image captured in step a), and optionally obtain information from the marker, such as information regarding the position and / or orientation of the color reference card relative to the camera. The color reference card may include other markers containing further information, or the position marker may further include further information. Such further information may include at least one of: an identifier for identifying the color reference card and / or the type of the color reference card, such as at least one of a label, a barcode, or a QR code; a designator for specifying details of the color reference card, such as a reference color value, a gray value, and / or a further gray background value, such as by using at least one of a label, a barcode, or a QR code.

[0030] The term "gray field" as used herein is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. The term may specifically refer to any item having known optical characteristics, such as, but not limited to, known reference color values. Specifically, the gray fields included in the color reference card may be two-dimensional structures such as rectangles, squares, polygons, circles and / or ellipses with uniform gray values, i.e., dedicated gray values. The gray values ​​of the gray fields may specifically be one or more of predetermined, known or determinable. For example, a gray value occurs when r=g=b, i.e., the red, green and blue color values ​​or color channels of an image point / pixel have equal values. The gray fields may be included and / or disposed within the surface of the color reference card, specifically, at least a portion of one or more gray fields may be visible in the image captured in step a).

[0031] The grey fields of the colour reference card may be arranged in a regular pattern on the surface of the colour reference card, for example in a rectangular pattern, such as a rectangular matrix pattern. The pattern arrangement may in particular enable identifying the grey fields, such as by searching at a predetermined distance in the x and / or y directions from one or more position markers.

[0032] Furthermore, the grey field or fields may be distributed locally across the colour reference card, and specifically across the portion of the colour reference card that is visible in the image.

[0033] The term "color camera" as used herein is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. The term may specifically refer to a device having at least one imaging element configured to record or capture spatially resolved one-, two-, or three-dimensional optical data or information, but is not limited to such a device. The camera may specifically comprise one or more imaging elements, such as a camera chip or imaging chip, such as a CCD chip and / or a CMOS chip. A color camera, and in particular an imaging element, may comprise a one- or two-dimensional array of image sensors, such as pixels. By way of example, a color camera may comprise at least 10 pixels in at least one dimension, such as at least 10 pixels in each dimension. However, it should be noted that other cameras may be used. A color camera may comprise, in addition to at least one camera chip or imaging chip, further elements, such as one or more optical elements, such as one or more lenses. By way of example, the camera may be a fixed focus camera, having at least one lens that is fixedly adjusted relative to the camera. However, alternatively, the camera may also include one or more variable lenses that may be adjusted automatically or manually. The camera may also include an aperture to control the opening (so that the amount of light reaching the camera sensor can be controlled). The aperture functions similarly to the iris of the eye and controls the effective diameter of the lens opening (called the aperture). The camera may further include an ambient light sensor to measure light reflected by the scene that is captured in the image.

[0034] The invention is intended to be particularly applicable to cameras typically used in mobile applications, such as notebook computers, tablets or mobile phones, particularly smartphones. In particular, the color camera may therefore be part of a mobile device that comprises, in addition to at least one camera, one or more data processing devices, such as one or more data processors. However, other color cameras are also feasible.

[0035] For each pixel of a color camera, color information such as three color values ​​of R, G, B may be provided or generated. More color values ​​are also feasible, such as four color values ​​per pixel, for example R, G, G, B. Color cameras are generally known to those skilled in the art. Thus, by way of example, a camera chip may consist of three or more different color sensors, color recording pixels, such as one pixel for red (R), one pixel for green (G), and one pixel for blue (B). Depending on the intensity of the respective color, for each pixel of R, G, B, etc., a value may be recorded by the pixel, such as a digital value ranging from 0 to 255. By way of example, instead of using a color triple such as R, G, B, a quadruple such as R, G, G, B may be used. The color sensitivity of the pixel may be generated by a color filter or by the appropriate intrinsic sensitivity of the sensor element used in the camera pixel. For example, the gain may be a digital camera setting that controls the amplification of the signal from the camera sensor.

[0036] The term "image" as used herein is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to any special or customized meaning. The term may specifically, but is not limited to, refer to a set of spatially resolved optical data. Specifically, the term may relate to data recorded by using a camera, such as multiple electronic readouts from an imager, such as the pixels of a camera chip. Additionally, the term may refer to, but is not limited to, a color digital image consisting of pixels that contain color information for at least three different colors, i.e., at least three different wavelengths of light. For example, each pixel is composed of a combination of color values, particularly for different primary colors, such as red, green, and blue. In an embodiment, the image may include multiple color pixels, each color pixel including an n-tuple of at least three color values ​​of three different colors. In an alternative embodiment, the image may further include at least three different color channels, and the color channel may refer to a grayscale image of the same size as the image, consisting of only one of the at least three colors.

[0037] The term "capturing an image" as used herein is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. The term may specifically refer to, but is not limited to, one or more of imaging, image recording, image acquisition, image capture. The term "capturing an image" may include capturing at least one single image, i.e., one image, and / or multiple images, such as a series of images. For example, capturing an image may include continuously recording a series of images, such as a video or movie. Capturing an image may be initiated by a user action or may be initiated automatically, for example, upon automatically detecting the presence of at least one object within the camera's field of view and / or within a predetermined portion of the field of view. These automatic image capture techniques are known, for example, in the field of automatic barcode readers, such as automatic barcode reading apps. Capturing an image may be done, for example, by capturing a stream or "live stream" of images with a camera, and automatically, or by user interaction, such as pressing a button, one or more of the images are stored and one of them is used as the at least one image. Image capture may be supported by a processor of the mobile device, and image storage may occur within a data storage device of the mobile device.

[0038] The capture of an image of at least a part of one or more gray fields of a color reference card by using a color camera may mean capturing an image that includes at least an area of ​​interest within at least one gray field. Thus, by way of example, one or more gray fields may be automatically detected in the image, for example by pattern recognition techniques commonly known to those skilled in the art, and at least one area of ​​interest, for example a rectangular, square, polygonal, elliptical or circular area of ​​interest, may be selected within each gray field. The taking of the image may be initiated by a user action or may be initiated automatically upon automatic detection of the presence of at least one gray field within the field of view of the camera and / or within a predefined sector of the field of view.

[0039] The term "region of interest" as used herein is a broad term and should be given its ordinary and customary meaning to one of ordinary skill in the art and should not be limited to any special or customized meaning. The term may specifically, but is not limited to, refer to a selected portion of an image, including one or more regions within an image or within the entire image. The region of interest may be determined by selecting a region within the image, for example, by automatically determining and selecting a region within the image that includes at least one gray field. The region or portion of the image may refer to a corresponding portion of pixels of the image, for example color pixels, and / or a corresponding portion of each color channel of the image.

[0040] The terms "overexposed" and "overexposure" as used herein are broad terms. They may specifically refer, but are not limited to, to an amount of light per unit area, or of a particular wavelength or range of wavelengths, that exceeds a preset threshold or exceeds the amount that can be reliably captured, for example, when the intensity in an image area deviates from the maximum intensity that can be represented. Alternatively, but are not limited to, the terms may refer to said guide values, such as the mean or median or maximum value determined for a region of interest where the guide values ​​exceed a preset threshold.

[0041] Thus, "determining, for a region of interest in an image, the number of gray fields for which color information for a particular color indicates overexposure" may refer, but is not limited to, to determining, for a region of interest in an image, the number of gray fields for which color information for a particular color exceeds a preset threshold.

[0042] "Overexposure of color information for a particular color of the region of interest" may refer to, but is not limited to, the color information of the region of interest exceeding a preset threshold or corresponding to a maximum value. Alternatively, but is not limited to, it may refer to one, multiple, or all pixels in the region of interest exceeding a preset threshold or corresponding to a preset maximum value. In another alternative, but is not limited to, it may refer to a guide value exceeding a preset threshold or corresponding to a preset maximum value, the guide value being determined for the region of interest. The guide value may refer to, but is not limited to, an average, median, or maximum value determined for each pixel. Furthermore, the guide value may be determined based on, but is not limited to, one, multiple, or all pixels of the region of interest.

[0043] According to steps d) and e), the method jumps back to step a) and if the first and / or the second overexposure number or the sum of the first and the second overexposure number exceeds the respective pre-set overexposure threshold, a new image is captured, i.e. the image is re-captured. This may mean that either one overexposure number or both overexposure numbers or the sum or combination of the aforementioned values ​​is compared with a suitable pre-set threshold and re-capture is performed if one or all of the values ​​compared with the threshold exceed the respective threshold. The value of either or the sum of both overexposure numbers is related to the degree of overexposure of the image.

[0044] For recapturing the image, a new white balance mode is selected based on the analysis of the determined overexposure number. If the first overexposure number is greater than the second overexposure number, i.e., if the overexposure for the light with a smaller wavelength, i.e., a lower color temperature, exceeds the overexposure for the larger, higher wavelength, a white balance mode associated with a higher color temperature, i.e., a higher white balance mode, is selected. Correspondingly, if a higher overexposure for the larger, higher wavelength, i.e., the second color, is determined, a white balance mode associated with a lower color temperature (lower white balance mode) is selected. For example, the following lower or higher white balance modes are selected, respectively. Alternatively, any one of the lower or higher white balance modes is selected. According to an embodiment, a difference between the two overexposure numbers is determined, and a higher or lower white balance mode is selected based on the determined difference.

[0045] An advantage of the described method is that the best mode is selected in a directed recursive process based on pre-defined device conditions, i.e. pre-defined white balance modes available for the device, and thus the described method is particularly easy and fast, helping to increase the reliability of color measurements via mobile devices.

[0046] In the described recursive process, the pre-set white balance modes may be selected according to a pre-set order, for example, the next hotter or colder mode is selected, or the next but one hotter or colder mode is selected, or any one of the hotter or colder modes is selected.

[0047] Furthermore, it is understood that the recursive process may terminate at any reasonable interruption point, which may be, for example, the fact that all pre-set white balance modes have been used, or the fact that the first and second overexposure numbers are equal, or the fact that the difference between the first and second overexposure numbers has reached a minimum, or the fact that said difference starts to increase again.

[0048] According to an embodiment, method steps b) and c) include identifying at least a portion of color information for each color corresponding to a gray field within the region of interest, determining a guide value for the identified portion of color information, and determining whether the guide value exceeds a predetermined overexposure threshold.

[0049] Identifying may refer to finding all or a portion of the image information and / or color information corresponding to the gray field, for example based on a predetermined condition. For example, multiple color pixels or pixels of color channels and corresponding respective values ​​of color are identified.

[0050] The term "guide value" as used herein is a broad term and should be given its ordinary and customary meaning to one of ordinary skill in the art and should not be limited to any special or customized meaning. The term may specifically refer to, but is not limited to, the mean, median, and / or maximum values ​​determined for the identified color information, such as a plurality of values ​​for a particular color of an identified color pixel, or a plurality of values ​​for an identified pixel of a particular color channel.

[0051] According to another embodiment, the second color is red and the first color is blue or green. Preferably, the first color is blue, in which case the difference between the corresponding wavelengths of the first and second colors is greater than when the first color is green.

[0052] According to a further embodiment, the color-based measurement is an analyte measurement based on a color reaction, which occurs in a test strip upon application of the sample to a test area of ​​the test strip.

[0053] The term "color reaction" as used herein is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. The term may specifically refer to, but is not limited to, a chemical, biological or physical reaction in which the color, specifically the reflectance, of at least one element involved in the reaction changes as the reaction progresses. Thus, as an example, reference may be made to the biochemical reaction mentioned above, which typically is used to detect blood glucose, involving a color change. Other types of color change or color reactions, such as the typical chemical reaction for determining pH value, are known to those skilled in the art.

[0054] The term "test strip" as used herein is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. The term may specifically, but not limited to, refer to a strip-like element or device configured to perform a color change detection reaction. A test strip is also referred to herein simply as a test strip, and these terms may refer to the same element. A test strip may have, in particular, a test field that includes at least one test chemical for detecting at least one analyte. By way of example, a test strip may comprise at least one substrate, such as at least one carrier, to which at least one test field is applied or incorporated therein. In particular, a test strip may further comprise at least one white area, such as a white field, in particular adjacent to, for example surrounding or surrounding, the test field. The white area may be a separate field independently disposed on the substrate or carrier. However, additionally or alternatively, the substrate or carrier itself may be or comprise a white area. At least one carrier may be in the form of a strip, thereby providing the basic form of a test strip. These test strips are widely available and in common use. A test strip may carry a single test field or multiple test fields having the same or different test agents contained therein.

[0055] As further used herein, the term "test field" is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. The term may specifically refer to, but is not limited to, a concentrated amount of test chemical, such as a circular, polygonal or rectangular shaped field or area having one or more layers of material with at least one layer of the test field containing the test chemical.

[0056] The test strip may be placed over a color reference card and / or the color reference card may include one or more windows, and the color reference card having one or more windows is placed over the test strip such that the test field or at least a portion of the test field is visible through the window.

[0057] According to another embodiment, the color reference card further includes a plurality of different color reference fields having known reference color values.

[0058] The term "color reference field" as used herein is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. The term may specifically refer to any item having known optical characteristics, such as, but not limited to, known reference color values. Specifically, the color reference fields included in the color reference card may be two-dimensional structures, such as rectangles, squares, polygons, circles and / or ellipses, having uniform color values. The color values ​​of the color reference fields may specifically be one or more of predetermined, known, or determinable. The color reference fields may specifically be constituted by and / or disposed within the surface of the color reference card such that at least a portion of the multiple different color reference fields (e.g., one color reference field) is visible in the image captured in step c).

[0059] Furthermore, the color reference fields may have color values ​​in a subspace of the color coordinate system that corresponds to the color space of the chromogenic reaction of the reagent test area. The color reference fields of the color reference card may be specifically arranged in a regular pattern on the surface of the color reference card, such as a rectangular pattern, such as a rectangular matrix pattern. The pattern arrangement may specifically allow for identifying the color reference fields, such as by searching at a predetermined distance in the x-direction and / or y-direction from one or more position markers.

[0060] According to a further embodiment, the image captured in step a) further captures at least a portion of the test area of ​​the test strip. This helps to ensure that the captured image can be used to perform a color-based measurement, for example to determine an analyte concentration. Thus, no further step of capturing an image for the color-based measurement is required.

[0061] According to another embodiment, the method further comprises determining the concentration of the analyte in the sample by using the image captured in step a) if the sum of the first overexposure number and the second overexposure number is less than a pre-defined overall overexposure threshold and / or if the difference between the first overexposure number and the second overexposure number is less than a pre-defined balance threshold. Alternatively, if the above mentioned conditions are met, further images are captured and used to determine the concentration of the analyte in the sample.

[0062] According to a further embodiment, steps a) to e) are repeated until the difference between the first overexposure number and the second overexposure number is less than a preset balance threshold, or until step a) has been performed for all white balance modes of the plurality of white balance modes, or until the difference between the first overexposure number and the second overexposure number starts to increase compared to the difference determined based on previously captured images, and then the concentration of the analyte in the sample is determined using the image corresponding to the smallest difference between the first overexposure number and the second overexposure number, or using an extra image captured using the auto white balance mode corresponding to the smallest difference between the first overexposure number and the second overexposure number.

[0063] According to another embodiment, the mobile device includes a display, and the method further includes providing visual guidance on the display for positioning the camera relative to a scene, the scene including at least a portion of a test strip and / or at least a portion of a color reference card. The visual guidance includes, for example, one or more arrows, frames, or lines indicating a preferred positioning of the camera relative to the object. As an example, the visual guidance includes an outline corresponding to a shape of an object superimposed on the display of the mobile device.

[0064] Providing visual guidance on the display to position the camera relative to an object on the display to determine exposure parameters and / or capture at least one image may in particular relate to providing visual guidance on the display to guide the user to position the camera such that the scene includes at least a portion of one or more gray fields of the color reference card. The guidance may further provide visual guidance to guide the user to position the camera such that the scene further includes further structures such as at least a portion of a test field of a test strip to which a sample has been applied and / or at least a portion of a plurality of different color reference fields of the color reference card. The visual guidance may, for example, include a contour corresponding to a shape of at least a portion of the color reference card and / or at least a portion of the test strip superimposed on the display of the mobile device.

[0065] According to a further embodiment, the color reference card includes at least one position marker.

[0066] According to another embodiment, the method is initiated automatically when the camera is determined to be in a defined position relative to the color reference card based on the at least one position marker.

[0067] According to a further embodiment, the test field is at a defined position relative to the color reference card during capture of the at least one image.

[0068] According to a further aspect of the invention, a mobile device is disclosed having at least one camera and at least one display, the mobile device being configured to perform the method described above.

[0069] According to another aspect of the present invention, a kit for determining the concentration of an analyte in a body fluid is disclosed, comprising the mobile device as described above, at least one test strip having at least one test area, and at least one color reference card, the color reference card including a plurality of different color reference areas having known reference color values ​​and one or more gray fields having defined gray values.

[0070] According to a further aspect of the invention, a computer program comprising instructions, which when executed by a mobile device having a camera, cause the mobile device to perform at least steps a) to d) of the method described above.

[0071] According to another aspect of the present invention, a computer readable storage medium, particularly a non-transitory storage medium, containing instructions which, when executed by a mobile device having a camera, cause the mobile device to perform at least steps a) to e) of the method described above is disclosed. [Brief description of the drawings]

[0072] Further optional features and embodiments are disclosed in more detail in the following description of the embodiments, preferably in conjunction with the dependent claims. Here, each optional feature may be realized in an independent manner as well as in any feasible combination, as understood by a person skilled in the art. The scope of the present invention is not limited by the preferred embodiments. The embodiments are illustrated diagrammatically in the figures, where the same reference signs in these figures refer to the same or functionally equivalent elements.

[0073] In the diagram it looks like this: [Figure 1] A color reference card is shown in top view. [Diagram 2] 1 illustrates an embodiment of a kit comprising a mobile device, a color reference card, and a test strip. [Diagram 3] 1 shows the situation when capturing an image of a color reference card. [Figure 4] 1 shows a flow chart of an embodiment of a method for controlling white balance settings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0074] Detailed Description of the Preferred Embodiments 1 illustrates in plan view an exemplary embodiment of a color reference card 110. The color reference card 110 includes one or more gray fields 114 having known gray values ​​and a plurality of color reference fields 112 having known reference color values. The gray fields 114 and the color reference fields 112 may be disposed on a surface of the color reference card 110, such as on a substrate of the color reference card 110.

[0075] The color reference fields 112 may be evenly distributed across the surface of the color reference card 110, in particular such that a plurality of color reference fields 112 may be distributed across the surface of the color reference card 110. By way of example, the color reference fields 112 may be arranged in a matrix pattern, such as a rectangular matrix pattern. However, the color reference fields 112 may be arranged in other ways.

[0076] One or more gray fields 114 may surround the color reference fields 112 and / or frame the color reference card and / or be distributed along the rows and / or columns between the matrix pattern of color reference fields 112. The color reference fields 112 and the gray fields 114 do not have to overlap each other.

[0077] The color reference card 110 may further include at least one window 116. Thus, at least one test strip 118, or a portion thereof, may be visible through the window 116 when the color reference card 110 is placed over the test strip 118. Specifically, at least one test field 120 included in the test strip 118 may be visible through the window 116 of the color reference card 110. As another example, the color reference card 110 may include a test strip 118 having at least one test field 120, specifically such that the at least one test field 120 is accessible and visible.

[0078] Furthermore, the color reference card 110 may include at least one position marker 122. The position marker 122 may be, in particular, an ArUco code. In FIG. 1, the position marker 122 may specifically include one or more ArUco codes, such as at the corners of a rectangular matrix including the color reference field 112 and a portion of the gray field 114 and surrounded by the gray field 114. Thus, in general, the position marker 122 may be located at at least one corner 124 of the color reference card 110. For example, at least one position marker 122 may be located at each of the corners 124 of the color reference card 110, in particular such that the position marker 122 is visible with the multiple color reference fields 112. Furthermore, the position marker 122 may include information regarding the orientation of the color reference card 110.

[0079] An exemplary embodiment of the kit is shown in perspective view in Figure 2. The kit comprises at least one mobile device 128 and at least one color reference card 110. Additionally, the kit comprises at least one test strip 118.

[0080] The mobile device 128 may be or may comprise at least one of a mobile phone, a smartphone, a tablet computer, etc. Additionally, the mobile device 128 has at least one color camera 130. The color camera 130 of the mobile device 128 may be configured to record images, specifically color images. For example, the color camera 130 comprises at least three color sensor types, such as at least one color sensor type each of red, green, and blue.

[0081] Furthermore, the mobile device 128 may comprise at least one processor 132. The processor 132 may be configured, in particular by software programming, to execute one or more of the method steps a) to d) of the method for controlling an automatic white balance setting. Exemplary embodiments of the above-mentioned methods are respectively illustrated in Fig. 4 and described in further detail below. Reference may therefore be made to the description of Fig. 4.

[0082] The processor 132 may be specifically configured to support setting of an area of ​​interest and to determine a first overexposure number and a second overexposure number of a gray field within the area of ​​interest. The processor 132 may be further specifically configured to capture at least one image including a scene 126. In the illustrated embodiment, the scene 126 comprises the entire color reference card 110 and the test field 120 of the test strip 118. Specifically, the processor 132 may prompt a user of the mobile device 128 to capture an image. Additionally or alternatively, the processor 132 may be configured to automatically capture an image of the color reference card 110, specifically when the color reference card 110 is within the field of view of the camera 130.

[0083] Color reference card 110 is described above (see FIG. 1).

[0084] At least one marker 122 of the color reference card 110 may be used to identify the color reference card 110. Specifically, the processor 132 of the mobile device 128 may be configured to detect the position marker 122 within the field of view of the color camera 130.

[0085] The color reference card 110 and the test strip 118 may be visible on at least one image captured by a color camera 130 of the mobile device 128. Specifically, at least one test area 120 of the test strip 118 may be visible through a window 116 of the color reference card 110.

[0086] 3 illustrates the situation when capturing an image. In this embodiment, visual guidance is provided on the display 134 of the mobile device 128 in the form of an outline 136 of the color reference card superimposed on the display to guide the user to position the camera such that the captured scene 126 corresponding to the outline 136 includes at least a portion of one or more gray fields. In the illustrated example, the scene within the set exposure metering area includes the entire color reference card.

[0087] Control of the automatic white balance setting and image capture may be initiated automatically, for example, when a color reference card is detected within the field of view of the camera 130 based on the position marker 122.

[0088] FIG. 4 illustrates a flow chart of an exemplary embodiment of a method for controlling auto-exposure settings of a mobile device 128 .

[0089] The method is performed by a processor and / or other processing means and begins with acquiring an image of at least a portion of one or more gray fields 114 of a color reference card 110 using a camera 130 and one of the white balance modes, the image including a plurality of pixels and color information for at least three different colors of the plurality of pixels, and in the illustrated embodiment, the color reference card is detected in the image based on its ArUco code.

[0090] For at least one region of interest ROI in the image, the number of grey fields 114 for which color information for a first color, channel B, indicates overexposure is determined as a first overexposure number. Correspondingly, for the same region of interest ROI in the image, the number of grey fields 114 for which color information for a second color, channel R, indicates overexposure is determined as a second overexposure number, where the first color, here blue, is associated with a smaller wavelength or a smaller range of wavelengths than the second color, here red.

[0091] It is determined whether the sum of the first overexposure number and the second overexposure number exceeds a threshold N, and if there is no undue overexposure, i.e., if the sum of the first overexposure number and the second overexposure number does not exceed the threshold N, a color-based measurement is performed on the current image or a newly captured image using the current white balance mode.

[0092] In the case of excessive overexposure, ie the sum of the first overexposure number and the second overexposure number exceeding the threshold N, the first overexposure number of channel B and the second overexposure number of channel R are compared.

[0093] In the embodiment shown, if the first overexposure number of B is greater than the second overexposure number of R, it is checked whether the used white balance mode is already the hottest, and if so, a color-based measurement is performed with the current image or a newly captured image using the current white balance mode. If the used white balance mode is not the hottest one, a white balance mode associated with a hotter color temperature is selected from the multiple white balance modes, and the described process begins again to acquire an image using the newly selected white balance mode.

[0094] In the embodiment shown, if the first overexposure number of B is not greater than the second overexposure number of R, it is checked whether the used white balance mode is already the coolest, and if so, a color-based measurement is performed with the current image or a newly captured image using the current white balance mode. If the used white balance mode is not the coolest, a white balance mode associated with a cooler color temperature is selected from the multiple white balance modes, and the described process starts again to acquire an image using the newly selected white balance mode.

[0095] The method may include additional steps not described above. [Explanation of symbols]

[0096] 110 Color Reference Card 112 Color Reference Field 114 Greyfield 116 Windows 118 Test Strips 120 Test Field 122 Position Marker 124 angle 126 Scenes 128 Mobile Devices 130 Camera 132 processors 134 Display 136 Contour

Claims

1. 1. A method for controlling an automatic white balance setting of a mobile device (128) for performing color-based measurements using the mobile device (128) and a color reference card (110), comprising: the mobile device (128) comprises a processor (132), a display (134), a color camera (130), and a plurality of pre-set white balance modes, each white balance mode being associated with a different color temperature; - said color reference card (110) comprises one or more grey fields (114), each grey field having a known dedicated grey value; The method comprises: capturing an image of at least a portion of the one or more gray fields (114) of the color reference card (110) using the camera (130) and one of the white balance modes; i. each white balance mode includes a set of camera parameters for color balancing or color matching for a particular color temperature; ii. capturing an image of at least a portion of the one or more gray fields (114) of the color reference card (110), the image comprising a plurality of pixels and color information for at least three different colors of the plurality of pixels, each color corresponding to a wavelength of light; b) determining, for a region of interest in the image, the number of gray fields (114) for which the color information for a first color exceeds a predetermined threshold, thereby indicating overexposure, as a first overexposure number; c) determining, for the region of interest in the image, the number of gray fields (114) for which the color information for a second color exceeds a predetermined threshold, thereby indicating overexposure, as a second overexposure number; i. determining a second overexposure number as the number of gray fields (114) in which the first color is associated with a smaller wavelength or a smaller range of wavelengths than the second color; d. If one of the first overexposure number and the second overexposure number, or the sum of the first overexposure number and the second overexposure number, exceeds a respective preset overexposure threshold, and the first overexposure number is greater than the second overexposure number, recapturing the image of step a) in a white balance mode associated with a warmer color temperature [pp. 15, 1. 1-8]; e. recapturing the image of step a) in a white balance mode associated with a cooler color temperature when one of the first overexposure number and the second overexposure number, or the sum of the first overexposure number and the second overexposure number, exceeds a respective preset overexposure threshold, and the first overexposure number is less than the second overexposure number; A method comprising:

2. Step b) and step c) identifying at least a portion of the color information for each color corresponding to the gray field within the region of interest; b. determining a guide value for the identified portion of the color information; c. determining whether the guide value exceeds a predetermined overexposure threshold; The method of claim 1 , comprising:

3. The method of claim 1 , wherein the second color is red and the first color is blue or green.

4. 2. The method of claim 1, wherein the color-based measurement is an analyte measurement based on a color reaction, and the color reaction occurs in an optical test strip (118) to which a sample has been applied in a test area (120) of the test strip (118).

5. The method of claim 1 , wherein the color reference card (110) further comprises a plurality of different color reference fields (112) having known reference color values.

6. The method of claim 1 , wherein the image captured in step a) further captures at least a portion of the test area (120) of the optical test strip (118).

7. The method comprises: a. i. the sum of the first overexposure number and the second overexposure number is less than a pre-set overall overexposure threshold; and / or ii. The method of claim 1, further comprising determining a concentration of the analyte in the sample by using the image captured in step a) if a difference between the first overexposure number and the second overexposure number is less than a preset balance threshold.

8. a. until the difference between the first overexposure number and the second overexposure number is less than a preset balance threshold; or b. Step a) is performed for all white balance adjustment modes of the plurality of white balance adjustment modes, or c. steps a) through e) are repeated until the difference between the first overexposure number and the second overexposure number begins to increase compared to the difference determined based on previously captured images; Next, d. using the image corresponding to the smallest difference between the first overexposure number and the second overexposure number; or e. The method of claim 1, wherein the concentration of the analyte in the sample is determined using an additional image captured using an automatic white balancing mode corresponding to the smallest difference between the first overexposure number and the second overexposure number.

9. The mobile device (128) comprises a display (134), and the method comprises:

10. The method of claim 1, further comprising: providing visual guidance on the display for positioning the camera relative to a scene, the scene including at least a portion of the optical test strip and / or the color reference card.

10. The method of claim 1 , wherein the color reference card includes at least one position marker (122).

11. 11. The method of claim 10, wherein the method is automatically initiated when the camera (130) is determined to be in a specified position relative to the color reference card (110) based on the at least one position marker (122).

12. A mobile device (128) having at least one camera (130) and at least one display (134), the mobile device (128) being configured to perform the method of any one of claims 1 to 11.

13. 1. A kit for determining the concentration of an analyte in a body fluid, comprising: a. a mobile device according to claim 12; b. at least one optical test strip (118) having at least one reagent test field (120); c. at least one color reference card (110), said color reference card (110) including a plurality of different color reference fields (112) having known reference color values ​​and one or more gray fields (114) having defined gray values; A kit comprising:

14. 12. A computer program comprising instructions that, when executed by a mobile device (128) having a camera (130), cause the mobile device (128) to perform at least steps a) to d) of the method of any one of claims 1 to 11.

15. 12. A computer-readable storage medium, in particular a non-transitory storage medium, comprising instructions that, when executed by a mobile device (128) having a camera (130), cause the mobile device (128) to perform at least steps a) to e) of the method of any one of claims 1 to 11.

Images