Urine test paper image detection method based on mobile equipment

Abstract

The invention discloses a urine test paper image detection method based on mobile equipment. The urine test paper image detection method comprises the following steps: acquiring camera parameters of a plurality of mobile equipment, establishing an equipment optical feature library, acquiring test paper images shot by the mobile equipment in different environments, and associating the test paper images with the equipment optical feature library to establish a multi-environment sample library; the multi-environment sample library serves as sample input, a color block detection model is established, and anchor points of samples are dynamically adjusted in the training process of the color block detection model; converting the RGB value of the test paper color block of the sample into a standard CIE Lab space; performing direction judgment and abnormal point filtering on the sample based on the checking color block space relationship; and constructing a standard urinalysis pigment feature library, matching the sample with the standard urinalysis pigment feature library, and outputting a detection result. The method has the beneficial effects that the cross-equipment color difference problem is effectively solved by establishing the equipment optical feature library, introducing the double-branch detection network and improving the anchor point adjustment mechanism of the SSD, and the accuracy of a urine detection result is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of urine test strip detection, and particularly to a method for detecting urine test strip images based on a mobile device. Background Art

[0002] With the development of medical technology, urine detection, as a routine detection method, has been more and more widely used. Usually, a urine analyzer is required to detect the urine to be analyzed.

[0003] In practical applications, due to the large volume and high price of urine analyzers, the popularization and application of urine detection methods are restricted. At present, with the popularization of smart devices and the application of image analysis algorithms, pictures are taken of test strips soaked in urine and test strips not soaked in urine using smart devices, and then the taken pictures are analyzed using image analysis algorithms to obtain the color of the urine test strip; then, according to the color vector value displayed by the urine test color block soaked in urine, the matching result of the color vector value in the set of reference color vector values of the color block matching the urine test color block is determined to obtain the urine detection result.

[0004] However, the lens resolutions of different smart devices are different, and the built-in image processing algorithms of the devices are also different, resulting in large color differences in the images obtained by different devices, leading to different detection results of the same sample on different devices, and further resulting in inaccurate urine detection results. Summary of the Invention

[0005] In view of the above problems, the present invention proposes a method for detecting urine test strip images based on a mobile device, mainly solving the problems in the background art.

[0006] To solve the above technical problems, the technical solution of the present invention is as follows:

[0007] A method for detecting urine test strip images based on a mobile device includes the following steps:

[0008] Collect the camera parameters of a number of mobile devices, establish a device optical feature library, and collect the test strip images taken by the mobile devices in different environments. The test strip images are associated with the device optical feature library to establish a multi-environment sample library;

[0009] Use the multi-environment sample library as a sample input to establish a color block detection model. The color block detection model dynamically adjusts the anchor points of the samples during the training process;

[0010] Convert the RGB values of the test strip color blocks of the samples to the standard CIE Lab space;

[0011] Based on the verification of the color block space relationship, perform direction determination and outlier filtering on the samples;

[0012] Construct a standard urine test pigment feature library, match the sample with the standard urine test pigment feature library, and output the test result.

[0013] In some embodiments, the device optical feature library includes collecting CMOS sensor parameters, lens MTF curves, and ISP processing pipeline configurations of the mobile device, and obtaining white balance coefficients, color correction matrices, and gamma curve parameters through the API of the mobile device.

[0014] In some embodiments, the process of acquiring the test paper image includes:

[0015] Under three environments, namely, standard light source, indoor fluorescent light and natural light, a 24-color standard color card and a urine test strip were placed side by side. Different mobile devices were used to capture images of the test strip in original RAW format, and the EXIF ​​metadata of each test strip image was recorded.

[0016] In some implementations, dynamically adjusting the anchor point of the sample includes: setting a scale level of the anchor point, and then dynamically adjusting the anchor point ratio according to an actual aspect ratio of the color block of the sample.

[0017] In some embodiments, the color block detection model also includes a dual-branch detection network, wherein the dual-branch detection network simultaneously detects the color blocks of the standard color card, as well as the color blocks of the test paper and the verification blocks, and finally uses a loss function to evaluate the accuracy of the dual-branch detection network.

[0018] In some embodiments, converting the RGB value of the test paper color block of the sample to the standard CIE Lab space includes: establishing a conversion matrix from RGB to CIE Lab of the mobile device, and then using Bradford to transform the conversion matrix to convert the RGB value of the test paper color block of the sample to the standard CIE Lab space.

[0019] In some implementations, the step of determining the direction of the sample and filtering outliers based on the spatial relationship of the verification color blocks includes:

[0020] The slope θ of the line connecting the center points of the verification block is calculated. When |θ|>5°, the direction correction is triggered. The nine-square grid sampling method is used to obtain 9 sampling points evenly distributed in the color block. The Mahalanobis distance of the Lab value of each point is calculated, and the abnormal points exceeding 3σ are eliminated.

[0021] In some embodiments, matching the sample with the standard urine test pigment signature library comprises:

[0022] For each element in the standard urine test pigment feature library, an ellipsoid model in the Lab color space is established, and the central coordinates (L0, a0, b0) and covariance matrix Σ are recorded. The Mahalanobis distance method is used to match the distance between the sample and the element, and the component corresponding to the minimum distance value is taken as the detection result. When the minimum distance value is less than 5, the sample is determined to be an abnormal sample.

[0023] The beneficial effects of the present invention are as follows: By establishing an optical feature library of devices, introducing a dual-branch detection network, and improving the anchor adjustment mechanism of SSD, the problem of color difference between devices is effectively solved, and the accuracy of urine test results is improved. Brief Description of the Drawings

[0024] Figure 1 It is a schematic flow chart of a urine test strip image detection method based on a mobile device disclosed in an embodiment of the present invention. Detailed Embodiments

[0025] To make the objectives, technical solutions and advantages of the present invention clearer and more definite, the content of the present invention will be further described in detail below with reference to the drawings and specific embodiments. It can be understood that the specific embodiments described herein are only used to explain the present invention, rather than limiting the present invention. Additionally, it should be noted that for the sake of description, only parts related to the present invention are shown in the drawings, rather than all the content.

[0026] As Figure 1 shown, this embodiment proposes a urine test strip image detection method based on a mobile device, which is applicable to urine test strips based on chemiluminescence or dry chemistry methods, and includes the following steps:

[0027] Step 1: Collect the camera parameters of several mobile devices, establish an optical feature library of devices, collect the test strip images taken by the mobile device in different environments, and establish a multi-environment sample library by associating the test strip images with the optical feature library of devices.

[0028] In an example, the optical feature library of devices includes collecting the CMOS sensor parameters, lens MTF curve, and ISP processing pipeline configuration of the mobile device, and obtaining the white balance coefficient, color correction matrix (CCM), and gamma curve parameters through the API of the mobile device, so as to construct the device feature vector D = {sensor configuration, CCM, gamma value}. The introduction of the above-mentioned optical feature library of devices aims to eliminate the hardware imaging differences. By collecting the underlying hardware data such as CMOS sensor parameters and ISP processing pipeline configuration, an optical feature library of devices is established, enabling the model to actively identify the imaging characteristic differences of different devices. The quantization and storage of the white balance coefficient and gamma value provide a computable physical parameter basis for subsequent color correction, converting the device differences into variables that can be mathematically compensated.

[0029] The process of obtaining the test strip image includes:

[0030] Under three environments of standard light source, indoor fluorescent lamp and natural light respectively, place the 24-color standard color card side by side with the urine test strip, use different mobile devices to take the test strip images in the original RAW format, record the EXIF metadata (ISO, exposure time, color temperature) of each test strip image, associate this EXIF metadata with the device feature vector, and establish a multi-environment sample library. The synchronous shooting of the standard color card and the test strip establishes an objective color reference system in the Lab space to eliminate the interference of ambient light color temperature on color recognition. The storage of RAW format images avoids color distortion caused by JPEG compression, and the recording of EXIF metadata provides a timestamp evidence chain for subsequent white balance correction.

[0031] Step two, use the multi-environment sample library as the sample input to establish a color block detection model, and the color block detection model dynamically adjusts the anchor points of the samples during the training process.

[0032] In one example, dynamically adjusting the anchor points of the samples includes: setting the scale levels of the anchor points (from 8×8 to 32×32 pixels) to cover different shooting distances, and then dynamically adjusting the anchor point ratio according to the actual length-to-width ratio of the color blocks of the samples (preset to 1:1.5). The improved anchor point adjustment mechanism can match the live frame of the color blocks, making this method adaptable to multiple platforms and resolutions.

[0033] More preferably, the color block detection model also includes a dual-branch detection network to establish a dynamic mapping relationship between the device color response and the standard color space. Among them, the dual-branch detection network synchronously detects the color blocks of the standard color card, as well as detects the color blocks and calibration blocks of the test strip, and finally uses a loss function to evaluate the accuracy of the dual-branch detection network. For example, use Focal Loss to solve the problem of class imbalance in color block detection, and the application of FocalLoss improves the detection accuracy of edge-blurred color blocks.

[0034] Step three, convert the RGB values of the test strip color blocks of the samples to the standard CIE Lab space.

[0035] In one example, converting the RGB values of the test strip color blocks of the samples to the standard CIE Lab space includes: establishing a conversion matrix M from mobile device RGB to CIE Lab = argmin∑(XYZ_measured - M·RGB_device) 2 , and then use Bradford to transform the conversion matrix (CAT) to convert the RGB values of the test strip color blocks of the samples to the standard CIE Lab space. In this implementation scheme, the device-related RGB is converted to the standard CIE Lab space through the M matrix to eliminate the influence of the device color rendering algorithm. The Bradford color adaptation transformation (CAT) makes the color matching under different light sources conform to the CIE human visual model, meeting the requirements of professional color management.

[0036] Step 4: Determine the direction of the sample and filter outliers based on the spatial relationship of the verification color blocks.

[0037] In one example, determining the direction of a sample and filtering outliers based on the spatial relationship of the verification color blocks includes:

[0038] Calculate the slope θ of the line connecting the center points of the calibration block. When |θ|>5°, trigger the direction correction. Use the nine-square grid (3×3 grid) sampling method to obtain 9 sampling points evenly distributed in the color block, calculate the Mahalanobis distance of the Lab value of each point, and remove abnormal points exceeding 3σ. The above optional scheme is based on the slope detection of the calibration color block connection line to realize the automatic recognition of the rotation angle of the test paper image, and the 9 sampling points evenly distributed in the color block can improve the sampling accuracy.

[0039] More preferably, the collection points whose RGB are not in the reagent reaction color range are eliminated, the remaining RGB collection points are mathematically averaged, and finally the process is skipped to step five to match the sample with the standard urine test pigment feature library, thereby improving the accuracy of the test.

[0040] Step 5: Build a standard urine test pigment feature library, match the sample with the standard urine test pigment feature library, and output the test results.

[0041] Matching samples to a library of standard urine pigment profiles includes:

[0042] Each element in the standard urine test pigment feature library corresponds to the ellipsoid model in the Lab space, and the center coordinates (L0, a0, b0) and the covariance matrix Σ are recorded. The Mahalanobis distance method is used to match the distance between the sample and the element, D_m = sqrt((x-μ)^TΣ^{-1}(x-μ)), and the component corresponding to the minimum distance value is taken as the detection result. When the minimum distance value is less than 5, it is judged as an abnormal sample, which can effectively identify abnormal situations such as test paper failure and sampling error, and improve the accuracy of matching.

[0043] In step five, the ellipsoid model has higher matching accuracy than the traditional cubic color gamut container, especially improving the recognition stability of purple (wavelength 450nm) and dark green (wavelength 550nm), which is particularly suitable for urine test solution detection tasks.

[0044] Through the above solution, users can scan urine test strip samples at home with their mobile phones, and the images of the test strips are uploaded to the cloud for encrypted calculation. The test results can be obtained within 30 seconds, enabling multiple physical examinations to be completed quickly, and promptly detecting and monitoring diabetes, liver and gallbladder diseases, kidney diseases, hemolytic diseases, etc., achieving early detection and diagnosis. Among them, 14 test indicators such as urine protein, occult blood, creatinine, urine sugar, urinary calcium, and bilirubin are completed at one time, and the abnormal indicators are interpreted simultaneously to guide medical treatment, which is fast and convenient. There is also dynamic blood glucose monitoring without finger pricking, and the detection of uric acid, blood lipids, etc. can also be quickly completed at home, and the test results can be viewed at any time.

[0045] The above embodiments are only for illustrating the technical concept and characteristics of the present invention, and the purpose is to enable ordinary technicians in the field to understand the content of the present invention and implement it accordingly, and it cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the essence of the content of the present invention should be covered within the protection scope of the present invention.

Claims

1. A urine test strip image detection method based on a mobile device, characterized in that, The following steps are involved: Collect camera parameters of several mobile devices, establish a device optical feature library, collect test paper images taken by the mobile devices in different environments, and associate the test paper images with the device optical feature library to establish a multi-environment sample library; The multi-environment sample library is used as a sample input to establish a color block detection model, and the color block detection model dynamically adjusts the anchor point of the sample during the training process; Convert the RGB value of the sample's test paper color patch to the standard CIE Lab space; Determine the direction of samples and filter outliers based on the spatial relationship of the verification color blocks; Construct a standard urine test pigment feature library, match the sample with the standard urine test pigment feature library, and output the test result.

2. The urine test strip image detection method based on a mobile device according to claim 1, characterized in that, The device optical feature library includes collecting the CMOS sensor parameters, lens MTF curve and ISP processing pipeline configuration of the mobile device, and obtaining the white balance coefficient, color correction matrix and gamma curve parameters through the API of the mobile device.

3. The urine test strip image detection method based on a mobile device according to claim 1, wherein The test paper image acquisition process includes: Under three environments, namely, standard light source, indoor fluorescent light and natural light, a 24-color standard color card and a urine test strip were placed side by side. Different mobile devices were used to capture images of the test strip in original RAW format, and the EXIF ​​metadata of each test strip image was recorded.

4. The urine test strip image detection method based on a mobile device according to claim 1, wherein, The dynamically adjusting the anchor point of the sample includes: setting the scale level of the anchor point, and then dynamically adjusting the anchor point ratio according to the actual aspect ratio of the color block of the sample.

5. The urine test strip image detection method based on a mobile device according to claim 1, characterized in that, The color block detection model also includes a dual-branch detection network, wherein the dual-branch detection network simultaneously detects the color blocks of the standard color card, as well as the color blocks of the test paper and the calibration blocks, and finally uses a loss function to evaluate the accuracy of the dual-branch detection network.

6. The urine test strip image detection method based on a mobile device according to claim 1, characterized in that, The converting of the RGB value of the test paper color block of the sample into the standard CIE Lab space includes: establishing a conversion matrix from RGB to CIE Lab of the mobile device, and then using Bradford to transform the conversion matrix to convert the RGB value of the test paper color block of the sample into the standard CIE Lab space.

7. The urine test strip image detection method based on a mobile device according to claim 5, characterized in that, The direction determination and outlier filtering of the sample based on the spatial relationship of the verification color block includes: The slope θ of the line connecting the center points of the verification block is calculated. When |θ|>5°, the direction correction is triggered. The nine-square grid sampling method is used to obtain 9 sampling points evenly distributed in the color block. The Mahalanobis distance of the Lab value of each point is calculated, and the abnormal points exceeding 3σ are eliminated.

8. The urine test strip image detection method based on a mobile device according to claim 7, characterized in that, The matching of the sample with the standard urine test pigment feature library comprises: Each element in the standard urine test pigment feature library corresponds to the ellipsoid model in the Lab space, and the center coordinates (L0, a0, b0) and the covariance matrix Σ are recorded. The distance between the sample and the element is matched using the Mahalanobis distance method, and the component corresponding to the minimum distance value is taken as the detection result. When the minimum distance value is less than 5, it is judged as an abnormal sample.

Images