A method for urine test paper mobile phone image detection analysis

Abstract

The application discloses a method for urine test paper mobile phone image detection and analysis, which comprises the following steps: collecting urine test paper JPEG images under multiple light sources under a standard light source box, taking the test paper image under the D65 light source as a standard image, converging the RGB color performance of the test paper block in the corresponding area of the test paper image under the standard light source to the test paper image under the remaining light source environment, that is, training and obtaining a compensation model, fully considering the test paper color features and the mobile phone camera characteristic features in the model characteristic parameters, ensuring the accuracy of the color correction model by analyzing the maximum value, minimum value and variance of the features during preprocessing, establishing a training TabNet model, fusing multiple regression models such as LightGBM, ElasticNet Regression and Support Vector Regression to optimize the color correction effect, and realizing a strengthened compensation model. Through the color correction of the test paper image by the compensation model, the error caused by the color performance of the urine test paper image affected by the environmental light is reduced, the most real color of the test paper image is restored, the color recognition degree is improved, the accuracy of the test paper detection and analysis is improved, and the model is small and has a fast recognition speed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, and particularly relates to a method for detecting and analyzing urine test paper mobile phone images. BACKGROUND

[0002] With the continuous development of computer technology, portable medical treatment is also rapidly developing, so that more and more medical data can be processed and judged by computers, and disease diagnosis is more convenient. With the improvement of living conditions, high-sugar and high-protein dietary patterns have made kidney diseases and diabetes common diseases in our lives, and such diseases almost require urine tests every day. Medical resources are expensive, and the conventional method of urine test paper detection is to use manual colorimetric cards or urine test paper analyzers. When using manual colorimetric cards to judge urine components, different people have subjective differences in judging the same test paper, and the needs of patients with eye diseases cannot be met. As for urine test paper analyzers, the cost is too high for individual users, and ordinary families cannot afford it; and patients with mild conditions have no reason to buy them. These methods have brought a lot of trouble to the people. Generally, urine analyzers are controlled by microcomputers, and the color change on the test strip is measured by a ball area instrument that receives double-wavelength reflected light for semi-quantitative measurement. We provide mobile medical treatment for urine tests. The first point to note is that the environmental light source has a great influence on the color of the image, which is the main factor affecting the accuracy of urine test paper detection. If a high-precision mobile phone urine test paper detection system can be developed based on the scientific method of medical imageomics and the principle of spectroscopy to reduce the error caused by environmental light in the imaging of test paper, people can realize accurate urine test paper detection at home, which will bring people a new medical experience. SUMMARY

[0003] The purpose of the present application is to overcome the shortcomings of conventional technology and provide a urine test paper mobile phone image detection and analysis method. The method uses a color correction model built by combining spectroscopy principles and artificial intelligence technology to detect and analyze urine test paper images obtained by a mobile phone. The method fully considers the interference factors such as mobile phone camera hardware parameters and light, reduces the requirements for the shooting environment to a certain extent, and improves the detection accuracy. The model constructed by the present application is small, runs fast, and can quickly give detection results.

[0004] The technical solution for achieving the purpose of the present application is as follows:

[0005] A urine test paper mobile phone image detection and analysis method, comprising the following steps:

[0006] 1) Establishing urine test paper image dataset: five groups of urine test paper images are collected under D65, D50, F, TL83, TL84 light sources using a self-developed mobile test paper shooting APP. The APP automatically extracts the test paper strip, while retaining the corresponding camera ISO, exposure, focal length and ambient light brightness at the time of shooting each image. The test paper test block RGB value, standard block RGB value, camera ISO, exposure, focal length and ambient light brightness are integrated as training samples and used as the data range of step 2);

[0007] 2) Test paper image parameter preprocessing: the color block RGB value, camera shooting time corresponding exposure, sensitivity, focal length and ambient light brightness are extracted as features and integrated into a CSV file for data analysis. The final data obtained after feature description is used as the training and test data of step 3);

[0008] 3) Establishing color correction regression model: a color correction model for color compensation is constructed, and the training set and test set are used for model training to obtain the trained model;

[0009] 4) Test paper detection: the test paper image is obtained by shooting on the mobile APP and uploaded to the trained model. After color correction processing of the test paper by the model, the color of each color block of the test paper is compared with the color of the color card stored in the model. The error size of each color block of the test paper is obtained and the result is output. One detection is completed.

[0010] In step 1), the urine test paper image obtained by APP shooting has a pixel size of 394*2661, and each image is accompanied by a txt file retaining the four-dimensional features of camera sensitivity, exposure, focal length and ambient light brightness at the time of shooting. The 5*5 pixel size sample image block is taken at equal intervals along the center of the test paper, and the average pixel value is calculated to obtain the RGB value feature of the test paper.

[0011] In step 2), the test paper image parameter preprocessing is a statistical analysis and data science analysis after step 1). First, the same dimension features of the sample are extracted and sorted by size, and the value range, average value, variance and data stability are calculated. The excessive exposure of the camera and the influence of ambient light will cause the RGB value of some white blocks to rise to the maximum value 255. More than three test paper blocks with RGB value reaching 255 in the same test paper sample are considered as invalid samples and are excluded.

[0012] In step 3), the color correction regression model is established by constructing a TabNet regression model, training the model with the training and test data obtained in step 2) as the training set and test set, and finally obtaining a regression model for compensating the pigment value. The model that retains a smaller color correction result error relative to the standard condition test paper color is linearly fused in step 4); the establishment process of the TabNet model is as follows:

[0013] 3-1) Each sample data obtained after preprocessing has eleven features, including ambient light mode, ambient light brightness measured by the mobile phone light sensor, focal length, sensitivity, exposure, standard color block RGB value, and test color block RGB value. The RGB values are processed separately as three-dimensional features.

[0014] 3-2) After BN (Batch Normalization, BN) processing, the data is input into the feature calculation layer of the TabNet model to calculate the selected features and obtain the feature calculation results.

[0015] 3-3) After the feature calculation results are calculated by the Attentive Transformer layer, the features that need to be deleted are calculated again, and the feature calculation is repeated. The operation of each layer of neurons in TabNet is calculated through multiple layers and enters the fully connected layer to finally obtain the compensation value output of the target pixel.

[0016] During model training, the root mean square error (RMSE) is used as the loss function, and the Adam function is used as the backpropagation optimizer.

[0017] After the TabNet model deep learning regression network is trained, all test papers need to be compensated for the pigment value. After compensation, all test papers will approach the pigment value under the standard light, and after comparing the compensated test paper with the standard color card, the accurate urine test result is obtained.

[0018] The TabNet model is optimized by fusing LightGBM, ElasticNet Regression, and Support Vector Regression three learners after establishing the TabNet model. The linear fusion method is used to weight the fusion of all models obtained by training. The TabNet, LightGBM, ElasticNet Regression, and Support Vector Regression four learners are used to obtain a new prediction value. The weight value is 1, and the weighted fusion formula is as follows:

[0019]

[0020] is the weight; each learner is assigned a weight , According to the advantages and disadvantages of the compensation results of the respective models, weights are obtained, and finally a final compensation model is obtained through fusion.

[0021] The urine test paper mobile phone image detection and analysis method provided by the application has small model size, fast recognition speed and high accuracy, and has important significance for patients who often need to do urine analysis, and can provide convenient medical services for diagnosis and treatment of related diseases. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is a technical circuit diagram of a urine test paper mobile phone image detection and analysis method;

[0023] Figure 2 is a schematic diagram of the compensation model establishment process. DETAILED DESCRIPTION

[0024] The application will be further described below in combination with the drawings and examples, but is not limited to the application.

[0025] Examples:

[0026] As shown in Figure 1 A urine test paper mobile phone image detection and analysis method, specifically comprising the following steps.

[0027] 1) The user uses the APP to take the urine test paper and transmits the test paper image to the server end running the color correction model, the color correction model restores the true color of the test paper (with D65 as the standard), the standard color card in the server end model is compared with the uploaded test paper image, and the test paper test result is analyzed, and the server returns the result to the user end after operation.

[0028] 2) The center point of the purple reference color block and the center point of the upper frame of the test paper strip are calibrated in the urine test paper strip picture by using the OpenCV contour detection method based on Python, the 5x5 pixel image block of the test paper color block center is extracted along the center, the test block RGB value and the standard block RGB value are obtained by summing the pixel points after obtaining the image block, and the sensitivity, exposure, focal length and environmental brightness are extracted from the camera parameter file as the data set of the compensation model. After sampling analysis on the data set obtained according to the above method, it is found that the value position of each color block is located at the center of the color block, and after maximum value, minimum value and variance analysis on the data set, each group of data is within the reasonable error range, which shows that the batch of data can be used for subsequent model establishment.

[0029] Specifically, 5*5 pixel image blocks are taken from the center of each of the 14 color blocks of a JPEG test paper image with a pixel size of 394*2661, the sum of the image block pixels is averaged to obtain the RGB value and the camera state parameters (sensitivity, exposure, ambient light, focal length) at the corresponding time as the feature input of the regression network model.

[0030] 3) An artificial intelligence regression algorithm model is established for data compensation, such as Figure 2 As shown in the figure, the final data described in step 2) is taken as the feature input of the regression model, and after training, a urine test paper data compensation model is obtained.

[0031] An application program is written and provided to users, and when the user needs to analyze the test paper, the APP can be directly used to shoot the test paper and upload it to the server side. All data processing and analysis processes are completed on the server side to ensure that different mobile phones can have the same high-speed and accurate analysis speed. After the test paper data is sent to the server side, data compensation is performed first, and the final result obtained is sent to the user side to display the result.

[0032] The innovation of the present application is that it can reduce the large error caused by different color performances of the images of the test paper shot under different environmental lights. D65 light source is the most commonly used artificial daylight in standard light sources. We select D65 light source as the standard light source in the test paper analysis process. Because it is impossible to require users to shoot test paper only in a certain environment, it is necessary to compensate the color of the test paper data shot by the user under different environmental lights, so that the color performance of the photo shot under the standard light source is close to the color performance of the test paper shot under the standard light source. After training the color compensation model, the photo shot by the user under any environmental light can obtain accurate detection result, which greatly reduces the requirement for environmental light when using mobile phone to shoot urine test paper image for urine analysis, and improves the accuracy of detecting and analyzing urine test paper image.

Claims

1. A method for mobile phone image detection and analysis of urine test strips, characterized in that, Includes the following steps: 1) Establish a urine test strip image dataset: Five sets of urine test strip images were collected using a self-developed mobile test strip shooting APP under D65, D50, F, TL83, and TL84 light sources. The APP automatically extracted the test strips and retained the mobile phone camera's ISO, exposure, focal length, and ambient light at the time of each image capture. The parameters such as the RGB values ​​of the test color block, the RGB values ​​of the standard block, the camera's ISO, exposure, focal length, and ambient light were integrated into training samples and used as the data range for step 2). 2) Preprocessing of test strip image parameters: Extract the RGB values ​​of color blocks, the corresponding exposure, sensitivity, focal length, and ambient light brightness at the time of camera shooting as features and integrate them into a CSV file for data analysis. The final data obtained after feature description is used as the training and testing data in step 3). 3) Establish a color correction regression model: Build a color correction model to achieve color compensation, and train the model using the training set and test set to obtain a well-trained model; In step 3), establishing the color correction regression model involves constructing a TabNet regression model. The training and testing data obtained in step 2) are used as the training and testing sets to train the model, ultimately resulting in a regression model that compensates for pigment values. Models with smaller color correction errors compared to test paper color errors under standard conditions are retained for linear fusion in step 4). The process of establishing the TabNet model is as follows: 3-1) Each sample data obtained after preprocessing has eleven features, namely: ambient light mode, ambient light brightness measured by the mobile phone light sensor, focal length, sensitivity, exposure, standard color block RGB value, and test color block RGB value. Among them, RGB is processed separately as three-dimensional features. 3-2) After processing the data with BN (Batch Normalization), it is input into the feature calculation layer of the TabNet model to calculate the features to be selected and obtain the feature calculation results; 3-3) After the feature calculation results are combined and calculated again through the Attentive Transformer layer to calculate the features that need to be deleted, the feature calculation is repeated again. This is the operation performed on the data by each neuron of TabNet. After multiple layers of calculation, it enters the fully connected layer and finally obtains the compensation value output of the target pixel. During model training, the root mean square error (RMSE) is used as the loss function, and the Adam function is used as the backpropagation optimizer. After training, the TabNet deep learning regression network regresses the pigment values ​​that all test strips need to compensate for. After compensation, the pigment values ​​of all test strips will be infinitely close to the pigment values ​​under standard light. After the compensated test strips are compared with the standard color chart, accurate urine test results can be obtained. After building the TabNet model, three learners—LightGBM, ElasticNet Regression, and SupportVector Regression—are fused together. All trained models are then weighted using a linear fusion method. A new predicted value is obtained by weighting the four learners (TabNet, LightGBM, ElasticNet Regression, and SupportVector Regression) with a sum of weights equal to 1. The weighted fusion formula is as follows: As weight; 4) Test paper detection: Take a picture of the test paper on the mobile APP, upload it to the trained model, and after the model performs color correction on the test paper, compare it with the color chart stored in the model. After obtaining the error of the color of each color block of the test paper with the color chart, the result is output, and one test is completed.

Images