Intelligent kidney disease type identification method and system based on tongue coating image, and storage medium

Abstract

The invention discloses an intelligent kidney disease type identification method and system based on a tongue coating image and a storage medium, relates to the technical field of intelligent medical treatment, and aims to solve the problem that a chronic kidney disease prediction method based on the tongue coating image is lacked in the prior art. The method comprises the following steps: S1, collecting tongue coating image data and medical record data of a patient, preprocessing the medical record data, and inserting the tongue coating image data of the patient into the corresponding medical record data; s2, constructing a tongue coating image feature extraction model, and training the tongue coating image feature extraction model by using the tongue coating image data; s3, performing feature extraction on the tongue coating image data by adopting the trained tongue coating image feature extraction model to obtain a tongue coating feature vector, combining the tongue coating feature vector with a corresponding medical record vector to obtain an embedded vector, and training an SVM classifier; and S4, integrating the trained tongue coating image feature extraction model and the SVM classifier to obtain a kidney disease type identification model based on the tongue coating image.

Description

Technical Field

[0001] The present invention relates to the field of smart medical technology, and in particular to a method, system and storage medium for intelligently identifying kidney disease types based on tongue coating images. Background Art

[0002] Chronic kidney disease has now become a major disease threatening global public health. Early diagnosis and intervention of the disease process of chronic kidney disease are key factors affecting its prognosis. Tongue diagnosis is a more important part of the four diagnoses of traditional Chinese medicine. It can objectively reflect the deficiency and excess of the patient's internal organs, the rise and fall of qi and blood, the surplus and deficiency of body fluids, the shallowness and depth of the disease, and the prognosis. The systematic and standardized algorithm analysis and diagnosis of tongue coating may be more accurate, efficient, and stable to improve the prognosis and quality of life of patients with chronic kidney disease.

[0003] In the actual clinical treatment of chronic kidney disease, it is found that the tongue images of different kidney disease patients change significantly during the course of the disease. [2] found that patients with chronic renal failure and liver and kidney yin deficiency syndrome often had a pale red or reddish tongue, while patients with spleen and stomach qi deficiency, yang deficiency, qi and yin deficiency, and yin and yang deficiency were more likely to have a pale tongue. [4] It is found that patients with chronic uremia who have both qi and blood deficiency and spleen yang deficiency often have pale tongue. Patients with chronic renal failure with dampness and turbidity syndrome or spleen and kidney yang deficiency have pale tongue color. Patients with spleen and kidney yang deficiency also have different degrees of pale purple, dark purple, ecchymosis and other manifestations of blood stasis. [5-6] . Cheng Xiaoxia et al. [7] Through observation, it was found that the change of tongue color is correlated with the stage of renal function. For example, the tongue of patients in the compensatory stage of renal function is mostly light red or reddish. As the renal function is impaired, the patient's tongue will gradually change from red to light, and from light to white. In the late stage of uremia, the tongue of patients is almost pale and dull. Wang Yiqin et al. [8] It was found that chronic renal failure with spleen and kidney qi deficiency is mainly characterized by thin white or greasy tongue coating; liver and kidney yin deficiency is mainly characterized by less tongue coating and thin yellow or yellow greasy tongue coating. It is believed that the identification of tongue coating quality and color can provide good guidance for the diagnosis of the disease, especially for the deficiency syndrome. However, the identification of tongue images has always been limited by the doctor's subjective visual physiology, knowledge level, thinking ability and diagnostic skills, and lacks objective evaluation standards. In some important theoretical aspects, there are still different schools of thought, different opinions and lack of unified standards. For example, Shi Qiang [1]Through the observation and analysis of 40 tongue photos by 13 traditional Chinese medicine practitioners, it was found that there were certain differences in the judgment of the 13 doctors on the tongue color, tongue texture, and tongue shape in the tongue photos, which had a great impact on the diagnosis results. With the continuous development of computer science, artificial intelligence technology is increasingly entering the medical field. Some scholars have introduced computer technology into tongue research, mainly by photographing the patient's tongue in a closed, constant light source environment, and then after pre-processing steps such as color correction, tongue segmentation, and tongue coating separation, the tongue color, tongue texture, tongue shape, and other features are identified, and then more specific tongue color, texture and other feature information are obtained, obtaining more scientific and rigorous research results than naked eye observation. For example, Xu Yifeng et al. [3] The correlation between each tongue parameter and different symptoms of type 2 diabetes was analyzed, and it was found that the main symptoms of type 2 diabetes patients had significant differences in tongue (coating) color parameters, tongue crack parameters, and coating peeling parameters, which provided objective and standardized tongue image indicators for TCM diagnosis and syndrome differentiation treatment of type 2 diabetes. In the existing field of intelligent medical technology, there is a lack of research on chronic kidney disease prediction based on tongue coating images to achieve stable and accurate chronic kidney disease prediction. Summary of the invention

[0004] The technical problems to be solved by the present invention are:

[0005] The existing technology lacks a chronic kidney disease prediction method based on tongue coating images.

[0006] The present invention adopts the following technical solutions to solve the above technical problems:

[0007] The present invention provides a method for identifying the type of kidney disease based on tongue coating images, comprising the following steps:

[0008] S1, collecting the patient's tongue coating image data and medical record data, preprocessing the medical record data, inserting the patient's tongue coating image data into the corresponding medical record data, and randomly dividing the data into a training set and a test set;

[0009] S2, constructing a tongue coating image feature extraction model, and using the tongue coating image data to train the tongue coating image feature extraction model;

[0010] S3, using the trained tongue coating image feature extraction model to extract features from the tongue coating image data to obtain a tongue coating feature vector, converting the patient medical record data into a medical record vector, combining the tongue coating feature vector with the corresponding medical record vector to obtain an embedding vector, and using the embedding vector to train the SVM classifier;

[0011] S4. Integrate the trained tongue coating image feature extraction model and SVM classifier to obtain a kidney disease type recognition model based on tongue coating images.

[0012] Furthermore, the preprocessing of the medical record data described in S1 includes deleting features irrelevant to kidney disease, deleting features with a high missing rate, filling in missing values ​​of data features, and deleting features with less correlation.

[0013] Furthermore, the filling of missing values ​​of data features includes filling missing values ​​of data features using mean filling and hot card filling.

[0014] Furthermore, the specific process of deleting the features with less correlation includes performing regression analysis on the data, deleting irrelevant or unimportant features, analyzing the correlation, and optimizing the highly correlated features.

[0015] Furthermore, the tongue coating image feature extraction model described in S2 adopts a VGG network model, which includes 13 convolutional layers, 3 fully connected layers and 5 pooling layers. The convolutional layer uses multiple 3×3 convolutional kernels instead of large convolutional kernels. The number of convolutional kernel channels is consistent with the number of channels of the input image. The resolution of the feature map of the convolutional layer gradually decreases under the action of the pooling layer. According to the position of the pooling layer, the convolutional layer is divided into 5 different blocks, which are numbered block1, block2, block3, block4 and block5 respectively.

[0016] Furthermore, in S2, block1 includes 2 conv3-64 convolutional layers, block2 includes 2 conv3-128 convolutional layers, block3 includes 3 conv3-256 convolutional layers, block4 includes 3 conv3-512 convolutional layers, and block5 includes 3 conv3-512 convolutional layers.

[0017] Furthermore, the kidney disease type output by the kidney disease type recognition model based on tongue coating images described in S4 includes a binary classification of kidney disease and non-kidney disease, or a four-class classification of renal insufficiency, proteinuria, hematuria and non-kidney disease.

[0018] Furthermore, the output of the kidney disease type recognition model based on tongue coating images also includes TCM symptom results.

[0019] The present invention provides a kidney disease type identification system based on tongue coating images. The system has a program module corresponding to the steps of the method described in any one of the above technical solutions, and executes the steps in the above-mentioned kidney disease type identification method based on tongue coating images when running.

[0020] The present invention provides a computer-readable storage medium, which stores a computer program. The computer program is configured to implement the steps in the method for identifying kidney disease types based on tongue coating images described in any one of the above-mentioned technical solutions when called by a processor.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] The present invention provides a method, system and storage medium for intelligently identifying the type of kidney disease based on tongue coating images, which fuses the medical record data and tongue coating image data to obtain richer and more comprehensive features, reduces the differences between different tongue coating image representations, and achieves more stable and accurate prediction and auxiliary judgment of chronic kidney disease. The present invention fully integrates the objectification technology of tongue diagnosis into the diagnosis and treatment prediction process of chronic kidney disease, which facilitates doctors to conduct targeted analysis of lesions, and is of great significance for early intervention of the disease process, disease warning and early treatment of kidney disease patients, and has good application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart of a method for intelligently identifying kidney disease types based on tongue coating images in an embodiment of the present invention;

[0024] Figure 2 Schematic diagram of the VGG16 network model structure in an embodiment of the present invention;

[0025] Figure 3 Schematic diagram of the Resnet50 network structure in an embodiment of the present invention;

[0026] Figure 4 This is part of the tongue coating image data in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, exemplary implementations or embodiments of the present invention will be described below in conjunction with the accompanying drawings. Obviously, the described implementations or embodiments are only implementations or embodiments of a part of the present invention, not all of them. Based on the implementations or embodiments of the present invention, all other implementations or embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present invention.

[0028] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0029] The present invention provides a method for identifying kidney disease types based on tongue coating images. Figure 1 As shown, the following steps are included:

[0030] S1, collecting the patient's tongue coating image data and medical record data, preprocessing the medical record data, inserting the patient's tongue coating image data into the corresponding medical record data, and randomly dividing the data into a training set and a test set;

[0031] S2, constructing a tongue coating image feature extraction model, and using the tongue coating image data to train the tongue coating image feature extraction model;

[0032] S3, using the trained tongue coating image feature extraction model to extract features from the tongue coating image data to obtain a tongue coating feature vector, converting the patient medical record data into a medical record vector, combining the tongue coating feature vector with the corresponding medical record vector to obtain an embedding vector, and using the embedding vector to train the SVM classifier;

[0033] S4. Integrate the trained tongue coating image feature extraction model and SVM classifier to obtain a kidney disease type recognition model based on tongue coating images.

[0034] Characteristics irrelevant to kidney disease, including personal information such as address and mobile phone number, were deleted, as well as characteristics of usual medication during the consultation.

[0035] Delete features with high missing rates.

[0036] The missing values ​​of data features were filled; the missing values ​​of height, weight, waist circumference and hip circumference were filled by the mean filling method; the missing values ​​of biochemical test values ​​were filled by hot card filling, and the most similar samples were found in the data set according to the defined similarity measurement method to fill the missing values.

[0037] Perform regression analysis on the data to delete irrelevant or unimportant features; use the Pearson correlation coefficient to analyze the correlation and optimize the highly correlated features.

[0038] Preferably, the tongue coating image feature extraction model in S2 adopts a VGG16 network model, such as Figure 2 As shown in the figure, the model includes 13 convolutional layers, 3 fully connected layers and 5 pooling layers. The convolutional layer uses multiple 3×3 convolutional kernels instead of large convolutional kernels. The number of convolutional kernel channels is consistent with the number of channels of the input image. The resolution of the feature map of the convolutional layer gradually decreases under the action of the pooling layer. According to the position of the pooling layer, the convolutional layer is divided into 5 different blocks, numbered as block1, block2, block3, block4 and block5 respectively; block1 includes 2 conv3-64 convolutional layers, block2 includes 2 conv3-128 convolutional layers, block3 includes 3 conv3-256 convolutional layers, block4 includes 3 conv3-512 convolutional layers, and block5 includes 3 conv3-512 convolutional layers.

[0039] Preferably, the kidney disease type output by the kidney disease type recognition model based on the tongue coating image described in S4 includes a binary classification of kidney disease and non-kidney disease, or a quadruple classification of renal insufficiency, proteinuria, hematuria and non-kidney disease, and the output of the model also includes the results of TCM symptomology, including: Kidney Qi deficiency: manifested as pale tongue coating and enlarged tongue, indicating kidney essence deficiency. Damp-heat accumulation: manifested as yellow and greasy tongue coating, indicating damp heat in the body, which may be related to the inflammatory response of kidney disease. Kidney Yin deficiency: manifested as red tongue with less coating, indicating kidney Yin deficiency, which may be accompanied by chronic renal damage.

[0040] The intelligent identification method (algorithm) of kidney disease types based on tongue coating images proposed in the present invention is the underlying technical core of the present invention, and various products can be derived based on the algorithm.

[0041] Based on the method proposed in the present invention, a system for intelligently identifying kidney disease types based on tongue coating images is developed using a programming language. The system has program modules corresponding to the steps of the above-mentioned technical solution, and executes the steps in the above-mentioned method for intelligently identifying kidney disease types based on tongue coating images during operation.

[0042] The computer program of the developed system (software) is stored on a computer-readable storage medium, and the computer program is configured to implement the steps of the above-mentioned method for intelligently identifying the type of kidney disease based on tongue coating images when called by a processor, that is, the present invention is materialized on a carrier to become a computer program product.

[0043] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0044] The computer programs (also referred to as programs, software, software applications, or codes) of the present invention include machine instructions for programmable processors, and these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or device (e.g., disk, optical disk, memory, programmable logic device PLD) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0045] Example 1

[0046] The deep learning framework used in the feature extraction part of this embodiment is pytorch, the language is Python 3.10, the integrated development environment is PyCharm 2022.3 version, and all experiments are carried out under the Windows 11 operating platform.

[0047] In this embodiment, the tongue coating image dataset contains a total of 718 images, including 397 images of non-nephropathy, 74 images of renal insufficiency, 184 images of proteinuria, and 63 images of hematuria. Figure 4 shown.

[0048] The feature extraction networks compared in this embodiment include: ResNet50, RESNET101 and EfficientNetB2 networks.

[0049] The Resnet50 network structure is divided into five parts: STAGE 0, STAGE 1, STAGE 2, STAGE 3 and STAGE 4. Figure 3 As shown. The first part has a 7×7×64 convolution, and the last four parts have residual units. Each residual unit contains three layers of convolution. The network has a total of 1+3×(3+4+6+3)=49 convolution layers, plus the final fully connected layer, a total of 50 layers. If the input image size of the ResNet50 network is 224×224, after the convolution operation of the first part, the output is 128×128, and after the maximum pooling layer, the output image size becomes 56×56. The operations of STAGE 1, STAGE 2, STAGE 3 and STAGE 4 are performed in sequence. After each part, the image size is reduced to half of the original size, and the number of channels becomes twice the original. The final output of the network is 2048×4×4.

[0050] The Resnet101 feature extraction network in STAGE3 on resnet50 changes from 6 times to 23 times, namely:

[0051]

[0052] The EfficientNet network uniformly scales the three dimensions of network depth, network width and resolution through a set of fixed scaling factors, improving network performance in these three dimensions. EfficientNetB0 was searched through the neural network structure search technology, and Efficien tNetB2 is a multiple of EfficientNetB0, with a model width and depth of 1.1 and 1.2 times that of EfficientNetB0, respectively, namely:

[0053]

[0054] EfficientNetB2 further increases the width and depth of the network based on EfficientNetB0.

[0055] The sample ratio of the training set and the test set is set to 6:4, and the classification results under different feature extraction networks are obtained in Table 1. The evaluation indicators include classification accuracy, recall rate, f1 coefficient and average accuracy.

[0056] As can be seen from Table 1, the four networks have the highest retrieval rates for images corresponding to hematuria, reaching 100%, 89%, 90%, and 85% respectively, and the average accuracy rates of the four networks are 83%, 80%, 82%, and 80% respectively. Overall, the kidney disease type recognition model of the present invention has a relatively good effect, with an average accuracy rate of 83%, which is at least one percentage point higher than the other three networks.

[0057] Table 1 Nephropathy type recognition model of the present invention

[0058]

[0059] RESNET50

[0060]

[0061]

[0062] RESNET 101

[0063]

[0064] EfficientNetB2

[0065]

[0066] Renal insufficiency, proteinuria, and hematuria are combined into the kidney disease category, resulting in two major categories: kidney disease and no kidney disease. The sample ratio of the training set and the test set is still set to 6:4. Table 2 shows the classification results under different feature extraction networks. The evaluation indicators include classification accuracy, recall rate, f1 coefficient, and average accuracy.

[0067] It can be seen from Table 2 that the classification accuracy of the four networks for renal disease and non-renal disease tongue coating images exceeds 83%. Specifically, in the binary classification task, the effect of the renal disease type recognition model of the present invention is relatively good, with an average accuracy of 93%, which is significantly better than the other three networks.

[0068] Table 2 Nephropathy type recognition model of the present invention

[0069]

[0070] RESNET50

[0071]

[0072]

[0073] RESNET 101

[0074]

[0075] EfficientNetB2

[0076]

[0077] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the protection scope of the present invention.

[0078] The documents cited in this invention include:

[0079] [1] Shi Qiang. Study on the consistency of tongue observation results of different traditional Chinese medicine practitioners [J]. Jiangsu Traditional Chinese Medicine, 2007(08):60-61.

[0080] [2] Ding Jianwen, Wang Yongtai, Xia Yanling. Observation of tongue features in 162 patients with chronic renal insufficiency and its clinical significance [J]. Gansu Traditional Chinese Medicine, 1994, 7(2): 18-19.

[0081] [3] Xu Yifeng, Wang Yiqin, Hao Yiming. Analysis of objective parameters of tongue diagnosis of TCM main symptoms in patients with type 2 diabetes [J]. World Science and Technology-Modernization of Traditional Chinese Medicine, 2021, 23(02): 402-409.

[0082] [4] Gu Youyi, Yang Wen. Observation of tongue features in 118 cases of chronic uremia [J]. Journal of Liaoning College of Traditional Chinese Medicine, 2003, 5(3): 21.

[0083] [5] Zhou Xiaofang, Li Xue, Li Fufeng. Study on the tongue characteristics of chronic renal failure with deficiency and dampness syndrome[J]. Chinese Journal of Traditional Chinese Medicine, 2018, 36(9): 2146-2148.

[0084] [6] Song Jintao, Wang Yaoguang. Study on tongue diagnosis of chronic renal failure - Analysis of 51 cases of tongue diagnosis data [J]. Tianjin Traditional Chinese Medicine, 1992, 9(6): 34.

[0085] [7] Cheng Xiaoxia. Analysis of tongue images in 127 cases of chronic primary glomerular disease[J]. Journal of Traditional Chinese Medicine, 1993, 34(2): -109.

[0086] [8] Wang Yiqin, Li Fufeng, He Liqun, et al. Quantitative analysis of tongue images in patients with chronic renal failure of different syndromes [J]. Journal of Shanghai University of Traditional Chinese Medicine, 2002, 16(4): 38-40.

Claims

1. A method for identifying kidney disease types based on tongue coating images, characterized in that: The steps include: S1, collecting the patient's tongue coating image data and medical record data, preprocessing the medical record data, inserting the patient's tongue coating image data into the corresponding medical record data, and randomly dividing the data into a training set and a test set; S2, constructing a tongue coating image feature extraction model, and using the tongue coating image data to train the tongue coating image feature extraction model; S3, using the trained tongue coating image feature extraction model to extract features from the tongue coating image data to obtain a tongue coating feature vector, converting the patient medical record data into a medical record vector, combining the tongue coating feature vector with the corresponding medical record vector to obtain an embedding vector, and using the embedding vector to train the SVM classifier; S4. Integrate the trained tongue coating image feature extraction model and SVM classifier to obtain a kidney disease type recognition model based on tongue coating images.

2. The method for identifying kidney disease types based on tongue coating images according to claim 1, characterized in that: The medical record data are preprocessed as described in S1, including the deletion of features irrelevant to kidney disease, the deletion of features with a high missing rate, the filling of missing values ​​of data features, and the deletion of features with less correlation.

3. The method for identifying kidney disease types based on tongue coating images according to claim 2, characterized in that: The filling of missing values ​​of data features includes filling the missing values ​​of data features by using mean filling and hot card filling.

4. The method for identifying kidney disease types based on tongue coating images according to claim 3, characterized in that: The specific process of deleting the features with less correlation includes performing regression analysis on the data, deleting irrelevant or unimportant features, analyzing the correlation, and optimizing the highly correlated features.

5. The method for identifying kidney disease types based on tongue coating images according to claim 1, characterized in that: The tongue coating image feature extraction model described in S2 adopts the VGG network model, which includes 13 convolutional layers, 3 fully connected layers and 5 pooling layers. The convolutional layer uses multiple 3×3 convolutional kernels instead of large convolutional kernels. The number of convolutional kernel channels is consistent with the number of channels of the input image. The resolution of the feature map of the convolutional layer gradually decreases under the action of the pooling layer. According to the position of the pooling layer, the convolutional layer is divided into 5 different blocks, which are numbered as block1, block2, block3, block4 and block5 respectively.

6. The method for identifying kidney disease types based on tongue coating images according to claim 5, characterized in that: In S2, block1 includes 2 conv3-64 convolutional layers, block2 includes 2 conv3-128 convolutional layers, block3 includes 3 conv3-256 convolutional layers, block4 includes 3 conv3-512 convolutional layers, and block5 includes 3 conv3-512 convolutional layers.

7. The method for identifying kidney disease types based on tongue coating images according to claim 1, characterized in that: The kidney disease type output by the tongue coating image-based kidney disease type recognition model described in S4 includes a binary classification of kidney disease and non-kidney disease, or a four-class classification of renal insufficiency, proteinuria, hematuria, and non-kidney disease.

8. The method for identifying kidney disease types based on tongue coating images according to claim 7, characterized in that: The output of the kidney disease type recognition model based on tongue coating images also includes TCM symptom results.

9. A kidney disease type recognition system based on tongue coating images, characterized in that: The system has a program module corresponding to the steps of the method described in any one of claims 1 to 8, and executes the steps in the above-mentioned method for identifying the type of kidney disease based on tongue coating images when running.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps in the method for identifying the type of kidney disease based on tongue coating images according to any one of claims 1 to 8 when called by a processor.

Images