Color-coded tongue color classification method and device

Abstract

The application provides a tongue color classification method and device based on color coding, the method comprising: obtaining multiple sample tongue images of different tongue color categories, taking the respective tongue color categories as label values; performing tongue body segmentation on each sample tongue image, and converting the segmented sample tongue image to an HSV color space; performing K-means clustering on each sample tongue image to obtain multiple subgraphs contained in each sample tongue image and area proportions of the subgraphs; clustering the subgraphs of all sample tongue images according to color information, so that each subgraph obtains a color classification number; mapping the color classification number of each subgraph to the corresponding sample tongue image to construct the color coding of each sample tongue image; taking the color coding of each sample tongue image and the combination vector composed of the area proportions of the subgraphs in each sample tongue image as variables, and taking the label values of the sample tongue images as results, training a tongue color classifier; and applying the tongue color classifier to tongue color classification of a tongue image to be classified.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and objective technology of TCM inspection, and in particular to a tongue color classification method and device based on color coding. BACKGROUND

[0002] TCM tongue diagnosis is to observe the changes of tongue appearance, understand the physiological function and pathological changes of the body, and plays an important role in TCM clinical diagnosis. Common TCM tongue diagnosis features include tongue color, moss color, thickness of moss, rotten and greasy, moist and dry, and cracks, etc. Among them, tongue color is an important diagnosis feature in TCM diagnosis and treatment. The traditional tongue diagnosis method relies on visual observation and language description of TCM, which is easily affected by the environment and the subjective consciousness of the doctor. Therefore, the objective, modernization and standardization of tongue diagnosis have become an important development direction of TCM tongue diagnosis.

[0003] With the development of computer technology and the continuous maturity of machine learning, machine learning is increasingly applied to intelligent discrimination of tongue color. In the traditional machine learning for tongue color determination, the artificial feature selection and classifier mode are generally adopted. In the artificial feature selection process, the tongue color features of the whole tongue appearance are usually selected for statistical analysis, and the tongue color feature with the highest proportion of colors is analyzed as the tongue color feature of the tongue appearance. Or select the label color block, and calculate the similarity between the tongue color of the to-be-detected tongue appearance and the label color block. However, the visual determination of tongue color is a complex process, and the color with the highest proportion is not necessarily the color of the whole tongue. For the determination of each tongue type, it is not determined by one color, but a combination of colors. The current analysis of tongue color lacks a method consistent with human visual perception, which leads to low tongue color classification accuracy and large difference from the clinical needs. Therefore, it is objectively necessary to establish a tongue color classification method consistent with human visual perception. SUMMARY

[0004] To solve the above problems, the present application provides a tongue color classification method and device based on color coding, which constructs color coding of tongue appearance, trains tongue color classifier based on color coding, and can obtain more accurate tongue color recognition results to assist TCM diagnosis.

[0005] In one aspect, the present application provides a tongue color classification method based on color coding, comprising:

[0006] obtaining a plurality of sample tongue appearances of different tongue color categories, each sample tongue appearance taking a respective tongue color category as a label value;

[0007] segmenting the tongue body of each sample tongue appearance, and converting the segmented sample tongue appearance to HSV color space;

[0008] K-means clustering is performed on each sample tongue image in HSV color space to obtain a plurality of subgraphs contained in each sample tongue image and an area ratio of each subgraph in each sample tongue image;

[0009] The subgraphs of all sample tongue images are clustered according to color information, so that each subgraph obtains a color classification number;

[0010] The color classification number of each subgraph is mapped to the corresponding sample tongue image to construct a color code of each sample tongue image;

[0011] The combination vector composed of the color code of each sample tongue image and the area ratio of each subgraph in each sample tongue image is taken as a variable, and the label value of each sample tongue image is taken as a result, and a tongue color classifier is trained;

[0012] The tongue color classifier is applied to tongue color classification of a to-be-classified tongue image.

[0013] Optionally, the sample tongue images are collected by an image collection device that simulates natural light, and the collection range includes people of all ages, including teenagers, middle-aged people and old people.

[0014] Optionally, the sample tongue images are divided into 5 categories according to tongue color categories, including light white, light red, red, deep red and purple red.

[0015] Optionally, the label value of light white is 1, the label value of light red is 2, the label value of red is 3, the label value of deep red is 4, and the label value of purple red is 5.

[0016] Optionally, the sample amount of sample tongue images of each tongue color category is at least 200.

[0017] Optionally, the K-means clustering of each sample tongue image in HSV color space includes:

[0018] Each sample tongue image is divided into n1 subgraphs by clustering adjacent regions with similar colors into the same region, and n1 is greater than or equal to the number of tongue color categories plus 1;

[0019] Each subgraph is named to record the subgraph contained in each sample tongue image and the area ratio of each subgraph in each sample tongue image.

[0020] Optionally, the clustering of the subgraphs of all sample tongue images according to color information so that each subgraph obtains a color classification number includes:

[0021] The subgraphs of all sample tongue images except the background subgraph are put into a database;

[0022] The clustering algorithm is used to color classify all subgraphs in the database, and the subgraphs are divided into n2 classes according to the image quality of the subgraphs, where n2 is less than or equal to n1 minus 1.

[0023] Optionally, the color coding of each sample tongue appearance is obtained by arranging the color classification numbers of each subgraph of the sample tongue appearance in descending order according to the area proportion of each subgraph.

[0024] Optionally, the tongue color classifier is trained by using an XGBoost method.

[0025] In another aspect, the present application provides a tongue color classification device based on color coding, comprising:

[0026] An acquisition module is configured to acquire a plurality of sample tongue appearances of different tongue color categories, each sample tongue appearance taking a respective tongue color category as a label value;

[0027] A segmentation and conversion module is configured to segment each sample tongue appearance into a tongue body and convert the segmented sample tongue appearance to an HSV color space;

[0028] A first clustering module is configured to perform K-means clustering on each sample tongue appearance in the HSV color space, to obtain a plurality of subgraphs contained in each sample tongue appearance and an area proportion of each subgraph in each sample tongue appearance;

[0029] A second clustering module is configured to cluster the subgraphs of all sample tongue appearances according to color information, so that each subgraph obtains a color classification number;

[0030] A mapping module is configured to map the color classification number of each subgraph to the corresponding sample tongue appearance, to construct a color coding of each sample tongue appearance;

[0031] A training module is configured to train a tongue color classifier by taking a combination vector composed of the color coding of each sample tongue appearance and the area proportion of each subgraph in each sample tongue appearance as a variable, and taking the label value of each sample tongue appearance as a result;

[0032] A classification module is configured to apply the tongue color classifier to tongue color classification of a tongue appearance to be classified.

[0033] The application provides a tongue color classification method based on color coding, acquires multiple sample tongue images of different tongue color categories, each sample tongue image taking a respective tongue color category as a label value; performs tongue body segmentation on each sample tongue image, and converts the segmented sample tongue image to an HSV color space; performs K-means clustering on each sample tongue image in the HSV color space, to obtain multiple subgraphs contained in each sample tongue image and an area proportion of each subgraph in each sample tongue image; clusters the subgraphs of all sample tongue images according to color information, so that each subgraph obtains a color classification number; maps the color classification number of each subgraph to the corresponding sample tongue image, to construct a color coding of each sample tongue image; takes a combination vector composed of the color coding of each sample tongue image and the area proportion of each subgraph in each sample tongue image as a variable, and takes the label value of each sample tongue image as a result, to train a tongue color classifier; and applies the tongue color classifier to tongue color classification of a tongue image to be classified. The application can effectively improve the accuracy of tongue color automatic analysis, and provides a new idea for machine learning in traditional Chinese medicine tongue color classification. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 FIG. 1 is a flowchart of a tongue color classification method based on color coding according to an embodiment of the application;

[0035] Figure 2 FIG. 2 is a diagram of subgraphs obtained by clustering analysis according to an embodiment of the application;

[0036] Figure 3 FIG. 3 is a diagram of a tongue color classification device based on color coding according to an embodiment of the application. DETAILED DESCRIPTION

[0037] To make the objectives, technical solutions and advantages of the embodiments of the application clearer, the technical solutions in the embodiments of the application will be described below in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the application.

[0038] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0039] Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The following examples and features in the examples can be combined with each other without conflict.

[0040] An embodiment of the present application provides a color coding-based tongue image and tongue color classification method, as shown in Figure 1 The method comprises steps S101-S107.

[0041] In step S101, a plurality of sample tongue images of different tongue color categories are obtained, each sample tongue image being labeled with a respective tongue color category.

[0042] Collect standard image samples of each TCM tongue color as sample tongue images. The collected sample tongue images are collected by an image collection device with simulated natural light. The collection range includes people of different age groups, such as teenagers, middle-aged people and elderly people.

[0043] The collected sample tongue images are divided into 5 categories according to tongue color categories, including light white, light red, red, deep red and purple red. A label value is designed for each tongue color category, represented by a number. For details, refer to Table 1 below.

[0044] Table 1

[0045]

[0046]

[0047] Each sample tongue image of the above tongue color category needs to contain a sample amount of more than 200 samples, and the tongue color category of the sample tongue image is labeled by three or more TCM physicians.

[0048] In step S102, the tongue body of each sample tongue image is segmented, and the segmented sample tongue image is converted to HSV color space.

[0049] The tongue body of the sample tongue image is segmented by manual or automatic method to obtain the tongue body region, and the edge of the tongue body is ensured to be complete. The segmented sample tongue image is converted to HSV color space corresponding to human eye perception.

[0050] In step S103, K-means clustering is performed on each sample tongue image in the HSV color space to obtain a plurality of subgraphs contained in each sample tongue image and an area ratio of each subgraph in each sample tongue image.

[0051] The K-means clustering analysis of each sample tongue image in the HSV color space is as follows:

[0052] S1, in the data set, randomly select n objects, each object represents the initial mean (center) of a cluster;

[0053] S2, for each object, respectively calculate the Euclidean distance (in fact, it is to calculate the similarity) of each object to the center of each cluster, and distribute it to the most similar cluster;

[0054] S3, update the cluster center. For each cluster, it is to recalculate the mean of each cluster according to the current objects in the cluster, and then take the mean as the center of the new cluster.

[0055] S4, then repeat steps S2 and S3 until the assignment is stable.

[0056] That is, each sample tongue image is clustered according to the same region of adjacent regions with similar colors. Generally, the K-means clustering analysis is set to n1 categories, and finally a sample tongue image is divided into n1 subgraphs. Because the tongue includes tongue color and coating color, when the human eye observes the tongue color, it will filter out the coating or determine through the coating. The coating color is also an influencing factor for determining the tongue color. Therefore, the whole tongue is taken as input for clustering. The number of categories n1 can be set according to the image quality of the sample tongue image. n1 is greater than or equal to the number of categories of the classifier plus 1. According to the image quality of the sample tongue image, n1 can be appropriately taken as a larger value. If the image color quality of the sample tongue image is low, n1 can be appropriately taken as a smaller value. For example, the number of categories of the classifier is 5, so n1 is taken as 6 at least. In this embodiment, n is taken as 9.

[0057] According to the K-means clustering result, each sample tongue image can be divided into 9 subgraphs, which are specifically referred to Figure 2 The main purpose of obtaining the subgraphs is: first, to obtain the tongue color zoning graph, and second, to decompose the tongue color to obtain the tongue color database. Remove the background subgraph, and name the remaining 8 subgraphs as (A1, A2, A3, A4, A5, A6, A7, A8). Record the subgraph contained in each sample tongue image and the area ratio of each subgraph in each sample tongue image.

[0058] At step S104, the subgraphs of all sample tongue images are clustered according to color information, so that each subgraph obtains a color classification number.

[0059] The subgraphs of all sample tongue images except the background subgraph are put into a database, and a clustering algorithm is used to classify the subgraphs in the database based on color information, so that each subgraph obtains a color classification number. The subgraphs are divided into n2 classes according to the image quality of the subgraphs, and n2 is less than or equal to n1 minus 1. The color classification number is denoted as (01, 02, …, 0n2). In this embodiment, the subgraphs are divided into 8 classes, and a color classification number (01, 02, …, 08) is set for each color class.

[0060] At step S105, the color classification number of each subgraph is mapped to the corresponding sample tongue image to construct the color coding of each sample tongue image.

[0061] The color classification number of each subgraph is mapped to each sample tongue image according to the mapping relationship between each sample tongue image and the subgraph, and the color coding of each sample tongue image is constructed. The color coding of each sample tongue image is obtained by arranging the color classification numbers of the subgraphs of the sample tongue image in descending order according to the area proportion of each subgraph. Table 2 lists the label values of three sample tongue images and the color classification numbers and area proportions of their subgraphs.

[0062] Table 2

[0063]

[0064] As can be seen from Table 2, the color coding of sample tongue N1 is [1 7 6 5 7 3 4 2], which is a combination vector composed of the color coding of the sample tongue and the area proportion of each subgraph. The combination vector is one-to-one corresponding to the label value of the sample tongue image.

[0065] At step S106, the combination vector composed of the color coding of each sample tongue image and the area proportion of each subgraph in each sample tongue image is used as a variable, and the label value of each sample tongue image is used as a result to train the tongue color classifier.

[0066] Training the tongue color classifier includes training the feature data of the color coding of each sample tongue image and the area proportion of the subgraph corresponding to the color coding, and the tongue color category of each tongue image is labeled as a label value, which is represented by a number. The tongue color classifier is trained by using the XGBoost method.

[0067] At step S107, the tongue color classifier is applied to classify the tongue color of the tongue image to be classified.

[0068] The tongue color classification method based on color coding provided by the embodiment of the present application comprises the following steps: obtaining a plurality of sample tongue images of different tongue color categories, each sample tongue image taking a respective tongue color category as a label value; performing tongue body segmentation on each sample tongue image and converting the segmented sample tongue image to an HSV color space; performing K-means clustering on each sample tongue image in the HSV color space to obtain a plurality of subgraphs contained in each sample tongue image and an area proportion of each subgraph in each sample tongue image; clustering the subgraphs of all sample tongue images according to color information so that each subgraph obtains a color classification number; mapping the color classification number of each subgraph to a corresponding sample tongue image to construct a color coding of each sample tongue image; taking a combination vector composed of the color coding of each sample tongue image and the area proportion of each subgraph in each sample tongue image as a variable and taking the label value of each sample tongue image as a result to train a tongue color classifier; and applying the tongue color classifier to tongue color classification of a tongue image to be classified.

[0069] In another aspect, the embodiment of the present application further provides a tongue image tongue color classification device based on color coding, as shown in the accompanying drawings, the device comprises: Figure 3

[0070] The obtaining module 301 is configured to obtain a plurality of sample tongue images of different tongue color categories, each sample tongue image taking a respective tongue color category as a label value.

[0071] The segmentation and conversion module 302 is configured to perform tongue body segmentation on each sample tongue image and convert the segmented sample tongue image to an HSV color space.

[0072] The first clustering module 303 is configured to perform K-means clustering on each sample tongue image in the HSV color space to obtain a plurality of subgraphs contained in each sample tongue image and an area proportion of each subgraph in each sample tongue image.

[0073] The second clustering module 304 is configured to cluster the subgraphs of all sample tongue images according to color information so that each subgraph obtains a color classification number.

[0074] The mapping module 305 is configured to map the color classification number of each subgraph to a corresponding sample tongue image to construct a color coding of each sample tongue image.

[0075] The training module 306 is configured to take a combination vector composed of the color coding of each sample tongue image and the area proportion of each subgraph in each sample tongue image as a variable and take the label value of each sample tongue image as a result to train a tongue color classifier.

[0076] The classification module 307 is configured to apply the tongue color classifier to tongue color classification of a tongue image to be classified.

[0077] ​The tongue image and tongue color classification device based on color coding provided in this embodiment of the invention is used to execute the above method embodiment. Its specific process and details can be referred to the above method embodiment, and will not be repeated here.

[0078] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0079] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for classifying tongue appearance and color based on color coding, characterized in that, The method includes: Obtain multiple sample tongue images of different tongue color categories, with each sample tongue image labeled with its respective tongue color category; For each sample tongue image, the tongue body is segmented, and the segmented sample tongue images are converted to the HSV color space; K-means clustering was performed on each sample tongue image in the HSV color space to obtain multiple sub-images contained in each sample tongue image and the area ratio of each sub-image in each sample tongue image. All sample tongue images are clustered according to color information, so that each sub-image is assigned a color classification number; The color classification number of each sub-image is mapped to the corresponding sample tongue image according to the mapping relationship between each sample tongue image and the sub-image, thus constructing the color code of each sample tongue image; The tongue color classifier is trained using a combination vector consisting of the color code of each sample tongue image and the area ratio of each sub-image in each sample tongue image as variables, and the label value of each sample tongue image as the result. The tongue color classifier described above is used to classify the tongue images to be classified by tongue color.

2. The method according to claim 1, characterized in that, The sample tongue images were acquired by an image acquisition device that simulates natural light, and the acquisition range included people of all ages, including teenagers, middle-aged people, and the elderly.

3. The method according to claim 1, characterized in that, The sample tongue images were divided into 5 categories according to tongue color: pale white, pale red, red, crimson, and purplish red.

4. The method according to claim 3, characterized in that, The label value for pale white is 1, for pale red it is 2, for red it is 3, for crimson it is 4, and for purplish red it is 5.

5. The method according to claim 1, characterized in that, The sample size for each tongue color category must be at least 200.

6. The method according to claim 1, characterized in that, The step of performing K-means clustering on each sample tongue image in the HSV color space to obtain multiple sub-images contained in each sample tongue image and the area ratio of each sub-image in each sample tongue image includes: Clustering is performed based on adjacent areas with similar colors, and each sample tongue image is divided into n1 sub-images, where n1 is greater than or equal to the number of categories of tongue color plus 1; Each sub-image is named, and the sub-images contained in each sample tongue image and the area ratio of each sub-image in each sample tongue image are recorded.

7. The method according to claim 6, characterized in that, The process of clustering sub-images of all sample tongue images according to color information, so that each sub-image obtains a color classification number, includes: All sample tongue images, excluding the background sub-image, are placed into the database; A clustering algorithm is used to classify all sub-images in the database by color. The sub-images are divided into n2 classes according to their image quality, where n2 is less than or equal to n1 minus 1.

8. The method according to claim 1, characterized in that, The color code for each sample tongue image is obtained by arranging the color classification numbers of each sub-image of the sample tongue image in descending order of the area percentage of each sub-image.

9. The method according to claim 1, characterized in that, The tongue color classifier was trained using the XGBoost method.

10. A tongue image and tongue color classification device based on color coding, characterized in that, The device includes: The acquisition module is used to acquire multiple sample tongue images of different tongue color categories, with each sample tongue image using its own tongue color category as a label value; The segmentation and conversion module is used to segment the tongue body of each sample tongue image and convert the segmented sample tongue image to the HSV color space. The first clustering module is used to perform K-means clustering on each sample tongue image in the HSV color space to obtain multiple sub-images contained in each sample tongue image and the area ratio of each sub-image in each sample tongue image. The second clustering module is used to cluster the sub-images of all sample tongue images according to color information, so that each sub-image obtains a color classification number; The mapping module is used to map the color classification number of each sub-image to the corresponding sample tongue image, and construct the color code of each sample tongue image; The training module is used to train a tongue color classifier by using a combination vector consisting of the color code of each tongue image and the area ratio of each sub-image in each tongue image as variables, and the label value of each tongue image as the result. The classification module is used to classify the tongue images to be classified by applying the tongue color classifier.

Images