A tongue coating and tongue texture separation method based on entropy clustering and convolutional neural network

Abstract

This invention relates to a method for separating tongue texture and coating based on entropy clustering and convolutional neural networks, belonging to the fields of deep learning and image processing technology. It solves the problems of large annotation and computational loads, insufficient accuracy and speed in existing methods for tongue texture and coating separation. The method involves processing the original image, calculating the overall image entropy of the tongue image, fitting the number of cluster centers, performing K-means clustering on the tongue image to obtain clustered tongue image sub-images, labeling them with categories to obtain a dataset, training a convolutional neural network model, and obtaining new clustered tongue image sub-images. The trained model is then used to classify these sub-images, obtaining classification results, which are verified using a similarity index. Once verified, tongue image sub-images of the same category are merged into pixels of the same category, and the images are overlaid to obtain the final classification result. This method significantly reduces the annotation workload and improves the speed and accuracy of the algorithm.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and image processing technology, and in particular to a method for separating tongue texture and tongue coating based on entropy clustering and convolutional neural networks. Background Technology

[0002] In Traditional Chinese Medicine (TCM), tongue diagnosis is a crucial aspect of diagnosis. The appearance of the tongue can indicate a person's health status. The tongue is interconnected with many organs in the body, and the condition of different areas of the tongue corresponds to the condition of each organ. The tongue's appearance is primarily composed of the tongue body and the tongue coating. Understanding the distribution ratio and color of the tongue body and coating greatly aids in the analysis of the tongue's characteristics.

[0003] Current methods for separating tongue body and tongue coating mainly include two approaches: supervised deep learning and traditional image processing methods. Deep learning methods utilize semantic segmentation algorithms to separate tongue body and tongue coating, relying heavily on pixel-level labeling and fine-grained segmentation models. This results in a large amount of annotation work and a large number of network model parameters. Traditional image processing methods, while not requiring a large amount of labeled data, are less robust than deep learning methods and do not perform well in separating tongue body and tongue coating.

[0004] In summary, current methods for separating tongue texture and tongue coating have drawbacks such as large amounts of labeling and computation, and insufficient algorithm accuracy and speed. Summary of the Invention

[0005] Based on the above analysis, the embodiments of the present invention aim to provide a tongue texture and tongue coating separation method based on entropy clustering and convolutional neural networks, in order to solve the problems of large amount of labeling and computation and insufficient robustness of existing tongue texture and tongue coating separation methods.

[0006] The objective of this invention is mainly achieved through the following technical solutions:

[0007] This invention provides a method for separating tongue texture and tongue coating based on entropy clustering and convolutional neural networks, comprising the following steps:

[0008] Obtain a set of tongue images containing information about the tongue's texture and coating; based on the overall image entropy of the tongue image set, obtain the number of cluster centers, and then obtain a sub-set of tongue images after clustering;

[0009] The clustered tongue image sub-atlas is labeled with categories to obtain labeled tongue image sub-atlases, forming a dataset; the categories include tongue texture and tongue coating.

[0010] Using the dataset, the convolutional neural network model is trained to obtain the trained convolutional neural network model;

[0011] The trained convolutional neural network model is used to classify the tongue image sub-images to be classified, and the classification result is obtained; the tongue image sub-images to be classified are obtained by clustering the tongue image images to be classified.

[0012] The classification results are verified, and the tongue image sub-images of the same category in the verified classification results are merged and superimposed to obtain the final classification result.

[0013] Based on further improvements to the above method, a set of tongue imagery containing information on tongue texture and coating is obtained, including:

[0014] The tongue image was photographed to obtain the original image set;

[0015] Based on the lighting conditions during the shooting process, the original image set is color-corrected using CCM to obtain a color-corrected image set;

[0016] The color-corrected image set is resized and fed into a tongue image segmentation network for segmentation to obtain a segmented tongue image set.

[0017] Based on further improvements to the above method, the overall image entropy of the tongue image set is obtained in the following way:

[0018] Convert the color space of the tongue image set from RGB color space to HSV color space;

[0019] For each tongue image in the tongue image set, calculate the two-dimensional entropy H of the tongue image in the H channel. H The image entropy H of the tongue image in the S channel S ;

[0020] For tongue images in the tongue image set, use H H and H S Perform weighted calculations to obtain the overall image entropy.

[0021] Based on a further improvement to the above method, the number of cluster centers is obtained based on the overall image entropy of the tongue image set, including:

[0022] Establish a nonlinear relationship between the number of cluster centers and the overall image entropy;

[0023] Based on the relationship between the number of cluster centers and the degree of aggregation, the nonlinear relationship is fitted to obtain the coefficient of the nonlinear relationship when the degree of aggregation is maximized.

[0024] Based on the nonlinear relationship obtained after fitting and the overall image entropy, the number of cluster centers is calculated;

[0025] Round up the number of cluster centers to get the total number of cluster centers.

[0026] Based on further improvements to the above method, the relationship between the number of cluster centers and the degree of clustering includes:

[0027]

[0028] Where S is the degree of aggregation, N pix It is the number of cluster centers, TP pix This represents the number of pixels in the same cluster.

[0029] By making N pix Minimum, TP pix If the maximum value is found, such that S is maximized, then the final fitting objective is argmax S(N). pix ).

[0030] Based on further improvements to the above method, the clustering yields a tongue image sub-atlas, including:

[0031] Based on the number of cluster centers N, the initial points of K-means clustering, i.e. the initial cluster center positions, are calculated for tongue images in RGB color space.

[0032] Divide the tongue image into a preset number of grids, then calculate the entropy of the pixels in each grid, and take the N grids with the largest entropy as the initial cluster center positions.

[0033] Based on the initial cluster centers, iterative K-means clustering is performed to obtain the clustered tongue image sub-image;

[0034] Each tongue image in the tongue image set is processed to obtain a clustered tongue image subset.

[0035] Based on further improvements to the above method, the clustered tongue image sub-atlas is labeled with categories to obtain labeled tongue image sub-atlases, forming a dataset, including:

[0036] Based on two categories, tongue body and tongue coating, the clustered tongue image sub-images are labeled to obtain labeled tongue image sub-images. The labeled tongue image sub-images are then divided into training set and validation set according to a preset ratio to form a dataset.

[0037] Further improvements to the above method are made to verify the classification results, including:

[0038] The classification results are verified using a similarity index, and the verified classification results are obtained by following these steps:

[0039] Step 1: Input the data of each cluster of tongue image sub-images in the classification results and the thresholds D, E and F of each similarity index; where D, E and F are measured experimentally in advance;

[0040] Step 2: Calculate the overall histogram similarity HB;

[0041] Calculate the total cosine similarity (COS);

[0042] Calculate the overall structural similarity (SSIM);

[0043] Step 3: Determine if HB>D, COS>E, and SSIM>F are satisfied;

[0044] Step 4: If the conditions are met, the verification passes; otherwise, it fails.

[0045] The overall histogram similarity is obtained in the following way:

[0046] Histogram statistics are performed using the RGB histogram of the classification results, and black pixels in the classification results are excluded while the number of color pixels is normalized.

[0047] The normalized histograms of the R, G, and B channels of the classification results can be represented as follows:

[0048]

[0049]

[0050]

[0051] Where, N not-black N represents the number of non-zero pixels. R N G N B These represent the number of pixels under each index; index is an integer between 0 and 255, representing the brightness, and is used as the index of the pixel value under that color component.

[0052] The similarity was calculated using the Bach coefficient, and the histogram similarity ρ between two tongue image sub-images in the classification results was obtained as follows:

[0053]

[0054] Where p and p′ represent the histogram data of the tongue image sub-images of the source region and candidate region in the classification results, respectively. p and p′ contain the R-channel normalized histogram, G-channel normalized histogram and B-channel normalized histogram of the corresponding tongue image sub-image.

[0055] Obtain the histogram similarity between two tongue image sub-images;

[0056] For all tongue image sub-images in the classification results, calculate the histogram similarity of each pair and sum them to obtain the total histogram similarity HB.

[0057] Based on further improvements to the above method, the total cosine similarity is obtained in the following way:

[0058] Divide the tongue image sub-image into m regions, each region having n pixels. Each pixel in the tongue image sub-image is represented by three colors: red, green, and blue. The region feature code of the tongue image sub-image is then calculated using the following method:

[0059]

[0060] Where, n r Let n be the red component value of the i-th pixel's color. g Let n be the green component value of the i-th pixel's color. b The value of the blue component of the color of the i-th pixel;

[0061] When calculating the cosine similarity between two tongue image sub-images, the region feature code of one tongue image sub-image is denoted as A, and the region feature code of the other tongue image sub-image is denoted as B. A and B are then substituted into the cosine similarity calculation formula to calculate the cosine similarity of a specified region between the two tongue image sub-images.

[0062]

[0063] Where i represents the i-th pixel out of n pixels;

[0064] Calculate the cosine similarity of m regions and sum them to obtain the cosine similarity between two tongue image sub-images;

[0065] For all tongue image sub-images in the classification results, calculate the cosine similarity of each pair and sum them to obtain the total cosine similarity (COS).

[0066] Based on further improvements to the above method, the overall structural similarity is obtained in the following way:

[0067] The structural similarity between two tongue image sub-images in the classification results is calculated as follows:

[0068]

[0069] Wherein, the subscripts x and y represent the labels of the two tongue image sub-images, μ represents the brightness contrast and σ represents the contrast contrast;

[0070] For all tongue image sub-images in the classification results, the structural similarity is calculated pairwise to obtain the structural similarity S between each pair of tongue image sub-images. SSIM .

[0071] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0072] 1. This invention calculates the two-dimensional entropy of the image and K-means clustering. It only requires manual annotation of the sub-image type of the cluster, i.e. the tongue texture and tongue coating type, during training. Compared with the fine-grained segmentation annotation in the prior art, the annotation workload of this invention is greatly reduced.

[0073] 2. In its implementation, this invention combines clustering and deep learning algorithms, which reduces the parameters of the algorithm model and improves the speed of the algorithm.

[0074] 3. This invention determines the optimal number of cluster centers by using indicators such as the degree of aggregation, integrates the same features of the image, and performs feature classification in the deep learning classification stage, which greatly improves the accuracy of pixel classification results compared with traditional algorithms.

[0075] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0076] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0077] Figure 1 This is a flowchart of the algorithm of the present invention;

[0078] Figure 2 This is a diagram of the convolutional neural network architecture of the present invention;

[0079] Figure 3 This refers to the SE (squeeze and excitation) module, which is the third stage in the convolutional neural network architecture of this invention.

[0080] Figure 4(a) is the original image of the present invention;

[0081] Figure 4(b) shows the final classification results of this invention. Detailed Implementation

[0082] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0083] Example 1

[0084] A specific embodiment of the present invention discloses a method for separating tongue texture and tongue coating based on entropy clustering and convolutional neural networks. The process of this method is as follows: Figure 1 As shown, it includes the following steps:

[0085] S1. Collect a set of original images containing information about the tongue's texture and coating, and process the original image set to obtain a set of tongue images; calculate the overall image entropy of the set of tongue images; based on the overall image entropy, obtain the number of cluster centers by fitting; based on the number of cluster centers, perform K-means clustering on the set of tongue images to obtain a clustered set of tongue images.

[0086] S2. Label the clustered tongue image sub-images to obtain labeled tongue image sub-images, forming a dataset;

[0087] Using the dataset, the convolutional neural network model is trained to obtain the trained convolutional neural network model;

[0088] A new clustered tongue image sub-image is obtained as the tongue image sub-image to be classified. The trained convolutional neural network model is used to classify the tongue image sub-image to be classified to obtain the classification result.

[0089] S3. Validate the classification results using a similarity index to obtain valid classification results; then merge tongue image sub-images of the same category into pixels of the same category in the valid classification results, overlay the images, and obtain the final classification result.

[0090] Example 2

[0091] Based on Example 1, step S1 can be further refined into the following steps:

[0092] S11. Acquire a raw image set containing information about the tongue's texture and coating, and process the raw image set to obtain a tongue image set. The processing procedure is as follows:

[0093] First, images of the tongue are captured using an image acquisition device to obtain the original image set;

[0094] Then, based on the lighting environment during the acquisition process, the original image is color-corrected using a CCM (color correction matrix) to obtain a color-corrected image set.

[0095] Finally, the color-corrected image set is adjusted to a suitable size and then fed into the tongue image segmentation network for segmentation to obtain the segmented image set, i.e., the tongue image set.

[0096] Specifically, the tongue image segmentation network uses the Unet (U-shaped network) segmentation structure to segment the color-corrected and size-adjusted image set to obtain the tongue image set.

[0097] S12. Calculate the overall image entropy of the obtained tongue image set.

[0098] Entropy is a statistical feature that reflects the amount of information in a system. The more complex the content of an image, the more information it contains, and the greater the entropy of the image. Because the tongue body and coating in a tongue image contain rich information and are complex in color and texture, using entropy to characterize the features of a tongue image is accurate and effective.

[0099] In the RGB color space, brightness has a significant impact on the image. Therefore, by converting the color space of the tongue image from RGB to HSV and using HSV to calculate the two-dimensional entropy of the image, the influence of brightness on the result can be ignored.

[0100] Specifically, when calculating the two-dimensional entropy of an image, we only calculate the two-dimensional entropy of the image determined by the colors and hues we are interested in, and only select the H and S channels in the HSV color space.

[0101] Based on tongue images, channel pixel values ​​are set from 0 to 255. To characterize the spatial features of channel pixel information, the image pixel and its neighborhood information are introduced to form a new feature tuple. The two-dimensional entropy H of the tongue image in channel H is calculated as follows: H :

[0102]

[0103]

[0104]

[0105] In the H channel, i1 represents the pixel value of the i1th pixel (0≤i1≤255), j1 represents the average value of the neighboring pixels of the i1th pixel (0≤j1≤255), f(i1,j1) is the number of pixels under the condition (i1,j1), and j1(k1) is the pixel value of the k1th neighboring pixel of i1 (0≤k1≤8). It is the probability of the feature (i1,j1) appearing in the H channel, width is the width of the tongue image, and height is the height of the tongue image;

[0106] The two-dimensional entropy H of the tongue image in the S channel is calculated as follows: S :

[0107]

[0108]

[0109]

[0110] In the S channel, i2 represents the pixel value of the i2th pixel (0≤i2≤255), j2 represents the average value of the neighboring pixels of the i2th pixel (0≤j2≤255), f(i2,j2) is the number of pixels under the condition (i2,j2), and j2(k2) is the pixel value of the k2th neighboring pixel of i2 (0≤k2≤8). It represents the probability of the feature (i2,j2) appearing in the S channel; width is the width of the tongue image, and height is the height of the tongue image.

[0111] Obtain the two-dimensional entropy H of the H channel image H and the two-dimensional entropy H of the S-channel image S Then, for the tongue images in the tongue image set, H was processed in the following way. H and H S Perform weighted calculations to obtain the overall image entropy:

[0112] H image =a'*H H +b'*H s

[0113] Among them, a' and b' were obtained experimentally, and a' is H. H The weight is set to 0.7; b' is H. S The weight of H is set to 0.3; image The total image entropy.

[0114] S13. Based on the overall image entropy, the number of cluster centers is obtained by fitting.

[0115] Based on the overall image entropy, an accurate fit for the number of cluster centers can be established. A challenge in image clustering analysis is determining the specific number of cluster centers. To achieve pixel classification, we need to cluster with the minimum number of cluster centers while maintaining classification accuracy; that is, using the fewest subimages to accurately separate similar features. The specific implementation involves defining an index related to clustering accuracy. We use the degree of aggregation as the evaluation metric. The optimal number of cluster centers is determined based on the degree of aggregation of pixels of the same category after clustering.

[0116] Based on the overall image entropy, the number of cluster centers is obtained through fitting, including:

[0117] The degree of aggregation is calculated as follows:

[0118]

[0119] Where S is the degree of aggregation, N pix It is the number of cluster centers, TP pix This represents the number of pixels in the same cluster.

[0120] By making N pix Minimum, TP pix If the maximum value is found, such that S is maximized, then the final fitting objective is argmax S(N). pix );

[0121] The aggregation degree S and the overall image entropy H image Between them, by the number of cluster centers N pix To establish a relationship, use the following expression:

[0122] N pix =H image 3 *a+H image 2 *b+H image *c+d

[0123] Where a, b, c, and d are the coefficients to be obtained through fitting;

[0124] By fitting the data, we obtain the coefficients a, b, c, and d corresponding to the maximum degree of aggregation S, and then obtain the number of cluster centers N corresponding to the maximum degree of aggregation S. pix and for N pix Round up to get the number of cluster centers N.

[0125] S14. Based on the number of cluster centers N, perform K-means clustering on the tongue images to obtain the clustered tongue image sub-sets.

[0126] Based on the number of cluster centers N, the initial points for K-means clustering, i.e., the initial cluster center positions, are calculated for the tongue image set in the RGB color space of S11. Specifically, the tongue images in the tongue image set are divided into an s×s grid, and the entropy of the pixels in each grid is calculated. The center coordinates of the first N grids with the largest entropy are taken as the initial cluster center positions.

[0127] In K-means clustering, the metric for clustering is the distance between the H and S indices on the HSV image. For example, based on the obtained initial cluster centers, the K-means clustering algorithm iterates to obtain the clustered tongue image sub-set. The algorithm flow is as follows:

[0128] Step 1: Load the initial cluster centers;

[0129] Step 2: Calculate the Euclidean distance from each sample point to the center point and assign a category number;

[0130] Step 3: Recalculate the cluster centers for clusters of the same category;

[0131] Step 4: Determine if the sum of squares at the cluster center has been reached, i.e., the iteration threshold sum:

[0132] If the target is not reached, return to step 2 to continue iterating; if the target is reached, terminate the algorithm to obtain the tongue image sub-sets for each cluster.

[0133] Preferably, step S2 can be further refined into the following steps:

[0134] S21. Label the clustered tongue image sub-images to obtain labeled tongue image sub-images, forming a dataset.

[0135] Specifically, the tongue image sub-images after clustering are manually labeled according to two categories: tongue body and tongue coating, to obtain labeled tongue image sub-images. The labeled tongue image sub-images are then divided into training and validation sets in a 7:3 ratio to form a dataset.

[0136] S22. Using the dataset, train the convolutional neural network model to obtain the trained convolutional neural network model.

[0137] Specifically, in the process of building the convolutional neural network, ResNet is used as the extraction network, combined with the feature pyramid and attention module, which enables the convolutional neural network to detect fine-grained information.

[0138] During the training of a convolutional neural network, the dataset undergoes the following convolution process:

[0139] Network computing is divided into three stages, such as Figure 2 As shown:

[0140] The first stage samples the features, which is the feedforward of the network. The feedforward network uses ResNet as its backbone. The network architecture used in this stage is ResNet50. The ResNet50 network has five convolutional stages that add residual units through a short-circuit mechanism. Finally, it is processed by the average pooling layer to extract features at different levels of the image. The first stage is a bottom-up path stage, which is mainly to extract features at different levels. The feature maps of different levels, namely c2, c3, c4, and c5, are sent to the second stage for processing.

[0141] The second stage is the top-down path stage, which mainly constructs a new set of features from the features obtained in the first stage. Each feature layer, p2, p3, p4, and p5, is the result of fusing different convolutional layers in ResNet50, ensuring more hierarchical information. The bottom-down feature map is passed through a 1×1 convolutional layer to fuse information from each channel while reducing channel dimensionality. It is then fused with the top-down feature map, followed by a 3×3 convolution to eliminate aliasing.

[0142] The third stage is the feature fusion and attention extraction stage. Here, the features from each layer in the second stage are first processed as follows: Figure 3 The SE (squeeze and excitation) module shown extracts channel attention. For each output channel, the SE module predicts a constant weight, weighting each channel accordingly. Essentially, the SE module performs attention or gating operations along the channel dimension. This attention mechanism allows the model to focus more on the most informative channel features while suppressing less important ones, thus improving the network's performance in recognizing fine-grained features. Through a single deconvolution, the size of each feature map is synchronized to the same size, then concatenated. All concatenated features are then fed into a convolutional network for feature extraction. Finally, the features are flattened and fed into a fully connected network, where a softmax function is used to obtain the probability value for each class, resulting in the classification result.

[0143] S22. Obtain a new clustered tongue image sub-image as the tongue image sub-image to be classified, and use the trained convolutional neural network model to classify the tongue image sub-image to be classified to obtain the classification result.

[0144] Obtain new clustered tongue image sub-images. Re-acquire new original images containing information about the tongue's texture and coating, process these new original images to obtain new tongue image sub-images, calculate the overall image entropy of these tongue image sub-images, and based on this overall image entropy, obtain the number of cluster centers through fitting. Then, based on the number of cluster centers N, perform K-means clustering on the new tongue image sub-images to obtain the clustered tongue image sub-images.

[0145] Preferably, step S3 can be further refined into the following steps:

[0146] S31. Validate the classification results using similarity metrics to obtain valid classification results.

[0147] Regarding result verification, relying solely on human visual verification, which primarily depends on subjective judgment, has numerous drawbacks in terms of accuracy. Therefore, after obtaining the classification results, to ensure their accuracy, three different objective quantitative indicators are needed for verification. These three objective quantitative indicators are histogram similarity, cosine similarity, and structural similarity. The histogram similarity indicator compares the similarity between tongue image sub-images in the classification results in terms of pixel statistics. The cosine similarity indicator measures the cosine similarity of colors between tongue image sub-images in the classification results. The structural similarity indicator, based on the rough judgment that most of the tongue coating is distributed in the center of the tongue and most of the tongue body is distributed in the edges, also considers specific structural information. Combining these different indicators allows for the objective verification of the classification results.

[0148] The overall histogram similarity is obtained in the following way:

[0149] Histogram statistics are performed using the RGB histogram of the classification results, and black pixels in the classification results are excluded while the number of color pixels is normalized.

[0150] The normalized histograms of the R, G, and B channels of the classification results can be represented as follows:

[0151]

[0152]

[0153]

[0154] Where, N not-black N represents the number of non-zero pixels. R N G N B These represent the number of pixels under each index; index is an integer between 0 and 255, representing the brightness, and is used as the index of the pixel value under that color component.

[0155] The similarity was calculated using the Bach coefficient, and the histogram similarity ρ between two tongue image sub-images in the classification results was obtained as follows:

[0156]

[0157] Where p and p′ represent the histogram data of the tongue image sub-images of the source region and candidate region in the classification results, respectively. p and p′ contain the R-channel normalized histogram, G-channel normalized histogram and B-channel normalized histogram of the corresponding tongue image sub-image.

[0158] Obtain the histogram similarity between two tongue image sub-images;

[0159] For all tongue image sub-images in the classification results, calculate the histogram similarity of each pair and sum them to obtain the total histogram similarity HB.

[0160] The cosine similarity index measures the similarity between two vectors by measuring the cosine of the angle between them. The cosine value for 0° is 1, and the cosine value for any other angle is no greater than 1; its minimum value is -1. Therefore, the cosine of the angle between two vectors can determine whether the two vectors point in roughly the same direction. When two vectors point in the same direction, the cosine similarity value is 1; when the angle between the two vectors is 90°, the cosine similarity value is 0; when the two vectors point in completely opposite directions, the cosine similarity value is -1. This result is independent of the length of the vectors, only related to their direction. Cosine similarity is typically used in positive space, so the value is set between 0 and 1. The tongue image sub-images in the classification results are represented as vectors, and the similarity between two tongue image sub-images in the classification results is characterized by calculating the cosine distance between the vectors.

[0161] Here, the points on the tongue image sub-image in the classification results are divided into different subsets. These subsets often belong to isolated points, continuous curves, or continuous regions. After grouping these points into subsets according to regions and extracting the features of the subsets, the features of each subset are used as a feature term of the tongue image sub-image for calculation.

[0162] The total cosine similarity is obtained in the following way:

[0163] Divide the tongue image sub-image into m regions, each region having n pixels. Each pixel in the tongue image sub-image is represented by three colors: red, green, and blue. The region feature code of the tongue image sub-image is then calculated using the following method:

[0164]

[0165] Where, n r Let n be the red component value of the i-th pixel's color. g Let n be the green component value of the i-th pixel's color. b The value of the blue component of the color of the i-th pixel;

[0166] When calculating the cosine similarity between two tongue image sub-images, the region feature code of one tongue image sub-image is denoted as A, and the region feature code of the other tongue image sub-image is denoted as B. A and B are then substituted into the cosine similarity calculation formula to calculate the cosine similarity of a specified region between the two tongue image sub-images.

[0167]

[0168] Where i represents the i-th pixel out of n pixels;

[0169] Calculate the cosine similarity of m regions and sum them to obtain the cosine similarity between two tongue image sub-images;

[0170] For all tongue image sub-images in the classification results, calculate the cosine similarity of each pair and sum them to obtain the total cosine similarity (COS).

[0171] The overall structural similarity was obtained in the following way:

[0172] The structural similarity between two tongue image sub-images in the classification results is calculated as follows:

[0173]

[0174] Wherein, the subscripts x and y represent the labels of the two tongue image sub-images, μ represents the brightness contrast and σ represents the contrast contrast;

[0175] For all tongue image sub-images in the classification results, calculate the structural similarity of each pair and sum them to obtain the total structural similarity SSIM.

[0176] The classification results are verified using a similarity index, and the verified classification results are obtained by following these steps:

[0177] Step 1: Input the data of each cluster of tongue image sub-images in the classification results and the thresholds D, E and F of each similarity index; where D, E and F are measured experimentally in advance;

[0178] Step 2: Calculate the overall histogram similarity HB;

[0179] Calculate the total cosine similarity (COS);

[0180] Calculate the overall structural similarity (SSIM);

[0181] Step 3: Determine if HB>D, COS>E, and SSIM>F are satisfied;

[0182] Step 4: If the conditions are met, the verification passes; otherwise, it fails.

[0183] S32. In the verified classification results, the tongue image sub-images of the same category are merged into pixels of the same category, and the images are superimposed to obtain the final classification result, as shown in Figure 4. Figure 4(a) is the original image, and Figure 4(b) is the final classification result.

[0184] By using the above method, the calculation of image two-dimensional entropy and K-means clustering only requires manual annotation of the sub-image types of the clusters, i.e., tongue texture and tongue coating types, during training. Compared with the fine-grained segmentation annotation in existing technologies, the annotation workload of this invention is greatly reduced. By combining clustering and other algorithms with deep learning algorithms, the parameters of the algorithm model are reduced, while the speed of the algorithm is improved. The optimal number of cluster centers is determined by indicators such as the degree of aggregation, and the same features of the image are integrated. In the deep learning classification stage, feature classification is performed, which greatly improves the accuracy of pixel classification results compared with traditional algorithms.

[0185] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0186] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A tongue coating and tongue texture separation method based on entropy clustering and convolutional neural network, characterized in that, Includes the following steps: Obtain a set of tongue images containing information about the tongue's texture and coating; based on the overall image entropy of the tongue image set, obtain the number of cluster centers, and then obtain a sub-set of tongue images after clustering; The clustered tongue image sub-atlas is labeled with categories to obtain labeled tongue image sub-atlases, forming a dataset; the categories include tongue texture and tongue coating. Using the dataset, the convolutional neural network model is trained to obtain the trained convolutional neural network model; The trained convolutional neural network model is used to classify the tongue image sub-images to be classified, and the classification result is obtained; the tongue image sub-images to be classified are obtained by clustering the tongue image images to be classified. The classification results are verified, and the tongue image sub-images of the same category in the verified classification results are merged and superimposed to obtain the final classification result. The overall image entropy of the tongue image set is obtained in the following way: Convert the color space of the tongue image set from RGB color space to HSV color space; For the tongue image in the tongue image set, respectively calculate the image two-dimensional entropy of the tongue image in H the channel H H and the image two-dimensional entropy of the tongue image in S the channel H S ; Using the tongue images in the tongue image set, H H and H S Perform weighted calculations to obtain the overall image entropy; The number of cluster centers is obtained based on the overall image entropy of the tongue image set, including: Establish a nonlinear relationship between the number of cluster centers and the overall image entropy; Based on the relationship between the number of cluster centers and the degree of aggregation, the nonlinear relationship is fitted to obtain the coefficient of the nonlinear relationship when the degree of aggregation is maximized. Based on the nonlinear relationship obtained after fitting and the overall image entropy, the number of cluster centers is calculated; Round up the number of cluster centers to get the total number of cluster centers.

2. The tongue texture and coating separation method based on entropy clustering and convolutional neural networks according to claim 1, characterized in that, Obtain a set of tongue images containing information about the tongue's texture and coating, including: The tongue image was photographed to obtain the original image set; Based on the lighting conditions during the shooting process, the original image set is color-corrected using CCM to obtain a color-corrected image set; The color-corrected image set is resized and fed into a tongue image segmentation network for segmentation to obtain a segmented tongue image set.

3. The tongue texture and coating separation method based on entropy clustering and convolutional neural networks according to claim 1, characterized in that, The relationship between the number of cluster centers and the degree of clustering includes: in, S It is the degree of aggregation. It is the number of cluster centers. This represents the number of pixels in the same cluster. By making Minimum Maximum, making S If the maximum is reached, then the final fitting objective is... .

4. The tongue texture and coating separation method based on entropy clustering and convolutional neural networks according to claim 3, characterized in that, Clustering yields a tongue image sub-atlas, including: Based on the number of cluster centers N For tongue images in RGB color space, perform K The initial point of mean clustering, i.e. the calculation of the initial cluster center position; The tongue image is divided into a predetermined number of grids, and then the entropy of the pixels in each grid is calculated. The first few pixels are selected. N The grid with the largest entropy is used as the initial cluster center position; Based on the initial cluster centers, K The mean clustering iteration yields the clustered tongue image sub-image; Each tongue image in the tongue image set is processed to obtain a clustered tongue image subset.

5. The tongue texture and coating separation method based on entropy clustering and convolutional neural networks according to claim 4, characterized in that, The clustered tongue image sub-atlas is then labeled with categories to obtain labeled tongue image sub-atlases, forming a dataset, including: Based on two categories, tongue body and tongue coating, the clustered tongue image sub-images are labeled to obtain labeled tongue image sub-images. The labeled tongue image sub-images are then divided into training set and validation set according to a preset ratio to form a dataset.

6. The tongue texture and coating separation method based on entropy clustering and convolutional neural networks according to claim 5, characterized in that, Validating the classification results includes: The classification results are verified using a similarity index, and the verified classification results are obtained by following these steps: Step 1: Input the data of each cluster of tongue image sub-images in the classification results and the threshold values ​​of each similarity index. D , E and F ;in, D , E and F Measured in advance by experiments; Step 2: Calculate the overall histogram similarity HB ; Calculate the total cosine similarity COS ; Calculate the overall structural similarity SSIM ; Step 3: Determine if the condition is met. HB > D , COS > E and SSIM > F ; Step 4: If the conditions are met, the verification passes; otherwise, it fails. The overall histogram similarity is obtained in the following way: Histogram statistics are performed using the RGB histogram of the classification results, and black pixels in the classification results are excluded while the number of color pixels is normalized. RGB of the classification results R Channel normalized histogram G Channel normalized histogram and B The channel-normalized histogram can be represented as follows: in, The number of non-zero pixels. Each index The number of pixels below; index An integer between 0 and 255, representing the brightness, used as the index of the pixel value for that color component; Similarity was calculated using the Bach coefficient, and the histogram similarity between two tongue image sub-images in the classification results was obtained as follows: ρ : in Histogram data representing the tongue image sub-images of the source region and candidate region in the classification results, respectively. Includes corresponding tongue image sub-images R Channel normalized histogram G Channel normalized histogram and B Channel normalized histogram; Obtain the histogram similarity between two tongue image sub-images; For all tongue image sub-images in the classification results, calculate the histogram similarity for each pair of images, and sum them to obtain the total histogram similarity. HB .

7. The tongue texture and coating separation method based on entropy clustering and convolutional neural networks according to claim 6, characterized in that, The total cosine similarity is obtained in the following way: Divide the tongue image into m There are 10 regions, and each region has 100 regions. n If each pixel in the tongue image sub-image is represented by red, green, and blue colors, then the region feature code of the tongue image sub-image is calculated as follows: in, n r For the first i The red component value of the color of each pixel n g For the first i The green component value of the color of each pixel. n b For the first i The blue component value of the color of each pixel; When calculating the cosine similarity between two tongue image sub-images, the region feature code of one of the tongue image sub-images is denoted as... A The region feature code of another tongue image sub-image is denoted as B ,Will A and B Substitute the values ​​into the cosine similarity formula to calculate the cosine similarity of a specified region between two tongue image sub-images: in, i express n The first pixel in the nth pixel i 1 pixel; right m The cosine similarity between the two tongue image sub-images is calculated and summed for each region. For all tongue image sub-images in the classification results, calculate the cosine similarity pairwise and sum them to obtain the total cosine similarity. COS .

8. The tongue texture and coating separation method based on entropy clustering and convolutional neural networks according to claim 6, characterized in that, The overall structural similarity was obtained in the following way: The structural similarity between two tongue image sub-images in the classification results is calculated as follows: Among them, subscript x , y The labels represent the two tongue image sub-images. For brightness contrast, For contrast comparison, and The constant for calculating structural similarity; For all tongue image sub-images in the classification results, calculate the structural similarity of each pair of images and sum them to obtain the total structural similarity. SSIM .

Images