Traditional Chinese medicine tongue image feature extraction method and system based on deep model

Abstract

The invention discloses a method and system for extracting features of tongue images in traditional Chinese medicine based on a deep model, and specifically relates to the technical field of tongue images in traditional Chinese medicine. The method uses a gray-world method to perform color correction on tongue images to ensure the standardization of input images, uses a Faster R-CNN model to accurately locate the tongue area, and uses a UNet model to perform semantic segmentation on the tongue area to generate a high-precision tongue mask image, uses a deep convolutional neural network to extract key features such as tongue quality, tongue coating and tongue cracks, and uses a support vector machine classifier to perform multi-dimensional classification and recognition on these features, and dynamically evaluates and adjusts model parameters according to the degree of coincidence between the segmentation boundary and the real boundary and the resource consumption to ensure continuous optimization of classification and recognition effects; the method effectively solves the problems of being greatly affected by subjective factors, low segmentation accuracy, and incomplete feature extraction in traditional tongue diagnosis, and significantly improves the recognition accuracy and diagnostic reliability of tongue image features.

Description

Technical Field

[0001] The present invention relates to the technical field of TCM tongue images, and in particular to a method and system for extracting TCM tongue image features based on a deep model. Background Art

[0002] Tongue diagnosis in traditional Chinese medicine is an important part of traditional Chinese medicine diagnosis and treatment. It is used to judge the health of the human body by observing the shape, color, coating and other characteristics of the tongue. With the development of modern medical technology, tongue diagnosis has been further scientifically verified and has been developed intelligently by combining technologies such as artificial intelligence. Especially in the past few decades, researchers have proposed the argument of combining tongue diagnosis with AI, and have done a lot of research on the objectification of tongue diagnosis, the development of tongue diagnosis instruments, and intelligent diagnosis and treatment methods.

[0003] Traditional Chinese medicine tongue diagnosis mainly relies on the doctor's naked eye observation, and the diagnosis results are easily affected by the doctor's experience, subjective judgment, and external environment (such as light, temperature, etc.). Existing tongue image segmentation methods, such as threshold method and energy functional-based segmentation methods, are easily affected by the shooting environment, lighting conditions, and other parts of the tongue, resulting in inaccurate segmentation results, which in turn affects the accuracy of subsequent feature extraction. In addition, existing tongue image feature extraction methods are mostly based on shallow image processing technology and fail to fully utilize the advantages of deep learning models, resulting in low recognition rate and incomplete feature extraction when extracting and identifying complex features (such as tongue coating thickness, tongue crack distribution, etc.). Therefore, how to improve the accuracy of tongue image segmentation and feature extraction, especially when dealing with subtle and critical pathological features, is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for extracting features of tongue images in traditional Chinese medicine based on a deep model to address the deficiencies in the background technology.

[0005] In order to achieve the above object, the present invention provides the following technical solution: a method for extracting features of tongue images in traditional Chinese medicine based on a deep model, comprising the following steps:

[0006] S1: The gray world method is used to perform color correction on the tongue image to eliminate the color deviation caused by external environmental factors and obtain a standardized tongue image;

[0007] S2: The Faster R-CNN model is used to locate the tongue body in the standardized tongue image. The Faster R-CNN model generates multiple candidate boxes through a region proposal network, and classifies and regresses each candidate box to determine the tongue body region.

[0008] S3: semantically segmenting the tongue region using a UNet model according to the positioning result to obtain a tongue mask image, wherein the UNet model has a symmetrical structure and a jump connection between an encoder and a decoder;

[0009] S4: After obtaining the tongue mask image, a deep convolutional neural network is used to extract tongue features in the tongue image. The deep convolutional neural network model automatically extracts the color and glossiness features of the tongue, the color, thickness and distribution features of the tongue coating, and the length, width, depth and distribution pattern of the tongue cracks through multi-layer convolution operations;

[0010] S5: inputting the extracted tongue image features into a support vector machine classifier for classification and recognition, wherein the deep convolutional neural network model classifies and recognizes the multi-dimensional features of tongue quality, tongue coating, and tongue cracks by constructing a multi-category classifier;

[0011] S6: During the process of tongue feature classification and recognition, the classification and recognition effects of the tongue image are evaluated based on the degree of fit between the segmentation boundary and the true boundary and the resource consumption of the segmentation and feature extraction algorithm, and whether the model parameters should be adjusted is determined based on the evaluation results.

[0012] Preferably, in S2, the Faster R-CNN model is used to locate the tongue body on the standardized tongue image, specifically:

[0013] The convolutional neural network is used as the feature extraction network of Faster R-CNN. The input tongue image is convolved to extract the high-level feature map. The feature map is input into the region proposal network. Multiple anchor boxes are generated on the feature map through a sliding window. Each anchor box corresponds to a potential tongue position. Binary classification and regression adjustment are performed on each anchor box to output a set of candidate boxes. The candidate boxes are made closer to the real boundary of the tongue through regression adjustment. Non-maximum suppression is performed on the generated candidate boxes to remove the candidate boxes with high overlap and retain the candidate boxes with low overlap. The filtered candidate boxes are compared with the original feature map. Figure 1 The image is input into the classification and bounding box regression network of Faster R-CNN, and each candidate box is further classified to determine whether it belongs to the tongue category, and the position of the candidate box is accurately regressed and adjusted; the classification network outputs the probability value of each candidate box belonging to different categories, and selects the category with the highest probability as the final classification result, and the regression network outputs the bounding box offset value of each candidate box, adjusts the position of the candidate box to make it more consistent with the actual position of the tongue; the final position of the tongue is determined, and the candidate box that belongs to the tongue category and has been regressed is selected as the bounding box of the tongue, which is used as the precise position of the tongue, and the bounding box coordinates are used for subsequent tongue image feature extraction and analysis.

[0014] Preferably, in S3, according to the tongue area image located by the Faster R-CNN model, the UNet model is used to perform semantic segmentation on the area; the UNet model includes a symmetrical structure of an encoder and a decoder, and the low-level features in the encoder are fused with the high-level features in the decoder through jump connections; the encoder part extracts features of the tongue image through a series of convolutional layers, each convolutional layer is followed by an activation function, and after the convolution operation, downsampling is performed through a 2x2 maximum pooling operation to gradually reduce the spatial dimension of the feature map, while increasing the depth of the feature map to capture more contextual information; the decoder part gradually restores the spatial resolution through an upsampling operation, and realizes the fusion of low-level detail information and high-level semantic information by splicing the decoder feature map with the corresponding jump connection feature map; after each splicing operation, the spliced ​​feature map is further processed through a convolution layer to refine the segmentation result; the last layer of the decoder maps the feature map to the target classification number through a 1x1 convolutional layer, and outputs a category probability map for each pixel; the probability map is thresholded to generate a binary tongue mask image.

[0015] Preferably, in S4, after obtaining the tongue mask image, a deep convolutional neural network is used to extract the tongue image features in the tongue image, specifically:

[0016] The tongue mask image generated from the UNet model is input into the deep convolutional neural network, where the mask image only contains the tongue area. The deep convolutional neural network performs multi-layer convolution operations on the tongue image, and uses the convolution kernel to slide and calculate the weighted sum of the local area to generate a feature map, where the initial convolution layer is used to extract low-level features. After each convolution operation, an activation function is applied to set the negative values ​​in the convolution result to zero, introducing the nonlinear ability of the model; a pooling operation is performed to reduce the size of the feature map by taking the maximum value of the local area, reducing the amount of calculation and introducing invariance; the tongue is extracted layer by layer. The high-level features of the image are obtained by convolutional layer, including the color and glossiness of the tongue, the color, thickness and distribution characteristics of the tongue coating, and the length, width, depth and distribution pattern of tongue cracks; the output of the convolutional layer is flattened into a one-dimensional vector as the input of the fully connected layer, and the flattened feature vector is converted into a specific feature category or attribute through the fully connected layer; the output of the fully connected layer is mapped to a specific feature category through an activation function; the feature vector is output, which specifically includes the color and glossiness characteristics of the tongue, the color, thickness and distribution pattern of the tongue coating, and the length, width, depth and distribution pattern of tongue cracks.

[0017] Preferably, in S5, the extracted tongue image features are input into a support vector machine classifier for classification and recognition, specifically:

[0018] The tongue image features extracted from the deep convolutional neural network are organized into a one-dimensional feature vector. The input feature vector is standardized so that its mean is 0 and its variance is 1. According to the complexity of the tongue image features, a kernel function is selected and a training data set is input, including the standardized feature vector and the corresponding category label. The support vector machine model is trained to determine the optimal hyperplane to distinguish different tongue image feature categories. The optimization goal is to maximize the classification interval and minimize the classification error. The standardized test feature vector is input into the trained support vector machine classifier, and the value of the decision function is calculated according to the input feature vector to determine the category to which the tongue image feature belongs. For multi-dimensional tongue image features, the support vector machine classifier classifies and identifies each feature dimension separately, and integrates the classification results to form the final tongue image diagnosis result. The performance of the support vector machine classifier is evaluated by cross-validation or an independent test set, and the parameters of the support vector machine are adjusted according to the verification results.

[0019] Preferably, in S6, during the process of tongue image feature classification and recognition, the classification and recognition effect of the tongue image is evaluated according to the degree of coincidence between the segmentation boundary and the real boundary and the resource consumption of the segmentation and feature extraction algorithm, specifically:

[0020] The edge over-detection coefficient is generated according to the degree of coincidence between the segmentation boundary and the true boundary. The method for obtaining the edge over-detection coefficient is:

[0021] The boundary area of ​​the segmentation result obtained from the segmentation model is recorded as S, and the manually annotated real tongue boundary is recorded as T. The difference between the segmentation area S and the real boundary T is calculated to obtain the over-detected redundant area E, which is expressed as: ; In the formula, S∩T represents the overlapping part of the segmented area and the true boundary, and E represents the area that only exists in the segmented area but not in the true boundary; the area of ​​the excess area E that is over-detected is calculated as ; Calculate the area of ​​the real tongue boundary T and record it as ; The edge over-detection coefficient ODC is expressed as the ratio of the over-detection area to the true boundary area.

[0022] Preferably, the per-pixel calculation time deviation index is generated according to the resource consumption of the segmentation and feature extraction algorithms, and the method for obtaining the per-pixel calculation time deviation index is:

[0023] Perform segmentation and feature extraction on multiple images or batches, recording the total computation time for each image or batch and the total number of pixels in the image , calculate the per-pixel computation time for each image or batch , the expression is: ; is the total computation time for the ith image or batch, For the total number of pixels in the i-th image or batch, calculate the average value of the per-pixel calculation time TPP for all images or batches , the calculation expression is: ; N is the total number of images or batches, and the deviation value of the calculation time per pixel is calculated , the expression is: ; By calculating the ratio of the standard deviation to the mean, the time deviation index TPPD for each pixel is obtained.

[0024] Preferably, the edge over-detection coefficient and the per-pixel calculation time deviation index are converted into a first eigenvector, and the first eigenvector is used as the input of the machine learning model. The machine learning model uses each group of first eigenvectors to predict the classification and recognition effect analysis value labels of the tongue image as a prediction target, and takes minimizing the sum of prediction errors of the classification and recognition effect analysis value labels of all tongue images as a training target. The machine learning model is trained until the sum of prediction errors converges, and the model training is stopped. The classification and recognition effect analysis value of the tongue image is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

[0025] Preferably, the acquired classification and recognition effect analysis value of the tongue image is compared with a pre-set reference threshold of the classification and recognition effect analysis value. If the classification and recognition effect analysis value of the tongue image is greater than or equal to the pre-set reference threshold of the classification and recognition effect analysis value, it means that the classification and recognition effect of the tongue image is good, and a normal image recognition classification signal is generated at this time, and no parameter adjustment is required; if the classification and recognition effect analysis value of the tongue image is less than the pre-set reference threshold of the classification and recognition effect analysis value, it means that the classification and recognition effect of the tongue image is poor, and an abnormal image recognition classification signal is generated at this time, and parameter adjustment is required.

[0026] The present invention also provides a TCM tongue image feature extraction system based on a deep model, including a color correction module, a tongue positioning module, a semantic segmentation module, a feature extraction module, a classification recognition module and a dynamic adjustment module;

[0027] Color correction module: Use the gray world method to perform color correction on the tongue image to eliminate the color deviation caused by external environmental factors and obtain a standardized tongue image;

[0028] Tongue localization module: The Faster R-CNN model is used to locate the tongue in the standardized tongue image. The Faster R-CNN model generates multiple candidate boxes through a region proposal network, and classifies and regresses each candidate box to determine the tongue area.

[0029] Semantic segmentation module: Based on the positioning results, the UNet model is used to perform semantic segmentation on the tongue area to obtain a tongue mask image. The UNet model has a symmetrical structure and jump connection between the encoder and the decoder;

[0030] Feature extraction module: After obtaining the tongue mask image, the deep convolutional neural network is used to extract the tongue image features in the tongue image. The deep convolutional neural network model automatically extracts the color and glossiness characteristics of the tongue, the color, thickness and distribution characteristics of the tongue coating, and the length, width, depth and distribution pattern of the tongue cracks through multi-layer convolution operations;

[0031] Classification and recognition module: The extracted tongue image features are input into the support vector machine classifier for classification and recognition. The deep convolutional neural network model constructs a multi-category classifier to classify and recognize the multi-dimensional features of tongue quality, tongue coating, and tongue cracks;

[0032] Dynamic adjustment module: During the process of tongue feature classification and recognition, the classification and recognition effects of the tongue image are evaluated based on the degree of fit between the segmentation boundary and the true boundary and the resource consumption of the segmentation and feature extraction algorithm, and whether the model parameters should be adjusted is determined based on the evaluation results.

[0033] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0034] 1. The present invention uses deep convolutional neural networks, Faster R-CNN models and UNet models to achieve accurate segmentation and feature extraction of tongue images. It can automatically extract complex features such as tongue quality, tongue coating and tongue cracks, and use support vector machine classifiers to perform multi-dimensional classification and recognition of these features. This method shows significant advantages in processing complex pathological features, significantly improves the accuracy of tongue feature extraction and recognition, and provides a more objective and standardized diagnostic basis for traditional Chinese medicine tongue diagnosis.

[0035] 2. The present invention further improves the stability and robustness of the system by dynamically evaluating and adjusting model parameters. By introducing the edge over-detection coefficient and the per-pixel calculation time deviation index, the system can fine-tune the model according to the effects of segmentation and feature extraction to ensure the accuracy and efficiency of classification and recognition. Finally, based on the feedback mechanism of the machine learning model, when the system detects that the classification and recognition effects are not good, it can automatically adjust the parameters and continuously optimize the model performance. This closed-loop optimization mechanism not only improves the adaptability of the model, but also ensures the continuous efficiency and reliability of the system in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0037] Figure 1 The figure is a flow chart of the method of the present invention.

[0038] Figure 2 This is the flow chart of TCM tongue image feature extraction.

[0039] Figure 3 This is the Faster R-CNN model architecture. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Example 1, please refer to Figure 1 , Figure 2 , Figure 3 As shown, the method for extracting features of tongue images in traditional Chinese medicine based on a deep model described in this embodiment includes the following steps:

[0042] S1: The gray world method is used to perform color correction on the tongue image to eliminate the color deviation caused by external environmental factors and obtain a standardized tongue image;

[0043] S2: The Faster R-CNN model is used to locate the tongue body in the standardized tongue image. The Faster R-CNN model generates multiple candidate boxes through a region proposal network, and classifies and regresses each candidate box to determine the tongue body region.

[0044] S3: semantically segmenting the tongue region using a UNet model according to the positioning result to obtain a tongue mask image, wherein the UNet model has a symmetrical structure and a jump connection between an encoder and a decoder;

[0045] S4: After obtaining the tongue mask image, a deep convolutional neural network is used to extract tongue features in the tongue image. The deep convolutional neural network model automatically extracts the color and glossiness features of the tongue, the color, thickness and distribution features of the tongue coating, and the length, width, depth and distribution pattern of the tongue cracks through multi-layer convolution operations;

[0046] S5: inputting the extracted tongue image features into a support vector machine classifier for classification and recognition, wherein the deep convolutional neural network model classifies and recognizes the multi-dimensional features of tongue quality, tongue coating, and tongue cracks by constructing a multi-category classifier;

[0047] S6: During the process of tongue feature classification and recognition, the classification and recognition effects of the tongue image are evaluated based on the degree of fit between the segmentation boundary and the true boundary and the resource consumption of the segmentation and feature extraction algorithm, and whether the model parameters should be adjusted is determined based on the evaluation results.

[0048] Among them, in S1, the gray world method is used to perform color correction on the tongue image to eliminate the color deviation caused by external environmental factors and obtain a standardized tongue image, specifically:

[0049] Obtaining the original tongue image: Obtaining the original tongue image from the image acquisition device. The image may be affected by external environmental factors such as light source color temperature, light intensity, and shooting angle, resulting in color distortion.

[0050] Convert image to grayscale: Convert the original RGB image to grayscale. Grayscale is achieved by calculating the weighted average of the red, green, and blue channels of each pixel using the formula: ;in, is the pixel value of the grayscale image, , and are the pixel values ​​of the red, green, and blue channels in the RGB image, respectively. Calculate the average grayscale value of all pixels in the grayscale image , which represents the average brightness of the entire image.

[0051] Calculate the average value of the red, green and blue channels in the original image respectively, denoted as , and According to the assumption of the gray world method, the color of each channel is adjusted to make it close to the average gray value of the image. The calculation expression of the color correction coefficient is: ;in, , and They are the correction coefficients for the red, green, and blue channels, respectively. Multiply the red, green, and blue channels of each pixel in the image by the corresponding correction coefficients to generate a corrected RGB image. The specific calculation expression is: ;in, , and is the corrected image channel value; the corrected pixel values ​​are clipped to ensure that they are in the reasonable RGB range (i.e. between 0 and 255). If the value of a channel exceeds this range, it will be clamped between 0 or 255: ; Among them, the clip function is used to limit the pixel value to a specified range.

[0052] The color-corrected red, green, and blue channels are recombined to generate a standardized tongue image. This image eliminates the color deviation caused by external environmental factors and provides a stable data basis for subsequent image processing and feature extraction.

[0053] S2: The Faster R-CNN model is used to locate the tongue body in the standardized tongue image. The Faster R-CNN model generates multiple candidate boxes through a region proposal network, and classifies and regresses each candidate box to determine the tongue body area.

[0054] The color-corrected and normalized tongue image is input into the Faster R-CNN model. This image is a preprocessed RGB image.

[0055] Using a convolutional neural network (such as ResNet or VGG) as the feature extraction network of Faster R-CNN, the input tongue image is convolved to extract high-level feature maps. These feature maps contain important information in the tongue image, such as edges, textures, and shapes.

[0056] The feature map is input into the region proposal network (RPN). RPN is a lightweight network that generates multiple anchor boxes on the feature map through a sliding window, each of which corresponds to a potential tongue position.

[0057] Each anchor box is classified into two categories (whether it contains the tongue) and regressed to output a set of proposals. These proposals include areas that may contain the tongue, and are regressed to make the proposals closer to the actual boundaries of the tongue.

[0058] Perform non-maximum suppression (NMS) on the generated candidate boxes to remove candidate boxes with high overlap and retain fewer but more accurate candidate boxes. Non-maximum suppression sets an overlap threshold to retain only the candidate boxes with the highest score and eliminate redundant detection results.

[0059] The filtered candidate box is combined with the original feature Figure 1The image is then fed into the second part of Faster R-CNN, the classification and bounding box regression network. This network will further classify each candidate box to determine whether it belongs to the "tongue" category and make precise regression adjustments to the position of the candidate box.

[0060] The classification network outputs the probability value of each candidate box belonging to different categories (such as tongue and non-tongue), and selects the category with the highest probability as the final classification result.

[0061] The regression network outputs the bounding box offset value of each candidate box and adjusts the position of the candidate box to make it more consistent with the actual position of the tongue.

[0062] According to the output of the classification and regression network, the final position of the tongue is determined. The candidate box belonging to the "tongue body" category and adjusted by regression is selected as the bounding box of the tongue, that is, the tongue body area. The coordinates of this bounding box are used as the precise position of the tongue for subsequent tongue image feature extraction and analysis.

[0063] Output the bounding box position of the tongue. The bounding box includes the coordinate information of the tongue in the tongue image and identifies the position area of ​​the tongue in the image. According to the determined tongue position, the tongue area is cropped from the standardized tongue image to generate an image containing only the tongue part, which provides input for subsequent tongue feature extraction and analysis.

[0064] S3: According to the positioning results, the UNet model is used to perform semantic segmentation on the tongue area to obtain a tongue mask image. The UNet model has a symmetrical structure and jump connection between the encoder and the decoder.

[0065] The tongue area image located by the Faster R-CNN model is used as input and passed to the UNet model for further processing. This image is cropped from the standardized tongue image and only contains the tongue part.

[0066] In the encoder part, the tongue image is first passed through a series of convolutional layers (usually 3x3 convolution kernels) for feature extraction. Each convolutional layer is followed by an activation function (such as ReLU) to introduce nonlinearity.

[0067] After each convolution operation, a 2x2 maximum pooling operation is performed to gradually reduce the spatial dimension of the feature map through downsampling while increasing the depth of the feature map. The role of downsampling is to capture more contextual information.

[0068] Through multiple convolution and pooling operations, the multi-scale features of the tongue image are extracted layer by layer, and the feature map generated by the encoder gradually shifts from detail information to semantic information.

[0069] In order to preserve the spatial detail information of the image, the output feature map of the encoder is directly passed to the corresponding level of the decoder through a skip connection before each sampling. In this way, the decoder can combine high-level semantic information and low-level detail information to perform refined image restoration.

[0070] In the decoder, the feature map is upsampled (usually using 2x2 deconvolution or bilinear interpolation) to gradually restore its spatial resolution. After each upsampling, the size of the decoder's feature map increases while the depth decreases.

[0071] At each stage after upsampling, the feature map in the decoder is concatenated with the corresponding jump connection feature map to fuse the low-level features of the encoder and the high-level features of the decoder. This fusion enables the decoder to restore the image while retaining more spatial detail information.

[0072] After each splicing operation, the spliced ​​feature map is further processed through a series of convolutional layers to refine the segmentation result.

[0073] In the last layer of the decoder, a 1x1 convolutional layer is used to map the decoder’s feature map to the target classification number, that is, the category of each pixel (such as tongue or background).

[0074] The output mask image is a probability map, where each pixel represents the probability of belonging to the tongue area. By thresholding these probabilities, a binary tongue mask image can be generated, where "1" represents the tongue area and "0" represents the background.

[0075] The generated tongue mask image is subjected to necessary denoising to remove small isolated noise points; optionally, morphological operations (such as erosion and dilation) are applied to smooth the tongue edge and improve the accuracy of segmentation.

[0076] According to the bounding box of the tongue mask image, the tongue image is cropped and standardized to ensure that the tongue image is suitable for subsequent feature extraction and analysis.

[0077] The final tongue mask image shows the precise boundary of the tongue for further tongue feature extraction. This mask image can be used for subsequent image analysis tasks, such as calculating the tongue area, shape features, etc.

[0078] S4: After obtaining the tongue mask image, a deep convolutional neural network is used to extract the tongue image features in the tongue image. The deep convolutional neural network model automatically extracts the color and glossiness characteristics of the tongue, the color, thickness and distribution characteristics of the tongue coating, and the length, width, depth and distribution pattern of the tongue cracks through multi-layer convolution operations.

[0079] The tongue mask image generated from the UNet model is input into the deep convolutional neural network (CNN) as the input data for feature extraction. This mask image only contains the tongue area and removes background interference. The input tongue mask image is convolved, and the convolution kernel (filter) is used to slide on the image to calculate the weighted sum of the local area and generate a feature map. The calculation expression is: ; In the formula, In the Layer, feature map value at position (i,j), The weights of the mth and nth convolution kernels in the lth layer, For the The pixel value of the input image of the layer at position (i+m,j+n), is the bias term of the lth layer, and k is the convolution kernel size (such as 3x3). The initial few layers of convolution are usually used to extract low-level features such as edges, lines, and basic shapes.

[0080] After each convolution operation, the ReLU (Rectified Linear Unit) activation function is applied to set the negative values ​​in the convolution result to 0, introducing the nonlinear ability of the model. The expression is: ; is the feature map value after the ReLU activation function. Pooling operation is performed on the feature map. By taking the maximum value of the local area, the size of the feature map is reduced, the amount of calculation is reduced, and invariance is introduced. The expression is: ; is the feature map value after pooling at the first layer. Through multi-layer convolution operations, the high-level features of the tongue image are extracted layer by layer. These high-level features can capture complex image patterns, such as the color and glossiness of the tongue, the thickness and distribution characteristics of the tongue coating, and the length, width, depth and distribution pattern of the tongue cracks. The expression is: ; For the The convolution feature map of the layer; the cumulative effect of multiple layers of convolution enables CNN to learn more abstract and semantically rich features. The output of the convolution layer is flattened into a one-dimensional vector as the input of the fully connected layer. The flattened feature vector is converted into a specific feature category or attribute through the fully connected layer, and the expression is: ; The output value of the fully connected layer corresponds to a certain attribute of the tongue image feature (such as the color of the tongue or the length of the tongue crack). is the flattened input feature, is the weight parameter of the fully connected layer, is the bias term of the fully connected layer.

[0081] The output of the fully connected layer is mapped to a specific feature category through the Softmax activation function or other classification methods, such as the color category of the tongue, the thickness of the tongue coating, or the severity of the tongue cracks. The calculation expression is: ; is the score of the jth category. The Softmax function is used for multi-classification tasks and outputs the probability value of each category.

[0082] The output feature vector specifically includes: Tongue quality feature: output information such as the color and glossiness of the tongue quality. Tongue coating feature: output the color, thickness and distribution pattern of the tongue coating on the tongue surface. Tongue crack feature: output the length, width, depth and distribution pattern of the tongue crack.

[0083] S5: The extracted tongue image features are input into the support vector machine classifier for classification and recognition. The deep convolutional neural network model accurately recognizes and classifies the multi-dimensional features of tongue quality, tongue coating, and tongue cracks by constructing a multi-category classifier.

[0084] The tongue features extracted from the deep convolutional neural network (CNN) are organized into a one-dimensional feature vector. This feature vector includes: the color and glossiness features of the tongue (e.g., the mean and variance of multiple color channels). The color, thickness and distribution features of the tongue coating (e.g., the statistics of the area covered by the tongue coating and its thickness). The length, width, depth and distribution pattern features of the tongue cracks (e.g., the number of cracks, average length and width, etc.).

[0085] The input feature vector is standardized to make its mean 0 and variance 1 to eliminate the dimensional differences between features and ensure the stability and efficiency of the SVM classifier. The standardized calculation expression is: ; In the formula, is the original eigenvalue, is the mean of feature i, is the standard deviation of feature i, is the standardized feature value. According to the complexity of tongue image features, select the appropriate kernel function. Commonly used kernel functions include: Linear kernel: suitable for linearly separable features.

[0086] Polynomial kernel: Applicable to features with interactive relationships. Radial basis kernel function (RBF): Applicable to nonlinear features, the calculation expression is as follows: ; In the formula, is the width parameter of the kernel function, which controls the smoothness of the decision boundary. Since tongue image features may involve multiple classification tasks (such as tongue color classification, tongue coating thickness classification, etc.), a one-to-many or one-to-one strategy is adopted to expand the SVM into a multi-category classifier.

[0087] Input the training data set (including standardized feature vectors and corresponding category labels), train the SVM model to determine the optimal hyperplane and distinguish different tongue feature categories. Optimization goal: maximize the classification interval (Margin) while minimizing the classification error, expressed as: ; w is the weight vector of the hyperplane, b is the bias term. C is the penalty parameter, which is used to control the slack variable of the model. is a slack variable that allows a certain amount of classification error.

[0088] The normalized test feature vector is input into the trained SVM classifier.

[0089] SVM calculates the value of the decision function based on the input feature vector to determine which category the sample belongs to. For linear SVM, the decision function is: ; f(x) is the classification score, and the category is determined by the sign of the score. For nonlinear SVM, the kernel function is used for mapping and then the decision function is calculated.

[0090] According to the output of the decision function of SVM, determine to which category the input tongue image feature belongs (for example, tongue color classification, tongue coating thickness classification, etc.).

[0091] If the probability output extension of SVM is used (such as Platt scaling), the confidence of the classification results can also be output, providing the probability of each classification result.

[0092] For the multiple dimensions of tongue features (tongue quality, tongue coating, tongue cracks, etc.), the SVM classifier classifies and identifies each feature dimension separately, and then combines these classification results to form the final tongue diagnosis result.

[0093] Different classification results can be weighted according to the importance of different features or expert knowledge to form a final judgment of weighted combination.

[0094] The performance of the SVM classifier is evaluated through cross-validation or independent test sets to determine indicators such as classification accuracy, precision, and recall.

[0095] According to the verification results, adjust the parameters of SVM (such as penalty parameter C and kernel function parameter γ), or use feature selection technology to further optimize the model performance.

[0096] S6: During the process of tongue feature classification and recognition, the classification and recognition effects of the tongue image are evaluated based on the degree of fit between the segmentation boundary and the true boundary and the resource consumption of the segmentation and feature extraction algorithm, and whether the model parameters should be adjusted is determined based on the evaluation results.

[0097] The trained Faster R-CNN and UNet models are used to segment the input tongue image and generate a tongue mask image. The deep convolutional neural network (CNN) is used to extract the features of tongue quality, tongue coating, and tongue cracks from the segmented tongue area to form a feature vector.

[0098] The edge over-detection coefficient is generated according to the degree of coincidence between the segmentation boundary and the true boundary. The method for obtaining the edge over-detection coefficient is:

[0099] The boundary area of ​​the segmentation result obtained from the segmentation model is recorded as S, and the manually annotated real tongue boundary is recorded as T. The difference between the segmentation area S and the real boundary T is calculated to obtain the over-detected redundant area E, which is expressed as: ; where S∩T represents the overlap between the segmented region and the true boundary, and E represents the region that exists only in the segmented region but not in the true boundary; the area (i.e., the number of pixels) of the excess region E that is over-detected is calculated as ; Calculate the area (i.e., number of pixels) of the actual tongue boundary T and record it as ; The edge over-detection coefficient ODC is expressed as the ratio of the over-detection area to the true boundary area.

[0100] The larger the edge over-detection coefficient, the more redundant areas there are in the segmentation results of the tongue image, that is, the segmentation model has significantly over-expanded when detecting the tongue boundary. This situation indicates that the model mistakenly includes the part of the image that does not belong to the tongue in the tongue area, resulting in inaccurate boundary recognition. This inaccurate segmentation directly affects the extraction of tongue image features, which has a negative impact on the classification and recognition effects, may introduce noise features or erroneous features, and ultimately reduce the accuracy and reliability of tongue image feature classification and recognition. Therefore, the larger the edge over-detection coefficient, the worse the classification and recognition effects of the tongue image are.

[0101] The per-pixel calculation time deviation index is generated according to the resource consumption of the segmentation and feature extraction algorithms. The method for obtaining the per-pixel calculation time deviation index is:

[0102] Perform segmentation and feature extraction on multiple images or batches, recording the total computation time for each image or batch and the total number of pixels in the image , calculate the per-pixel computation time for each image or batch , the expression is: ; is the total computation time for the ith image or batch, For the total number of pixels in the i-th image or batch, calculate the average value of the per-pixel calculation time TPP for all images or batches , the calculation expression is: ; N is the total number of images or batches, and the deviation value of the calculation time per pixel is calculated , the expression is: ; By calculating the ratio of the standard deviation to the mean, the per-pixel computation time deviation index TPPD is obtained. The lower the TPPD, the more consistent the per-pixel computation time of the algorithm when processing different images or batches, and the higher the stability. A higher TPPD indicates that the algorithm has large fluctuations in computation time, which may be affected by factors such as image complexity and algorithm branching paths. By analyzing TPPD, the bottlenecks that cause computation time fluctuations in the algorithm can be identified, and the implementation of the algorithm can be optimized, such as adjusting memory management, optimizing branching logic, or improving parallel computing strategies, reducing the variability of computation time, and improving overall processing efficiency.

[0103] The edge over-detection coefficient and the per-pixel calculation time deviation index are converted into the first eigenvector, and the first eigenvector is used as the input of the machine learning model. The machine learning model predicts the classification and recognition effect analysis value label of the tongue image with each group of first eigenvectors as the prediction target, and minimizes the sum of the prediction errors of the classification and recognition effect analysis value labels of all tongue images as the training target. The machine learning model is trained until the sum of the prediction errors converges, and the model training is stopped. The classification and recognition effect analysis value of the tongue image is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

[0104] The method for obtaining the classification and recognition effect analysis value of the tongue image is as follows: from the first feature vector training data of the trained machine learning model, the corresponding function expression is obtained: ; In the formula, is the output function of the model, is the edge over-detection coefficient, The temporal deviation index is calculated for each pixel, It is the analysis value of the classification and recognition effect of tongue images.

[0105] The acquired classification and recognition effect analysis value of the tongue image is compared with the pre-set reference threshold of the classification and recognition effect analysis value. If the classification and recognition effect analysis value of the tongue image is greater than or equal to the pre-set reference threshold of the classification and recognition effect analysis value, it means that the classification and recognition effect of the tongue image is good. At this time, a normal image recognition classification signal is generated and no parameter adjustment is required. If the classification and recognition effect analysis value of the tongue image is less than the pre-set reference threshold of the classification and recognition effect analysis value, it means that the classification and recognition effect of the tongue image is poor. At this time, an abnormal image recognition classification signal is generated and parameter adjustment is required.

[0106] Based on the abnormal signals generated, determine which parameters need to be adjusted. Dynamic adjustment strategies include:

[0107] If your model is slow to converge or not very accurate, you may need to adjust the learning rate. Use Learning Rate Decay or an adaptive optimization method such as Adam to dynamically adjust the learning rate.

[0108] If the model is overfitting, which is manifested as good performance on the training set but poor performance on the test set, you can increase the regularization parameters (such as the L2 regularization coefficient) and reduce the complexity of the model.

[0109] If the classification effect is poor, it may be that the capacity of the model is insufficient or too high. You can adjust the number of layers (depth) of the model or the number of neurons in each layer (width) to find the optimal structure.

[0110] If the classification result is biased towards one category, it may be caused by data imbalance. The bias of the model can be improved by data augmentation, increasing training samples, or adjusting sample weights.

[0111] After adjusting the parameters, retrain the model and perform the classification and recognition of tongue images again. Use the new model to recalculate the effect analysis value and compare it with the reference threshold to verify the performance of the adjusted model. If the model performance improves and meets the preset standard, save the model parameters; otherwise, continue to adjust. It may take multiple rounds of adjustments to achieve the best results. After each adjustment, retrain, verify, and evaluate until the model performance is stable and meets the expected standards. After multiple rounds of dynamic adjustments, select the best performing model for finalization and deploy it in actual applications. During the application process, continue to monitor the classification and recognition effects and make further adjustments as needed.

[0112] In this embodiment, the gray world method is first used to perform color correction on the tongue image to eliminate the color cast of the external environment and obtain a standardized tongue image. Next, the Faster R-CNN model is used to locate the tongue in the image, and a candidate frame is generated through the region proposal network and classified and regressed to determine the tongue area. Subsequently, the UNet model is used to perform semantic segmentation on the located tongue area to generate a tongue mask image. The tongue image features in the mask image are extracted using a deep convolutional neural network, and these features are classified and identified in multiple categories using a support vector machine classifier. In the process of feature classification and identification, the classification and recognition effects are evaluated based on the degree of fit between the segmentation boundary and the true boundary and the resource consumption of the algorithm, and it is determined whether to adjust the model parameters based on the evaluation results to continuously optimize the performance of the model.

[0113] Embodiment 2, the TCM tongue image feature extraction system based on the deep model described in this embodiment includes a color correction module, a tongue positioning module, a semantic segmentation module, a feature extraction module, a classification recognition module and a dynamic adjustment module;

[0114] Color correction module: Use the gray world method to perform color correction on the tongue image to eliminate the color deviation caused by external environmental factors and obtain a standardized tongue image;

[0115] Tongue localization module: The Faster R-CNN model is used to locate the tongue in the standardized tongue image. The Faster R-CNN model generates multiple candidate boxes through a region proposal network, and classifies and regresses each candidate box to determine the tongue area.

[0116] Semantic segmentation module: Based on the positioning results, the UNet model is used to perform semantic segmentation on the tongue area to obtain a tongue mask image. The UNet model has a symmetrical structure and jump connection between the encoder and the decoder;

[0117] Feature extraction module: After obtaining the tongue mask image, the deep convolutional neural network is used to extract the tongue image features in the tongue image. The deep convolutional neural network model automatically extracts the color and glossiness characteristics of the tongue, the color, thickness and distribution characteristics of the tongue coating, and the length, width, depth and distribution pattern of the tongue cracks through multi-layer convolution operations;

[0118] Classification and recognition module: The extracted tongue image features are input into the support vector machine classifier for classification and recognition. The deep convolutional neural network model constructs a multi-category classifier to classify and recognize the multi-dimensional features of tongue quality, tongue coating, and tongue cracks;

[0119] Dynamic adjustment module: During the process of tongue feature classification and recognition, the classification and recognition effects of the tongue image are evaluated based on the degree of fit between the segmentation boundary and the true boundary and the resource consumption of the segmentation and feature extraction algorithm, and whether the model parameters should be adjusted is determined based on the evaluation results.

[0120] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0121] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0122] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0123] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for extracting features of TCM tongue images based on a deep model, characterized by: The steps include: S1: The gray world method is used to perform color correction on the tongue image to eliminate the color deviation caused by external environmental factors and obtain a standardized tongue image; S2: The Faster R-CNN model is used to locate the tongue body in the standardized tongue image. The Faster R-CNN model generates multiple candidate boxes through a region proposal network, and classifies and regresses each candidate box to determine the tongue body region. S3: semantically segmenting the tongue region using a UNet model according to the positioning result to obtain a tongue mask image, wherein the UNet model has a symmetrical structure and a jump connection between an encoder and a decoder; S4: After obtaining the tongue mask image, a deep convolutional neural network is used to extract tongue features in the tongue image. The deep convolutional neural network model automatically extracts the color and glossiness features of the tongue, the color, thickness and distribution features of the tongue coating, and the length, width, depth and distribution pattern of the tongue cracks through multi-layer convolution operations; S5: inputting the extracted tongue image features into a support vector machine classifier for classification and recognition, wherein the deep convolutional neural network model classifies and recognizes the multi-dimensional features of tongue quality, tongue coating, and tongue cracks by constructing a multi-category classifier; S6: In the process of tongue image feature classification and recognition, the classification and recognition effect of the tongue image is evaluated according to the degree of coincidence between the segmentation boundary and the real boundary and the resource consumption of the segmentation and feature extraction algorithm, and whether to adjust the model parameters is determined according to the evaluation results; The step S6 comprises: generating an edge over-detection coefficient according to the degree of coincidence between the segmentation boundary and the real boundary, and the method for obtaining the edge over-detection coefficient is: The boundary area of ​​the segmentation result obtained from the segmentation model is recorded as S, and the manually annotated real tongue boundary is recorded as T. The difference between the segmentation area S and the real boundary T is calculated to obtain the over-detected redundant area E, which is expressed as: ; In the formula, S∩T represents the overlapping part of the segmented area and the true boundary, and E represents the area that only exists in the segmented area but not in the true boundary; the area of ​​the excess area E that is over-detected is calculated as ; Calculate the area of ​​the real tongue boundary T and record it as ; The edge over-detection coefficient ODC is expressed as the ratio of the over-detection area to the true boundary area; The per-pixel calculation time deviation index is generated according to the resource consumption of the segmentation and feature extraction algorithms. The method for obtaining the per-pixel calculation time deviation index is: Perform segmentation and feature extraction on multiple images or batches, recording the total computation time for each image or batch and the total number of pixels in the image , calculate the per-pixel computation time for each image or batch , the expression is: ; is the total computation time for the ith image or batch, For the total number of pixels in the i-th image or batch, calculate the average value of the per-pixel calculation time TPP for all images or batches , the calculation expression is: ; N is the total number of images or batches, and the deviation value of the calculation time per pixel is calculated , the expression is: ; By calculating the ratio of the standard deviation to the mean, the time deviation index TPPD for each pixel is obtained; The edge over-detection coefficient and the per-pixel calculation time deviation index are converted into the first eigenvector, and the first eigenvector is used as the input of the machine learning model. The machine learning model predicts the classification and recognition effect analysis value label of the tongue image with each group of first eigenvectors as the prediction target, and minimizes the sum of the prediction errors of the classification and recognition effect analysis value labels of all tongue images as the training target. The machine learning model is trained until the sum of the prediction errors converges, and the model training is stopped. The classification and recognition effect analysis value of the tongue image is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

2. The method for extracting features of tongue images in traditional Chinese medicine based on a deep model according to claim 1, characterized in that: In S2, the Faster R-CNN model is used to locate the tongue body in the standardized tongue image, specifically: The convolutional neural network is used as the feature extraction network of Faster R-CNN. The input tongue image is convolved to extract high-level feature maps. The feature maps are input into the region proposal network. Multiple anchor boxes are generated on the feature maps through sliding windows. Each anchor box corresponds to a potential tongue position. Binary classification and regression adjustment are performed on each anchor box to output a set of candidate boxes. The candidate boxes are made closer to the real boundary of the tongue through regression adjustment. Non-maximum suppression is performed on the generated candidate boxes to remove candidate boxes with high overlap and retain candidate boxes with low overlap. The filtered candidate boxes are input into Faster together with the original feature maps. R-CNN's classification and bounding box regression network further classifies each candidate box to determine whether it belongs to the tongue category, and performs precise regression adjustment on the position of the candidate box; the classification network outputs the probability value of each candidate box belonging to different categories, and selects the category with the highest probability as the final classification result; the regression network outputs the bounding box offset value of each candidate box, adjusts the position of the candidate box to make it more consistent with the actual position of the tongue; determines the final position of the tongue, selects the candidate box that belongs to the tongue category and has been regressed as the bounding box of the tongue, uses it as the precise position of the tongue, and uses the bounding box coordinates for subsequent tongue image feature extraction and analysis.

3. The method for extracting features of tongue images in traditional Chinese medicine based on a deep model according to claim 2, characterized in that: In S3, based on the tongue area image located by the Faster R-CNN model, the UNet model is used to perform semantic segmentation on the area; the UNet model includes a symmetrical structure of an encoder and a decoder, and the low-level features in the encoder are fused with the high-level features in the decoder through jump connections; the encoder part extracts features of the tongue image through a convolution layer, each convolution layer is followed by an activation function, and after the convolution operation, downsampling is performed through a 2x2 maximum pooling operation to gradually reduce the spatial dimension of the feature map, while increasing the depth of the feature map to capture more context information; the decoder part gradually restores the spatial resolution through an upsampling operation, and realizes the fusion of low-level detail information and high-level semantic information by splicing the decoder feature map with the corresponding jump connection feature map; after each splicing operation, the spliced ​​feature map is further processed through a convolution layer to refine the segmentation result; the last layer of the decoder maps the feature map to the target classification number through a 1x1 convolution layer, and outputs a category probability map for each pixel; the probability map is thresholded to generate a binary tongue mask image.

4. The method for extracting features of tongue images in traditional Chinese medicine based on a deep model according to claim 3, characterized in that: In S4, after obtaining the tongue mask image, the deep convolutional neural network is used to extract the tongue image features in the tongue image, specifically: The tongue mask image generated from the UNet model is input into the deep convolutional neural network, where the mask image only contains the tongue area. The tongue image is subjected to multi-layer convolution operations through the deep convolutional neural network, and the weighted sum of the local area is calculated by sliding the convolution kernel to generate a feature map, wherein the initial convolution layer is used to extract low-level features. After each convolution operation, an activation function is applied to set the negative values ​​in the convolution result to zero, thereby introducing the nonlinear ability of the model. A pooling operation is performed to reduce the size of the feature map by taking the maximum value of the local area, thereby reducing the amount of calculation and introducing invariance. High-level features of the tongue image are extracted layer by layer, including the color and glossiness of the tongue, the color, thickness and distribution characteristics of the tongue coating, and the length, width, depth and distribution pattern of the tongue cracks. The output of the convolution layer is flattened into a one-dimensional vector as the input of the fully connected layer, and the flattened feature vector is converted into a specific feature category or attribute through the fully connected layer. The output of the fully connected layer is mapped to a specific feature category through the activation function. The output feature vector includes the color and glossiness characteristics of the tongue, the color, thickness and distribution pattern of the tongue coating, and the length, width, depth and distribution pattern of the tongue cracks.

5. The method for extracting features of tongue images in traditional Chinese medicine based on a deep model according to claim 4, characterized in that: In S5, the extracted tongue image features are input into the support vector machine classifier for classification and recognition, specifically: The tongue image features extracted from the deep convolutional neural network are organized into a one-dimensional feature vector, and the input feature vector is standardized to make its mean 0 and variance 1. According to the complexity of the tongue image features, a kernel function is selected, and a training data set including the standardized feature vector and the corresponding category label is input. The support vector machine model is trained to determine the optimal hyperplane to distinguish different tongue image feature categories, and the optimization goal is to maximize the classification interval and minimize the classification error; the standardized test feature vector is input into the trained support vector machine classifier, and the value of the decision function is calculated according to the input feature vector to determine the category to which the tongue image feature belongs; For multidimensional tongue image features, the support vector machine classifier classifies and identifies each feature dimension separately, and integrates the classification results to form the final tongue image diagnosis result; the performance of the support vector machine classifier is evaluated through cross-validation or independent test sets, and the parameters of the support vector machine are adjusted according to the verification results.

6. The method for extracting features of tongue images in traditional Chinese medicine based on a deep model according to claim 1, characterized in that: The acquired classification and recognition effect analysis value of the tongue image is compared with the pre-set reference threshold of the classification and recognition effect analysis value. If the classification and recognition effect analysis value of the tongue image is greater than or equal to the pre-set reference threshold of the classification and recognition effect analysis value, it means that the classification and recognition effect of the tongue image is good. At this time, a normal image recognition classification signal is generated and no parameter adjustment is required. If the classification and recognition effect analysis value of the tongue image is less than the pre-set reference threshold of the classification and recognition effect analysis value, it means that the classification and recognition effect of the tongue image is poor. At this time, an abnormal image recognition classification signal is generated and parameter adjustment is required.

7. A system for extracting features of tongue images in traditional Chinese medicine based on a deep model, used to implement the method for extracting features of tongue images in traditional Chinese medicine based on a deep model according to any one of claims 1 to 6, characterized in that: It includes color correction module, tongue positioning module, semantic segmentation module, feature extraction module, classification and recognition module and dynamic adjustment module; Color correction module: Use the gray world method to perform color correction on the tongue image to eliminate the color deviation caused by external environmental factors and obtain a standardized tongue image; Tongue localization module: The Faster R-CNN model is used to locate the tongue in the standardized tongue image. The Faster R-CNN model generates multiple candidate boxes through a region proposal network, and classifies and regresses each candidate box to determine the tongue area. Semantic segmentation module: Based on the positioning results, the UNet model is used to perform semantic segmentation on the tongue area to obtain a tongue mask image. The UNet model has a symmetrical structure and jump connection between the encoder and the decoder; Feature extraction module: After obtaining the tongue mask image, the deep convolutional neural network is used to extract the tongue image features in the tongue image. The deep convolutional neural network model automatically extracts the color and glossiness characteristics of the tongue, the color, thickness and distribution characteristics of the tongue coating, and the length, width, depth and distribution pattern of the tongue cracks through multi-layer convolution operations; Classification and recognition module: The extracted tongue image features are input into the support vector machine classifier for classification and recognition. The deep convolutional neural network model constructs a multi-category classifier to classify and recognize the multi-dimensional features of tongue quality, tongue coating, and tongue cracks; Dynamic adjustment module: During the process of tongue feature classification and recognition, the classification and recognition effects of the tongue image are evaluated based on the degree of fit between the segmentation boundary and the true boundary and the resource consumption of the segmentation and feature extraction algorithm, and whether the model parameters should be adjusted is determined based on the evaluation results.

Images