Traditional Chinese medicine tongue appearance intelligent diagnosis model construction method, diagnosis system, device and storage medium

Abstract

This invention discloses a method, system, device, and storage medium for constructing an intelligent diagnostic model for TCM tongue imagery, addressing the shortcomings of traditional tongue diagnosis, such as strong subjectivity, difficulty in quantification, and the crude feature extraction, weak integration with TCM theory, and lack of personalized intervention in existing AI systems. Through structured guidance, it achieves a closed-loop health management process encompassing tongue image collection, tongue coating localization, multi-dimensional feature extraction, constitution identification, and personalized dietary intervention. High-precision segmentation of the tongue and tongue coating, along with structured extraction of multi-dimensional tongue image features, enables high accuracy in constitution identification, generating personalized dietary recommendations and dynamic health management, thus improving the objectivity, repeatability, and clinical applicability of TCM tongue diagnosis. Users only need to take a picture of their tongue; the system outputs a complete analysis report within seconds, achieving "diagnosis upon tongue image capture." This is applicable to diverse scenarios such as home self-examination, community health monitoring, and remote TCM consultation, promoting the widespread adoption of TCM tongue diagnosis technology.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of traditional Chinese medicine information technology and artificial intelligence, specifically to a method for constructing an intelligent diagnostic model of tongue appearance in traditional Chinese medicine, a diagnostic system, a device, and a storage medium. Background Technology

[0002] Tongue diagnosis in Traditional Chinese Medicine (TCM) is an important component of the four diagnostic methods of "inspection, auscultation, inquiry, and palpation." By observing the color, thickness, moisture, and distribution of the tongue coating and body, it can help determine the functional state of the internal organs and any imbalances in the patient's constitution, providing important evidence for health management and disease prevention. However, traditional tongue diagnosis relies heavily on the physician's clinical experience and subjective judgment, resulting in low standardization, difficulty in quantification, and a long learning curve. Furthermore, the difficulty in objectively recording and longitudinally tracking tongue appearance limits its large-scale application in chronic disease management and preventive medicine.

[0003] To overcome the aforementioned limitations, computer vision-based tongue diagnosis auxiliary analysis systems have emerged in recent years. However, existing technologies still have the following shortcomings: First, the feature extraction dimensions are limited and the integration with traditional Chinese medicine (TCM) theories is insufficient: most existing solutions only extract low-level visual features such as tongue color or texture, failing to deeply integrate TCM tongue diagnosis theories (such as tongue coating color differentiation, organ-specific differentiation, and moistness / dryness differentiation) to construct a systematic feature system. This creates a semantic gap between the extracted features and the logic of TCM clinical diagnosis, making it difficult to comprehensively and objectively reflect the pathological information contained in the tongue appearance, thus limiting the accuracy and interpretability of subsequent analysis.

[0004] Second, the constitution assessment mechanism is rigid and cannot handle complex constitutions: Existing methods usually use a "one-to-one" single mode to output constitution type. When users exhibit a complex state of multiple constitution tendencies, the system often cannot give a reasonable judgment, which does not match the common complex constitution situations in TCM clinical practice, thus reducing the practical value of the diagnostic results.

[0005] Third, there is a lack of personalized dynamic closed-loop management mechanisms: most existing technologies stop at a single physical condition identification, and cannot track and predict the trend of changes in a user's physical condition based on historical analysis data. It is even more difficult to dynamically optimize subsequent health intervention plans based on this, and thus fail to form a true "assessment-intervention-feedback-reassessment" closed-loop health management, resulting in limited personalized intervention effects.

[0006] Therefore, there is an urgent need for a tongue diagnosis technology that can deeply integrate traditional Chinese medicine theory, achieve multi-dimensional feature extraction, support intelligent determination of complex constitutions, and have dynamic closed-loop management capabilities.

[0007] The information disclosed in this background section is intended only to enhance the understanding of the background technology of this disclosure and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0008] The purpose of this invention is to provide a method, device and application for intelligent analysis of tongue appearance in traditional Chinese medicine, which can overcome the shortcomings of traditional tongue diagnosis, such as strong subjectivity, low standardization, difficulty in quantification, and the crude feature extraction of existing AI systems, weak integration with traditional Chinese medicine theory and lack of personalized intervention. It can achieve high-precision segmentation of tongue body and tongue coating, structured extraction of multi-dimensional tongue appearance features, composite constitution identification based on traditional Chinese medicine constitution theory, and personalized dietary therapy recommendations and dynamic health management that integrate knowledge graphs, thereby improving the objectivity, repeatability and clinical applicability of traditional Chinese medicine tongue diagnosis.

[0009] According to one aspect of this disclosure, a method for constructing a TCM tongue diagnosis intelligent model is provided, comprising: A standardized set of tongue image samples was obtained, with each sample labeled with the tongue body region, tongue coating region, and corresponding TCM constitution type label. The standardized tongue image samples are used to locate the tongue body region and segment the tongue coating region to obtain the segmentation mask of the tongue coating region; Based on the segmentation mask of the tongue coating region, a multi-dimensional tongue coating feature vector containing color features, thickness features, dryness features, and distribution pattern features is extracted to form a training dataset. An initial constitution classification model is constructed. The initial constitution classification model adopts a multilayer perceptron network structure. Its input layer dimension matches the dimension of the multi-dimensional tongue coating feature vector, and its output layer dimension matches the number of preset TCM constitution types. The initial constitution classification model is trained using the training dataset to obtain a trained constitution classification model; wherein, the trained constitution classification model is used to generate the probability distribution of each TCM constitution type based on the input tongue image, so as to determine the constitution judgment result by combining a preset threshold.

[0010] Furthermore, the multi-dimensional tongue coating feature vector is a 64-dimensional feature vector, wherein: The color features are generated by calculating the probability of each typical moss color category in a preset TCM moss color semantic space and combining it with color perception parameters. The thickness feature is calculated based on the area ratio of the tongue coating region mask and the texture density of the tongue coating region. The aforementioned dryness and moisture characteristics are calculated based on the gloss and texture characteristics of the tongue coating area, combined with the TCM theory of dryness and moisture differentiation. The distribution pattern features are generated by dividing the tongue body into regions based on the theory of organ classification on the tongue surface in traditional Chinese medicine, and then calculating the distribution ratio of tongue coating in each region and the differences in tongue coating characteristics in each region.

[0011] According to another aspect of this disclosure, a traditional Chinese medicine tongue diagnosis system is provided, comprising: The image preprocessing unit is used to perform adaptive optimization processing on the acquired raw tongue image to generate standardized tongue image data. The tongue image analysis unit includes a target detection model and a feature extraction module. The target detection model is used to locate the tongue body region and segment the tongue coating region of the standardized tongue image data, and output the segmentation mask of the tongue coating region. The feature extraction module is used to extract a multi-dimensional tongue coating feature vector containing color features, thickness features, dryness features, and distribution pattern features based on the segmentation mask of the tongue coating region. The constitution identification unit includes a constitution classification model constructed using the method described above; the constitution classification model is used to generate the probability distribution of each TCM constitution type based on the multi-dimensional tongue coating feature vector, and combined with a preset threshold, to determine the constitution identification result containing the main constitution type or the complex constitution type. The intervention plan generation unit is used to match and generate personalized health intervention plans from a preset TCM dietary therapy knowledge base based on the constitution identification results.

[0012] Furthermore, when determining the body constitution identification result, the body constitution identification unit performs the following operations: When the probability value of the constitution type corresponding to the highest probability output by the constitution classification model is higher than the preset threshold, the constitution type is determined as the main constitution type. When the highest probability value is lower than the preset threshold, the top 2 or 3 body types in terms of probability are selected as the composite body type. Based on the main constitution type or the complex constitution type, and combined with the TCM constitution knowledge graph, corresponding constitution tendency and constitution characteristic descriptions are generated.

[0013] Furthermore, the system also includes: The health record management unit is used to establish user health records and store the physical constitution identification results and / or health intervention plans obtained from previous analyses. The trend analysis unit is used to analyze the changing trends of the results of previous physical fitness identifications. The intervention plan generation unit is also used to dynamically optimize subsequent health intervention plans based on the results of the trend analysis, forming a closed-loop health management process.

[0014] Furthermore, the object detection model is a pre-trained Yolox object detection model, which uses CSPDarknet53 as the backbone network, FPN as the feature pyramid network, and a decoupling head as the detection head; the pre-training process is as follows: Collect no fewer than 50,000 standard tongue images, and have licensed TCM physicians annotate the tongue body bounding boxes and tongue coating area masks; The data is augmented using an enhancement strategy that simulates real-world shooting disturbances; Based on general visual pre-training parameters, the tongue diagnosis task is fine-tuned by jointly optimizing the synchronous training of tongue detection and tongue coating segmentation tasks. During training, a focus-sensitive loss function is introduced to give higher weight to rare sample categories, and a boundary geometric consistency constraint is introduced to improve the recall rate of small targets and the accuracy of edge localization.

[0015] According to another aspect of this disclosure, a traditional Chinese medicine tongue diagnosis intelligent diagnostic device is provided, comprising: The image acquisition module is used to acquire raw tongue images; A memory for storing computer programs and a preset model library, the model library including the target detection model and the body constitution classification model; A processor for executing the computer program to perform the following steps: ① The original tongue image is subjected to adaptive optimization processing for illumination and color to generate standardized tongue image data; ②The target detection model is invoked to locate the tongue body region and segment the tongue coating region of the standardized tongue image data, and the segmentation mask of the tongue coating region is obtained; ③ Based on the segmentation mask of the tongue coating region, extract a multi-dimensional tongue coating feature vector containing color features, thickness features, dryness features, and distribution pattern features; ④ The constitution classification model is invoked to process the multi-dimensional tongue coating feature vector to generate the probability distribution of each TCM constitution type; ⑤ Combine the preset threshold and determine the physical condition identification result according to the probability distribution; ⑤ Based on the constitution identification results, a personalized health intervention plan is generated by matching from the preset TCM dietary therapy knowledge base; The output module is used to output the physical constitution identification results and / or the personalized health intervention plan.

[0016] Furthermore, the processor is also used for: Establish a user health record and store the physical condition identification results and / or the personalized health intervention plan in the memory; Obtain the user's past physical constitution identification results and analyze the changing trends; Based on the results of the trend analysis, the subsequently generated health intervention plan is dynamically optimized.

[0017] According to another aspect of this disclosure, a computer device is provided, comprising: a storage module, a processing module, and a transceiver module that are sequentially and communicatively connected, wherein the storage module is used to store a computer program, the transceiver module is used to send and receive messages, and when the processing module executes the computer program, it realizes the function of the TCM tongue diagnosis model constructed by the above-described model construction method, or realizes the function performed by the TCM tongue intelligent diagnosis system.

[0018] According to another aspect of this disclosure, a computer-readable storage medium is provided, on which instructions are stored, which, when executed on a computer, implement the function of the TCM tongue diagnosis model constructed by the above-described model construction method, or implement the function performed by the TCM tongue intelligent diagnosis system.

[0019] In the training process of the object detection model, Focal Loss is used as the loss function to handle the class imbalance problem; IoU Loss is used to optimize the bounding box regression accuracy; and the theory of traditional Chinese medicine tongue diagnosis is deeply integrated to enhance the diagnostic ability. Furthermore, the model is strengthened in the following five aspects: (1) Alignment of category weights with tongue diagnosis semantics: For tongue coating categories that are key to diagnosis but have sparse samples (such as black coating, coarse coating, thick coating, etc.), semantic classification is performed, and the weight coefficients in the cross-entropy loss are dynamically adjusted accordingly. This guides the model to learn rare but high-value tongue features more fully during training, significantly improving the sensitivity and discrimination ability of tongue coating types related to diagnosis, thereby enhancing the clinical reliability of the overall analysis results.

[0020] (2) Multi-task loss is consistent with the four-dimensional features of tongue diagnosis: The training adopts a multi-task framework of tongue detection and tongue coating segmentation. The mask of the segmentation output is directly used to extract the four-dimensional features of color, thickness, moisture, and distribution. Its accuracy is highly dependent on the boundary precision. The segmentation loss is assigned a syndrome-related weight in the total loss, and the boundary optimization of the corresponding areas of the internal organs such as the root and sides of the tongue is particularly strengthened. This makes the IoU and boundary quality optimization not only improve the segmentation index, but also ensure the syndrome-related usability of the distribution features. Thus, the optimization objectives of Focal Loss and IoU Loss are semantically consistent with the extraction of four-dimensional features in traditional Chinese medicine.

[0021] (3) Consistency Design of Difficult Samples and Key Samples for TCM Diagnosis: Focal Loss adaptively reduces the loss weight of easily classified samples through modulation factors, which strengthens the learning of difficult samples while still ensuring the accuracy of easily classified samples. This application further integrates "difficult samples" with key samples for TCM diagnosis: In the annotation stage, key tongue appearances for TCM diagnosis are explicitly marked (such as mixed cold and heat, complex constitution-related manifestations, or rare combinations of tongue color and dryness); and they are given higher weights in the loss calculation, or a weighted Focal Loss strategy is adopted. Accordingly, the training of the model in this application not only responds to the classification difficulty driven by data, but also actively incorporates the "keyness of diagnosis" under the guidance of TCM theory, thereby significantly enhancing the ability to diagnose key clinical scenarios without sacrificing the overall accuracy.

[0022] (4) IoU Loss and Tongue Diagnosis Zone Boundaries: According to the theory of tongue diagnosis in Traditional Chinese Medicine, the tongue surface is divided into regions such as the tip (heart and lungs), middle (spleen and stomach), root (kidneys), and sides (liver and gallbladder). The distribution of tongue coating in each region corresponds to different organ functions. The boundary quality of the tongue coating region mask directly affects the accuracy of the distribution characteristics; regions such as the root and sides are prone to missed detection or blurred boundaries. In IoU Loss or segmentation loss, spatial weighting can be applied to the above-mentioned key diagnostic regions to make the boundary regression consistent with the anatomy of "organ division" and the theory of tongue diagnosis, thereby improving the credibility of the distribution characteristics and the diagnostic ability.

[0023] (5) Closed loop with physical fitness knowledge graph The constitution classification model takes 64-dimensional tongue coating features as input and outputs nine constitution types and their tendencies, depending on the accuracy of the tongue coating features. When training the detection / segmentation model, the "downstream constitution judgment accuracy" or "consistency between tongue coating features and constitution labels" can be used as indirect objectives. By jointly adjusting the weights or sampling strategies of Focal Loss and IoU Loss through feedback from diagnostic-related indicators (such as constitution-tongue coating consistency) on the joint training or validation sets, a closed-loop optimization of "detection / segmentation—tongue coating features—constitution judgment" is formed, enhancing the overall diagnostic ability and connecting with the constitution knowledge graph and dietary recommendation chain of this invention.

[0024] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. Multi-dimensional feature fusion enhances the objectivity and accuracy of tongue image analysis: By constructing a 64-dimensional tongue coating feature vector encompassing four dimensions—color, thickness, moisture, and distribution—this invention deeply integrates traditional Chinese medicine tongue diagnosis theories (such as tongue coating color differentiation, organ-specific differentiation, and moisture differentiation). Compared to traditional methods that only extract single dimensions like color or texture, this invention can more comprehensively and objectively quantify tongue image information, providing a more discriminative structured data foundation for subsequent constitution identification, thereby significantly improving the accuracy of the analysis results.

[0025] 2. Supports intelligent determination of complex constitutions, better aligning with TCM clinical practice: By introducing a preset threshold mechanism, when the probability of a single constitution type is below the threshold, the system automatically selects the top 2 or 3 constitution types as the complex constitution type for output. This technology effectively solves the problem that existing methods cannot handle complex constitutions, making the diagnostic results more consistent with TCM clinical practice and improving the system's clinical usability and user acceptance.

[0026] 3. Constructing an "Assessment-Intervention-Feedback" Closed Loop for Dynamic and Personalized Health Management: By establishing user health records, storing analysis results from each analysis, and further analyzing trends in physical condition changes, subsequent intervention plans are dynamically optimized based on these trends. This closed-loop management process represents a leap from single static diagnosis to long-term dynamic tracking, enabling health intervention recommendations to self-adjust as the user's physical condition changes. This truly achieves personalized and precise health management, significantly improving user experience and intervention effectiveness.

[0027] In summary, the overall technical solution of this application effectively alleviates the problems of strong subjectivity, lack of standardization, and generalization of intervention measures in traditional tongue diagnosis. Users only need to take a picture of the tongue, and the system will output a complete analysis report within seconds, realizing "diagnosis by taking a picture of the tongue". No professional physicians are required to participate on-site, which is applicable to diverse scenarios such as home self-examination, community health monitoring, and remote TCM consultation, and promotes the popularization and implementation of TCM tongue diagnosis technology. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the data processing flow of a TCM tongue diagnosis intelligent diagnostic model constructed according to an embodiment of the present invention.

[0029] Figure 2 This is a schematic diagram illustrating the process of inputting tongue image data, locating the tongue body and segmenting the tongue coating using a Yolox object detection model, and generating tongue coating region data in one embodiment of the present invention.

[0030] Figure 3 This is a schematic diagram of the process for judging body constitution based on tongue coating feature vectors in one embodiment of the present invention.

[0031] Figure 4 This is a schematic diagram of the process of generating a personalized diet recommendation plan based on the results of a physical condition assessment in one embodiment of the present invention.

[0032] Figure 5 This is a schematic diagram of the structure of a traditional Chinese medicine tongue diagnosis device according to an embodiment of the present invention.

[0033] Figure 6 This is a schematic diagram of the main interface of the client described in one embodiment of this application.

[0034] Figure 7 This is a schematic diagram of the standardized tongue image acquisition interface of the client described in one embodiment of this application.

[0035] Figure 8 This is a schematic diagram of the tongue detection and tongue coating segmentation results interface of the client described in one embodiment of this application.

[0036] Figure 9 This is a schematic diagram of the physical fitness identification result interface of the client described in one embodiment of this application.

[0037] Figure 10 This is a schematic diagram of the organ risk analysis interface of the client described in one embodiment of this application.

[0038] Figure 11 This is a schematic diagram of the personalized diet recommendation interface of the client described in one embodiment of this application.

[0039] Figure 12 This is a schematic diagram of the user health record management interface of the client described in one embodiment of this application. Detailed Implementation

[0040] To better understand the technical solution of this application, the above technical solution will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] It should be noted that, although the embodiments in this application are based on... Figure 1 Steps S1 to S6 will be described sequentially, but this does not mean that the steps must be performed in a strict order. The reason this embodiment follows this order is... Figure 1 The order in which steps S1 to S6 are described is provided to facilitate understanding of the technical solutions of the embodiments of this application by those skilled in the art. In other words, the order of the steps in the embodiments of this application can be appropriately adjusted according to actual needs.

[0042] This application discloses a TCM tongue image intelligent analysis method. Its core lies in achieving a closed-loop health management process, encompassing tongue image acquisition, tongue coating localization, multi-dimensional feature extraction, constitution identification, and personalized dietary intervention, through structured guidance. The method can be implemented on various terminal devices with image acquisition and processing capabilities, including but not limited to mobile applications, web clients, embedded tongue diagnostic instruments, or cloud service systems. Example 2 illustrates a specific application of the model building method described in this invention, but should not be construed as limiting the implementation of this invention.

[0043] Example 1: A Method for Constructing an Intelligent Diagnostic Model for Tongue Appearance in Traditional Chinese Medicine

[0044] like Figure 1As shown in the figure, this embodiment discloses a method for constructing a TCM tongue diagnosis intelligent model, which specifically includes the following steps: S1. Obtain a standardized tongue image sample set This step aims to generate high-quality, reproducible, standardized tongue image data, providing reliable input for subsequent intelligent analysis. Specifically, it includes the following sub-processes: (1) Standardized tongue image acquisition Tongue images are acquired using a professional tongue diagnostic camera or a mobile device camera under standardized lighting conditions. These standardized acquisition conditions include: CIE D65 standard daylight simulation light source; color temperature 6500K; fixed shooting distance of 15cm to ensure consistency and comparability of image acquisition; the user naturally extends their tongue with the tongue surface flat, avoiding excessive force or contraction to ensure the authenticity of the tongue image.

[0045] (2) Multi-stage image preprocessing Perform the following operations sequentially on the original images generated in the previous stage to ensure that the true physiological state of the tongue is restored: Color correction: White balance is performed based on D65 light source parameters, and the image is converted from the original color space (such as RGB) to the standard color space (such as HSV) in order to more accurately represent and process the true color information of the tongue.

[0046] Noise Removal: A bilateral filtering algorithm is used to remove high-frequency noise while preserving the tongue edges and moss texture.

[0047] Contrast Enhancement: Employs CLAHE (Contrast-Limited Adaptive Histogram Equalization) to locally enhance the visibility of tongue coating areas, especially suitable for highlighting subtle features such as thin coating and transitions between moist and dry areas.

[0048] (3) Multi-dimensional quality inspection and access control The preprocessed tongue images need to be evaluated jointly by three quantitative indicators: Sharpness ≥ 0.8, calculated using the Laplacian operator response and gradient magnitude; Integrity ≥ 0.9, determined by the tongue detection model to determine whether the tongue surface is completely within the field of view; Illumination uniformity ≥0.85, evaluated based on the standard deviation of image zone brightness.

[0049] Only images that simultaneously meet all three quality indicators can be identified as standardized tongue image data and proceed to subsequent processing stages.

[0050] (4) Robust compatibility mechanism for non-standard scenarios To address the complex lighting and low contrast scenarios commonly encountered in homes or on mobile devices, this invention further designs a dual-track processing strategy: For non-standard images, local illumination estimation and normalization are first performed based on the tongue region (such as the adaptive Retinex method), and the lightness (V) channel is constrained in the HSV color space to avoid distortion of the tongue coating color; then the enhancement branch is triggered, and multi-scale contrast enhancement is used to improve the visibility of the tongue coating. All images (regardless of whether they meet the standard conditions) are input into the same set of Yolox tongue localization and tongue coating segmentation models, and the subsequent feature extraction, constitution identification and diet recommendation processes are completed to ensure algorithm consistency. For non-standard acquired images, the system adds prompts such as "It is recommended to combine with clinical review" in the output results, and records the quality label and confidence level to ensure interpretability and safety.

[0051] S2. Tongue region localization and tongue coating region segmentation, generating tongue coating region data. This step aims to use Yolox object detection technology to accurately locate the tongue body and precisely segment the tongue coating area, providing an accurate data foundation for subsequent feature analysis, such as... Figure 2 As shown.

[0052] First, standardized tongue image data is input into a pre-trained Yolox object detection model for tongue region localization. The Yolox model uses CSPDarknet 53 as the backbone network to extract multi-scale features; it uses FPN as the feature pyramid network to fuse feature information at different scales; and it uses a decoupled head as the detection head to predict classification and localization tasks separately, improving detection accuracy. The model is pre-trained on the COCO dataset and then fine-tuned on over 50,000 standardized tongue image data, achieving a tongue detection accuracy of 96% and an mAP of 0.94.

[0053] Based on the tongue bounding box coordinates output by the Yolox model, the standardized tongue image data is cropped to generate a tongue region image. The bounding box coordinates include the coordinates of the top-left corner (x1, y1) and the bottom-right corner (x2, y2), as well as the detection confidence score. When the detection confidence score is greater than 0.9, the detection result is considered reliable and further processing is performed.

[0054] The tongue region image is input into a pre-trained Yolox segmentation model for tongue coating region segmentation. The Yolox segmentation model adds a segmentation branch to the Yolox detection model, achieving pixel-level tongue coating region segmentation. The segmentation branch uses a fully convolutional network structure to classify each pixel and determine whether it belongs to the tongue coating region. The model employs a multi-task learning strategy, simultaneously training for both tongue detection and tongue coating segmentation tasks to improve model performance. The tongue coating segmentation accuracy reaches 92%, and the IoU reaches 0.89.

[0055] Tongue coating region data is generated based on the tongue body bounding box coordinates, the tongue coating region mask, and the tongue coating region confidence score. The tongue coating region data includes: tongue body bounding box coordinates, used to locate the tongue body in the original image; the tongue coating region mask, used to identify the pixel positions of the tongue coating region; and the tongue coating region confidence score, used to evaluate the reliability of the segmentation results.

[0056] S3. Based on the segmentation mask of the tongue coating region, extract a multi-dimensional tongue coating feature vector containing color features, thickness features, dryness / moisture features, and distribution pattern features. The color feature analysis involves extracting the average hue (H value), saturation (S value), and brightness (V value) of the tongue coating area within the HSV color space. Based on traditional Chinese medicine tongue diagnosis theory, the hue values ​​are mapped to four basic tongue coating color types: white coating (H value in the range of 0-30 or 330-360), yellow coating (H value in the range of 30-90), gray coating (H value in the range of 90-150), and black coating (H value in the range of 150-210). By calculating the probability distribution of each coating color type, a color feature vector is obtained, including the probability of white coating, yellow coating, gray coating, and black coating, as well as the average hue, saturation, and brightness values—a total of seven dimensions of color features.

[0057] The thickness feature analysis involves calculating the area ratio of the tongue coating region mask, i.e., the ratio of the tongue coating area to the tongue body area. Simultaneously, combining the texture density of the tongue coating region, the thickness is assessed by calculating the Local Binary Pattern (LBP) features of the tongue coating region. According to Traditional Chinese Medicine tongue diagnosis theory, the thickness is divided into three levels: no coating (area ratio less than 0.1), thin coating (area ratio between 0.1 and 0.5), and thick coating (area ratio greater than 0.5). By calculating the confidence level for each level, a thickness feature vector is obtained, including the probability of no coating, the probability of thin coating, the probability of thick coating, as well as the area ratio value and texture density value, for a total of 5 dimensions of thickness features.

[0058] The analysis of the moistness / dryness characteristics involves: extracting the glossiness features of the tongue coating area and assessing the glossiness by calculating the specular reflection intensity of the tongue coating area; simultaneously, extracting the texture / roughness features of the tongue coating area and assessing the roughness by calculating the gradient amplitude variance of the tongue coating area. Based on traditional Chinese medicine tongue diagnosis theory, the degree of moistness / dryness is divided into three levels: moist coating (high gloss, low roughness), dry coating (medium gloss, medium roughness), and rough coating (low gloss, high roughness). By calculating the confidence level of each level, a moistness / dryness feature vector is obtained, including the probability of moist coating, dry coating, and rough coating, as well as glossiness and roughness values, for a total of 5 dimensions of moistness / dryness characteristics.

[0059] The distribution feature analysis analyzes the distribution pattern of tongue coating in different areas of the tongue surface. According to Traditional Chinese Medicine (TCM) tongue diagnosis theory, the tongue surface is divided into four regions: the tip region (corresponding to the heart and lungs), the middle region (corresponding to the spleen and stomach), the root region (corresponding to the kidneys), and the sides region (corresponding to the liver and gallbladder). The distribution ratio of tongue coating in each region and the differences in tongue coating characteristics in each region are calculated. By analyzing the distribution pattern, the functional state of different organs can be determined. The distribution feature vector includes the distribution ratio of the tip, middle, root, and sides of the tongue, as well as the differences in tongue coating characteristics in each region, totaling 47 dimensions of distribution features.

[0060] Finally, the color features (7 dimensions), thickness features (5 dimensions), moisture features (5 dimensions), and distribution features (47 dimensions) are fused to generate a 64-dimensional tongue coating feature vector. The feature fusion method combines weighted averaging and principal component analysis (PCA) to retain the most important feature information while reducing the feature dimensionality, thereby improving the performance of the subsequent classification model.

[0061] This application combines the theory of tongue diagnosis in traditional Chinese medicine to construct a multi-dimensional tongue coating feature analysis system, which realizes the comprehensive, objective and quantitative extraction of tongue coating features, and provides high-information structured feature support for constitution identification.

[0062] S4. Construct a body constitution classification model A knowledge graph of TCM constitutions was constructed. The knowledge graph includes nine basic constitution types and their corresponding tongue coating characteristic patterns: Balanced constitution (pale red tongue, thin white coating), Qi deficiency constitution (pale tongue, thin white coating), Yang deficiency constitution (pale and swollen tongue, white and slippery coating), Yin deficiency constitution (red tongue with little coating), Phlegm-dampness constitution (pale and swollen tongue, white and greasy coating), Damp-heat constitution (red tongue, yellow and greasy coating), Blood stasis constitution (dark purple tongue, thin coating), Qi stagnation constitution (pale red tongue, thin white coating), and Special constitution (various tongue appearances). Each constitution type corresponds to a set of typical tongue coating characteristic patterns, including combinations of color characteristics, thickness characteristics, moisture / dryness characteristics, and distribution characteristics.

[0063] Tongue coating feature vectors are input into a pre-trained constitution classification model to predict constitution type. The constitution classification model employs a multilayer perceptron (MLP) network structure, including an input layer (64-dimensional), a first hidden layer (128-dimensional, ReLU activation function), a second hidden layer (64-dimensional, ReLU activation function), and an output layer (9-dimensional, Softmax activation function). The model is trained on over 30,000 labeled data examples using cross-entropy loss and the Adam optimizer, with a learning rate of 0.001, for 100 epochs. The model achieves an accuracy of 88% and an F1-score of 0.86 on the validation set.

[0064] Based on the output of the constitution classification model, the probability distribution of each constitution type is calculated. The model outputs nine probability values, corresponding to the probabilities of the nine constitution types. The constitution type with the highest probability is selected as the primary constitution type. When the probability of the primary constitution type is less than 0.6, it is considered that the judgment of a single constitution type is not accurate enough. The top two or three constitution types with the highest probabilities are selected as the composite constitution type, and the tendency of each constitution is calculated.

[0065] Incorporating Traditional Chinese Medicine (TCM) constitution theory, the system generates a constitution characteristic description and a constitution tendency score. The constitution characteristic description includes basic constitution features, common symptoms, susceptible diseases, and conditioning principles. The constitution tendency score uses a 0-100 point rating system, with higher scores indicating more pronounced constitution characteristics.

[0066] Based on the mapping relationship between the tongue area and the internal organs, the user's health / disease tendency scores for the heart, spleen and stomach, kidneys and liver and gallbladder are assessed, and comprehensive inferences and suggestions are given based on this.

[0067] like Figure 3 As shown, this application achieves intelligent constitution judgment based on tongue coating characteristics through in-depth integration with traditional Chinese medicine constitution theory, providing a scientific basis for personalized health management.

[0068] S5. Based on the constitution assessment results, a personalized health intervention plan is generated by matching from a preset TCM dietary therapy knowledge base. First, a knowledge base for traditional Chinese medicine (TCM) dietary therapy will be constructed. This knowledge base is based on classic TCM texts such as the *Huangdi Neijing*, *Compendium of Materia Medica*, and *Dietary Therapy Materia Medica*, as well as modern research findings in TCM dietary therapy. It includes recommended foods, prohibited foods, dietary therapy formulas, and dietary principles corresponding to different body constitution types.

[0069] Based on the main constitution type identified in the constitution assessment results, corresponding basic dietary recommendations are retrieved from the dietary therapy knowledge base. These recommendations include: a list of recommended ingredients (10-15 ingredients), a list of prohibited ingredients (5-8 ingredients), dietary therapy formulas (2-3 formulas), and explanations of dietary principles (3-5 principles).

[0070] Based on the body constitution assessment results, adjust the intensity of the dietary recommendations. When the body constitution tendency is high (greater than 80 points), increase the quantity and frequency of recommended ingredients, and it is recommended to consume them daily; when the body constitution tendency is low (less than 60 points), adjust the recommendations appropriately, and it is recommended to consume them 2-3 times per week.

[0071] Based on individual factors such as age, gender, and season, the basic dietary recommendations are adjusted accordingly. For example, older adults have weaker digestive functions, so easily digestible foods are recommended; women are advised to increase their intake of blood-nourishing foods before and after menstruation; liver-nourishing foods are recommended in spring; and heat-clearing foods are recommended in summer.

[0072] Based on the above adjustments, a personalized dietary recommendation plan is generated. The plan includes: a list of recommended ingredients (including dosage and preparation methods), a list of prohibited ingredients (including reasons), dietary therapy formulas (including preparation methods), dietary principles (including precautions), and a description of expected results (including the improvement period). The recommendation plan is presented in various formats such as text, charts, and videos to facilitate user understanding and implementation.

[0073] like Figure 4 As shown, this application achieves precise dietary recommendations based on constitution assessment through the personalized application of traditional Chinese medicine dietary therapy theory, providing users with scientific and practical guidance for health conditioning.

[0074] S6. Establish user health records and dynamically optimize subsequent dietary plans. Establish user health records to record the results of each tongue image analysis, constitution identification conclusions, and corresponding personalized dietary recommendations; through regular follow-ups, re-collect user tongue image images and compare them with historical data to analyze the trend of constitution evolution; based on the trend of constitution change, dynamically optimize dietary intervention strategies, thereby constructing a closed-loop TCM health management mechanism of "assessment-intervention-feedback-reassessment".

[0075] In summary, this application achieves precise localization of the tongue body and accurate segmentation of the tongue coating region through Yolox object detection technology. Combined with multi-dimensional feature analysis based on traditional Chinese medicine (TCM) tongue diagnosis theory, it realizes the objective identification of tongue coating symptoms, achieving an accuracy rate of 92%, a 25 percentage point improvement compared to traditional methods. Through deep integration with TCM constitution theory, it achieves intelligent constitution assessment based on tongue coating characteristics, with an accuracy rate of 88%, a 30 percentage point improvement compared to traditional methods. Through personalized application of TCM dietary therapy theory, it achieves precise dietary recommendations based on constitution assessment, increasing the personalization of dietary recommendations by 75%. This effectively solves the problems of strong subjectivity, low standardization, and insufficient personalization in traditional tongue diagnosis, providing scientific, objective, and convenient technical support for TCM health management and dietary therapy guidance.

[0076] Example 2: A client application based on a traditional Chinese medicine tongue diagnosis system Figure 5This is a schematic diagram of the tongue coating symptom analysis and diagnostic device based on Yolox according to the present invention. Based on the same inventive concept, embodiments of the present invention also provide a client application for a traditional Chinese medicine tongue image intelligent diagnostic system, the interactive interface of which is shown in the schematic diagram below. Figures 6 to 12 As shown.

[0077] The operation process includes: 1. Collecting tongue images ( Figure 7 ); 2. The system automatically performs tongue detection, tongue coating segmentation, and regional feature extraction. Figure 8 ); 3. Output the constitution identification results (example user is identified as having phlegm-dampness constitution). Figure 9 ); 4. Generate risk warnings based on Traditional Chinese Medicine's theory of Zang-Fu organs (such as phlegm-dampness / damp-heat risk warnings in the spleen and stomach area). Figure 10 );, 5. Push personalized diet lists and conditioning suggestions ( Figure 11 ); 6. Automatically generate personal health records and archive historical records, and provide relevant health advice based on historical comparative analysis. Figure 12 ).

[0078] Users only need to complete the tongue image capture, and the system can output the constitution identification conclusion and the actionable health management plan within seconds, significantly reducing the technical threshold of TCM tongue diagnosis and providing efficient and accessible technical support for routine health monitoring in the home setting.

[0079] Although some preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0080] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this application and their equivalents, this invention is also intended to include these modifications and variations.

Claims

1. A method for constructing an intelligent diagnostic model for tongue appearance in Traditional Chinese Medicine, characterized in that, include: A standardized set of tongue image samples was obtained, with each sample labeled with the tongue body region, tongue coating region, and corresponding TCM constitution type label. The standardized tongue image samples are used to locate the tongue body region and segment the tongue coating region to obtain the segmentation mask of the tongue coating region; Based on the segmentation mask of the tongue coating region, a multi-dimensional tongue coating feature vector containing color features, thickness features, dryness features, and distribution pattern features is extracted to form a training dataset. An initial constitution classification model is constructed. The initial constitution classification model adopts a multilayer perceptron network structure. Its input layer dimension matches the dimension of the multi-dimensional tongue coating feature vector, and its output layer dimension matches the number of preset TCM constitution types. The initial constitution classification model is trained using the training dataset to obtain a trained constitution classification model; wherein, the trained constitution classification model is used to generate the probability distribution of each TCM constitution type based on the input tongue image, so as to determine the constitution judgment result by combining a preset threshold.

2. The construction method according to claim 1, characterized in that, The multi-dimensional tongue coating feature vector is a 64-dimensional feature vector, wherein: The color features are generated by calculating the probability of each typical moss color category in a preset TCM moss color semantic space and combining it with color perception parameters. The thickness feature is calculated based on the area ratio of the tongue coating region mask and the texture density of the tongue coating region. The aforementioned dryness and moisture characteristics are calculated based on the gloss and texture characteristics of the tongue coating area, combined with the TCM theory of dryness and moisture differentiation. The aforementioned distribution pattern characteristics are generated by dividing the tongue body into regions based on the theory of organ classification on the tongue surface in traditional Chinese medicine, and then calculating the distribution ratio of tongue coating in each region and the differences in tongue coating characteristics in each region.

3. A traditional Chinese medicine tongue diagnosis system, characterized in that, include: The image preprocessing unit is used to perform adaptive optimization processing on the acquired raw tongue image to generate standardized tongue image data. The tongue image analysis unit includes a target detection model and a feature extraction module. The target detection model is used to locate the tongue body region and segment the tongue coating region of the standardized tongue image data, and output the segmentation mask of the tongue coating region. The feature extraction module is used to extract a multi-dimensional tongue coating feature vector containing color features, thickness features, dryness features, and distribution pattern features based on the segmentation mask of the tongue coating region. The constitution identification unit includes a constitution classification model constructed using the method described in claim 1 or 2; the constitution classification model is used to generate the probability distribution of each TCM constitution type based on the multi-dimensional tongue coating feature vector, and combined with a preset threshold, to determine the constitution identification result containing the main constitution type or the complex constitution type. The intervention plan generation unit is used to match and generate personalized health intervention plans from a preset TCM dietary therapy knowledge base based on the constitution identification results.

4. The intelligent diagnostic system for tongue appearance in Traditional Chinese Medicine according to claim 3, characterized in that, When determining the body constitution identification result, the body constitution identification unit performs the following operations: When the probability value of the constitution type corresponding to the highest probability output by the constitution classification model is higher than the preset threshold, the constitution type is determined as the main constitution type. When the highest probability value is lower than the preset threshold, the top 2 or 3 body types in terms of probability are selected as the composite body type. Based on the main constitution type or the complex constitution type, and combined with the TCM constitution knowledge graph, corresponding constitution tendency and constitution characteristic descriptions are generated.

5. The intelligent diagnostic system for tongue appearance in Traditional Chinese Medicine according to claim 3, characterized in that, The system also includes: The health record management unit is used to establish user health records and store the physical constitution identification results and / or health intervention plans obtained from previous analyses. The trend analysis unit is used to analyze the changing trends of the results of previous physical fitness identifications. The intervention plan generation unit is also used to dynamically optimize subsequent health intervention plans based on the results of the trend analysis, forming a closed-loop health management process.

6. The intelligent diagnostic system for tongue appearance in Traditional Chinese Medicine according to claim 3, characterized in that, The target detection model is a model obtained through the following training process: Collect no fewer than 50,000 standard tongue images, and have licensed TCM physicians annotate the tongue body bounding boxes and tongue coating area masks; The data is augmented using an enhancement strategy that simulates real-world shooting disturbances; Based on general visual pre-training parameters, the tongue diagnosis task is fine-tuned by jointly optimizing the synchronous training of tongue detection and tongue coating segmentation tasks. During training, a focus-sensitive loss function is introduced to give higher weight to rare sample categories, and a boundary geometric consistency constraint is introduced to improve the recall rate of small targets and the accuracy of edge localization.

7. A smart diagnostic device for tongue appearance in Traditional Chinese Medicine, characterized in that, include: The image acquisition module is used to acquire raw tongue images; A memory for storing computer programs and a preset model library, the model library including the target detection model of claim 3 and the body constitution classification model of claim 1; A processor for executing the computer program to perform the following steps: ① The original tongue image is subjected to adaptive optimization processing for illumination and color to generate standardized tongue image data; ②The target detection model is invoked to locate the tongue body region and segment the tongue coating region of the standardized tongue image data, and the segmentation mask of the tongue coating region is obtained; ③ Based on the segmentation mask of the tongue coating region, extract a multi-dimensional tongue coating feature vector containing color features, thickness features, dryness features, and distribution pattern features; ④ The constitution classification model is invoked to process the multi-dimensional tongue coating feature vector to generate the probability distribution of each TCM constitution type; ⑤ Combine the preset threshold and determine the physical condition identification result according to the probability distribution; ⑤ Based on the constitution identification results, a personalized health intervention plan is generated by matching from the preset TCM dietary therapy knowledge base; The output module is used to output the physical constitution identification results and / or the personalized health intervention plan.

8. The intelligent diagnostic device for tongue appearance in Traditional Chinese Medicine according to claim 7, characterized in that, The processor is also used for: Establish a user health record and store the physical condition identification results and / or the personalized health intervention plan in the memory; Obtain the user's past physical constitution identification results and analyze the changing trends; Based on the results of the trend analysis, the subsequently generated health intervention plan is dynamically optimized.

9. A computer device, characterized in that, include: The system comprises a storage module, a processing module, and a transceiver module that are sequentially connected in communication. The storage module is used to store computer programs, the transceiver module is used to send and receive messages, and the processing module, when executing the computer program, implements the function of the TCM tongue diagnosis model constructed as described in claim 1 or 2, or implements the function of the TCM tongue diagnosis intelligent system as described in any one of claims 3 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer, implement the function of the TCM tongue diagnosis model constructed as claimed in claim 1 or 2, or implement the function performed by the system as claimed in any one of claims 3 to 6.

Images