Traditional Chinese medicine tongue diagnosis auxiliary diagnosis system based on artificial intelligence

Abstract

The invention provides a traditional Chinese medicine tongue diagnosis auxiliary diagnosis system based on artificial intelligence. The traditional Chinese medicine tongue diagnosis auxiliary diagnosis system comprises a tongue picture acquisition and preprocessing module, a tongue picture feature intelligent extraction module, a traditional Chinese medicine syndrome differentiation intelligent reasoning module and a diagnosis result integration and output module which are connected in sequence. Through multi-module cooperation and algorithm fusion, the pain points of traditional tongue diagnosis and an existing intelligent system are effectively solved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and traditional Chinese medicine tongue diagnosis auxiliary diagnostic technology, and in particular to an artificial intelligence-based traditional Chinese medicine tongue diagnosis auxiliary diagnostic system. Background Technology

[0002] In traditional Chinese medicine tongue diagnosis, diagnostic results heavily rely on the physician's accumulated experience, leading to strong subjectivity and inconsistent standards. For example, different physicians may disagree on the judgment of characteristics such as tongue color and the thickness of the tongue coating due to differences in experience, affecting diagnostic consistency. Furthermore, quantifying tongue characteristics is difficult; traditional methods struggle to transform intuitive descriptions such as "red tongue" and "greasy tongue coating" into calculable objective indicators, hindering the modernization and intelligentization of tongue diagnosis.

[0003] While existing AI-based tongue diagnosis assistance systems attempt to extract features through image recognition technology, they mostly remain at the level of two-dimensional image analysis. This makes them susceptible to distortions in feature extraction due to factors such as shooting angle and lighting conditions (e.g., tongue edge stretching, color deviation). Furthermore, the correlation between features and TCM syndrome elements and patterns largely relies on linear models or single-rule reasoning, making it difficult to capture complex nonlinear relationships such as "red tongue + yellow coating → heat syndrome." This results in limited diagnostic accuracy and fails to meet the actual needs of clinical auxiliary diagnosis. Summary of the Invention

[0004] This invention provides an artificial intelligence-based auxiliary diagnostic system for tongue diagnosis in traditional Chinese medicine. Through multi-module collaboration and algorithm integration, it effectively solves the pain points of traditional tongue diagnosis and existing intelligent systems.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A TCM tongue diagnosis auxiliary system based on artificial intelligence includes a tongue image acquisition and preprocessing module, a tongue image feature intelligent extraction module, a TCM syndrome differentiation intelligent reasoning module, and a diagnosis result integration and output module connected in sequence.

[0007] The tongue image acquisition and preprocessing module is used to acquire the original tongue image and shooting environment parameters, and to correct, segment and denoise the original tongue image, and output a standardized tongue image and a tongue body segmentation mask. The tongue body segmentation mask is a binary image that marks the tongue body region.

[0008] The tongue image feature intelligent extraction module receives a standardized tongue image and a tongue segmentation mask. First, based on the tongue region marked by the tongue segmentation mask, the standardized tongue image is converted into a standard viewpoint corrected image and surface coefficients are obtained through a three-dimensional surface fitting algorithm. Then, deep semantic features are extracted by combining the surface coefficients with an improved CNN model. At the same time, shallow features are extracted by combining the surface coefficients with traditional computer vision methods. Finally, the deep semantic features and shallow features are fused to obtain a fused feature vector.

[0009] The TCM syndrome differentiation intelligent reasoning module receives the fused feature vector, first obtains the preliminary score of the syndrome element through the syndrome element correlation surface fitting algorithm, then corrects the preliminary score of the syndrome element through the knowledge graph rule reasoning to obtain the corrected syndrome element score, and finally obtains the final syndrome element score and syndrome type matching degree by combining the preliminary score and the corrected syndrome element score through the random forest model.

[0010] The diagnostic result integration and output module receives the final syndrome element score and syndrome type matching degree, and generates and outputs an auxiliary diagnostic report.

[0011] In this specification, the improved CNN model in the intelligent tongue feature extraction module has a surface perception attention layer after the Conv3 stage. The attention weights of this layer are calculated based on the surface coefficients and are used to perform weighted processing on the feature map output by the Conv3 stage.

[0012] In this specification, when extracting shallow features using traditional computer vision methods, the standard viewpoint calibrated image is first converted from RGB space to HSV space. The average hue, saturation, and brightness of the tongue body area and tongue coating area are calculated. Then, the brightness is corrected by combining surface coefficients. At the same time, the entropy value and thickness of the tongue coating area are calculated to obtain shallow features containing the corrected brightness, entropy value, and thickness.

[0013] In this specification, in the intelligent tongue feature extraction module, the fusion weight of deep semantic features and shallow features is calculated by the Sigmoid function based on the quadratic term coefficient in the surface coefficient. The fusion weight is used to weight and concatenate the deep semantic features and shallow features to obtain the fused feature vector.

[0014] In this specification, the TCM syndrome differentiation intelligent reasoning module uses a quadratic polynomial surface fitting algorithm for syndrome element association. It takes the fused feature vector as input, solves the model coefficients by combining the least squares method with the training data, and then obtains the preliminary scores of 8 syndrome elements. The 8 syndrome elements include cold, heat, wind, dampness, dryness, deficiency, excess and blood stasis syndrome elements.

[0015] In this specification, the TCM syndrome differentiation intelligent reasoning module uses knowledge graph rule reasoning based on a triplet knowledge graph containing tongue appearance features, correlation relationships, syndrome elements, and rule weights. First, it matches and fuses the features in the feature vector that correspond to the knowledge graph, then calculates the syndrome element correction amount, and finally combines the initial syndrome element score and the syndrome element correction amount to obtain the corrected syndrome element score.

[0016] In this specification, in the TCM syndrome differentiation intelligent reasoning module, the input of the random forest model is a concatenated vector of the initial score and the corrected score of the syndrome element. The final score of the syndrome element is obtained by averaging the prediction results of the decision tree, and the matching degree of the syndrome type is calculated based on the correlation weight between the final score of the syndrome element and the syndrome type.

[0017] In this specification, the intelligent tongue feature extraction module and the intelligent TCM syndrome differentiation reasoning module form a closed-loop iteration. The iteration process is as follows: the intelligent TCM syndrome differentiation reasoning module generates feature correction values ​​based on the final syndrome element scores; the intelligent tongue feature extraction module updates the fused feature vector based on the feature correction values; and the intelligent TCM syndrome differentiation reasoning module recalculates the final syndrome element scores and syndrome matching degree based on the updated fused feature vector.

[0018] In this specification, the termination condition of the closed-loop iteration is any of the following: the number of iterations reaches the preset maximum number of iterations of 3; in two consecutive iterations, the Euclidean distance of the evidence element scores does not exceed 0.05, the change rate of the first evidence type matching degree does not exceed 0.1, the order of the first 3 evidence types is consistent, and the deviation of the core feature value from the theoretical range does not exceed 15%.

[0019] In this specification, the shooting environment parameters output by the tongue image acquisition and preprocessing module include light intensity, color temperature, and shooting angle. The shooting angle is used to adjust the number of iterations of the tongue body three-dimensional surface fitting algorithm in the tongue image feature intelligent extraction module, and the light intensity is used to adjust the brightness correction coefficient in traditional computer vision methods.

[0020] In summary, the present invention has at least the following beneficial effects:

[0021] The objectivity and accuracy of feature extraction have been improved: a three-dimensional surface fitting algorithm for the tongue body has been introduced to correct the perspective distortion of two-dimensional images. Combined with improved CNN and traditional computer vision fusion feature extraction, features such as tongue texture, tongue coating, and abnormal signs have been accurately quantified, reducing feature deviations caused by image quality.

[0022] The logic and robustness of dialectical reasoning are enhanced: the nonlinear relationship between features and syndrome elements is captured by fitting the correlation surface of syndrome elements, and the constraints of traditional Chinese medicine theory are introduced by combining knowledge graph rule reasoning. Then, random forest optimization and fusion are performed so that the dialectical results not only conform to the data pattern, but also follow the logic of traditional Chinese medicine diagnosis and treatment, reducing the limitations of a single algorithm.

[0023] It achieves closed-loop optimization of "feature extraction-diagnostic reasoning": through dynamic iterative correction, it ensures the consistency between feature extraction and diagnostic results, and the final output auxiliary diagnostic report has both quantitative basis and clinical readability, providing reliable reference for physicians and promoting the standardization and intelligent development of tongue diagnosis. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the artificial intelligence-based TCM tongue diagnosis auxiliary diagnostic system involved in this invention.

[0026] Figure 2 This is a schematic diagram of the tongue image acquisition and preprocessing module involved in this invention.

[0027] Figure 3 This is a schematic diagram of the intelligent tongue feature extraction module process involved in this invention.

[0028] Figure 4 This is a schematic diagram of the process of the intelligent reasoning module for TCM syndrome differentiation involved in this invention.

[0029] Figure 5 This is a schematic diagram of the diagnostic result integration and output module process involved in this invention. Detailed Implementation

[0030] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the embodiments of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0031] The following disclosure provides many different implementations or examples for carrying out different structures of the embodiments of the present invention. To simplify the disclosure of the embodiments of the present invention, specific examples of components and arrangements are described below. Of course, these are merely examples and are not intended to limit the embodiments of the present invention. Furthermore, reference numerals and / or reference letters may be repeated in different examples of the embodiments of the present invention; such repetition is for simplification and clarity and does not in itself indicate a relationship between the various implementations and / or arrangements discussed.

[0032] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0033] like Figure 1 As shown, this embodiment provides an artificial intelligence-based TCM tongue diagnosis auxiliary diagnostic system, including a tongue image acquisition and preprocessing module, a tongue image feature intelligent extraction module, a TCM syndrome differentiation intelligent reasoning module, and a diagnostic result integration and output module connected in sequence.

[0034] The tongue image acquisition and preprocessing module is used to acquire the original tongue image and shooting environment parameters, and to correct, segment and denoise the original tongue image, and output a standardized tongue image and a tongue body segmentation mask. The tongue body segmentation mask is a binary image that marks the tongue body region.

[0035] The tongue image feature intelligent extraction module receives a standardized tongue image and a tongue segmentation mask. First, based on the tongue region marked by the tongue segmentation mask, the standardized tongue image is converted into a standard viewpoint corrected image and surface coefficients are obtained through a three-dimensional surface fitting algorithm. Then, deep semantic features are extracted by combining the surface coefficients with an improved CNN model. At the same time, shallow features are extracted by combining the surface coefficients with traditional computer vision methods. Finally, the deep semantic features and shallow features are fused to obtain a fused feature vector.

[0036] The TCM syndrome differentiation intelligent reasoning module receives the fused feature vector, first obtains the preliminary score of the syndrome element through the syndrome element correlation surface fitting algorithm, then corrects the preliminary score of the syndrome element through the knowledge graph rule reasoning to obtain the corrected syndrome element score, and finally obtains the final syndrome element score and syndrome type matching degree by combining the preliminary score and the corrected syndrome element score through the random forest model.

[0037] The diagnostic result integration and output module receives the final syndrome element score and syndrome type matching degree, and generates and outputs an auxiliary diagnostic report.

[0038] In some embodiments, in the tongue image feature intelligent extraction module, the improved CNN model has a surface perception attention layer after the Conv3 stage. The attention weights of this layer are calculated based on the surface coefficients and are used to perform weighted processing on the feature map output by the Conv3 stage.

[0039] In some embodiments, in the intelligent extraction module for tongue features, when extracting shallow features using traditional computer vision methods, the standard viewpoint calibrated image is first converted from RGB space to HSV space, the average hue, saturation, and brightness of the tongue body area and the tongue coating area are calculated, and then the brightness is corrected by combining the surface coefficient. At the same time, the entropy value and thickness of the tongue coating area are calculated to obtain shallow features containing the corrected brightness, entropy value, and thickness.

[0040] In some embodiments, in the tongue image feature intelligent extraction module, the fusion weight of deep semantic features and shallow features is calculated by the Sigmoid function based on the quadratic term coefficient in the surface coefficient. The fusion weight is used to weight and concatenate the deep semantic features and shallow features to obtain the fused feature vector.

[0041] In some embodiments, the TCM syndrome differentiation intelligent reasoning module uses a quadratic polynomial surface fitting algorithm for syndrome element association. It takes the fused feature vector as input, solves the model coefficients by combining the least squares method with the training data, and then obtains the preliminary scores of 8 syndrome elements, which include cold, heat, wind, dampness, dryness, deficiency, excess and blood stasis syndrome elements.

[0042] In some embodiments, the TCM syndrome differentiation intelligent reasoning module uses knowledge graph rule reasoning based on a triplet knowledge graph containing tongue appearance features, correlation relationships, syndrome elements, and rule weights. First, it matches and fuses the features in the feature vector that correspond to the knowledge graph, then calculates the syndrome element correction amount, and finally combines the initial syndrome element score and the syndrome element correction amount to obtain the corrected syndrome element score.

[0043] In some embodiments, in the TCM syndrome differentiation intelligent reasoning module, the input of the random forest model is a concatenated vector of the initial syndrome element score and the corrected syndrome element score. The final syndrome element score is obtained by averaging the prediction results of the decision tree, and the matching degree of the syndrome type is calculated based on the correlation weight between the final syndrome element score and the syndrome type.

[0044] In some embodiments, the tongue image feature intelligent extraction module and the TCM syndrome differentiation intelligent reasoning module form a closed-loop iteration. The iteration process is as follows: the TCM syndrome differentiation intelligent reasoning module generates feature correction amount based on the final syndrome element score, the tongue image feature intelligent extraction module updates the fused feature vector based on the feature correction amount, and the TCM syndrome differentiation intelligent reasoning module recalculates the final syndrome element score and syndrome matching degree based on the updated fused feature vector.

[0045] In some embodiments, the termination condition of the closed-loop iteration is any of the following: the number of iterations reaches the preset maximum number of iterations of 3; in two consecutive iterations, the Euclidean distance of the evidence element scores does not exceed 0.05, the change rate of the first evidence type matching degree does not exceed 0.1, the order of the first 3 evidence types is consistent, and the deviation of the core feature value from the theoretical range does not exceed 15%.

[0046] In some embodiments, the shooting environment parameters output by the tongue image acquisition and preprocessing module include light intensity, color temperature, and shooting angle. The shooting angle is used to adjust the number of iterations of the tongue body three-dimensional surface fitting algorithm in the tongue image feature intelligent extraction module, and the light intensity is used to adjust the brightness correction coefficient in traditional computer vision methods.

[0047] The technical concept of this invention is as follows:

[0048] This invention proposes an artificial intelligence-based TCM tongue diagnosis auxiliary system, which includes four core modules: tongue image acquisition and preprocessing module, tongue image feature intelligent extraction module, TCM syndrome differentiation intelligent reasoning module, and diagnosis result integration and output module.

[0049] The system takes tongue images as input. First, it acquires standardized raw images and environmental parameters through the acquisition and preprocessing module. Then, the feature extraction module introduces three-dimensional surface fitting to correct viewpoint distortion and integrates the deep semantic features of improved CNN with the shallow quantitative features of traditional computer vision to generate comprehensive fused features. Subsequently, the syndrome differentiation and reasoning module, based on the fused features, outputs the final syndrome element score and candidate syndrome type through a three-level fusion of syndrome element association surface fitting, knowledge graph rule reasoning, and random forest optimization. Finally, the results integration module transforms the syndrome differentiation results into a clinical report that includes feature annotations and consultation suggestions.

[0050] The modules achieve dynamic optimization through closed-loop data iteration, ensuring that the entire process from image to diagnosis conforms to objective data patterns and follows traditional Chinese medicine theory, providing accurate and reliable intelligent auxiliary support for clinical tongue diagnosis.

[0051] Module 1: Tongue Image Acquisition and Preprocessing Module (process as follows) Figure 2 (As shown)

[0052] As the "input end" of the system, this module needs to provide high-quality raw data to module 2, and its processing accuracy directly affects the accuracy of tongue body three-dimensional surface fitting.

[0053] Core functionality expansion:

[0054] 1. Multi-device compatibility: Supports image input from tongue diagnostic instruments (resolution of 5 million pixels or higher), smartphones (automatically activates fill light), and tablet devices. By standardizing image scale through device calibration parameters (such as focal length and sensor size), it avoids distortion of tongue size caused by device differences.

[0055] 2. Real-time recording of environmental parameters: Synchronously collect and store the light intensity (unit: lux), color temperature (unit: K), and shooting angle (angle with the tongue perpendicular to the shooting angle, unit: degree) during shooting as metadata. (Set of shooting environment parameters), where L is the light intensity. Color temperature For the shooting angle. These parameters are used to assist module 2 in judging the difficulty of surface fitting (e.g., (Geometric correction needs to be strengthened when the angle is >30°).

[0056] 3. Basic noise reduction and cropping:

[0057] To address high-frequency noise in mobile phone photography, a bilateral filter (instead of a standard Gaussian filter) is used to preserve tongue edge details while reducing noise.

[0058] ;

[0059] in : The pixel value at coordinates (x, y) in the image after bilateral filtering; : The pixel value at coordinates (i, j) of the original tongue image; For the neighborhood of (x, y), For normalization coefficients, the standard deviation in the spatial domain is... =5. Standard deviation of the range =20 is the filter parameter.

[0060] The image is cropped based on the approximate outline of the tongue (preliminarily located using a simple thresholding method), and areas occupying more than 30% of the background are removed, resulting in a preliminary processed image of 1024×1024 pixels. .

[0061] Connection with Module 2:

[0062] Output and To module 2, where Shooting angle in Used in module 2-1 to adjust the number of iterations for surface fitting ( The larger the value, the more iterations are increased from 50 to 100 to ensure correction accuracy.

[0063] Module 2: Intelligent Extraction Module for Tongue Features (process as follows) Figure 3 (As shown)

[0064] This module is the core of the system's feature quantification process. Through a workflow of "tongue 3D surface fitting → improved CNN deep feature extraction → traditional computer vision shallow feature extraction → multi-feature fusion," it transforms tongue images into quantifiable features suitable for diagnostic purposes. Compared to traditional methods, the tongue 3D surface fitting algorithm solves the problem of tongue morphological distortion caused by shooting angles (such as tilting the head up or down) in 2D images. Furthermore, the algorithm's output is integrated into all subsequent feature extraction stages, achieving end-to-end synergy between "geometric correction, feature enhancement, and fusion optimization."

[0065] Submodule 2.1: Algorithm for Fitting Three-Dimensional Surfaces of the Tongue

[0066] Core function: Reconstructing a three-dimensional curved surface model from a two-dimensional tongue image, correcting distortions such as tongue stretching and compression caused by the shooting angle, and providing a standardized geometric benchmark for subsequent feature extraction.

[0067] Model building process:

[0068] 1. Feature Point Selection: In the "tongue segmentation mask" (binary image, tongue region as 1, background as 0) output by module 1, 20 key anatomical feature points of the tongue are automatically marked using an edge detection algorithm, specifically including:

[0069] One point on the tip of the tongue: the most prominent point at the front of the tongue. ;

[0070] One frenulum attachment point: the point where the center of the base of the tongue connects to the floor of the mouth. ;

[0071] Eight points on the left side of the tongue: evenly distributed points on the left edge from the tip of the tongue to the root of the tongue. ;

[0072] Eight points on the right side of the tongue: points evenly distributed along the right edge from the tip to the root of the tongue. ;

[0073] There are two center points on the tongue surface: the center point of the anterior 1 / 3 of the tongue body. The center point of the middle third of the tongue .

[0074] All feature points constitute a set The two-dimensional coordinates of each point

[0075] Determined by the image pixel coordinates.

[0076] 2. Definition of Surface Equation: The three-dimensional morphology of the tongue is described using a quadratic polynomial surface model (assuming the tongue is a smooth surface). The equation is as follows:

[0077] (2.1)

[0078] in:

[0079] This represents the height of the tongue at two-dimensional coordinates (x, y) (the dimension perpendicular to the image plane). The surface coefficients to be solved are ( For constant terms, , The coefficient of the linear term, , , (The coefficient of the quadratic term); the introduction of the quadratic term is to capture the non-linear characteristics of the natural curvature of the tongue (such as the slightly convex tip and slightly concave middle of the tongue).

[0080] 3. Solving for surface coefficients: The feature point set P is fitted by the least squares method, with the goal of minimizing the error between the height value predicted by the model and the prior height value.

[0081] Prior height value: Based on 3D scan data of 300 standard tongue bodies (without morphological distortion), the average height of 20 feature points was calculated. (such as the tip of the tongue) The average height is 12mm, tongue frenulum point (5mm)

[0082] Error function:

[0083] (2.2)

[0084] Solution results: Surface coefficients are obtained through matrix operations. This coefficient fully describes the three-dimensional morphological characteristics of the tongue (such as...). The larger the absolute value, the more pronounced the left-right curvature of the tongue.

[0085] Model application process:

[0086] Based on the solved surface model, the original corrected image (output of module 1) is converted into an image with a standard viewing angle (direct front vertical shooting angle). Specific steps:

[0087] 1. Define a tongue coordinate system (x', y') from a standard perspective, where the x' axis is the left-right direction of the tongue and the y' axis is the direction from the tip of the tongue to the root of the tongue, with a fixed size of 512×512 pixels;

[0088] 2. Establish the mapping relationship between the original coordinates (x, y) and the standard coordinates (x', y'):

[0089] (2.3)

[0090] Where M is the mapping function, derived from the surface model. The derivation (essentially, it involves "attaching" a plane from a standard perspective to the curved surface of a three-dimensional tongue).

[0091] 3. Generate corrected image:

[0092] (2.4)

[0093] That is, the pixel value at the standard coordinates (x', y') is equal to the pixel value of the corresponding mapped point (x, y) in the original image, ensuring that there is no distortion in any part of the tongue under the standard view.

[0094] Output:

[0095] Standard viewpoint corrected image (For use in subsequent feature extraction);

[0096] Surface coefficients (Used for subsequent feature weighting and correction).

[0097] Submodule 2.2: Improved Deep Feature Extraction in CNNs

[0098] Core function: Learn deep semantic features of the tongue image from the corrected image (such as the distribution pattern of cracks and the morphological features of teeth marks), and use surface coefficients to dynamically adjust the network attention, so that the network pays more attention to the tongue body morphology sensitive areas (such as the parts of the tongue edge where teeth marks are likely to appear).

[0099] Model building process:

[0100] 1. Basic Network Structure: Based on ResNet-50, it includes 5 convolutional stages (Conv1~Conv5) and a global average pooling layer. The improvement over the original ResNet-50 is the insertion of a "surface-aware attention layer" after the Conv3 stage. The weights of this layer are derived from the surface coefficients output by submodule 2.1. control.

[0101] 2. Design of the surface perception attention layer:

[0102] Input: Feature map output from the Conv3 stage (C=256 channels, H=64, W=64);

[0103] Attention weight calculation: For each channel k (k=1, 2, ..., 256), its attention weight is... Defined as:

[0104] (2.5)

[0105] Where: the Sigmoid function ensures ; For channel k and surface coefficients The correlation coefficient (learned through training, with initial values ​​randomly generated);

[0106] for The element in (i=0 corresponds to) (and so on).

[0107] Physical meaning: If the curvature of a certain area of ​​the tongue is more obvious (such as the edge of the tongue) If the weight of a channel is relatively large, the weight of that channel increases, and the network pays more attention to the features of that region (such as serrations).

[0108] Feature map weighting:

[0109] (2.6)

[0110] in This is the weighted feature map, where (k, h, w) represents the channel, height, and width dimensions. Let be the attention weight for the k-th channel.

[0111] 3. Deep Feature Output: After the weighted feature map is processed through Conv4 and Conv5 stages, a one-dimensional feature vector is obtained through global average pooling.

[0112] (2.7)

[0113] in This is a global average pooling operation. It includes abstract features such as cracks, teeth marks, and ecchymosis (e.g., the 1200th dimension may correspond to "the depth of teeth marks on the tongue edge").

[0114] Model training process:

[0115] 1. Training data: 10,000 tongue images (including correction images) Surface coefficients Each image is labeled with 7 categories of tongue features (presence or absence of cracks, presence or absence of teeth marks, presence or absence of ecchymosis, etc., with each category labeled as 0 or 1).

[0116] 2. Loss Function: Multi-label cross-entropy loss is used to calculate the difference between the predicted features and the labeled features.

[0117] (2.8)

[0118] in The label (0 or 1) for the c-th feature. The probability of the CNN predicting this feature (by...) (Output via fully connected layer)

[0119] 3. Optimization strategy: Use the Adam optimizer with an initial learning rate of 0.001. The learning rate is reduced to 1 / 10 of its original value every 20 rounds, and the training continues for a total of 100 rounds until the loss function converges (fluctuation is less than 0.001).

[0120] Output: Deep semantic feature vector .

[0121] Submodule 2.3: Shallow Feature Extraction in Traditional Computer Vision

[0122] Core function: Quantify the color and texture features of the tongue body and tongue coating (such as the degree of redness of the tongue body and the thickness of the tongue coating), complementing the deep features of CNN (traditional methods are good at quantifying intuitive features, while CNN is good at capturing complex patterns), and the feature calculation process introduces surface coefficients to correct for lighting deviations.

[0123] Processing procedure:

[0124] 1. Division of tongue body and tongue coating regions:

[0125] Based on the corrected image K-means clustering (K=2) was used to divide the tongue body into tongue tissue regions. and tongue coating area (The clustering feature is RGB color value, and the cluster centers are initialized using 300 manually labeled data examples.)

[0126] Output region mask: =1 indicates that (x, y) belongs to the tongue body area. =1 indicates that it belongs to the tongue coating area, and 0 indicates that it does not belong to the area. + =1 (within the tongue area).

[0127] 2. Color Feature Extraction and Surface Correction:

[0128] Color space conversion: Converting from RGB space to HSV space (which better matches human color perception) yields three channels: hue H(x,y), saturation S(x,y), and lightness V(x,y).

[0129] Calculation of average color in the area:

[0130] ; ; (2.9)

[0131] ; ; (2.10)

[0132] in , , The average hue, saturation, and brightness of the tongue's surface area; , , The average hue, saturation, and brightness of the tongue coating area; This represents the total number of pixels in the tongue's surface area. This represents the total number of pixels in the tongue coating area.

[0133] Curved surface correction for brightness values: The raised areas of the tongue (such as the tip) receive more light and have higher brightness values, requiring correction using curved surface height.

[0134] (2.11)

[0135] (2.12)

[0136] in The average height of the tongue body area (calculated by formula 2.1). This represents the average height of the tongue coating area. =0.15 is the correction factor (determined through experiments to ensure that the brightness is not affected by the protrusion after correction). , This is the corrected brightness value.

[0137] 3. Texture feature extraction:

[0138] Tongue coating greasiness: The entropy value of the tongue coating area was calculated using the gray-level co-occurrence matrix (GLCM). (The higher the entropy value, the more complex the texture, and the more greasy the tongue coating.)

[0139] (2.13)

[0140] Where P(i,j) is the probability that gray values ​​i and j appear simultaneously in GLCM (distance 1, angle 0°). Avoid log(0);

[0141] Tongue coating thickness: defined as the proportion of the total area of ​​the tongue body to the number of pixels in the tongue coating area.

[0142] (2.14)

[0143] Output: Shallow feature vectors .

[0144] Module 2 Algorithm Fusion and Output

[0145] Fusion logic: CNN features (deep semantics) and traditional CV features (shallow quantization) need to be dynamically weighted and fused. The weights are determined by the shape of the tongue surface (the higher the curvature of the surface, the greater the weight of the CNN features, because deep features are better able to capture features under complex shapes).

[0146] Fusion formula:

[0147] (2.15)

[0148] Among them, the fusion weight The calculation is as follows:

[0149] (2.16)

[0150] , Surface coefficients The quadratic coefficient (reflecting the degree of curvature of the tongue in the left-right and front-back directions); For the Sigmoid function, ensure (The greater the degree of curvature, the better) The closer to 1, the greater the proportion of CNN features.

[0151] Output: Fusion feature vector , as input to module 3.

[0152] Module 3: Intelligent Reasoning Module for Traditional Chinese Medicine Diagnosis (process as follows) Figure 4 (As shown)

[0153] This module is the core of the system's "feature → syndrome element → syndrome type" reasoning process. Through the process of "syndrome element correlation surface fitting → knowledge graph rule reasoning → random forest optimization," it transforms fused features into TCM syndrome differentiation results. The newly added syndrome element correlation surface fitting algorithm is used to model the nonlinear relationship between features and syndrome elements, while the knowledge graph rule reasoning incorporates TCM theoretical constraints. The outputs of both serve as the input to the random forest, realizing a fusion reasoning approach that is both "data-driven" and "knowledge-driven."

[0154] Submodule 3.1: Algorithm for Fitting Surfaces Related to Evidence Elements

[0155] Core function: Constructing fusion features The nonlinear mapping relationship with TCM syndrome elements outputs preliminary scores for the syndrome elements (data-driven preliminary judgment).

[0156] Model building process:

[0157] 1. Definition of Syndrome Elements: Eight core syndrome elements of TCM tongue diagnosis are selected, specifically:

[0158] : Cold syndrome (such as a pale tongue with a white coating, which corresponds to cold symptoms);

[0159] Heat syndrome elements (such as the heat signs corresponding to a red tongue with a yellow coating);

[0160] : Wind syndrome elements (such as the wind phenomenon corresponding to redness on the sides and tip of the tongue);

[0161] : Dampness syndrome elements (such as the dampness symptoms corresponding to a greasy tongue coating);

[0162] Dryness-related factors (such as the dryness symptoms corresponding to cracks on the tongue surface);

[0163] : Deficiency syndrome elements (such as the deficiency symptoms corresponding to a swollen and tender tongue);

[0164] : Empirical elements (such as the actual phenomena corresponding to thick moss);

[0165] : Blood stasis syndrome (such as blood stasis signs corresponding to ecchymosis on the tongue).

[0166] The score range for each element is [0, 1], where 0 indicates that the element is not present and 1 indicates that the element is extremely strong.

[0167] 2. Definition of surface equation: For each element (m=1, 2, ..., 8), construct a quadratic surface model that integrates feature and syndrome element scores (to capture the interaction between features, such as the enhancing effect of "red tongue + yellow coating" on heat syndrome elements):

[0168] (3.1)

[0169] in:

[0170] Evidence elements for surface fitting output Preliminary score (superscript "(1)" indicates the first stage output);

[0171] For fusion features The kth dimension (k=1, 2, ..., 2056); For fusion features The One dimension;

[0172] The coefficient of the constant term, The coefficient of the first-order term (the influence of a single feature on the evidence element). The interaction term coefficients (feature k and) Synergistic effects);

[0173] Only retain k< To avoid double counting, a total of 2 million items (sparsed using L1 regularization, retaining only important interaction items).

[0174] 3. Coefficient Calculation: Historical case data are fitted using regularized least squares method, with the objective function being:

[0175] (3.2)

[0176] in:

[0177] N=5000 is the number of training cases;

[0178] Evidence elements annotated by experts in the nth case Score;

[0179] The prediction score for the nth case is given by formula (3.1);

[0180] =0.01 is the L1 regularization coefficient (to suppress redundant coefficients and avoid overfitting).

[0181] Model training process:

[0182] 1. Training data: 5000 cases, each containing The expert annotation scores (0-1) of the feature vector and 8 evidence elements;

[0183] 2. Optimization method: The coordinate descent method is used to solve formula (3.2), and the coefficients of the constant term, linear term and interaction term are optimized in stages, iterating for 50 rounds until convergence;

[0184] 3. Model validation: Calculate the mean squared error (MSE) on a 1000-case test set. The model can be put into use if the MSE is less than 0.05.

[0185] Model application process:

[0186] The output of module 2 Substituting into formula (3.1), calculate the preliminary scores for the eight evidence elements:

[0187] (3.3)

[0188] Output: Preliminary score vector of evidence elements (Preliminary judgment based on data).

[0189] Submodule 3.2: Knowledge Graph Rule Reasoning

[0190] Core function: Based on the rules of traditional Chinese medicine theory, correct the initial score of syndrome elements and solve the "data bias" that may exist in surface fitting (such as misjudgment of rare tongue features due to insufficient training data).

[0191] Model building process:

[0192] 1. Knowledge Graph Construction: Construct a knowledge graph of tongue image features and evidence elements, containing three types of entities: tongue image features (F), relationships (R), and evidence elements (S), stored as triples (F, R, S, ...). ),in The rule weights range from 0.1 to 1.0, determined by consensus among three TCM experts. Specific examples of ternary rules (100 rules in total):

[0193] (F: <0.1 (red tongue), R: Support, S: (Heat syndrome element) =0.8);

[0194] (F: <0.3 (pale tongue), R: Support, S: (Cold syndrome) =0.7);

[0195] (F: >0.8 (greasy tongue coating), R: Support, S: (Dampness syndrome element) =0.6);

[0196] (F: >0.7 (thick tongue coating), R: Support, S: (Empirical element) =0.5);

[0197] (F: >0.9 (obvious crack characteristics), R: Support, S: (Dryness syndrome element) =0.6);

[0198] (F: >0.8 (white tongue coating), R: inhibition, S: (Heat syndrome element) =0.4) (White tongue coating inhibits heat syndrome).

[0199] 2. Rule matching mechanism: for the input Extract features that can be matched to the knowledge graph:

[0200] For traditional CV features: direct matching (e.g.) <0.1 corresponds to "red tongue");

[0201] For CNN features: matching via predefined thresholds (e.g.) A value >0.9 corresponds to "obvious cracks," and the threshold is determined by expert-annotated data.

[0202] Reasoning process:

[0203] 1. For each evidence element Collect all association rules (like The association rules include rules that support heat-related elements and rules that inhibit heat-related elements.

[0204] 2. Calculation rule correction amount :

[0205] (3.4)

[0206] in:

[0207] To support The set of rules, To suppress The set of rules;

[0208] The feature matching degree (1 indicates a match, 0 indicates no match);

[0209] 3. Correction of evidence element scores:

[0210] (3.5)

[0211] in =0.2 is the rule influence coefficient (controlling the strength of rule modification), and the truncation function. Ensure the score is within the range [0, 1].

[0212] Output: Rule-corrected element score vector .

[0213] Submodule 3.3: Random Forest Classification Optimization

[0214] Core function: It integrates the results of surface fitting and rule-based reasoning to output the final evidence element score and evidence type prediction, thus overcoming the limitations of single methods (such as insufficient generalization of surface fitting to new samples and the subjectivity of rule-based reasoning).

[0215] Model building process:

[0216] 1. Input features: and The input features are concatenated into a random forest:

[0217] (3.6)

[0218] 2. Random Forest Structure:

[0219] It contains 100 CART decision trees, each with a depth limit of 10 (to avoid overfitting);

[0220] Training samples for each tree are selected from the training set using Bootstrap sampling (70% sampling rate).

[0221] Node splitting criterion: The Gini index is used for the node sample set D.

[0222] (3.7)

[0223] in For the elements in sample set D The percentage of samples with a score greater than 0.5.

[0224] Model training process:

[0225] 1. Training data: 5000 cases, each containing input feature X and expert-annotated final evidence score. (0-1) Certificate Type Label (The only correct certificate type among the 10 types);

[0226] 2. Loss Function: For element score prediction, the mean squared error is used.

[0227] (3.8)

[0228] For pattern prediction, cross-entropy loss is used:

[0229] (3.9)

[0230] in This is the syndrome type label for the nth case (1 for the correct syndrome type, and 0 for the rest). Let be the probability of random forest predicting type p;

[0231] 3. Optimization strategy: By adjusting the number and depth of trees, the joint loss can be optimized. Minimize the time and train for 50 rounds until convergence.

[0232] Model application process:

[0233] 1. Final element score: The average of the prediction results for 100 trees:

[0234] (3.10)

[0235] in For the t-th tree, the evidence element is... The predicted score.

[0236] 2. Syndrome Type Prediction: The system includes 10 common syndrome types, specifically:

[0237] : Wind-cold syndrome affecting the exterior; : Wind-heat attacking the exterior syndrome; Damp-heat accumulation in the spleen syndrome; : Cold-dampness trapping the spleen syndrome; : Excessive stomach fire syndrome;

[0238] Lung dryness and fluid depletion syndrome; Spleen Qi deficiency syndrome; Kidney Yin Deficiency Syndrome; Qi stagnation and blood stasis syndrome; Phlegm-dampness obstructing the lungs syndrome.

[0239] Each certificate type The matching degree is calculated as follows:

[0240] (3.11)

[0241] in For proof type With evidence The association weights (determined by TCM experts, such as...) (Damp-heat accumulation in the spleen syndrome) =0.6 (heat) =0.7 (wet)).

[0242] Output: Final evidence element score Certificate type matching degree .

[0243] Module 3 Algorithm Fusion and Output

[0244] Fusion logic: The input X of the random forest directly depends on the surface fitting. and rule-based reasoning These three elements form a cascaded fusion of "data-driven preliminary judgment → knowledge-driven correction → integrated model optimization," ensuring that the output results not only conform to data patterns but also follow traditional Chinese medicine theory.

[0245] Output: Final evidence element score The certificate type matching degree T (as input to module 4).

[0246] Algorithm interaction relationships between modules

[0247] 1. Interaction between Module 2 and Module 3:

[0248] Fusion features of Module 2 It is the core input of the surface fitting algorithm in Module 3 (in Formula 3.1)

[0249] ), directly determining the initial score of the evidence element. ;

[0250] Surface coefficients of module 2 Indirectly affecting the rule reasoning of Module 3: through Correction features in (e.g.) Affects rule matching degree (For example, the brightness of the tongue body is more accurate after surface correction, and the accuracy of matching the "pale tongue body" rule is improved).

[0251] 2. Interactions within Module 3:

[0252] Surface fitting output It is the foundation of rule-based reasoning (in Formula 3.5) rely ), the amount of correction for rule-based reasoning It needs to be combined Dynamic size adjustment (e.g.) When the value is close to 1, the positive correction amount will be weakened.

[0253] and Both are used as inputs to the random forest (Equation 3.6), and the random forest learns the complementary relationship between them. Skilled at identifying patterns in data (Adept at correcting theoretical biases) Outputs final results And T.

[0254] Summary of core algorithm contributions

[0255] 1. Tongue 3D Surface Fitting Algorithm: By reconstructing the 3D shape, it solves the perspective distortion problem of 2D images, and its output surface coefficients... The CNN attention adjustment (Equation 2.5), color correction (Equation 2.12), and feature fusion (Equation 2.16) throughout Module 2 make feature extraction more adaptable to the natural shape of the tongue.

[0256] 2. Syndrome Element Correlation Surface Fitting Algorithm: By using a quadratic surface model (Formula 3.1) to capture the interaction between features (such as the enhancement of heat syndrome elements by "red tongue + yellow coating"), it breaks through the limitations of traditional linear models and provides a more accurate data-driven basis for syndrome differentiation.

[0257] 3. Cross-module collaboration mechanism: The feature quality of module 2 directly determines the diagnostic accuracy of module 3, while the rule reasoning of module 3 can correct the feature deviation of module 2 in reverse (such as avoiding misjudging "pale tongue with white coating" as a feature related to heat syndrome by using the constraints of traditional Chinese medicine theory), forming a closed-loop optimization of "feature extraction-diagnostic reasoning".

[0258] The closed loop between Module 2 (Intelligent Tongue Feature Extraction Module) and Module 3 (Intelligent Reasoning Module for Traditional Chinese Medicine Diagnosis) refers to an iterative optimization process of "feature extraction - diagnosis reasoning - feature correction." Its core is to use the reasoning results of Module 3 to correct the feature extraction deviations of Module 2 until the preset convergence conditions or clinical accuracy requirements are met. The closed loop termination condition must simultaneously meet the following four quantitative standards to ensure that the results are both stable and consistent with the logic of Traditional Chinese Medicine diagnosis:

[0259] 1. The number of iterations has reached the preset limit.

[0260] To avoid wasting computational resources due to infinite loops, an upper limit needs to be set for the number of closed-loop iterations. Based on clinical data, the correlation between tongue appearance features and syndrome elements usually stabilizes within three iterations; therefore, a maximum number of iterations is set. =3.

[0261] Iteration process: Module 2 output features →Module 3 Reasoning yields evidence scores →If the termination condition is not met, module 3 generates feature correction values. →Module 2 is based on Update features Then proceed to the next iteration (k is the current iteration number).

[0262] When k= At that time, regardless of whether it converges or not, the closed loop is forcibly terminated to avoid excessive iteration.

[0263] 2. The change in the element score is less than the threshold.

[0264] The final evidence score output by module 3 The score needs to remain stable in two consecutive iterations, meaning the difference between the two scores is less than a preset threshold, indicating that feature extraction and dialectical reasoning have reached a consensus.

[0265] Quantitative indicators: defining the change in evidence element scores The Euclidean distance between the element scores in the two iterations:

[0266] ;

[0267] End threshold: (Based on clinical expert consensus, this threshold corresponds to fluctuations in the symptom score within the range of "no clinical significance").

[0268] 3. The fluctuation in the type matching degree is less than the threshold.

[0269] The matching degree of the first candidate certificate type needs to remain stable in two consecutive iterations, and the top 3 certificate types (the top 3 candidate certificate types in the ranking) The order of the results remained unchanged, ensuring the consistency of the dialectical results.

[0270] Quantitative Indicator 1: Change Rate of First-Choice Certificate Type Matching :

[0271] ;(like ,but )in Let be the first-order type matching degree of the k-th iteration.

[0272] Quantitative Indicator 2: Consistency of Top 3 Evidence Type Ranking If the top 3 proof types and rankings in round k are exactly the same as those in round k-1, then =1, otherwise =0.

[0273] End threshold: (Primary certificate type matching degree fluctuation ≤10%) and =1 (sorting remains unchanged).

[0274] 4. The correlation error between the core features and the syndrome type is less than the threshold.

[0275] fused features output by module 2 Certificate type output by module 3 It must meet the clinical logic of the "feature-syndrome" association, that is, the support error of the core feature to the syndrome is within an acceptable range.

[0276] Core feature definition: For the first-order certificate type Select association weights >0.6 evidence (For example, the core symptom of damp-heat accumulation in the spleen is heat) and wet ), and then in the reverse positioning module 2, features strongly correlated with these elements (such as Related (Red tongue) Related (Greasy tongue coating)

[0277] Association error calculation: Let the theoretical value range of the core feature be... (Based on statistics of typical cases annotated by TCM experts), the actual characteristic value is Then the error for:

[0278] ;

[0279] End threshold: (The core eigenvalues ​​are within ±15% of the theoretical range).

[0280] Logic for determining the end of the closed loop

[0281] The closed loop terminates when any of the following conditions are met simultaneously:

[0282] 1. Number of iterations (Forced termination);

[0283] 2. In two consecutive iterations, and and =1 and (Natural convergence).

[0284] Core significance: By using multi-dimensional quantitative conditions, we can ensure that the feature extraction results and dialectical reasoning results at the end of the closed loop converge at the data level and conform to the clinical logic of traditional Chinese medicine, thus avoiding diagnostic errors caused by the deviation of a single indicator.

[0285] Module 4: Diagnostic Result Integration and Output Module (process as follows) Figure 5 (As shown)

[0286] Final evidence score based on the output of Module 3 The module for matching syndrome type T needs to transform abstract data into diagnostic suggestions that conform to the clinical practice of traditional Chinese medicine, and realize the visual association of "feature-syndrome type-suggestion" to assist physicians in making quick decisions.

[0287] Core functionality expansion:

[0288] 1. Priority ranking and explanation of certificate types:

[0289] The output of module 3 Sort by matching degree in descending order and select the top 3 candidate certificate types. ,in .

[0290] Generate a "feature support list" for each candidate certificate type: extract... Medium weight >0.5 evidence elements (such as (Damp-heat accumulation in the spleen syndrome) Related (Heat) and (wet), and trace back to the fusion features of module 2. ,For example:

[0291] “ (Match degree 0.82) Main supported features: Red tongue ( =0.08), yellow tongue coating (

[0292] =0.2), greasy tongue coating ( =0.85).

[0293] 2. Visual annotation of tongue features:

[0294] In standard corrected image Above, use different colors to mark the areas related to the core evidence elements:

[0295] The red box marks the red area of ​​the tongue (corresponding to...). Heat syndrome elements);

[0296] The yellow box marks the areas with thick, greasy tongue coating (corresponding to...) (Dampness syndrome elements)

[0297] The cracked area is marked in blue (corresponding to...) Dryness syndrome element).

[0298] Generate feature heatmaps: This involves extracting the dimension from the CNN features that has the highest correlation with the pattern (e.g., [the dimension with the highest correlation with the pattern]). (Corresponding cracks) Visualization: the darker the color, the more prominent the feature.

[0299] 3. Generation of clinical decision recommendations:

[0300] against Based on the TCM diagnosis and treatment guidelines, "further consultation directions" are provided, for example:

[0301] like First and foremost, it is recommended to ask "whether it is accompanied by abdominal distension and loose stools" (typical accompanying symptoms of damp-heat accumulation in the spleen syndrome);

[0302] like (Spleen Qi Deficiency Syndrome) is the next most important factor; it is recommended to ask "Do you feel easily tired after eating?"

[0303] Output "Key Points for Differential Diagnosis" to distinguish similar syndrome types, for example: and Differential diagnosis of (cold-dampness trapping the spleen syndrome): The former has a yellow and greasy tongue coating, while the latter has a white and greasy tongue coating.

[0304] Output format:

[0305] The structured report (PDF format) includes: Patient ID, image capture time, candidate syndrome types and matching degree, feature-annotated images, and consultation suggestions;

[0306] Data interface output (JSON format) is provided for the hospital's HIS system to use. Fields include... , Key features, etc.

[0307] Connection with Module 3: Directly read from Module 3 And T, where the evidence element score Used to quantify feature support strength (e.g.) A score >0.7 is labeled as "significant thermal phenomenon" (syndrome matching degree). Determine the priority of the recommendations.

[0308] Strengthening inter-module collaboration

[0309] Module 1 → Module 2: Environmental Parameters It is used not only for image correction, but also as a "difficulty coefficient" for surface fitting in Module 2, dynamically adjusting algorithm parameters (such as increasing the color correction coefficient when there is insufficient light). Up to 0.2).

[0310] Module 3 → Module 4: Evidence Element Scoring Provide quantitative basis for visual annotation (the higher the score, the darker the annotation color), and the degree of matching of the syndrome type T determines the level of detail of the suggestion (the explanation of the first syndrome type is twice as long as that of the second).

[0311] The embodiments described above are for illustrative purposes only and are not intended to limit the invention. Therefore, any changes in numerical values ​​or substitutions of equivalent elements should still fall within the scope of this invention.

[0312] The above detailed description will enable those skilled in the art to understand that the present invention can indeed achieve the aforementioned objectives and has complied with the provisions of the Patent Law.

[0313] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention. The above descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. It should be noted that any modifications, equivalent substitutions, and improvements made within the spirit and principles of the invention should be included within the scope of protection of the invention.

[0314] It should be noted that the above description of the process is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to the process under the guidance of this specification. However, these modifications and changes remain within the scope of this specification.

[0315] The basic concepts have been described above. Obviously, for those skilled in the art who have read this application, the above disclosure is merely illustrative and does not constitute a limitation of this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are suggested in this application, and therefore, such modifications, improvements, and corrections still fall within the spirit and scope of the exemplary embodiments of this application.

[0316] Furthermore, this application uses specific terms to describe its embodiments. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that "an embodiment," "one embodiment," or "an alternative embodiment" mentioned twice or more in different positions in this specification do not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application can be appropriately combined.

[0317] Furthermore, those skilled in the art will understand that aspects of this application can be described and illustrated through several patentable types or situations, including any new and useful combination of processes, machines, products, or substances, or any new and useful improvements thereof. Therefore, aspects of this application can be implemented entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. All of the above hardware or software can be referred to as a “unit,” “module,” or “system.” Furthermore, aspects of this application can take the form of a computer program product embodied in one or more computer-readable media, wherein computer-readable program code is contained therein.

[0318] The computer program code required for the operation of each part of this application can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C, VB.NET, and Python; general programming languages ​​such as C; Visual Basic, Fortran2103, Perl, COBOL2102, PHP, and ABAP; dynamic programming languages ​​such as Python, Ruby, and Groovy; or other programming languages. This program code can run entirely on the user's computer, or as a standalone software package on the user's computer, or partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any network, such as a local area network (LAN) or wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service such as Software as a Service (SaaS).

[0319] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this application are not intended to limit the order of the processes and methods of this application. Although some currently considered useful embodiments of the invention have been discussed in the foregoing disclosure by way of various examples, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the substance and scope of the embodiments of this application. For example, although the implementation of the various components described above can be embodied in a hardware device, it can also be implemented as a purely software solution, such as an installation on an existing server or mobile device.

[0320] Similarly, it should be noted that, in order to simplify the description of the present application and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of the embodiments of the present application sometimes combines multiple features into a single embodiment, drawing, or description thereof. However, this approach of the present application should not be construed as reflecting an intention that the claimed subject matter requires more features than expressly recited in each claim. Rather, the subject of the invention should possess fewer features than in any single embodiment described above.

Claims

1. A TCM tongue diagnosis-assisted diagnostic system based on artificial intelligence, characterized in that, It includes a tongue image acquisition and preprocessing module, a tongue image feature intelligent extraction module, a traditional Chinese medicine syndrome differentiation intelligent reasoning module, and a diagnosis result integration and output module connected in sequence. The tongue image acquisition and preprocessing module is used to acquire the original tongue image and shooting environment parameters, and to correct, segment and denoise the original tongue image, and output a standardized tongue image and a tongue body segmentation mask. The tongue body segmentation mask is a binary image that marks the tongue body region. The tongue image feature intelligent extraction module receives a standardized tongue image and a tongue segmentation mask. First, based on the tongue region marked by the tongue segmentation mask, the standardized tongue image is converted into a standard viewpoint corrected image and surface coefficients are obtained through a three-dimensional surface fitting algorithm. Then, deep semantic features are extracted by combining the surface coefficients with an improved CNN model. At the same time, shallow features are extracted by combining the surface coefficients with traditional computer vision methods. Finally, the deep semantic features and shallow features are fused to obtain a fused feature vector. The TCM syndrome differentiation intelligent reasoning module receives the fused feature vector, first obtains the preliminary score of the syndrome element through the syndrome element correlation surface fitting algorithm, then corrects the preliminary score of the syndrome element through the knowledge graph rule reasoning to obtain the corrected syndrome element score, and finally obtains the final syndrome element score and syndrome type matching degree by combining the preliminary score and the corrected syndrome element score through the random forest model. The diagnostic result integration and output module receives the final syndrome element score and syndrome type matching degree, and generates and outputs an auxiliary diagnostic report.

2. The artificial intelligence-based TCM tongue diagnosis auxiliary system according to claim 1, characterized in that, In the intelligent tongue image feature extraction module, the improved CNN model has a surface perception attention layer after the Conv3 stage. The attention weights of this layer are calculated based on the surface coefficients and are used to perform weighted processing on the feature map output by the Conv3 stage.

3. The artificial intelligence-based TCM tongue diagnosis auxiliary system according to claim 1, characterized in that, In the intelligent extraction module for tongue features, when extracting shallow features using traditional computer vision methods, the standard viewpoint calibrated image is first converted from RGB space to HSV space, the average hue, saturation, and brightness of the tongue body area and the tongue coating area are calculated, and then the brightness is corrected by combining the surface coefficient. At the same time, the entropy value and thickness of the tongue coating area are calculated to obtain shallow features containing the corrected brightness, entropy value, and thickness.

4. The artificial intelligence-based TCM tongue diagnosis auxiliary diagnostic system according to claim 1, characterized in that, In the intelligent tongue feature extraction module, the fusion weight of deep semantic features and shallow features is calculated by the Sigmoid function based on the quadratic term coefficient in the surface coefficient. The fusion weight is used to weight and concatenate the deep semantic features and shallow features to obtain the fused feature vector.

5. The artificial intelligence-based TCM tongue diagnosis auxiliary diagnostic system according to claim 1, characterized in that, In the TCM syndrome differentiation intelligent reasoning module, the syndrome element correlation surface fitting algorithm adopts a quadratic polynomial surface model, takes the fused feature vector as input, and solves the model coefficients by combining the least squares method with the training data, thereby obtaining the preliminary scores of 8 syndrome elements, which include cold, heat, wind, dampness, dryness, deficiency, excess and blood stasis syndrome elements.

6. The artificial intelligence-based TCM tongue diagnosis auxiliary system according to claim 1, characterized in that, In the TCM syndrome differentiation intelligent reasoning module, the knowledge graph rule reasoning is based on a triplet knowledge graph containing tongue appearance features, correlation relationships, syndrome elements and rule weights. First, it matches and fuses the features in the feature vector that correspond to the knowledge graph, then calculates the syndrome element correction amount, and finally combines the initial syndrome element score and the syndrome element correction amount to obtain the corrected syndrome element score.

7. The artificial intelligence-based TCM tongue diagnosis auxiliary system according to claim 1, characterized in that, In the TCM syndrome differentiation intelligent reasoning module, the input of the random forest model is a concatenated vector of the initial score and the corrected score of the syndrome element. The final score of the syndrome element is obtained by averaging the prediction results of the decision tree, and the matching degree of the syndrome type is calculated based on the correlation weight between the final score of the syndrome element and the syndrome type.

8. The artificial intelligence-based TCM tongue diagnosis auxiliary system according to claim 1, characterized in that, The intelligent tongue feature extraction module and the intelligent TCM syndrome differentiation reasoning module form a closed-loop iteration. The iteration process is as follows: the intelligent TCM syndrome differentiation reasoning module generates feature correction values ​​based on the final syndrome element scores; the intelligent tongue feature extraction module updates the fused feature vector based on the feature correction values; and the intelligent TCM syndrome differentiation reasoning module recalculates the final syndrome element scores and syndrome matching degree based on the updated fused feature vector.

9. The artificial intelligence-based TCM tongue diagnosis auxiliary system according to claim 8, characterized in that, The closing loop iteration ends under any of the following conditions: the number of iterations reaches the preset maximum number of iterations, 3 times; in two consecutive iterations, the Euclidean distance between the evidence element scores does not exceed 0.05, the change rate of the first evidence type matching degree does not exceed 0.1, the order of the first 3 evidence types is consistent, and the deviation of the core feature value from the theoretical range does not exceed 15%.

10. The artificial intelligence-based TCM tongue diagnosis auxiliary system according to claim 1, characterized in that, The shooting environment parameters output by the tongue image acquisition and preprocessing module include light intensity, color temperature, and shooting angle. The shooting angle is used to adjust the number of iterations of the tongue body three-dimensional surface fitting algorithm in the tongue image feature intelligent extraction module, and the light intensity is used to adjust the brightness correction coefficient in traditional computer vision methods.

Images