Tongue feature analysis method and apparatus, and electronic device and storage medium

Abstract

The present application relates to a tongue feature analysis method and apparatus, and an electronic device and a storage medium. The method comprises: fusing a trained tongue detection model and a trained tongue segmentation model to obtain a target segmentation model, and performing segmentation processing on an image dataset by means of the target segmentation model, so as to obtain a segmented tongue image dataset; labeling the segmented tongue image dataset to obtain a tongue feature classification dataset, and on the basis of the tongue feature classification dataset, training a tongue feature classification model to obtain a trained tongue feature classification model; acquiring target image data to be processed, and inputting said target image data into the target segmentation model to obtain segmented tongue image data of said target image data; and performing feature analysis on the segmented tongue image data by means of the trained tongue feature classification model, so as to obtain a tongue feature analysis result, thereby not only improving the training efficiency of a model, but also significantly improving the stability and reliability of classification.

Description

Tongue feature analysis methods, devices, electronic equipment and storage media

[0001] This disclosure claims priority to Chinese Patent Application No. 202510134818.0, filed on February 7, 2025, entitled “Tongue Feature Analysis Method, Apparatus, Electronic Device and Storage Medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of feature analysis technology, and in particular to a tongue feature analysis method, apparatus, electronic device and storage medium. Background Technology

[0003] In the field of Traditional Chinese Medicine (TCM), tongue observation and analysis are crucial diagnostic tools. The tongue, encompassing its coating and texture, reflects various aspects of the body's health. Traditional tongue analysis relied heavily on the doctor's experience, involving visual observation and diagnosing based on that experience. However, with the rapid development of computer vision and artificial intelligence technologies, tongue analysis is gradually shifting from manual diagnosis towards automation and intelligence. Deep learning technology has provided powerful support for the analysis of tongue features.

[0004] Currently, tongue feature analysis mainly employs the following approaches: 1) Traditional machine learning combined with feature engineering, which involves manually designing feature extraction for tongue characteristics such as color and texture, and then inputting these features into a machine learning classifier for classification; 2) Deep learning-based tongue segmentation and classification models, which use convolutional neural networks (CNNs) to segment tongue images and combine them with classification models to classify features such as tongue texture, tongue coating, and tongue color. To improve classification accuracy, this type of method typically uses an independent classification model for each feature, for example, one model for tongue texture classification and another model for tongue coating classification; 3) Using improved segmentation algorithms and transfer learning models, the tongue region is first extracted from the tongue image using a segmentation model, and then the tongue features are classified using a pre-trained classification model.

[0005] While the three schemes mentioned above have achieved the function of segmenting the tongue region and classifying its features to a certain extent, they also have some drawbacks.

[0006] 1) Solution 1 relies on manually designed feature extraction, such as low-level features like color and texture. These features are typically defined by experts based on experience and are difficult to fully and deeply reflect the complex characteristics of the tongue. Therefore, manually extracted features cannot adequately capture the deep information in tongue images, resulting in limited classification performance, insufficient generalization ability, and poor performance when dealing with different tongue features and individual differences.

[0007] 2) Option 2 relies on manual separation of tongue coating and tongue body, and the quality of annotation directly affects the performance of the model; secondly, since multiple independent classification models are required, the complexity of the system increases, the collaboration and information sharing between models are insufficient, and detailed information in the tongue image is easily lost, resulting in limited classification effect.

[0008] 3) The effectiveness of transfer learning in Scheme 3 largely depends on the characteristics of the original data of the pre-trained model. If the pre-trained data differs greatly from the data in the actual application scenario, the model's performance will be greatly affected. The generalization ability and robustness of the transfer learning model are still insufficient in some complex tasks, especially in the fine classification of tongue features. Summary of the Invention

[0009] This disclosure provides a tongue feature analysis method, apparatus, electronic device, and storage medium that can solve the aforementioned technical problems.

[0010] In a first aspect, embodiments of this disclosure provide a tongue feature analysis method, the method comprising: constructing an image dataset, the image dataset including a large amount of image data containing tongue images; training a tongue detection model and a tongue segmentation model based on the image dataset to obtain trained tongue detection models and trained tongue segmentation models; fusing the trained tongue detection models and trained tongue segmentation models to obtain a target segmentation model, and segmenting the image dataset using the target segmentation model to obtain a tongue segmentation image dataset; labeling the tongue segmentation image dataset to obtain a tongue feature classification dataset, and training a tongue feature classification model based on the tongue feature classification dataset to obtain a trained tongue feature classification model; acquiring target image data to be detected, inputting the target image data into the target segmentation model to obtain tongue segmentation image data of the target image data to be detected; and performing feature analysis on the tongue segmentation image data using the trained tongue feature classification model to obtain tongue feature analysis results.

[0011] In some preferred embodiments, the tongue detection model uses a lightweight network as the backbone network and employs depthwise separable convolutions to reduce the number of parameters and computational cost while maintaining high detection accuracy. Spatial attention and channel attention modules are introduced before the final detection head to enhance the model's ability to represent tongue region features. The tongue feature classification model adopts a multi-task learning architecture, constructing different tongue feature classification tasks into a unified fusion model. The overall architecture of the tongue feature classification model includes the following parts: a low-level feature extraction network, a task branch module, a shared feature layer, a loss function and optimization strategy, and model output.

[0012] The model comprises several layers: a low-level feature extraction network for extracting low- and high-level features from the tongue image; a task branch module consisting of a network structure including multi-layer feature extraction, skip connections, fully connected layers, and activation functions for classifying and predicting tongue color, tongue body, tongue coating color, tongue coating, and overall features; a shared feature layer for fusing and re-encoding the low- and high-level features output by the low-level feature extraction network, providing rich semantic features for each task branch; each task branch has an independent loss function, with cross-entropy loss function applied to all task branches for tongue color, tongue body, tongue coating color, tongue coating, and overall features; and a multi-task classification model that simultaneously outputs classification predictions for tongue color, tongue body, tongue coating color, tongue coating, and overall features, with confidence scores for different categories for each classification task. The tongue feature classification model extracts multi-level features, fuses these features through feature concatenation, and uses these features as input for each classification task. An independent classification head is designed for each task to enable the model to perform fine-grained learning of specific features for each task, ensuring classification accuracy for each task. The tongue segmentation model is based on a multi-layer convolutional neural network. In the encoder, a CBAM attention mechanism and a residual module are introduced to enable the model to focus on more important tongue feature regions, improving gradient flow and feature learning capabilities. In the decoder, multi-scale feature fusion is introduced to better capture detailed information about the tongue edges. The tongue detection model and the tongue segmentation model are jointly trained. After fusing the trained tongue detection and segmentation models, a unified interactive segmentation module is constructed to improve segmentation accuracy and efficiency. Using an interactive segmentation strategy, after obtaining the segmented tongue image, tongue coating separation is not performed; instead, multi-task classification is directly performed based on the complete segmented tongue image to enhance classification accuracy and stability.

[0013] After training the tongue feature classification model based on the tongue feature classification dataset to obtain a trained tongue feature classification model, the method further includes: integrating the trained tongue feature classification model, color space conversion algorithm, channel analysis algorithm, and verification algorithm to obtain a statistical analysis module; inputting the tongue segmentation image data into the statistical analysis module, performing feature analysis on the tongue segmentation image data using the trained tongue feature classification model to obtain the confidence level of each feature category; using the color space conversion algorithm to convert the tongue segmentation image data into color-opposite space channel image data; inputting the tongue segmentation image data and the color-opposite space channel image data into the channel analysis algorithm respectively for channel analysis to obtain red-green-blue channel analysis results and color-opposite space channel analysis results; and performing distribution verification on the tongue segmentation image data and the color-opposite space channel image data using the verification algorithm to obtain the verification values ​​of the red-green-blue channels and the color-opposite space channels.

[0014] The method further includes: extracting various tongue feature analysis and channel statistical test results based on the inference results of the statistical analysis module, and outputting a structured statistical analysis report. The statistical analysis report includes multi-task feature classification prediction results, color channel statistics, and KS test statistics. The multi-task feature classification prediction results include predicted values ​​and confidence scores for tongue color, tongue texture, tongue coating color, tongue texture, and overall features, generating corresponding charts to display the multi-task feature classification prediction results. The color channel statistics include the mean and standard deviation of the RGB and Lab channels, as well as the distance of the channel spatial mean from the eight vertices. The KS test statistics include the KS test p-value comparing the tongue image distribution with the standard distribution, used to determine the significant difference between the distribution of each color channel and the standard distribution, generating corresponding KS test statistic charts for the KS test statistics.

[0015] Secondly, embodiments of this disclosure provide a tongue feature analysis device, the device comprising: a construction unit for constructing an image dataset, the image dataset including a large amount of image data containing tongue images; a first training unit for training a tongue detection model and a tongue segmentation model based on the image dataset, respectively, to obtain trained tongue detection models and trained tongue segmentation models; a first obtaining unit for fusing the trained tongue detection models and trained tongue segmentation models to obtain a target segmentation model, and segmenting the image dataset using the target segmentation model to obtain a tongue segmentation image dataset; a second training unit for labeling the tongue segmentation image dataset to obtain a tongue feature classification dataset, and training a tongue feature classification model based on the tongue feature classification dataset to obtain a trained tongue feature classification model; a second obtaining unit for acquiring target image data to be detected, inputting the target image data into the target segmentation model to obtain tongue segmentation image data of the target image data to be detected; and a feature analysis unit for performing feature analysis on the tongue segmentation image data using the trained tongue feature classification model to obtain tongue feature analysis results.

[0016] In some preferred embodiments, the tongue detection model employs a lightweight network as its backbone, using depthwise separable convolutions to reduce the number of parameters and computational cost while maintaining high detection accuracy. Spatial attention and channel attention modules are introduced before the final detection head to enhance the model's ability to represent tongue region features. The tongue feature classification model adopts a multi-task learning architecture, constructing different tongue feature classification tasks within a unified, fused large model. The overall architecture of the tongue feature classification model includes the following components: a low-level feature extraction network, a task branching module, a shared feature layer, a loss function and optimization strategy, and model output.

[0017] The model comprises several layers: a low-level feature extraction network for extracting low- and high-level features from the tongue image; a task branch module consisting of a network structure including multi-layer feature extraction, skip connections, fully connected layers, and activation functions for classifying and predicting tongue color, tongue body, tongue coating color, tongue coating, and overall features; a shared feature layer for fusing and re-encoding the low- and high-level features output by the low-level feature extraction network, providing rich semantic features for each task branch; each task branch has an independent loss function, with cross-entropy loss function applied to all task branches for tongue color, tongue body, tongue coating color, tongue coating, and overall features; and a multi-task classification model that simultaneously outputs classification predictions for tongue color, tongue body, tongue coating color, tongue coating, and overall features, with confidence scores for different categories for each classification task. The tongue feature classification model extracts multi-level features, fuses these features through feature concatenation, and uses these features as input for each classification task. An independent classification head is designed for each task to enable the model to perform fine-grained learning of specific features for each task, ensuring classification accuracy for each task. The tongue segmentation model is based on a multi-layer convolutional neural network. In the encoder, a CBAM attention mechanism and a residual module are introduced to enable the model to focus on more important tongue feature regions, improving gradient flow and feature learning capabilities. In the decoder, multi-scale feature fusion is introduced to better capture detailed information about the tongue edges. The tongue detection model and the tongue segmentation model are jointly trained. After fusing the trained tongue detection and segmentation models, a unified interactive segmentation module is constructed to improve segmentation accuracy and efficiency. Using an interactive segmentation strategy, after obtaining the segmented tongue image, tongue coating separation is not performed; instead, multi-task classification is directly performed based on the complete segmented tongue image to enhance classification accuracy and stability.

[0018] The tongue feature analysis device further includes: an integration unit; the integration unit is used to integrate the trained tongue feature classification model, color space conversion algorithm, channel analysis algorithm, and verification algorithm to obtain a statistical analysis module; the tongue segmentation image data is input into the statistical analysis module, and the trained tongue feature classification model is used to perform feature analysis on the tongue segmentation image data to obtain the confidence level of each feature category; the color space conversion algorithm is used to convert the tongue segmentation image data into color-opposite space channel image data; the tongue segmentation image data and the color-opposite space channel image data are respectively input into the channel analysis algorithm for channel analysis to obtain the red-green-blue channel analysis results and the color-opposite space channel analysis results; the verification algorithm is used to perform distribution verification on the tongue segmentation image data and the color-opposite space channel image data to obtain the verification values ​​of the red-green-blue channels and the color-opposite space channels.

[0019] The integration unit is also used to extract various tongue feature analysis and channel statistical test results based on the inference results of the statistical analysis module, and output a structured statistical analysis report. The statistical analysis report includes multi-task feature classification prediction results, color channel statistics, and KS test statistics. The multi-task feature classification prediction results include predicted values ​​and confidence scores for tongue color, tongue texture, tongue coating color, tongue texture, and overall features, and generate corresponding charts to display the multi-task feature classification prediction results. The color channel statistics include the mean and standard deviation of RGB and Lab channels, as well as the distance of the channel space mean from the eight vertices. The KS test statistics include the KS test p-value of the test tongue image distribution and the standard distribution, used to determine the significant difference between the distribution of each color channel and the standard distribution, and generate corresponding KS test statistics charts for the KS test statistics.

[0020] Thirdly, embodiments of this disclosure provide an electronic device, including: a memory; a processor; and a computer program; wherein the computer program is stored in the memory and configured to be executed by the processor to implement the method as described in the first aspect.

[0021] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method as described in the first aspect.

[0022] Fifthly, embodiments of this disclosure also provide a computer program product comprising a computer program or instructions that, when executed by a processor, implement the method described in the first aspect.

[0023] The tongue feature analysis method, apparatus, electronic device, and storage medium provided in this disclosure construct an image dataset, which includes a large number of image data containing tongue images. Based on this image dataset, a tongue detection model and a tongue segmentation model are trained to obtain trained tongue detection and segmentation models. These trained models are then fused to obtain a target segmentation model. The target segmentation model is used to segment the image dataset to obtain a tongue segmentation image dataset. This dataset is then labeled to obtain a tongue feature classification dataset. Based on this dataset, a tongue feature classification model is trained to obtain a trained tongue feature classification model. Target image data to be detected is acquired and input into the target segmentation model to obtain tongue segmentation image data of the target image data. The trained tongue feature classification model is then used to perform feature analysis on the tongue segmentation image data to obtain tongue feature analysis results. This disclosure enables feature sharing and information complementarity, thereby improving the accuracy and robustness of feature classification. By training directly on complete tongue segmentation images, the potential information loss during the separation of tongue coating and tongue body is effectively avoided. The model can more comprehensively learn the complex relationships and detailed information between features such as tongue color, tongue body, and coating color, ensuring more accurate and objective classification results. This reduces reliance on manual annotation and errors in tongue coating separation, not only improving the training efficiency of the model but also significantly enhancing the stability and reliability of classification.

[0024] The advantages of this disclosure include: compared to traditional deep learning methods that design separate classification models for each feature, this disclosure overcomes the problem of information sharing difficulties through a feature-sharing strategy. The tongue feature classification model designed in this disclosure first extracts multi-level features and then fuses these features together through feature concatenation operations, using them as input for each classification task. This feature-sharing approach allows multiple classification tasks to maintain their independence while using the same set of basic features. Under this information-sharing mechanism, different classification tasks can benefit from each other's features, thus utilizing more comprehensive feature information during classification. Especially when there is overlap and similarity between tasks (such as tongue coating and tongue body), this method can better preserve the potential connections between different tasks. Furthermore, by designing independent classification heads for each task, the model can perform fine-grained learning of specific features for each task, ensuring the classification accuracy of each task. Therefore, by combining the low-level feature extraction network with the information-sharing module, this disclosure achieves information sharing among multiple tasks, reducing the information loss problem common in independent model design and ensuring the completeness and accuracy of the classification results. Simultaneously, based on achieving information sharing, this disclosure further reduces redundant computation, improves model operating efficiency, and makes the training process faster and more efficient.

[0025] Traditional classification methods in machine learning and deep learning typically rely on manually labeled image data after separating the tongue coating from the tongue body to train and test the model. However, the quality of the tongue coating separation process and subsequent manual labeling directly affects the model's performance. Inaccurate separation or incorrect labeling leads to information loss or missing details. Furthermore, the tongue coating and tongue body share significant similarities in color and texture, making objective ground truth difficult. Therefore, manually separating the tongue coating and tongue body is extremely challenging. This disclosure abandons this method that relies on tongue coating separation, employing an automated interactive segmentation strategy. It eliminates the need for tongue coating separation, directly utilizing large-scale complete tongue segmentation images for training, thereby reducing reliance on manual labeling and errors in the tongue coating separation process. The tongue feature classification model can learn more complete feature information from large-scale complete tongue segmentation images, including subtle changes in the tongue coating and tongue body, thus enhancing the accuracy and stability of classification.

[0026] Traditional segmentation methods in machine learning and deep learning typically use the original image for processing. However, original images often contain a large amount of complex background information, especially when the image size is large. This background noise can easily interfere with the segmentation model, thus reducing the model's segmentation accuracy and efficiency. Furthermore, large images significantly increase computational resource consumption during processing, leading to higher memory requirements and processing time. This is particularly true in high-resolution image applications, where memory overflow is more likely, affecting model deployment and performance. In contrast, interactive segmentation strategies accurately locate the tongue region using a tongue detection model, generate bounding boxes, and then crop the tongue. This not only effectively filters out complex background noise from the original image but also ensures the accuracy of the segmented region. The cropped image is smaller, reducing computational complexity while increasing processing speed, and ensuring that the model uses high-quality tongue segmentation image data for feature extraction and classification. This method provides clean and accurate data input for subsequent classification and analysis, greatly improving the overall performance of tongue diagnosis analysis. It also demonstrates greater device adaptability, making it particularly suitable for computationally resource-sensitive applications such as mobile or embedded devices.

[0027] Traditional tongue diagnosis relies on the doctor's experience and manual observation, which is easily influenced by subjective factors, leading to poor repeatability and consistency of results. This disclosure utilizes a multi-dimensional tongue feature analysis and statistical quantification method, incorporating both qualitative and quantitative results, to objectively and quantitatively analyze key features such as tongue color and coating color. By combining RGB and Lab color space conversion, channel analysis, and the KS test algorithm, it can deeply quantify the color distribution in tongue images. Furthermore, by analyzing the differences between the channel distribution of test samples and the standard distribution, it provides doctors with quantitative analysis results of tongue color and coating color. This quantitative analysis method not only improves the scientific rigor and objectivity of tongue diagnosis but also provides standardized data references for clinical diagnosis, reducing errors caused by human intervention and improving the accuracy and repeatability of tongue diagnosis analysis. This disclosure achieves a fully automated workflow for tongue feature analysis, from image acquisition, tongue detection, tongue segmentation, tongue classification to statistical analysis, eliminating human intervention, reducing subjective errors, and greatly improving the efficiency and accuracy of intelligent tongue diagnosis. Compared with traditional methods, this disclosure ensures the consistency of analysis results through a standardized automated workflow, making it suitable for large-scale data processing and analysis, and possessing higher reliability and practicality. Attached Figure Description

[0028] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0029] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 is a flowchart of the tongue feature analysis method provided in an embodiment of this disclosure;

[0031] Figure 2 is a schematic diagram of the overall architecture of the tongue feature analysis method provided in the embodiments of this disclosure;

[0032] Figure 3 is a flowchart of a tongue feature analysis method provided in another embodiment of this disclosure;

[0033] Figure 4 is a flowchart of a tongue feature analysis method provided in another embodiment of this disclosure;

[0034] Figure 5 is a diagram of the tongue detection model architecture provided in an embodiment of this disclosure;

[0035] Figure 6 is a diagram of the tongue segmentation model architecture provided in an embodiment of this disclosure;

[0036] Figure 7 is a diagram of the tongue feature classification model architecture provided in an embodiment of this disclosure;

[0037] Figure 8 is a schematic diagram of the workflow of the tongue feature analysis method for interactive segmentation and fusion large model provided in the embodiments of this disclosure;

[0038] Figure 9 is a schematic diagram of the workflow of the interactive segmentation module provided in the embodiments of this disclosure;

[0039] Figure 10 is a schematic diagram of the workflow of the statistical analysis module provided in the embodiments of this disclosure;

[0040] Figure 11 is a schematic diagram of the tongue feature analysis device provided in an embodiment of this disclosure;

[0041] Figure 12 is a schematic diagram of the structure of the electronic device provided in the embodiment of this disclosure. Detailed Implementation

[0042] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0043] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0044] All actions taken to obtain information or data in this application were carried out in accordance with the relevant data protection laws and policies of the country where the information or data was located, and with the authorization of the relevant information or data owner.

[0045] To address this problem, this disclosure provides a tongue feature analysis method, which will be described below with reference to specific embodiments.

[0046] Figure 1 is a flowchart of the tongue feature analysis method provided in this embodiment. The execution subject of this method is an electronic device, which can be a tongue feature analysis device, specifically a tablet computer, a laptop computer, or a personal computer. This method can be applied to scenarios involving tongue feature analysis, improving the accuracy and efficiency of tongue feature classification, reducing reliance on manual annotation, and solving the problems of information loss and low efficiency in the classification and segmentation process of traditional methods. It is understood that the tongue feature analysis method provided in this embodiment can also be applied to other scenarios.

[0047] The tongue feature analysis method shown in Figure 1 is described below. The specific steps of this method are as follows.

[0048] S101. Construct an image dataset, which includes a large number of image data containing images of tongues.

[0049] As shown in Figure 2, the user performs image acquisition. In response to this operation, the electronic device constructs an image dataset, which includes a large number of images containing tongue images. Constructing this image dataset is fundamental to model training. Through systematic and standardized image data acquisition, annotation, and dataset creation, the model is trained with diverse and high-quality data support. This provides a solid foundation for subsequent tongue detection and segmentation models, effectively improving the model's performance and reliability.

[0050] In some embodiments, S101 may include, but is not limited to, S1011, S1012, S1013, S1014, and S1015.

[0051] S1011. Collect a large amount of image data containing images of the tongue.

[0052] S1012. Perform color calibration processing on each of the image data to obtain color-calibrated image data for each image.

[0053] In this step, as shown in Figure 8, the electronic device can perform color calibration on each image data to obtain color-calibrated image data. Optionally, a high-resolution camera with color calibration capabilities can be used to acquire the color-calibrated image data. This avoids color deviation problems caused by differences in the characteristics of the shooting equipment or changes in ambient lighting conditions. The color-calibrated tongue image serves as the basic data source for the model, providing reliable data support for subsequent tongue feature analysis, segmentation, and classification, thereby ensuring the reliability and consistency of the tongue feature analysis results.

[0054] Tongue images were collected from individuals with different health conditions. Their age composition, gender ratio, and health status are shown in Table 1 below.

[0055] Table 1. Statistics on the health status of data collection personnel by age group

[0056] Image acquisition not only requires the support of high-resolution camera equipment, but also needs to consider the following key factors.

[0057] 1) The collected data must cover the tongue characteristics of different individuals and the tongue's appearance under different health conditions. Diverse sample data ensures that the model can handle various complex tongue image scenarios during training and inference, thereby improving the model's generalization ability.

[0058] 2) To ensure data consistency, images should be acquired in a standardized environment, including multiple constant lighting intensities and fixed shooting distances. This minimizes the impact of external environmental factors on image color and detail.

[0059] 3) During image acquisition, a standard color chart is used to calibrate the image at each shooting time to correct for color deviations in the camera equipment. This step ensures that the color information of the tongue image remains consistent across different devices and shooting environments.

[0060] The specific sample image acquisition strategy is shown in Table 2.

[0061] Table 2 Sample Image Acquisition Strategy

[0062] Furthermore, after data cleaning and screening, images with poor quality due to factors such as blurry imaging, overexposure, or underexposure were removed.

[0063] S1013. Crop each of the color-calibrated image data to obtain each tongue region image data.

[0064] The color-calibrated image data is cropped. A rectangle approximately twice the length and width of the tongue region is used to completely enclose it. This cropping method effectively preserves the details around the tongue, providing high-quality training data for the subsequent tongue segmentation model. The cropped image will serve as input data for the tongue segmentation model, improving its accuracy and robustness in real-world applications.

[0065] S1014. Each color-calibrated image data is labeled to mark the tongue bounding box of each image, thereby obtaining a tongue detection dataset. The tongue detection dataset is then divided into a training set, a validation set, and a test set for the tongue detection model based on a preset first ratio.

[0066] After acquiring the image data, it is labeled using Labelme. Specifically, the color-calibrated image data is imported into Labelme, and the rectangle tool is used to label the tongue bounding box for each image, resulting in the tongue detection dataset. A stratified sampling strategy is employed to ensure a uniform distribution of samples from different categories across the datasets, thus avoiding class imbalance during model training. The tongue detection dataset is divided into training, validation, and test sets in an 8:1:1 ratio. Using stratified sampling, samples are randomly assigned to the training, validation, and test sets while maintaining a balanced distribution across different health states, age groups, and genders. This ensures that the model fully learns the features of different categories during training, improving its generalization ability in practical applications.

[0067] S1015. Each image of the tongue region is labeled to mark the tongue region segmentation box of each image, thereby obtaining a tongue segmentation dataset. The tongue segmentation dataset is then divided into a training set, a validation set, and a test set for the tongue segmentation model based on a preset second ratio.

[0068] After acquiring the image data, it was labeled using Labelme. Specifically, the bounding boxes of the tongue in each image were labeled, resulting in a tongue detection dataset. This dataset was then imported into Labelme, and the polygon tool was used to label the cropped tongue region images, creating segmentation bounding boxes for each image, thus obtaining a tongue segmentation dataset. A stratified sampling strategy was employed, dividing the tongue segmentation dataset into training, validation, and test sets in a 70:15:15 ratio. This ensures that samples of different classes are evenly distributed across all datasets, avoiding class imbalance that could negatively impact model performance. Compared to the tongue detection dataset, the tongue segmentation dataset has a higher validation and test ratio to better evaluate the model's segmentation performance.

[0069] It should be noted that the preset first ratio and the preset second ratio can also be other ratio values. The embodiments disclosed herein are only for illustrative purposes and are not intended to limit the specific implementation.

[0070] S102. Based on the image dataset, train the tongue detection model and the tongue segmentation model respectively to obtain the trained tongue detection model and the trained tongue segmentation model.

[0071] In this step, as shown in Figure 2, after constructing the image dataset, the model is trained using the image dataset to obtain a trained tongue detection model and a trained tongue segmentation model. Optionally, the image dataset includes a training set, a test set, and a validation set. During training, the annotation information in the training set guides the model to learn how to accurately complete the tongue detection and segmentation tasks. The data in the validation set is used to adjust the model's hyperparameters to ensure the model's performance outside the training set and to avoid overfitting. After the model training is complete, performance is evaluated based on the test set to confirm the model's actual detection capability and robustness.

[0072] S103. The trained tongue detection model and the trained tongue segmentation model are fused to obtain a target segmentation model. The image dataset is segmented using the target segmentation model to obtain a tongue segmentation image dataset.

[0073] In this step, the electronic device fuses the trained tongue detection model and the trained tongue segmentation model to obtain a target segmentation model. The image dataset is then input into the target segmentation model, ultimately generating a high-quality tongue segmentation image dataset. In some embodiments, the trained tongue detection model and the trained tongue segmentation model can be fused to construct a unified interactive segmentation module. The core objective of the interactive segmentation module is to quickly locate the approximate area of ​​the tongue using the tongue detection model, and then perform fine segmentation of the detected tongue area using the tongue segmentation model, thereby improving segmentation accuracy and efficiency. As shown in Figure 9, the specific process of the interactive segmentation module is as follows: 1) First, the trained tongue detection model is used to identify the approximate location of the tongue in the input image and output a bounding box containing the tongue region; 2) The bounding box of the tongue region output by the tongue detection model is extended, and the length and width of the tongue region are enlarged to twice the original bounding box (if the bounding box exceeds the image boundary after enlargement, it is taken as the image boundary value), forming a cropping box containing the tongue region; 3) The original image is cropped according to the cropping box, and the region containing the tongue is extracted separately; 4) The cropped tongue region image is input into the tongue segmentation model, and then pixel-level fine segmentation is performed to output a high-precision tongue segmentation image containing only the tongue.

[0074] S104. Label the tongue segmentation image dataset to obtain a tongue feature classification dataset. Train the tongue feature classification model based on the tongue feature classification dataset to obtain a trained tongue feature classification model.

[0075] In this step, after generating the tongue segmentation image dataset, classification and labeling are performed to obtain the tongue feature classification dataset. Further, the tongue feature classification model is trained based on the tongue feature classification dataset to obtain the trained tongue feature classification model.

[0076] In some embodiments, the tongue feature classification dataset can be divided into a training set, a validation set, and a test set for the tongue feature classification model. The specific partitioning strategy is as follows: 1) The dataset is divided into training, validation, and test sets in a 75:15:10 ratio, meaning 75% of the data is used for model training, 15% for model validation, and 10% for model testing; 2) To ensure a relatively balanced distribution of tongue feature categories in each dataset, a stratified sampling strategy is used during partitioning to ensure that the sample proportions of each tongue feature category in each dataset are as consistent as possible, avoiding the impact of class imbalance on model performance.

[0077] S105. Obtain the target image data to be detected, and input the target image data into the target segmentation model to obtain the tongue segmentation image data of the target image data to be detected.

[0078] In this step, the electronic device acquires the target image data to be detected, which includes a tongue image. The target image data is then input into the target segmentation model to obtain tongue segmentation image data of the target image data to be detected.

[0079] S106. Perform feature analysis on the segmented tongue image data using the trained tongue feature classification model to obtain the tongue feature analysis results.

[0080] In this step, after obtaining the segmented tongue image data of the target image data to be detected, the segmented tongue image data is input into the trained tongue feature classification model to obtain the tongue feature analysis results. As shown in Figure 8, the tongue feature analysis results include, but are not limited to, tongue color prediction results, coating color prediction results, tongue texture prediction results, coating texture prediction results, and overall prediction results.

[0081] This disclosure embodiment constructs an image dataset, which includes a large number of images containing tongue images. Based on this image dataset, a tongue detection model and a tongue segmentation model are trained separately to obtain trained tongue detection and segmentation models. These trained models are then fused to obtain a target segmentation model. The target segmentation model is used to segment the image dataset, resulting in a tongue segmentation image dataset. This dataset is then labeled to obtain a tongue feature classification dataset. Based on this dataset, a tongue feature classification model is trained to obtain a trained tongue feature classification model. Target image data to be detected is acquired and input into the target segmentation model to obtain tongue segmentation image data of the target image data. The trained tongue feature classification model is then used to perform feature analysis on the tongue segmentation image data to obtain tongue feature analysis results. This disclosure embodiment can achieve feature sharing and information complementarity, thereby improving the accuracy and robustness of feature classification. By training directly on complete tongue segmentation images, the potential information loss during the separation of tongue coating and tongue body is effectively avoided. The model can more comprehensively learn the complex relationships and detailed information between features such as tongue color, tongue body, and coating color, ensuring more accurate and objective classification results. This reduces reliance on manual annotation and errors in tongue coating separation, not only improving the training efficiency of the model but also significantly enhancing the stability and reliability of classification.

[0082] Figure 3 is a flowchart of a tongue feature analysis method provided in another embodiment of the present disclosure. As shown in Figure 3, the method includes the following steps.

[0083] S201. Construct an image dataset, which includes a large amount of image data containing images of tongues.

[0084] Specifically, the implementation process and principle of S201 and S101 are the same, and will not be repeated here.

[0085] S202. Construct a tongue detection model and a tongue segmentation model.

[0086] To accurately locate the tongue region in complex tongue images and minimize computational resource consumption while maintaining model performance, this disclosure presents a lightweight model architecture with good detection performance. As shown in Figure 5, the detection model uses a MobileNetV3 convolutional neural network structure as the backbone, employing depthwise separable convolutions to reduce the number of parameters and computational cost while maintaining high detection accuracy. Furthermore, spatial attention and channel attention modules are introduced before the final detection head to enhance the model's ability to represent tongue region features.

[0087] In Figure 5, Conv_BN_HSwish is the combined operation module, mainly containing convolutional layers, batch normalization layers, and the Hard-Swish activation function. The CBAM module is a combination of attention mechanisms, combining spatial and channel attention modules to enhance the model's ability to capture spatial and channel information, allowing the model to focus more on important feature regions and channels, thus improving overall model performance. C2f is an optimized convolutional module used to better capture multi-scale feature information of feature maps, while improving model performance while keeping computation and parameter counts relatively low. Here, C stands for "Conv," meaning convolution operation; 2 indicates that two different feature extraction paths are used in this module; and f stands for "Focus," indicating that this module uses a new structure to better focus on and capture features at different scales. Upsample is used to increase the spatial resolution (height and width) of the input feature map to a higher resolution. Concat is used to concatenate multiple tensors along a specific dimension (axis), merging them into a new tensor.

[0088] To accurately segment the tongue region, as shown in Figure 6, the segmentation model is based on a multi-layer convolutional neural network. The encoder incorporates a CBAM attention mechanism module and a residual block, enabling the model to focus on more important tongue feature regions and improving gradient flow and feature learning capabilities. At the decoder, multi-scale feature fusion is introduced to better capture detailed information about the tongue edges.

[0089] In Figure 6, the residual convolutional module (R-ConvBlock) consists of two convolutional layers (Conv), a batch normalization layer (BatchNorm, BN), a ReLU activation function, and a residual connection. Each R-ConvBlock module is responsible for extracting image features, and the residual module effectively alleviates the gradient vanishing problem in deep networks. Features are extracted using the residual convolutional module, and then further optimized by the attention module. Layers are connected via downsampling to reduce the spatial resolution of the feature maps and increase the receptive field of the features. Downsampling reduces the spatial resolution or sampling rate of the input feature map or signal, extracting higher-level features by reducing the spatial size of the data, while simultaneously reducing computation and memory usage. After each downsampling, the feature map is upsampled and concatenated with the higher-level feature maps for multi-scale fusion. Finally, the output tongue segmentation mask is generated through convolutional layers. This network structure is well-suited for fine-grained tongue segmentation tasks and can maintain high tongue segmentation accuracy even in complex hospital scenarios.

[0090] S203. Train the tongue detection model based on the tongue detection dataset, update the model parameters of the tongue detection model, and if the tongue detection result of each image output by the updated tongue detection model has converged to the labeled tongue bounding box of each image, then the trained tongue detection model is obtained.

[0091] S204. Train the tongue segmentation model based on the tongue segmentation dataset, update the model parameters of the tongue segmentation model, and if the tongue segmentation result of each image output by the updated tongue segmentation model has converged to the labeled tongue region segmentation box of each image, then the trained tongue segmentation model is obtained.

[0092] In steps S203 and S204, the tongue detection model and the tongue segmentation model are jointly trained based on the training and validation sets in their respective datasets. The ultimate goal is to train high-precision tongue detection and segmentation models. First, data augmentation techniques are applied to the training sets in the detection and segmentation datasets to increase the model's generalization ability. These techniques mainly include the following operations.

[0093] 1) Rotation: Randomly rotate the image with an angle range of [-30°, 30°].

[0094] 2) Translation: Randomly translates the width and height, ranging from -10% to 10%.

[0095] 3) Flip: Randomly flips the image horizontally or vertically.

[0096] 4) Brightness / Contrast Adjustment: Randomly adjusts brightness and contrast, ranging from [0.8, 1.2].

[0097] 5) Noise: Add random noise to the image.

[0098] For the training set in the detection and segmentation dataset, one or more of the above operations will be randomly selected to perform data augmentation operations on the original tongue image, thus expanding the training set data to twice the size of the original training set data.

[0099] The training process for the tongue detection and segmentation models employs a series of carefully designed hyperparameters to achieve optimal training results and final performance. Appropriate hyperparameters can significantly improve the model's detection and segmentation performance on tongue images, ensuring rapid convergence and good generalization ability during training. Table 3 details the hyperparameters used in the training of the tongue detection and segmentation models, including key factors such as learning rate, batch size, optimizer type, loss function, and data augmentation strategies. Specific hyperparameters are shown in Table 3 below.

[0100] Table 3. Training hyperparameters of the tongue detection model and tongue segmentation model

[0101] Table 3 above shows that cosine annealing is a learning rate scheduling strategy that controls the model's training process by adjusting the learning rate curve. Unlike traditional learning rate decay strategies, cosine annealing's learning rate changes smoothly along a cosine curve, allowing for fine-tuning of the model with a lower learning rate in the later stages of training. This effectively avoids overfitting during training and promotes rapid model convergence. The specific formula is as follows.

[0102] Where, η t η is the current learning rate. min This is the current minimum learning rate; η max It is the maximum learning rate (initial learning rate); T cur This is the current training round; T maxThis is the largest round of the annealing cycle.

[0103] Step decay is another learning rate scheduling strategy used to gradually reduce the learning rate during training. It reduces the learning rate proportionally when a predetermined number of training iterations (epochs) is reached. The learning rate adjustment process of the step decay strategy can be expressed by the following formula.

[0104] Where η is the current learning rate; η0 is the initial learning rate; γ is the decay coefficient (usually less than 1, for example, 0.1); s is the decay step size, indicating how many epochs between each decay; and t is the current training epoch.

[0105] In some embodiments, after training the tongue detection model and the tongue segmentation model based on the image dataset to obtain the trained tongue detection model and the trained tongue segmentation model, the method further includes steps A, B, and C.

[0106] Step A: Test the trained tongue detection model and the trained tongue segmentation model using their respective test sets to obtain the tongue detection output and tongue segmentation output results of the test sets.

[0107] In this step, after training the model, to comprehensively evaluate the performance of the tongue detection and segmentation models, the trained tongue detection and segmentation models are tested using test sets from the tongue detection and segmentation datasets. The tongue detection and segmentation outputs on these test sets are then obtained. This process aims to verify the model's performance in real-world application environments, ensuring that the model can run stably and accurately in diverse scenarios.

[0108] Step B: Evaluate the tongue detection output and tongue segmentation output of the test set using preset test indicators to obtain the test results of the tongue detection model and the tongue segmentation model.

[0109] Optional evaluation metrics for tongue detection models include: Mean Average Precision (mAP), Recall, Precision, and F1 Score. Evaluation metrics for tongue segmentation models include: Dice Coefficient, Intersection over Union (IoU), and Pixel Accuracy.

[0110] Optionally, the Dice Coefficient is an evaluation metric used in image segmentation tasks to measure the similarity between the model's segmentation result and the ground truth segmented region. The Dice Coefficient ranges from 0 to 1, where 1 indicates a perfect match between the segmentation result and the ground truth region, and 0 indicates no overlap. The formula for calculating the Dice Coefficient is as follows.

[0111] Where A is the segmentation region predicted by the model; B is the ground truth segmentation region; A is the total number of pixels in the model prediction region; B is the total number of pixels in the ground truth segmentation region; A∩B is the number of pixels at the intersection of the model prediction region and the ground truth segmentation region; TP is the number of pixels that are both true positives and predicted as positives by the model; FP is the number of pixels that are true negatives but predicted as positives by the model; and FN is the number of pixels that are true positives but predicted as negatives by the model.

[0112] Pixel accuracy (PA) is a metric used to evaluate the performance of a model in image segmentation tasks. It measures the accuracy with which the model classifies all pixels. It assesses the overall segmentation accuracy by calculating the ratio of correctly classified pixels to the total number of pixels. The formula for calculating pixel accuracy is as follows.

[0113] Wherein, TP (True Positive) is the number of pixels correctly classified as the target category; TN (True Negative) is the number of pixels correctly classified as the background; FP (False Positive) is the number of pixels incorrectly classified as the target category; and FN (False Negative) is the number of pixels incorrectly classified as the background.

[0114] Step C: When the test results of the tongue detection model and the tongue segmentation model both meet the target index values, the training of the tongue detection model and the tongue segmentation model is considered complete.

[0115] In this step, if the test results of the tongue detection model and the tongue segmentation model both meet the target index values, then the tongue detection model and the tongue segmentation model are considered to have completed training and can be used.

[0116] S205. The trained tongue detection model and the trained tongue segmentation model are fused to obtain a target segmentation model. The image dataset is segmented using the target segmentation model to obtain a tongue segmentation image dataset.

[0117] Specifically, the implementation process and principle of S205 and S103 are the same, and will not be repeated here.

[0118] S206. Remove image data from the tongue segmentation image dataset that does not meet the preset image quality requirements to obtain the tongue segmentation image dataset after removal.

[0119] In this step, the images in the tongue segmentation image dataset are reviewed. Samples with serious problems identified during the evaluation process (such as blurred segmentation boundaries or incomplete tongue region cropping) are removed to ensure that all samples entering the tongue segmentation image dataset are high-quality samples. Finally, tongue segmentation images that meet the image quality requirements are included in the dataset.

[0120] S207. Based on the preset tongue feature classification standard, the tongue segmentation image dataset after removal is labeled with features, and the tongue features of each image are labeled to obtain the tongue feature classification dataset. The tongue features include tongue color, tongue texture, coating color, coating texture and overall features.

[0121] After data cleaning, based on preset feature classification standards, the segmented tongue images that meet the image quality requirements are subjected to fine feature classification and annotation to create a tongue feature classification dataset. The tongue features include tongue color, tongue body, coating color, coating texture, and overall features. Specifically, tongue color features are classified as: pale red, pale white, red, crimson, bluish-purple, and purple; tongue body features are classified as: swollen tongue, thin tongue, cracked tongue, teeth-marked tongue, prickly tongue, and awned tongue; coating color features are classified as: pale white, dark white, pale yellow, medium yellow, dark yellow, gray, yellowish-black, scorched black, pale green, and dark green; coating texture features are classified as: thin coating, thick coating, moist coating, dry coating, greasy coating, rough coating, rotten coating, and peeling coating; overall features include all categories of tongue color, tongue body, coating color, and coating texture.

[0122] S208. Train the tongue feature classification model based on the tongue feature classification dataset to obtain a trained tongue feature classification model.

[0123] S209. Obtain the target image data to be detected, and input the target image data into the target segmentation model to obtain the tongue segmentation image data of the target image data to be detected.

[0124] Specifically, the implementation process and principle of S209 and S105 are the same, and will not be repeated here.

[0125] S210. The tongue segmentation image data is analyzed using a trained tongue feature classification model to obtain tongue feature analysis results.

[0126] Specifically, the implementation process and principle of S210 and S106 are the same, and will not be repeated here.

[0127] This embodiment of the disclosure constructs an image dataset, which includes a large number of images containing tongue images, constructs a tongue detection model and a tongue segmentation model, trains the tongue detection model based on the tongue detection dataset, updates the model parameters of the tongue detection model, and obtains a trained tongue detection model if the tongue detection result of each image output by the updated tongue detection model converges to the labeled tongue bounding box of each image. Similarly, it trains a tongue segmentation model based on the tongue segmentation dataset, updates the model parameters of the tongue segmentation model, and obtains a trained tongue segmentation model if the tongue segmentation result of each image output by the updated tongue segmentation model converges to the labeled tongue region segmentation box of each image. Furthermore, the trained tongue detection model and the trained tongue segmentation model are fused to obtain a target segmentation model. This target segmentation model is then used to segment the image dataset, resulting in a tongue segmentation image dataset. Images in this dataset that do not meet preset image quality requirements are removed, resulting in a discarded tongue segmentation image dataset. Based on preset tongue feature classification standards, feature annotation is performed on the discarded tongue segmentation image dataset, annotating the tongue features of each image to obtain a tongue feature classification dataset. These tongue features include tongue color, tongue texture, tongue coating color, tongue coating texture, and overall features. Next, the tongue feature classification model is trained based on this dataset, resulting in a trained tongue feature classification model. The target image data to be detected is then acquired and input into the target segmentation model to obtain tongue segmentation image data of the target image data. Finally, the trained tongue feature classification model is used to perform feature analysis on the tongue segmentation image data to obtain tongue feature analysis results. This method can improve the accuracy and robustness of feature classification, reduce reliance on manual annotation and errors in tongue coating separation, not only improving the training efficiency of the model, but also significantly improving the stability and reliability of classification.

[0128] Figure 4 is a flowchart of a tongue feature analysis method provided in another embodiment of this disclosure. As shown in Figure 4, the method includes the following steps.

[0129] S301. Construct an image dataset, which includes a large amount of image data containing images of tongues.

[0130] Specifically, the implementation process and principle of S301 and S101 are the same, and will not be repeated here.

[0131] S302. Based on the image dataset, train the tongue detection model and the tongue segmentation model respectively to obtain the trained tongue detection model and the trained tongue segmentation model.

[0132] Specifically, the implementation process and principle of S302 and S102 are the same, and will not be repeated here.

[0133] S303. The trained tongue detection model and the trained tongue segmentation model are fused together to obtain a target segmentation model. The image dataset is then segmented using the target segmentation model to obtain a tongue segmentation image dataset.

[0134] Specifically, the implementation process and principle of S303 and S103 are the same, and will not be repeated here.

[0135] S304. Label the tongue segmentation image dataset to obtain a tongue feature classification dataset. Train the tongue feature classification model based on the tongue feature classification dataset to obtain a trained tongue feature classification model.

[0136] Specifically, the implementation process and principle of S304 and S104 are the same, and will not be repeated here.

[0137] In some embodiments, in S304, the tongue feature classification model is trained based on the tongue feature classification dataset to obtain a trained tongue feature classification model, which may include, but is not limited to, S3041 and S3042.

[0138] S3041. Construct a tongue feature classification model.

[0139] As shown in Figure 7, the tongue feature classification model adopts a multi-task learning (MTL) architecture, which integrates different tongue feature classification tasks into a unified fusion model. The overall model architecture can be divided into the following main parts.

[0140] 1) Low-level feature extraction network

[0141] The low-level feature extraction network, based on EfficientNet, is used to extract low-level and high-level features from tongue images. EfficientNet is a convolutional neural network architecture that improves model performance while minimizing computational cost through automated model expansion. The low-level feature extraction network employs a composite expansion strategy, simultaneously expanding the network's depth, width, and resolution. This achieves a good balance between model size, computational cost, and accuracy, enabling it to capture information such as edges, texture, and color from tongue images, providing rich tongue feature representations for subsequent tasks.

[0142] 2) Task Branch Module

[0143] The task branch module is used to classify and predict tongue color, tongue texture, tongue coating color, tongue coating, and overall features. The task branch module is a network structure consisting of multiple layers of feature extraction, skip connections, fully connected layers, and activation functions. The output layer uses the Softmax activation function to output the predicted probabilities for different categories.

[0144] 3) Feature sharing

[0145] Following the underlying feature extraction network, a shared feature layer is used to achieve information sharing and feature complementarity between different tasks. The shared feature layer is designed as a multi-layer convolutional neural network structure to integrate feature information from multiple tasks at different levels. Its main function is to fuse and re-encode the low-level and high-level features output by the underlying feature extraction network, providing rich semantic features for each task branch.

[0146] 4) Loss Function and Optimization Strategy

[0147] Each task branch has an independent loss function. The tongue color, tongue texture, tongue coating color, tongue texture, and overall feature branches all use the cross-entropy loss function to optimize the classification output. To better balance the learning process of each task, the total loss function is a weighted sum of the losses from each task.

[0148] 5) Model Output

[0149] The model output adopts multi-task classification, and simultaneously outputs classification predictions for tongue color, tongue texture, tongue coating color, tongue coating, and overall feature results. Each classification task outputs confidence scores for different categories, and for each classification task, a comprehensive evaluation is performed using metrics such as accuracy, precision, and recall to fully measure the model's performance in multi-task classification.

[0150] In Figure 7, Dropout is a regularization technique to prevent overfitting in deep neural networks. It reduces the network's over-reliance on certain features by randomly dropping (i.e., disconnecting) a portion of neurons and their connections during training, thereby improving the model's generalization ability. Global Average Pooling (GAP) is a pooling operation used to reduce the spatial dimensions (height and width) of each feature map to 1, thus outputting the average of the entire feature map. A Fully Connected Layer (FC) is a basic layer type in neural networks where each input node is connected to every output node. Therefore, each output node is a weighted sum of all input nodes, used to integrate extracted features for final classification. Softmax is an activation function used as the output layer for multi-class classification problems.

[0151] In Figure 7, Conv3_BN_Swish is a module for feature extraction and nonlinear transformation. It consists of a 3x3 convolutional kernel, a batch normalization layer, and a Swish activation function concatenated together. The 3x3 convolutional kernel balances computational complexity and feature extraction capability, capturing local spatial information. Conv1_BN_Swish mainly consists of a 1x1 convolutional kernel, followed by a batch normalization layer, and finally the Swish activation function. The 1x1 convolutional kernel changes the number of channels in the feature map without changing the spatial size of the input feature map, thus compressing the channel dimension. This allows information from multiple channels to be fused together, increasing the model's nonlinear expressive power.

[0152] In Figure 7, MBConv1 is a convolutional module, an extended inverted bottleneck structure, mainly consisting of a depthwise convolutional layer, two batch normalized layers, a pointwise convolutional layer, a residual connection, and a ReLU6 activation function. In MBConv1, the expansion factor is 1, therefore there are no expansion layers, and the number of input and output channels remains the same. MBConv1 with its residual structure can provide high performance while maintaining model computational efficiency, making it particularly suitable for deep learning applications in mobile and embedded devices.

[0153] In Figure 7, MBConv6_SE is a convolutional module based on MBConv1, with added extension layers and SiLU replacing ReLU. It increases the number of input channels by a factor of 6, aiming to enable subsequent deep convolutional layers to learn more features in a higher-dimensional space. SE represents feature reweighting. It first performs global average pooling on the feature map in the spatial dimension, then calculates channel attention through two fully connected layers, and finally scales the original features. Through the SE module, the model can adaptively focus on and select specific channel features, improving the model's feature selectivity and classification ability.

[0154] S3042. Train the tongue feature classification model based on the tongue feature classification dataset, update the model parameters of the tongue feature classification model, and if the tongue feature analysis results of each image output by the updated tongue feature classification model have converged to the labeled tongue features of each image, then the trained tongue feature classification model is obtained.

[0155] The tongue feature classification model is trained using a pre-defined tongue feature classification dataset. To ensure that the model can comprehensively and accurately capture the multi-dimensional features of the tongue, this step employs a multi-task learning approach, aiming to simultaneously optimize and train multiple tasks, including tongue color, tongue texture, tongue coating color and texture, and overall features.

[0156] First, the cross-entropy loss function is applied to optimize each classification task. The cross-entropy loss function exhibits good convergence in multi-class classification problems and effectively measures the deviation between the model's predictions and the true values, thus guiding the model to continuously improve its prediction accuracy during training. To achieve optimal performance across multiple tasks, a total loss function is used as a weighted sum of the losses from each task. Therefore, the total loss function L... total It can be represented as.

[0157] Among them, L i It is the loss function for the i-th task (such as the cross-entropy loss function); α i It is the weight of the i-th task. ， This indicates the importance of the task in the total loss; N is the total number of tasks (in this paper, N=5, namely, five tasks: tongue color, tongue body, tongue coating color, tongue coating, and overall characteristics).

[0158] The loss weights for each task are adjusted based on their importance in the overall model, ensuring that each task receives appropriate optimization during training. Ultimately, this results in a trained tongue feature classification model.

[0159] Table 4 lists the hyperparameters used in the training process of the tongue feature classification model, including key factors such as learning rate, batch size, optimizer type, and loss function.

[0160] Table 4. Training hyperparameters of the tongue feature classification model

[0161] In Table 4 above, Xavier weight initialization is a method used for weight initialization in deep neural networks. Its aim is to ensure that the variances of the inputs and outputs of each layer remain consistent during initialization, thereby avoiding gradient vanishing or exploding problems and promoting rapid convergence of the neural network. Assuming the input of a layer is X and the output is Y, Xavier aims to initialize the weight matrix W to guarantee the following variance relationship: Var(Y) = Var(W·X) = Var(W)·Var(X)

[0162] To make the variance of the output Y as close as possible to the variance of the input X, Xavier initialization samples the elements of the weight matrix W from the following uniform distribution.

[0163] Where, n in This is the number of input nodes (neurons) in the current layer; n out It represents the number of output nodes (neurons) in the current layer.

[0164] In some embodiments, after S304, the method further includes steps a, b, and c.

[0165] Step a: Test the trained tongue feature classification model using the test set in the tongue feature classification dataset to obtain the tongue feature classification output results of the test set.

[0166] Step b: Evaluate the tongue feature classification output results of the test set using preset test indicators to obtain the test results of the tongue feature classification model.

[0167] Step c: When the test results of the tongue feature classification model meet the target index value, the training of the tongue feature classification model is considered complete.

[0168] In this embodiment, a comprehensive performance test will be conducted on the trained tongue feature classification model. This step aims to verify the model's actual performance in different tongue feature classification tasks and evaluate its reliability and generalization ability in real-world application scenarios. The test dataset consists of tongue segmentation image data that was not involved in model training. Each image in the test set is input into the trained tongue feature classification model, which will predict five classification tasks: tongue color, tongue texture, tongue coating color, tongue coating texture, and overall features, and output the prediction result and confidence score for each task. To comprehensively measure the model's overall performance in multi-task classification, accuracy, precision, and recall will be used to evaluate the real-world performance and obtain the test results of the tongue feature classification model. When the test results of the tongue feature classification model meet the target metric values, the training of the tongue feature classification model is considered complete.

[0169] S305. Integrate the trained tongue feature classification model, color space conversion algorithm, channel analysis algorithm, and verification algorithm to obtain the statistical analysis module.

[0170] As shown in Figure 10, the trained tongue feature classification model is combined with color space conversion algorithms, channel analysis algorithms, and validation algorithms to construct a comprehensive statistical analysis module. This module can receive tongue segmentation images, perform multi-dimensional tongue feature analysis and statistical validation, and output detailed tongue feature classification prediction results and various channel analysis data.

[0171] S306. Input the tongue segmentation image data into the statistical analysis module, and perform feature analysis on the tongue segmentation image data using the trained tongue feature classification model to obtain the credibility of each feature category.

[0172] The preprocessed segmented tongue image is input into the tongue feature classification model, which performs multi-task feature classification inference on the input image. The model outputs predicted information for tongue color, tongue texture, tongue coating color, tongue coating, and overall feature classification, and provides a confidence score for each prediction, thus providing a reliability reference for the prediction results of each feature category.

[0173] S307. Use a color space conversion algorithm to convert the tongue segmentation image data into color-opposite space channel image data.

[0174] To further analyze the color features of the image, as shown in Figure 8, a color space conversion algorithm was used to convert the RGB channel image, i.e., the original tongue segmentation image data, into the Lab color space. Separating the luminance information (L channel) and color information (a and b channels) in the tongue image is crucial for subsequent analysis of tongue color and tongue coating.

[0175] S308. Input the tongue segmentation image data and the color contrast space channel image data into the channel analysis algorithm for channel analysis to obtain the red, green and blue channel analysis results and the color contrast space channel analysis results.

[0176] As shown in Figure 8, the RGB and converted Lab channel images are input into the channel analysis algorithm, respectively, and detailed statistical analysis is performed on the RGB and Lab channels. The analysis includes the mean, standard deviation, and distance of the channel spatial mean from the eight vertices for each channel. These statistics help quantify the color distribution characteristics of the tongue image, thus providing richer reference data for tongue feature analysis.

[0177] S309. The tongue segmentation image data and color-opposite space channel image data are subjected to distribution testing using a testing algorithm to obtain the test values ​​of the red, green and blue channels and the test values ​​of the color-opposite space channels.

[0178] The RGB and Lab channel images are input into the KS test algorithm to test the spatial distribution of the image channels against the standard distribution, and the KS test p-value for each channel is calculated. This p-value can be used to assess the significant differences in tongue characteristics under different health conditions, thus providing a statistical basis for intelligent tongue diagnosis results.

[0179] Ultimately, the statistical analysis module will output multi-dimensional tongue feature analysis data, including prediction results and confidence scores for tongue color, tongue body, tongue coating color and texture, and overall characteristics, as well as detailed information such as various statistics for RGB and Lab channels and KS test p-values. This analytical data will be presented in a structured format for use by medical analysts and researchers to better understand and analyze the relationships between different tongue features and their indicative role in health status.

[0180] The Kolmogorov-Smirnov (KS) test is a nonparametric statistical method used to compare the distributions of two samples, or the difference between a sample and a reference distribution. Given two samples X1, X2, ..., X... n and Y1,Y2,...,Y n Then their cumulative distribution functions (CDF) are respectively F n (x) and G m The formula for calculating F(x) is as follows: F(x) = P(X≤x).

[0181] In the above formula, X represents a given random variable.

[0182] As shown in the CDF formula above, CDFF(x) represents the probability that the random variable X takes a value less than or equal to x. As x increases, F(x) is monotonically increasing, and the range of CDF is (0, 1). When x approaches negative infinity, F(x) approaches 0, and when x approaches positive infinity, F(x) approaches 1. For two samples with known cumulative distribution functions, the formula for calculating the p-value output by the KS test algorithm is as follows.

[0183] Among them, F n (x) represents the expression for samples X1, X2, ..., X... n The cumulative distribution function G; m (x) represents the expression for samples Y1, Y2, ..., Y... n The cumulative distribution function.

[0184] In some embodiments, various tongue feature analyses and channel statistical test results are extracted from the inference results of the statistical analysis module, and a structured statistical analysis report is output. The specific output content is as follows.

[0185] 1) Multi-task feature classification prediction results. This includes predicted values ​​and confidence scores for multiple tasks such as tongue color, tongue texture, tongue coating color, tongue coating texture, and overall features, along with corresponding charts.

[0186] 2) Color channel statistics. These include the mean and standard deviation of the RGB and Lab channels, as well as the distance of the channel space mean from the eight vertices.

[0187] 3) KS test statistic. The KS test p-value is used to test the distribution of the tongue image compared with the standard distribution to determine the significant difference between the distribution of each color channel and the standard distribution, and then the corresponding KS test statistic chart is output.

[0188] This embodiment of the disclosure constructs an image dataset, which includes a large number of images containing tongue images. Based on this image dataset, a tongue detection model and a tongue segmentation model are trained separately to obtain trained tongue detection and segmentation models. Then, the trained tongue detection and segmentation models are fused to obtain a target segmentation model. This target segmentation model is used to segment the image dataset, resulting in a tongue segmentation image dataset. This tongue segmentation image dataset is then labeled to obtain a tongue feature classification dataset. Based on this tongue feature classification dataset, a tongue feature classification model is trained to obtain a trained tongue feature classification model. Next, the trained tongue feature classification model, a color space conversion algorithm, a channel analysis algorithm, and a verification algorithm are integrated to obtain a statistical analysis module. The tongue segmentation image data is input into the statistical analysis module, and the trained tongue feature classification model is used to perform feature analysis on the tongue segmentation image data to obtain the confidence level of each feature category. Furthermore, a color space conversion algorithm is used to convert the segmented tongue image data into color-opposite space channel image data. The segmented tongue image data and the color-opposite space channel image data are then input into a channel analysis algorithm for channel analysis, yielding red-green-blue channel analysis results and color-opposite space channel analysis results. Subsequently, a distribution test algorithm is used to perform distribution tests on the segmented tongue image data and the color-opposite space channel image data, obtaining test values ​​for the red-green-blue channels and the color-opposite space channels. This method enables quantitative analysis of key features such as tongue color and tongue coating color. Through channel analysis, the model can deeply extract and quantify color features in tongue images, and statistical methods such as tests are used to evaluate the significance of differences between samples. The analysis results provide doctors with detailed quantitative data on tongue color, tongue coating color, and other features, providing strong data support for clinical applications.

[0189] Figure 11 is a schematic diagram of the structure of the tongue feature analysis device provided in this embodiment. The tongue feature analysis device can be an electronic device as described in the above embodiments, or it can be a component or assembly within that electronic device. The tongue feature analysis device provided in this embodiment can execute the processing flow provided in the tongue feature analysis method embodiment. As shown in Figure 11, the tongue feature analysis device 40 includes: a construction unit 41, a first training unit 42, a first obtaining unit 43, a second training unit 44, a second obtaining unit 45, and a feature analysis unit 46; wherein, the construction unit 41 is used to construct an image dataset, which includes a large amount of image data containing tongue images; the first training unit 42 is used to train the tongue detection model and the tongue segmentation model based on the image dataset, respectively, to obtain the trained tongue detection model and the trained tongue segmentation model; the first obtaining unit 43 is used to process the trained tongue detection model and the trained tongue segmentation model... The system performs fusion to obtain a target segmentation model, and then segments the image dataset using the target segmentation model to obtain a tongue segmentation image dataset. A second training unit 44 annotates the tongue segmentation image dataset to obtain a tongue feature classification dataset, and trains a tongue feature classification model based on the tongue feature classification dataset to obtain a trained tongue feature classification model. A second obtaining unit 45 acquires the target image data to be detected, inputs the target image data into the target segmentation model, and obtains the tongue segmentation image data of the target image data to be detected. A feature analysis unit 46 performs feature analysis on the tongue segmentation image data using the trained tongue feature classification model to obtain tongue feature analysis results.

[0190] Optionally, when constructing the image dataset, the construction unit 41 is specifically used for: collecting a large amount of image data containing tongue images; performing color calibration processing on each image data to obtain each color-calibrated image data; cropping each color-calibrated image data to obtain each tongue region image data; labeling each color-calibrated image data to mark the tongue bounding box of each image to obtain a tongue detection dataset, and dividing the tongue detection dataset into a training set, a validation set, and a test set of the tongue detection model based on a preset first ratio; labeling each tongue region image data to mark the tongue region segmentation box of each image to obtain a tongue segmentation dataset, and dividing the tongue segmentation dataset into a training set, a validation set, and a test set of the tongue segmentation model based on a preset second ratio.

[0191] Optionally, the image dataset includes a tongue detection dataset and a tongue segmentation dataset.

[0192] When the first training unit 42 trains the tongue detection model and the tongue segmentation model based on the image dataset to obtain the trained tongue detection model and the trained tongue segmentation model, it is specifically used for: constructing the tongue detection model and the tongue segmentation model; training the tongue detection model based on the tongue detection dataset, updating the model parameters of the tongue detection model, and obtaining a trained tongue detection model if the tongue detection result of each image output by the updated tongue detection model has converged to the labeled tongue bounding box of each image; training the tongue segmentation model based on the tongue segmentation dataset, updating the model parameters of the tongue segmentation model, and obtaining a trained tongue segmentation model if the tongue segmentation result of each image output by the updated tongue segmentation model has converged to the labeled tongue region segmentation box of each image.

[0193] Optionally, after training the tongue detection model and the tongue segmentation model based on the image dataset to obtain the trained tongue detection model and the trained tongue segmentation model, the tongue feature analysis device 40 further includes: a first testing unit 47; the first testing unit 47 is used to test the trained tongue detection model and the trained tongue segmentation model using their respective test sets to obtain the tongue detection output result and the tongue segmentation output result of the test set; to evaluate the tongue detection output result and the tongue segmentation output result of the test set using preset test indicators to obtain the test result of the tongue detection model and the test result of the tongue segmentation model; when the test result of the tongue detection model and the test result of the tongue segmentation model both meet the target indicator value, it is determined that the training of the tongue detection model and the tongue segmentation model is complete.

[0194] Optionally, when the second training unit 44 annotates the tongue segmentation image dataset to obtain a tongue feature classification dataset, it is specifically used to: remove image data in the tongue segmentation image dataset that does not meet the preset image quality requirements to obtain a removed tongue segmentation image dataset; and, in conjunction with the preset tongue feature classification criteria, perform feature annotation on the removed tongue segmentation image dataset to annotate the tongue features of each image to obtain a tongue feature classification dataset, wherein the tongue features include tongue color, tongue texture, coating color, coating texture, and overall features.

[0195] Optionally, when the second training unit 44 trains the tongue feature classification model based on the tongue feature classification dataset to obtain a trained tongue feature classification model, it is specifically used for: constructing a tongue feature classification model; training the tongue feature classification model based on the tongue feature classification dataset; updating the model parameters of the tongue feature classification model; and if the tongue feature analysis results of each image output by the updated tongue feature classification model have converged to the labeled tongue features of each image, then a trained tongue feature classification model is obtained.

[0196] After training the tongue feature classification model based on the tongue feature classification dataset to obtain the trained tongue feature classification model, the tongue feature analysis device 40 further includes: a second testing unit 48; the second testing unit 48 is used to test the trained tongue feature classification model using the test set in the tongue feature classification dataset to obtain the tongue feature classification output result of the test set; to evaluate the tongue feature classification output result of the test set using preset test indicators to obtain the test result of the tongue feature classification model; when the test result of the tongue feature classification model meets the target indicator value, it is determined that the tongue feature classification model training is complete.

[0197] Optionally, after training the tongue feature classification model based on the tongue feature classification dataset to obtain a trained tongue feature classification model, the tongue feature analysis device 40 further includes: an integration unit 49; the integration unit 49 is used to integrate the trained tongue feature classification model, color space conversion algorithm, channel analysis algorithm, and verification algorithm to obtain a statistical analysis module; inputting the tongue segmentation image data into the statistical analysis module, performing feature analysis on the tongue segmentation image data through the trained tongue feature classification model to obtain the confidence level of each feature category; using the color space conversion algorithm to convert the tongue segmentation image data into color-opposite space channel image data; inputting the tongue segmentation image data and the color-opposite space channel image data into the channel analysis algorithm respectively for channel analysis to obtain red-green-blue channel analysis results and color-opposite space channel analysis results; and performing distribution verification on the tongue segmentation image data and the color-opposite space channel image data through the verification algorithm to obtain the verification values ​​of the red-green-blue channels and the color-opposite space channels.

[0198] The tongue feature analysis device shown in Figure 11 can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0199] Figure 12 is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. The electronic device can be a tongue feature analysis device. Referring specifically to Figure 12 below, it shows a schematic diagram of the structure suitable for implementing the electronic device 600 in the embodiment of this disclosure. The electronic device shown in Figure 12 is merely an example and should not impose any limitation on the function and scope of use of the embodiments of this disclosure.

[0200] As shown in FIG12, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603 to implement the tongue feature analysis method as described in the embodiments of this disclosure. The RAM 603 also stores various programs and data required for the operation of the electronic device 600. The processing device 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604. Optionally, the computer program can be downloaded and installed from a network via a communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the methods of the embodiments of this disclosure.

[0201] It should be noted that the computer-readable medium described above in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0202] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0203] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, enable the electronic device to perform the tongue feature analysis methods described in the various embodiments above.

[0204] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0205] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0206] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for analyzing tongue features, characterized in that, The method includes: Construct an image dataset, which includes a large amount of image data containing images of tongues; The tongue detection model and the tongue segmentation model are trained based on the image dataset to obtain the trained tongue detection model and the trained tongue segmentation model. The trained tongue detection model and the trained tongue segmentation model are fused together to obtain a target segmentation model. The image dataset is then segmented using the target segmentation model to obtain a tongue segmentation image dataset. The tongue segmentation image dataset is labeled to obtain a tongue feature classification dataset. The tongue feature classification model is trained based on the tongue feature classification dataset to obtain a trained tongue feature classification model. Acquire the target image data to be detected, input the target image data into the target segmentation model, and obtain the tongue segmentation image data of the target image data to be detected; The tongue segmentation image data is analyzed by a trained tongue feature classification model to obtain tongue feature analysis results; The tongue detection model uses a lightweight network as the backbone network and uses depthwise separable convolution to reduce the number of parameters and computation while maintaining high detection accuracy. Spatial attention module and channel attention module are introduced in front of the final detection head to improve the model's ability to express tongue region features. The tongue feature classification model adopts a multi-task learning architecture, which integrates different tongue feature classification tasks into a unified fusion model. The overall architecture of the tongue feature classification model includes the following parts: a low-level feature extraction network, a task branch module, a shared feature layer, a loss function and optimization strategy, and model output. The underlying feature extraction network is used to extract low-level and high-level features of the tongue image. The task branch module is a network structure consisting of multi-layer feature extraction, skip connections, fully connected layers, and activation functions, used to classify and predict tongue color, tongue body, tongue coating color, tongue coating, and overall features. The shared feature layer is used to fuse and re-encode the low-level and high-level features output by the underlying feature extraction network, providing rich semantic features for each task branch. Each task branch has an independent loss function, and the cross-entropy loss function is used for each task branch of tongue color, tongue body, tongue coating color, tongue coating, and overall features. The model output adopts multi-task classification, simultaneously outputting classification predictions for tongue color, tongue body, tongue coating color, tongue coating, and overall features, with confidence scores for different categories output for each classification task. Multi-level features are extracted through a tongue feature classification model. Features from different levels are then fused together through feature concatenation operations and used as input for various classification tasks. An independent classification head is designed for each task so that the tongue feature classification model can perform fine-grained learning of specific features for each task, ensuring the classification accuracy of each task. The tongue segmentation model is based on a multi-layer convolutional neural network. The encoder part introduces a CBAM attention mechanism combination module and a residual module to enable the model to focus on more important tongue feature regions, improve the model's gradient flow and feature learning ability, and introduces multi-scale feature fusion at the decoder end to better capture the detailed information of the tongue edge. The tongue detection model and the tongue segmentation model were obtained through joint training; By fusing the trained tongue detection model and the trained tongue segmentation model, a unified interactive segmentation module is constructed to improve segmentation accuracy and efficiency. An interactive segmentation strategy is adopted. After obtaining the tongue segmentation image, tongue coating separation is not performed on the tongue segmentation image. Instead, multi-task classification is performed directly based on the complete tongue segmentation image to enhance the accuracy and stability of classification. After training the tongue feature classification model based on the tongue feature classification dataset to obtain the trained tongue feature classification model, the method further includes: The trained tongue feature classification model, color space conversion algorithm, channel analysis algorithm, and testing algorithm are integrated to obtain the statistical analysis module; The segmented tongue image data is input into the statistical analysis module, and the trained tongue feature classification model is used to perform feature analysis on the segmented tongue image data to obtain the confidence level of each feature category. The tongue segmentation image data is converted into color-opposite space channel image data using a color space conversion algorithm. The tongue segmentation image data and the color-opposite space channel image data are respectively input into the channel analysis algorithm for channel analysis to obtain the red, green and blue channel analysis results and the color-opposite space channel analysis results. The distribution of tongue segmentation image data and color-opposite space channel image data is tested using a test algorithm to obtain the test values ​​of the red, green and blue channels and the test values ​​of the color-opposite space channels. The method further includes: Based on the inference results of the statistical analysis module, various tongue feature analyses and channel statistical test results are extracted, and a structured statistical analysis report is output. This report includes multi-task feature classification prediction results, color channel statistics, and KS test statistics. The multi-task feature classification prediction results include predicted values ​​and confidence scores for tongue color, tongue texture, tongue coating color, tongue texture, and overall features, generating corresponding charts to display these results. The color channel statistics include the mean and standard deviation of the RGB and Lab channels, as well as the distance of the channel spatial mean from the eight vertices. The KS test statistics include the KS test p-value comparing the test tongue image distribution with the standard distribution, used to determine the significant difference between the distribution of each color channel and the standard distribution, generating corresponding KS test statistic charts.

2. The method according to claim 1, characterized in that, The construction of the image dataset includes: Collect a large amount of image data containing images of tongues; Perform color calibration processing on each of the image data to obtain color-calibrated image data; Each color-calibrated image data is cropped to obtain image data of each tongue region; Each color-calibrated image data is labeled to mark the tongue bounding box of each image, resulting in a tongue detection dataset. The tongue detection dataset is then divided into a training set, a validation set, and a test set for the tongue detection model based on a preset first ratio. Each image of the tongue region is labeled to mark the tongue region segmentation box of each image, resulting in a tongue segmentation dataset. The tongue segmentation dataset is then divided into a training set, a validation set, and a test set for the tongue segmentation model based on a preset second ratio.

3. The method according to claim 1, characterized in that, The image dataset includes a tongue detection dataset and a tongue segmentation dataset; The step of training the tongue detection model and the tongue segmentation model based on the image dataset to obtain the trained tongue detection model and the trained tongue segmentation model includes: Construct a tongue detection model and a tongue segmentation model; The tongue detection model is trained based on the tongue detection dataset, and the model parameters of the tongue detection model are updated. If the tongue detection result of each image output by the updated tongue detection model has converged to the labeled tongue bounding box of each image, then the trained tongue detection model is obtained. The tongue segmentation model is trained based on the tongue segmentation dataset, and the model parameters of the tongue segmentation model are updated. If the tongue segmentation result of each image output by the updated tongue segmentation model has converged to the labeled tongue region segmentation box of each image, then the trained tongue segmentation model is obtained.

4. The method according to claim 1, characterized in that, After training the tongue detection model and the tongue segmentation model based on the image dataset to obtain the trained tongue detection model and the trained tongue segmentation model, the method further includes: The trained tongue detection model and the trained tongue segmentation model are tested using their respective test sets to obtain the tongue detection output and tongue segmentation output results of the test sets. The tongue detection output and tongue segmentation output of the test set are evaluated using preset test indicators to obtain the test results of the tongue detection model and the tongue segmentation model. When the test results of the tongue detection model and the tongue segmentation model both meet the target index values, the training of the tongue detection model and the tongue segmentation model is considered complete.

5. The method according to claim 1, characterized in that, The annotation of the tongue segmentation image dataset to obtain a tongue feature classification dataset includes: Image data that does not meet the preset image quality requirements in the tongue segmentation image dataset are removed to obtain the tongue segmentation image dataset after removal. Based on the preset tongue feature classification criteria, the segmented tongue image dataset after removal is labeled with features, and the tongue features of each image are labeled to obtain the tongue feature classification dataset. The tongue features include tongue color, tongue texture, coating color, coating texture and overall features.

6. The method according to claim 1, characterized in that, The process of training a tongue feature classification model based on the tongue feature classification dataset to obtain a trained tongue feature classification model includes: Construct a tongue feature classification model; The tongue feature classification model is trained based on the tongue feature classification dataset, and the model parameters of the tongue feature classification model are updated. If the tongue feature analysis results of each image output by the updated tongue feature classification model have converged to the labeled tongue features of each image, then the trained tongue feature classification model is obtained. After training the tongue feature classification model based on the tongue feature classification dataset to obtain the trained tongue feature classification model, the method further includes: The trained tongue feature classification model is tested using the test set in the tongue feature classification dataset to obtain the tongue feature classification output results of the test set; The tongue feature classification output results of the test set are evaluated using preset test indicators to obtain the test results of the tongue feature classification model. When the test results of the tongue feature classification model meet the target index value, the tongue feature classification model is considered to have completed training.

7. A tongue feature analysis device, characterized in that, include: A construction unit is used to construct an image dataset, which includes a large amount of image data containing images of tongues; The first training unit is used to train the tongue detection model and the tongue segmentation model based on the image dataset, respectively, to obtain the trained tongue detection model and the trained tongue segmentation model. The first obtaining unit is used to fuse the trained tongue detection model and the trained tongue segmentation model to obtain a target segmentation model, and to segment the image dataset using the target segmentation model to obtain a tongue segmentation image dataset. The second training unit is used to annotate the tongue segmentation image dataset to obtain a tongue feature classification dataset, and to train the tongue feature classification model based on the tongue feature classification dataset to obtain a trained tongue feature classification model. The second obtaining unit is used to acquire the target image data to be detected, input the target image data into the target segmentation model, and obtain tongue segmentation image data of the target image data to be detected. The feature analysis unit is used to perform feature analysis on the segmented tongue image data using a trained tongue feature classification model to obtain tongue feature analysis results. The tongue detection model uses a lightweight network as the backbone network and uses depthwise separable convolution to reduce the number of parameters and computation while maintaining high detection accuracy. Spatial attention module and channel attention module are introduced in front of the final detection head to improve the model's ability to express tongue region features. The tongue feature classification model adopts a multi-task learning architecture, which integrates different tongue feature classification tasks into a unified fusion model. The overall architecture of the tongue feature classification model includes the following parts: a low-level feature extraction network, a task branch module, a shared feature layer, a loss function and optimization strategy, and model output. The underlying feature extraction network is used to extract low-level and high-level features of the tongue image. The task branch module is a network structure consisting of multi-layer feature extraction, skip connections, fully connected layers, and activation functions, used to classify and predict tongue color, tongue body, tongue coating color, tongue coating, and overall features. The shared feature layer is used to fuse and re-encode the low-level and high-level features output by the underlying feature extraction network, providing rich semantic features for each task branch. Each task branch has an independent loss function, and the cross-entropy loss function is used for each task branch of tongue color, tongue body, tongue coating color, tongue coating, and overall features. The model output adopts multi-task classification, simultaneously outputting classification predictions for tongue color, tongue body, tongue coating color, tongue coating, and overall features, with confidence scores for different categories output for each classification task. Multi-level features are extracted through a tongue feature classification model. Features from different levels are then fused together through feature concatenation operations and used as input for various classification tasks. An independent classification head is designed for each task so that the tongue feature classification model can perform fine-grained learning of specific features for each task, ensuring the classification accuracy of each task. The tongue segmentation model is based on a multi-layer convolutional neural network. The encoder part introduces a CBAM attention mechanism combination module and a residual module to enable the model to focus on more important tongue feature regions, improve the model's gradient flow and feature learning ability, and introduces multi-scale feature fusion at the decoder end to better capture the detailed information of the tongue edge. The tongue detection model and the tongue segmentation model were obtained through joint training; By fusing the trained tongue detection model and the trained tongue segmentation model, a unified interactive segmentation module is constructed to improve segmentation accuracy and efficiency. An interactive segmentation strategy is adopted. After obtaining the tongue segmentation image, tongue coating separation is not performed on the tongue segmentation image. Instead, multi-task classification is performed directly based on the complete tongue segmentation image to enhance the accuracy and stability of classification. The tongue feature analysis device further includes: an integration unit; The integration unit integrates the trained tongue feature classification model, color space conversion algorithm, channel analysis algorithm, and verification algorithm to obtain a statistical analysis module. The tongue segmentation image data is input into the statistical analysis module, and the trained tongue feature classification model performs feature analysis on the tongue segmentation image data to obtain the confidence level of each feature category. The color space conversion algorithm converts the tongue segmentation image data into color-opposite space channel image data. The tongue segmentation image data and the color-opposite space channel image data are then input into the channel analysis algorithm for channel analysis to obtain the red-green-blue channel analysis results and the color-opposite space channel analysis results. The verification algorithm performs distribution verification on the tongue segmentation image data and the color-opposite space channel image data to obtain the verification values ​​for the red-green-blue channels and the color-opposite space channels. The integration unit is also used to extract various tongue feature analysis and channel statistical test results based on the inference results of the statistical analysis module, and output a structured statistical analysis report. The statistical analysis report includes multi-task feature classification prediction results, color channel statistics, and KS test statistics. The multi-task feature classification prediction results include predicted values ​​and confidence scores for tongue color, tongue texture, tongue coating color, tongue texture, and overall features, and generate corresponding charts to display the multi-task feature classification prediction results. The color channel statistics include the mean and standard deviation of RGB and Lab channels, as well as the distance of the channel space mean from the eight vertices. The KS test statistics include the KS test p-value of the test tongue image distribution and the standard distribution, used to determine the significant difference between the distribution of each color channel and the standard distribution, and generate corresponding KS test statistics charts for the KS test statistics.

8. An electronic device, characterized in that, include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images