Constitution and fatigue state evaluation system based on tongue image recognition
Patent Information
- Application Number
- CN202611285475.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-24
- Publication Date
- 2026-09-25
AI Technical Summary
[0002]对于慢性肝病患者而言，其常常出现疲劳症状，但早期阶段疲劳感往往隐匿，患者可能表现为轻微不适或无明显自觉症状，易被忽视或误判，传统临床评估主要依赖于患者主观描述和医生经验性问诊，缺乏客观量化标准，导致疲劳程度判断不准确，延误干预时机；
[0021]本申请通过整合舌象图像特征与肝功能指标数据实现多源信息融合，并引入动态反馈优化机制有利于持续调整舌象评估模型，有效解决了隐匿性疲劳早期客观难以量化的问题，具有提供客观量化评估、实现多源数据深度整合、建立动态反馈优化机制以提高疲劳状态评估准确性的优点。
Smart Images
Figure CN122805216A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of tongue image assessment technology, specifically a system for assessing physical condition and fatigue status based on tongue image recognition. Background Technology
[0002] For patients with chronic liver disease, fatigue symptoms are common, but in the early stages, fatigue is often hidden. Patients may experience mild discomfort or no obvious symptoms, which are easily overlooked or misdiagnosed. Traditional clinical assessment mainly relies on the patient's subjective description and the doctor's experience in consultation, lacking objective quantitative standards, which leads to inaccurate judgment of the degree of fatigue and delays in intervention.
[0003] Tongue appearance is an important basis for diagnosis in traditional Chinese medicine, and its characteristics are closely related to the metabolic state of the liver. However, existing tongue appearance analysis technology has failed to achieve deep integration of tongue appearance image features with liver function biochemical data, and has not established a dynamic feedback optimization mechanism based on patient follow-up data. As a result, the tongue appearance assessment model cannot adapt to individual differences and disease progression, and it is difficult to objectively quantify hidden fatigue in the early stage. To address the above problems, existing technologies urgently need to be improved. Summary of the Invention
[0004] The purpose of this application is to provide a physical condition and fatigue state assessment system based on tongue image recognition, which has the advantages of providing objective quantitative assessment, realizing deep integration of multi-source data, and establishing a dynamic feedback optimization mechanism to improve the accuracy of fatigue state assessment.
[0005] The objective of this application can be achieved through the following technical solution: Firstly, a system for assessing physical condition and fatigue status based on tongue image recognition, comprising the following modules:
[0006] The data acquisition module is used to acquire tongue image data and liver function index data;
[0007] The feature extraction module is used to generate a tongue image feature vector based on the tongue image data and a liver function feature vector based on the liver function index data.
[0008] An anomaly detection module is used to obtain the degree of difference at each point on the tongue surface based on the tongue image data, identify abnormal areas on the tongue surface based on the degree of difference, and generate an anomaly feature vector based on the geometric features of the abnormal areas.
[0009] The data fusion module is used to generate a fused feature vector based on the abnormal feature vector, the tongue image feature vector, and the liver function feature vector;
[0010] The status assessment module is used to input the fused feature vector into a preset tongue image assessment model to output the fatigue level assessment, generate an initial assessment report, send the initial assessment report to the institution, and receive the corrected fatigue level from the institution.
[0011] The feedback optimization module is used to obtain an error coefficient based on the corrected fatigue level and the assessed fatigue level, optimize the weight of the fused feature vector based on the error coefficient to obtain an optimized fused feature vector, and input it into a preset tongue image assessment model to generate a final assessment report.
[0012] Secondly, the method for assessing physical condition and fatigue status based on tongue image recognition includes the following steps:
[0013] Acquire tongue image data and liver function index data;
[0014] A tongue image feature vector is generated based on the tongue image data, and a liver function feature vector is generated based on the liver function index data.
[0015] The difference degree at each point on the tongue surface is obtained based on the tongue image data, abnormal areas on the tongue surface are identified based on the difference degree, and abnormal feature vectors are generated based on the geometric features of the abnormal areas.
[0016] A fusion feature vector is generated based on the abnormal feature vector, the tongue image feature vector, and the liver function feature vector;
[0017] The fused feature vector is input into a preset tongue image assessment model to output the fatigue level assessment and generate an initial assessment report. The initial assessment report is sent to the institution and the corrected fatigue level is received from the institution.
[0018] The error coefficient is obtained based on the corrected fatigue level and the assessed fatigue level. The weight of the fused feature vector is optimized based on the error coefficient to obtain an optimized fused feature vector, which is then input into a preset tongue image assessment model to generate a final assessment report.
[0019] Thirdly, a computer storage medium storing computer-executable instructions, which, when executed, implement the physical condition and fatigue state assessment system based on tongue image recognition described in the first aspect.
[0020] Compared with the prior art, the beneficial effects of this application are:
[0021] This application integrates tongue image features with liver function index data to achieve multi-source information fusion, and introduces a dynamic feedback optimization mechanism to facilitate continuous adjustment of the tongue image assessment model. It effectively solves the problem of the difficulty in objectively quantifying the early stage of hidden fatigue, and has the advantages of providing objective quantitative assessment, realizing deep integration of multi-source data, and establishing a dynamic feedback optimization mechanism to improve the accuracy of fatigue state assessment. Attached Figure Description
[0022] Figure 1 This is a schematic diagram of the modules of the physical condition and fatigue state assessment system based on tongue image recognition according to this application;
[0023] Figure 2 This is a schematic diagram illustrating the steps of the method for assessing physical condition and fatigue status based on tongue image recognition in this application. Detailed Implementation
[0024] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but only to illustrate selected embodiments of this application.
[0025] Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item has been defined in one figure, it does not need to be further defined and explained in subsequent figures. The terms "first", "second", etc. are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.
[0026] In the clinical assessment of patients with chronic liver disease, the insidious nature of early fatigue makes it impossible for traditional medical history taking methods to provide objective quantitative indicators. Because fatigue symptoms lack clear physiological markers, the assessment process relies heavily on patient subjective descriptions and physician experience, making it difficult to establish a correlation between fatigue and liver metabolic status, thus affecting the accuracy and consistency of the assessment results. Furthermore, the potential of tongue appearance as a non-invasive indicator has not been fully explored, and the limitations of single-modality data further exacerbate assessment bias, failing to meet the needs of early intervention.
[0027] For example, during an outpatient follow-up, a patient with chronic liver disease reported mild fatigue symptoms, but their liver function biochemical indicators were within the normal range. Images obtained through tongue imaging showed increased diameter and darker color of the sublingual veins, as well as increased stickiness of the tongue coating. However, the doctor, relying solely on the single consultation result, rated the fatigue level as low, failing to identify the potential correlation between local abnormalities in the tongue image and metabolic stagnation. In this scenario, the tongue image features were not extracted in a structured manner, and the liver function index data and tongue feature vectors lacked effective fusion, resulting in the assessment results failing to reflect the dynamic changes in liver function, thus delaying the objective determination of the fatigue state.
[0028] If these problems are not addressed, the insidious nature of fatigue will cause a disconnect between assessment results and actual pathological progression, making it impossible to monitor and intervene in liver metabolic abnormalities in a timely manner. Furthermore, the reliability of the assessment system will decrease, individual differences and disease progression factors will not be incorporated into the dynamic optimization mechanism, ultimately affecting the effectiveness of clinical decision-making and potentially accelerating the disease's deterioration.
[0029] Therefore, this application provides a system for assessing physical condition and fatigue status based on tongue image recognition, such as... Figure 1 As shown, it includes the following modules:
[0030] The data acquisition module is used to acquire tongue image data and liver function index data;
[0031] The feature extraction module is used to generate a tongue image feature vector based on the tongue image data and a liver function feature vector based on the liver function index data.
[0032] An anomaly detection module is used to obtain the degree of difference at each point on the tongue surface based on the tongue image data, identify abnormal areas on the tongue surface based on the degree of difference, and generate an anomaly feature vector based on the geometric features of the abnormal areas.
[0033] The data fusion module is used to generate a fused feature vector based on the abnormal feature vector, the tongue image feature vector, and the liver function feature vector;
[0034] The status assessment module is used to input the fused feature vector into a preset tongue image assessment model to output the fatigue level assessment, generate an initial assessment report, send the initial assessment report to the institution, and receive the corrected fatigue level from the institution.
[0035] The feedback optimization module is used to obtain an error coefficient based on the corrected fatigue level and the assessed fatigue level, optimize the weight of the fused feature vector based on the error coefficient to obtain an optimized fused feature vector, and input it into a preset tongue image assessment model to generate a final assessment report.
[0036] In practical applications, multimodal data fusion and closed-loop feedback mechanisms are used to objectively quantify the fatigue state of patients with chronic liver disease. The data acquisition module obtains tongue image data and liver function index data. The tongue image data includes RGB images of the tongue surface and sublingual region to capture the visual features of the tongue body, tongue coating, and sublingual veins. The liver function index data includes bilirubin metabolism indicators, enzyme indicators, and synthetic function indicators to quantify the liver's metabolic state.
[0037] The feature extraction module extracts tongue surface feature parameters (such as the thickness and greasiness of the tongue coating) and sublingual feature parameters (such as the diameter, color, and tortuosity of sublingual veins) from tongue image data, and generates a structured tongue image feature vector. Simultaneously, it standardizes liver function index data to generate a liver function feature vector, thus transforming complex visual and biochemical information into a quantifiable vector representation. The anomaly identification module analyzes the feature comparison between each point in the tongue image and its neighboring areas using a difference detection algorithm, identifying anomalies and generating anomaly feature vectors. This algorithm can keenly capture subtle local changes in the tongue image, providing a sensitive basis for early fatigue assessment.
[0038] The data fusion module integrates abnormal feature vectors, tongue image feature vectors, and liver function feature vectors to generate a fusion feature vector that comprehensively represents multi-source information, overcoming the limitations of single-modal data. The state assessment module inputs the fusion feature vector into a preset tongue image assessment model, calculates its proximity to a reference feature vector, outputs an assessment level of fatigue, and generates an initial assessment report based on an intervention strategy library, achieving an objective determination of fatigue status. The feedback optimization module dynamically adjusts the weights of the feature vectors based on the error coefficient between the corrected fatigue level and the assessed fatigue level, generating an optimized fusion feature vector and a final assessment report, forming a closed-loop feedback mechanism that enables the system to adapt to individual differences and disease progression.
[0039] Therefore, by integrating tongue image features with liver function biochemical indicators, this system constructs a multimodal data closed-loop assessment framework, effectively solving the problems of concealed early fatigue in patients with chronic liver disease and the difficulty in quantifying it through traditional medical history taking. Specifically, the structured extraction of tongue image features and anomaly identification enable sensitive monitoring of concealed pathological changes, while the data fusion module integrates multi-source information to enhance the comprehensiveness of the assessment and avoid the limitations of a single data source.
[0040] The objective judgment mechanism of the condition assessment module eliminates reliance on subjective experience, ensuring standardized assessment results. The closed-loop mechanism of the feedback optimization module, through dynamic weight adjustment, enables the system to adapt to individual differences and disease progression, significantly improving the accuracy and personalization of fatigue state assessment. This technical solution transforms tongue imagery from static observation to dynamic process monitoring, closely linking it to the pathophysiological processes of liver metabolism, achieving objective and early assessment of fatigue state, and providing reliable technical support for the physical condition and fatigue management of patients with chronic liver disease.
[0041] It should be further explained that, in the specific implementation process, the acquisition of tongue image data and liver function indicator data includes:
[0042] The tongue image data refers to the RGB images of the tongue obtained through a standardized acquisition process, including RGB images of the tongue surface and RGB images of the sublingual region. These image data are used to extract visual features such as tongue texture, tongue coating, tongue shape, and sublingual veins.
[0043] Using a camera equipped with a supplementary light device, RGB images of the tongue are captured under standard lighting conditions. The user naturally extends their tongue, and a front view of the tongue and a close-up image under the tongue are taken. The timestamp and user ID are saved at the same time.
[0044] The liver function index data refers to a series of indicators related to liver function obtained through blood biochemical analysis, including bilirubin metabolism indicators (total bilirubin, direct bilirubin), enzyme indicators (alanine aminotransferase), and synthetic function indicators (albumin). These indicators are used to quantitatively assess the liver's metabolic, excretory, and synthetic functions.
[0045] Total bilirubin (TBIL, normal range 3.4-20.5 μmol / L), direct bilirubin (DBIL, normal range 0-6.8 μmol / L), alanine aminotransferase (ALT, normal range 0-40 U / L), and albumin (ALB, normal range 35-55 g / L) are associated with tongue image data through the same timestamp and user ID.
[0046] Specifically, the solution in this application standardizes the image acquisition conditions through a standardized acquisition process, ensuring the consistency of tongue image data under different acquisition scenarios; the coordinated configuration of the supplementary lighting device and the standard light source environment eliminates ambient light interference, ensuring that the color features of the tongue surface RGB image and the sublingual RGB image are stable and comparable; the operation of taking frontal views of the tongue surface and close-up images of the sublingual image respectively covers the visual features of key areas of the tongue, providing a complete data foundation for the subsequent extraction of parameters such as tongue texture and tongue coating;
[0047] Liver function index data are obtained through blood biochemical analysis. The quantitative characteristics of its bilirubin metabolism index, enzyme index, and synthetic function index provide an objective basis for liver function assessment. The synchronous recording mechanism of timestamp and user ID accurately correlates tongue image data with liver function index data in the time dimension, avoiding feature misalignment caused by data acquisition time difference. The above links form a closed-loop data acquisition system to ensure that the data input to the feature extraction module has high quality, high synchronization and low subjective interference characteristics.
[0048] This application effectively solves the problem of feature distortion caused by ambient light interference in tongue images, avoids the correlation error between liver function indicators and tongue image data caused by time misalignment, reduces the subjective bias in correcting the assessment level, and thus provides a reliable data basis for the assessment of physical condition and fatigue status, significantly improving the accuracy and reliability of the assessment results.
[0049] It should be further explained that, in the specific implementation process, the process of generating a tongue image feature vector based on the tongue image data and a liver function feature vector based on the liver function index data includes:
[0050] Based on the RGB image of the tongue surface in the tongue image data, tongue surface feature parameters are extracted, including the thickness and greasiness of the tongue coating, which reflects the thickness and greasiness of the tongue coating in various parts of the tongue surface. The thickness and greasiness of the tongue coating is calculated through texture contrast, using the following formula: T represents the thickness and greasiness of the tongue coating. Let be the joint probability of gray levels a and b in the gray-level co-occurrence matrix;
[0051] Based on the RGB images of the tongue image data, the sublingual feature parameters are extracted, including the diameter of the sublingual veins, the color of the sublingual veins, and the tortuosity of the sublingual veins, which are used to reflect the diameter, color, and tortuosity of blood vessels in various parts of the tongue.
[0052] The diameter of the sublingual veins is obtained through pixel distance conversion, using the formula: Where D is the diameter of the sublingual vein at any location, p is the pixel width of the corresponding location of the blood vessel in the image, and s is the calibration coefficient, which is usually 0.1-0.3 mm / pixel.
[0053] The color of the sublingual veins is calculated using a blood color index, with the formula as follows: HI represents the color of the sublingual veins, while R, G, and B represent the average red, green, and blue channel values of the blood vessels in various locations under the tongue, respectively.
[0054] The tortuosity of the sublingual veins is obtained by curvature integral, and the formula is as follows: Where C is the tortuosity of the sublingual vein, k(s) is the curvature of the blood vessel at any location, and L is the length of the blood vessel at the corresponding location.
[0055] The maximum thickness of tongue coating, diameter of sublingual veins, color of sublingual veins, and tortuosity of sublingual veins are extracted from the RGB images of the tongue surface and the RGB images of the sublingual surface under the same time stamp, and these are combined into a 4-dimensional vector, which is denoted as the tongue image feature vector. All values are normalized to the interval [0, 1].
[0056] The Z-score method was used to standardize the total bilirubin, direct bilirubin, alanine aminotransferase and albumin in the liver function index data at the same time point. The values obtained after standardization were combined into a 4-dimensional vector, which is denoted as the liver function feature vector.
[0057] Specifically, the solution of this application first extracts the thickness of the tongue coating from the RGB image of the tongue surface. This process uses the gray-level co-occurrence matrix to analyze the spatial distribution characteristics of texture features, transforming subjective visual evaluation into objective values based on gray-level joint probability. At the same time, the diameter, color, and tortuosity of the sublingual veins are extracted from the RGB image of the sublingual veins. The quantification of vascular features is achieved through the physical mapping of pixel width and calibration coefficient, the color calculation of RGB channels, and the integral accumulation of curve curvature, respectively.
[0058] Subsequently, the maximum values of each feature at the same timestamp were obtained and combined and normalized into a tongue image feature vector to ensure that key pathological features are prominent and the numerical range is uniform; the liver function indicators were standardized by Z-score to generate liver function feature vectors, eliminating the dimensional differences of the original indicators such as total bilirubin, direct bilirubin, alanine aminotransferase, and albumin.
[0059] Ultimately, this process enables tongue features and liver function indicators to form comparable data on the same scale, providing an objective basis for multimodal fusion. Through standardized feature extraction and vectorization mechanisms, this process transforms multi-source heterogeneous data into a unified quantitative representation, effectively solving the problems of subjectivity in feature extraction and data inconsistency.
[0060] This application achieves objective quantification of the feature extraction process, avoiding data inconsistencies caused by subjective visual judgment. At the same time, the multi-source data undergoes unified standardization processing, ensuring the comparability of tongue features and liver function indicators on the same scale, thereby improving the accuracy and reliability of subsequent fatigue state assessment.
[0061] It should be further explained that, in the specific implementation process, the process of obtaining the gray-level co-occurrence matrix and the joint probability includes:
[0062] The gray-level co-occurrence matrix is a tool for describing the texture features of an image. It reflects texture information by studying the spatial correlation characteristics of image gray levels. Its core idea is to analyze the frequency of co-occurrence of pixel pairs of different gray levels in an image under specific spatial relationships.
[0063] First, the RGB image of the tongue surface is converted into a grayscale image and the grayscale level is quantized. Then, the frequency of co-occurrence of pixel pairs of grayscale levels is counted according to the preset distance (d=1) and direction (0°, 45°, 90°, 135°) to form an initial matrix.
[0064] Next, the initial matrix is normalized, that is, each element is divided by the sum of all elements of the initial matrix, thus obtaining the gray-level co-occurrence matrix, where each value represents the joint probability of the corresponding gray level co-occurring under a specific spatial relationship.
[0065] It should be further explained that, in the specific implementation process, the process of obtaining the degree of difference at various points on the tongue surface based on the tongue image data, identifying abnormal regions on the tongue surface based on the degree of difference, and generating an abnormal feature vector based on the geometric features of the abnormal regions includes:
[0066] A difference detection algorithm is used to compare the features of each pixel in the RGB image of the tongue surface with its neighboring regions to obtain the difference of each pixel on the tongue surface. The neighboring regions refer to the circular regions with each pixel as the center and a radius of 10 pixels.
[0067] ;
[0068] Among them, D p For the difference of a single pixel, F p F represents the chromaticity value of this single pixel in the CIELAB color space. n(i) is the chromaticity value of each pixel i in the neighborhood of the single pixel in the CIELAB color space, and n is the number of pixels in the neighborhood of the single pixel;
[0069] Set a difference threshold, and initially identify the connected regions composed of pixels with a difference higher than the difference threshold as outliers. For each outlier, use a region growing algorithm to obtain its boundary, and take the region within the boundary as the outlier region of the corresponding outlier.
[0070] Obtain the roundness and elongation corresponding to a single abnormal region, wherein the roundness , among which, S y C refers to the area of the single anomalous region. y The term refers to the perimeter of the single abnormal region, and the term "extension length" refers to the aspect ratio of the smallest bounding rectangle of the single abnormal region.
[0071] Obtain each abnormal region and its corresponding roundness and elongation from the RGB image of the tongue surface at the same timestamp. Combine the largest roundness and elongation into a 2D vector, which is denoted as the abnormal feature vector.
[0072] Specifically, the proposed solution performs pixel-by-pixel analysis of the RGB image of the tongue surface using a difference detection algorithm. Leveraging the sensitivity of the CIELAB color space to human visual perception, each pixel is compared in color to pixels within a 10-pixel radius circular neighborhood. The average absolute deviation is calculated as the difference, thus systematically detecting local inconsistencies. After setting a difference threshold, pixels with high differences are clustered into connected regions to form preliminary anomalies. A region growing algorithm is then used to accurately obtain the boundaries of these anomaly regions, ensuring that the boundary range accurately reflects the pathological area.
[0073] Subsequently, roundness and elongation were calculated for each abnormal region: roundness quantifies the degree of roundness based on the relationship between area and perimeter, while elongation describes the shape's extensibility using the aspect ratio of the minimum bounding rectangle. Together, they constitute the quantitative basis for distinguishing different pathological types such as petechiae and fissures. Finally, the combination of the maximum roundness and elongation was selected as a 2D abnormal feature vector, focusing on the most significant pathological features, avoiding information redundancy, and effectively integrating local abnormal information into subsequent fusion analysis, thereby compensating for the limitations of global feature extraction.
[0074] This application can effectively identify local abnormal areas on the tongue surface and quantify their geometric features, thereby capturing subtle pathological changes such as petechiae and cracks that are ignored by traditional methods, significantly improving the accuracy and sensitivity of physical condition and fatigue status assessment, and is especially suitable for the accurate identification of early hidden fatigue in patients with chronic liver disease.
[0075] It should be further explained that, in the specific implementation process, the process of generating a fused feature vector based on the abnormal feature vector, the tongue image feature vector, and the liver function feature vector includes:
[0076] The abnormal feature vector, the tongue appearance feature vector, and the liver function feature vector are all normalized to the interval [0, 1]. Initial weight values are then assigned to each feature vector. ;
[0077] After multiplying each element in different feature vectors by their corresponding initial weight values, they are combined sequentially into a 10-dimensional vector in the order of abnormal feature vector, tongue image feature vector, and liver function feature vector, which is denoted as the fusion feature vector.
[0078] Specifically, the solution in this application uses a normalization step to uniformly scale the abnormal feature vector, tongue image feature vector, and liver function feature vector to the [0, 1] interval, eliminating the fusion barrier caused by the different dimensions of tongue image features (such as the thickness of the tongue coating and the diameter of the sublingual veins) and liver function indicators (such as total bilirubin and alanine aminotransferase); then, an initial weight value is assigned to each feature vector through a weight setting step. This allows for differentiated representation of the sensitivity of abnormal feature vectors to pathological abnormalities, the ability of tongue image feature vectors to characterize the overall tongue image state, and the quantitative effect of liver function feature vectors on metabolic function.
[0079] Finally, the elements of each feature vector are multiplied by their corresponding weights through a weighted combination step and then concatenated in order to form a fused feature vector. This process not only preserves the independent value of anomaly information, overall tongue features, and liver function indicators, but also amplifies the contribution of key features through weight adjustment, so that the fused feature vector can comprehensively reflect the intrinsic relationship between the user's physical condition and fatigue state, thereby providing balanced and information-rich input features for the subsequent evaluation model.
[0080] Through the above scheme, this application effectively solves the problems of dimensional differences and uneven importance in multi-source feature vector fusion, enabling the fused feature vector to more accurately represent the comprehensive correlation between tongue abnormalities, overall tongue condition and liver function indicators, thereby improving the reliability and accuracy of fatigue state assessment.
[0081] It should be further explained that, in the specific implementation process, the process of inputting the fused feature vector into a preset tongue image assessment model to output the fatigue level assessment and generate an initial assessment report, sending the initial assessment report to the institution, and receiving the corrected fatigue level from the institution includes:
[0082] Different fatigue states are set, including mild fatigue state, moderate fatigue state, and severe fatigue state, and corresponding reference feature vectors are set for different fatigue states. The reference feature vectors are all 10-dimensional vectors.
[0083] The tongue image assessment model is as follows: , where d k To fuse the proximity between the feature vector and the reference feature vector of the k-th fatigue state, j=1,2,...,m,m=10, f j To fuse the j-th element in the feature vector, r kj It is the j-th element in the reference feature vector of the k-th fatigue state;
[0084] The fused feature vector is input into the tongue image assessment model to obtain the proximity between the corresponding fused feature vector and the reference feature vector of each fatigue state. The fatigue state with the smallest proximity is selected as the fatigue level. If there are cases with the same distance, the fatigue state with the higher level is selected first, with severe fatigue state being higher than moderate fatigue state being higher than mild fatigue state.
[0085] An intervention strategy library is set up, which stores descriptions and suggestions corresponding to different fatigue states. For example, mild fatigue state corresponds to "increase water intake", moderate fatigue state corresponds to "adjust diet structure" and severe fatigue state corresponds to "seek medical check-up". The corresponding descriptions and suggestions are retrieved according to the output fatigue level assessment and combined into an initial assessment report with corresponding fusion feature vector.
[0086] The user can choose whether to send the initial assessment report to the institution. If yes, the initial assessment report, along with its tongue image data and liver function index data, will be sent from the user's terminal to the institution's terminal. The user's terminal refers to the user's terminal, and the institution's terminal refers to the medical institution's terminal. If no, no other operations will be performed.
[0087] Based on the initial assessment report and its tongue image data and liver function index data, relevant personnel from the medical institution provide the user with a corrected fatigue status, which is recorded as the corrected fatigue level, and the corrected fatigue level is sent back to the user's terminal.
[0088] The proposed solution inputs the fused feature vector into the tongue image assessment model, calculates its proximity to the reference feature vectors of each fatigue state, automatically determines the fatigue level based on the minimum distance principle, and generates a structured report by combining the intervention strategy library.
[0089] Specifically, the system first receives a 10-dimensional fusion feature vector from the data fusion module, and then calculates the Euclidean distance between the vector and the reference feature vectors corresponding to mild, moderate and severe fatigue states in parallel in the tongue image assessment model to obtain three proximity values. Then, these proximity values are compared, and the fatigue state with the smallest value is selected as the preliminary assessment result. If there is an equal value, the higher-level state is selected according to the preset priority rules.
[0090] Finally, the system queries the intervention strategy library based on the determined fatigue level assessment, extracts the corresponding descriptive text and health suggestions, and automatically combines them into a complete report containing assessment results and guidance recommendations. This design achieves an end-to-end automated process from multi-source feature fusion to clinical decision support, ensuring the objectivity of the assessment process and the interpretability of the results.
[0091] Through the above technical solution, the system can objectively map the fused feature vector to a specific fatigue state, realizing automated quantitative assessment of fatigue state, avoiding the defects of traditional methods that rely on subjective judgment; at the same time, the generated initial assessment report contains targeted health guidance suggestions, which significantly improves the user's understanding and clinical applicability of the assessment results, and effectively solves the technical problem of early quantitative assessment of fatigue state in patients with chronic liver disease.
[0092] It should be further explained that, in the specific implementation process, the process of obtaining the error coefficient based on the corrected fatigue level and the assessed fatigue level includes:
[0093] For different fatigue states, corresponding baseline values are set, with the baseline values ranging from [0, 1]. Tongue image data, liver function index data, and corresponding corrected assessment levels and fatigue assessment levels at the same time stamp are included in the same sample. The latest samples at each time stamp are obtained, and the error coefficient W is:
[0094] ;
[0095] Where b = 1, 2, ..., h, h is the number of samples obtained, PR b PG is the baseline value corresponding to the corrected assessment level in the b-th sample. b Let be the proximity of the fatigue level assessment in the b-th sample.
[0096] Specifically, the solution of this application solves the problem of the incomputability of grade differences by mapping discrete fatigue levels to continuous benchmark values; the standardization of the benchmark value range to the [0, 1] interval ensures the stability of numerical calculation and prevents the dimension from interfering with the subsequent optimization process; the strict binding of tongue image data, liver function index data and corresponding fatigue levels under the same timestamp eliminates the evaluation distortion caused by time misalignment.
[0097] Focusing on the most recent samples at the timestamps enables error calculation to respond promptly to dynamic changes in individual physiological states and disease progression trends; the error coefficient W is calculated using the mean squared error method, and significant errors are amplified through squaring to prioritize critical issues, while averaging smooths random noise; PR b The revised assessment grade will be converted into a continuous benchmark value as a true reference. b By utilizing the model's internal proximity to preserve fine-grained evaluation, the two work together to ensure that the error signal accurately reflects the model's performance deviation, thus providing a reliable basis for weight optimization.
[0098] Through the above technical solution, this application realizes the quantification of the difference between the assessed fatigue level and the corrected assessment level, enabling the system to dynamically calibrate the model weights to adapt to individual physiological changes and data deviations, thereby improving the accuracy and adaptability of fatigue state assessment.
[0099] It should be further explained that, in the specific implementation process, the process of optimizing the weights of the fused feature vector based on the error coefficient to obtain an optimized fused feature vector, and then inputting it into the preset tongue image assessment model to generate the final assessment report includes:
[0100] The initial weights corresponding to the abnormal feature vector, tongue image feature vector, and liver function feature vector are determined based on the obtained error coefficients. Perform optimizations separately to obtain the latest optimized weight values;
[0101] ;
[0102] ;
[0103] ;
[0104] in, These are the optimized weight values corresponding to the abnormal feature vector, tongue appearance feature vector, and liver function feature vector, respectively. This is a preset adjustment factor, with a value range of 0.01-0.1.
[0105] Each element in the subsequently obtained feature vectors is multiplied by its latest optimized weight value, and then combined in the order of abnormal feature vector, tongue image feature vector, and liver function feature vector to form an optimized fusion feature vector.
[0106] The optimized fusion feature vector is input into the tongue image assessment model, and the corresponding fatigue level is output. The final assessment report is generated by combining the intervention strategy library and fed back to the user.
[0107] Specifically, the scheme in this application drives the weight optimization process through the gradient information of the error coefficients, using the error coefficients as the direct basis for weight updates, ensuring that the weight adjustment directions of the abnormal feature vector, tongue image feature vector, and liver function feature vector are consistent with the direction of reducing the evaluation bias; during the weight optimization process, the adjustment factor... Constrain the weight update range to avoid drastic changes in weights due to fluctuations in the error coefficient, and ensure that the optimization process is within a reasonable range;
[0108] The optimized weight values are applied to the subsequent weighted fusion of feature vectors, enabling the fused feature vectors to dynamically reflect the real-time contributions of abnormal tongue features, tongue ontological features, and liver function indicators. After the optimized fused feature vectors are input into the tongue assessment model, they are combined with the final assessment report generated by the intervention strategy library to form a closed-loop learning mechanism through feedback, enabling the system to continuously absorb error information from new data and gradually improve its adaptability to individual differences and disease evolution.
[0109] Through the above technical solution, this application realizes the dynamic adaptive adjustment of feature fusion weights, so that the relative importance allocation of tongue image features, abnormal features and liver function features is closely related to the actual assessment deviation, effectively improving the stability and accuracy of the system's quantitative assessment of fatigue status under individual differences, data fluctuations or disease progression scenarios, thereby ensuring that the assessment results can reliably reflect the user's true fatigue status in the long term.
[0110] In another embodiment, this application also provides a method for assessing physical condition and fatigue status based on tongue image recognition, such as... Figure 2 As shown, it includes the following steps:
[0111] Acquire tongue image data and liver function index data;
[0112] A tongue image feature vector is generated based on the tongue image data, and a liver function feature vector is generated based on the liver function index data.
[0113] The difference degree at each point on the tongue surface is obtained based on the tongue image data, abnormal areas on the tongue surface are identified based on the difference degree, and abnormal feature vectors are generated based on the geometric features of the abnormal areas.
[0114] A fusion feature vector is generated based on the abnormal feature vector, the tongue image feature vector, and the liver function feature vector;
[0115] The fused feature vector is input into a preset tongue image assessment model to output the fatigue level assessment and generate an initial assessment report. The initial assessment report is sent to the institution and the corrected fatigue level is received from the institution.
[0116] The error coefficient is obtained based on the corrected fatigue level and the assessed fatigue level. The weight of the fused feature vector is optimized based on the error coefficient to obtain an optimized fused feature vector, which is then input into a preset tongue image assessment model to generate a final assessment report.
[0117] The core innovation of this embodiment lies in constructing a multimodal assessment framework by integrating tongue image features and liver function index data, and introducing a feedback optimization mechanism to achieve dynamic adjustment. This transforms tongue image observation from static to dynamic monitoring, closely linking it to the pathophysiological process of liver metabolism. It solves the problems of difficulty in quantifying fatigue status and strong subjectivity in patients with chronic liver disease, and realizes early objective assessment and personalized dynamic monitoring of hidden fatigue.
[0118] In another embodiment, this application also provides a computer storage medium storing computer-executable instructions, which, when executed, implement the aforementioned physical condition and fatigue state assessment system based on tongue image recognition.
[0119] The core innovation of this embodiment lies in combining tongue image features with liver function biochemical indicators through multimodal data fusion and introducing a feedback optimization mechanism based on error coefficients. This transforms tongue image observation from static to dynamic process monitoring, closely linking it to the pathophysiological process of liver metabolism. It solves the problems of hidden fatigue in the early stages of chronic liver disease patients and the difficulty in quantifying it through traditional consultation, and achieves objective quantification and personalized assessment of fatigue status.
[0120] Specifically, this technical solution utilizes computer storage media as the physical carrier of executable instructions, ensuring that the evaluation system can operate independently of a specific hardware environment. It encodes the logic of modules such as data acquisition, feature extraction, anomaly detection, data fusion, state assessment, and feedback optimization into executable code, enabling the system to automatically process tongue images and liver function indicators. For example, it extracts tongue surface feature parameters from tongue image data and generates tongue feature vectors. By integrating standardized image processing algorithms, it quantifies key parameters such as tongue coating thickness and sublingual vein diameter, transforming subjective visual observation into objective numerical features.
[0121] Simultaneously, the instruction-defined weight optimization mechanism dynamically integrates abnormal feature vectors, tongue image feature vectors, and liver function feature vectors to generate a fused feature vector. This fused feature vector is then mapped to fatigue levels using a tongue image assessment model, and personalized suggestions are output based on an intervention strategy library. Thus, the system forms a closed-loop process from data input to assessment output. By continuously collecting clinical follow-up data and adaptively adjusting feature weights, it effectively overcomes the limitations of single-modality data, significantly improves the objectivity and adaptability of fatigue state assessment, and avoids assessment biases caused by reliance on human experience in traditional methods.
[0122] The above embodiments are only used to illustrate the technical methods of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of this application without departing from the spirit and scope of the technical methods of this application.
Claims
1. A system for assessing physical condition and fatigue status based on tongue image recognition, characterized in that, Includes the following modules: The data acquisition module is used to acquire tongue image data and liver function index data; The feature extraction module is used to generate a tongue image feature vector based on the tongue image data and a liver function feature vector based on the liver function index data. An anomaly detection module is used to obtain the degree of difference at each point on the tongue surface based on the tongue image data, identify abnormal areas on the tongue surface based on the degree of difference, and generate an anomaly feature vector based on the geometric features of the abnormal areas. The data fusion module is used to generate a fused feature vector based on the abnormal feature vector, the tongue image feature vector, and the liver function feature vector; The status assessment module is used to input the fused feature vector into a preset tongue image assessment model to output the fatigue level assessment, generate an initial assessment report, send the initial assessment report to the institution, and receive the corrected fatigue level from the institution. The feedback optimization module is used to obtain an error coefficient based on the corrected fatigue level and the assessed fatigue level, optimize the weight of the fused feature vector based on the error coefficient to obtain an optimized fused feature vector, and input it into a preset tongue image assessment model to generate a final assessment report.
2. The system for assessing physical condition and fatigue status based on tongue image recognition according to claim 1, characterized in that, The tongue image data includes RGB images of the tongue surface and RGB images of the sublingual region; the liver function index data includes total bilirubin, direct bilirubin, alanine aminotransferase, and albumin. The thickness and greasiness of the tongue coating are extracted from the RGB image of the tongue surface as a characteristic parameter of the tongue surface. , Let be the joint probability of gray levels a and b in the gray-level co-occurrence matrix; The diameter, color, and tortuosity of the sublingual veins are extracted from the RGB images of the sublingual veins as sublingual feature parameters. The diameter of the sublingual veins is obtained by pixel distance conversion, the color of the sublingual veins is calculated by blood color index, and the tortuosity of the sublingual veins is obtained by curvature integral. The tongue coating thickness, sublingual vein diameter, sublingual vein color, and sublingual vein tortuosity extracted from the tongue surface RGB image and sublingual RGB image at the same time stamp are sequentially combined into a tongue image feature vector. The Z-score method was used to standardize the total bilirubin, direct bilirubin, alanine aminotransferase, and albumin in the liver function index data at the same time point. The standardized parameters were then combined sequentially to form a liver function feature vector.
3. The system for assessing physical condition and fatigue status based on tongue image recognition according to claim 2, characterized in that, The process of obtaining the gray-level co-occurrence matrix and joint probability is as follows: The RGB image of the tongue surface is converted into a grayscale image and the grayscale levels are quantized. The frequency of co-occurrence of pixel pairs of grayscale levels is statistically analyzed according to preset distance and direction to form an initial matrix. The initial matrix is normalized to obtain a grayscale co-occurrence matrix, where each value represents the joint probability of the corresponding grayscale level co-occurring under a specific spatial relationship.
4. The system for assessing physical condition and fatigue status based on tongue image recognition according to claim 2, characterized in that, The process of obtaining the difference and identifying abnormal regions, as well as generating abnormal feature vectors, includes: A difference detection algorithm is used to obtain the difference between each pixel in the RGB image of the tongue surface. D p For the difference of a single pixel, F p F represents the chromaticity value of this single pixel in the CIELAB color space. n(i) is the chromaticity value of each pixel i in the CIELAB color space within the neighborhood of the single pixel, and n is the number of pixels in the neighborhood of the single pixel. The neighborhood refers to a circular area with a fixed radius centered on the single pixel. The connected regions formed by pixels with a difference degree higher than the preset difference threshold are taken as outliers. The boundary of each outlier is obtained by using a region growing algorithm. The region within the boundary is taken as the outlier region of the corresponding outlier. The circularity and elongation are obtained according to the area and perimeter of a single outlier region and the aspect ratio of the minimum bounding rectangle. The maximum circularity and elongation of each abnormal region extracted from the RGB image of the tongue surface at the same timestamp are sequentially combined to form an abnormal feature vector.
5. The system for assessing physical condition and fatigue status based on tongue image recognition according to claim 1, characterized in that, The process of generating fused feature vectors includes: The abnormal feature vector, the tongue image feature vector, and the liver function feature vector are all normalized to the interval [0, 1]. An initial weight value is set for each feature vector. Each element in different feature vectors is multiplied by its corresponding initial weight value, and then combined in the order of abnormal feature vector, tongue image feature vector, and liver function feature vector to form a fusion feature vector.
6. The system for assessing physical condition and fatigue status based on tongue image recognition according to claim 5, characterized in that, The process of outputting the fatigue level assessment and generating an initial assessment report includes: Reference feature vectors are set for different fatigue states, including mild fatigue state, moderate fatigue state, and severe fatigue state. The tongue image assessment model is as follows: d k To fuse the proximity between the feature vector and the reference feature vector of the k-th fatigue state, j=1,2,...,m,m=10, f j To fuse the j-th element in the feature vector, r kj It is the j-th element in the reference feature vector of the k-th fatigue state; The proximity between the fused feature vector and the reference feature vector of each fatigue state is obtained using the tongue image assessment model. The fatigue state with the smallest proximity is selected as the fatigue level assessment. An intervention strategy library is set up, which stores descriptions and suggestions corresponding to different fatigue states. The corresponding descriptions and suggestions are retrieved according to the output fatigue level assessment and combined into an initial assessment report for the corresponding fused feature vector.
7. The physical condition and fatigue state assessment system based on tongue image recognition according to claim 6, characterized in that, The process of obtaining the error coefficients includes: The corrected fatigue level refers to the fatigue state that the relevant personnel at the institution correct for the user based on the initial assessment report and its tongue image data and liver function index data, and set a benchmark value for different fatigue states. Tongue image data, liver function index data, and corresponding corrected fatigue levels and assessed fatigue levels at the same time stamp are included in the same sample. Several of the latest samples at each time stamp are then obtained. The error coefficient... ; Where b = 1, 2, ..., h, h is the number of samples obtained, PR b PG is the baseline value corresponding to the corrected fatigue level in the b-th sample. b Let be the proximity of the fatigue level assessment in the b-th sample.
8. The physical condition and fatigue state assessment system based on tongue image recognition according to claim 7, characterized in that, The process of obtaining the optimized fusion feature vector and its final evaluation report includes: The initial weights for the abnormal feature vector, tongue image feature vector, and liver function feature vector are determined based on the obtained error coefficients. Perform optimizations separately to obtain the latest optimized weight values. ; ， ， ； in, To pre-set the adjustment factor, each element in the different feature vectors obtained subsequently is multiplied by its latest optimized weight value and then combined sequentially to form an optimized fusion feature vector. The optimized fusion feature vector is input into the tongue image assessment model, and the corresponding assessment fatigue level is output. The final assessment report is generated by combining the intervention strategy library and fed back to the user.
9. A method for assessing physical condition and fatigue status based on tongue image recognition, characterized in that, Includes the following steps: Acquire tongue image data and liver function index data; A tongue image feature vector is generated based on the tongue image data, and a liver function feature vector is generated based on the liver function index data. The difference degree at each point on the tongue surface is obtained based on the tongue image data, abnormal areas on the tongue surface are identified based on the difference degree, and abnormal feature vectors are generated based on the geometric features of the abnormal areas. A fusion feature vector is generated based on the abnormal feature vector, the tongue image feature vector, and the liver function feature vector; The fused feature vector is input into a preset tongue image assessment model to output the fatigue level assessment and generate an initial assessment report. The initial assessment report is sent to the institution and the corrected fatigue level is received from the institution. The error coefficient is obtained based on the corrected fatigue level and the assessed fatigue level. The weight of the fused feature vector is optimized based on the error coefficient to obtain an optimized fused feature vector, which is then input into a preset tongue image assessment model to generate a final assessment report.
10. A computer storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed, they implement the physical condition and fatigue status assessment system based on tongue image recognition as described in any one of claims 1-8.