Traditional Chinese medicine tongue diagnosis image three-dimensional reconstruction and pathological feature quantitative analysis method

Abstract

The invention provides a traditional Chinese medicine tongue diagnosis image three-dimensional reconstruction and pathological feature quantitative analysis method. A thermal infrared imager, near-infrared multispectral imaging equipment, a high-resolution RGB camera, a structured light three-dimensional scanner and a wettability sensor are used for synchronously collecting tongue body infrared thermal images, multispectral reflection, color images, three-dimensional point cloud and wettability data; and after statistical filtering and denoising are performed on the three-dimensional point cloud, reconstruction is performed by using a Poisson algorithm, and three-dimensional modeling is completed in combination with UV mapping. Features of dynamic microcirculation, geometry, color texture and the like are extracted through Fourier transform, a gray-level co-occurrence matrix and the like, and multi-modal feature adaptive fusion is realized by means of dynamic weight and an attention mechanism. And orthogonalizing the fused features, inputting the orthogonalized fused features into a deep learning model to assess disease risks, and combining clustering analysis to realize health state classification and generate personalized suggestions. According to the method, multi-modal data and dynamic feature tracking are creatively fused, the pathological feature extraction precision is remarkably improved, and digital and intelligent development of traditional Chinese medicine tongue diagnosis is promoted.

Description

Technical Field

[0001] This invention relates to the field of digital diagnostic technology in traditional Chinese medicine, specifically to a method for three-dimensional reconstruction and quantitative analysis of pathological features of tongue diagnosis images in traditional Chinese medicine based on multimodal image processing. Background Technology

[0002] Tongue diagnosis in Traditional Chinese Medicine (TCM), an important component of the four diagnostic methods of "inspection, auscultation and olfaction, inquiry, and palpation," utilizes the tongue's color, shape, texture, and moisture to aid in assessing the body's physiological and pathological states. It plays a crucial role in disease diagnosis, treatment evaluation, and health management. However, traditional tongue diagnosis relies heavily on the subjective experience of TCM practitioners and is influenced by individual cognitive differences and environmental factors, resulting in limitations such as strong subjectivity in diagnostic results, a lack of quantitative standards, and difficulty in dynamic monitoring.

[0003] With the development of computer vision, sensor technology, and artificial intelligence, digital tongue diagnosis technology has gradually become a research hotspot. Existing technologies mostly focus on two-dimensional tongue diagnosis image analysis, extracting features such as tongue color and texture through RGB image acquisition and processing to achieve preliminary diagnosis and classification of diseases. However, two-dimensional images can only reflect planar information of the tongue surface and cannot obtain the three-dimensional morphological structure of the tongue (such as tongue thickness and the degree of curvature of the tongue edges), while these geometric features also contain important pathological information. At the same time, the dynamic physiological characteristics of the tongue (such as changes in microcirculation and fluctuations in moisture) are closely related to disease development, but traditional two-dimensional analysis methods struggle to effectively capture and quantify them.

[0004] In the area of ​​multimodal data fusion, some studies have attempted to combine multispectral imaging to obtain blood oxygenation information of the tongue or analyze the temperature distribution of the tongue surface through infrared thermography. However, existing fusion methods often employ simple data splicing or superficial feature fusion, failing to fully explore the intrinsic connections between different modalities. This results in incomplete extraction of pathological features and limited accuracy and stability of diagnostic models. Furthermore, existing tongue diagnosis analysis techniques are mostly static, single-test methods, lacking the tracking and analysis of dynamic changes in tongue diagnosis features, making it difficult to meet the needs of disease progression monitoring and personalized health management. Therefore, there is an urgent need for a TCM tongue diagnosis technique that can achieve high-precision three-dimensional reconstruction of the tongue, comprehensively quantify multimodal pathological features, and support dynamic analysis, in order to promote the standardization and intelligent development of TCM diagnosis. Summary of the Invention

[0005] This invention aims to provide a method for three-dimensional reconstruction and quantitative analysis of pathological features of tongue diagnosis images in traditional Chinese medicine. Through multimodal data fusion, high-precision three-dimensional reconstruction and dynamic feature analysis, it achieves comprehensive quantification of tongue diagnosis features and intelligent assessment of disease risks, improves the objectivity, accuracy and traceability of traditional Chinese medicine diagnosis, and promotes the standardization and modernization of traditional Chinese medicine tongue diagnosis.

[0006] The method for three-dimensional reconstruction and quantitative analysis of pathological features of TCM tongue diagnosis images according to the present invention includes the following steps: The multimodal data acquisition: Synchronous data acquisition: An integrated multimodal sensor system is employed, simultaneously activating an infrared thermal imager, a near-infrared multispectral imaging device, a high-resolution RGB camera, a structured light 3D scanner, and a humidity sensor. Each device achieves synchronous data acquisition via hardware trigger signals, ensuring a synchronization time error of no more than ±10ms to guarantee the spatiotemporal consistency of the multimodal data.

[0007] The data acquisition parameter settings are as follows: Infrared thermal imager: Employs an uncooled focal plane array detector with a resolution of 640×480 pixels, a data acquisition frequency of 25 frames / second, and an operating wavelength of 8-14μm. The ambient temperature is controlled at 22±1℃, and the relative humidity at 50±5% to minimize the impact of ambient temperature on tongue surface temperature measurement. The distance between the device and the tongue is maintained at 15-20cm to ensure the field of view covers the entire tongue.

[0008] Near-infrared multispectral equipment: Three characteristic wavelengths were selected: 760nm (hemoglobin absorption peak), 850nm (good tissue penetration), and 940nm (water absorption peak). Each wavelength was sampled three times to improve signal stability, with each sampling exposure time being 50ms. The blood oxygen saturation and blood flow velocity of the tongue can be calculated using multispectral reflectance data.

[0009] RGB camera: Equipped with a 50-megapixel sensor, an aperture of f / 2.8 for greater depth of field, and a fixed focal length of 35mm to reduce distortion. It uses a standard D65 light source (color temperature 5500K, color rendering index Ra≥95) for illumination to ensure accurate color reproduction. The acquisition distance is 10-15cm to obtain tongue images with sufficient resolution.

[0010] Structured light 3D scanner: Projects a sinusoidal striped structured light pattern with a stripe period of 8 pixels and a phase shift step of π / 2. Phase information is calculated using a four-step phase shift method, combined with triangulation principles to reconstruct the 3D point cloud data of the tongue. The scanning range covers the entire tongue, achieving a point cloud density of over 1000 points / cm².

[0011] Moisture sensor: A contact sensor made of high-molecular polymer material, it makes gentle contact with the tongue surface to avoid damage. It collects data at a frequency of 1 time per second, recording the moisture content and evaporation rate of the tongue surface in real time. The measurement range is 0-100%RH, with an accuracy of ±3%RH.

[0012] Data integration and preprocessing: The collected infrared thermal imaging data, near-infrared multispectral reflectance data, RGB color images, 3D point cloud data, and humidity data are aligned and synchronized according to timestamps. Preliminary preprocessing is performed on each modality of data, including non-uniformity correction of infrared images, color standardization of RGB images, and downsampling of point cloud data, to generate a multimodal dataset with a unified time reference.

[0013] The three-dimensional reconstruction: Point cloud denoising (S21): Statistical filtering is used to denoise the original point cloud data. The number of neighboring points K is set to 20, and the Euclidean distance between each point and its K neighbors is calculated. The distance distribution of all points is statistically analyzed, and the mean μ and standard deviation σ of the distances are calculated. A threshold T = μ + 2σ is set. Points with distances exceeding the threshold T are considered noise points and removed to retain the true point cloud of the tongue surface. To improve the denoising effect, the number of neighboring points K can be dynamically adjusted within the range of 15-25, and the threshold T can be adaptively set within the range of μ + (1.5-2.5)σ according to the point cloud density.

[0014] Surface Reconstruction (S22): The Poisson reconstruction algorithm is used to reconstruct the 3D surface of the denoised point cloud data. The depth parameter of the algorithm is set to 8, and a continuous 3D model of the tongue surface is generated by solving an implicit function. The reconstruction parameters are adjusted to control the level of detail and the number of triangular faces in the model, keeping the number of triangular faces between 100,000 and 150,000 to ensure model accuracy while avoiding waste of computational resources. The reconstructed surface model is smoothed using the Laplacian smoothing algorithm with 5-10 iterations to eliminate surface jaggedness generated during the reconstruction process.

[0015] Texture Mapping (S23): The RGB color image is mapped to UV coordinates, dividing it into 100×100 grid cells, each corresponding to a region on the 3D model. For each vertex on the 3D model, its projected coordinates on the image plane are calculated, and the center of the nearest grid cell is found. The texture pixel value of that grid cell is then assigned to the vertex of the 3D model. A bilinear interpolation algorithm is used to smooth the texture boundaries, avoiding noticeable jagged edges after texture mapping. The final result is a complete 3D tongue model with realistic color texture.

[0016] Quantitative analysis of the pathological features: Dynamic Microcirculation Feature Extraction (S31): Infrared Thermographic Time Series Analysis. Time series analysis is performed on infrared thermographic data, using a sliding window of 10 frames to move across the time series. Within each window, Fourier transform is applied to convert the time-domain signal to the frequency-domain signal, extracting features such as temperature fluctuation amplitude, inter-regional temperature difference, and frequency components of temperature fluctuations. To reduce spectral leakage, a Hanning window function is used for windowing, with a window length of 8-12 frames. Multispectral Data Processing. Multispectral reflectance data is normalized, normalizing the data for each wavelength to the [0,1] interval. Using the Lambert-Beer law, combined with the known absorption and scattering coefficients of tissue components such as hemoglobin and water, the blood oxygen saturation and blood flow velocity of the tongue are calculated. Feature Integration and Dimensionality Reduction. The extracted temperature features and the calculated blood oxygen saturation and blood flow velocity features are integrated to form a high-dimensional feature vector. Principal Component Analysis (PCA) is used to reduce the dimensionality of the feature vector, retaining 95% of the variance information to reduce feature redundancy and improve the efficiency of subsequent analysis.

[0017] The geometric feature extraction (S32): Curvature Calculation: The curvature is calculated for each point in the complete 3D model of the tongue, using a combination of mean curvature and Gaussian curvature. The specific calculation formula is: Mean Curvature Gaussian curvature ,in and These are the two principal curvatures at this point. Curvature analysis can identify features such as concave and convex areas and cracks on the tongue surface.

[0018] Region Division: Based on the anatomical characteristics of the tongue, and using the connection between the root of the tongue and the oral cavity, and the tip of the tongue as reference points, the 3D model is divided into three regions: the tip of the tongue (occupying the front 1 / 3 of the tongue), the middle of the tongue (the middle 1 / 3 of the tongue), and the root of the tongue (the back 1 / 3 of the tongue). Simultaneously, based on the morphology of the tongue's edges, the tongue's lateral region is further subdivided.

[0019] Parameter calculation: Calculate the volume (by summing the volumes of triangular facets), surface area (by accumulating the areas of triangular facets), height difference (the difference in z-coordinates between the highest and lowest points), and surface area to volume ratio for each region to form a complete three-dimensional morphological feature vector.

[0020] The color and texture feature extraction (S33): Color Feature Extraction: Using the RGB image, the RGB color values ​​are converted to the HSV color space using the RGB-to-HSV color space conversion formula. The hue (H), saturation (S), and brightness (V) features of the tongue color are then extracted. To reduce the influence of lighting, the HSV color space is normalized.

[0021] Texture feature extraction: A texture analysis algorithm based on the Gray-Level Co-occurrence Matrix (GLCM) is used, with gray levels set to 8-16, distance parameter d to 1-3 pixels, and angle parameter θ taken in four directions: 0°, 45°, 90°, and 135°. Texture feature parameters such as energy, entropy, contrast, and correlation are calculated to describe the texture characteristics of the tongue surface, including its smoothness and roughness.

[0022] Moisture feature extraction: The mean moisture content and evaporation rate of the tongue surface are extracted from the moisture sensor data, and combined with color and texture features to form a tongue moisture feature vector.

[0023] The multimodal feature fusion: Feature splicing: The color features, texture features, moisture features, dynamic microcirculation feature vectors, and morphology-microcirculation integrated feature matrix of the tongue are spliced ​​together to form an initial joint feature vector.

[0024] Feature space mapping: through a linear transformation matrix Mapping each modality feature distribution to a unified embedding space, the linear transformation formula is as follows: ,in The initial feature vector, The mapped feature vector, The weight matrix is ​​a learnable matrix. Let be the bias vector. The optimal mapping parameters are learned by minimizing the difference between the mapped feature distribution and the target distribution.

[0025] Dynamic weight fusion: A dynamic weight mechanism is introduced to construct an attention mechanism network. The network structure includes an input layer, hidden layers, and an output layer. The number of neurons in the hidden layer is 64-128, and the activation function is ReLU or LeakyReLU. Based on the importance of different modalities in different pathological states, the weights of each modality are automatically learned through the attention mechanism. The features of different modalities are weighted and fused to form a unified feature encoding for high-dimensional feature representation, ultimately generating a multimodal pathological feature vector.

[0026] The dynamic feature tracking and analysis (S35): Time series model construction (S351): An autoregressive integrated moving average model (ARIMA(p,d,q)) is used to model continuously collected multimodal pathological feature vectors. Non-stationarity of the feature vectors is eliminated by setting the difference order d=1; the autoregressive order p is determined based on the partial autocorrelation function (PACF) plot, and p=k is taken when PACF is truncated after lag k; the moving average order q is determined based on the autocorrelation function (ACF) plot, and q=k is taken when ACF is truncated after lag k. Model parameters are optimized using maximum likelihood estimation.

[0027] Kalman filter noise reduction (S352): The noise in the feature vector is filtered and optimized iteratively through the following steps: State prediction: ,in Here is the state transition matrix. This is the optimal estimate from the previous moment. To control the input matrix, For control input; Covariance prediction: ,in Let be the covariance matrix of the previous time step. For process noise covariance; Kalman gain calculation: ,in For the observation matrix, To measure the noise covariance; Status Update: ,in The observed value at the current moment; Covariance update: .

[0028] State transition matrix Set as a diagonal matrix, with diagonal elements dynamically adjusted based on the rate of change of the eigenvectors over the first 5-10 time steps, with adjustment coefficients ranging from 0.8 to 1.2; process noise covariance. and measurement noise covariance Online estimation is performed using the EM algorithm.

[0029] Dynamic Time Warping Analysis (S353): Utilizes the DTW algorithm to compare eigenvectors at different time points and defines an eigenvector sequence. and Construct the cost matrix ,in , The distance is Euclidean. A Sakoe-Chiba band constraint is applied, with the window width set to 10% of the feature vector length to limit the search range of the curved path. The optimal path is found through backtracking, and the cumulative distance is calculated as a similarity measure. When the DTW distance exceeds a threshold (set to 1.5 times the average DTW distance of historical data), it is considered a significant feature change.

[0030] Trend Feature Extraction (S354): Extracting slope features from the DTW-aligned feature sequence: ; Calculate the characteristics of the rate of change: Multinomial fitting (quadratic polynomial) is used to capture the trend of feature changes, and the fitting coefficients are used as trend features. Based on the trend features, a Markov chain model is constructed to predict the probability of feature changes in the next three time steps.

[0031] The disease risk assessment and health status classification: Feature preprocessing: The multimodal pathological feature vectors are orthogonalized using the Gram-Schmidt orthogonalization method to eliminate the correlation between features and improve the model training efficiency.

[0032] Intelligent Diagnostic Model: The preprocessed feature vectors are input into an intelligent diagnostic model trained based on a hybrid convolutional neural network (CNN) and recurrent neural network (RNN). The CNN part extracts spatial features and contains three convolutional layers with kernel sizes of 3×3, 5×5, and 7×7, respectively. Max pooling is used in the pooling layers. The RNN part uses a 256-unit LSTM network to process the temporal information of the features. Model training parameters: The Adam optimizer is used, with a learning rate of 0.0005-0.0015, a batch size of 16-32, and 80-120 epochs.

[0033] Disease Risk Score: Based on the diagnostic probability of each disease output by the diagnostic model, and combined with the degree of abnormality of each indicator in the pathological feature vector, a weighted summation method is used to calculate the disease risk score. The score ranges from 0 to 100, with a higher score indicating a higher disease risk.

[0034] Health Status Classification: Based on the risk score results and the pathological characteristics of different areas of the tongue, a health score (0-10 points) is calculated for each tongue region. The K-means clustering algorithm is used to classify the patient's health status into five levels: healthy, sub-healthy, mildly abnormal, moderately abnormal, and severely abnormal. Personalized health recommendations are generated based on the classification results, including dietary adjustments, lifestyle modifications, and traditional Chinese medicine treatment plans.

[0035] S5. Verification of Diagnostic Results (Optional Step) The intelligent diagnostic results are compared and verified with classic TCM tongue diagnosis atlases such as "Atlas of Tongue Diagnosis in Traditional Chinese Medicine" and "Illustrated Guide to Tongue Diagnosis." A template matching algorithm is used to calculate the similarity between the patient's tongue image and standard tongue image samples in the atlases to verify the accuracy of the diagnostic results. If the similarity is lower than a set threshold (e.g., 80%), an expert review process is triggered, and a senior TCM physician conducts a manual review.

[0036] The above technical solution can bring about the following three significant technical effects: 1. Multimodal Deep Fusion, Overcoming Information Limitations: Traditional tongue diagnosis relies heavily on two-dimensional RGB images, making it difficult to obtain three-dimensional morphological and physiological dynamic information about the tongue. This solution comprehensively covers tongue characteristics such as color, texture, temperature, geometric shape, and moisture by simultaneously acquiring multimodal data including infrared thermography, near-infrared spectroscopy, and three-dimensional point clouds. For example, infrared thermography data can accurately reflect tongue surface temperature fluctuations, closely related to TCM's differentiation of cold and heat patterns; near-infrared spectroscopy can quantitatively analyze blood oxygen saturation and blood flow velocity, revealing the state of Qi and blood circulation. Through dynamic weighting and attention mechanisms, multimodal feature adaptive fusion is achieved, improving information completeness by over 60% compared to single-modal analysis, providing a more comprehensive and accurate basis for TCM diagnosis.

[0037] 2. High-precision 3D reconstruction for quantifying pathological features: Addressing the issue that traditional 2D analysis cannot accurately describe the 3D structure of the tongue, this solution utilizes statistical filtering combined with the Poisson reconstruction algorithm to process 3D point cloud data, controlling the tongue surface reconstruction error to within 0.5mm. By calculating geometric parameters such as average curvature and Gaussian curvature, and dividing the tongue into regions such as the tip, middle, and root, and quantifying indicators such as volume and surface area, it can accurately capture pathological morphological features such as tongue size, teeth marks, and cracks. Taking scalloped tongue as an example, traditional methods can only provide a qualitative description; this solution can accurately measure parameters such as the depth and area of ​​teeth marks through a 3D model, improving the accuracy of quantitative analysis of tongue pathological features by more than 40%, and promoting the transformation of TCM tongue diagnosis from subjective experience-based judgment to objective quantitative diagnosis.

[0038] 3. Dynamic Feature Intelligent Tracking, Empowering Precision Medicine: Existing tongue diagnosis techniques are mostly static, single-shot tests, making it difficult to capture the dynamic changes in tongue features during disease development. This solution introduces the ARIMA time series model, Kalman filtering, and dynamic time warping algorithm to track and analyze continuously acquired multimodal pathological feature vectors, enabling real-time monitoring of the evolution trends of tongue microcirculation, moisture, and other features. For example, in the follow-up monitoring of chronic disease patients, abnormal changes in tongue features can be detected 3-5 testing cycles in advance. Combined with intelligent diagnostic models, disease risk can be predicted with an accuracy rate of over 85%. Personalized health recommendations generated through dynamic feature analysis can provide targeted guidance for patients to adjust their lifestyles or undergo intervention treatment, providing strong support for chronic disease management and personalized medicine. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the system architecture and data acquisition equipment of the present invention. Figure 2 This is a flowchart of the three-dimensional reconstruction process of the present invention. Detailed Implementation The following will refer to the appendices in the embodiments of the present invention. Figure 1 and Figure 2The technical solutions in the embodiments of the present invention are clearly and completely described herein. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0040] This invention can be implemented through an integrated intelligent tongue diagnosis system, which consists of two main parts: hardware devices and software algorithms. The specific implementation process is detailed below with reference to the key technical points: I. Hardware Equipment Deployment and Data Acquisition Multimodal sensor array: The system integrates a high-precision infrared thermal imager (640×480 resolution, 25 frames / second), a near-infrared multispectral imaging device (760nm, 850nm, 940nm three-band), a 50-megapixel RGB camera (f / 2.8 aperture, 35mm focal length), a structured light 3D scanner (8-pixel stripe period), and a contact humidity sensor. All devices achieve synchronous data acquisition through a hardware trigger module, with time errors controlled within ±10ms, ensuring spatiotemporal consistency of multimodal data.

[0041] Standardized acquisition environment: The device is placed in a customized acquisition box with a built-in D65 standard light source (color temperature 5500K, color rendering index ≥95). During acquisition, the ambient temperature is controlled at 22±1℃ and the humidity at 50±5%. The patient's tongue is 15cm away from the device, and the patient completes the natural tongue extension action through voice prompts, reducing human interference.

[0042] Synchronous Data Acquisition: After the system is started with one click, the infrared thermal imager captures the tongue surface temperature distribution in real time, the near-infrared device acquires blood oxygen-related data, the RGB camera captures color images, the structured light scanner acquires 3D point clouds, and the humidity sensor records the tongue surface humidity. All data is synchronously stored to the solid-state drive according to the timestamp, forming the original multimodal dataset.

[0043] II. Software Algorithm Processing Flow 3D Reconstruction Module Point cloud preprocessing: Apply a statistical filtering algorithm to the 3D point cloud data, set the neighborhood points K=20, calculate the Euclidean distance, and remove noise points with a threshold T=μ+2σ (μ is the mean and σ is the standard deviation) to retain the true surface information of the tongue.

[0044] Surface generation and texture mapping: The Poisson reconstruction algorithm (depth parameter 8) is used to generate a continuous 3D model, with the number of triangular facets controlled at around 120,000; the RGB image is divided into 100×100 grids, and texture mapping is completed by the nearest neighbor interpolation method to construct a high-precision textured 3D tongue model.

[0045] Pathological Feature Quantitative Analysis Module Dynamic microcirculation analysis: Temperature fluctuation features were extracted from infrared thermographic sequences using a 10-frame sliding window combined with Fourier transform; after normalizing the multispectral data, blood oxygen saturation and blood flow velocity were calculated based on Lambert-Beer's law, and PCA was used to reduce the dimension while retaining 95% of the variance information to form a microcirculation feature vector.

[0046] Geometric and texture feature extraction: Calculate the average curvature and Gaussian curvature of each point in the 3D model, divide the tongue tip, middle and root regions, and measure parameters such as volume and surface area respectively; convert the RGB image to HSV space to extract color features, obtain texture parameters through gray-level co-occurrence matrix (16 gray levels, θ=0° / 45° / 90° / 135°), and combine with humidity data to form a complete feature set.

[0047] Multimodal fusion and dynamic tracking: The above features are concatenated and mapped to a unified space through linear transformation. A ReLU activation network with 128 neurons is used to learn modal weights to achieve adaptive fusion. An ARIMA model is used for modeling, Kalman filtering is used for noise reduction, and the DTW algorithm is used to compare time-series features and extract trend parameters such as slope and rate of change to construct a dynamic pathological feature library.

[0048] Intelligent Diagnosis and Health Management Module Model training and inference: Based on a CNN-RNN hybrid network (3 convolutional layers + 256-unit LSTM), the model is trained for 100 rounds using the Adam optimizer (learning rate 0.001, batch size=32). The input is the orthogonalized multimodal feature vector, and the output is the disease diagnosis probability.

[0049] Risk assessment and recommendation generation: The disease risk score (0-100 points) is calculated by weighting the diagnostic probability and the degree of abnormality of the features. The health level is divided by K-means clustering (K=5). The results are compared and verified with the "Traditional Chinese Medicine Tongue Diagnosis Atlas". If the similarity is >85%, personalized health recommendations are automatically generated, covering diet (e.g., "For those with damp-heat constitution, it is recommended to eat barley and red bean porridge"), work and rest, and TCM conditioning plan.

[0050] III. Implementation Results and Advantages Improved diagnostic accuracy: In a clinical trial at a top-tier hospital, the system detected 92.3% accuracy in identifying pathological features of the tongue in 300 patients (including 8 common syndromes such as spleen deficiency and damp-heat), which is 21% higher than traditional two-dimensional analysis. In particular, the error in measuring the depth of the teeth marks on the tongue is only 0.3mm, which is significantly better than manual judgment.

[0051] Advantages of dynamic monitoring: In a study of 20 diabetic patients who were followed up for 3 consecutive months, the system detected abnormal fluctuations in tongue temperature in 4 patients in advance through dynamic feature analysis. Combined with risk scores, the system could provide early warning of disease progression and provide a basis for intervention 2-3 weeks earlier than conventional testing.

[0052] Clinical application value: The system takes only 12 seconds to perform a single test and generates an electronic report that includes 3D model visualization, quantitative indicators, and health recommendations. It supports remote transmission and cloud storage and has been successfully applied to remote TCM consultations in primary healthcare units, effectively alleviating the problem of insufficient high-quality TCM resources.

[0053] Through the coordinated implementation of the aforementioned hardware and software, this invention achieves a breakthrough in TCM tongue diagnosis, moving from subjective experience to objective quantification and from static testing to dynamic monitoring, significantly improving the standardization and intelligence level of TCM diagnosis.

[0054] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A traditional Chinese medicine tongue diagnosis image three-dimensional reconstruction and pathological characteristic quantification analysis method, characterized in that, Comprising the following steps: S1. Multi-modal data acquisition: Using a high-precision infrared thermal imager, set the resolution to 640x480 pixels, and the acquisition frequency to 25 frames / s, under the conditions of ambient temperature 22±1℃ and relative humidity 50±5%, at a distance of 15-20 cm from the tongue body, collect infrared thermal image data of the tongue surface for subsequent analysis of tongue surface temperature fluctuation characteristics; Using a near-infrared multispectral imaging device, select three wavelengths of 760 nm, 850 nm, and 940 nm, collect data at each wavelength three times, and each time the exposure time is 50 ms, to obtain near-infrared multispectral reflectance data of the tongue body, so as to calculate the blood oxygen saturation and blood flow velocity of the tongue body; Using a high-resolution RGB camera with a 50 million-pixel sensor, set the aperture to f / 2.8 and the focal length to 35 mm, under standard D65 light source conditions, at a distance of 10-15 cm from the tongue body, collect RGB color images for extracting tongue color, texture, and other features; Using a structured light three-dimensional scanner, project a sinusoidal fringe structured light pattern with a fringe period of 8 pixels and a phase shift step of π / 2, and scan the entire tongue body to obtain three-dimensional point cloud data, providing basic data for three-dimensional reconstruction; Using a contact-type humidity sensor, the sensor probe is made of high-molecular polymer, and is gently in contact with the tongue surface, with a collection frequency of 1 / s, recording the moisture and evaporation rate of the tongue surface to obtain moisture data; Align and integrate the above collected infrared thermal image data, near-infrared multispectral reflectance data, RGB color images, three-dimensional point cloud data, and moisture data according to the time stamp to generate a multi-modal data set; S2. Three-dimensional reconstruction: S21. Using statistical filtering method to remove noise points from the collected three-dimensional point cloud data: set the number of neighborhood points K = 20, calculate the Euclidean distance of each point and its K neighborhood points; Calculate the distance distribution of all points, calculate the mean μ and standard deviation σ of the distance, and set the threshold T = μ + 2σ; Remove the points whose distance exceeds the threshold T as noise points; S22. Using Poisson reconstruction algorithm to reconstruct the three-dimensional surface of the denoised point cloud data: set the depth parameter of the Poisson reconstruction algorithm to 8, solve the implicit function to generate a continuous three-dimensional model of the tongue surface, and control the number of triangular facets of the model to 100,000-150,000; S23. Using the collected RGB color images, map the texture information of the tongue surface to the continuous three-dimensional model based on the nearest neighbor interpolation UV mapping method: first map the UV coordinates of the RGB color image, and divide the image into 100x100 grid cells; For each vertex on the three-dimensional model, find the nearest grid cell center by calculating its projection coordinates on the image plane, and assign the texture pixel value of the grid cell to the three-dimensional model vertex to obtain a complete three-dimensional model of the tongue body; S3. Pathological feature quantitative analysis: S31. Dynamic microcirculation feature analysis: Time series analysis is performed on the infrared thermal image data, a sliding window with a length of 10 frames is used, and in each window, Fourier transform is used to extract features such as temperature fluctuation amplitude, inter-regional temperature difference and temperature fluctuation frequency component, and the frequency resolution is set to 0.1 Hz; the multispectral reflection data is normalized, the data of each wavelength is normalized to the interval [0, 1], the Lambert-Beer law is used, combined with the known tissue absorption coefficient and scattering coefficient, the blood oxygen saturation and blood flow velocity are calculated; the above extracted temperature features and calculated blood oxygen saturation and blood flow velocity features are integrated, and the dimensionality is reduced by principal component analysis (PCA), retaining 95% of the variance information, to obtain the dynamic microcirculation feature vector; S32. Geometric feature extraction: Calculate the curvature of each point in the complete tongue three-dimensional model, adopt the combination of average curvature and Gaussian curvature, the specific calculation formula is: average curvature , Gaussian curvature , wherein and are the two principal curvatures of the point; according to the anatomical features of the tongue, take the junction of the tongue root and the oral cavity and the tongue tip endpoint as the reference points, divide the three-dimensional model into three regions: tongue tip (occupying the front 1 / 3 region of the tongue), tongue middle (the middle 1 / 3 region), and tongue root (the rear 1 / 3 region); calculate the volume (calculated by the sum of triangular facet volume), surface area (cumulative triangular facet area), height difference (z coordinate difference value of the highest point and the lowest point) and other parameters of each region respectively, and obtain the complete three-dimensional morphological features; S33. Color and texture feature extraction: Through the RGB image, the color space conversion formula of RGB to HSV is used to extract the tongue color feature; the texture analysis algorithm of gray level co-occurrence matrix is used, the gray level number is set to 16, the texture feature parameters in 0°, 45°, 90° and 135° directions are calculated, including energy, entropy, contrast and correlation, and the texture feature of the tongue surface is extracted; at the same time, the average moisture of the tongue surface and the evaporation rate in the moisture data are extracted, and the tongue moisture feature is obtained; S34. Multimodal feature fusion: The color feature, texture feature, wetness feature, dynamic microcirculation feature vector and morphology-microcirculation comprehensive feature matrix of the tongue body are spliced to form an initial joint feature vector; a linear transformation matrix is used to map the feature vector to a unified embedding space Each modal feature distribution is mapped to a unified embedding space, and the linear transformation formula is: Wherein is the initial feature vector, is the mapped feature vector, is a learnable weight matrix, is a bias vector; a dynamic weight mechanism is introduced to construct an attention mechanism network, the network structure includes an input layer, a hidden layer and an output layer, the number of hidden layer neurons is 128, and the activation function adopts a ReLU function; according to the importance of different modal data under different pathological states, the weights of each modal are automatically learned through the attention mechanism, the different modal features are weighted and fused to form a unified feature code of high-dimensional feature representation, and finally a multimodal pathological feature vector is generated; S35. Dynamic feature tracking and analysis: S351. Time series model construction: Autoregressive integrated moving average model (ARIMA(p, d, q)) is used to model the continuously collected multi-modal pathological feature vector, and the non-stationarity of the feature vector is eliminated by difference order d=1; the autoregressive order p is determined according to the partial autocorrelation function (PACF) graph, when PACF is truncated after lag k, p=k is taken; the moving average order q is determined according to the autocorrelation function (ACF) graph, when ACF is truncated after lag k, q=k is taken; the model parameters are optimized by maximum likelihood estimation method; S352. Kalman filter denoising: filtering the noise in the eigenvector, state prediction: ; covariance prediction: ; Kalman gain calculation: ; state update: ; covariance update: ; state transition matrix is set to a diagonal matrix, and the diagonal elements are adaptively adjusted according to the historical change rule of the eigenvector; process noise covariance and measurement noise covariance estimated online by EM algorithm; S353. Dynamic time warping analysis: compare feature vectors at different time points using DTW algorithm, define feature vector sequence and ; construct cost matrix where , is the Euclidean distance; use Sakoe-Chiba band constraint, window width set to 10% of feature vector length, limit search range of curved path; find optimal path by backtracking, calculate cumulative distance as similarity measure; when DTW distance exceeds threshold (threshold set to 1.5 times average DTW distance of historical data), determine as significant change in features; S354. Trend feature extraction: Extract the slope feature from the DTW aligned feature sequence: ; Calculate the rate of change feature: ; Capture the trend of feature change using polynomial fitting (quadratic polynomial), and the fitting coefficient as the trend feature; Based on the trend feature, a Markov chain model is constructed to predict the feature change probability of the next 3 time steps; S4. Disease risk assessment and health status classification: The multi-modal pathological feature vector is orthogonalized and input into an intelligent diagnosis model trained based on a convolutional neural network-recurrent neural network hybrid model, and the diagnosis probability of each disease is output; according to the diagnosis probability and the abnormality degree of the indicators in the pathological feature vector, the disease risk score is calculated by weighted summation; according to the risk score result, combined with the pathological features of different regions of the tongue, the health score of each tongue region is calculated, and the K-means clustering algorithm is used to classify the health status of the patient, and then personalized health advice is generated.

2. The traditional Chinese medicine tongue image three-dimensional reconstruction and pathological feature quantification analysis method according to claim 1, characterized in that, In the multi-modal data acquisition of S1, the time error of each device triggered synchronously does not exceed ±10 ms, to ensure the spatio-temporal consistency of the collected multi-modal data.

3. The traditional Chinese medicine tongue image three-dimensional reconstruction and pathological feature quantification analysis method according to claim 1, characterized in that, In the statistical filtering method for removing noise points in S21, the value range of the number of neighborhood points K is 15-25, and the threshold T is set to μ+(1.5-2.5)σ, where μ is the distance mean and σ is the distance standard deviation.

4. The traditional Chinese medicine tongue image three-dimensional reconstruction and pathological feature quantification analysis method according to claim 1, characterized in that, In the step S31, the Fourier transform is used to extract the temperature fluctuation characteristics, and a Hanning window function is used for window processing, and the window length is 8-12 frames, so as to reduce the spectral leakage.

5. The traditional Chinese medicine tongue image three-dimensional reconstruction and pathological feature quantification analysis method according to claim 1, characterized in that, In the step S33, the gray level is 8-16 levels, the distance parameter d is 1-3 pixels, and the angle parameter θ is 0°, 45°, 90° and 135°.

6. The traditional Chinese medicine tongue image three-dimensional reconstruction and pathological feature quantification analysis method according to claim 1, characterized in that, In the step S34, the number of hidden layer neurons of the attention mechanism network is 64-128, and the activation function is ReLU or LeakyReLU function.

7. The traditional Chinese medicine tongue image three-dimensional reconstruction and pathological feature quantification analysis method according to claim 1, characterized in that, The state transition matrix of the Kalman filter in the S352 The diagonal elements are dynamically adjusted according to the eigenvector change rate of the previous 5-10 time steps, and the adjustment coefficient is 0.8-1.

2. 8.The method of claim 1, wherein, In the step S4, the Adam optimizer is used for training the intelligent diagnosis model, the learning rate is 0.0005-0.0015, the batch size is 16-32, and the training round is 80-120 epochs. 9.The method of claim 1, wherein, The method further comprises the step S5 of comparing and verifying the diagnosis result with a traditional Chinese medicine classic tongue diagnosis atlas, and the atlas comprises standard tongue sample in the “Chinese Medicine Tongue Diagnosis Atlas” and “Color Picture Tongue Identification Guide”.

Images