A method for analyzing sublingual and tongue surface images in the four diagnostic methods of traditional Chinese medicine based on deep learning

Abstract

The present invention discloses a method for analyzing tongue bottom and tongue surface images in the four diagnostic methods of traditional Chinese medicine based on deep learning, including (1) an image acquisition module: used for users to upload images of the tongue bottom and tongue surface. (2) A deep learning analysis unit module: This module contains a pre-trained neural network for analyzing the uploaded tongue surface and tongue bottom images. The network identifies and classifies health indicators related to the four diagnostic methods of traditional Chinese medicine by extracting key features of the images. (3) A feature database module. (4) A diagnostic result analysis and evaluation module. (5) A result display and report module: Displays the analysis and diagnostic results in a user-friendly visual format, including a health assessment report and possible medical suggestions, to help users or medical professionals understand and use.

Description

Technical Field

[0001] The present invention relates to the field of image processing in deep learning and image recognition in computer vision; specifically, it relates to a method for analyzing sublingual and tongue surface images in the four traditional Chinese medicine diagnostic methods based on deep learning. Background Art

[0002] As an important traditional diagnostic method, traditional Chinese medicine tongue diagnosis infers the health status of the human body by observing various manifestations such as the color, shape, and tongue coating of the tongue. However, traditional tongue diagnosis methods rely on doctors' personal experience and subjective judgment, which makes the diagnostic results have certain uncertainties and variabilities, especially among different doctors. With the development of technology, especially the progress in the fields of image processing and machine learning, modern medical diagnostic technologies are moving towards automation and precision. In this context, applying deep learning to traditional Chinese medicine tongue diagnosis can not only standardize the diagnostic process but also improve the accuracy and repeatability of diagnosis.

[0003] Currently, although there have been studies attempting to automatically analyze tongue images through computer vision technology, most of these methods focus on basic image processing such as tongue edge detection and color recognition, and are insufficient for in-depth analysis of tongue image features and comprehensive judgment of health status. In addition, existing technologies often require a large amount of manual intervention when dealing with complex and ever-changing tongue image features, which limits their wide application in clinical practice. Summary of the Invention

[0004] To solve the above problems, the present invention discloses a method for analyzing sublingual and tongue surface images based on deep learning. Based on the analysis of sublingual and tongue surface images in traditional Chinese medicine tongue diagnosis, by combining image analysis and machine learning technologies, tongue image features are extracted, and a pattern of health status is identified through intelligent algorithms to achieve rapid prediction of an individual's health status.

[0005] To solve the above technical problems, the present invention provides a technology for analyzing sublingual and tongue surface images in the four traditional Chinese medicine diagnostic methods based on deep learning, which includes the following components:

[0006] (1) Image acquisition module: used for users to upload images of the sublingual and tongue surface.

[0007] (2) Deep learning analysis unit module: This module contains a pre-trained neural network for analyzing the uploaded tongue surface and sublingual images. The network identifies and classifies health indicators related to the four traditional Chinese medicine diagnostic methods by extracting key features of the images.

[0008] (3) Feature database module.

[0009] (4) Diagnostic result analysis and evaluation module.

[0010] (5) Result display and reporting module: Displays the analysis and diagnosis results in a user-friendly visual format, including a health assessment report and possible medical advice, to assist users or medical professionals in understanding and using.

[0011] Preferably, in component (1), the image acquisition module specifically includes the following components:

[0012] (11) The Gaussian bilateral filtering algorithm is used to remove random noise in the image. This algorithm smooths the image by combining the weights in the spatial domain and the pixel value domain, and can retain the edges and texture details of the image while removing noise, improving the image quality.

[0013] (12) The Laplace operator is used to enhance the edges and texture details of the image. The Laplace operator is a second-order derivative operator that sharpens the edges and enhances the details by detecting the regions where the pixel gray values change significantly in the image (i.e., the edges), which helps the subsequent deep learning model to accurately extract the tongue image features.

[0014] Preferably, in component (2), the deep learning analysis unit module receives and processes the images of the underside and surface of the tongue and extracts relevant information related to the four traditional Chinese medicine diagnostic methods. Specifically, it includes the following components:

[0015] (21) A deep convolutional neural network with multiple convolutional layers is used to extract the basic features of the image. The convolutional layer can capture local patterns and structural information in the image, such as texture and contour.

[0016] (22) Combine local binary patterns to extract key tongue diagnosis features from the preprocessed image data. LBP is an effective texture descriptor that generates a binary pattern map by comparing the gray values of each pixel with those of its neighboring pixels, and can capture the microscopic texture features in the tongue image.

[0017] (23) Use Gabor filters to further extract the texture and edge information of the image. Gabor filters are linear filters that can simulate the texture perception characteristics of the human visual system and extract the texture features of the image through multi-scale and multi-directional filtering operations.

[0018] (24) Apply principal component analysis to reduce the dimensionality of the high-dimensional image data, retain the main feature components, reduce the data dimension, and improve the computational efficiency and generalization ability of the model. Use autoencoders for non-linear dimensionality reduction and reconstruction of image features. Autoencoders can extract the hidden features in high-dimensional data through compression and decoding of input features, and further optimize the feature representation.

[0019] Preferably, in component (3), use the existing feature database to compare and match the data of the uploaded images of the underside and surface of the tongue. Specifically, it includes the following components:

[0020] (31) The feature database uses a hash table as the indexing system. By performing hash encoding on the feature vectors, fast lookup and matching are achieved. The hash table has efficient insertion and query performance, can quickly locate the position of the feature vector in the database, and significantly improves the speed of comparison and matching.

[0021] (32) Using the classification algorithm that combines the support vector machine (SVM) with deep learning, systematic annotation and classification are carried out on the tongue diagnosis image data in the database. Through the trained deep learning classification model, automatic annotation is performed on the newly added tongue image data, generating detailed feature labels and classification results to ensure the continuous update and accuracy of the database.

[0022] (33) Using the machine learning algorithm K-Nearest Neighbor algorithm (KNN) to preliminarily screen the uploaded image data, finding the database records closest to the features of the uploaded image to support automated disease recognition and health status analysis.

[0023] Preferably, in component (4), the diagnosis result analysis and evaluation module: used to analyze and interpret the image data, specifically including the following components: The present invention respectively uses the correlation coefficient and cosine similarity to calculate the similarity between the feature data of the sublingual and dorsal tongue images and the samples in the database; uses the support vector machine for the recognition and classification of disease patterns;

[0024] (41) According to the results of similarity matching and query processing, return the tongue diagnosis feature information and health status mapping relationship most similar to the new sample. If the matching result does not meet the requirements or there is no corresponding data in the database, consider updating the database, adding new tongue diagnosis feature data, and performing corresponding data management and index update;

[0025] Preferably, in component (5), the result display and report module: displays the obtained health assessment results in a visual form, presents the results to the user through an output device, and generates a health analysis report, specifically including the following components

[0026] (51) According to specific data and analysis requirements, present the health assessment and analysis results to the user in a visual form, including charts corresponding to tongue diagnosis data, health heat maps, etc., to present information such as the change trend and relative proportion of health parameters;

[0027] (52) The present invention draws and displays the health assessment data results obtained through the deep learning model using computer visualization tools;

[0028] (53)Based on the analysis results, a health analysis report is generated by integrating information. The report includes health assessment results, data charts, and diagnostic-related information.

[0029] Advantages of the present invention:

[0030] 1. By using advanced image recognition and machine learning models, not only can the basic features of the tongue image be automatically recognized and analyzed, but also deeper health information can be extracted from the tongue image. By constructing a large-scale tongue image feature database and training a deep neural network model, this technology can automatically and accurately analyze the tongue image and perform intelligent matching with existing health records, providing scientific and objective diagnostic basis for doctors.

[0031] 2. The application of the present invention can not only reduce the workload of doctors and improve the diagnostic efficiency, but also provide more personalized health advice and treatment plans for patients through accurate tongue image analysis. In addition, the development and improvement of this technology are expected to promote the traditional Chinese medicine tongue diagnosis towards a more modern and scientific direction, enhancing its status and role in the modern medical system. Brief Description of the Drawings

[0032] Figure 1 It is a flowchart of the tongue bottom and tongue surface image analysis technology for the four diagnostic methods of traditional Chinese medicine based on deep learning.

[0033] Figure 2 It is a diagram of the tongue image feature database;

[0034] Figure 3 It is a pattern recognition diagram. Detailed Embodiment

[0035] The present invention will be further clarified below in conjunction with the drawings and detailed embodiments. It should be understood that the following detailed embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. It should be noted that the terms "front", "rear", "left", "right", "upper" and "lower" used in the following description refer to the directions in the drawings, and the terms "inner" and "outer" refer to the directions towards or away from the geometric center of a specific component respectively.

[0036] This embodiment provides a tongue bottom and tongue surface image analysis technology for the four diagnostic methods of traditional Chinese medicine based on deep learning, including the following steps:

[0037] (1) Image acquisition module: used for users to upload images of the tongue bottom and tongue surface.

[0038] (2) Deep learning analysis unit module: This module contains a pre-trained neural network for analyzing the uploaded tongue surface and tongue bottom images. The network identifies and classifies health indicators related to the four diagnostic methods of traditional Chinese medicine by extracting key features of the images.

[0039] (3) Feature database module.

[0040] (4) Diagnostic result analysis and evaluation module.

[0041] (5) Result display and reporting module: Displays the analysis and diagnostic results in a user-friendly visual format, including a health assessment report and possible medical advice, to assist users or medical professionals in understanding and using.

[0042] In a specific embodiment, the present invention applies the tongue bottom and tongue surface image analysis technology of traditional Chinese medicine four diagnostic methods based on deep learning. By combining the image data of the tongue bottom and tongue surface, a comprehensive assessment of the human health status is carried out, as Figure 1 shown. In this embodiment, the system specifically analyzes tongue image features, such as the color, thickness, and humidity of the tongue coating, as well as other tongue surface features, so as to provide a comprehensive diagnosis of the human health condition. By identifying and classifying common health indicators and possible symptom manifestations, such as blood stasis, damp-heat, and spleen deficiency.

[0043] In a specific embodiment, in the image acquisition module, the user uploads the images of the tongue bottom and tongue surface, and uses the Gaussian bilateral filtering algorithm to remove the image noise. For example, there may be some random noise points in the original image. After filtering, the noise is effectively smoothed.

[0044]

[0045] Among them, I(x, y) is the pixel value of the image at (x, y), and f r is the gray difference function, which is used to calculate the difference between pixels, and f d is the spatial distance function, which is used to calculate the distance between pixels, and W p is the normalization constant, which is used to ensure the smoothness of the filtering result. This filtering algorithm retains the edge information of the image while removing the noise.

[0046] Use the Laplace operator to enhance the edge and texture details of the image. For example, the tongue print details in the original image may not be obvious. After edge enhancement, the details become more prominent.

[0047]

[0048] Among them, represents the Laplace transform of the image, and respectively represent the second-order derivatives of the image in the x and y directions, which are used to detect the edges in the image. The Laplace operator enhances the details and edge information of the image by calculating the change rate of the image gray value. After preprocessing, the clear image has reduced noise and enhanced edge and texture details.

[0049] The preprocessed image enters the deep learning analysis unit for further processing. The convolutional neural network (CNN) is used to extract the local features of the image. For example, through multiple convolutional layers and pooling layers, CNN can identify the red spots, textures, and morphological features on the tongue surface.

[0050]

[0051] Among them, I(i,j) is the input image, and K(m,n) is the convolutional kernel. The local features are extracted through the convolution operation. The convolution operation captures the local information such as edges and textures in the image by applying a filter (convolutional kernel) to the image. Combining the local binary pattern (LBP) and Gabor filter further extracts the texture and directional features of the image. For example, LBP is used to identify the texture pattern on the tongue surface, and the Gabor filter is used to capture the directional features on the tongue surface.

[0052]

[0053] Among them, LBP(x,y) is the LBP eigenvalue at (x,y), I i is the neighborhood pixel value, and s is the sign function used to compare the gray difference between the neighborhood pixel and the central pixel. LBP generates a binary pattern by comparing the gray values of the central pixel and its neighborhood pixels, which is used to describe the local texture features.

[0054]

[0055] Among them, G(x,y;λ,θ,ψ,σ,γ) is the Gabor filter function, x′ and y′ are the rotated coordinates of x and y respectively, λ is the wavelength, θ is the direction angle, ψ is the phase offset, σ is the standard deviation of the Gaussian envelope, and γ is the spatial aspect ratio. The Gabor filter extracts the directional and scale features in the image by simulating the receptive field of the human visual system. The extracted high-dimensional feature vector contains various information such as the color, texture, and shape of the tongue surface.

[0056] The high-dimensional feature vector is dimensionally reduced using principal component analysis (PCA) to reduce the feature dimension and retain the main information. For example, it is dimensionally reduced from the original high-dimensional feature vector to a smaller dimension while retaining the main feature information.

[0057] Z = XW

[0058] Among them, Z is the dimensionally reduced feature vector, X is the original feature matrix, and W is the feature vector matrix. PCA projects the high-dimensional data into a low-dimensional space by finding the principal components of the data, reducing the computational complexity while retaining the main information of the data.

[0059] Use an autoencoder to further reduce the dimension and fuse features. For example, the feature vector is encoded into a lower-dimensional latent representation by the autoencoder, and then the decoder restores it.

[0060] h = σ(W e x + b e ),

[0061] where h is the encoded feature vector, W e and W d are the weight matrices of the encoder and decoder respectively, b e and b d are the bias terms, and σ is the activation function. The autoencoder extracts the latent representation of the data through unsupervised learning and retains important features during the dimensionality reduction process.

[0062] The reduced-dimensional feature vector uses a recurrent neural network (RNN) to integrate time-series features. For example, through RNN processing, the changing features of the dorsal and ventral tongue images at different time points are captured.

[0063] h t = σ(W h h t-1 + W x x t + b)

[0064] where h t is the hidden state at time t, W h and W x are the weight matrices of the hidden state and input respectively, b is the bias term, and σ is the activation function. The RNN uses the hidden state (hidden layer) to memorize the information in the time series and captures the dynamic changes of the image features. The integrated feature vector contains dynamic change information.

[0065] The integrated feature vector is used in the feature database module to compare and match the uploaded image features. As Figure 2 shown, the construction process of the feature database is as follows: The feature vectors are stored in a hash table for quick indexing and searching. The hash function is defined as follows:

[0066] h(k) = k mod m

[0067] where h(k) is the hash value, k is the key value, and m is the size of the hash table. The hash table maps the feature vector to the hash value through the hash function for quick searching and matching. Support vector machines (SVMs) are used to classify and label the tongue diagnosis images in the database. For example, healthy and unhealthy image samples are classified according to the features of the dorsal and ventral tongue.

[0068]

[0069] Among them, f(x) is the classification result, and α i is the Lagrange multiplier, y i is the class label, K is the kernel function, and b is the bias term. SVM classifies feature vectors by finding the hyperplane with the maximum margin.

[0070] Use the K-Nearest Neighbor algorithm (KNN) to compare and match the uploaded image features with the samples in the database. For example, find the database sample most similar to the uploaded image features through the KNN algorithm.

[0071]

[0072] Among them, d(x,y) is the Euclidean distance between the feature vectors x and y. KNN finds the most similar samples by calculating the distances between feature vectors.

[0073] According to the comparison and matching results, conduct diagnostic result analysis and evaluation: calculate the similarity between the feature data and the database samples. For example, use the correlation coefficient and cosine similarity to calculate the similarity between the uploaded image features and the database samples. As Figure 3 shown.

[0074]

[0075] Among them, r is the correlation coefficient, x i and y i are the components of two feature vectors respectively, and are the means of the feature vectors respectively. The correlation coefficient is used to measure the linear correlation between feature vectors.

[0076]

[0077] Among them, cos(θ) is the cosine similarity between feature vectors A and B. The cosine similarity is used to measure the similarity between feature vectors, and the closer the value is to 1, the more similar the feature vectors are. Use the Support Vector Machine (SVM) to identify and classify disease patterns. For example, use the SVM classifier to identify whether the uploaded image has disease features. The preliminary diagnostic results include health assessment and potential disease patterns.

[0078] Present the analysis and diagnostic results in a user-friendly form: generate health assessment charts and heatmaps through visualization tools. For example, display the change trends and relative proportion information of the health parameters on the tongue surface and the underside of the tongue. Generate a health analysis report containing health assessment results, data charts, and diagnostic-related information.

[0079] The technical means disclosed by the solution of the present invention are not limited to the technical means disclosed in the above embodiments, and also include technical solutions composed of any combination of the above technical features.

Claims

1. A method for analyzing the tongue base and tongue surface images of the four diagnostic methods of traditional Chinese medicine based on deep learning, characterized in that: Specifically, it includes the following components: (1) Image acquisition module: used by users to upload images of the tongue bottom and tongue surface; in component (1), a Gaussian bilateral filtering algorithm is used to remove random noise in the image, and a Laplacian operator is used to enhance the edge and texture details of the image; (2) Deep learning analysis unit module: This module contains a pre-trained neural network for analyzing uploaded images of the tongue surface and tongue base. The network identifies and classifies health indicators related to the four diagnostic methods of traditional Chinese medicine by extracting key features of the images. In component (2), the deep learning analysis unit module receives and processes images of the tongue base and tongue surface and extracts relevant information about the four diagnostic methods of traditional Chinese medicine. Specifically, it includes the following components: (21) Use a deep convolutional neural network with multiple convolutional layers, combined with local binary pattern LBP and Gabor filter to extract basic features of the image; Among them, I(i,j) is the input image, K(m,n) is the convolution kernel, and local features are extracted through convolution operation; Among them, LBP(x,y) is the LBP eigenvalue at (x,y), I i is the neighborhood pixel value, s is the sign function used to compare the grayscale difference between the neighborhood pixel and the center pixel; Where G(x,y;λ,θ,ψ,σ,γ) is the Gabor filter function, x′ and y′ are the rotation coordinates of x and y respectively, λ is the wavelength, θ is the direction angle, ψ is the phase shift, σ is the standard deviation of the Gaussian envelope, and γ is the spatial aspect ratio; (22) Principal component analysis (PCA) and autoencoder technology are used to reduce the dimensionality of high-dimensional image data, and recurrent neural network (RNN) is used to integrate feature changes in time series; Z = XW Among them, Z is the eigenvector after dimensionality reduction, X is the original feature matrix, and W is the eigenvector matrix; h=σ(W e x+b e ), Among them, h is the encoded feature vector, W e and W d are the weight matrices of the encoder and decoder, respectively, and b e and b d is the bias term, σ is the activation function; h t =σ(W h h t-1 +W x x t +b) Among them, h t is the hidden state at time t, W h and W x are the weight matrices of the hidden state and input, b is the bias term, and σ is the activation function (3) Using the existing feature database, the uploaded tongue base and tongue surface images are compared and matched; (4) Diagnosis result analysis and evaluation module: used to analyze and interpret image data, specifically including the following components: using correlation coefficient and cosine similarity to calculate the similarity between the feature data of tongue base and tongue surface images and samples in the database; using support vector machine to identify and classify disease patterns; (5) Result presentation and reporting module: The analysis and diagnosis results are presented in a user-friendly visual format, including health assessment reports and possible medical recommendations, to help users or medical professionals understand and use them.

2. A method for analyzing the tongue base and tongue surface images of the four diagnostic methods of traditional Chinese medicine based on deep learning according to claim 1, characterized in that: Component (1) Among them, I(x,y) is the pixel value of the image at (x,y), f r is the grayscale difference function, used to calculate the difference between pixels, f d is the spatial distance function, used to calculate the distance between pixels, W p is a normalization constant used to ensure the smoothness of the filtering result; in, represents the Laplace transform of the image, and Represent the second-order derivatives of the image in the x and y directions respectively.

3. A method for analyzing the tongue base and tongue surface images of the four diagnostic methods of traditional Chinese medicine based on deep learning according to claim 1, characterized in that: Component (3) specifically includes the following components: (31) The feature database uses a hash table as the index system; h(k)=kmodm Among them, h(k) is the hash value, k is the key value, and m is the size of the hash table; (32) Use the deep learning classification algorithm support vector machine (SVM) to systematically label and classify the tongue diagnosis image data in the database; Among them, f(x) is the classification result, α i is the Lagrange multiplier, y i is the category label, K is the kernel function, and b is the bias term; (33) Using the machine learning K-nearest neighbor algorithm KNN, the uploaded image data is compared and matched with the data in the feature database; where d(x,y) is the Euclidean distance between feature vectors x and y.

4. According to claim 1, a method for analyzing the tongue base and tongue surface images of the four diagnostic methods of traditional Chinese medicine based on deep learning, characterized in that: Component (4) is specifically: Among them, r is the correlation coefficient, x i and i are the components of the two eigenvectors, and are the means of the eigenvectors respectively; the correlation coefficient is used to measure the linear correlation between the eigenvectors; Among them, cos(θ) is the cosine similarity between feature vectors A and B; cosine similarity is used to measure the similarity between feature vectors, and the closer the value is to 1, the more similar the feature vectors are.

5. The deep learning-based tongue base and tongue surface image analysis technology of the four diagnostic methods of traditional Chinese medicine as claimed in claim 1, characterized in that: In component (5), the result display and report module: displays the health assessment results obtained by analysis in a visual form, presents the results to the user through the output device, and generates a health analysis report, which specifically includes the following components (51) Based on specific data and analysis requirements, health assessment and analysis results are presented to users in a visual form, including graphs and health heat maps corresponding to tongue diagnosis data, to present the changing trends and relative proportion information of health parameters; (52) The health assessment data results obtained by the deep learning model are plotted and displayed using computer visualization tools; (53) Based on the analysis results, the information is integrated to generate a health analysis report, which includes health assessment results, data charts, and diagnosis-related information.

Images