Tongue picture quantitative analysis method and system based on texture and hyperspectral features
Through the tongue image quantization analysis method based on texture and hyperspectral features, the deep learning network and convolutional neural network model are used to extract the texture and spectral characteristics of tongue coating, which solves the problem of lack of quantification of tongue coating parameter judgment in traditional tongue diagnosis, and realizes the tongue coating Accurate quantitative analysis and greasy rating evaluation.
Patent Information
- Application Number
- CN202411260155.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-12-31
AI Technical Summary
Traditional Chinese medicine tongue diagnosis relies on doctors’ subjective judgments, lacks objective standards, and the existing technology ignores the characteristic information of tongue coating spectral, resulting in the judgment of tongue coating parameter mostly general descriptions and lacks quantitative numerical evaluation.
Using the tongue image quantization analysis method based on texture and hyperspectral features, the tongue coating area was segmented through the 3D U-net deep learning network, the convolutional neural network model was constructed, the tongue coating texture and hyperspectral features were extracted, and the tongue coating thickness and greasy level discrimination model was trained. , realize quantitative analysis of tongue coating.
The accurate quantitative description of the relative area and volume of tongue coating is achieved, the quantitative evaluation problem of greasy results is solved, and more accurate tongue image analysis results are provided.
Smart Images
Figure CN119229140A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of tongue image processing, and in particular to a tongue image quantitative analysis method and system based on texture and hyperspectral features. Background Art
[0002] Tongue diagnosis is an important basis for TCM disease diagnosis, syndrome identification, and treatment and medication, and has important clinical significance and application value. Traditional TCM tongue diagnosis relies on the doctor's subjective judgment, and its diagnosis results are affected by the doctor's experience and external environment. At the same time, when defining the degree of tongue coating, the results of different doctors vary greatly, making it difficult to form an objective and unified judgment standard, which brings many inconveniences to the inheritance and promotion of TCM.
[0003] With the continuous development of technologies such as computer science and image processing, objective research on TCM tongue diagnosis with the help of digital images has also been carried out more and more, forming a series of research results such as tongue diagnosis expert software and tongue image automatic analyzer. However, most of the current research is based on color images, ignoring the reflectivity information of other spectral bands, resulting in the problem of missing spectral feature information. In the prior art, the parameter discrimination of tongue coating is mostly focused on the classification and descriptive results of the thickness, moistness, greasy coating, etc. of the tongue coating, which are mostly general descriptions and lack quantitative numerical discrimination. Therefore, for those skilled in the art, how to analyze and calculate the tongue image with the help of multi-dimensional features to obtain the numerical quantitative evaluation results of the tongue coating is an urgent problem to be solved. Summary of the invention
[0004] The purpose of the present invention is to provide a method and system for quantitative analysis of tongue images based on texture and hyperspectral features, so as to solve the problems raised in the background technology and realize quantitative analysis of tongue images.
[0005] To achieve the above object, the present invention provides the following scheme: a tongue image quantitative analysis method based on texture and hyperspectral features, the specific steps of which include the following:
[0006] Collect hyperspectral images of the tongue and mark the tongue coating position contour, greasy coating level information, and tongue coating thickness level information;
[0007] Establishing a tongue coating segmentation model based on a 3D U-net deep learning network, segmenting the tongue coating position area of the hyperspectral image, and obtaining the relative area of the tongue coating area;
[0008] In the tongue coating region, calculating and extracting the texture features and hyperspectral features of the tongue coating;
[0009] Constructing a first convolutional neural network model, taking the hyperspectral features of the tongue coating area and the tongue coating thickness level information as input, training the first convolutional neural network model, and obtaining a tongue coating thickness discrimination model;
[0010] Constructing a second convolutional neural network model, taking the texture features of the tongue coating area, the hyperspectral features, and the greasy coating grade information as input, training the second convolutional neural network model, and obtaining a greasy coating grade discrimination model;
[0011] The tongue image and tongue coating are quantitatively analyzed by using the tongue coating thickness discrimination model and the greasy coating grade discrimination model.
[0012] Preferably, the texture features include gray-level co-occurrence matrix features and Tamura texture features; the hyperspectral features are the average values of hyperspectral reflectance curves of all pixels in the tongue coating area.
[0013] Preferably, the thickness result of the tongue coating area is calculated pixel by pixel by the tongue coating thickness discrimination model, and the relative volume of the tongue coating is calculated according to the thickness calculation result. The calculation formula of the relative volume of the tongue coating is: The thickness calculation result of the i-th tongue coating pixel, where N is the total number of pixels corresponding to the tongue coating.
[0014] Preferably, marking the greasy coating level information includes marking no greasy coating as 0, mild greasy coating as 1, moderate greasy coating as 2, and severe greasy coating as 3.
[0015] Preferably, the relative area of the tongue coating region is the number of pixels determined to be the tongue coating region / the total number of pixels in the tongue body region.
[0016] Preferably, the method further includes performing a preprocessing operation on the hyperspectral image of the tongue to eliminate image noise.
[0017] Preferably, the specific steps of the pretreatment operation are:
[0018] Performing black-and-white verification on the hyperspectral image to complete correction and registration to obtain a first image;
[0019] discarding the front and rear fixed bands of the first image to obtain a second image;
[0020] The minimum noise separation method is used to concentrate the energy of the second image to remove the noise in the band.
[0021] Preferably, the first convolutional neural network model is a deep learning model based on DenseNet.
[0022] Preferably, the tongue image and tongue coating are quantitatively analyzed by using the tongue coating thickness discrimination model and the greasy coating grade discrimination model, and the results of the tongue image and tongue coating quantitative analysis include the relative area of the tongue coating, the relative volume of the tongue coating, the tongue coating thickness grade, and the greasy coating grade.
[0023] On the other hand, a tongue image quantitative analysis system based on texture and hyperspectral features is provided, including an image acquisition module, an image segmentation module, a feature extraction module, a tongue coating thickness discrimination module, a greasy coating grade discrimination module, and a tongue image analysis module; wherein,
[0024] The image acquisition module is used to collect the hyperspectral image of the tongue and mark the tongue coating position contour, greasy coating level information, and tongue coating thickness level information;
[0025] The image segmentation module is used to establish a tongue coating segmentation model based on a 3D U-net deep learning network, segment the tongue coating position area of the hyperspectral image, and obtain the relative area of the tongue coating area;
[0026] The feature extraction module is used to calculate and extract the texture features and hyperspectral features of the tongue coating in the tongue coating area;
[0027] The tongue coating thickness discrimination module is used to construct a first convolutional neural network model, take the hyperspectral features of the tongue coating area and the tongue coating thickness level information as input, train the first convolutional neural network model, and obtain a tongue coating thickness discrimination model;
[0028] The greasy fur grade discrimination module is used to construct a second convolutional neural network model, take the texture features of the tongue fur area, the hyperspectral features, and the greasy fur grade information as input, train the second convolutional neural network model, and obtain a greasy fur grade discrimination model;
[0029] The tongue image analysis module is used to perform quantitative analysis of tongue image and tongue coating through the tongue coating thickness discrimination model and the greasy coating grade discrimination model.
[0030] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:
[0031] (1) Based on the hyperspectral features, the tongue image and tongue coating are identified and analyzed to obtain the relative area and volume of the tongue coating area compared to the whole tongue area, so as to more accurately quantify the properties of the tongue coating.
[0032] (2) A quantitative evaluation method for greasy tongue coating based on texture and hyperspectral features is used to effectively solve the problem that the existing methods have insufficient data feature dimensions and cannot quantitatively evaluate the results of greasy tongue coating.
[0033] (3) On the basis of RGB color images, the spectral feature dimensions of hyperspectral images are fully considered, and regional segmentation and quantification calculation of tongue coating are performed based on hyperspectral images. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0035] Figure 1 is a flow chart of the method of the present invention;
[0036] Figure 2 It is a system structure diagram of the present invention. DETAILED DESCRIPTION
[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0038] The current basic process for the classification and discrimination of greasy tongue coating is as follows: (1) data acquisition; (2) tongue coating parameter labeling; (3) tongue coating segmentation model training; (4) identification of parameters such as thick coating / greasy coating / moist coating. The obtained results do not have quantitative results, and the coverage range and coverage area of the tongue coating area are not clearly marked. In addition, the calculation of the tongue coating volume based on the results of thick coating / thin coating is not involved. There is a lack of quantitative output of the relative area and relative volume of the tongue coating area, resulting in the current results being mostly general descriptions and lack of quantitative numerical discrimination. In addition, the current greasy coating recognition results are mostly binary classification results of presence / absence, but the current results cannot fully describe the occurrence and development process of greasy coating. Therefore, it is necessary to be able to quantify the changes in greasy coating, which will help evaluate clinical efficacy and disease outcomes.
[0039] The purpose of this invention is to provide a tongue image quantitative analysis method based on texture and hyperspectral features. Figure 1 As shown, the specific steps include the following:
[0040] S1, collecting hyperspectral images of the tongue and marking the tongue coating position contour, greasy coating level information, and tongue coating thickness level information;
[0041] S2. A tongue coating segmentation model is established based on the 3D U-net deep learning network to segment the tongue coating location area of the hyperspectral image and obtain the relative area of the tongue coating area;
[0042] S3, in the tongue coating area, calculating and extracting the texture features and hyperspectral features of the tongue coating;
[0043] S4, constructing a first convolutional neural network model, taking the hyperspectral features of the tongue coating area and the tongue coating thickness level information as input, training the first convolutional neural network model, and obtaining a tongue coating thickness discrimination model;
[0044] S5, constructing a second convolutional neural network model, taking the texture features of the tongue coating area, the hyperspectral features, and the greasy coating grade information as input, training the second convolutional neural network model, and obtaining a greasy coating grade discrimination model;
[0045] S6. Quantitative analysis of tongue image and tongue coating is performed using the tongue coating thickness discrimination model and greasy coating grade discrimination model.
[0046] In the present invention, tongue coating parameters are output in the form of numerical quantification, which is different from the classification results in the prior art. For example, if the current tongue coating results are thick coating and greasy coating, the output results of the present invention are thick coating thickness 0.35 and greasy coating level 2.5.
[0047] Furthermore, a hyperspectral image acquisition device is used to acquire hyperspectral image data of TCM tongue images in S 1. In order to ensure the reliability and stability of tongue coating segmentation and measurement model construction, no less than 500 cases need to be collected, and the range of collected greasy coating samples should cover different forms of greasy coating as much as possible.
[0048] Through the diagnosis of clinical experts in traditional Chinese medicine, the collected tongue coating position contour and tongue coating thickness level information and greasy coating level information are marked. Among them, the greasy coating level information mainly includes: no greasy coating, mild greasy coating, moderate greasy coating, and severe greasy coating. No greasy coating is marked as 0, mild greasy coating is marked as 1, moderate greasy coating is marked as 2, and severe greasy coating is marked as 3. The tongue coating thickness information needs to be judged at different levels, where no tongue coating is marked with a thickness of 0; mild tongue coating coverage is marked with a thickness of 0.33; moderate tongue coating coverage is marked with a thickness of 0.66; moderate tongue coating coverage is marked with a thickness of 1; that is, the thickness of the tongue coating is divided into four different levels, and each level is quantitatively marked using the corresponding numerical value, where no tongue coating is 0; the thickest tongue coating is 1.
[0049] No greasy coating refers to a normal thin white coating on the surface of the tongue without a dense greasy coating. Mild greasy coating refers to a layer of dense greasy coating on the surface of the tongue, with a relatively slight coverage and thickness. Moderate greasy coating refers to a greasy coating on the surface of the tongue with a larger coverage, denser texture, and increased thickness than mild greasy coating. Severe greasy coating refers to a greasy coating on the surface of the tongue with a larger coverage, very dense texture, and thicker appearance.
[0050] Due to the influence of the imaging system itself and the external environment, the hyperspectral image has a certain amount of noise. Therefore, it also includes preprocessing operations on the hyperspectral image of the tongue to eliminate image noise. The specific steps of the preprocessing operation are:
[0051] S11, performing black-and-white verification on the hyperspectral image to complete correction and registration to obtain a first image;
[0052] S12, discarding the front and rear fixed bands of the first image to obtain a second image;
[0053] S13, using the minimum noise separation method to concentrate energy on the second image to remove noise within the band.
[0054] The process of calculating and extracting the texture features and hyperspectral features of the tongue coating in step S3 is as follows:
[0055] Texture feature extraction method: perform principal component analysis on the tongue coating area obtained in step S2, select the first principal component result, and calculate its gray level co-occurrence matrix and Tamura feature as the texture feature of the tongue coating area.
[0056] Hyperspectral feature extraction method: The hyperspectral reflectance values of all pixels in the tongue coating area are extracted, and the average value of the hyperspectral reflectance of all pixels in the tongue coating area is calculated as the hyperspectral feature of the tongue coating area.
[0057] In order to realize the recognition and analysis of tongue color and tongue coating color, it is necessary to perform tongue coating separation operation on the hyperspectral tongue image, and establish a tongue coating segmentation model based on the 3D U-net deep learning network to segment and identify the tongue quality area and tongue coating area.
[0058] In the segmented tongue coating area, texture features are calculated and extracted from the color image data, including gray-level co-occurrence matrix features and Tamura texture features; hyperspectral features are calculated and extracted from the hyperspectral image data, specifically the average value of the hyperspectral reflectance curve of all pixels in the tongue coating area.
[0059] In this embodiment, the first convolutional neural network model is a deep learning model based on Dense Net.
[0060] The tongue image and tongue coating were quantitatively analyzed using the tongue coating thickness discrimination model and the greasy coating grade discrimination model. The results of the tongue image and tongue coating quantitative analysis included the relative area of the tongue coating, the relative volume of the tongue coating, the thickness grade of the tongue coating, the greasy coating grade, etc.
[0061] Furthermore, the thickness of the tongue coating area is calculated pixel by pixel through the tongue coating thickness discrimination model. At this time, the two extreme cases are: if there is no tongue coating, the thickness of the tongue coating is 0, and the volume of the tongue coating is 0; if the tongue coating completely covers the tongue body and is the thickest, the volume of the tongue coating is 100%*1=100. The relative volume calculation formula of the tongue coating in other cases is: The thickness calculation result of the i-th tongue coating pixel point, where N is the total number of pixels corresponding to the tongue coating, the calculated relative volume range of the tongue coating is 0-100, and the relative area and relative volume of the tongue coating are output.
[0062] The relative area of the tongue coating region is the number of pixels determined to be the tongue coating region / the total number of pixels in the tongue region.
[0063] Furthermore, the greasy coating is discriminated by using a greasy coating grade discrimination model, and the output result is the severity of the greasy coating, with no greasy coating marked as 0, mild greasy coating marked as 1, moderate greasy coating marked as 2, and severe greasy coating marked as 3.
[0064] On the other hand, a tongue image quantitative analysis system based on texture and hyperspectral features is provided. Figure 2 As shown, it includes an image acquisition module, an image segmentation module, a feature extraction module, a tongue coating thickness determination module, a greasy coating grade determination module, and a tongue image analysis module; wherein,
[0065] An image acquisition module is used to collect hyperspectral images of the tongue and mark the tongue coating position contour, greasy coating level information, and tongue coating thickness level information;
[0066] An image segmentation module is used to establish a tongue coating segmentation model based on a 3D U-net deep learning network, segment the tongue coating position area of the hyperspectral image, and obtain the relative area of the tongue coating area;
[0067] A feature extraction module, used for calculating and extracting the texture features and hyperspectral features of the tongue coating in the tongue coating area;
[0068] A tongue coating thickness discrimination module is used to construct a first convolutional neural network model, take the hyperspectral features of the tongue coating area and the tongue coating thickness level information as input, train the first convolutional neural network model, and obtain a tongue coating thickness discrimination model;
[0069] A greasy fur grade discrimination module is used to construct a second convolutional neural network model, take the texture features of the tongue fur area, the hyperspectral features, and the greasy fur grade information as input, train the second convolutional neural network model, and obtain a greasy fur grade discrimination model;
[0070] The tongue image analysis module is used to perform quantitative analysis of tongue image and tongue coating through the tongue coating thickness discrimination model and the greasy coating grade discrimination model.
[0071] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of the present invention. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A tongue image quantitative analysis method based on texture and hyperspectral features, characterized in that: The specific steps include the following: Collect hyperspectral images of the tongue and mark the tongue coating position contour, greasy coating level information, and tongue coating thickness level information; Establishing a tongue coating segmentation model based on a 3D U-net deep learning network, segmenting the tongue coating position area of the hyperspectral image, and obtaining the relative area of the tongue coating area; In the tongue coating region, calculating and extracting the texture features and hyperspectral features of the tongue coating; Constructing a first convolutional neural network model, taking the hyperspectral features of the tongue coating area and the tongue coating thickness level information as input, training the first convolutional neural network model, and obtaining a tongue coating thickness discrimination model; Constructing a second convolutional neural network model, taking the texture features of the tongue coating area, the hyperspectral features, and the greasy coating grade information as input, training the second convolutional neural network model, and obtaining a greasy coating grade discrimination model; The tongue image and tongue coating are quantitatively analyzed by using the tongue coating thickness discrimination model and the greasy coating grade discrimination model.
2. The method for quantitative analysis of tongue images based on texture and hyperspectral features according to claim 1, characterized in that: The texture features include gray-level co-occurrence matrix features and Tamura texture features; the hyperspectral features are the average values of the hyperspectral reflectance curves of all pixels in the tongue coating area.
3. The method for quantitative analysis of tongue images based on texture and hyperspectral features according to claim 1, characterized in that: The thickness result of the tongue coating area is calculated pixel by pixel by the tongue coating thickness discrimination model, and the relative volume of the tongue coating is calculated according to the thickness calculation result. The calculation formula of the relative volume of the tongue coating is: The thickness calculation result of the i-th tongue coating pixel, where N is the total number of pixels corresponding to the tongue coating.
4. The method for quantitative analysis of tongue images based on texture and hyperspectral features according to claim 1, characterized in that: Marking the greasy coating level information includes marking no greasy coating as 0, mild greasy coating as 1, moderate greasy coating as 2, and severe greasy coating as 3.
5. The method for quantitative analysis of tongue images based on texture and hyperspectral features according to claim 1, characterized in that: The relative area of the tongue coating region is the number of pixels determined as the tongue coating region / the total number of pixels in the tongue body region.
6. The method for quantitative analysis of tongue images based on texture and hyperspectral features according to claim 1, characterized in that: The method also includes a preprocessing operation on the hyperspectral image of the tongue to eliminate image noise.
7. The method for quantitative analysis of tongue images based on texture and hyperspectral features according to claim 6, characterized in that: The specific steps of the pre-processing operation are: Performing black-and-white verification on the hyperspectral image to complete correction and registration to obtain a first image; discarding the front and rear fixed bands of the first image to obtain a second image; The minimum noise separation method is used to concentrate the energy of the second image to remove the noise in the band.
8. The method for quantitative analysis of tongue images based on texture and hyperspectral features according to claim 1, characterized in that: The first convolutional neural network model is a deep learning model based on Dense Net.
9. The method for quantitative analysis of tongue images based on texture and hyperspectral features according to claim 1, characterized in that: The tongue image and tongue coating quantitative analysis is performed using the tongue coating thickness discrimination model and the greasy coating grade discrimination model. The tongue image and tongue coating quantitative analysis results include the tongue coating relative area, the tongue coating relative volume, the tongue coating thickness grade, and the greasy coating grade.
10. A tongue image quantitative analysis system based on texture and hyperspectral features, characterized in that: It includes image acquisition module, image segmentation module, feature extraction module, tongue coating thickness determination module, greasy coating grade determination module and tongue image analysis module; among them, The image acquisition module is used to collect the hyperspectral image of the tongue and mark the tongue coating position contour, greasy coating level information, and tongue coating thickness level information; The image segmentation module is used to establish a tongue coating segmentation model based on a 3D U-net deep learning network, segment the tongue coating position area of the hyperspectral image, and obtain the relative area of the tongue coating area; The feature extraction module is used to calculate and extract the texture features and hyperspectral features of the tongue coating in the tongue coating area; The tongue coating thickness discrimination module is used to construct a first convolutional neural network model, take the hyperspectral features of the tongue coating area and the tongue coating thickness level information as input, train the first convolutional neural network model, and obtain a tongue coating thickness discrimination model; The greasy fur grade discrimination module is used to construct a second convolutional neural network model, take the texture features of the tongue fur area, the hyperspectral features, and the greasy fur grade information as input, train the second convolutional neural network model, and obtain a greasy fur grade discrimination model; The tongue image analysis module is used to perform quantitative analysis of tongue image and tongue coating through the tongue coating thickness discrimination model and the greasy coating grade discrimination model.