A tongue image feature extraction method and system based on hyperspectral images

Abstract

The application discloses a tongue image feature extraction method and system based on hyperspectral images, and relates to the field of spectroscopy. The method comprises the following steps: acquiring a to-be-tested hyperspectral image; the to-be-tested hyperspectral image is a tongue body part hyperspectral image of a to-be-tested tongue body; performing data feature extraction on the to-be-tested hyperspectral image by using a principal component analysis method to obtain a first principal component, a second principal component and a third principal component of the to-be-tested tongue body; determining a space-spectrum feature of the to-be-tested tongue body according to the first principal component, the second principal component and the third principal component of the to-be-tested tongue body and the to-be-tested hyperspectral image; and the space-spectrum feature comprises a tongue fur space feature, a tongue fur spectrum and color feature, a crack space feature, a tooth mark space feature and a tongue shape space feature. The application can extract various spectral features and space features covering color, texture and shape information of the tongue body from the hyperspectral image of the tongue body, so that the tongue body can be classified efficiently and accurately.

Description

Technical Field

[0001] This invention relates to the field of spectroscopy, and in particular to a method and system for extracting tongue features based on hyperspectral images. Background Technology

[0002] Traditional Chinese medicine (TCM) considers the tongue to be the "sprout of the heart" and that "the stomach's essence manifests in the tongue." Tongue diagnosis is an important component of TCM's observational diagnosis. Abnormal changes in the tongue, including its color, texture, and shape, can explain diseases of corresponding organs. Therefore, TCM tongue diagnosis has gained widespread attention in recent years due to its non-invasive nature, rapid diagnosis, and ability to reveal "pre-disease" conditions. Clinically, changes in the tongue's appearance are the most rapid and significant; however, traditional tongue diagnosis relies on the physician's subjective analysis and experience, lacks unified diagnostic standards, and is easily influenced by the external environment, thus limiting its development and application.

[0003] Hyperspectral imaging offers advantages such as speed, non-destructive nature, and rich information. When combined with tongue diagnosis, it can better reconstruct tongue image information, overcoming the limitations of traditional Chinese medicine's naked-eye observation and the imaging capabilities of traditional Chinese medicine's four diagnostic instruments, thus providing a larger and more multi-dimensional dataset for tongue diagnosis. However, existing inventions involving the digitization of hyperspectral images and tongue diagnosis, such as a D-ResNet-based hyperspectral Chinese medicine tongue coating and tongue body classification method (CN111259954A), primarily propose a method to separate the tongue coating from the tongue body, without extracting the spatial and spectral features of the tongue, or addressing research on the extraction of the tongue's texture, edges, and color features.

[0004] In summary, existing inventions for digital tongue diagnosis based on hyperspectral images have limitations in covering tongue features and fail to fully utilize the spatial and spectral information of hyperspectral tongue images. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for extracting tongue features based on hyperspectral images, so as to extract a variety of spectral and spatial features covering the color, texture and shape information of the tongue from the hyperspectral image of the tongue, thereby enabling efficient and accurate classification of the tongue.

[0006] To achieve the above objectives, the present invention provides the following solution: A method for tongue image feature extraction based on hyperspectral images, comprising: Acquire a hyperspectral image to be tested; the hyperspectral image to be tested is a hyperspectral image of the tongue body portion of the tongue body to be tested. Principal component analysis was used to extract data features from the hyperspectral image to be tested, resulting in the first principal component, the second principal component, and the third principal component of the tongue body to be tested. The first principal component, the second principal component, and the third principal component are all two-dimensional matrices with the same spatial resolution as the hyperspectral image to be tested. The spatial-spectral features of the tongue under test are determined based on the first, second, and third principal components of the tongue under test and the hyperspectral image of the tongue under test. These spatial-spectral features include: tongue coating spatial features, tongue coating spectral and color features, crack spatial features, teeth mark spatial features, and tongue shape spatial features. The tongue coating spatial features include: the white tongue coating's spectral-to-matrix ratio parameter and the yellow tongue coating's spectral-to-matrix ratio parameter. The tongue coating's spectral and color features include: the sum of the yellow tongue coating's spectral intensities, the maximum value of the yellow tongue coating's spectral intensity, the band containing the maximum value of the yellow tongue coating's spectral intensity, the RGB color parameters of the yellow tongue coating, the HSV color parameters of the yellow tongue coating, and the white tongue coating's spectral characteristics. The total intensity, maximum spectral intensity of white tongue coating, spectral intensity band of maximum white tongue coating, RGB color parameters of white tongue coating, and HSV color parameters of white tongue coating; the spatial characteristics of cracks include: crack length, crack density, average crack length, crack circumference, and crack compactness; the spatial characteristics of tooth marks include: number of tooth marks, average tooth width, maximum tooth width, minimum tooth width, average tooth depth, maximum tooth depth, minimum tooth depth, average tooth mark angle, maximum tooth mark angle, and minimum tooth mark angle; the spatial characteristics of tongue shape include: aspect ratio and tongue tip angle; the spatial spectral characteristics of the tongue body under test are used to classify the tongue body under test.

[0007] Optionally, the method for determining the spatial features of the tongue coating includes: The first principal component, the second principal component, and the third principal component are binarized respectively to obtain a first binary matrix, a second binary matrix, and a third binary matrix; Perform pixel-by-pixel multiplication on the second binary matrix and the first binary matrix to obtain the tongue coating distribution matrix; Perform pixel-by-pixel multiplication on the third binary matrix and the tongue coating distribution matrix to obtain the yellow tongue coating distribution matrix; The white tongue coating distribution matrix is ​​determined based on the tongue coating distribution matrix and the yellow tongue coating distribution matrix; The white tongue coating texture ratio parameter is determined based on the total number of white tongue coating pixels and the total number of tongue body pixels in the white tongue coating distribution matrix. The yellow tongue coating quality ratio parameter is determined based on the total number of yellow tongue coating pixels and the total number of tongue body pixels in the yellow tongue coating distribution matrix.

[0008] Optionally, the method for determining the spectral and color characteristics of the tongue coating includes: Based on the white tongue coating distribution matrix and the yellow tongue coating distribution matrix, the white tongue coating spectral curve and the yellow tongue coating spectral curve are extracted from the hyperspectral image to be tested, respectively. Based on the spectral curves of the white tongue coating and the yellow tongue coating, determine the maximum spectral intensity of the white tongue coating, the band containing the maximum spectral intensity of the white tongue coating, the total spectral intensity of the white tongue coating, the maximum spectral intensity of the yellow tongue coating, the band containing the maximum spectral intensity of the yellow tongue coating, and the total spectral intensity of the yellow tongue coating, respectively. Using the CIE color standard, the RGB color parameters of the white tongue coating and the yellow tongue coating are determined based on the spectral curves of the white tongue coating and the yellow tongue coating, respectively. The RGB color parameters of the white tongue coating are converted to obtain the HSV color parameters of the white tongue coating, and the RGB color parameters of the yellow tongue coating are converted to obtain the HSV color parameters of the yellow tongue coating.

[0009] Optionally, the method for determining the spatial characteristics of the crack includes: Based on the first principal component, tongue region information is extracted and binarized to obtain the tongue information matrix; Edge detection is performed on the information matrix in the tongue to obtain the crack edge matrix; The crack length is determined based on the total number of crack pixels in the information matrix of the tongue. The crack density is determined based on the crack length and the total number of pixels in the information matrix of the tongue. The average crack length is determined based on the crack length and the number of closed intervals in the crack edge matrix; The crack perimeter is determined based on the total number of crack edge pixels in the crack edge matrix. Crack compactness is determined based on the crack perimeter and the crack length.

[0010] Optionally, the method for determining the spatial features of the tooth marks includes: Based on the first principal component, tongue side region information is extracted and binarized to obtain the tongue side information matrix; The tongue side information matrix is ​​sequentially subjected to opening and closing operations to obtain the tongue side morphology matrix; Edge coordinates are extracted from the tongue-side morphological matrix and then subjected to moving average filtering to obtain a set of edge row coordinates and a set of edge column coordinates. The concavity and convexity of edge pixels are determined based on the set of edge row coordinates and the set of edge column coordinates, and edge pixels whose concavity and convexity exceed a set threshold are identified as tooth mark concavity and convexity points, thus obtaining a set of tooth mark concavity and convexity points. The number of all tooth marks and bumps in the set of tooth marks and bumps is determined as the number of tooth marks; A triangle is determined by any three tooth marks in the set of tooth marks, and the width, height and vertex angle of the triangle are calculated with the middle point as the vertex, so as to obtain the width array, height array and vertex angle array. The maximum, minimum, and average values ​​in the wide array are respectively determined as the maximum tooth width, minimum tooth width, and average tooth width; The maximum, minimum, and average values ​​in the high-order array are respectively determined as the maximum tooth depth, minimum tooth depth, and average tooth depth; The maximum, minimum, and average values ​​in the vertex angle array are respectively determined as the maximum tooth mark angle, the minimum tooth mark angle, and the average tooth mark angle.

[0011] Optionally, the method for determining the tongue-shaped spatial features includes: The hyperspectral image to be tested is mapped to obtain a tongue body distribution matrix; the pixel value of the tongue body part in the tongue body distribution matrix is ​​1, and the pixel value of the background part is 0. The minimum rectangle is determined based on the tongue distribution matrix; the minimum rectangle encloses all non-zero regions in the tongue distribution matrix and is tangent to the non-zero regions. The ratio of the length to the width of the smallest rectangle is defined as the aspect ratio. Edge extraction is performed on the tongue distribution matrix to obtain the tongue edge matrix; the pixel value of the tongue edge part in the tongue edge matrix is ​​1, and the pixel value of other parts is 0. The angle between the tongue tip and the tongue edge is determined based on the position of the bottommost pixel in the tongue edge matrix and the positions of the left and right adjacent pixels. The bottommost pixel is the pixel with a pixel value of 1 located at the bottom of the tongue edge matrix. The left and right adjacent pixels include: a pixel with a pixel value of 1 located at a predetermined distance to the left of the bottommost pixel and a pixel with a pixel value of 1 located at a predetermined distance to the right of the bottommost pixel.

[0012] Optionally, the tongue image feature extraction method based on hyperspectral images further includes: The spatial spectral features of the tongue to be tested are input into the tongue image classification model to obtain the classification result of the tongue to be tested; the classification result of the tongue to be tested includes at least one of the following: whether the tongue to be tested is a thick and greasy tongue, whether the tongue to be tested is a teeth-marked tongue, and whether the tongue to be tested is a fissured tongue; the tongue image classification model includes a thick and greasy tongue classification model, a teeth-marked tongue classification model, and a fissured tongue classification model in parallel; the thick and greasy tongue classification model, the teeth-marked tongue classification model, and the fissured tongue classification model are all binary classification models.

[0013] Optionally, the method for determining the tongue image classification model specifically includes: Obtain a sample dataset; the sample dataset includes several sample hyperspectral images and corresponding category labels; the sample hyperspectral images are hyperspectral images of the tongue body of the sample; the category labels include: thick and greasy tongue category label, teeth-marked tongue category label, and cracked tongue category label; Principal component analysis was used to extract data features from the hyperspectral image of the sample to obtain the first principal component, the second principal component, and the third principal component of the tongue of the sample. The spatial spectral characteristics of the sample tongue are determined based on the first principal component, the second principal component, and the third principal component of the sample tongue. The following parameters are defined as characteristic parameters of a thick, greasy tongue coating: the white tongue coating's material-to-texture ratio, the yellow tongue coating's material-to-texture ratio, the sum of the yellow tongue coating's spectral intensities, the maximum value of the yellow tongue coating's spectral intensities, the band containing the maximum value of the yellow tongue coating's spectral intensities, the RGB color parameters of the yellow tongue coating, the HSV color parameters of the yellow tongue coating, the sum of the white tongue coating's spectral intensities, the maximum value of the white tongue coating's spectral intensities, the band containing the maximum value of the white tongue coating's spectral intensities, the RGB color parameters of the white tongue coating, and the HSV color parameters of the white tongue coating. The number of tooth marks, the average tooth width, the maximum tooth width, the minimum tooth width, the average tooth depth, the maximum tooth depth, the minimum tooth depth, the average tooth mark angle, the maximum tooth mark angle, the minimum tooth mark angle, the aspect ratio, and the tongue tip angle are determined as the characteristic parameters of the tooth mark tongue. The crack length, crack density, average crack length, crack perimeter, and crack compactness are defined as characteristic parameters of the crack tongue. A random forest classifier is trained based on the thick, greasy tongue feature parameters and corresponding category labels of the sample tongue to obtain a thick, greasy tongue classification model; A random forest classifier is trained based on the tooth-marked tongue feature parameters and corresponding category labels of the sample tongue to obtain a tooth-marked tongue classification model; A random forest classifier is trained based on the cracked tongue feature parameters and corresponding category labels of the sample tongue to obtain a cracked tongue classification model.

[0014] A tongue image feature extraction system based on hyperspectral images, comprising: The acquisition module is used to acquire the hyperspectral image to be tested; the hyperspectral image to be tested is a hyperspectral image of the tongue body portion of the tongue body to be tested; The data feature extraction module is used to extract data features from the hyperspectral image to be tested using principal component analysis to obtain the first principal component, the second principal component, and the third principal component of the tongue body to be tested; the first principal component, the second principal component, and the third principal component are all two-dimensional matrices with the same spatial resolution as the hyperspectral image to be tested. The spatial spectral feature determination module is used to determine the spatial spectral features of the tongue under test based on the first principal component, the second principal component, and the third principal component of the tongue under test, as well as the hyperspectral image of the tongue under test. The spatial spectral features include: tongue coating spatial features, tongue coating spectral and color features, crack spatial features, tooth mark spatial features, and tongue shape spatial features. The tongue coating spatial features include: white tongue coating material-to-texture ratio parameters and yellow tongue coating material-to-texture ratio parameters. The tongue coating spectral and color features include: the sum of yellow tongue coating spectral intensities, the maximum value of yellow tongue coating spectral intensity, the band containing the maximum value of yellow tongue coating spectral intensity, the RGB color parameters of yellow tongue coating, and the HSV color parameters of yellow tongue coating. The parameters include: total spectral intensity of white tongue coating, maximum spectral intensity of white tongue coating, spectral band containing the maximum spectral intensity of white tongue coating, RGB color parameters of white tongue coating, and HSV color parameters of white tongue coating; the spatial characteristics of cracks include: crack length, crack density, average crack length, crack circumference, and crack compactness; the spatial characteristics of tooth marks include: number of tooth marks, average tooth width, maximum tooth width, minimum tooth width, average tooth depth, maximum tooth depth, minimum tooth depth, average tooth mark angle, maximum tooth mark angle, and minimum tooth mark angle; the spatial characteristics of tongue shape include: aspect ratio and tongue tip angle; the spatial spectral characteristics of the tongue body under test are used to classify the tongue body under test.

[0015] Optionally, the tongue image feature extraction system based on hyperspectral images further includes: The classification module is used to input the spatial spectral features of the tongue body to be tested into the tongue image classification model to obtain the classification result of the tongue body to be tested; the classification result of the tongue body to be tested includes at least one of the following: whether the tongue body to be tested is a thick and greasy tongue, whether the tongue body to be tested is a teeth-marked tongue, and whether the tongue body to be tested is a fissured tongue; the tongue image classification model includes a thick and greasy tongue classification model, a teeth-marked tongue classification model, and a fissured tongue classification model in parallel; the thick and greasy tongue classification model, the teeth-marked tongue classification model, and the fissured tongue classification model are all binary classification models.

[0016] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: This invention first employs principal component analysis to extract data features from the hyperspectral image under test, obtaining the first, second, and third principal components of the tongue under test. Then, based on the first, second, and third principal components of the tongue under test and the hyperspectral image under test, the spatial spectral features of the tongue under test are determined. These spatial spectral features include tongue coating spatial features, tongue coating spectral and color features, crack spatial features, teeth mark spatial features, and tongue shape spatial features, covering various aspects of the tongue's color, texture, and shape, and can more comprehensively reflect the condition of the tongue. Furthermore, combined with a tongue image classification model, the tongue under test can be accurately classified, providing data support for subsequent diagnosis by doctors. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart of the tongue image feature extraction method based on hyperspectral images provided by the present invention; Figure 2 This is a block diagram of the tongue image feature extraction system based on hyperspectral images provided by the present invention; Figure 3 A comparative diagram showing the classification results of various pathological tongues provided by this invention.

[0019] Symbol explanation: Acquisition Module-1, Data Feature Extraction Module-2, Spatial Spectral Feature Determination Module-3. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] The purpose of this invention is to provide a method and system for extracting tongue features based on hyperspectral images, so as to extract a variety of spectral and spatial features covering the color, texture and shape information of the tongue from the hyperspectral image of the tongue, thereby enabling efficient and accurate classification of the tongue.

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] Example 1 This invention provides a method for tongue image feature extraction based on hyperspectral images. For example... Figure 1 As shown, the tongue image feature extraction method includes: Step S1: Obtain the hyperspectral image to be tested; the hyperspectral image to be tested is a hyperspectral image of the tongue part of the tongue to be tested.

[0024] The hyperspectral image of the tongue body is obtained by preprocessing the original hyperspectral image of the tongue body; the preprocessing includes: background removal segmentation, Gaussian smoothing filtering for noise reduction, and Z-score normalization.

[0025] In this embodiment, a hyperspectral image of the human tongue was acquired using a hyperspectral imager, comprising 31 bands between 400nm and 700nm, with a spectral interval of 10nm. The spatial resolution of the image was 512×512. Because the hyperspectral data acquired by the imager contains noise due to the equipment and exhibits significant variations in spectral intensity between bands, preprocessing is necessary. First, Gaussian smoothing is used to remove noise. Then, Z-score normalization is employed to reduce the impact of intensity differences on data analysis. Since the captured images have complex backgrounds, background removal segmentation is performed on each image, retaining only the tongue portion.

[0026] Step S2: Principal component analysis is used to extract data features from the hyperspectral image to be tested, and the first principal component, the second principal component, and the third principal component of the tongue body to be tested are obtained; the first principal component, the second principal component, and the third principal component are all two-dimensional matrices with the same spatial resolution as the hyperspectral image to be tested.

[0027] Specifically, principal component analysis (PCA) was used for data feature extraction. PCA is an existing method for dimensionality reduction of hyperspectral image data. The principal components obtained are the processed results, with the first principal component indicating the most information after dimensionality reduction, followed by the second principal component, and so on. By revealing information at different locations in the tongue hyperspectral data through a small number of principal components, dimensionality reduction is achieved while preserving as much original variable information as possible. The sum of the contribution rates of the first three principal components obtained through PCA reached 99%.

[0028] Step S3: Determine the spatial-spectral features of the tongue body to be tested based on the first principal component, second principal component, and third principal component of the tongue body to be tested, as well as the hyperspectral image to be tested. The spatial-spectral features include: tongue coating spatial features, tongue coating spectral and color features, crack spatial features, tooth mark spatial features, and tongue shape spatial features. The tongue coating spatial features include: the white tongue coating's material-to-paste ratio parameter and the yellow tongue coating's material-to-paste ratio parameter. The tongue coating spectral and color features include: the sum of the yellow tongue coating's spectral intensities, the maximum value of the yellow tongue coating's spectral intensity, the band containing the maximum value of the yellow tongue coating's spectral intensity, the RGB color parameters of the yellow tongue coating, and the yellow tongue... The parameters for tongue coating HSV color, total spectral intensity of white tongue coating, maximum spectral intensity of white tongue coating, spectral band containing the maximum spectral intensity of white tongue coating, RGB color parameters of white tongue coating, and HSV color parameters of white tongue coating are included. The spatial characteristics of the cracks include: crack length, crack density, average crack length, crack circumference, and crack compactness. The spatial characteristics of the tooth marks include: number of tooth marks, average tooth width, maximum tooth width, minimum tooth width, average tooth depth, maximum tooth depth, minimum tooth depth, average tooth mark angle, maximum tooth mark angle, and minimum tooth mark angle. The spatial characteristics of the tongue shape include: aspect ratio and tongue tip angle. These spatial spectral characteristics of the tongue under test are used for classification.

[0029] The characteristics of hyperspectral data mainly include spatial and spectral features (which can be simply referred to as spatial-spectral features). According to traditional Chinese medicine theory, the spatial features of tongue hyperspectral data include the teeth marks on the sides of the tongue, the cracks in the center of the tongue, the shape of the tongue, and the ratio of tongue coating to its texture. The spectral features include the spectral response of the tongue coating and the color of the tongue coating. These features are important observation objects in traditional Chinese medicine tongue observation and are important indicators for clinical diagnosis. This invention extracts the above-mentioned spatial-spectral features to classify the tongue under test, obtaining a relatively accurate classification structure and providing data support for subsequent diagnosis by physicians. It should be noted that this invention does not contain a direct description of disease diagnosis methods; its output is limited to spatial-spectral features and classification results for physicians to make more efficient diagnoses. Whether a disease is diagnosed depends on the physician's professional knowledge and skills.

[0030] Traditional Chinese medicine believes that tongue coating is formed by the upward steaming of the spleen and stomach qi to the stomach yin, and is a layer of coating-like substance attached to the surface of the tongue. The observation of tongue coating mainly includes observing the texture and color of the coating. This is reflected in the tongue hyperspectral data as the spatial characteristics of the coating texture (i.e., the spatial characteristics of the tongue coating) and the spectral and color characteristics of the tongue coating area (i.e., the spectral and color characteristics of the tongue coating).

[0031] The method for determining the spatial characteristics of the tongue coating includes: (1) Binarize the first principal component, the second principal component, and the third principal component respectively to obtain the first binary matrix. Second binary matrix and the third binary matrix .in, , and All are two-dimensional matrices with length and width equal to the hyperspectral image of the tongue body and composed of 0s and 1s. Since the two-dimensional matrix referred to in this paper is obtained after a series of processing of the hyperspectral image, it can still be considered as an image in essence. Therefore, the elements in it will be referred to as pixels in the following text.

[0032] (2) Perform pixel-by-pixel multiplication on the second binary matrix and the first binary matrix, i.e. The tongue coating distribution matrix was obtained. .

[0033] (3) Perform pixel-by-pixel multiplication on the third binary matrix and the tongue coating distribution matrix, i.e. The distribution matrix of yellow tongue coating was obtained. .

[0034] in, and All are two-dimensional matrices with length and width equal to the hyperspectral images of the tongue body and consisting of 0s and 1s. The location of the yellow tongue coating is indicated by a pixel value of 1. The location of the tongue coating is indicated by a pixel value of 1.

[0035] In this embodiment, the obtained and This is the binarized result after removing the background, teeth marks, and cracks outside the tongue. It is a 512×512 matrix composed of 0s and 1s, with the positive values ​​having a higher overlap with the tongue coating in the original image. Regions with a value of 1 are defined as the locations of the tongue coating and the yellow, greasy tongue coating, respectively. The area with a value of 1 is defined as the location of the white tongue coating.

[0036] (4) Determine the white tongue coating distribution matrix based on the tongue coating distribution matrix and the yellow tongue coating distribution matrix, specifically as follows: .

[0037] (5) Determine the white tongue coating quality ratio parameter based on the total number of white tongue coating pixels and the total number of tongue body pixels in the white tongue coating distribution matrix. The specific formula is as follows: .

[0038] (6) Determine the yellow tongue coating quality ratio parameter based on the total number of yellow tongue coating pixels and the total number of tongue body pixels in the yellow tongue coating distribution matrix. The specific formula is as follows: .

[0039] in, The parameter for the ratio of white tongue coating to its texture. The parameter for the ratio of tongue coating to its texture in yellow tongue is... This represents the total number of pixels in the tongue region of the image, i.e., the total number of pixels in the tongue itself.

[0040] The methods for determining the spectral and color characteristics of the tongue coating include: (1) Extract the white tongue coating spectral curve and the yellow tongue coating spectral curve from the hyperspectral image to be tested according to the white tongue coating distribution matrix and the yellow tongue coating distribution matrix respectively.

[0041] Specifically, after obtaining the tongue coating area, the spectral curve characteristics of the white tongue coating and the yellow, greasy tongue coating can be further analyzed. The spectral curve of the white tongue coating portion is defined as follows: , where i is the band number in the spectral curve of the white tongue coating, and the spectral curve of the yellow tongue coating is: j represents the band number in the spectral curve of yellow tongue coating.

[0042] (2) Determine the maximum spectral intensity of the white tongue coating based on the spectral curves of the white tongue coating and the yellow tongue coating, respectively. The spectral intensity of white tongue coating is located in the spectral band of maximum value. Total spectral intensity of white tongue coating Maximum spectral intensity of yellow tongue coating The band containing the maximum spectral intensity of yellow tongue coating The sum of the spectral intensities of yellow tongue coating .

[0043] in, This indicates the maximum spectral intensity of the white tongue coating. This indicates the spectral intensity band where the white tongue coating reaches its maximum. express inverse function, This represents the sum of the intensities of all bands on the curve representing the white tongue coating, i.e., the sum of the spectral intensities of the white tongue coating. This indicates the maximum spectral intensity of the yellow tongue coating. This indicates the spectral intensity band where the yellow tongue coating reaches its maximum. express inverse function, This represents the sum of the intensities of all bands on the curve for the yellow tongue coating, i.e., the sum of the spectral intensities of the yellow tongue coating.

[0044] (3) Using the CIE color standard, determine the RGB color parameters of the white tongue coating and the yellow tongue coating respectively based on the spectral curve of the white tongue coating and the spectral curve of the yellow tongue coating.

[0045] Specifically, using the CIE color standard, the corresponding bands are selected from the spectral curve of the location of the yellow tongue coating to obtain the RGB color parameters of the yellow tongue coating, which are denoted as follows: , representing the red, green, and blue parameters of the yellow tongue coating; using the CIE color standard, the corresponding bands are selected from the spectral curve of the location of the white tongue coating to obtain the RGB color parameters of the white tongue coating, denoted as: , representing the red parameter, green parameter, and blue parameter for white tongue coating.

[0046] (4) The RGB color parameters of the white tongue coating are converted to obtain the HSV color parameters of the white tongue coating, and the RGB color parameters of the yellow tongue coating are converted to obtain the HSV color parameters of the yellow tongue coating.

[0047] Specifically, the HSV color parameters of the yellow tongue coating are calculated from the RGB color parameters of the yellow tongue coating using a conversion formula, and are denoted as follows: , representing the hue parameter, saturation parameter, and value parameter of the yellow tongue coating; the HSV color parameter of the white tongue coating is calculated from the RGB color parameters of the white tongue coating using a conversion formula, and is denoted as: , representing the hue parameter, saturation parameter, and value parameter of the white tongue coating.

[0048] In this embodiment, after obtaining the tongue coating area, the color characteristics of the tongue coating area can also be used to analyze whether it is tongue coating on the tongue body. Using the CIE color standard, the values ​​of the 450nm, 550nm, and 700nm wavelengths are selected from the spectral curves of the locations of yellow and white tongue coatings, and are denoted as follows: The HSV (Hue, Saturation, Value) values ​​are recovered from the RGB (Red, Green, Blue) parameters using a conversion formula, denoted as: .

[0049] Fissured tongue refers to the presence of cracks of varying depths and shapes on the tongue. Traditional Chinese medicine believes that "spleen and stomach deficiency with dampness and weakness in digestion" will lead to fissured tongue. It is an important spatial feature of tongue observation in traditional Chinese medicine, and can be regarded as the texture feature of data in hyperspectral data.

[0050] The method for determining the spatial characteristics of the crack includes: (1) Extract and binarize the tongue region information based on the first principal component to obtain the tongue information matrix. .

[0051] Specifically, the Region of Interest (ROI) in the tongue region of the first principal component result is selected, and the region is binarized and denoted as a two-dimensional matrix. ,definition The total number of elements is N, and the number of closed intervals is S.

[0052] (2) The tongue information matrix Edge detection is performed to obtain the crack edge matrix. .

[0053] in, and Both are two-dimensional matrices with length and width equal to the region in the first principal component tongue and consisting of 0s and 1s. A pixel value of 1 indicates the location where the crack appears; This is the result of enhancing the crack edge, with the pixel value at the edge position being 1 and the rest being 0; (3) Determine the crack length based on the total number of crack pixels in the information matrix of the tongue, i.e. .

[0054] (4) Determine the crack density based on the crack length and the total number of pixels in the information matrix of the tongue, i.e. .

[0055] (5) Determine the average crack length based on the crack length and the number of closed intervals in the crack edge matrix, i.e. .

[0056] (6) Determine the crack perimeter based on the total number of crack edge pixels in the crack edge matrix, i.e. .

[0057] (7) Determine the crack compactness based on the crack perimeter and the crack length, i.e. .

[0058] Where N is the total number of pixels in the information matrix of the tongue, S is the number of closed intervals in the crack edge matrix, L is the crack length, and D is the crack density. The average crack length is... The crack perimeter is... For crack compactness.

[0059] Furthermore, through a minimum pair of ellipses The ellipse encloses all non-zero regions, and the ellipse can be exactly aligned with... The non-zero region boundary is tangent; the angle A between the major axis direction of the ellipse (defined as "from bottom to top" as the positive direction) and the horizontal direction (defined as "horizontal to the right" as the positive direction) is denoted as the crack direction; the ratio of the major axis to the minor axis of the ellipse is... , denoted as crack shape factor.

[0060] Scalloped tongue refers to the appearance of teeth marks on the edge of the tongue. It is one of the abnormal tongue shapes in tongue diagnosis. It is caused by the enlarged tongue being compressed by the teeth. Clinically, it manifests as spleen deficiency. In hyperspectral data, it can be regarded as the edge feature of the data.

[0061] The method for determining the spatial features of the tooth marks includes: (1) Extract and binarize the tongue side region information based on the first principal component to obtain the tongue side information matrix. .

[0062] Specifically, the tongue-side portion of the first principal component is extracted and binarized to obtain... .

[0063] (2) Perform opening and closing operations on the tongue side information matrix in sequence to obtain the tongue side morphology matrix. .

[0064] Specifically, for First perform the closing operation, then perform the opening operation to obtain... .in, and Both are two-dimensional matrices with length and width equal to the region in the first principal component tongue and composed of 0 and 1.

[0065] (3) From the lingual morphological matrix Extracting edge coordinates and performing moving average filtering yields a set of edge row coordinates and a set of edge column coordinates, denoted as […]. and , where x is the index of the edge point.

[0066] In this embodiment, since the teeth marks appear on the tongue side, the tongue side region ROI of the first principal component result is selected for subsequent feature extraction. Binarization is performed, and the result contains some noise, requiring preprocessing to remove holes and connect edge images. Morphological closing operations can fill small holes and mend small cracks, while opening operations can remove small holes at the edges and break narrow edge connections. To smooth the edges and reduce burrs while preserving the tongue's edge features, this embodiment uses a disk structure with a radius of 10 to perform closing operations followed by opening operations. Subsequently, the edge coordinates of the binary boundary in the result are extracted, and a 5-point moving average filter is used for data smoothing to obtain a set of row coordinates. and column coordinate set .

[0067] (4) Determine the concavity / convexity of edge pixels based on the set of edge row coordinates and the set of edge column coordinates, and identify edge pixels whose concavity / convexity exceeds a set threshold as serrated concavity / convexity points, thereby obtaining a set of serrated concavity / convexity points, specifically formulated as follows: .

[0068] in, This represents the degree of concavity (i.e., curvature) of the x-th edge pixel. This is the gradient operator.

[0069] (5) The number of all tooth marks in the set of tooth marks is determined as the number of tooth marks.

[0070] (6) Determine a triangle using any three tooth marks in the set of tooth marks and use the midpoint of these three tooth marks as the vertex. Calculate the width, height, and apex angle of the triangle to obtain the width array, height array, and apex angle array.

[0071] (7) The maximum, minimum and average values ​​in the wide array are respectively determined as the maximum tooth width, minimum tooth width and average tooth width.

[0072] (8) The maximum, minimum and average values ​​in the high array are respectively determined as the maximum tooth depth, minimum tooth depth and average tooth depth.

[0073] (9) The maximum, minimum and average values ​​in the vertex angle array are respectively determined as the maximum tooth mark angle, the minimum tooth mark angle and the average tooth mark angle.

[0074] In this embodiment, Points greater than 0.005 or less than -0.01 are denoted as serration points. Let P be the number of serration points, i.e., the number of serrations. Every three points can form a triangle. The width, height, and vertex angle of the triangle are defined as the serration width, serration depth, and serration angle, respectively. From this, the number of points containing... The dent depth vector H, dent width vector W, and dent angle vector A of each element are used as characteristic parameters, with the maximum, minimum, and mean values ​​of these three vectors retained.

[0075] Tongue shape is an important parameter in Traditional Chinese Medicine (TCM) diagnosis. The length-to-width ratio of the tongue can indicate whether it is swollen or enlarged, which clinically suggests spleen and stomach weakness. A tongue with a pointed tip, not round, or with a notch indicates excessive internal heat and insufficient spleen and kidney yang, which can be judged by the angle between the tip and the tongue.

[0076] The method for determining the spatial features of the tongue shape includes: (1) The hyperspectral image to be tested is mapped to obtain a tongue body distribution matrix; the pixel value of the tongue body part in the tongue body distribution matrix is ​​1, and the pixel value of the background part is 0.

[0077] Specifically, a hyperspectral image of the tongue is selected, the background is removed, the tongue portion is set to 1, and the background portion is set to 0, resulting in a binary image of the tongue. That is, the tongue body distribution matrix.

[0078] (2) Determine the minimum rectangle according to the tongue distribution matrix; the minimum rectangle wraps all non-zero regions in the tongue distribution matrix and is tangent to the non-zero regions.

[0079] (3) The ratio of the length to the width of the smallest rectangle is determined as the aspect ratio.

[0080] Specifically, through a minimum rectangle pair The rectangle encloses all non-zero regions, and the rectangle can be exactly aligned with... Tangent to the non-zero region boundary in the rectangle. The ratio of the rectangle's length to its width is... , denoted as aspect ratio.

[0081] (4) Extract the edges of the tongue distribution matrix to obtain the tongue edge matrix; the pixel value of the tongue edge part in the tongue edge matrix is ​​1, and the pixel value of other parts other than the edge is 0.

[0082] Specifically, for After edge extraction, the result is That is, the tongue edge matrix. Among them, and All are two-dimensional matrices with length and width equal to the hyperspectral images of the tongue body and consisting of 0s and 1s. The region where the pixel value is 1 is the area where the tongue is located; The position with a pixel value of 1 is the position of the edge of the tongue.

[0083] (5) Determine the tongue tip angle based on the position of the bottommost pixel in the tongue edge matrix and the positions of the left and right adjacent pixels; the bottommost pixel is the pixel with a pixel value of 1 located at the bottom of the tongue edge matrix; the left and right adjacent pixels include: a pixel with a pixel value of 1 located at a set distance to the left of the bottommost pixel and a pixel with a pixel value of 1 located at a set distance to the right of the bottommost pixel.

[0084] In this embodiment, take Take the 10th pixel on each side of the bottommost pixel. Then, find the angle between the bottommost pixel and the 10th pixel on each side. , which is recorded as the angle between the tip of the tongue.

[0085] Furthermore, this tongue image feature extraction method also includes: Step S4: Input the spatial spectral features of the tongue body to be tested into the tongue image classification model to obtain the classification result of the tongue body to be tested; the classification result of the tongue body to be tested includes at least one of the following: whether the tongue body to be tested is a thick and greasy tongue, whether the tongue body to be tested is a teeth-marked tongue, and whether the tongue body to be tested is a fissured tongue; the tongue image classification model includes a thick and greasy tongue classification model, a teeth-marked tongue classification model, and a fissured tongue classification model in parallel; the thick and greasy tongue classification model, the teeth-marked tongue classification model, and the fissured tongue classification model are all binary classification models.

[0086] The method for determining the tongue image classification model specifically includes: Step S401: Obtain a sample dataset; the sample dataset includes several sample hyperspectral images and corresponding category labels; the sample hyperspectral images are hyperspectral images of the tongue body of the sample; the category labels include: thick and greasy tongue category label, teeth-marked tongue category label, and cracked tongue category label.

[0087] Step S402: Principal component analysis is used to extract data features from the hyperspectral image of the sample to obtain the first principal component, the second principal component, and the third principal component of the tongue of the sample.

[0088] Step S403: Determine the spatial spectral features of the sample tongue body based on the first principal component, the second principal component, and the third principal component of the sample tongue body.

[0089] Step S404: The following parameters are determined as characteristic parameters of a thick, greasy tongue coating: the white tongue coating's material-to-texture ratio parameter, the yellow tongue coating's material-to-texture ratio parameter, the sum of the yellow tongue coating's spectral intensities, the maximum value of the yellow tongue coating's spectral intensities, the band containing the maximum value of the yellow tongue coating's spectral intensities, the RGB color parameters of the yellow tongue coating, the HSV color parameters of the yellow tongue coating, the sum of the white tongue coating's spectral intensities, the maximum value of the white tongue coating's spectral intensities, the band containing the maximum value of the white tongue coating's spectral intensities, the RGB color parameters of the white tongue coating, and the HSV color parameters of the white tongue coating.

[0090] Step S405: The number of tooth marks, the average tooth width, the maximum tooth width, the minimum tooth width, the average tooth depth, the maximum tooth depth, the minimum tooth depth, the average tooth mark angle, the maximum tooth mark angle, the minimum tooth mark angle, the aspect ratio, and the tongue tip angle are determined as the tooth mark tongue characteristic parameters.

[0091] Step S406: The crack length, crack density, average crack length, crack perimeter, and crack compactness are determined as crack tongue characteristic parameters.

[0092] Step S407: Train a random forest classifier based on the thick, greasy tongue feature parameters of the sample tongue and the corresponding category labels to obtain a thick, greasy tongue classification model.

[0093] Step S408: Train a random forest classifier based on the tooth-marked tongue feature parameters and corresponding category labels of the sample tongue body to obtain a tooth-marked tongue classification model.

[0094] Step S409: Train a random forest classifier based on the cracked tongue feature parameters and corresponding category labels of the sample tongue to obtain a cracked tongue classification model.

[0095] In this embodiment, to demonstrate the classification performance of the extracted spatial spectral feature parameters on common pathological tongues, the training set and test set are divided in a 7:3 ratio by combining multiple feature parameters. Random forest is used as a classifier to train a binary classification model to determine whether the tongue is a thick, greasy tongue, a tongue with teeth marks, or a tongue with fissures. The model is then tested.

[0096] 1. The characteristic parameters for judging the combination of teeth marks and tongue include: number of teeth marks, average tooth width, maximum tooth width, minimum tooth width, average tooth depth, maximum tooth depth, minimum tooth depth, average tooth mark angle, maximum tooth mark angle, minimum tooth mark angle, length-to-width ratio, and tongue tip angle.

[0097] 2: The characteristic parameters for judging the combination of crack tongues are: crack length, crack density, average crack length, crack perimeter, and crack compactness.

[0098] 3. The characteristic parameters for judging a thick, greasy tongue coating include: the ratio of white tongue coating to substance, the ratio of yellow tongue coating to substance, the total spectral intensity of yellow tongue coating, the maximum spectral intensity of yellow tongue coating, the band containing the maximum spectral intensity of yellow tongue coating, the total spectral intensity of white tongue coating, the maximum spectral intensity of white tongue coating, the band containing the maximum spectral intensity of white tongue coating, the RGB color parameters of yellow tongue coating, the HSV color parameters of yellow tongue coating, and the RGB color parameters and HSV color parameters of white tongue coating.

[0099] The classification results of the feature parameters extracted using the method of this invention for scalloped tongue, fissured tongue, and thick, greasy tongue are as follows: Figure 3 As shown in the figure. This embodiment uses AUC (Area Under the Curve) as the evaluation metric. A closer AUC to 1 indicates better classification performance, and an AUC greater than 0.75 indicates a relatively good result. Figure 3 It can be seen that the method of the present invention achieves very good results in the classification of pathological tongue bodies in this embodiment.

[0100] Example 2 To implement the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, a tongue image feature extraction system based on hyperspectral images is provided below. For example... Figure 2 As shown, the tongue image feature extraction system includes: Acquisition module 1 is used to acquire the hyperspectral image to be tested; the hyperspectral image to be tested is a hyperspectral image of the tongue part of the tongue to be tested.

[0101] The data feature extraction module 2 is used to extract data features from the hyperspectral image to be tested using principal component analysis to obtain the first principal component, the second principal component, and the third principal component of the tongue body to be tested; the first principal component, the second principal component, and the third principal component are all two-dimensional matrices with the same spatial resolution as the hyperspectral image to be tested.

[0102] The spatial spectral feature determination module 3 is used to determine the spatial spectral features of the tongue under test based on the first principal component, the second principal component, and the third principal component of the tongue under test, as well as the hyperspectral image under test. The spatial spectral features include: tongue coating spatial features, tongue coating spectral and color features, crack spatial features, tooth mark spatial features, and tongue shape spatial features. The tongue coating spatial features include: white tongue coating material-to-texture ratio parameters and yellow tongue coating material-to-texture ratio parameters. The tongue coating spectral and color features include: the sum of yellow tongue coating spectral intensities, the maximum value of yellow tongue coating spectral intensity, the band containing the maximum value of yellow tongue coating spectral intensity, yellow tongue coating RGB color parameters, and yellow tongue coating HSV color parameters. The parameters include: total spectral intensity of white tongue coating, maximum spectral intensity of white tongue coating, spectral band containing the maximum spectral intensity of white tongue coating, RGB color parameters of white tongue coating, and HSV color parameters of white tongue coating; the spatial characteristics of cracks include: crack length, crack density, average crack length, crack circumference, and crack compactness; the spatial characteristics of tooth marks include: number of tooth marks, average tooth width, maximum tooth width, minimum tooth width, average tooth depth, maximum tooth depth, minimum tooth depth, average tooth mark angle, maximum tooth mark angle, and minimum tooth mark angle; the spatial characteristics of tongue shape include: aspect ratio and tongue tip angle; the spatial spectral characteristics of the tongue body under test are used to classify the tongue body under test.

[0103] Furthermore, the aforementioned tongue image feature extraction system also includes: The classification module is used to input the spatial spectral features of the tongue body to be tested into the tongue image classification model to obtain the classification result of the tongue body to be tested; the classification result of the tongue body to be tested includes at least one of the following: whether the tongue body to be tested is a thick and greasy tongue, whether the tongue body to be tested is a teeth-marked tongue, and whether the tongue body to be tested is a fissured tongue; the tongue image classification model includes a thick and greasy tongue classification model, a teeth-marked tongue classification model, and a fissured tongue classification model in parallel; the thick and greasy tongue classification model, the teeth-marked tongue classification model, and the fissured tongue classification model are all binary classification models.

[0104] Example 3 This invention also provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to perform the tongue feature extraction method based on hyperspectral images as described in Embodiment 1. The electronic device may be a server.

[0105] In addition, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the tongue image feature extraction method based on hyperspectral images in Embodiment 1.

[0106] In summary, this invention provides a method and system for tongue feature extraction based on hyperspectral images. It can extract tongue color, texture, and edge features that are key observations in Traditional Chinese Medicine (TCM) theory, and analyze them in digital form. This invention, based on TCM theory and combined with hyperspectral data, increases the amount of information obtained about the tongue, describes tongue features using digital parameters, and provides a new approach for the digitization of TCM tongue imaging.

[0107] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0108] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A tongue image feature extraction method based on hyperspectral image, characterized in that, include: Acquire a hyperspectral image to be tested; the hyperspectral image to be tested is a hyperspectral image of the tongue body portion of the tongue body to be tested. Principal component analysis was used to extract data features from the hyperspectral image to be tested, resulting in the first principal component, the second principal component, and the third principal component of the tongue body to be tested. The first principal component, the second principal component, and the third principal component are all two-dimensional matrices with the same spatial resolution as the hyperspectral image to be tested. The spatial-spectral characteristics of the tongue under test are determined based on the first principal component, the second principal component, and the third principal component of the tongue under test, as well as the hyperspectral image of the tongue under test. The spatial spectral features include: tongue coating spatial features, tongue coating spectral and color features, crack spatial features, teeth mark spatial features, and tongue shape spatial features; the tongue coating spatial features include: white tongue coating material-to-paste ratio parameters and yellow tongue coating material-to-paste ratio parameters; the tongue coating spectral and color features include: the sum of yellow tongue coating spectral intensities, the maximum value of yellow tongue coating spectral intensity, the band containing the maximum value of yellow tongue coating spectral intensity, yellow tongue coating RGB color parameters, yellow tongue coating HSV color parameters, the sum of white tongue coating spectral intensities, the maximum value of white tongue coating spectral intensity, and the maximum value of white tongue coating spectral intensity. The maximum value band, RGB color parameters of white tongue coating, and HSV color parameters of white tongue coating; the crack spatial characteristics include: crack length, crack density, average crack length, crack circumference, and crack compactness; the tooth mark spatial characteristics include: number of tooth marks, average tooth width, maximum tooth width, minimum tooth width, average tooth depth, maximum tooth depth, minimum tooth depth, average tooth mark angle, maximum tooth mark angle, and minimum tooth mark angle; the tongue shape spatial characteristics include: aspect ratio and tongue tip angle; the spatial spectral characteristics of the tongue body under test are used to classify the tongue body under test; The method for determining the spatial features of the tongue coating includes: performing binarization processing on the first principal component, the second principal component, and the third principal component respectively to obtain a first binary matrix, a second binary matrix, and a third binary matrix; performing pixel-by-pixel multiplication on the second binary matrix and the first binary matrix to obtain a tongue coating distribution matrix; performing pixel-by-pixel multiplication on the third binary matrix and the tongue coating distribution matrix to obtain a yellow tongue coating distribution matrix; determining a white tongue coating distribution matrix based on the tongue coating distribution matrix and the yellow tongue coating distribution matrix; determining a white tongue coating quality ratio parameter based on the total number of white tongue coating pixels and the total number of tongue body pixels in the white tongue coating distribution matrix; and determining a yellow tongue coating quality ratio parameter based on the total number of yellow tongue coating pixels and the total number of tongue body pixels in the yellow tongue coating distribution matrix.

2. The tongue image feature extraction method based on hyperspectral images according to claim 1, characterized in that, The methods for determining the spectral and color characteristics of the tongue coating include: Based on the white tongue coating distribution matrix and the yellow tongue coating distribution matrix, the white tongue coating spectral curve and the yellow tongue coating spectral curve are extracted from the hyperspectral image to be tested, respectively. Based on the spectral curves of the white tongue coating and the yellow tongue coating, determine the maximum spectral intensity of the white tongue coating, the band containing the maximum spectral intensity of the white tongue coating, the total spectral intensity of the white tongue coating, the maximum spectral intensity of the yellow tongue coating, the band containing the maximum spectral intensity of the yellow tongue coating, and the total spectral intensity of the yellow tongue coating, respectively. Using the CIE color standard, the RGB color parameters of the white tongue coating and the yellow tongue coating are determined based on the spectral curves of the white tongue coating and the yellow tongue coating, respectively. The RGB color parameters of the white tongue coating are converted to obtain the HSV color parameters of the white tongue coating, and the RGB color parameters of the yellow tongue coating are converted to obtain the HSV color parameters of the yellow tongue coating.

3. The tongue image feature extraction method based on hyperspectral images according to claim 1, characterized in that, The method for determining the spatial characteristics of the crack includes: Based on the first principal component, tongue region information is extracted and binarized to obtain the tongue information matrix; Edge detection is performed on the information matrix in the tongue to obtain the crack edge matrix; The crack length is determined based on the total number of crack pixels in the information matrix of the tongue. The crack density is determined based on the crack length and the total number of pixels in the information matrix of the tongue. The average crack length is determined based on the crack length and the number of closed intervals in the crack edge matrix; The crack perimeter is determined based on the total number of crack edge pixels in the crack edge matrix. Crack compactness is determined based on the crack perimeter and the crack length.

4. The tongue image feature extraction method based on hyperspectral images according to claim 1, characterized in that, The method for determining the spatial features of the tooth marks includes: Based on the first principal component, tongue side region information is extracted and binarized to obtain the tongue side information matrix; The tongue side information matrix is ​​sequentially subjected to opening and closing operations to obtain the tongue side morphology matrix; Edge coordinates are extracted from the tongue-side morphological matrix and then subjected to moving average filtering to obtain a set of edge row coordinates and a set of edge column coordinates. The concavity and convexity of edge pixels are determined based on the set of edge row coordinates and the set of edge column coordinates, and edge pixels whose concavity and convexity exceed a set threshold are identified as tooth mark concavity and convexity points, thus obtaining a set of tooth mark concavity and convexity points. The number of all tooth marks and bumps in the set of tooth marks and bumps is determined as the number of tooth marks; A triangle is determined by any three tooth marks in the set of tooth marks, and the width, height and vertex angle of the triangle are calculated with the middle point as the vertex, so as to obtain the width array, height array and vertex angle array. The maximum, minimum, and average values ​​in the wide array are respectively determined as the maximum tooth width, minimum tooth width, and average tooth width; The maximum, minimum, and average values ​​in the high-order array are respectively determined as the maximum tooth depth, minimum tooth depth, and average tooth depth; The maximum, minimum, and average values ​​in the vertex angle array are respectively determined as the maximum tooth mark angle, the minimum tooth mark angle, and the average tooth mark angle.

5. The tongue image feature extraction method based on hyperspectral images according to claim 1, characterized in that, The method for determining the spatial features of the tongue shape includes: The hyperspectral image to be tested is mapped to obtain a tongue body distribution matrix; the pixel value of the tongue body part in the tongue body distribution matrix is ​​1, and the pixel value of the background part is 0. The minimum rectangle is determined based on the tongue distribution matrix; the minimum rectangle encloses all non-zero regions in the tongue distribution matrix and is tangent to the non-zero regions. The ratio of the length to the width of the smallest rectangle is defined as the aspect ratio. Edge extraction is performed on the tongue distribution matrix to obtain the tongue edge matrix; the pixel value of the tongue edge part in the tongue edge matrix is ​​1, and the pixel value of other parts is 0. The angle between the tongue tip and the tongue edge is determined based on the position of the bottommost pixel in the tongue edge matrix and the positions of the left and right adjacent pixels. The bottommost pixel is the pixel with a pixel value of 1 located at the bottom of the tongue edge matrix. The left and right adjacent pixels include: a pixel with a pixel value of 1 located at a predetermined distance to the left of the bottommost pixel and a pixel with a pixel value of 1 located at a predetermined distance to the right of the bottommost pixel.

6. The tongue image feature extraction method based on hyperspectral images according to claim 1, characterized in that, Also includes: The spatial spectral features of the tongue to be tested are input into the tongue image classification model to obtain the classification result of the tongue to be tested; The classification results of the tongue to be tested include at least one of the following: whether the tongue to be tested is a thick, greasy tongue, whether the tongue to be tested is a scalloped tongue, and whether the tongue to be tested is a fissured tongue; the tongue image classification model includes parallel thick, greasy tongue classification models, scalloped tongue classification models, and fissured tongue classification models; the thick, greasy tongue classification model, the scalloped tongue classification model, and the fissured tongue classification model are all binary classification models.

7. The tongue image feature extraction method based on hyperspectral images according to claim 6, characterized in that, The method for determining the tongue image classification model specifically includes: Obtain a sample dataset; the sample dataset includes several sample hyperspectral images and corresponding category labels; the sample hyperspectral images are hyperspectral images of the tongue body of the sample; the category labels include: thick and greasy tongue category label, teeth-marked tongue category label, and cracked tongue category label; Principal component analysis was used to extract data features from the hyperspectral image of the sample to obtain the first principal component, the second principal component, and the third principal component of the tongue of the sample. The spatial spectral characteristics of the sample tongue are determined based on the first principal component, the second principal component, and the third principal component of the sample tongue. The following parameters are defined as characteristic parameters of a thick, greasy tongue coating: the white tongue coating's material-to-texture ratio, the yellow tongue coating's material-to-texture ratio, the sum of the yellow tongue coating's spectral intensities, the maximum value of the yellow tongue coating's spectral intensities, the band containing the maximum value of the yellow tongue coating's spectral intensities, the RGB color parameters of the yellow tongue coating, the HSV color parameters of the yellow tongue coating, the sum of the white tongue coating's spectral intensities, the maximum value of the white tongue coating's spectral intensities, the band containing the maximum value of the white tongue coating's spectral intensities, the RGB color parameters of the white tongue coating, and the HSV color parameters of the white tongue coating. The number of tooth marks, the average tooth width, the maximum tooth width, the minimum tooth width, the average tooth depth, the maximum tooth depth, the minimum tooth depth, the average tooth mark angle, the maximum tooth mark angle, the minimum tooth mark angle, the aspect ratio, and the tongue tip angle are determined as the characteristic parameters of the tooth mark tongue. The crack length, crack density, average crack length, crack perimeter, and crack compactness are defined as characteristic parameters of the crack tongue. A random forest classifier is trained based on the thick, greasy tongue feature parameters and corresponding category labels of the sample tongue to obtain a thick, greasy tongue classification model; A random forest classifier is trained based on the tooth-marked tongue feature parameters and corresponding category labels of the sample tongue to obtain a tooth-marked tongue classification model; A random forest classifier is trained based on the cracked tongue feature parameters and corresponding category labels of the sample tongue to obtain a cracked tongue classification model.

8. A tongue image feature extraction system based on hyperspectral images, characterized in that, include: The acquisition module is used to acquire the hyperspectral image to be tested; the hyperspectral image to be tested is a hyperspectral image of the tongue body portion of the tongue body to be tested; The data feature extraction module is used to extract data features from the hyperspectral image to be tested using principal component analysis to obtain the first principal component, the second principal component, and the third principal component of the tongue body to be tested; the first principal component, the second principal component, and the third principal component are all two-dimensional matrices with the same spatial resolution as the hyperspectral image to be tested. The spatial spectral feature determination module is used to determine the spatial spectral features of the tongue body under test based on the first principal component, the second principal component, and the third principal component of the tongue body under test and the hyperspectral image under test. The spatial characteristics include: tongue coating spatial characteristics, tongue coating spectral and color characteristics, crack spatial characteristics, tooth mark spatial characteristics, and tongue shape spatial characteristics; the tongue coating spatial characteristics include: white tongue coating material ratio parameters and yellow tongue coating material ratio parameters; the tongue coating spectral and color characteristics include: total yellow tongue coating spectral intensity, maximum yellow tongue coating spectral intensity, the band containing the maximum yellow tongue coating spectral intensity, yellow tongue coating RGB color parameters, yellow tongue coating HSV color parameters; the white tongue coating spectral intensity includes: total white tongue coating spectral intensity, maximum white tongue coating spectral intensity, the band containing the maximum white tongue coating spectral intensity, white tongue coating RGB color parameters, and white tongue coating HSV color parameters; the crack spatial characteristics include: crack length, crack density, average crack length, crack circumference, and crack compactness; the tooth mark spatial characteristics include: number of tooth marks, average tooth width, maximum tooth width, minimum tooth width, average tooth depth, maximum tooth depth, minimum tooth depth, average tooth mark angle, maximum tooth mark angle, and maximum tooth mark angle. The small teeth mark angle; the tongue shape spatial features include: aspect ratio and tongue tip angle; the spatial spectral features of the tongue body to be tested are used to classify the tongue body to be tested; the method for determining the tongue coating spatial features includes: performing binarization processing on the first principal component, the second principal component and the third principal component respectively to obtain a first binary matrix, a second binary matrix and a third binary matrix; performing pixel-by-pixel multiplication operation on the second binary matrix and the first binary matrix to obtain a tongue coating distribution matrix; performing pixel-by-pixel multiplication operation on the third binary matrix and the tongue coating distribution matrix to obtain a yellow tongue coating distribution matrix; determining a white tongue coating distribution matrix based on the tongue coating distribution matrix and the yellow tongue coating distribution matrix; determining the white tongue coating texture ratio parameter based on the total number of white tongue coating pixels and the total number of tongue body pixels in the white tongue coating distribution matrix; determining the yellow tongue coating texture ratio parameter based on the total number of yellow tongue coating pixels and the total number of tongue body pixels in the yellow tongue coating distribution matrix.

9. The tongue image feature extraction system based on hyperspectral images according to claim 8, characterized in that, Also includes: The classification module is used to input the spatial spectral features of the tongue body to be tested into the tongue image classification model to obtain the classification result of the tongue body to be tested; The classification results of the tongue to be tested include at least one of the following: whether the tongue to be tested is a thick, greasy tongue, whether the tongue to be tested is a scalloped tongue, and whether the tongue to be tested is a fissured tongue; the tongue image classification model includes parallel thick, greasy tongue classification models, scalloped tongue classification models, and fissured tongue classification models; the thick, greasy tongue classification model, the scalloped tongue classification model, and the fissured tongue classification model are all binary classification models.

Images