A tongue image processing method for traditional Chinese medicine tongue diagnosis based on computer vision

Abstract

The present invention relates to the field of image processing, and in particular to a computer vision-based tongue image processing method for traditional Chinese medicine tongue diagnosis. The method comprises the following steps: photographing a tongue image and performing image preprocessing on the tongue image to obtain a preprocessed tongue image; separating the tongue region from the preprocessed tongue image to obtain a segmented tongue image; enhancing the segmented tongue image using an adaptive texture-color coupling enhancement algorithm to obtain an enhanced tongue image; and obtaining a prediction result of the tongue diagnosis category based on the enhanced tongue image. The method solves the problems of insufficient intelligence and standardization of traditional Chinese medicine tongue diagnosis, the presence of complex background interference in image processing, and the insufficient distinguishability of tongue diagnosis image features.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a computer vision-based tongue image processing method for traditional Chinese medicine tongue diagnosis. Background Art

[0002] Traditional Chinese Medicine (TCM) tongue diagnosis theory primarily infers a person's health by observing the tongue's color, shape, coating thickness, and moistness. Traditional tongue diagnosis relies on the practitioner's visual observation and clinical experience, often affected by individual differences and environmental factors. With advances in computer vision technology, a growing number of studies are exploring automated tongue image analysis through image processing. Leveraging modern technology, these efforts aim to achieve more objective and accurate diagnoses, furthering the scientific and precise nature of traditional TCM tongue diagnosis. With continued technological advancements, the automation and intelligence of tongue diagnosis will continue to improve, and it is expected to play an increasingly important role in clinical practice, contributing to the digital transformation of TCM health management.

[0003] However, the above-mentioned tongue image processing method for traditional Chinese medicine tongue diagnosis still has problems such as insufficient intelligence and standardization, complex background interference in image processing, and insufficient distinguishability of tongue diagnosis image features. Summary of the Invention

[0004] The present invention provides a computer vision-based tongue image processing method for traditional Chinese medicine tongue diagnosis, so as to solve the problems of insufficient intelligence and standardization of traditional Chinese medicine tongue diagnosis, complex background interference in image processing, and insufficient distinguishability of tongue diagnosis image features.

[0005] The present invention provides a computer vision-based tongue image processing method for traditional Chinese medicine tongue diagnosis, which specifically includes the following technical solutions:

[0006] S1, photographing a tongue image, and performing image preprocessing on the tongue image to obtain a preprocessed tongue image; separating a tongue region from the preprocessed tongue image to obtain a segmented tongue image;

[0007] S2. Enhance the segmented tongue image using an adaptive texture-color coupling enhancement algorithm to obtain an enhanced tongue image; based on the enhanced tongue image, obtain a prediction result of the tongue diagnosis category.

[0008] Preferably, the S1 specifically includes:

[0009] During the image preprocessing process, the illumination deviation of the tongue image is corrected to obtain a color-corrected tongue image; the color-corrected tongue image is subjected to noise reduction processing to obtain a noise-reduced tongue image; and the noise-reduced tongue image is subjected to image standardization processing to obtain a preprocessed tongue image.

[0010] Preferably, the S1 specifically includes:

[0011] Based on the preprocessed tongue image, an initial binary mask is generated. The initial binary mask is used as the initial label of the tongue area and the background, and the tongue area and background are separated through iterative optimization to obtain the final binary mask. The preprocessed tongue image and the final binary mask are element-wise multiplied to obtain the segmented tongue image.

[0012] Preferably, the S2 specifically includes:

[0013] In the implementation process of the adaptive texture-color coupling enhancement algorithm, the segmented tongue image is divided into grids. In each grid, the horizontal gradient and vertical gradient of the pixel are calculated respectively; based on the horizontal gradient and vertical gradient of the pixel, the gradient amplitude of the pixel is obtained; the gradient amplitude of the pixels of all grids is globally normalized to obtain the normalized gradient value of the pixel, and a normalized gradient map is generated.

[0014] Preferably, the S2 specifically includes:

[0015] Based on the normalized gradient value of the pixels in the grid, the grid variance is calculated and normalized. Combined with the normalized gradient value of the pixel, the texture feature value of the pixel is calculated; based on the texture feature value of the pixel, a texture feature map is constructed.

[0016] Preferably, the S2 specifically includes:

[0017] In the implementation process of the adaptive texture-color coupling enhancement algorithm, the number of pixels with 256 gray levels in each of the red, green, and blue channels of the segmented tongue image is counted, and histograms are constructed respectively; the histogram of each channel is normalized to obtain a normalized histogram, and the normalized histograms of the three channels are used as the global color distribution features.

[0018] Preferably, the S2 specifically includes:

[0019] Based on the normalized histogram, the color frequency of each pixel in the segmented tongue image is calculated, and a negative exponential function is introduced to perform nonlinear regulation of the color frequency.

[0020] Preferably, the S2 specifically includes:

[0021] Based on the texture feature value of the pixel and the negative exponential function, and introducing the enhancement weight, the enhancement amplitude is obtained; based on the enhancement amplitude and the pixel value of the segmented tongue image, the pixel value of the enhanced tongue image is obtained; based on the pixel value of the enhanced tongue image, the enhanced tongue image is constructed.

[0022] Preferably, the S2 specifically includes:

[0023] The enhanced tongue image is subjected to feature extraction and classification processing using a pre-trained convolutional neural network to obtain the prediction results of the tongue diagnosis category.

[0024] The beneficial effects of the technical solution of the present invention are:

[0025] 1. The segmented tongue image is divided into several grids. The Sobel operator is used in each grid to calculate the horizontal and vertical gradients of the pixels. The pixel gradient amplitude is obtained to reflect the local texture intensity (such as cracks or tongue coating boundaries). This method accurately captures the details of texture changes in the tongue image and highlights key diagnostic features such as cracks and tongue coating boundaries.

[0026] 2. Globally normalize the gradient amplitudes of pixels in all grids to eliminate inconsistencies in gradient amplitudes caused by differences in image brightness or contrast, enhance the comparability of texture features between different grids, ensure the stability of texture features under different lighting conditions or image quality, improve the consistency and robustness of tongue image processing, and make it suitable for diverse clinical image data.

[0027] 3. Based on the normalized gradient map, the variance of the normalized gradient value in each grid is calculated as a measure of texture complexity. The amplification factor is generated and multiplied with the normalized gradient value to obtain the texture feature value. This significantly enhances the texture features related to TCM diagnosis in the tongue image (such as cracks and tongue coating), providing more reliable texture information for automated tongue diagnosis.

[0028] 4. Based on the histogram frequency of each pixel's RGB value, the color frequency is calculated, and a negative exponential function is introduced for nonlinear regulation. Low-frequency abnormal colors (such as dark purple and spots) are preferentially enhanced, achieving adaptive enhancement of abnormal colors. The enhancement of the main hue area is relatively small, highlighting the abnormal color features related to the disease in the tongue image (such as dark purple tongue), and improving the visibility and distinguishability of tongue diagnosis categories (such as "dark purple tongue").

[0029] 5. The texture feature value is multiplied by the output of the negative exponential function, and enhancement weights and bias terms are introduced to achieve fine control of the pixel value enhancement amplitude, ensuring that the pixel values ​​of the enhanced tongue image remain in a reasonable range, avoiding excessive enhancement or pixel value overflow, and improving the stability and visual quality of tongue image color enhancement. The enhanced tongue image is more suitable for clinical diagnosis and convolutional neural network processing, ensuring the reliability of tongue diagnosis category prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a flow chart of the tongue image processing method for traditional Chinese medicine tongue diagnosis based on computer vision described in the present invention. DETAILED DESCRIPTION

[0031] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0032] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0033] The following describes in detail a specific solution of a tongue image processing method for traditional Chinese medicine tongue diagnosis based on computer vision provided by the present invention in conjunction with the accompanying drawings.

[0034] Refer to the attached Figure 1 , which shows a flow chart of a tongue image processing method for traditional Chinese medicine tongue diagnosis based on computer vision provided by one embodiment of the present invention, the method comprising the following steps:

[0035] S1, photographing a tongue image, and performing image preprocessing on the tongue image to obtain a preprocessed tongue image; separating a tongue region from the preprocessed tongue image to obtain a segmented tongue image;

[0036] In clinical diagnosis, a high-resolution digital camera or a dedicated tongue diagnosis device is used to capture a tongue image, and the tongue image is preprocessed to obtain a preprocessed tongue image;

[0037] The image preprocessing uses a white balance algorithm to correct the illumination deviation of the tongue image to obtain a color-corrected tongue image to ensure that the tongue color can reflect the physiological state; a Gaussian blur filter is applied to the color-corrected tongue image to reduce image noise to obtain a denoised tongue image; the denoised tongue image is subjected to image standardization: the resolution of the denoised tongue image is adjusted to a fixed size (such as 512×512 pixels), and the pixel values ​​in the denoised tongue image are normalized, and each pixel value in the denoised tongue image is scaled to the interval [0,1] to obtain a preprocessed tongue image, thereby ensuring the consistency of the size and pixel value range of the preprocessed tongue image in subsequent steps and avoiding inconsistencies caused by size or brightness differences between different tongue images; the white balance algorithm, Gaussian blur filter and image standardization processing method are technical means well known to those skilled in the art and will not be described in detail here;

[0038] Furthermore, the tongue area is separated from the pre-processed tongue image, and irrelevant background (such as lips, facial background, etc.) is removed to obtain the segmented tongue image. The specific implementation process is as follows: the pre-processed tongue image is converted from the red, green, and blue (RGB) color space to the HSV color space through the color space conversion algorithm, so as to facilitate the hue-based segmentation of the pre-processed tongue image; in the HSV color space, a hue range is set by the expert experience method. (such as [0,30]), for the preprocessed tongue image, the pixel value within the hue range is marked as 1 (tongue area), and the pixel value outside the hue range is marked as 0 (background), and finally the initial binary mask is generated ; Use GrabCut algorithm to initial binary mask As the initial mark of the tongue area and the background, the tongue area and the background are separated through iterative optimization and the final binary mask is output , where 1 represents the tongue area and 0 represents the background; the color space conversion algorithm and GrabCut algorithm are well known to those skilled in the art and will not be described in detail here;

[0039] The preprocessed tongue image and the final binary mask are multiplied element by element to obtain the segmented tongue image; the color space conversion algorithm is only an intermediate calculation process for separating the tongue area, and the segmented tongue image still maintains the RGB format.

[0040] S2. Enhance the segmented tongue image using an adaptive texture-color coupling enhancement algorithm to obtain an enhanced tongue image; and obtain a tongue diagnosis category prediction result based on the enhanced tongue image.

[0041] To improve the recognizability of detailed features in segmented tongue images, especially to enhance the visual salience of tongue coating, cracks, and areas of abnormal color (such as dark purple or spots), an adaptive texture-color coupling enhancement algorithm is proposed. This algorithm uses computer vision technology to process segmented tongue images to enhance the texture and color features relevant to traditional Chinese medicine tongue diagnosis, resulting in an enhanced tongue image.

[0042] The adaptive texture-color coupling enhancement algorithm divides the segmented tongue image into several grids through gridding processing to facilitate the subsequent fine calculation of texture features; the gridding processing is a technical means well known to those skilled in the art and will not be described in detail here; in each grid, the Sobel operator is used to calculate the gradients of the pixels in the segmented tongue image in the horizontal and vertical directions respectively, and the square sum of the horizontal gradient and vertical gradient of each pixel is calculated and the square root is taken to obtain the gradient amplitude of the pixel; the gradient amplitude of the pixel reflects the intensity of the local texture (such as cracks or tongue coating boundaries) and can capture the details of the texture changes in the segmented tongue image; to ensure that the gradient amplitudes of pixels between different grids are comparable, the gradient amplitudes of the pixels of all grids are globally normalized: the gradient amplitude of each pixel is divided by the maximum gradient amplitude of the pixels in all grids to obtain the normalized gradient value of the pixel, and then a normalized gradient map is generated. The normalized gradient value ranges from 0 to 1, which can eliminate the influence of differences in image brightness or contrast;

[0043] Based on the normalized gradient map, a texture feature map is further generated. For each grid, the variance of the normalized gradient values ​​of the pixels within the grid (i.e., the grid variance) is calculated to quantify the texture complexity. After the grid variance is normalized (i.e., the grid variance is divided by the maximum value of all grid variances), it is added to a constant of 1 to generate an amplification factor ranging from 1 to 2. The amplification factor is multiplied by the normalized gradient value of the pixel and then divided by 2 to calculate the texture eigenvalue of the pixel. The value range of the texture eigenvalue of the pixel is maintained between 0 and 1. Based on the texture eigenvalue of the pixel, a texture feature map is constructed. The texture feature map comprehensively reflects the local texture intensity (e.g., cracks) and regional texture complexity (e.g., uneven distribution of tongue coating).

[0044] The calculation formula of the texture feature value of a pixel is:

[0045] ,

[0046] in, Indicated in coordinates The texture feature value of the pixel at is in the range of ; Indicated in coordinates The normalized gradient value of the pixel at ; Respectively represent the row index and column index of the grid in the segmented tongue image; Indicates the Rank Grid of columns; Represented in the grid The variance of the normalized gradient values ​​of the inner pixels, i.e., the grid variance, can reflect the complexity of the texture within the grid (such as uneven distribution of tongue coating). The calculation of the variance is a technical means well known to those skilled in the art and will not be described in detail here. Indicates the maximum value of all grid variances, which is used to normalize the grid variance to ensure the variance factor The range is ; Represents the normalization factor, which is used to normalize the texture feature value of the pixel to ; represents the amplification factor;

[0047] The adaptive texture-color coupling enhancement algorithm counts the number of pixels at 256 grayscale levels in each of the red, green, and blue (RGB) channels of the segmented tongue image and constructs a histogram for each channel. The histogram of each channel is normalized to obtain a normalized histogram so that the sum of the normalized histograms of each channel is 1. The normalized histograms of the three channels are used as global color distribution features to characterize the global color distribution in the segmented tongue image. The histogram normalization method is a technical means well known to those skilled in the art and will not be described in detail here.

[0048] Furthermore, the texture feature map and the global color distribution feature are comprehensively utilized to adaptively adjust the value of each pixel in the segmented tongue image to obtain an enhanced tongue image; the specific implementation process is as follows: for each pixel in the segmented tongue image, a texture feature value is obtained to reflect the local texture intensity and regional complexity, and based on the normalized histogram, the color distribution feature of the pixel RGB value is calculated. The specific method is to find the corresponding normalized pixel frequency in the normalized histogram of each channel for the RGB value of any pixel, and add the normalized pixel frequencies of the red, green and blue channels to obtain the color frequency. The color frequency is an indicator used to characterize the commonness of the color. The higher the color frequency, the closer the pixel color is to the main hue in the segmented tongue image (such as light red), and the lower the color frequency, the abnormal hue of the pixel color (such as dark purple, spots); a negative exponential function is introduced to perform nonlinear regulation of the color frequency. When the color frequency is higher, the output of the negative exponential function is closer to 0 and the enhancement amplitude is smaller. When the color frequency is lower, the output of the negative exponential function is larger and the enhancement amplitude is more significant, so as to ensure that abnormal color features are enhanced first;

[0049] For each pixel in the segmented tongue image, multiply the pixel's texture feature value by the output of the negative exponential function and multiply it by the enhancement weight (recommended value is 0.3) to obtain the enhancement amplitude; add the enhancement amplitude to the pixel value of the segmented tongue image and subtract the bias term (recommended value is 0.1) to obtain the pixel value of the enhanced tongue image; based on the pixel value of the enhanced tongue image, construct an enhanced tongue image; the bias term is used to ensure that the pixel value of the enhanced tongue image remains in a reasonable range to prevent pixel value overflow or excessive enhancement;

[0050] The calculation formula of the pixel value of the enhanced tongue image is:

[0051] ,

[0052] in, Indicates that the enhanced tongue image is The pixel value at ; The segmented tongue image is The pixel value at ; Indicates the enhancement weight, which can be set according to the specific implementation scenario. It is not limited here and the recommended value is 0.3; Represents a negative exponential function based on color distribution, used to adaptively adjust the enhancement amplitude; Represents color frequency, which is used to reflect the degree to which the pixel color is close to the main hue. It is used to distinguish normal colors (high frequency) from abnormal colors (low frequency) to guide the adaptive adjustment of the enhancement amplitude. Represents the normalized histogram of each RGB channel, , ; The segmented tongue image is A color channel of the pixel at Pixel value of Represents the bias term, which is used to prevent pixel value overflow caused by excessive enhancement. It can be set according to the specific implementation scenario and is not limited here. The recommended value is 0.1;

[0053] The enhanced tongue image is input into a pre-trained convolutional neural network (e.g., ResNet-18) for feature extraction and classification, and a tongue diagnosis category prediction result is output, such as "dark purple tongue" or "thick fur." ​​The tongue diagnosis category is based on Traditional Chinese Medicine (TCM) tongue diagnosis standards (e.g., tongue color: light red, dark purple; fur color: white, yellow; fur quality: thick, thin), and is defined by professional TCM practitioners. The convolutional neural network and its training method are well known to those skilled in the art and are not described in detail here.

[0054] Efficient and accurate automated tongue diagnosis is suitable for clinical auxiliary diagnosis, which helps promote the intelligence and standardization of traditional Chinese medicine diagnosis and has significant practical value and promotion potential.

[0055] In summary, a tongue image processing method for traditional Chinese medicine tongue diagnosis based on computer vision has been completed.

[0056] The order in which the embodiments of the invention are presented is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0057] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0058] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A computer vision-based tongue image processing method for traditional Chinese medicine tongue diagnosis, characterized in that: The following steps are involved: S1, photographing a tongue image, and performing image preprocessing on the tongue image to obtain a preprocessed tongue image; separating a tongue region from the preprocessed tongue image to obtain a segmented tongue image; S2. Enhance the segmented tongue image using an adaptive texture-color coupling enhancement algorithm to obtain an enhanced tongue image. In the implementation of the adaptive texture-color coupling enhancement algorithm, the segmented tongue image is divided into grids. Within each grid, the normalized gradient value of the pixel is calculated, and an amplification factor is introduced to calculate the texture feature value of the pixel. Obtaining pixel values ​​of an enhanced tongue image based on the texture feature values ​​of the pixels and the pixel values ​​of the segmented tongue image; Based on the pixel values ​​of the enhanced tongue image, an enhanced tongue image is formed; based on the enhanced tongue image, a prediction result of the tongue diagnosis category is obtained; The formula for the texture eigenvalue of a pixel is: ， in, is The texture feature value of the pixel at ; is The normalized gradient value of the pixel at ; are the row index and column index of the grid in the segmented tongue image, respectively; It is Rank Grid of columns; Is in the grid The variance of the normalized gradient value of the inner pixel; is the normalization factor; The formula for the pixel value of the enhanced tongue image is: ， in, The enhanced tongue image is The pixel value at ; The tongue image after segmentation is The pixel value at ; It is to enhance the weight; It is the normalized histogram of each RGB channel; The tongue image after segmentation is A color channel of the pixel at Pixel value of is the bias term.

2. The method for processing tongue images in traditional Chinese medicine tongue diagnosis based on computer vision according to claim 1, characterized in that: Said S1 specifically includes: During the image preprocessing process, the illumination deviation of the tongue image is corrected to obtain a color-corrected tongue image; the color-corrected tongue image is subjected to noise reduction processing to obtain a noise-reduced tongue image; and the noise-reduced tongue image is subjected to image standardization processing to obtain a preprocessed tongue image.

3. The method for processing tongue images in traditional Chinese medicine tongue diagnosis based on computer vision according to claim 2, characterized in that: Said S1 specifically includes: Based on the preprocessed tongue image, an initial binary mask is generated. The initial binary mask is used as the initial label of the tongue area and the background, and the tongue area and background are separated through iterative optimization to obtain the final binary mask. The preprocessed tongue image and the final binary mask are element-wise multiplied to obtain the segmented tongue image.

4. The method for processing tongue images in traditional Chinese medicine tongue diagnosis based on computer vision according to claim 1, characterized in that: Said S2 specifically includes: In each grid, the horizontal gradient and vertical gradient of the pixel are calculated separately; the square sum of the horizontal gradient and vertical gradient of each pixel is calculated and the square root is taken to obtain the gradient amplitude of the pixel; the gradient amplitudes of the pixels of all grids are globally normalized to obtain the normalized gradient value of the pixel, and a normalized gradient map is generated.

5. The method for processing tongue images in traditional Chinese medicine tongue diagnosis based on computer vision according to claim 4, characterized in that: Said S2 specifically includes: Based on the normalized gradient value of the pixels in the grid, the grid variance is calculated and normalized to generate an amplification factor. Combined with the normalized gradient value of the pixel, the texture feature value of the pixel is calculated; based on the texture feature value of the pixel, a texture feature map is constructed.

6. The method for processing tongue images in traditional Chinese medicine tongue diagnosis based on computer vision according to claim 1, characterized in that: Said S2 specifically includes: In the implementation process of the adaptive texture-color coupling enhancement algorithm, the number of pixels with 256 gray levels in each of the red, green, and blue channels of the segmented tongue image is counted, and histograms are constructed respectively; the histogram of each channel is normalized to obtain a normalized histogram, and the normalized histograms of the three channels are used as the global color distribution features.

7. The method for processing tongue images in traditional Chinese medicine tongue diagnosis based on computer vision according to claim 6, characterized in that: Said S2 specifically includes: Based on the normalized histogram, the color frequency of each pixel in the segmented tongue image is calculated, and a negative exponential function is introduced to perform nonlinear regulation of the color frequency.

8. The method for processing tongue images in traditional Chinese medicine tongue diagnosis based on computer vision according to claim 7, characterized in that: Said S2 specifically includes: Based on the texture feature value of the pixel and the negative exponential function, and introducing the enhancement weight, the enhancement amplitude is obtained; based on the enhancement amplitude and the pixel value of the segmented tongue image, the pixel value of the enhanced tongue image is obtained.

9. The method for processing tongue images in traditional Chinese medicine tongue diagnosis based on computer vision according to claim 8, characterized in that: Said S2 specifically includes: The enhanced tongue image is subjected to feature extraction and classification processing using a pre-trained convolutional neural network to obtain the prediction results of the tongue diagnosis category.

Images