A traditional Chinese medicine tongue image recognition method and system based on image processing

Abstract

The present application relates to the technical field of image processing, and especially to a traditional Chinese medicine tongue image recognition method and system based on image processing, wherein the present application firstly collects a tongue base image and a tongue surface image, then performs conventional image preprocessing operation and illumination correction on the tongue base image and the tongue surface image, adopts different illumination correction work, adjusts the brightness of the tongue surface image by using a linear change method, and realizes illumination correction on the tongue base image by using a block correction method, and when performing illumination correction on the tongue base image, an improved Gaussian weight function formula is used to independently perform illumination estimation on each small block, the gradient calculation is introduced into the specific application scene of illumination estimation of the block image, and when performing illumination estimation on the pixels of the block tongue base image, the gradient term of the block image pixel is introduced, so that the improved Gaussian weight function can determine the improved term of the Gaussian weight function according to the image details of each block tongue base image, and the estimated pixels can better retain the detail information of the tongue base image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for traditional Chinese medicine tongue image recognition based on image processing. Background Technology

[0002] Traditional Chinese medicine (TCM) tongue diagnosis relies heavily on the physician's subjective visual observation and accumulated experience, which has several limitations. First, different physicians may differ in their observation and judgment of the tongue, leading to inconsistent diagnostic results. Second, the physician's experience level significantly impacts diagnostic accuracy; experienced physicians often have a much higher accuracy rate in tongue diagnosis than beginners. Furthermore, subtle changes in the tongue may be difficult to detect accurately with the naked eye, affecting diagnostic precision. Moreover, in real-world medical settings, physicians need to diagnose a large number of patients in a short period, which may result in insufficient observation and consequently affect the quality of diagnosis.

[0003] With the continuous advancement of technology, image recognition technology has been widely applied in many fields, and its application in the medical field has gradually attracted attention. Image recognition technology can accurately analyze and identify various features in images, providing the possibility for objective and accurate disease diagnosis. In the field of tongue diagnosis in traditional Chinese medicine, using image recognition technology to collect, process, and analyze tongue images is expected to overcome the shortcomings of traditional methods and achieve standardization and objectification of tongue diagnosis.

[0004] Significant progress has been made in the digitalization and intelligentization of TCM tongue diagnosis. For example, Chen Keju et al. proposed a TCM syndrome identification model (SI-MTF) based on multi-view tongue image feature fusion. By collecting a large number of tongue surface and under-tongue images, they used deep learning technology to extract the color, shape, and texture features of the tongue image, and combined a university-hospital network to improve the model's ability to fuse tongue surface and under-tongue features, ultimately achieving intelligent identification of TCM syndromes. Zeng Lili et al. proposed a cross-task transfer learning tongue image diagnosis method based on multi-source data (M2T-Net). Through multi-source data pre-training and a cross-attention mechanism, they achieved accurate classification of patients with coronary heart disease and their accompanying diabetes. Zhang Mo et al. explored the intelligent application of TCM tongue diagnosis based on a multimodal large model. By combining tongue image images and the doctor's initial description, they used a multimodal large model to achieve efficient diagnosis of Helicobacter pylori infection.

[0005] However, existing TCM tongue diagnosis methods suffer from low accuracy in preprocessing tongue images because these images, which include both the surface and underside of the tongue, are located at different positions and are subject to varying lighting conditions during acquisition. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a method and system for TCM tongue image recognition based on image processing, which solves the problems existing in the prior art.

[0007] This invention provides a method for TCM tongue image recognition based on image processing, comprising the following steps:

[0008] S1: Acquire images of the tongue surface and underside using an image acquisition device;

[0009] S2: Perform a first image preprocessing operation on the tongue surface and tongue underside images to obtain a first preprocessed tongue surface image and a first preprocessed tongue underside image;

[0010] S3: Perform a second image preprocessing operation on the first preprocessed tongue surface image and the first preprocessed tongue underside image to obtain a second preprocessed tongue surface image and a second preprocessed tongue underside image;

[0011] The second image preprocessing operation specifically involves: adjusting the brightness of the first preprocessed tongue surface image using a linear transformation method; and performing illumination correction on the first preprocessed tongue underside image using a block correction method.

[0012] The illumination correction for the first preprocessed tongue image is achieved using a block-based correction method, specifically as follows:

[0013] Sa: Divide the first preprocessed tongue underside image into multiple small blocks;

[0014] Sb: Illumination estimation is performed independently for each small block using an improved Gaussian weighting function formula;

[0015] Sc: Perform image correction on each small block based on the pixel local illumination estimate of each small block;

[0016] Sd: The corrected small blocks are image fused to achieve illumination correction of the first preprocessed tongue image using a block correction method.

[0017] S4: Input the second preprocessed tongue surface image and the second preprocessed tongue underside image into the tongue image recognition model to obtain the tongue image recognition result.

[0018] Preferably, in Sb, a local weighted average method is used to calculate the illumination estimation component. Within the neighborhood of each pixel in each small block, different weights are assigned based on the distance between the pixel and the center pixel. The illumination estimation component of the pixel is obtained by calculating the weighted average of the pixel grayscale values ​​within the neighborhood. The specific formula is:

[0019] ;

[0020] Where N(x,y) is the neighborhood centered at pixel (x,y), I(i,j) is the gray value of pixel (i,j), and w(i,j) is the weight of pixel (i,j). illumination (i,j) is the illumination estimation component of pixel (i,j).

[0021] Preferably, the improved Gaussian weighting function formula is as follows:

[0022] ;

[0023] In the formula, w(i,j) is the weight, (x,y) is the position of the center pixel, (i,j) is the position of the pixel in the neighborhood, and σ is the standard deviation of the Gaussian weight function. It is the gradient value of each image at pixel (i,j). It is the maximum value of the gradient for each image block.

[0024] Preferably, the Sd specifically involves: in the overlapping region of small blocks, assigning different weights to each pixel based on its distance from the center of the small block, performing smooth fusion, calculating the weights using the improved Gaussian weight function formula, and calculating the fused pixel value I of the overlapping region based on the weights. fused(i,j) ;

[0025] The specific formula is as follows:

[0026] ;

[0027] Among them, I corrected,k (i,j) is the gray value of the corrected image of the k-th small block at pixel (i,j), w k (i,j) is the weight of the k-th block at pixel (i,j).

[0028] Preferably, the first image preprocessing operation includes image denoising, image segmentation, and image enhancement.

[0029] Preferably, the image denoising is performed by using median filtering to remove random noise from the image, and the image segmentation is performed by using the Canny edge detection algorithm to extract the tongue contour in the tongue surface image and using a color threshold-based segmentation method to extract the tongue contour in the tongue floor image.

[0030] Preferably, the tongue contour extracted from the tongue image using a color threshold-based segmentation method specifically involves: converting the tongue image from RGB to HSV color space using a color space conversion function; setting a color threshold based on the color characteristics of the tongue image; segmenting the tongue image using the color threshold, setting pixels that meet the conditions to white, and setting other pixels to black, thereby extracting the tongue region.

[0031] Preferably, setting the color threshold based on the color characteristics of the tongue image specifically involves: using a histogram analysis tool to observe the color distribution of the tongue image in the HSV color space, then selecting a typical color range of the tongue based on the observation results, and setting the color threshold range based on the typical color range.

[0032] Preferably, the color threshold range is:

[0033] H (hue): 0 to 30 degrees;

[0034] S (saturation): 0 to 100 degrees;

[0035] V (brightness): 100 to 255 degrees.

[0036] According to another aspect of the present invention, a TCM tongue image recognition system based on image processing is provided. The system employs the aforementioned TCM tongue image recognition method based on image processing, and the system includes:

[0037] Image acquisition device to capture images of the tongue surface and underside;

[0038] The first preprocessing module is used to perform a first image preprocessing operation on the tongue surface and tongue underside images to obtain a first preprocessed tongue surface image and a first preprocessed tongue underside image.

[0039] The second preprocessing module is used to perform a second image preprocessing operation on the first preprocessed tongue surface image and the first preprocessed tongue underside image to obtain a second preprocessed tongue surface image and a second preprocessed tongue underside image.

[0040] The tongue image recognition module is used to input the second preprocessed tongue surface image and the second preprocessed tongue underside image into the tongue image recognition model to obtain the tongue image recognition result.

[0041] The embodiments of the present invention have the following technical effects:

[0042] This invention first acquires images of the underside and surface of the tongue, then performs conventional image preprocessing and illumination correction on these images. During illumination correction, different methods are employed due to the varying sensitivities of the tongue surface and underside to lighting conditions caused by their different structures and locations. For the surface image, a linear variation method is used to adjust brightness; for the underside image, a block-based correction method is used. Furthermore, an improved Gaussian weighting function is used to independently estimate the illumination of each small block during illumination correction. Gradient calculation is introduced into this specific application scenario of block-based image illumination estimation. When estimating the illumination of pixels in the block-based underside image, considering the varying details contained in each block, a pixel gradient term is introduced. This allows the improved Gaussian weighting function to determine its improvement term based on the image details of each underside image, thereby improving the targeting of illumination estimation and ensuring that the estimated pixels better preserve the detailed information of the underside image. Attached Figure Description

[0043] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0044] Figure 1 This is a flowchart of a traditional Chinese medicine tongue image recognition method based on image processing provided in an embodiment of the present invention;

[0045] Figure 2 This is a flowchart of an embodiment of the present invention for performing illumination correction on the first preprocessed tongue image using a block correction method;

[0046] Figure 3 This is a comparison image of the tongue surface before and after preprocessing provided in an embodiment of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0048] Example 1, Appendix Figure 1 A flowchart of a TCM tongue image recognition method based on image processing is shown in the attached figure. Figure 1As shown, a TCM tongue image recognition method based on image processing includes the following steps:

[0049] S1: Acquire images of the tongue surface and underside using an image acquisition device;

[0050] The image acquisition device includes a high-resolution camera, a uniform light source, and image acquisition software.

[0051] The high-resolution camera is used to capture images of the tongue surface and under the tongue. The high-resolution camera should have high resolution to ensure that it can clearly capture subtle features of the tongue image, such as tongue color, coating color, tongue shape, and coating texture. At the same time, the high-resolution camera should have an autofocus function to adapt to the tongue position and shape of different patients.

[0052] The uniform light source is used to provide stable lighting conditions. The uniform light source can ensure the color accuracy and consistency of the tongue image and reduce the image quality degradation caused by uneven lighting. In this step, the uniform light source adopts a cold light source to avoid causing discomfort to the patient or interfering with the color performance of the image.

[0053] The image acquisition software is used to control the camera's shooting parameters and the image acquisition process. The image acquisition software has the function of automatically adjusting the shooting angle and lighting to ensure that the acquired tongue image is clear and stable.

[0054] The process of acquiring images of the tongue surface and tongue underside includes:

[0055] Device initialization: Before starting to acquire images of the tongue surface and under the tongue, first start the image acquisition device and ensure that the high-resolution camera and uniform light source are in normal working condition.

[0056] Tongue image acquisition: The person whose tongue image is to be acquired extends their tongue, and the operator controls the camera to capture the tongue image through the image acquisition software, including images of the tongue surface and the underside of the tongue. The image acquisition software automatically adjusts the shooting angle of the camera and the intensity of the light source to ensure that the acquired tongue image is clear and stable.

[0057] S2: Perform a first image preprocessing operation on the tongue surface and tongue underside images to obtain a first preprocessed tongue surface image and a first preprocessed tongue underside image;

[0058] The first image preprocessing operation includes image denoising, image segmentation, and image enhancement.

[0059] In this step, the image denoising is performed by using median filtering to remove random noise from the image. The basic principle is to replace each pixel value in the image with the median value in the neighborhood of that pixel, thereby achieving the purpose of smoothing the image and removing noise.

[0060] The image segmentation involves using the Canny edge detection algorithm to extract the tongue contour from the tongue surface image and using a color threshold-based segmentation method to extract the tongue contour from the tongue floor image.

[0061] Furthermore, the Canny edge detection algorithm is used to extract the tongue contour from the tongue image. Specifically, the following steps are performed: gradient calculation is performed on the tongue image to calculate the gradient intensity and direction of each pixel; for each pixel, its neighboring pixels in the gradient direction are checked, and if the gradient value of the pixel is not a local maximum, its gradient value is set to 0; two thresholds are set, a high threshold for detecting strong edges and a low threshold for detecting weak edges, and the strong edges and weak edges are connected by an edge tracking algorithm to form a complete edge; through the above steps, the Canny edge detection algorithm can effectively extract the tongue contour from the tongue image.

[0062] Specifically, gradient calculation is performed on the tongue image. The calculation of the gradient intensity and direction of each pixel in the tongue image is as follows: the Sobel operator is used to calculate the gradient of the tongue image in the horizontal and vertical directions respectively, and then the gradient intensity and direction of each pixel in the tongue image are calculated.

[0063] The blood vessel texture and color features of the tongue underside image are relatively obvious. The tongue contour can be extracted by color threshold segmentation. Furthermore, the tongue contour extraction method based on color threshold segmentation is specifically as follows: the tongue underside image is converted from RGB to HSV color space using a color space conversion function; an appropriate color threshold is set according to the color features of the tongue underside image; the tongue underside image is segmented using the color threshold, and pixels that meet the conditions are set to white and other pixels are set to black, thereby extracting the tongue region.

[0064] The color of the underside of the tongue is usually lighter, pink or light red, while the blood vessels are more prominent and darker, usually dark red or purple. Based on the color characteristics of the underside of the tongue image, the tongue outline can be effectively extracted. To better perform color segmentation, the underside of the tongue image usually needs to be converted from the RGB color space to the HSV color space. The HSV color space is a commonly used choice because it separates color information from brightness information, making it easier to set the color threshold. In the HSV color space, H (Hue) represents the hue, indicating the type of color; S (Saturation) represents the saturation, indicating the purity of the color; and V (Value) represents the brightness, indicating the lightness or darkness of the color.

[0065] Setting an appropriate color threshold based on the color characteristics of the tongue image specifically involves: using a histogram analysis tool to observe the color distribution of the tongue image in the HSV color space, then selecting a typical color range of the tongue based on the observation results, and setting an appropriate color threshold range based on the typical color range.

[0066] For example, the typical color of the underside of the tongue is usually between pink and light red, corresponding to an H value range of 0 to 30 degrees, an S value range of 0 to 100, and a V value range of 100 to 255; therefore, the suitable color threshold range is:

[0067] H (hue): 0 to 30 degrees;

[0068] S (saturation): 0 to 100 degrees;

[0069] V (brightness): 100 to 255 degrees;

[0070] Specifically, the tongue region is extracted by segmenting the tongue image using the color threshold, setting the pixels that meet the conditions to white, and setting the other pixels to black. This is achieved by: segmenting the tongue image using the color threshold range, extracting the pixels within the color threshold range, setting the extracted pixels to white (255), and setting the other pixels to black (0), generating a binary image; and refining the extracted contours through morphological operations such as dilation, erosion, opening, and closing operations to remove noise and irregular parts.

[0071] The above steps can effectively extract the tongue contour from the tongue base image and refine the contour through morphological operations, providing more accurate data support for subsequent feature extraction and analysis.

[0072] Image enhancement is an important step in image preprocessing. Its purpose is to improve the visual effect of the tongue surface and tongue underside images, enhance useful information in the images, and improve image recognizability. Histogram equalization is a commonly used image enhancement technique, particularly suitable for improving image contrast, thereby enhancing the texture and color features of the image. More specifically, image enhancement of the tongue surface and tongue underside images involves:

[0073] Calculate the number of pixels for each gray value in the images of the tongue surface and tongue base; calculate the cumulative histogram: calculate the cumulative distribution function (CDF) of the histogram to obtain the cumulative histogram; map the gray values ​​of the original image to new gray values ​​according to the cumulative histogram, so that the distribution of the new gray values ​​is more uniform, thereby achieving image enhancement.

[0074] S3: Perform a second image preprocessing operation on the first preprocessed tongue surface image and the first preprocessed tongue underside image to obtain a second preprocessed tongue surface image and a second preprocessed tongue underside image;

[0075] The surface and underside of the tongue have different structural characteristics. The surface of the tongue is relatively flat, while the underside has more folds and vascular patterns. As a result, the reflection and absorption characteristics of light on the surface and underside of the tongue are different when taking images of the surface and underside of the tongue. This leads to differences in their sensitivity to lighting conditions. Therefore, in order to accurately extract tongue image features, it is necessary to perform effective lighting correction on the surface and underside images of the tongue respectively.

[0076] Specifically, the second image preprocessing operation involves: adjusting the brightness of the first preprocessed tongue surface image using a linear transformation method; and performing illumination correction on the first preprocessed tongue underside image using a block correction method.

[0077] Furthermore, adjusting the brightness of the first preprocessed tongue image using a linear transformation method specifically involves: setting a target brightness range, and using a linear transformation formula to adjust the grayscale value of the first preprocessed tongue image to the target brightness range, thereby achieving the adjustment of the brightness of the first preprocessed tongue image.

[0078] In this embodiment, the target brightness range is [mean-offset, mean+offset], where mean is the target average brightness of the first preprocessed tongue image, and offset is a preset offset value;

[0079] The linear transformation formula is:

[0080] ;

[0081] In the formula, I old I represents the grayscale value of the first preprocessed tongue surface image. new This refers to the adjusted grayscale value of the first preprocessed tongue surface image; the above formula can be used to adjust the image brightness to the target range, making the overall brightness of the image more uniform; Appendix Figure 3 The accompanying image shows a comparison of the tongue surface before and after preprocessing. Figure 3 As can be seen, the brightness of the tongue image can be accurately adjusted through the first preprocessing process of this embodiment.

[0082] Due to the numerous folds and vascular patterns in tongue floor images, illumination correction requires more meticulous handling of local illumination variations. Therefore, as shown in the attached... Figure 2 As shown, the illumination correction method for the first preprocessed tongue image is specifically implemented using a block correction method:

[0083] Sa: Divide the first preprocessed tongue underside image into multiple small blocks;

[0084] Specifically, in this step, the size of the small block can be adjusted according to the resolution of the first preprocessed tongue image. For example, the image can be divided into small blocks of 32×32 or 64×64. When dividing the blocks, ensure that there is a certain overlapping area between the small blocks so as to perform smoothing in the subsequent fusion process. The size of the overlapping area can be adjusted according to the size of the small block, usually 1 / 4 to 1 / 2 of the side length of the small block.

[0085] Sb: Illumination estimation is performed independently for each small block using an improved Gaussian weighting function formula;

[0086] In this step, a local weighted average method is used to estimate the illumination components. Within the neighborhood of each pixel in each small block, different weights are assigned based on the distance between the pixel and the center pixel; the closer the distance, the greater the weight. The illumination component of the pixel is obtained by calculating the weighted average of the gray values ​​of the pixels in the neighborhood. The specific formula is as follows:

[0087] ;

[0088] Where N(x,y) is the neighborhood centered at pixel (x,y), w(i,j) is the weight of pixel (i,j), and I illumination (i,j) is the illumination estimation component of pixel (i,j);

[0089] In this step, the determination of weights has a significant impact on the calculation of the illumination estimate; existing technologies generally use a Gaussian function to calculate the weights, i.e.

[0090] ;

[0091] in,( x , y ) is the position of the center pixel, ( i , j ) represents the pixel position within the neighborhood. σ The standard deviation of the Gaussian function controls the decay rate of the weights. However, for tongue images, which contain a lot of detailed information such as wrinkles and blood vessel textures, the above Gaussian weight function may cause some loss of detailed information when processing the edges and details of the image. In order to improve this, this embodiment proposes an improved Gaussian weight function to determine the weights.

[0092] The improved Gaussian weighting function formula is as follows:

[0093] ;

[0094] In the formula, w(i,j) is the weight, (x,y) is the position of the center pixel, (i,j) is the position of the pixel in the neighborhood, and σ is the standard deviation of the Gaussian weight function. It is the gradient value of each image at pixel (i,j). It is the maximum value of the gradient for each image block;

[0095] By introducing the improved Gaussian weight function formula, gradient calculation is applied to the specific application scenario of illumination estimation for segmented images. When performing illumination estimation on the pixels of segmented tongue images, the improved Gaussian weight function can determine the improved term of the Gaussian weight function based on the image details of each segmented tongue image, thus improving the processing targeting of illumination estimation and enabling the estimated pixels to better retain the detailed information of the tongue image.

[0096] Sc: Perform image correction on each small block based on the pixel local illumination estimate of each small block;

[0097] The corrected image for each small block is obtained by dividing the pixel grayscale value of the original image by the illumination estimation component.

[0098] Sd: The corrected small blocks are image fused to achieve illumination correction of the first preprocessed tongue image using a block correction method.

[0099] Specifically, during the fusion process, the main thing to pay attention to is the smoothing of the edges of the small blocks to avoid obvious splicing marks. Therefore, the Sd is specifically to assign different weights to each pixel in the overlapping area of ​​the small blocks according to the distance of each pixel from the center of the small block, and to perform smooth fusion. The improved Gaussian weight function formula is also used to calculate the weights, and the fused pixel values ​​of the overlapping area are calculated based on the weights.

[0100] The specific formula is as follows:

[0101] ;

[0102] Among them, I corrected,k (i,j) is the gray value of the corrected image of the k-th small block at pixel (i,j), w k (i,j) is the weight of the k-th block at pixel (i,j).

[0103] In this step, the weights are gradually reduced in the edge regions of small blocks to avoid obvious stitching marks. At the same time, the smooth decay characteristics of the improved Gaussian weight function ensure that the grayscale values ​​of the fused image transition naturally in the edge regions.

[0104] S4: Input the second preprocessed tongue surface image and the second preprocessed tongue underside image into the tongue image recognition model to obtain the tongue image recognition result.

[0105] In tongue image recognition, convolutional neural networks (CNNs) are one of the most commonly used deep learning models. CNNs have powerful feature extraction capabilities and can automatically learn complex patterns and features in images. Therefore, the image recognition model in this embodiment is a convolutional neural network model.

[0106] The convolutional neural network model consists of multiple convolutional layers, pooling layers, and fully connected layers. It can automatically extract features from images. The convolutional layers slide convolutional kernels across the image to extract local features. Each convolutional kernel extracts a specific feature, such as edges or textures. The pooling layers reduce the spatial dimension of the feature map, reducing computation while retaining important features. The fully connected layers flatten the feature map into a one-dimensional vector, which is then used for classification tasks through a multi-layer neural network. The output of the fully connected layers is the final prediction result of the image recognition model.

[0107] When using the convolutional neural network model for tongue image recognition, the model needs to be trained using labeled tongue image data. During training, the weights of the convolutional neural network model are updated through backpropagation algorithm to minimize the loss function. In this embodiment, the loss function is the cross-entropy loss function.

[0108] Furthermore, during training, the hyperparameters of the convolutional neural network model, such as learning rate, batch size, and optimizer, are adjusted to improve the training effect of the model.

[0109] The tongue image recognition results include: pale red tongue, pale white tongue, red tongue, crimson tongue; white tongue coating, yellow tongue coating, gray tongue coating, black tongue coating, large tongue, thin tongue, tongue with teeth marks, and tongue with cracks.

[0110] By inputting tongue images into a trained deep learning model, automated TCM image recognition can be achieved. Convolutional neural network models have powerful feature extraction capabilities, enabling them to automatically learn complex patterns and features in images, thereby improving the accuracy and efficiency of image recognition and providing strong support for TCM image recognition.

[0111] Example 2: The present invention also provides a TCM tongue image recognition system based on image processing. The system employs a TCM tongue image recognition method based on image processing as described in Example 1. The system includes:

[0112] Image acquisition device to capture images of the tongue surface and underside;

[0113] The first preprocessing module is used to perform a first image preprocessing operation on the tongue surface and tongue underside images to obtain a first preprocessed tongue surface image and a first preprocessed tongue underside image.

[0114] The second preprocessing module is used to perform a second image preprocessing operation on the first preprocessed tongue surface image and the first preprocessed tongue underside image to obtain a second preprocessed tongue surface image and a second preprocessed tongue underside image.

[0115] The tongue image recognition module is used to input the second preprocessed tongue surface image and the second preprocessed tongue underside image into the tongue image recognition model to obtain the tongue image recognition result.

[0116] Example 3: The present invention also provides an electronic device, including one or more processors and a memory.

[0117] A processor can be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and can control other components in an electronic device to perform desired functions.

[0118] The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and a processor may execute the program instructions to implement the image processing-based TCM tongue image recognition method described above in any embodiment of this application, and / or other desired functions. Various contents such as initial extrinsic parameters and thresholds may also be stored in the computer-readable storage medium.

[0119] In one example, the electronic device may also include input and output devices, which are interconnected via a bus system and / or other forms of connection (not shown). The input device may include, for example, a keyboard, a mouse, etc. The output device may output various information to the outside, including warning messages, braking force, etc. The output device may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0120] Of course, for simplicity, components such as buses and input / output interfaces have been omitted. In addition, depending on the specific application, the electronic device may include any other appropriate components.

[0121] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the function of a traditional Chinese medicine tongue image recognition method based on image processing provided in any embodiment of this application.

[0122] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0123] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to implement a TCM tongue image recognition method based on image processing provided in any embodiment of this application.

[0124] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A traditional Chinese medicine tongue image recognition method based on image processing, characterized in that, The method comprises the following steps: S1: collecting a tongue surface image and a tongue bottom image by an image acquisition device; S2: performing a first image preprocessing operation on the tongue surface and tongue bottom images to obtain a first preprocessed tongue surface image and a first preprocessed tongue bottom image; S3: performing a second image preprocessing operation on the first preprocessed tongue surface image and the first preprocessed tongue bottom image to obtain a second preprocessed tongue surface image and a second preprocessed tongue bottom image; The second image preprocessing operation specifically comprises: adjusting the brightness of the first preprocessed tongue surface image by using a linear variation method; and achieving illumination correction of the first preprocessed tongue bottom image by using a block correction method; Achieving illumination correction of the first preprocessed tongue bottom image by using the block correction method specifically comprises: Sa: dividing the first preprocessed tongue bottom image into a plurality of small blocks; Sb: independently performing illumination estimation on each small block by using an improved Gaussian weight function formula; in Sb, a local weighted average method is used to calculate an illumination estimation component, and in the neighborhood of each pixel in each small block, different weights are given to the pixel and the center pixel according to the distance between them, and the weighted average value of the gray values of the pixels in the neighborhood is calculated to obtain the illumination estimation component of the pixel; the specific formula is: ； where N(x, y) is a neighborhood centered at pixel (x, y), I(i, j) is the gray value of pixel (i, j), w(i, j) is the weight of pixel (i, j), I illumination (i, j) is the illumination estimation component of pixel (i, j); Sc: performing image correction on each small block according to the local illumination estimation value of the pixels in each small block; Sd: performing image fusion on each corrected small block to achieve illumination correction of the first preprocessed tongue bottom image by using the block correction method; S4: inputting the second preprocessed tongue surface image and the second preprocessed tongue bottom image into a tongue image recognition model to obtain a tongue image recognition result.

2. The image processing-based traditional Chinese tongue image recognition method according to claim 1, wherein the improved Gaussian weight function formula is:

3. The image processing-based traditional Chinese tongue image recognition method according to claim 1, wherein the specific formula is: ； where w(i,j) is the weight of pixel (i,j), (x,y) is the position of the center pixel, (i,j) is the pixel position within the neighborhood, and σ is the standard deviation of the Gaussian weighting function, is the gradient value at pixel (i,j) for each block image, is the maximum value of the gradient for each block image.

4. The image processing-based traditional Chinese tongue image recognition method according to claim 1, wherein the first image preprocessing operation comprises image denoising, image segmentation, and image enhancement. The Sd is specifically: in the small overlapping area, different weights are given according to the distance of each pixel from the center of the small block, smoothing fusion is carried out, the weight is calculated by using the improved Gaussian weight function formula, and the pixel value I of the overlapping area after fusion is calculated according to the weight fused(i,j) ; The image denoising is to remove random noise in the image by using median filtering, the image segmentation is to extract the tongue contour in the tongue surface image by using a Canny edge detection algorithm, and the tongue contour in the tongue bottom image is extracted by using a color threshold-based segmentation method. ； where I corrected,k (i,j) is the gray value of the corrected image of the kth patch at pixel (i,j), w k (i,j) is the weight of the kth patch at pixel (i,j).

6. The image processing-based traditional Chinese tongue image recognition method according to claim 5, wherein the color threshold-based segmentation method for extracting the tongue contour in the tongue bottom image specifically comprises: converting the tongue bottom image from RGB to HSV color space using a color space conversion function; and setting a color threshold according to the color features of the tongue bottom image; The color threshold is used to segment the tongue bottom image, the pixels meeting the conditions are set to white, and the other pixels are set to black, so as to extract the tongue area.

5. The image processing-based traditional Chinese tongue image recognition method according to claim 4, characterized in that:

7. The image processing-based traditional Chinese tongue image recognition method according to claim 6, wherein ​ ​ ​ ​ The color threshold is set according to the color feature of the tongue base image, specifically, a histogram analysis tool is used to observe the color distribution of the tongue base image in an HSV color space, then according to the observation result, a typical color range of the tongue base is selected, and a color threshold range is set according to the typical color range.

8. The image processing-based traditional Chinese tongue image recognition method according to claim 7, characterized in that: The color threshold range is: Hue: 0 to 30 degrees; Saturation: 0 to 100 degrees; Brightness: 100 to 255 degrees.

9. An image processing-based traditional Chinese medicine tongue image recognition system, characterized in that, The system adopts the image processing-based traditional Chinese tongue image recognition method according to any one of claims 1-8, and the system comprises: An image acquisition device for acquiring tongue surface and tongue base images; A first preprocessing module for performing a first image preprocessing operation on the tongue surface and tongue base images to obtain a first preprocessed tongue surface image and a first preprocessed tongue base image; A second preprocessing module for performing a second image preprocessing operation on the first preprocessed tongue surface image and the first preprocessed tongue base image to obtain a second preprocessed tongue surface image and a second preprocessed tongue base image; A tongue image recognition module for inputting the second preprocessed tongue surface image and the second preprocessed tongue base image into a tongue image recognition model to obtain a tongue image recognition result.

Images