A Traditional Chinese Medicine Tongue Diagnosis Intelligent Recognition Method Based on Tongue Image Processing

Abstract

The present invention discloses a method for intelligent recognition of traditional Chinese medicine tongue diagnosis based on tongue image processing. This method is implemented based on an intelligent terminal, and a preset App is installed on the intelligent terminal. The method includes the following steps: Step S1, the user uses the intelligent terminal to take a tongue image for the App to obtain the tongue image; Step S2, the App preprocesses the tongue image to obtain a tongue image that meets the parameter requirements; Step S3, input the tongue image preprocessed in Step S2 into the image recognition buffer of the App; Step S4, the App recognizes the tongue image according to the preset tongue image data set, and then obtains the corresponding disease recognition result and interpretation information. The present invention can provide the observation results of continuous image data for traditional Chinese medicine doctors, provide the change situation of diseases in a continuous time period, and record subtle changes and other data, so as to provide continuous data support for the disease diagnosis process.

Description

Technical Field

[0001] The present invention relates to an intelligent algorithm for human organ image recognition and diagnosis, and particularly to an intelligent recognition method for observing and diagnosing diseases by tongue inspection in traditional Chinese medicine based on tongue image processing. Background Art

[0002] In the prior art, image processing and recognition technologies are widely applied in aspects such as face recognition, object recognition, driverless, security and dangerous goods recognition, medical imaging, etc., while there are few corresponding technology applications in the field of traditional Chinese medicine. The reason is not that traditional Chinese medicine is a traditional medicine. On the one hand, the diagnosis of traditional Chinese medicine relies on the experience accumulation and inheritance of traditional Chinese medicine doctors through inspection, auscultation and olfaction, interrogation, and palpation. On the other hand, there are no applications and fit points of artificial intelligence technology in the field of traditional Chinese medicine. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide, in view of the deficiencies of the prior art, an innovative combination of image recognition and processing technology and the method of observing and diagnosing diseases by tongue inspection in traditional Chinese medicine, which can provide continuous image data observation results for traditional Chinese medicine doctors, provide the change situation of diseases during a continuous time period and record of subtle changes and other data, and further provide an intelligent recognition method with continuous data support for the disease diagnosis process.

[0004] To solve the above technical problem, the present invention adopts the following technical solutions.

[0005] An intelligent recognition method for observing and diagnosing diseases by tongue inspection in traditional Chinese medicine based on tongue image processing, which is implemented based on an intelligent terminal. The intelligent terminal is installed with a preset App. The method includes the following steps: Step S1, the user uses the intelligent terminal to take a tongue image for the App to obtain the tongue image; Step S2, the App preprocesses the tongue image to obtain a tongue image meeting the parameter requirements; Step S3, input the tongue image preprocessed in Step S2 into the image recognition buffer area of the App; Step S4, the App recognizes the tongue image according to a preset tongue image data set, and then obtains the corresponding disease recognition result and interpretation information.

[0006] Preferably, Step S1 includes the following process: The user opens the App through the intelligent terminal. The App provides a function of taking and obtaining the camera permission of the intelligent terminal. The camera permission includes the front camera permission and the rear camera shooting permission. The user switches the front and rear cameras according to the requirements.

[0007] Preferably, in step S2, the process in which the App preprocesses the tongue image includes: step S20, image format cropping: using the sample parameter comparison method, the image parameters uploaded by the user are compared according to the standard sample parameter values, and then the image uploaded by the user is cropped; step S21, image processing judgment: the image uploaded by the user is cropped and the parameter values ​​are calculated in real time, and the calculation results are saved as the input values ​​of the comparison algorithm. The standard sample parameter values ​​are used as the comparison input values, and the traversal algorithm is used to compare them item by item, and finally the comparison results are output. If the image is qualified, the subsequent steps are continued. If the image is unqualified, image processing is performed. During the image processing process, one or several parameters of the image are adjusted according to the results of the above-mentioned traversal algorithm.

[0008] Preferably, in step S21, when performing image processing judgment, parameterized judgment is performed for five preset situations. If all parameters of the standard sample are met at the same time, the image recognition buffer area is entered. Otherwise, the standard sample parameters are compared with the user-shot image item by item according to the preset judgment algorithm.

[0009] Preferably, in the process of comparing the standard sample parameters with the user-shot image parameters, the parameter types that need to be compared include: image fog parameters, image contrast parameters, lossless magnification, whether to stretch, stretching parameters, image clarity and color parameters.

[0010] Preferably, in step S21, the five preset situations include: image dehazing: changing the format of image features, extracting non-hazed image pixel data and reconstructing and combining them to generate a non-hazed image; image contrast enhancement: brightening and enhancing the overly dark image, and darkening the overly bright image or even the image with light explosion until the recognition feature requirements are met; image lossless enlargement: enlarging the image by 2-4 times the pixels according to the preset image pixel quality requirements to facilitate the extraction of subtle feature values ​​of the image; restoring the stretched image: compactly rearranging the image pixels and reorganizing them in combination with conventional controls; image clarity and color enhancement: denoising the compressed blurred and low-pixel images, and the denoising process includes image texture optimization, color saturation adjustment, brightness adjustment and contrast adjustment.

[0011] Preferably, the process of establishing the tongue image dataset includes: Step S10, collecting preprocessed tongue image samples. After the tongue images taken by the user through the intelligent terminal camera or professional camera are uploaded, the tongue images are cropped into corresponding specifications; Step S11, extracting the subtle features in the tongue image features through the special layer of the convolutional neural network, and then training item by item to obtain the convolutional neural network; Step S12, processing the features of each layer of the convolutional neural network to obtain the tongue image partition feature values; Step S13, presenting a feature value questionnaire for the user to perform precise training and feature description in image recognition; Step S14, using the intelligent terminal as a trainer to perform timely tongue image training and questionnaire submission during fragmented time; Step S15, performing continuous identification training on the same tongue image feature values to ensure the reliability and correctness of the image data in the tongue image dataset; Step S16, counting the answering results; Step S17, obtaining the tongue image feature value and disease interpretation dataset through machine training.

[0012] Preferably, in the said Step S11, the convolutional neural network includes: a fully connected layer, which is a hidden layer of the convolutional neural network and contains a weight vector W and an activation function; a convolutional layer, which retains the spatial features of the input image. Through the convolutional operation, an image with pixel particle dimensions can be obtained. Usually, one-dimensional convolution and two-dimensional convolution are used, and 6 different convolutional results are continuously stacked to output a convolutional Kernel to extract the tongue image feature values; a pooling layer, which is used to compress the information of the original feature layer.

[0013] In the intelligent recognition method for traditional Chinese medicine tongue diagnosis based on tongue image processing disclosed by the present invention, through the research on image processing and recognition technology and algorithm design, combined with the diagnosis method of traditional Chinese medicine tongue diagnosis, an innovation is made, enabling the integration of image processing and recognition technology with traditional Chinese medicine tongue diagnosis. Through the image acquisition, recognition modeling, and algorithm deduction of the user's tongue organ area and tongue coating, it helps doctors infer the characteristics of the tongue organ area, the color and thickness of the tongue coating, and the diseases corresponding to the special characteristics of serious diseases. This provides the observing results of continuous image data, the changes of diseases during a continuous time period, and the records of subtle changes for traditional Chinese medicine doctors, providing continuous data support for disease diagnosis. In addition, the present invention also provides scientific, controllable, and effective health management data for users in the science of personal health management, and can provide reliable health management suggestions for users from the perspective of traditional Chinese medicine. Description of the Drawings

[0014] Figure 1 It is a flowchart of the image processing for the pre - step of traditional Chinese medicine tongue diagnosis image recognition;

[0015] Figure 2 It is a flowchart of the execution process of the traditional Chinese medicine tongue diagnosis image recognition trainer;

[0016] Figure 3 It is a flowchart of the process of traditional Chinese medicine tongue diagnosis image recognition and disease result interpretation;

[0017] Figure 4 It is a flowchart of the process of traditional Chinese medicine tongue diagnosis image recognition and disease diagnosis device execution;

[0018] Figure 5 It is a system block diagram for implementing the intelligent recognition method of observing tongue and differentiating syndromes in traditional Chinese medicine of the present invention. Specific implementation manner

[0019] The present invention will be described in more detail below with reference to the accompanying drawings and embodiments.

[0020] The present invention discloses an intelligent recognition method for observing tongue and differentiating syndromes in traditional Chinese medicine based on tongue image processing. Combining Figures 1 to 5 As shown, this method is implemented based on an intelligent terminal, and a preset App is installed on the intelligent terminal. The method includes the following steps:

[0021] Step S1, the user uses the intelligent terminal to take a tongue image picture for the App to obtain the tongue image picture;

[0022] Step S2, the App preprocesses the tongue image picture to obtain a tongue image picture that meets the parameter requirements;

[0023] Step S3, input the tongue image picture preprocessed in step S2 into the image recognition buffer area of the App;

[0024] Step S4, the App recognizes the tongue image picture according to the preset tongue image data set, and then obtains the corresponding disease recognition result and interpretation information.

[0025] In the above method, through research on image processing and recognition technology and algorithm concept, combined with the diagnosis method of observing tongue and differentiating syndromes in traditional Chinese medicine, innovation is carried out, so that the image processing and recognition technology are applied and integrated with traditional Chinese medicine tongue diagnosis. Through image acquisition, recognition modeling and algorithm deduction of the user's tongue organ block and tongue coating, it helps doctors infer the characteristics of the tongue organ block, the color and thickness of the tongue coating, and the diseases corresponding to the special characteristics of serious diseases. This provides continuous image data observation results, changes in diseases during continuous time periods, and records of subtle changes for traditional Chinese medicine doctors, providing continuous data support for disease diagnosis. In addition, the present invention also provides scientific, controllable and effective health management data for users in the science of personal health management, and can provide reliable health management suggestions for users from the perspective of traditional Chinese medicine.

[0026] In this embodiment, the algorithms for image processing and recognition are transplanted into the App pre-installed on intelligent terminals such as mobile phones. Users can use the front or rear cameras of the intelligent terminal to take selfies or take pictures of the tongue image in other ways and upload the image in real time. In the App, first, the daily tongue image monitoring is enabled. The App will automatically activate the front or rear camera of the intelligent terminal. The user locates and takes pictures within the shooting outline drawn by the App, and after shooting, the App automatically uploads the picture to the server of the App. To facilitate users' understanding and acquaintance with the said App, the said App can be named "Tongue Observation App". The tongue diagnosis image processing algorithm processes the image when the image data is obtained. The purpose of image processing is that the final image can enable the recognition algorithm to identify diseases and correspond to individual physical signs. Therefore, during the image processing process, a series of special process judgments and processing of image processing are required.

[0027] Further, the step S1 includes the following processes:

[0028] The user opens the said App through the intelligent terminal. The App provides a function of shooting and obtaining the camera permission of the intelligent terminal. The camera permission includes the front camera permission and the rear camera shooting permission. The user switches the front and rear cameras according to the requirements. Since the collected images will vary due to different camera specifications and pixel arrangements, the App supports such differences and provides a button to submit the collected images.

[0029] In the step S2 of this embodiment, the process of the App preprocessing the tongue image includes:

[0030] Step S20, image format cropping: Using the sample parameter comparison method, compare the image parameters uploaded by the user according to the standard sample parameter values, and then crop the image uploaded by the user; this step is also called image screening;

[0031] Step S21, image processing judgment: After the image uploaded by the user is cropped, various parameter values are calculated in real time. The calculation results are saved as the input values of the comparison algorithm. The standard sample parameter values are used as the comparison input values. The traversal algorithm is used to compare item by item, and finally the comparison result is output. If the image is qualified, the subsequent steps are continued. If the image is unqualified, image processing is performed. During the image processing process, according to the results of the above traversal algorithm, one or several parameters of the image are adjusted.

[0032] As a preferred method, in the step S21, when performing image processing judgment, parametric judgment is performed for five preset situations. If all the parameters of the standard sample are satisfied at the same time, it enters the image recognition buffer area. Otherwise, according to the preset judgment algorithm, the standard sample parameters are compared with the user's captured image item by item.

[0033] Furthermore, in the process of comparing the standard sample parameters with the user-shot image parameters, the types of parameters that need to be compared include: image fog parameters, image contrast parameters, lossless magnification, whether to stretch, stretching parameters, image clarity and color parameters.

[0034] The specific algorithm principle of the above process is that after the user uploads the image and crops it, the parameter values ​​are calculated in real time, and the calculation results are stored as the input values ​​of the comparison algorithm. The parameter values ​​of the standard sample are the comparison input values, i.e. the objects. The traversal algorithm compares each item one by one, and the number of combinations for which the number of comparisons is C5 is also the factorial number of 5, resulting in 120 superimposed combinations for traversal comparison. Finally, the results are output. If qualified, the next step will be entered. If unqualified, image processing will be performed. Image processing adjusts the image parameters for one or several items based on the results of the above traversal algorithm.

[0035] In step S21 of this embodiment, the five preset situations include:

[0036] Image defogging: Change the format of image features, extract non-fogging image pixel data, reconstruct and combine them, and generate a non-fogging image; specifically, defogging is caused by lens fogging due to differences in climate and shooting environment temperature. The image is foggy and difficult to identify, so defogging is performed. If this condition is met, the defogging image processing step is performed. When the image is fogged, change the format of its features. Extract its non-fogging image pixel data, reconstruct and combine them;

[0037] Image contrast enhancement: Brighten and enhance the dark images, and darken the overly bright or even light-blown images until the recognition feature requirements are met;

[0038] Lossless image enlargement: Enlarge the image by 2-4 times according to the preset image pixel quality requirements to facilitate the extraction of subtle feature values ​​of the image;

[0039] Restoring stretched images: compactly rearrange the image pixels and reorganize them in combination with conventional reference objects. In practical applications, the stretched images are inevitably jittery during the shooting process, so the stretched images are restored. The restoration process is actually a process of compactly rearranging the image pixels and reorganizing them in combination with conventional reference objects.

[0040] Image clarity and color enhancement: De-noising is performed on compressed blurred and low-pixel images. The denoising process includes image texture optimization, color saturation adjustment, brightness adjustment, and contrast adjustment.

[0041] Since the images on the user side may exhibit the above five situations, or a superposition of certain concentrated situations may occur simultaneously, such as dehazing operation while also enhancing the image clarity, etc., regarding which algorithm processing is required, first, the App will give a standardized format, perform parameter and quantization on the standardized image, so as to complete the judgment on whether the image needs to go through the processing steps.

[0042] After going through the four steps of image processing, the tongue image has the basis for recognition. Before officially incorporating the tongue image recognition algorithm into the App, a tongue image acquisition and image recognition trainer based on the intelligent terminal will be designed and implemented first, and this trainer will be named Tongue image set (TIS, tongue image data set). TIS is a universal new type of human organ sign image data set different from face recognition and medical imaging. One image acquisition port of it integrates functions such as tongue image shooting, uploading, classification, disease corresponding questionnaire and submission, etc. Under the premise of seeking consent from Chinese medicine doctors during diagnosis, tongue image data of patients will be collected, and image acquisition will be carried out with different diseases, different diagnosis times, and different individuals as dimensions. It is expected to collect tongue image data of about 7000 or more different patients on mobile phones for 10 - 12 months with common cases as the first priority and severe and rare diseases as the opportunity, and establish the data basis of the TIS data set and the trainer.

[0043] The purpose of image recognition training is to calibrate the accuracy and recognition speed of the recognition algorithm. In this embodiment, the establishment process of the tongue image data set includes:

[0044] Step S10, collect pre - processed tongue image samples. After the tongue image taken by the user through the intelligent terminal camera or professional camera is uploaded, the tongue image is cropped into the corresponding specifications;

[0045] Step S11, extract the subtle features in the tongue image features through a special layer of the convolutional neural network, and then conduct item - by - item training to obtain the convolutional neural network;

[0046] Step S12, process the features of each layer of the convolutional neural network to obtain the tongue image partition feature values;

[0047] Step S13, present a feature value questionnaire for users to conduct accurate training and feature description in image recognition;

[0048] Step S14, the intelligent terminal serves as a trainer, and uses fragmented time for timely tongue image training and questionnaire submission;

[0049] Step S15, conduct continuous identification training on the same tongue image feature values to ensure the reliability and correctness of the image data in the tongue image data set;

[0050] Step S16, count the answers;

[0051] Step S17, obtain the tongue image eigenvalue and disease interpretation dataset through machine training.

[0052] Furthermore, in the step S11, the convolutional neural network includes:

[0053] Fully connected layer, which is a hidden layer of the convolutional neural network and contains a weight vector W and an activation function;

[0054] Convolutional layer, which retains the spatial features of the input image. Through convolution operations, an image with pixel granularity dimensions can be obtained. Usually, one-dimensional and two-dimensional convolutions are used, and 6 different convolution results are continuously stacked to output a convolution Kernel and extract the tongue image eigenvalue;

[0055] Pooling layer, used to compress the information of the original feature layer.

[0056] The function of the above tongue image trainer is to obtain a training dataset (TIS) through diverse data sample collection. The training algorithm is to train the recognition ability based on this dataset. This is the prerequisite for the final online real-time tongue image recognition, used to train a fast and accurate recognition ability, and provide basic preparation for the recognition and interpretation of the diseases corresponding to the tongue images. On the basis of having such an ability, the core recognition ability can be possessed. Therefore, after integrating the trained tongue image recognition and disease interpretation ability algorithm into the App, it will fully present the health management ability of the App based on tongue diagnosis.

[0057] The intelligent recognition method for traditional Chinese medicine tongue diagnosis based on tongue image processing disclosed in the present invention realizes the recognition of tongue images and the interpretation of disease results. Its core principle is to streamline the tongue image training samples, extract the tongue image eigenvalue through a convolutional neural network, then train the formed principal component analysis (PCA) model, perform dimensionality reduction on the key tongue features, train the Bayesian model, and calculate the Bayesian adjustment factor. The real-time tongue image preprocessing samples also go through the above processing, and finally, the posterior probability is calculated and compared. Thus, the eigenvalue recognition result and the corresponding explanation of the disease are obtained. The real-time tongue image recognition is strongly related to the training in the trainer, which is the function and purpose of the tongue image recognition trainer. The purpose of the tongue image training dataset is to train and improve the real-time recognition ability through large sample data.

[0058] After the tongue image recognition and symptom interpretation of the present invention are completed, there will be a diagnosis suggestion. On the one hand, the recognition algorithm can accurately track the changes in the user's tongue image. On the other hand, the App conducts targeted data collection based on the user's basic habits. These data collections mainly include the user's physical signs and information, including age, height, weight, pulse, heart rate, electrocardiogram, blood oxygen, body temperature, medication history, genetic history, allergy history, and living area (water quality, climate, diet). Combining the tongue image syndrome differentiation analysis and the online Chinese doctor's diagnosis, through deep learning and the recommendation system, it can clearly and accurately provide users with diet and medical treatment suggestions. The tongue image recognition syndrome differentiation analysis is the result of training with a large number of data samples. It can adapt cases according to the user's current symptoms and give users suggestions based on the experience results of the cases. The purpose of the online Chinese doctor's diagnosis is to verify the accuracy of the tongue image recognition. The combination of the two can improve the algorithm ability of deep learning.

[0059] Compared with the prior art, the present invention helps to realize the intelligentization of the Chinese medicine tongue diagnosis on the basis of tongue image processing and recognition. As a new health management and preventive treatment of disease in Chinese medicine mode that combines professional Chinese medicine tongue images and image recognition technology, it can generally provide users with convenient and professional Chinese medicine health management tools and diagnosis suggestions supported by data, which has practical and profound value and significance for user health management. Therefore, it is suitable for popularization and application in Chinese medicine diagnosis and treatment and has broad application prospects.

[0060] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, or improvements made within the technical scope of the present invention shall be included within the scope protected by the present invention.

Claims

1. An intelligent identification method for TCM tongue diagnosis based on tongue image processing, It is characterized in that The method is implemented based on a smart terminal, which has a preset App installed. The method includes the following steps: Step S1, the user uses a smart terminal to take a picture of the tongue, so that the App can obtain the picture of the tongue; Step S2, the App pre-processes the tongue image to obtain a tongue image that meets parameter requirements; Step S3, inputting the tongue image obtained by the preprocessing in step S2 into the image recognition buffer area of ​​the App; Step S4, the App recognizes the tongue image according to a preset tongue image data set, and then obtains corresponding disease recognition results and interpretation information; The step S1 includes the following process: The user opens the App through a smart terminal. The App provides the function of taking pictures and obtaining the camera permissions of the smart terminal. The camera permissions include the front camera permission and the rear camera shooting permission. The user switches the front and rear cameras as required; In step S2, the process of preprocessing the tongue image by the App includes: Step S20, image format cropping: using a sample parameter comparison method, the image parameters uploaded by the user are compared with the standard sample parameter values, and then the image uploaded by the user is cropped; Step S21, image processing judgment: After the image uploaded by the user is cropped, various parameter values ​​are calculated in real time, and the calculation results are saved as input values ​​of the comparison algorithm. The parameter values ​​of the standard sample are used as comparison input values, and the traversal algorithm is used to compare item by item, and finally the comparison results are output. If the image is qualified, the subsequent steps are continued. If the image is unqualified, the image is processed. During the image processing process, one or several parameters of the image are adjusted according to the results of the above traversal algorithm; In step S21, when performing image processing judgment, parameterized judgment is performed for five preset situations. If all parameters of the standard sample are met at the same time, the image recognition buffer area is entered. Otherwise, the parameters of the standard sample are compared with the user's captured image item by item according to the preset judgment algorithm. When comparing the parameters of the standard sample with the parameters of the image taken by the user, the types of parameters that need to be compared include: image fogging parameters, image contrast parameters, lossless magnification, whether to stretch, stretching parameters, image clarity and color parameters; In step S21, the five preset situations include: Image dehazing: Change the format of image features, extract non-hazed image pixel data, reconstruct and combine them to generate a non-hazed image; Image contrast enhancement: Brighten and enhance the dark images, and darken the overly bright or even light-blown images until the recognition feature requirements are met; Lossless image enlargement: Enlarge the image by 2-4 times according to the preset image pixel quality requirements to facilitate the extraction of subtle feature values ​​of the image; Restoring stretched images: compact rearrangement of image pixels, combined with conventional controls for reconstructing; Image clarity and color enhancement: Denoise the compressed blurred and low-pixel images. The denoising process includes image texture optimization, color saturation adjustment, brightness adjustment, and contrast adjustment; The establishment process of the tongue image dataset includes: Step S10, collect tongue image preprocessing samples. After the tongue image taken by the user through the intelligent terminal camera or professional camera is uploaded, crop the tongue image into the corresponding specifications; Step S11, extract the subtle features in the tongue image features through the special layer of the convolutional neural network, and then conduct item-by-item training to obtain the convolutional neural network; Step S12, process the features of each layer of the convolutional neural network to obtain the tongue image partition feature values; Step S13, present a feature value questionnaire for the user to conduct precise training and feature description in image recognition; Step S14, the intelligent terminal serves as a trainer, and uses fragmented time for timely tongue image training and questionnaire submission; Step S15, conduct continuous identification training on the same tongue image feature values to ensure the reliability and correctness of the image data in the tongue image dataset; Step S16, count the answer results; Step S17, obtain the tongue image feature value and disease interpretation dataset through machine training; In the said step S11, the convolutional neural network includes: Fully connected layer, which is a hidden layer of the convolutional neural network and contains a weight vector W and an activation function; Convolutional layer, which retains the spatial features of the input image. Through convolution operations, an image with pixel particle dimensions can be obtained. Usually, one-dimensional convolution and two-dimensional convolution are used, and 6 different convolution results are continuously stacked to output a convolution Kernel and extract the tongue image feature values; Pooling layer, used to compress the information of the original feature layer.

Images