Health management system based on tongue information
The health management system addresses subjective tongue diagnosis by using AI and machine learning to provide objective health scores and personalized strategies, enhancing the reliability and rationality of health management through automated tongue image analysis.
Patent Information
- Application Number
- CN202110181525.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-02-09
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-02-09
AI Technical Summary
Current tongue diagnosis in traditional Chinese medicine relies heavily on subjective observation, lacking objective judgment and leading to inconsistent results among practitioners.
A health management system utilizing tongue image acquisition, health status determination, intelligent recommendation, medical record management, search, and characteristic medicine recognition modules to provide automated and accurate tongue diagnosis, including health scores and personalized health management strategies.
Enables objective and accurate tongue diagnosis, facilitating personalized health management strategies and improving the reliability and rationality of health management by leveraging AI and machine learning for consistent and precise health assessments.
Smart Images

Figure CN112820370B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a health management system based on tongue image information. Background Art
[0002] In traditional Chinese medicine, observation, auscultation, inquiry, and palpation are collectively called the Four Diagnosis. Among them, "observation" refers to observing the complexion. Through thousands of years of practice, Chinese medicine has begun to realize that there is a close connection between physical manifestations, especially the tongue, and the internal organs. Many clinical studies have also confirmed that tongue images can reflect the overall state of the body, and play an important guiding role in clinical diagnosis, differentiation of symptoms, judgment of the severity of the disease, guidance of medication, and observation of treatment effects. Tongue diagnosis has been widely used in clinical practice, and occupies an important position in clinical Chinese medicine. It is an indispensable examination routine. It does not require additional instrument assistance. Doctors can understand a person's internal health status and treat symptoms just by observing the tongue.
[0003] However, current tongue diagnosis is mainly based on subjective observation of traditional Chinese medicine practitioners and lacks objective judgment basis. The results of tongue diagnosis are closely related to the doctor's medical skills and treatment ideas. For the same patient, two traditional Chinese medicine practitioners may get different results and take different measures. Therefore, how to use modern computer technology to achieve objective and accurate automated tongue diagnosis has become an important issue that needs to be solved urgently. Summary of the invention
[0004] In view of the problems existing in the prior art, the present invention provides a health management system based on tongue image information.
[0005] The present invention provides a health management system based on tongue image information, comprising: a tongue image acquisition module, used to acquire a tongue image to be identified; a health status determination module, used to obtain a comprehensive health status reflected by the tongue image according to the tongue image, wherein the comprehensive health status includes a health score.
[0006] According to a health management system based on tongue image information provided by the present invention, the system also includes an intelligent recommendation module, and the intelligent recommendation module is used to automatically recommend health management strategies according to the comprehensive health status.
[0007] According to a health management system based on tongue image information provided by the present invention, the health management strategy includes at least one of a diet therapy strategy, a drug therapy strategy, a physical therapy strategy, a music therapy strategy and an exercise prescription.
[0008] According to a health management system based on tongue image information provided by the present invention, the system also includes a medical record management module, which is used to: establish a tracking file for the patient; wherein the tracking file includes the tongue image, the comprehensive health status obtained based on the tongue image, the health management strategy and at least one of the data acquisition time.
[0009] According to a health management system based on tongue image information provided by the present invention, the system also includes a search module, and the search module is used to find similar cases by comparing the similarity of the tongue image pictures.
[0010] According to a health management system based on tongue image information provided by the present invention, the system also includes a characteristic medicine recognition module, and the characteristic medicine recognition module is used to: input the tongue image picture into at least one characteristic medicine recognition model corresponding to the preset health indicator respectively, and output the characteristic medicine recognition result according to the characteristic medicine recognition model; wherein the characteristic medicine recognition result includes the occurrence probability of the preset characteristic medicine in the treatment prescription of the tongue image, and the preset characteristic medicine is a medicine used to improve the problem of the preset health indicator, and the preset characteristic medicine is one or more; when the health status determination module is used to obtain the comprehensive health status reflected by the tongue image picture according to the tongue image picture, it is specifically used to: determine the status of the corresponding preset health indicator according to the characteristic medicine recognition result, and obtain the comprehensive health status reflected by the tongue image picture by comprehensively combining the status of the preset health indicators obtained respectively.
[0011] According to a health management system based on tongue image information provided by the present invention, the status of the preset health indicator includes the probability of problems with the preset health indicator; the health status determination module is used to comprehensively obtain the status of the preset health indicators obtained separately to obtain the comprehensive health status reflected by the tongue image, and is specifically used to: comprehensively obtain the probabilities of problems with the preset health indicators obtained separately to obtain the health score.
[0012] According to a health management system based on tongue image information provided by the present invention, the comprehensive health status also includes a feature vector whose dimension corresponds to the number of the preset health indicators, wherein the feature in the feature vector is the probability of problems occurring in the preset health indicators.
[0013] According to a health management system based on tongue image information provided by the present invention, the intelligent recommendation module is specifically used to: obtain the health management strategy corresponding to the status of the preset health indicator according to a preset mapping relationship table, and output and display at least one of the health management strategies; the system also includes a health status display module, and the health status display module is used to: sort the preset health indicators in order from high to low according to the probability of problems with the preset health indicators, obtain a preset number of the preset health indicators with top sorting, and output and display the symptom description when the preset number of the preset health indicators have problems, the probability of problems with the preset health indicators and the health score.
[0014] According to a health management system based on tongue image information provided by the present invention, the tongue image picture acquisition module is specifically used to: shoot a tongue image video, and select a frame of photo with the best image quality from the tongue image video as the tongue image to be identified.
[0015] The health management system based on tongue image information provided by the present invention realizes automated tongue diagnosis by obtaining a comprehensive health status including a health score based on tongue image, which is helpful for grasping the health level as a whole. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0017] Figure 1 It is a structural schematic diagram of the health management system based on tongue image information provided by the present invention. DETAILED DESCRIPTION
[0018] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0019] Combine the following Figure 1 The health management system based on tongue image information of the present invention is described.
[0020] Figure 1 Schematic diagram of the structure of the health management system based on tongue image information provided by the present invention. Figure 1 As shown, the system includes a tongue image acquisition module 10 and a health status determination module 20, wherein: the tongue image acquisition module 10 is used to: obtain the tongue image to be identified; the health status determination module 20 is used to: obtain the comprehensive health status reflected by the tongue image according to the tongue image, and the comprehensive health status includes a health score.
[0021] The tongue image acquisition module 10 is used to acquire the tongue image to be identified. In the health management system based on tongue image information provided by the present invention, the tongue image acquisition module 10 is free to take pictures when acquiring the tongue image to be identified, such as supporting mobile phone shooting, ordinary camera shooting, professional tongue diagnosis equipment shooting, etc., and qualified quality photos can be used. The box used to isolate external light on the general tongue diagnosis instrument can be eliminated, and the camera can be used directly to take pictures of the face and tongue, and the optimized design of the fill light and the image processing algorithm can be used to reduce the impact of changes in ambient light on the tongue color, so that patients can collect tongue images more comfortably, freely, and experience better.
[0022] The health status determination module 20 is used to obtain the comprehensive health status reflected by the tongue image according to the tongue image, and the comprehensive health status includes a health score. The health status determination module 20 can obtain the comprehensive health status according to the tongue image by a machine learning method. Taking obtaining a health score as an example, a health scoring model can be pre-trained, the scoring criteria and scoring level of the tongue image can be set, and then a large number of tongue image samples can be scored and annotated to obtain a scored and annotated sample. The health scoring model is obtained by training with the tongue image sample as input and the score annotation result as output. In actual use, the tongue image to be identified is input into the health scoring model, and the health score is output according to the health scoring model.
[0023] The health management system based on tongue image information provided by the present invention realizes automated tongue diagnosis by obtaining a comprehensive health status including a health score based on tongue image, which is helpful for grasping the health level as a whole.
[0024] According to a health management system based on tongue image information provided by the present invention, the system also includes an intelligent recommendation module, and the intelligent recommendation module is used to automatically recommend health management strategies according to the comprehensive health status.
[0025] The intelligent recommendation module can automatically recommend health management strategies based on the comprehensive health status. For example, a dietary strategy can be recommended based on the score range of the health score. When recommending a dietary strategy, for example, the lower the health score, the worse the physical condition, and the dietary strategy can be strengthened, such as increasing the intake of nutrients. Backend medical big data can be set up, and the backend medical big data can be queried based on the comprehensive health status to obtain the corresponding health management strategy and make automatic recommendations.
[0026] The health management system based on tongue image information provided by the present invention realizes the automatic acquisition of health management strategies by setting up an intelligent recommendation module to automatically recommend health management strategies according to the comprehensive health status.
[0027] According to a health management system based on tongue image information provided by the present invention, the health management strategy includes at least one of a diet therapy strategy, a drug therapy strategy, a physical therapy strategy, a music therapy strategy and an exercise prescription.
[0028] Health management strategies can include many types, such as diet therapy strategies, drug therapy strategies, physical therapy strategies, music therapy strategies, and exercise prescriptions. Diet therapy strategies include tea, health products, and ordinary food. Drug therapy strategies include prescription drugs, OTC non-prescription drugs, etc. Physical therapy strategies include massage, etc. Exercise prescriptions include exercise therapy plans, etc. In addition, music can also play a role in health care. For various health management strategies, detailed links can be displayed, such as corresponding drug names, displaying drug detailed links; for song names, displaying song links, etc.
[0029] The health management system based on tongue image information provided by the present invention enriches the types of health management strategies by setting health management strategies including at least one of diet therapy strategy, drug therapy strategy, physical therapy strategy, music therapy strategy and exercise prescription.
[0030] According to a health management system based on tongue image information provided by the present invention, the system also includes a medical record management module, which is used to: establish a tracking file for the patient; wherein the tracking file includes the tongue image, the comprehensive health status obtained based on the tongue image, the health management strategy and at least one of the data acquisition time.
[0031] A tracking file can be established for the patient through the medical record management module, and the tracking file includes the results of previous measurements. The measurement data can be uploaded regularly. For example, the tracking file can include at least one of the tongue image, the comprehensive health status obtained based on the tongue image, the health management strategy and the data acquisition time, so as to facilitate the observation of health change trends. The health management system based on tongue image information provided by the present invention can not only achieve the prevention of disease, that is, predict disease and intervene in advance; it can also achieve prognosis management.
[0032] The health management system based on tongue image information provided by the present invention improves the rationality of health management by establishing a tracking file.
[0033] According to a health management system based on tongue image information provided by the present invention, the system also includes a search module, and the search module is used to find similar cases by comparing the similarity of the tongue image pictures.
[0034] Similar cases can be found by comparing the similarity of tongue images, such as by extracting features from tongue images, comparing the similarity between features extracted by neural networks, or comparing the similarity between feature vectors output by neural networks. After finding similar cases, the tracking files of similar cases can be consulted for reference in health management.
[0035] This system can also provide an open network platform interface to support scientific research and applications.
[0036] The health management system based on tongue image information provided by the present invention further improves the reliability of health management by providing a search function for finding similar cases.
[0037] According to a health management system based on tongue image information provided by the present invention, the system also includes a characteristic medicine recognition module, and the characteristic medicine recognition module is used to: input the tongue image picture into at least one characteristic medicine recognition model corresponding to the preset health indicator respectively, and output the characteristic medicine recognition result according to the characteristic medicine recognition model; wherein the characteristic medicine recognition result includes the occurrence probability of the preset characteristic medicine in the treatment prescription of the tongue image, and the preset characteristic medicine is a medicine used to improve the problem of the preset health indicator, and the preset characteristic medicine is one or more; the health status determination module 20, when used to obtain the comprehensive health status reflected by the tongue image picture according to the tongue image picture, is specifically used to: determine the status of the corresponding preset health indicator according to the characteristic medicine recognition result, and obtain the comprehensive health status reflected by the tongue image picture by combining the status of the preset health indicators obtained respectively.
[0038] The characteristic drug recognition module is used to input the tongue image into at least one characteristic drug recognition model corresponding to the preset health index, and output the characteristic drug recognition result corresponding to the tongue image according to the characteristic drug recognition model. Multiple health indicators that are really useful in clinical and health care can be pre-set as preset health indicators, and the characteristic drug recognition models for each preset health index can be trained respectively. The end-to-end deep learning algorithm is used to solve the tongue diagnosis problem. By collecting the tongue image and the corresponding treatment prescription big data of the Chinese medicine doctor when treating the disease, the mapping relationship between the tongue image and the prescription is directly learned. The tongue image recognition problem is converted into a tongue image sample labeling problem. When training a characteristic drug recognition model corresponding to a preset health index, the tongue image sample can be used as input and the medication situation of this preset health index in the prescription can be used as the output label for machine learning training. For example, for a certain tongue image sample, if the prescription contains a drug for treating this health index problem, the corresponding drug is marked as 1. Otherwise, it is marked as 0. The end-to-end learning method can fully utilize the features reflected in the image, and can effectively avoid the disadvantages of artificially selected features that are not comprehensive, difficult to quantify, unprofessional, and difficult to combine with the treatment plan.
[0039] The characteristic medicine recognition module inputs the tongue image into at least one characteristic medicine recognition model corresponding to the preset health indicator, and outputs the characteristic medicine recognition result according to the characteristic medicine recognition model. The characteristic medicine recognition module can simultaneously input the tongue image into characteristic medicine recognition models corresponding to multiple preset health indicators, and each characteristic medicine recognition model outputs a characteristic medicine recognition result based on the tongue image and corresponding to each preset health indicator. Among them, the characteristic medicine recognition result corresponding to the tongue image includes the probability of occurrence of the preset characteristic medicine in the treatment prescription of the tongue image, and the preset characteristic medicine is a medicine used to improve the problem of the preset health indicator, and the preset characteristic medicine is one or more.
[0040] The health status determination module 20 is used to determine the status of the corresponding preset health indicator according to the characteristic drug identification result, and to obtain the comprehensive health status reflected by the tongue image by combining the status of the preset health indicators obtained respectively. Since each characteristic drug identification model outputs a characteristic drug identification result based on the tongue image and corresponding to each preset health indicator, the output characteristic drug identification result can indicate whether there is a problem with the corresponding preset health indicator, and can also indicate the degree or probability of the corresponding preset health indicator having a problem. For example, if there is no drug for treating the corresponding health indicator problem in the output characteristic drug identification result, it means that there is no problem with the corresponding health indicator; if there is a drug for treating the corresponding health indicator problem in the output characteristic drug identification result, it means that there is a problem with the corresponding health indicator; the more types of drugs for treating the corresponding health indicator problem appear and the larger the dosage, the more serious the problem of the corresponding health indicator or the higher the probability of the problem occurring.
[0041] Therefore, each characteristic drug recognition model outputs a characteristic drug recognition result corresponding to the corresponding preset health indicator based on the tongue image. The status of each preset health indicator can be obtained according to the characteristic drug recognition result output by each characteristic drug recognition model, and the comprehensive health status reflected by the tongue image can be obtained by combining the status of each preset health indicator. The comprehensive health status may include a collection of the status of each preset health indicator. According to the comprehensive health status, it can be known which preset health indicators have problems.
[0042] The health management system based on tongue image information provided by the present invention obtains the characteristic medicine recognition result corresponding to the tongue image by utilizing the characteristic medicine recognition model, obtains the comprehensive health status according to the characteristic medicine recognition result, and realizes the acquisition of health status refined to health indicators.
[0043] According to the health management system based on tongue image information provided by the present invention, the status of the preset health indicator includes the probability of problems with the preset health indicator; the health status determination module 20 is used to comprehensively obtain the status of the preset health indicators obtained separately to obtain the comprehensive health status reflected by the tongue image, and is specifically used to: comprehensively obtain the probabilities of problems with the preset health indicators obtained separately to obtain the health score.
[0044] The status of the preset health indicator includes the probability of a problem with the preset health indicator, and a probability threshold may be set. When the probability of a problem with the preset health indicator is greater than the probability threshold, it is considered that a problem occurs with the corresponding preset health indicator.
[0045] When the health status determination module 20 is used to comprehensively obtain the states of the preset health indicators respectively obtained to obtain the comprehensive health status reflected by the tongue image, the health score is obtained by comprehensively obtaining the probabilities of problems with the preset health indicators respectively obtained. For example, a probability value reflecting the overall health status is obtained by weighted summing the probabilities of problems with each preset health indicator, and then the probability value is multiplied by 100 to obtain a health score reflecting the comprehensive health status.
[0046] The health management system based on tongue image information provided by the present invention obtains the health score by comprehensively analyzing the probabilities of problems occurring in the preset health indicators, thereby improving the accuracy of the health score.
[0047] According to a health management system based on tongue image information provided by the present invention, the comprehensive health status also includes a feature vector whose dimension corresponds to the number of the preset health indicators, wherein the feature in the feature vector is the probability of problems occurring in the preset health indicators.
[0048] As described above, the comprehensive health status may include a set of states of various preset health indicators. The set of states of various preset health indicators may be represented by a feature vector having a dimension corresponding to the number of the preset health indicators, wherein the feature in the feature vector is the probability of a problem with the preset health indicator.
[0049] The health management system based on tongue image information provided by the present invention enriches and standardizes the output of comprehensive health status by outputting a feature vector whose feature is the probability of problems occurring in preset health indicators.
[0050] According to a health management system based on tongue image information provided by the present invention, the intelligent recommendation module is specifically used to: obtain the health management strategy corresponding to the status of the preset health indicator according to a preset mapping relationship table, and output and display at least one of the health management strategies; the system also includes a health status display module, and the health status display module is used to: sort the preset health indicators in order from high to low according to the probability of problems with the preset health indicators, obtain a preset number of the preset health indicators with top sorting, and output and display the symptom description when the preset number of the preset health indicators have problems, the probability of problems with the preset health indicators and the health score.
[0051] A mapping relationship table between the status of each preset health indicator and the health management strategy can be preset, and a preset health indicator can correspond to multiple categories and multiple health management strategies. After obtaining the comprehensive health status, the health management strategy corresponding to the status of each preset health indicator is obtained according to the mapping relationship table between the status of each preset health indicator and the health management strategy in the comprehensive health status, and the health management strategies corresponding to each preset health indicator are integrated to obtain the total health management strategy. At least one health management strategy can be output and displayed according to the total health management strategy. When displaying the health management strategy, it can be displayed in correspondence with the status of the corresponding preset health indicator.
[0052] The health management system based on tongue image information also includes a health status display module, which is used to sort the preset health indicators in descending order according to the probability of problems with the preset health indicators, obtain a preset number of preset health indicators with the top sorting, and output and display the symptom description when problems occur with the preset number of preset health indicators, the probability of problems with the preset health indicators, and the health score.
[0053] For example, the display content of the health status display module can be shown in Table 1:
[0054] Table 1
[0055]
[0056]
[0057] The health management system based on tongue image information provided by the present invention obtains the health management strategy corresponding to the state of the preset health indicators according to the preset mapping relationship table, and outputs and displays at least one health management strategy, thereby realizing the rapid acquisition of the health management strategy; by sorting the preset health indicators in order from high to low according to the probability of problems with the preset health indicators, obtaining a preset number of preset health indicators with top sorting, and outputting and displaying the symptom description when the preset number of preset health indicators have problems, the probability of problems with the preset health indicators and the health score, thereby realizing the reasonable display of the health status.
[0058] According to a health management system based on tongue image information provided by the present invention, the tongue image picture acquisition module 10 is specifically used to: shoot a tongue image video, and select a frame of photo with the best image quality from the tongue image video as the tongue image to be identified.
[0059] The better the image quality of the tongue image, the easier it is to perform feature recognition. The quality of the tongue image can be easily affected by hand shaking when taking the tongue image. Therefore, the image quality of the tongue image can be improved by taking a video of the tongue image and selecting a frame of the tongue image with the best image quality from the tongue image video as the tongue image to be identified. That is, by replacing the camera with a video camera, a continuous video image is dynamically captured, the best static photo is automatically taken out from it, and the dynamic process of tongue extension can be recorded and saved.
[0060] When collecting tongue images, the tongue detection and segmentation algorithm can be used to intelligently detect the degree of tongue extension and image quality. Based on the detection results, the instrument is controlled to start and complete the collection work in a timely manner.
[0061] In addition, after obtaining the tongue image, further image preprocessing can be performed. Image preprocessing can include, for example, adaptive correction of color to make the processed tongue color as close as possible to the color of the standard light source. In this way, the learning algorithm's dependence on color is reduced, the stability and consistency of each measurement are improved, and the restrictions on the environment are relaxed.
[0062] The health management system based on tongue image information provided by the present invention effectively improves the quality of tongue image pictures by shooting tongue image video and selecting a frame of photo with the best image quality from the tongue image video as the tongue image to be identified.
[0063] According to a health management system based on tongue image information provided by the present invention, the system also includes a training module, which is used to: construct training samples based on cases of a certain doctor or school, and perform machine learning training based on the training samples to obtain at least one characteristic drug identification model corresponding to the preset health indicators.
[0064] Different doctors or schools may give different prescriptions for the same tongue image. The health management system based on tongue image information provided by the present invention can learn the prescriptions of specific doctors or schools respectively, so as to obtain different characteristic drug recognition models. Each school and master corresponds to a set of AI neural networks, which can simulate the corresponding doctors and schools to give prescriptions. In addition, a multi-AI consultation situation can be formed to increase treatment ideas.
[0065] Therefore, when training the characteristic drug identification model, the training module can construct training samples based on the cases of a certain doctor or school, and perform machine learning training based on the training samples to obtain at least one characteristic drug identification model corresponding to the preset health indicators.
[0066] According to the health management system based on tongue image information provided by the present invention, training samples are constructed based on the cases of a certain doctor or school, and machine learning training is performed based on the training samples to obtain at least one characteristic drug recognition model corresponding to preset health indicators, thereby realizing the diagnosis and treatment simulation of a specific doctor or school.
[0067] According to a health management system based on tongue image information provided by the present invention, the training samples include tongue image picture samples and corresponding treatment prescriptions; when the training module is used to perform machine learning training based on the training samples to obtain the at least one characteristic drug recognition model corresponding to the preset health indicator, it is specifically used to: use the tongue image sample as input and the annotation result of whether the preset characteristic drug corresponding to the preset health indicator appears in at least one treatment prescription corresponding to the tongue image sample, or the annotation result of whether the preset characteristic drug appears and the dosage of the preset characteristic drug when it appears as output, to perform machine learning training to obtain the at least one characteristic drug recognition model corresponding to the preset health indicator.
[0068] Specifically, when training the characteristic drug recognition model corresponding to the preset health index, the training module can use the tongue image sample as input and the labeling result of whether at least one preset characteristic drug appears in the treatment prescription corresponding to the tongue image sample as output, and perform machine learning training to obtain the characteristic drug recognition model. For example, taking the qi and blood running index as an example, first determine the medicine for qi and blood running (the appearance of this medicine indicates that the qi and blood running is blocked and needs to be unblocked, for example, there are m in total). During training, the tongue image sample is used as input, and the output end labels the above m drugs. For example, if drug 1 appears, drug 1 is labeled as 1; if drug 2 appears, drug 2 is labeled as 2; ... If drug m appears, drug m is labeled as m; if the corresponding drug does not appear, it is labeled as 0. Then, the tongue image sample is used as input and the prescription labeling result is used as the output label to perform machine learning training to obtain the characteristic drug recognition model. When the characteristic drug recognition model obtained by the above training method is used, the input is the tongue image, and the output is the probability of occurrence of m drugs. A probability threshold may be set, and when the probability of a drug appearing is greater than the probability threshold, it indicates that the corresponding drug appears in the prescription.
[0069] The training module can also use the tongue image sample as input, and the labeling result of whether the preset characteristic medicine appears and the dosage when the preset characteristic medicine appears as output, to perform machine learning training, and obtain at least one characteristic medicine recognition model corresponding to the preset health index. Similarly, taking the qi and blood running index as an example, first determine the medicine for qi and blood running (the appearance of this medicine indicates that the qi and blood running is blocked and needs to be unblocked, for example, there are m in total). During training, the tongue image sample is used as input, and the output end labels the above m medicines. The labels corresponding to the dosage ranges of various drugs can be pre-set. For example, drug 1, dosage <10g, label 11; drug 1, 10g < dosage <20g, label 12; ... drug m, dosage <5g, label m1; drug m, 5g < dosage <10g, label m2, etc. If the corresponding drug does not appear, the label is 0. Then use the tongue image sample as input and the prescription labeling result as the output label to perform machine learning training to obtain the characteristic medicine recognition model. When using the characteristic drug recognition model obtained by the above training method, the input is the tongue image, and the output is the probability of m drugs not appearing and the probability of appearing in each dosage range. A probability threshold can be set to determine whether a drug appears and in which dosage range it appears.
[0070] End-to-end learning can achieve a certain accuracy rate using only the general basic process of deep learning, but it is far from enough. The main reason is that deep learning requires a large number of samples. These samples must cover every possible situation, and each situation must be guaranteed to have a considerable number in order to achieve a relatively ideal effect. In the problem of tongue diagnosis, it is impossible to get so many samples. In addition to generating virtual samples (such as adding noise) to overcome the problem of insufficient samples, the accuracy of model recognition can be improved by combining technical means with tongue diagnosis expertise when training and applying models. For example, through communication with experts and self-learning, it is necessary to find out what factors on the tongue map determine the use of a certain medication. Then analyze and count the tongue images that the model recognizes inaccurately, and find ways to improve the model. You can also use the compatibility and repulsion constraints between drugs (use this constraint relationship to correct the results) to improve the accuracy, etc. In addition, further improving the recognition accuracy of the model from a technical perspective may include: setting an appropriate image resolution suitable for recognition; setting a reasonable feature extraction area on the tongue image, which may be a single area or multiple areas; setting the color, position, size, and aspect ratio of the image of each area (because the shapes of different people are different, normalization can be performed).
[0071] The health management system based on tongue image information provided by the present invention uses tongue image samples as input and uses the annotation results of whether at least one preset characteristic drug appears in the treatment prescription corresponding to the tongue image samples, or the annotation results of whether the preset characteristic drug appears and the dosage of the preset characteristic drug when it appears, as output, to perform machine learning training, thereby obtaining at least one characteristic drug recognition model corresponding to the preset health indicators, thereby enriching the functions of the characteristic drug recognition model.
[0072] According to a health management system based on tongue image information provided by the present invention, the health status determination module 20, when used to determine the status of the corresponding preset health indicator according to the characteristic drug identification result, is specifically used to: determine the probability of a problem with the preset health indicator according to the number of the preset characteristic drugs appearing in the treatment prescription; or, determine the probability of a problem with the preset health indicator according to the number of the preset characteristic drugs appearing in the treatment prescription and the dosage of the preset characteristic drugs.
[0073] The training process of the characteristic medicine recognition model can be: taking the tongue image sample as input and taking the labeling result of whether at least one preset characteristic medicine appears in the prescription corresponding to the tongue image sample as output, performing machine learning training, and obtaining at least one characteristic medicine recognition model corresponding to the preset health indicator. In this case, when the characteristic medicine recognition model is used to identify the tongue image, the above-mentioned label setting is still used as an example. If the probability of the output of the neural network corresponding to drug 1 is 1, if it is greater than the preset probability threshold, it means that the prescription includes drug 1; if the probability of the output of the neural network corresponding to drug 2 is 2, if it is greater than the preset probability threshold, it means that the prescription includes drug 2; if the probability of the output of the neural network corresponding to drug 3 is 3, if it is less than the preset probability threshold, it means that the prescription does not include drug 3... By analogy, it can be obtained which medicines are included in the output prescription corresponding to the input tongue image, or which medicines appear. If the number of medicines that appear is greater, it means that the probability of the corresponding preset health indicator having a problem is greater, or the condition is more serious. Therefore, the probability of the preset health indicator having a problem can be determined according to the number of preset characteristic medicines appearing in the treatment prescription.
[0074] The training process of the characteristic drug recognition model can also be: taking the tongue image sample as input, taking the labeling result of whether at least one preset characteristic drug appears in the prescription corresponding to the tongue image sample and the dosage when the preset characteristic drug appears as output, performing machine learning training, and obtaining at least one characteristic drug recognition model corresponding to the preset health index. In this case, when the characteristic drug recognition model is used to identify the tongue image, the above-mentioned label setting is still used as an example. If the probability of one of the output ends of the neural network corresponding to drug 1 outputting 11 is greater than the preset probability threshold, it means that the prescription includes drug 1 and the dosage is <10g; if the probability of one of the output ends of the neural network corresponding to drug m outputting m2 is greater than the preset probability threshold, it means that the prescription includes drug m, and 5g<dose<10g, if the output probability is less than the preset probability threshold, it means that the prescription does not include drug m... By analogy, it is possible to obtain which medicines are included in the output prescription based on the input tongue image and the predicted value of the dosage of the medicine. If the number of medicines that appear is more and the dosage is larger, it means that the probability of the corresponding preset health index having problems is greater, or the condition is more serious. Therefore, the probability of problems with preset health indicators can be determined based on the number of preset characteristic drugs appearing in the treatment prescription and the dosage of the preset characteristic drugs.
[0075] The health management system based on tongue image information provided by the present invention improves the accuracy of determining the status of preset health indicators by determining the probability of problems with preset health indicators based on the number of preset characteristic drugs appearing in the treatment prescription, or by determining the probability of problems with preset health indicators based on the number of preset characteristic drugs appearing in the treatment prescription and the dosage of the preset characteristic drugs.
[0076] Table 2 shows the difference between the health management system based on tongue image information (TH-Health tongue diagnosis system) provided by the present invention and the general tongue diagnosis system.
[0077] Table 2
[0078]
[0079]
[0080] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.
[0081] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A health management system based on tongue image information, characterized in that: include: The tongue image acquisition module is used to: acquire the tongue image to be identified; A health status determination module, used to: obtain the comprehensive health status reflected by the tongue image according to the tongue image, wherein the comprehensive health status includes a health score; The system further includes a characteristic drug identification module, which is used to: The tongue image is input into at least one characteristic medicine recognition model corresponding to each preset health indicator, and a characteristic medicine recognition result is output according to the characteristic medicine recognition model; wherein the characteristic medicine recognition result includes the probability of occurrence of a preset characteristic medicine in the treatment prescription of the tongue image, the preset characteristic medicine is a medicine used to improve the problem of the preset health indicator, the preset characteristic medicine is one or more, and the characteristic medicine recognition result indicates whether there is a problem with the corresponding preset health indicator, and the degree or probability of the problem with the corresponding preset health indicator; When the health status determination module is used to obtain the comprehensive health status reflected by the tongue image according to the tongue image, it is specifically used to: The status of the corresponding preset health indicator is determined according to the characteristic drug identification result, and the comprehensive health status reflected by the tongue image is obtained by combining the status of the preset health indicators obtained respectively.
2. The health management system based on tongue image information according to claim 1, characterized in that: The system further includes an intelligent recommendation module, which is used to: Automatically recommend health management strategies based on the comprehensive health status.
3. The health management system based on tongue image information according to claim 2, characterized in that: The health management strategy includes at least one of a diet therapy strategy, a drug therapy strategy, a physical therapy strategy, a music therapy strategy and an exercise prescription.
4. The health management system based on tongue image information according to claim 2, characterized in that: The system also includes a medical record management module, which is used to: A tracking file is established for the patient; wherein the tracking file includes at least one of the tongue image, the comprehensive health status obtained based on the tongue image, the health management strategy and the data acquisition time.
5. The health management system based on tongue image information according to claim 4, characterized in that: The system further comprises a search module, wherein the search module is used to: Similar cases are found by comparing the similarities of the tongue image pictures.
6. The health management system based on tongue image information according to claim 2, characterized in that: The state of the preset health indicator includes the probability of a problem with the preset health indicator; when the health state determination module is used to obtain the comprehensive health state reflected by the tongue image by comprehensively obtaining the states of the preset health indicators obtained respectively, it is specifically used to: The health score is obtained by combining the probabilities of problems with the preset health indicators obtained respectively.
7. The health management system based on tongue image information according to claim 2, characterized in that: The comprehensive health status also includes a feature vector whose dimension corresponds to the number of the preset health indicators, wherein the feature in the feature vector is the probability of a problem with the preset health indicator.
8. The health management system based on tongue image information according to claim 2, characterized in that: The intelligent recommendation module is specifically used to: obtain the health management strategy corresponding to the state of the preset health indicator according to the preset mapping relationship table, and output and display at least one of the health management strategies; The system also includes a health status display module, which is used to: sort the preset health indicators in descending order according to the probability of problems with the preset health indicators, obtain a preset number of the preset health indicators with the top sorting, and output and display the symptom description when problems occur with the preset number of the preset health indicators, the probability of problems with the preset health indicators, and the health score.
9. The health management system based on tongue image information according to claim 1, characterized in that: The tongue image acquisition module is specifically used for: A tongue image video is taken, and a frame of photo with the best image quality is selected from the tongue image video as the tongue image to be identified.
