Traditional Chinese Medicine Health Status Analysis System Based on Artificial Intelligence

Through artificial intelligence technology, the tongue image is segmented and feature extracted, and combined with the gating mechanism to form a multi-scale feature representation of the tongue state, realizing intelligent discrimination of the shape and quality of tongue coating, solving the problem of traditional Chinese medicine's reliance on doctors' experience and improving the Diagnostic efficiency and accuracy.

CN118645250BActive Publication Date: 2024-12-03HANGZHOU LIUZHI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411121160.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-12-03
Estimated Expiration
2044-08-15

AI Technical Summary

Technical Problem

Traditional Chinese medicine tongue diagnosis and diagnosis methods require high experience for doctors, which leads to patients waiting for a long time when medical resources are tight, affecting the timely diagnosis and treatment of the disease.

Method used

Develop a traditional Chinese medicine health status analysis system based on artificial intelligence. Through tongue image acquisition, tongue area segmentation, LBP encoding, visual semantic feature extraction and gated dynamic interaction module, multi-scale features of tongue state are extracted to achieve tongue coating shape Quality automatic identification.

Benefits of technology

Users can automatically recognize tongue images online by uploading tongue images, saving long-term waiting and improving diagnostic efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118645250B_ABST
    Figure CN118645250B_ABST
Patent Text Reader

Abstract

The present application relates to a TCM health status analysis system based on artificial intelligence. It includes: combining artificial intelligence technology, after performing tongue image region segmentation on the tongue image, extracting its multi-scale tongue state features, and dynamically interacting the multi-scale tongue state features based on the gating mechanism to form a multi-scale feature representation of the tongue state, and intelligently distinguishing the type of tongue coating shape and quality based on the multi-scale feature representation of the tongue state. In this way, users can realize online automatic tongue image recognition by uploading their own tongue image, without waiting for a long time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of TCM health status, and specifically, to a TCM health status analysis system based on artificial intelligence. Background Art

[0002] As an important part of traditional Chinese medicine, tongue diagnosis is one of the diagnostic methods of TCM. It is to understand the physiological and pathological changes of the human body by observing the tongue, and then judge the health status and diagnose the disease. However, this diagnostic method requires a lot of experience from the doctor. In the case of tight medical resources, patients may have to wait for a long time, which not only increases the psychological and physical burden of patients, but also may affect the timely diagnosis and treatment of diseases.

[0003] With the rapid development of artificial intelligence technology, image recognition technology is increasingly used in the field of medical health. In view of the above technical problems, we look forward to an artificial intelligence-based TCM health status analysis system. Summary of the invention

[0004] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] In the first aspect, the present application provides a TCM health status analysis system based on artificial intelligence, the system comprising: a tongue image acquisition module for acquiring a tongue image uploaded by a user; a tongue region segmentation module for performing tongue region segmentation on the tongue image to obtain a tongue target region image; an LBP encoding module for encoding the tongue target region image using an LBP pattern operator to obtain a tongue image LBP encoding feature vector; a visual semantic feature extraction module for inputting the tongue target region image into a visual encoder to obtain a tongue state semantic encoding feature vector; a dynamic interaction module for inputting the tongue image LBP encoding feature vector and the tongue state semantic encoding feature vector into a feature vector dynamic interaction synthesis module under gated control to obtain a tongue state depth feature fusion representation vector; a tongue coating shape and quality analysis result generation module for determining a tongue coating shape and quality analysis result based on the tongue state depth and shallow feature fusion representation vector, the tongue coating shape and quality analysis result being used to represent a type label of the tongue coating shape and quality.

[0006] Optionally, the tongue area segmentation module is used to: input the tongue image into a tongue target area detection network based on a Yolo model to obtain the tongue target area image.

[0007] Optionally, the visual semantic feature extraction module is used to: input the tongue target area image into a visual encoder based on a ViT model to obtain the tongue state semantic encoding feature vector.

[0008] Optionally, the dynamic interaction module includes: a tongue multi-scale coding information union unit, used to input the tongue image LBP coding feature vector and the tongue state semantic coding feature vector into a feature union module to obtain a tongue LBP coding feature-tongue state semantic coding feature joint vector; a gated response unit, used to input the tongue LBP coding feature-tongue state semantic coding feature joint vector into a gated response function to obtain a response gate for tongue multi-scale coding information fusion; a tongue multi-scale coding information fusion representation unit, used to calculate the position-weighted sum of the tongue image LBP coding feature vector and the tongue state semantic coding feature vector using the response gate of the tongue multi-scale coding information fusion as a weight to obtain the tongue state depth feature fusion representation vector.

[0009] Optionally, the tongue multi-scale coding information combining unit is used to: cascade the tongue image LBP coding feature vector and the tongue state semantic coding feature vector to obtain the tongue LBP coding feature-tongue state semantic coding feature joint vector.

[0010] Optionally, the gated response unit is used to: calculate the matrix product of the weight parameter vector and the tongue LBP coding feature-tongue state semantic coding feature joint vector, add the value obtained by the matrix product to the bias term, and perform nonlinear processing on the value obtained by the addition to obtain the response gate of the tongue multi-scale coding information fusion.

[0011] Optionally, the nonlinear processing includes using a sigmoid function to process the value obtained by the addition to obtain a response gate for the fusion of the tongue multi-scale encoding information.

[0012] Optionally, the tongue multi-scale coding information fusion representation unit is used to: construct the tongue image LBP coding fusion weight and the tongue state semantic coding fusion weight based on the response gate of the tongue multi-scale coding information fusion; use the tongue image LBP coding fusion weight as the weight of the tongue image LBP coding feature vector, use the tongue state semantic coding fusion weight as the weight of the tongue state semantic coding feature vector, calculate the position-weighted sum of the tongue image LBP coding feature vector and the tongue state semantic coding feature vector to obtain the tongue state depth feature fusion representation vector.

[0013] Optionally, the tongue coating shape and quality analysis result generating module is used to: input the tongue state depth feature fusion representation vector into a classifier-based tongue coating shape and quality analyzer to obtain the tongue coating shape and quality analysis result.

[0014] By adopting the above technical solution, combined with artificial intelligence technology, after the tongue image is segmented, the multi-scale tongue state features are extracted, and the multi-scale tongue state features are dynamically interacted based on the gating mechanism to form a multi-scale feature representation of the tongue state, and the type of tongue coating shape and quality is intelligently distinguished based on the multi-scale feature representation of the tongue state. In this way, users can realize online automatic tongue image recognition by uploading their own tongue image, saving a long wait.

[0015] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and other features, advantages and aspects of the embodiments of the present application will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0017] In the attached picture: Figure 1 It is a block diagram of a TCM health status analysis system based on artificial intelligence according to an exemplary embodiment.

[0018] Figure 2 It is a flow chart of a method for analyzing health status of traditional Chinese medicine based on artificial intelligence according to an exemplary embodiment.

[0019] Figure 3 It is a schematic diagram of the thinking principle of TCM health status identification according to an exemplary embodiment.

[0020] Figure 4 It is a block diagram of an electronic device according to an exemplary embodiment.

[0021] Figure 5 This is an application scenario diagram of an artificial intelligence-based traditional Chinese medicine health status analysis system according to an exemplary embodiment. DETAILED DESCRIPTION

[0022] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0023] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0024] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0025] It should be noted that the concepts such as "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0026] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0027] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0028] TCM's understanding of health has been well documented in the Yellow Emperor's Classic of Internal Medicine, which can be divided into two levels: macro and meso. At the macro level, TCM emphasizes that a healthy person must achieve "harmony between man and nature" and have a good adaptation to nature and society; at the meso level, it emphasizes "moderation and peace" and "form and spirit are in harmony". In other words, health is the result of the harmony between man and nature, society and the harmony of yin and yang. It is a functional state in which man is in harmony with nature, qi is in circulation, form and spirit are in harmony, and yin and yang are in harmony. As the Suwen Shanggu Tianzhen Lun says: "The ancient sages' teachings to their followers are all about avoiding evil spirits and thieves' winds at certain times, being calm and empty, and letting the true qi follow. The spirit is kept inside, and there is no disease. Therefore, the mind is at ease and desires are few, the heart is at peace and not afraid, the body is tired but not exhausted, the qi follows and is smooth, everyone follows their desires, and everyone gets what they want. Therefore, they enjoy their food, wear whatever they want, and enjoy their customs. The high and the low do not envy each other, and their people are therefore called simple. Therefore, desires cannot tire their eyes, evil cannot confuse their hearts, and the foolish, the wise, the virtuous, and the unworthy are not afraid of things, so they are in harmony with the Tao. The reason why they can live to be a hundred years old and still be active is because their virtue is complete and not in danger." Therefore, Chinese medicine believes: Health is a state of the human body with moderation and peace, unity of man and nature as its basic goals, and harmony of man and nature, circulation of qi, coordination of body and spirit, and balance of yin and yang as its basic principles. It is influenced by factors such as innate endowment and acquired care, and its main characteristics include universality and individuality, stability and variability, integrity and locality, self-regulation and external regulation, etc.

[0029] The health status of traditional Chinese medicine covers the physiological and pathological characteristics, constitution, syndrome, disease, etc. at different stages of the individual's life cycle. According to the relationship between the whole and the part, the status can be classified vertically to explain the problem of the current local status being mixed, that is, showing physiological and pathological characteristics, constitution, syndrome, disease and other different states at the same time; the status can also be classified horizontally to explain the health status level of the current overall status, that is, not sick, about to be sick, and already sick.

[0030] The broad definition of health status includes healthy state and disease state. There are different physiological and pathological characteristics and individual differences at different stages of life. Therefore, the status also includes physiological and pathological characteristics, constitution, disease and syndrome.

[0031] Physiological characteristics refer to the laws of normal life activities of the human body, and pathological characteristics refer to the special laws reflected in the disease state. Humans are closely connected with nature, and different groups of people may show different physiological and pathological characteristics due to differences in gender, age, region, occupation, etc. For example, the southeast is more humid and hot, so the skin of people in this area is more loose, and the northwest is more cold and dry, so the skin of people in this area is more dense.

[0032] Disease and syndrome are both pathological states, but they are different in their categories, etiology, pathogenesis, characteristics, and outcomes. Disease and syndrome have different emphases on the laws and mechanisms of disease occurrence and development. In traditional Chinese medicine, "disease" is an abnormal state of imbalance of yin and yang, damage to internal organs and tissues, or physiological dysfunction caused by the action of pathogenic evil qi on the human body and the struggle of the human body's positive qi, reflecting the characteristics and laws of the whole process of the disease. "Syndrome" is a high-level summary of the cause, location, nature, and condition of the disease at a certain stage of development, reflecting the current nature of the disease. "Disease" has a certain continuity in time and space, and has a certain regularity in its occurrence, development, and evolution. Different patients with the same disease show common basic pathological characteristics, reflecting the basic contradictions of the disease. "Syndrome" is a conclusion made on the pathological nature of the current stage of the disease, reflecting the body's reaction state in different patients and at different stages, that is, special contradictions. The health state of traditional Chinese medicine includes physiological characteristics, pathological characteristics, disease, syndrome, etc. Therefore, understanding the status is the key to grasping health and disease. The bias in the status is the internal cause of the disease, and it is also an important factor in determining the development process of the disease and the evolution of symptoms, which determines the outcome of the syndrome and the prognosis of the disease.

[0033] Traditional Chinese medicine obtains disease information through the four examinations to diagnose and differentiate diseases, and a person's physical constitution, physiological characteristics, pathological characteristics and other states can also be reflected through external representation information. Therefore, the identification of physical constitution and the state of illness can refer to the method of differentiation of syndromes. For example, the principle of syndrome differentiation is applied, and the contribution of each symptom to the syndrome is weighted and integrated to establish a mathematical model of syndrome differentiation, so that the state representation and the description of the state elements are more objective, and the judgment of the state is also more objective and accurate.

[0034] The specific implementation of the present application is described in detail below with reference to the accompanying drawings.

[0035] Figure 1 FIG. 1 is a block diagram of a TCM health status analysis system based on artificial intelligence according to an exemplary embodiment. Figure 1As shown, the TCM health status analysis system 100 includes: a tongue image acquisition module 101, which is used to acquire a tongue image uploaded by a user; a tongue region segmentation module 102, which is used to perform tongue region segmentation on the tongue image to obtain a tongue target region image; an LBP encoding module 103, which is used to encode the tongue target region image using an LBP pattern operator to obtain a tongue image LBP encoding feature vector; a visual semantic feature extraction module 104, which is used to input the tongue target region image into a visual encoder to obtain a tongue state semantic encoding feature vector; a dynamic interaction module 105, which is used to input the tongue image LBP encoding feature vector and the tongue state semantic encoding feature vector into a feature vector dynamic interaction synthesis module under gate control to obtain a tongue state depth feature fusion representation vector; a tongue coating shape and quality analysis result generation module 106, which is used to determine a tongue coating shape and quality analysis result based on the tongue state depth feature fusion representation vector, and the tongue coating shape and quality analysis result is used to represent the type label of the tongue coating shape and quality.

[0036] In response to the above technical problems, the technical concept of this application is to combine artificial intelligence technology, perform tongue image region segmentation on the tongue image, extract its multi-scale tongue state features, and dynamically interact the multi-scale tongue state features based on the gating mechanism to form a multi-scale feature representation of the tongue state, and intelligently distinguish the type of tongue coating based on the multi-scale feature representation of the tongue state. In this way, users can realize online automatic tongue image recognition by uploading their own tongue image, saving a long wait.

[0037] Based on this, in the technical solution of the present application, first, a tongue image uploaded by a user is obtained; and the tongue image is input into a tongue target area detection network based on the Yolo model to obtain a tongue target area image. Here, the tongue image reflects the user's current health status and provides an important data basis for remote tongue image recognition. Considering that the tongue image may contain other non-tongue background information, such as teeth, lips, etc., which may interfere with subsequent analysis, in the technical solution of the present application, the tongue target area can be accurately located in the image through the tongue target area detection network based on the Yolo model, so that the model detects and extracts the tongue target area, thereby eliminating these interfering background information and focusing on the analysis of the tongue image.

[0038] In one embodiment of the present application, the tongue region segmentation module is used to: input the tongue image into a tongue target region detection network based on the Yolo model to obtain the tongue target region image.

[0039] Then, the tongue target area image is encoded using the LBP mode operator to obtain the tongue image LBP encoded feature vector. Among them, the local binary pattern (Local Binary Patterns, referred to as LBP) is capable of describing the texture changes in the image, thereby capturing the local texture information in the tongue target area image. In an embodiment of the present application, the LBP mode operator has a 3×3 local neighborhood. In the process of encoding the tongue target area image using the LBP mode operator, in each 3×3 local neighborhood, the grayscale value of the pixel at the center of the local neighborhood is used as a reference, and the other 8 pixels around the center of the local neighborhood are binarized, for example, the pixel value greater than the grayscale value of the pixel at the center of the local neighborhood is set to 1, otherwise it is set to 0. After that, the 8 binary numbers are arranged in order to obtain a binary sequence, and the binary sequence is used as the LBP value of the central pixel of the local neighborhood. In this way, the neighborhood center LBP value of each local neighborhood is obtained, and the tongue image LBP encoded feature vector is obtained by arranging them in order.

[0040] At the same time, the tongue target area image is input into a visual encoder based on the ViT model to obtain a tongue state semantic encoding feature vector. The ViT model is an image processing model based on deep learning, which can extract deep features from the tongue target area image. These features are richer and more abstract than traditional manual feature extraction methods. More specifically, the ViT model can understand the relationship between different areas in the image through the self-attention mechanism, thereby capturing richer semantic information, which is crucial for understanding the complex visual patterns in the tongue image.

[0041] In one embodiment of the present application, the visual semantic feature extraction module is used to: input the tongue target area image into a visual encoder based on the ViT model to obtain the tongue state semantic encoding feature vector.

[0042] It should be understood that the fusion of the tongue image LBP encoding feature vector and the tongue state semantic encoding feature vector can simultaneously utilize texture information and higher-level image semantic information to give full play to the advantages of the LBP pattern operator and the deep learning algorithm model, and realize the communication and complementarity between features. However, traditional fusion methods, such as cascading or adding by position, cannot simulate the complex interaction pattern between the two. Therefore, in the technical solution of the present application, the tongue image LBP encoding feature vector and the tongue state semantic encoding feature vector are input into the feature vector dynamic interaction synthesis module under gate control to obtain the tongue state depth feature fusion representation vector. Among them, the feature vector dynamic interaction synthesis module under gate control can dynamically adjust the contribution of each feature according to the importance of the feature, so that the model can pay more attention to the features that are more meaningful to the analysis of tongue coating shape and quality.

[0043] In one embodiment of the present application, the dynamic interaction module includes: a tongue multi-scale coding information union unit, which is used to input the tongue image LBP coding feature vector and the tongue state semantic coding feature vector into a feature union module to obtain a tongue LBP coding feature-tongue state semantic coding feature joint vector; a gated response unit, which is used to input the tongue LBP coding feature-tongue state semantic coding feature joint vector into a gated response function to obtain a response gate for tongue multi-scale coding information fusion; a tongue multi-scale coding information fusion representation unit, which is used to calculate the position-weighted sum of the tongue image LBP coding feature vector and the tongue state semantic coding feature vector using the response gate of the tongue multi-scale coding information fusion as a weight to obtain the tongue state depth feature fusion representation vector.

[0044] More specifically, the dynamic interaction synthesis module of the feature vector under the gate control effectively integrates the information from the LBP encoding feature vector of the tongue image and the semantic encoding feature vector of the tongue state through a dynamic interaction and weighting mechanism to improve the expression ability of the model. First, the LBP encoding feature vector of the tongue image and the semantic encoding feature vector of the tongue state are preliminarily integrated through the feature union module to obtain the tongue LBP encoding feature-tongue state semantic encoding feature joint vector. This step is the basis of feature fusion, which aims to provide input for subsequent dynamic interaction and weighting, realize the preliminary integration of features, and provide conditions for the interaction between features. Next, the gated response function is used to process the tongue LBP encoding feature-tongue state semantic encoding feature joint vector to adjust its weight according to the dynamic characteristics of the input feature, and obtain the response gate of the multi-scale encoding information fusion of the tongue. Here, the introduction of the gated response function enables the model to dynamically adjust its weight in the fusion process according to the importance of the feature, realize feature selection and weighting, thereby highlighting key features and suppressing noise features, and enhancing the model's ability to capture useful information. Finally, the response gate of the multi-scale coding information fusion of the tongue is used as a weight to perform a weighted sum by position on the LBP coding feature vector of the tongue image and the semantic coding feature vector of the tongue state, and obtain the fusion representation vector of the depth feature of the tongue state. This step reflects the dynamic interaction and fusion of feature vectors. Through the weighted sum operation, the information complementarity and enhancement between feature vectors are achieved, and the expressive ability of features is improved. Among them, the weighted sum by position not only considers the relative importance of feature vectors, but also considers the contribution of each element within the feature vector, so that the fused feature vector can more comprehensively reflect the characteristics of the original data.

[0045] In one embodiment of the present application, the tongue multi-scale coding information combination unit is used to: cascade the tongue image LBP coding feature vector and the tongue state semantic coding feature vector to obtain the tongue LBP coding feature-tongue state semantic coding feature joint vector.

[0046] Furthermore, in one embodiment of the present application, the gated response unit is used to: calculate the matrix product of the weight parameter vector and the tongue LBP coding feature-tongue state semantic coding feature joint vector, add the value obtained by the matrix product to the bias term, and process the added value through nonlinearization to obtain the response gate of the tongue multi-scale coding information fusion.

[0047] Furthermore, in one embodiment of the present application, the nonlinear processing includes using a sigmoid function to process the value obtained by the addition to obtain a response gate for the fusion of the tongue multi-scale encoding information.

[0048] Furthermore, in one embodiment of the present application, the tongue multi-scale coding information fusion representation unit is used to: construct the tongue image LBP coding fusion weight and the tongue state semantic coding fusion weight based on the response gate of the tongue multi-scale coding information fusion; use the tongue image LBP coding fusion weight as the weight of the tongue image LBP coding feature vector, use the tongue state semantic coding fusion weight as the weight of the tongue state semantic coding feature vector, calculate the position-weighted sum of the tongue image LBP coding feature vector and the tongue state semantic coding feature vector to obtain the tongue state depth feature fusion representation vector.

[0049] Specifically, the tongue image LBP encoding feature vector and the tongue state semantic encoding feature vector are input into the feature vector dynamic interactive synthesis module under gate control according to the following interactive synthesis formula to obtain the tongue state depth feature fusion representation vector; wherein the interactive synthesis formula is: ;in, and are respectively the tongue image LBP encoding feature vector and the tongue state semantic encoding feature vector, and are the weight parameter vector and the bias term, respectively. is the sigmoid function, It is the response gate for the fusion of multi-scale coding information of the tongue. represents dot product, It means adding by position. It is the tongue state depth feature fusion representation vector.

[0050] Furthermore, the tongue state depth feature fusion representation vector is input into a tongue coating shape and quality analyzer based on a classifier to obtain a tongue coating shape and quality analysis result, and the tongue coating shape and quality analysis result is used to represent the type label of the tongue coating shape and quality. Among them, the type label of the tongue coating shape and quality may include but is not limited to the following: thin coating, thick coating, dry coating, wet coating, greasy coating, peeling coating, cracked coating, yellow coating or white coating.

[0051] In one embodiment of the present application, the tongue coating shape and quality analysis result generation module is used to: input the tongue state depth feature fusion representation vector into a classifier-based tongue coating shape and quality analyzer to obtain the tongue coating shape and quality analysis result.

[0052] In a preferred embodiment, the tongue state depth feature fusion representation vector is input into a tongue coating shape analyzer based on a classifier to obtain a tongue coating shape analysis result, which includes: probabilizing each feature value of the tongue state depth feature fusion representation vector based on a probability activation function, such as a sigmoid function, to obtain a probabilistic tongue state depth feature fusion representation vector; obtaining the tongue state depth feature fusion representation vector and inputting it into a tongue coating shape analyzer based on a classifier to obtain a tongue coating shape analysis result. The type labels of tongue coating shape and quality correspond to label probability values; The label probability values ​​are subtracted from After dividing by one, calculate the square root of the sum of the squares and add Add the two fractions together to get the type probability value.

[0053] The tongue state depth and shallow feature fusion type recognition symbol value is determined based on the comparison between each eigenvalue of the probabilistic tongue state depth and shallow feature fusion representation vector and the type probability value, wherein the tongue state depth and shallow feature fusion type recognition symbol value is equal to one, zero and negative one respectively in response to the eigenvalue of the probabilistic tongue state depth and shallow feature fusion representation vector being greater than, equal to and less than the type probability value.

[0054] The mean of all eigenvalues ​​of the probabilistic tongue state depth and shallow feature fusion representation vector is calculated to obtain the overall phase shift value of the tongue state depth and shallow feature fusion type.

[0055] After multiplying each eigenvalue of the probabilistic tongue state depth and shallow feature fusion representation vector by the tongue state depth and shallow feature fusion type recognition symbol value and the tongue state depth and shallow feature fusion type overall phase shift value, a weighted difference calculation is performed and the absolute value is taken to obtain the optimized eigenvalue of the probabilistic tongue state depth and shallow feature fusion representation vector.

[0056] The optimized tongue state depth feature fusion representation vector composed of the optimized feature values ​​is input into a tongue coating shape and quality analyzer based on a classifier to obtain a tongue coating shape and quality analysis result.

[0057] Specifically, the optimization steps are expressed as: ;in, and are respectively the i-th eigenvalue of the probabilistic tongue state depth feature fusion representation vector and the optimized tongue state depth feature fusion representation vector, is the type probability value, is the j-th label probability value, is the tongue state depth feature fusion type cognitive symbol value, is the overall phase shift value of the tongue state depth feature fusion type, and are the first weight and the second weight respectively, Indicates taking the absolute value.

[0058] Here, the tongue image LBP encoding feature vector and the tongue state semantic encoding feature vector respectively express the local source image semantic space domain LBP pattern features of the tongue target area image and the image semantic features based on the global association of the local image semantic space source image semantic context. Therefore, after the tongue image LBP encoding feature vector and the tongue state semantic encoding feature vector are input into the feature vector dynamic interaction synthesis module under gated control, the obtained tongue state depth feature fusion representation vector will have a priori-posteriori class probability causal association missing relative to the source image semantic feature representation due to the dynamic interaction synthesis difference caused by the differentiated distribution of semantic modal features of different images, thereby affecting the accuracy of the classification result.

[0059] Therefore, in the above-mentioned preferred embodiment, the applicant of the present application obtains the type cognitive phase transformation response of the tongue state depth and shallow feature fusion representation vector by comparing the probabilistic amplitude of the eigenvalue of the tongue state depth and shallow feature fusion representation vector with the type probability representation, and performs a feature distribution sequence invariance transformation based on the differential distribution expansion of the type label probability on the phase shift response of the eigenvalue of the tongue state depth and shallow feature fusion representation vector relative to the type probability representation of the feature graph as a whole, so as to realize the causal constraint of the posterior class probability of the tongue state depth and shallow feature fusion representation vector on its prior feature distribution representation, so as to improve the accuracy of the tongue coating shape and quality analysis result obtained by inputting the tongue state depth and shallow feature fusion representation vector into the tongue coating shape and quality analyzer based on the classifier.

[0060] In summary, the above scheme is adopted, and after the tongue image is segmented into tongue regions by combining artificial intelligence technology, its multi-scale tongue state features are extracted, and the multi-scale tongue state features are dynamically interacted based on the gating mechanism to form a multi-scale feature representation of the tongue state, and the type of tongue coating shape and quality is intelligently distinguished based on the multi-scale feature representation of the tongue state. In this way, users can realize online automatic tongue image recognition by uploading their own tongue image, without waiting for a long time.

[0061] Figure 2 is a flowchart of a method for analyzing health status of traditional Chinese medicine based on artificial intelligence according to an exemplary embodiment, such as Figure 2 As shown, the method includes: step S201, obtaining a tongue image uploaded by a user.

[0062] Step S202: performing tongue region segmentation on the tongue image to obtain a tongue target region image.

[0063] Step S203: Encode the tongue target area image using the LBP mode operator to obtain a tongue image LBP encoded feature vector.

[0064] Step S204: input the tongue target area image into a visual encoder to obtain a tongue state semantic coding feature vector.

[0065] Step S205: input the tongue image LBP encoding feature vector and the tongue state semantic encoding feature vector into a feature vector dynamic interactive synthesis module under gate control to obtain a tongue state depth feature fusion representation vector.

[0066] Step S206: Determine a tongue coating shape and quality analysis result based on the tongue state depth feature fusion representation vector, wherein the tongue coating shape and quality analysis result is used to represent a type label of the tongue coating shape and quality.

[0067] In one embodiment of the present application, TCM health information includes basic information, basic indicators, physical and chemical indicators, TCM four diagnostic information, and tongue image. Collection methods: self-filling by users, taking tongue image photos with mobile phone cameras, photos of inspection reports, portable wearable devices, filling out symptom questionnaires, AI voice dialogue, TCM four diagnostic equipment, meridian detector equipment, health all-in-one equipment, and interface connection with existing medical information systems (HIS, LIS, PACS, RIS), etc.

[0068] The thinking principle of TCM health status identification is as follows Figure 3As shown in the figure, TCM health status identification is the doctor's judgment on the health status of the life process. Under the guidance of TCM theory, the external representation information of the individual is comprehensively analyzed, and the state elements of the individual's overall reaction state (including degree, location, and nature) are judged to identify the state of life. Whether it is a healthy or diseased state, it can be reflected through external representations (phenomena). Therefore, as long as the characteristics and identification methods of the state elements are mastered, the various states of the life process can be grasped by analyzing and summarizing the representation parameters.

[0069] The thinking process of TCM health status identification can be summarized as "identifying status elements and forming status names based on representation parameters". Judging status elements from representation parameters and finally forming status names are not only the principles and laws of status identification, but also the three levels, three steps and three steps in the thinking process of status identification. All three are "identification". Identifying representation parameters is the basis, judging status elements is the key and establishing status names is the purpose.

[0070] Among them, multi-dimensional analysis is conducted based on data collected from traditional Chinese medicine health information.

[0071] The BMI algorithm analysis is: BMI = weight (kg) ÷ height^ 2 (m).

[0072] The TCM constitution algorithm analysis is as follows: based on the degree of symptoms answered in the four TCM diagnostic information questions, 1 point is for none, 2 points is for mild, 3 points is for moderate (present), 4 points is for severe, and 5 points is for very severe.

[0073] Total score = sum of all answer scores.

[0074] Conversion factor = [(total score - number of questions) / (number of questions * 4)] * 100.

[0075] Other methods for judging constitution are: conversion coefficient >= 40 indicates that you have this constitution; 30<= conversion coefficient<= 40 indicates that you tend to have this constitution; conversion coefficient <30 indicates that you do not have this constitution.

[0076] The method for judging a peaceful constitution is: if the conversion coefficient is >= 60 and the conversion coefficients of other constitutions are all <30, then it is this constitution; if the conversion coefficient is >= 60 and the conversion coefficients of other constitutions are all <40, then it is basically this constitution; otherwise, it is not this constitution.

[0077] Furthermore, the TCM syndrome factor algorithm is analyzed as follows: a TCM syndrome factor intelligent differentiation method based on syndrome factor theory is used to calculate TCM syndrome factors.

[0078] Furthermore, the tongue image model analysis is: using a model of a traditional Chinese medicine tongue image recognition method based on image recognition technology to analyze the tongue image.

[0079] Based on the results of TCM health status analysis and assessment, personalized guidance intervention plans and early warning reminders are automatically recommended through the large model base.

[0080] Among them, the intervention principles include: actively carry out health education to improve the health awareness of the whole people; change unhealthy lifestyles and build five health cornerstones. Master health skills, strive to achieve a reasonable diet, moderate exercise, psychological balance, adequate sleep, quit smoking and limit alcohol; relieve tension and stress in a timely manner, effectively eliminate mental and physical fatigue; and carry out syndrome differentiation and adjustment guided by the theory of traditional Chinese medicine. Under the guidance of the theory of traditional Chinese medicine, corresponding intervention measures are given according to the physical condition of those in sub-health status and the specific characteristics and severity of discomfort. Such as Chinese medicine, acupuncture, massage, nutrient supplements, health foods, medicinal diet and traditional fitness; psychological counseling and behavioral guidance are carried out according to individual situations. For sub-healthy people with mental and psychological discomfort or difficulty in social interaction, psychological counseling or cognitive behavioral guidance can be given according to the specific situation.

[0081] Intervention methods include: (1) Emotional intervention: appropriately adjust your personality, cultivate a strong will, rational and stable personality, and enhance independence. Maintain a happy mood, cultivate an optimistic attitude towards life, and be good at regulating your emotions. Maintain emotional stability, adjust negative emotions in a timely manner, calm the mind, soothe emotions, and maintain psychological balance. According to the specific situation, you can choose the method of moderation, catharsis, transfer, and emotional control. (2) Daily life intervention: follow the body's biological clock to adjust your daily life, maintain a regular work and rest schedule, work and study during the day, and rest and sleep at night. Ensure adequate sleep time, develop good sleeping habits, and avoid staying up late. Adjust your work and rest schedule according to different seasons, and work and rest in moderation. (3) Diet intervention: Eat in moderation, at regular times and in fixed quantities, eat until you are 80% full, and avoid overeating. Reasonable allocation, balanced nutrition, and rich variety. Harmonize the five flavors without partiality. Pay attention to choosing food according to your physical characteristics. Eat slowly, be focused, and live in a clean and relaxing environment.

[0082] Reference below Figure 4 , which shows a schematic diagram of the structure of an electronic device 600 suitable for implementing the embodiment of the present application. The terminal device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0083] like Figure 4 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0084] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0085] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present application are executed.

[0086] It should be noted that the computer-readable medium mentioned above in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0087] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0088] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0089] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0090] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0091] The modules involved in the embodiments described in this application can be implemented by software or hardware. The name of the module does not limit the module itself in some cases. For example, the test parameter acquisition module can also be described as a "module for acquiring device test parameters corresponding to the target device".

[0092] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0093] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0094] Figure 5 FIG. 1 is an application scenario diagram of a TCM health status analysis system based on artificial intelligence according to an exemplary embodiment. Figure 5 As shown, in this application scenario, first, a tongue image uploaded by a user is obtained (for example, Figure 5 Then, the acquired tongue image is input to a server (for example, Figure 5 In S) shown in , the server is capable of processing the tongue image based on an artificial intelligence-based TCM health status analysis algorithm to determine a tongue coating morphology analysis result, and the tongue coating morphology analysis result is used to represent a type label of the tongue coating morphology.

[0095] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

[0096] In addition, although each operation is described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or to be performed in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the application. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0097] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely exemplary forms of implementation. With respect to the apparatus in the above-mentioned embodiments, the specific manner in which each module performs the operation has been described in detail in the embodiments related to the method, and will not be described in detail here.

Claims

1. A TCM health status analysis system based on artificial intelligence, characterized in that: include: A tongue image acquisition module is used to acquire the tongue image uploaded by the user; A tongue region segmentation module, used for performing tongue region segmentation on the tongue image to obtain a tongue target region image; LBP encoding module, used for encoding the tongue target area image using LBP mode operator to obtain the tongue image LBP encoding feature vector; visual A semantic feature extraction module, used for inputting the tongue target area image into a visual encoder to obtain a tongue state semantic coding feature vector; a dynamic interaction module, used for inputting the tongue image LBP coding feature vector and the tongue state semantic coding feature vector into a feature vector dynamic interaction synthesis module under gate control to obtain a tongue state depth feature fusion representation vector; a tongue coating shape and quality analysis result generation module, used for determining a tongue coating shape and quality analysis result based on the tongue state depth feature fusion representation vector, wherein the tongue coating shape and quality analysis result is used to represent a type label of the tongue coating shape and quality; The tongue coating shape and quality analysis result generation module is used to: input the tongue state depth feature fusion representation vector into the tongue coating shape and quality analyzer based on the classifier to obtain the tongue coating shape and quality analysis result; The method comprises: inputting the tongue state depth feature fusion representation vector into a tongue coating shape and quality analyzer based on a classifier to obtain a tongue coating shape and quality analysis result, comprising: probabilizing each feature value of the tongue state depth feature fusion representation vector based on a probability activation function to obtain a probabilistic tongue state depth feature fusion representation vector; obtaining the tongue state depth feature fusion representation vector and inputting it into a tongue coating shape and quality analyzer based on a classifier to obtain a tongue coating shape and quality analysis result. The type labels of tongue coating shape and quality correspond to label probability values; The label probability values ​​are subtracted After dividing by one, calculate the square root of the sum of the squares and add Add one half and get the type probability value; Determine the tongue state depth feature fusion type recognition symbol value based on the comparison between each feature value of the probabilistic tongue state depth feature fusion representation vector and the type probability value, wherein the tongue state depth feature fusion type recognition symbol value is equal to one, zero and negative one in response to the feature value of the probabilistic tongue state depth feature fusion representation vector being greater than, equal to and less than the type probability value, respectively; Calculate the mean of all eigenvalues ​​of the probabilistic tongue state depth feature fusion representation vector to obtain the overall phase shift value of the tongue state depth feature fusion type; After multiplying each eigenvalue of the probabilistic tongue state depth feature fusion representation vector by the tongue state depth feature fusion type recognition symbol value and the tongue state depth feature fusion type overall phase shift value, weighted difference calculation is performed, and the absolute value is taken to obtain the optimized eigenvalue of the probabilistic tongue state depth feature fusion representation vector; Inputting the optimized tongue state depth feature fusion representation vector composed of the optimized feature values ​​into a tongue coating shape and quality analyzer based on a classifier to obtain a tongue coating shape and quality analysis result; The optimization steps are expressed as: ;in, and are respectively the i-th eigenvalue of the probabilistic tongue state depth feature fusion representation vector and the optimized tongue state depth feature fusion representation vector, is the type probability value, is the j-th label probability value, is the tongue state depth feature fusion type cognitive symbol value, is the overall phase shift value of the tongue state depth feature fusion type, and are the first weight and the second weight respectively, Indicates taking the absolute value.

2. The artificial intelligence-based TCM health status analysis system according to claim 1 is characterized in that: The tongue area segmentation module is used to: input the tongue image into a tongue target area detection network based on the Yolo model to obtain the tongue target area image.

3. The artificial intelligence-based TCM health status analysis system according to claim 2 is characterized in that: The visual semantic feature extraction module is used to: input the tongue target area image into a visual encoder based on the ViT model to obtain the tongue state semantic coding feature vector.

4. The artificial intelligence-based TCM health status analysis system according to claim 3 is characterized in that: The dynamic interaction module includes: a tongue multi-scale coding information union unit, which is used to input the tongue image LBP coding feature vector and the tongue state semantic coding feature vector into a feature union module to obtain a tongue LBP coding feature-tongue state semantic coding feature joint vector; a gated response unit, which is used to input the tongue LBP coding feature-tongue state semantic coding feature joint vector into a gated response function to obtain a response gate for tongue multi-scale coding information fusion; and a tongue multi-scale coding information fusion representation unit, which is used to calculate the position-weighted sum of the tongue image LBP coding feature vector and the tongue state semantic coding feature vector using the response gate for the tongue multi-scale coding information fusion as a weight to obtain the tongue state depth feature fusion representation vector.

5. The artificial intelligence-based TCM health status analysis system according to claim 4 is characterized in that: The tongue multi-scale coding information combining unit is used to: cascade the tongue image LBP coding feature vector and the tongue state semantic coding feature vector to obtain the tongue LBP coding feature-tongue state semantic coding feature combined vector.

6. The artificial intelligence-based TCM health status analysis system according to claim 5 is characterized in that: The gated response unit is used to: calculate the matrix product of the weight parameter vector and the tongue LBP coding feature-tongue state semantic coding feature joint vector, add the value obtained by the matrix product to the bias term, and perform nonlinear processing on the value obtained by the addition to obtain the response gate of the tongue multi-scale coding information fusion.

7. The artificial intelligence-based TCM health status analysis system according to claim 6 is characterized in that: The nonlinear processing includes using a sigmoid function to process the numerical value obtained by the addition to obtain a response gate for the fusion of the tongue multi-scale coding information.

8. The artificial intelligence-based TCM health status analysis system according to claim 7 is characterized in that: The tongue multi-scale coding information fusion representation unit is used to: construct a tongue image LBP coding fusion weight and a tongue state semantic coding fusion weight based on a response gate of the tongue multi-scale coding information fusion; The tongue image LBP encoding fusion weight is used as the weight of the tongue image LBP encoding feature vector, and the tongue state semantic encoding fusion weight is used as the weight of the tongue state semantic encoding feature vector. The position-weighted sum of the tongue image LBP encoding feature vector and the tongue state semantic encoding feature vector is calculated to obtain the tongue state depth feature fusion representation vector.