Household traditional Chinese medicine robot

Abstract

The invention discloses a household traditional Chinese medicine robot which comprises a robot body, a face diagnosis acquisition module, a tongue diagnosis analysis module, an intelligent inquiry module, a pulse diagnosis detection module and a multi-modal data fusion center module. The system has the advantages that firstly, multi-mode traditional Chinese medicine diagnosis integration is achieved, intelligent fusion of four diagnosis modes of traditional Chinese medicine is achieved through collaborative analysis of face diagnosis, tongue diagnosis, pulse diagnosis and intelligent inquiry diagnosis, and the limitation of a traditional single diagnosis mode is broken through; 2, an accurate data acquisition technology: adopting advanced technologies such as five-color spectral imaging, tongue body dynamic tracking and a three-dimensional pulse pressure sensor to improve objectivity and accuracy of traditional Chinese medicine sign detection; and 3, dynamic personalized inquiry: based on the Bayesian network and the symptom knowledge graph, realizing real-time optimization of an inquiry path, and improving the dialectical efficiency. The method is suitable for household scenes, and the threshold of professional traditional Chinese medicine diagnosis is lowered.

Description

Technical Field

[0001] The present invention relates to the technical field of medical robots, and in particular to a household traditional Chinese medicine robot. Background Art

[0002] Traditional Chinese Medicine (TCM) diagnosis relies on a comprehensive approach based on observation, auscultation, inquiry, and palpation, analyzing the underlying condition of the disease based on the patient's constitution, symptoms, and signs. Its core principle is syndrome differentiation and treatment, developing a targeted treatment plan tailored to individual differences and evolving conditions, rather than simply prescribing the right medicine for the right symptoms. The four diagnostic methods include: 1. Observation, which examines the patient's overall condition, including complexion, tongue, posture, and skin. A red tongue is often associated with heat syndrome, while a thick, white tongue coating indicates dampness and turbidity. A pale complexion often indicates qi and blood deficiency, while a flushed face may indicate excess heat or deficiency heat. 2. Auscultation, which assesses the condition by listening to sounds such as coughs and voices and smelling breath and body odor. A low voice often indicates qi deficiency, while a foul odor in the mouth may indicate stomach heat or food accumulation. 3. Interview, which systematically inquires about the patient's symptoms, medical history, and lifestyle, encompassing the "Ten Questions" covering topics such as chills and fever, sweating, diet, and bowel movements. For example, a fever accompanied by chills is an external symptom; night sweats are often a sign of yin deficiency. Fourth, palpation, including pulse diagnosis and palpation. Pulse diagnosis involves palpating the radial artery to determine the pulse condition. Common pulse types include: a floating pulse, felt with light pressure, indicating superficial symptoms; a sinking pulse, felt with heavy pressure, indicating internal symptoms; and a stringy pulse, like pressing a guitar string, often associated with liver depression. Palpation involves pressing the abdomen and skin to detect abnormalities. For example, abdominal pain that resists palpation indicates a positive symptom, while pain that responds to palpation indicates a negative symptom. These TCM diagnoses typically require visits to TCM clinics, which are not only expensive but also extremely difficult to obtain. With the development of society, there is a clear lack of TCM diagnostic and treatment robots.

[0003] Existing TCM diagnosis and treatment equipment has the following defects: there is no automated, mechanized, and digital robot that can integrate the four diagnostic methods of looking, smelling, asking, and palpating. Summary of the Invention

[0004] The main technical problem solved by the present invention is to provide a household TCM robot, which realizes the intelligent integration of the four TCM examinations, breaks through the limitations of the traditional single diagnostic method, lowers the threshold of professional TCM diagnosis, and is suitable for home use.

[0005] In order to solve the above problems, the present invention provides a household traditional Chinese medicine robot, comprising: a robot body, a facial diagnosis collection module, a tongue diagnosis analysis module, an intelligent diagnosis inquiry module, a pulse diagnosis detection module and a multimodal data fusion center module; the facial diagnosis collection module, tongue diagnosis analysis module, intelligent diagnosis inquiry module, pulse diagnosis detection module and multimodal data fusion center module are assembled on the robot body, and each module is connected to the multimodal data fusion center module via an industrial bus.

[0006] Furthermore, the facial diagnosis acquisition module includes a five-color spectral imager, a micro-expression recognizer, and a facial organ mapping database; the five-color spectral imager includes a polarization interference filter group.

[0007] Furthermore, the five-color spectral imager is used to collect the facial reflection spectrum in the 400-1000nm band; the polarization interference filter group is used to eliminate the reflection interference of the skin stratum corneum.

[0008] Furthermore, the tongue diagnosis and analysis module includes a multi-angle 3D tongue image acquisition component, a tongue image feature extraction engine, and a tongue and vein ecchymosis detection algorithm; the multi-angle 3D tongue image acquisition component includes a ring LED fill light and binocular stereo vision.

[0009] Furthermore, the tongue diagnosis and analysis module also includes a tongue dynamic tracker and a tongue coating moistness analyzer.

[0010] Furthermore, the intelligent consultation module includes a dialect-adaptive speech recognizer, a dynamic consultation path generator, and a symptom knowledge graph.

[0011] Furthermore, the symptom knowledge graph contains dialectical rules for 30,000 clinical cases.

[0012] Furthermore, the pulse diagnosis detection module includes a three-dimensional pulse pressure sensor array, a pulse wave feature extractor, and a pulse simulation trainer; the three-dimensional pulse pressure sensor array is used to synchronously detect the pulse conditions of the inch, guan, and chi parts; the pulse wave feature extractor is used to calculate the 28-dimensional features of pulse position, pulse rate, and pulse shape; the pulse simulation trainer contains mechanical feedback models of 200 typical pulse conditions.

[0013] Furthermore, the pulse diagnosis detection module also includes a blood vessel stiffness compensation algorithm and a pulse diagnosis guiding robotic arm; the blood vessel stiffness compensation algorithm is used to correct the pulse assessment threshold of patients of different ages.

[0014] Furthermore, the multimodal data fusion center module includes a four-diagnosis weight allocator, a syndrome differentiation decision engine, and a contraindication verifier.

[0015] The beneficial effects of the present invention are: 1. Multimodal TCM diagnosis integration: through the collaborative analysis of facial diagnosis, tongue diagnosis, pulse diagnosis and intelligent questioning, the intelligent integration of the four diagnoses of TCM is realized, breaking through the limitations of the traditional single diagnosis method; 2. Accurate data acquisition technology: using advanced technologies such as five-color spectral imaging, tongue dynamic tracking, and three-dimensional pulse pressure sensor to improve the objectivity and accuracy of TCM physical sign detection; 3. Dynamic personalized questioning: based on Bayesian network and symptom knowledge graph, the real-time optimization of the questioning path is realized to improve the efficiency of syndrome differentiation. 4. Suitable for home scenarios, through miniaturized design and automated operation such as pulse diagnosis to guide the robotic arm, the threshold for professional TCM diagnosis is lowered; 5. Improve diagnostic reliability: multimodal data fusion and deep learning algorithms, namely U-Net and deep forest, reduce subjective errors and enhance the scientific nature of syndrome type judgment; 6. Reduce medical costs: Home application reduces the frequency of patients going to and from the hospital and alleviates the pressure on medical resources; 7. Promote the standardization of TCM: quantify indicators such as tongue and pulse to promote the modernization and internationalization of TCM diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the description are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 This is a product appearance diagram of embodiment 1 of the present invention.

[0018] Figure 2 It is a schematic diagram of the connection of various components of the present invention.

[0019] Figure 3 This is a product appearance diagram of Example 2 of the present invention.

[0020] The accompanying drawings are marked as follows: 10. Robot body, 20. Facial diagnosis acquisition module, 21. Five-color spectrum imager, 22. Micro-expression recognizer, 23. Facial viscera mapping database, 30. Tongue diagnosis analysis module, 31. Multi-angle 3D tongue image acquisition component, 32. Tongue image feature extraction engine, 33. Tongue pulse ecchymosis detection algorithm, 34. Tongue body dynamic tracker, 35. Tongue coating moistness analyzer, 40. Intelligent consultation module, 41. Dialect adaptive speech recognizer, 42. Dynamic consultation path generator, 43. Symptom knowledge graph, 50. Pulse diagnosis detection module, 51. Three-dimensional pulse pressure sensor array, 52. Pulse wave feature extractor, 53. Pulse simulation trainer, 54. Vascular stiffness compensation algorithm, 55. Pulse diagnosis guidance robot arm, 60. Multimodal data fusion center module, 61. Four-diagnosis weight distributor, 62. Syndrome differentiation decision engine, 63. Contraindication verifier.

[0021] The following is a further description of the purpose, functional features and advantages of the present invention with reference to the embodiments and accompanying drawings. DETAILED DESCRIPTION

[0022] To make the purpose, technical solutions, and advantages of the invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0023] Example 1:

[0024] like Figure 1 、 Figure 2 As shown, a household traditional Chinese medicine robot, the movement of the robot body 10 of this embodiment is in the form of wheel movement, which includes: a robot body 10, a facial diagnosis acquisition module 20, a tongue diagnosis analysis module 30, an intelligent diagnosis module 40, a pulse diagnosis detection module 50 and a multimodal data fusion center module 60; the facial diagnosis acquisition module 20, the tongue diagnosis analysis module 30, the intelligent diagnosis module 40, the pulse diagnosis detection module 50, and the multimodal data fusion center module 60 are installed on the robot body 10, and each module is connected to the multimodal data fusion center module 60 through an industrial bus.

[0025] like Figure 1 、 Figure 2 As shown, the facial diagnosis acquisition module 20 includes a five-color spectrum imager 21, a micro-expression recognizer 22, and a facial viscera mapping database 23; the five-color spectrum imager 21 includes a polarization interference filter group; the core component of the five-color spectrum imager 21 adopts a multi-spectral camera, which is specifically a high-sensitivity CMOS sensor in the 400-1000nm band; the micro-expression recognizer 22 adopts a high-speed camera, which is specifically an industrial-grade 60fps camera, and is a FLIR Blackfly S or Basler ace series; the facial viscera mapping database 23 is used to store the coordinates of the projection areas of the five internal organs and six bowels defined in the "Inner Canon of the Yellow Emperor".

[0026] like Figure 1 、 Figure 2 As shown, the five-color spectral imager 21 is used to collect the facial reflection spectrum in the 400-1000nm band; the polarization interference filter group is used to eliminate the reflection interference of the skin stratum corneum; the polarization interference filter group specifically adopts a liquid crystal tunable filter LCTF or an acousto-optic tunable filter AOTF.

[0027] like Figure 1 、 Figure 2 As shown, the tongue diagnosis and analysis module 30 includes a multi-angle 3D tongue image acquisition component 31, a tongue image feature extraction engine 32, and a tongue and vein ecchymosis detection algorithm 33; the multi-angle 3D tongue image acquisition component 31 includes a ring LED fill light and binocular stereo vision; the binocular stereo vision adopts the Intel RealSense D415 depth camera; the ring LED fill light adopts a Phlox medical-grade light source for adjusting the color temperature of the LED ring; the tongue image feature extraction engine 32 is used to quantify the tongue quality color, tongue coating thickness and crack index; the tongue image feature extraction engine adopts HSV / CIELab color space conversion for colorimetry analysis, and crack detection adopts the Canny operator, which is based on the edge detection algorithm; the tongue and vein ecchymosis detection algorithm 33 is used for the U-Net network to identify the degree of sublingual varicose veins.

[0028] like Figure 1 、 Figure 2 As shown, the tongue diagnosis and analysis module 30 also includes a tongue dynamic tracker 34 and a tongue coating moisture analyzer 35. The tongue dynamic tracker 34 is used to calculate the tongue vibration frequency by matching feature points. The tongue coating moisture analyzer 35 uses a capacitive humidity sensor to assess the distribution of body fluids. The capacitive humidity sensor specifically uses a Sensirion SHT40.

[0029] like Figure 1 、 Figure 2 As shown, the intelligent consultation module 40 includes a dialect-adaptive speech recognizer 41, a dynamic consultation path generator 42, and a symptom knowledge graph 43; the dialect-adaptive speech recognizer 41 covers seven major Chinese dialect variants, and the dialect-adaptive speech recognizer 41 adopts iFlytek or Alibaba Cloud speech recognition API, and its dialect model adopts Conformer, which is an end-to-end model based on deep learning; the dynamic consultation path generator 42 is based on the Bayesian network to adjust the inquiry strategy in real time, that is, when the tongue diagnosis shows damp-heat syndrome, it gives priority to asking about related symptoms such as yellow and dark urine and sticky stool; the Bayesian network tool library adopts pgmpy.

[0030] like Figure 1 、 Figure 2 As shown, the symptom knowledge graph 43 contains the syndrome differentiation rules of 30,000 clinical cases; the symptom knowledge graph 43 specifically adopts the graph database Neo4j, which stores the syndrome differentiation rules of 30,000 clinical cases.

[0031] like Figure 1 、 Figure 2As shown, the pulse diagnosis detection module 50 includes a three-dimensional pulse pressure sensor array 51, a pulse wave feature extractor 52, and a pulse simulation trainer 53. The three-dimensional pulse pressure sensor array 51 is used to synchronously detect the three pulses of Cun, Guan, and Chi. The pulse wave feature extractor 52 is used to calculate 28-dimensional features such as pulse position, pulse rate, and pulse shape. The pulse simulation trainer 53 includes mechanical feedback models for 200 typical pulse conditions. The three-dimensional pulse pressure sensor array 51 uses a MEMS pressure sensor, namely the TE Connectivity MS5837, and is also equipped with a multi-channel data acquisition card, which uses a National Instruments USB-6009.

[0032] like Figure 1 、 Figure 2 As shown, the pulse diagnosis detection module 50 also includes a vascular stiffness compensation algorithm 54 and a pulse diagnosis guiding robotic arm 55; the vascular stiffness compensation algorithm 54 is used to correct the pulse assessment threshold of patients of different ages; the pulse diagnosis guiding robotic arm 55 has six-degree-of-freedom adjustment and tactile feedback functions; the pulse diagnosis guiding robotic arm 55 adopts a UR3 collaborative robot, and its tactile feedback device adopts a Force-Torque sensor.

[0033] like Figure 1 、 Figure 2 As shown, the multimodal data fusion center module 60 includes a four-diagnosis weight allocator 61, a syndrome differentiation decision engine 62, and a contraindication checker 63; the four-diagnosis weight allocator 61 dynamically adjusts the diagnosis weight according to the "solitude hides treachery" principle of "Nanjing"; the syndrome differentiation decision engine 62 adopts Deep Forest algorithm to output syndrome type probability distribution; the contraindication checker 63 adopts a database based on the "Compendium of Materia Medica" to review the safety of prescriptions and a structured Chinese medicine contraindication table.

[0034] The working process of the household TCM robot of this embodiment:

[0035] 1. Simultaneous collection of data for four diagnoses: 1. Facial diagnosis: a five-color spectral imager collects facial reflection spectra, a polarization interference filter eliminates reflective interference, and a micro-expression recognizer records muscle movements; 2. Tongue diagnosis: under annular LED fill light, binocular stereo vision constructs a 3D tongue image model, a capacitive humidity sensor quantifies moistness, and a U-Net network analyzes sublingual varicose veins; 3. Interview: a Conformer dialect recognition engine parses voice input, and a Bayesian network dynamically generates query paths; 4. Pulse diagnosis: a MEMS pressure sensor array synchronously detects pulse pressure in the three parts of the body, namely, Cun, Guan, and Chi, and a six-degree-of-freedom robotic arm provides tactile feedback.

[0036] 2. Multimodal fusion analysis: The four-diagnosis weight allocator dynamically adjusts the contribution of each module according to the principles of "Nanjing". The DeepForest algorithm integrates tongue HSV / CIELab colorimetry, pulse 28-dimensional features, and medical consultation knowledge graph data. The contraindication verifier compares the "Compendium of Materia Medica" database to verify the safety of the prescription.

[0037] 3. During the interactive output stage, the touch screen displays a constitution identification report, including tongue and pulse images, and syndrome probability distribution; through HarmonyOS linkage with smart home devices, data is encrypted and uploaded to the cloud to establish a health record, supporting comparison of previous diagnoses. The entire process takes about 8-12 minutes, and the error in the core diagnostic link is controlled within ±1.5%.

[0038] Functions of the household TCM robot of this embodiment:

[0039] 1. Facial diagnosis function: 1. Five-color spectral analysis: using a multispectral camera to collect facial reflection spectra, combined with a liquid crystal tunable filter to eliminate skin reflection interference, to achieve quantitative analysis of the five-color pathological characteristics of Traditional Chinese Medicine; 2. Organ positioning diagnosis: Based on the facial organ mapping database of the "Inner Canon of Medicine", a 60fps industrial camera is used to capture micro-expression changes and accurately locate the diseased areas of the five internal organs.

[0040] 2. Tongue diagnosis functions: 1. 3D tongue image modeling, using the Intel RealSense D415 depth camera and medical-grade ring LED light source to construct a three-dimensional model of the tongue and quantify the thickness of the tongue coating and the crack index; 2. Sublingual vein analysis, using the U-Net network to identify the degree of sublingual varicose veins, combined with a capacitive humidity sensor to assess the degree of body fluid moistness; 3. Dynamic tracking, using a feature point matching algorithm to monitor the frequency of tongue vibration in real time, such as symptoms of liver wind internal movement.

[0041] 3. Medical consultation function: 1. Dialect interaction, using the Conformer end-to-end model to recognize seven major Chinese dialects to achieve barrier-free voice medical consultation; 2. Intelligent inquiry strategy, dynamically generating medical consultation paths based on the Bayesian network, such as asking about defecation and urination symptoms first for damp-heat syndrome; 3. Symptom matching, calling 30,000 clinical syndrome differentiation rules through the Neo4j graph database to generate a symptom-syndrome association map.

[0042] IV. Pulse diagnosis function: 1. Three-part and nine-sign detection, using a MEMS pressure sensor array to synchronously collect pulse pressure waveforms of the three parts of the body; 2. Pulse feature extraction, calculating 28-dimensional features such as pulse position, pulse rate, and pulse shape, and combining a vascular stiffness compensation algorithm to correct age differences; 3. Tactile simulation: Using the UR3 collaborative robotic arm to achieve six-degree-of-freedom touch and pressure feedback, reproducing 200 pulse mechanical models.

[0043] V. Fusion decision-making function: 1. Weight allocation of the four diagnoses: Dynamically adjust the weights of the observation, auscultation, questioning and palpation data according to the principle of "solitude hides treachery" in the "Nanjing"; 2. Syndrome determination: Use the Deep Forest algorithm to fuse multimodal data and output the syndrome probability distribution, such as the probability of phlegm and blood stasis syndrome is 82%; 3. Prescription safety verification: Automated review of Chinese medicine compatibility contraindications based on the "Compendium of Materia Medica" database.

[0044] 6. Extended functions: 1. Health management: output physical condition reports through the touch screen and link with HarmonyOS smart home devices such as adjusting moxibustion device parameters; 2. Data security: encrypt and upload diagnostic data to establish personal health records and support historical comparative analysis.

[0045] Example 2:

[0046] like Figure 3 As shown, the difference between the second embodiment and the first embodiment is that the robot body 10 is changed to a humanoid mechanical foot, and the positions of other corresponding components are also adjusted, but the components and functions are the same as those of the first embodiment.

[0047] From the products of Example 1 and Example 2, the products of the present invention fully possess the beneficial effects described in the content of the invention.

[0048] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention is disclosed through the above embodiments, the protection scope of the present invention is not limited thereto. Without departing from the concept of the present invention, any deformation or replacement of the above components shall fall within the scope of the claims of the present invention.

Claims

1. A household Chinese medicine robot, characterized in that: It includes a robot body, a facial diagnosis collection module, a tongue diagnosis analysis module, an intelligent questioning module, a pulse diagnosis detection module and a multimodal data fusion center module; the facial diagnosis collection module, tongue diagnosis analysis module, intelligent questioning module, pulse diagnosis detection module and multimodal data fusion center module are assembled on the robot body, and each module is connected to the multimodal data fusion center module through an industrial bus.

2. A household Chinese medicine robot according to claim 1, characterized in that: The facial diagnosis and acquisition module includes a five-color spectrum imager, a micro-expression recognizer, and a facial viscera mapping database; the five-color spectrum imager includes a polarization interference filter group.

3. A household Chinese medicine robot according to claim 2, characterized in that: The five-color spectrum imager is used to collect the facial reflection spectrum in the 400-1000nm band; the polarization interference filter group is used to eliminate the reflection interference of the skin stratum corneum.

4. A household Chinese medicine robot according to claim 1, characterized in that: The tongue diagnosis and analysis module includes a multi-angle 3D tongue image acquisition component, a tongue image feature extraction engine, and a tongue and vein ecchymosis detection algorithm; the multi-angle 3D tongue image acquisition component includes a ring LED fill light and binocular stereo vision.

5. A household Chinese medicine robot according to claim 4, characterized in that: The tongue diagnosis and analysis module also includes a tongue dynamic tracker and a tongue coating moistness analyzer.

6. A household Chinese medicine robot according to claim 1, characterized in that: The intelligent consultation module includes a dialect-adaptive speech recognizer, a dynamic consultation path generator, and a symptom knowledge graph.

7. A household Chinese medicine robot according to claim 6, characterized in that: The symptom knowledge graph contains syndrome differentiation rules for 30,000 clinical cases.

8. The household moxibustion intelligent robot according to claim 1, characterized in that: The pulse diagnosis detection module includes a three-dimensional pulse pressure sensor array, a pulse wave feature extractor, and a pulse simulation trainer; the three-dimensional pulse pressure sensor array is used to synchronously detect the pulse of the three parts of the inch, guan, and chi; the pulse wave feature extractor is used to calculate the 28-dimensional features of pulse position, pulse rate, and pulse shape; The pulse condition simulation trainer comprises 200 mechanical feedback models of typical pulse conditions.

9. A household Chinese medicine robot according to claim 8, characterized in that: The pulse diagnosis detection module also includes a blood vessel stiffness compensation algorithm and a pulse diagnosis guiding mechanical arm; the blood vessel stiffness compensation algorithm is used to correct the pulse assessment threshold of patients of different ages.

10. A household Chinese medicine robot according to claim 1, characterized in that: The multimodal data fusion center module includes a four-diagnosis weight allocator, a syndrome differentiation decision engine, and a contraindication checker.

Images