Intelligent physique analysis and disease prevention method based on multi-modal data fusion
In the field of health service management, the method of fusion of multimodal data based on neural network models has been solved, and more accurate and reliable physical condition analysis and disease prevention have been achieved.
Patent Information
- Application Number
- CN202510233420.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-28
- Publication Date
- 2025-06-13
AI Technical Summary
When the prior art integrates multimodal data, the feature dimensions are fewer, and the psychological characteristics, humanistic data, gender and age are not effectively integrated, which affects the accuracy and reliability of physical fitness analysis.
Based on the neural network model, the characteristics, psychological characteristics, gender, age and humanistic data of traditional Chinese and Western medicine are integrated, and a unified multimodal physique characterization and disease prevention results are generated through multimodal mapping and adaptive weight allocation mechanism.
It improves the accuracy and rapid response ability of physical fitness identification, provides personalized health advice and disease prediction, and enhances the comprehensive analysis ability of the health management system.
Smart Images
Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field:
[0001] The present invention relates to the field of health service management. More specifically, it relates to an intelligent physical constitution analysis and disease prevention method based on multi-modal data fusion. Background Art:
[0002] In the prior art, the fusion system usually fuses a small number of features and a small number of dimensions, and the classification of features and physical constitutions is not so detailed; for example, for traditional Chinese medicine tongue image features, these methods do not add the fusion of the tongue body position part (the tip of the tongue, the middle of the tongue, the root of the tongue, the sides of the tongue), and the physical constitution obtained by the fusion system finally is the nine traditional Chinese medicine physical constitutions. Some methods fuse to obtain the secondary traditional Chinese medicine physical constitution, but compared with this method, the fusion feature dimension is less.
[0003] The main problems of the prior art are how to fuse the features of each data source, perform multi-modal fusion, and comprehensively analyze to obtain suggestions. In addition, these methods fail to fuse psychological features, humanistic data, gender, and age together, and the small fusion surface will affect the accuracy and reliability of the fusion analysis.
[0004] Therefore, it is necessary to provide a method that can comprehensively analyze multi-modal data features and disease prevention to improve the accuracy of the fusion of features, physical constitutions, and disease prevention.
[0005] Related technologies: The multi-classifier neural network model (CCNN), Panda, and the weight calculation method are all mature technical methods.
[0006] Reference materials: "Classification and Judgment of Traditional Chinese Medicine Physical Constitution", "Huangdi Neijing", "Diagnostics of Traditional Chinese Medicine", "Traditional Chinese Medicine Constitution Science", "Basic Theory of Traditional Chinese Medicine", "Diagnostics of Western Medicine", "Phenological Psychology", "Health Meteorology" series, "Medical Meteorology: The Impact of Weather and Climate on Health", "Handbook of Common Psychological Assessment Scales". Summary of the Invention:
[0007] The present invention refers to teaching materials and related materials such as "Traditional Chinese Medicine Constitution Science", "Huangdi Neijing", "Diagnostics of Western Medicine", "Phenological Psychology", "Health Meteorology" series, "Medical Meteorology: The Impact of Weather and Climate on Health", "Handbook of Common Psychological Assessment Scales", etc., and based on the neural network model, comprehensively analyzes the human physical constitution by fusing multi-dimensional data such as traditional Chinese and Western medicine features, psychological features, gender, age, and human culture. On this basis, personalized suggestions for clothing, food, housing, and transportation and department recommendations are provided, and possible diseases are predicted, enhancing the accuracy of physical constitution identification; achieving a rapid response for physical constitution identification.
[0008] This system includes:
[0009] The data collection module is used to obtain the basic information filled in by the user (gender, age, weight, height, allergy history (optional), genetic history (optional)), uploaded audio and video, electronic medical records, and human factor data. Among them, the uploaded audio and video data includes ear images, tongue images, facial images, palm prints, eye diagnosis, respiratory rate, and psychological analysis data. The human factor data includes social environment, meteorological data, and family history;
[0010] The feature extraction module is used to extract physiological and psychological features from the audio and video data, and western medical features of the electronic medical records. Among them, the physiological and psychological features include traditional Chinese medicine features (ear image features, tongue image features, facial image features, palm print features, eye features), western medical features (respiratory rate, heart rate), and psychological features (emotional state, mental health state);
[0011] The dynamic fusion module integrates gender, age, and human data, performs multi-modal mapping on the physiological and psychological features output by the feature extraction module, converts them into traditional Chinese medicine secondary constitution classification and disease prevention, and generates unified multi-modal constitution representations, disease prevention results, and comprehensive suggestions through an adaptive weight allocation mechanism. Among them, the traditional Chinese medicine features need to include tongue image, facial image, palm print, and eye diagnosis data; for western medical features, the user needs to provide at least one of liver function, kidney function, blood routine, and urine routine, and achieve dynamic fusion with traditional Chinese medicine features, psychological features, gender, age, and human data to support accurate health analysis and prevention suggestions; various mapping tables have been placed in other supporting documents.
[0012] The result display module is used to display the final results in the form of pictures and texts. Among them, the picture and text data includes secondary constitution, psychological state, possible diseases, human factors (social environment, meteorology, family history), and suggestions for clothing, food, housing, transportation, and external treatment in the department are displayed on the test report interface.
[0013] For details, please refer to the system architecture Figure 1 。
[0014] Schematic diagram of constitution fusion process Figure 2 。
[0015] Schematic diagram of disease prediction fusion process Figure 3 。 Specific implementation method:
[0016] 1. Integrate the entire fusion system into the company's health management system. (Official website:
[0017] https: / / yinrk.com / WeChat mini-program name: Shenzhen Yinrunkang Technology Co., Ltd.
[0018] APP Name: Yinrun Kang (synchronized in OPPO, vivo, Xiaomi, Honor, and Huawei app markets) 2. Obtain the features extracted by traditional Chinese medicine and psychology, as well as the electronic medical records (related Western medicine indicators) uploaded by users. 3. Input the obtained features into the traditional Chinese medicine, Western medicine, and psychology integration system to obtain suggestions on clothing, food, housing, transportation, and departments. 4. Obtain the indicators of all Western medicine modules, identify the abnormal indicators, obtain the list of possible diseases corresponding to the abnormal indicators, obtain the list of possible diseases corresponding to the constitution and psychology output by the traditional Chinese medicine module, combine all the lists, calculate the number of diseases, and get the top three diseases with the largest number. According to the predicted diseases, obtain preventive suggestions.
[0019] 5. Upload all the results to the front end. Description of the Drawings:
[0020] Figure 1 It is the system architecture diagram.
[0021] Figure 2 It is the schematic diagram of the weight integration process.
[0022] Figure 3 It is the schematic diagram of the disease prediction integration process.
Claims
1. An intelligent physical analysis and disease prevention method based on multimodal data fusion, characterized in that: include: The data collection module is used to obtain the basic information filled in by the user (gender, age, weight, height, allergy history (optional), genetic history (optional)), uploaded audio and video, electronic medical records and human factors data, wherein the uploaded audio and video data includes ear image, tongue image, face image, palm print, eye diagnosis, respiratory rate, psychological analysis data, and the human factors data includes social environment, meteorological data, and family history; A feature extraction module is used to extract physiological and psychological features and Western medical features of electronic medical records from audio and video data, wherein the physiological and psychological features include Chinese medicine features (ear image features, tongue image features, facial image features, palm print features, eye features), Western medicine features (respiratory rate, heart rate, liver function, kidney function, blood routine, urine routine), and psychological features (emotional state, mental health state); The dynamic fusion module integrates gender, age and humanities data, performs multimodal mapping on the physiological and psychological characteristics output by the feature extraction module, converts them into TCM secondary constitution classification and disease prevention, and generates unified multimodal constitution representation, disease prevention results and comprehensive recommendations through an adaptive weight allocation mechanism (disease prevention does not have one). Among them, TCM features must include tongue image, facial image, palm print and eye diagnosis data; Western medicine features require the user to upload electronic medical records, provide at least one of liver function, kidney function, blood routine and urine routine, and dynamically integrate with TCM features, psychological characteristics, gender, age and humanities data to support accurate health analysis and disease prevention recommendations; The result display module is used to display the final results in the form of pictures and texts, wherein the picture and text data include secondary constitution, psychological state, possible diseases, human factors (social environment, weather, family history), food, clothing, housing, transportation and departmental external treatment recommendations are displayed on the test report interface.
2. An intelligent physical analysis and disease prevention method based on multimodal data fusion according to claim 1. It is characterized in that the dynamic fusion module consists of two parts: physical fusion and disease prevention fusion, wherein physical fusion consists of three sub-modules: traditional Chinese medicine fusion, western medicine fusion and psychological fusion, and adopts a phased fusion strategy: first, each part of the data (traditional Chinese medicine, western medicine, psychology) is independently fused, and then all sub-modules are globally fused in combination with gender, age and humanities data. Among them, Traditional Chinese medicine needs to include tongue image, facial image, palm print and eye examination data; Western medicine requires at least one of the following data in the electronic medical records uploaded by users: liver function, kidney function, blood routine and urine routine. Taking the fusion method of traditional Chinese medicine constitution as an example, its process includes: extracting the features of all traditional Chinese medicine modules; calculating the corresponding constitution probability based on the features; combining meteorological data and psychological characteristics to calculate their influence on constitution probability; multiplying each constitution probability by the internal weight of a single module and the global weight of each module in traditional Chinese medicine diagnosis to obtain a weighted new probability; accumulating all constitution probabilities and extracting the three constitution types with the highest probability; finally generating personalized food, clothing, housing and transportation recommendations based on the constitution type. Global fusion integrates the constitutions obtained by fusion of each submodule again to obtain the final constitution and give recommendations. The fusion of disease prevention is similar to the fusion of constitution (the difference is that the fusion of disease prevention does not include custom weight distribution).
3. An intelligent constitution analysis and disease prevention method based on multimodal data fusion according to claim 1. Characterized in that the result display module displays secondary constitution, psychological state, possible diseases, human factors (social environment, weather, family history), food, clothing, housing, transportation and departmental external treatment recommendations. The fused constitution is presented in a pie chart, divided into three parts to show the proportion of the three constitutions; psychological characteristics are presented in a bar chart according to the acquired user's emotional levels (anger, disgust, fear, happiness, sadness, surprise, calmness, positive, negative); intuitive visual design is used to ensure that users can quickly understand and obtain key result information.