Body data analysis system based on virtual avatar

Abstract

The invention provides a body data analysis system based on a virtual avatar, which can improve the accuracy of body data analysis. The body data analysis system based on the virtual cloned body comprises a data perception layer, a data analysis layer and a data analysis layer, wherein the data perception layer is used for collecting physiological data, psychological data and behavior data; the digital twinborn layer is used for constructing a virtual branch of the patient on the basis of the physiological data, the psychological data and the behavioral data; and the engine layer is used for acquiring a disease evolution path based on lesion data of a preset disease and the virtual avatar, and generating an intervention scheme based on the disease evolution path.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent medical treatment, and in particular to a body data analysis system based on virtual avatars. BACKGROUND

[0002] In the current era of rapid development of digital medicine, medical data, as an important basis for medical decision-making, disease research, and health management, is growing at an unprecedented rate. However, the management and utilization of medical data face many problems that need to be solved. Disease prediction is one of the important directions of modern medical research, and accurate disease prediction can help take preventive measures in advance to reduce the risk and harm of disease. However, traditional disease prediction models have obvious limitations, mainly manifested in static prediction. Traditional models mainly rely on historical data for modeling and analysis, and through statistics and induction of past cases, a correlation model between disease and various factors is established. However, this prediction method based on historical data cannot fully consider the dynamic development characteristics of the disease. Development and regression is a complex process affected by many factors, and the traditional model lacks the ability to dynamically capture and analyze real-time data, and cannot timely reflect the changes in disease state. SUMMARY

[0003] To solve the above technical problems, the present application is proposed. The embodiments of the present application provide a body data analysis system based on virtual avatars, which can improve the accuracy of body data analysis.

[0004] According to one aspect of the present application, a body data analysis system based on virtual avatars is provided, comprising: a data perception layer for collecting physiological data, psychological data and behavioral data; a digital twin layer for constructing a virtual avatar of a patient based on the physiological data, the psychological data and the behavioral data; an engine layer for obtaining a disease evolution path based on lesion data of a preset disease and the virtual avatar, and generating an intervention scheme based on the disease evolution path.

[0005] In an embodiment, the body data analysis system based on virtual avatars further comprises an interaction layer for supporting a third-party device to view the state of the virtual avatar and intervening in the state of the virtual avatar based on user needs.

[0006] In an embodiment, the physiological data comprises blood glucose data and blood pressure data, the psychological data comprises voice spectrum data and facial expression data, and the behavioral data comprises action data and diet data; the data perception layer comprises: a first acquisition module configured to acquire the blood glucose data and the blood pressure data; a second acquisition module configured to acquire the voice spectrum data and the facial expression data; and a third acquisition module configured to acquire the action data and the diet data.

[0007] In an embodiment, the data perception layer further comprises: an integration device module configured to integrate the blood glucose data and the blood pressure data; a psychological evaluation module configured to extract emotional features from the voice spectrum data and the facial expression data through voice frequency analysis and facial expression recognition; and a behavior capture module configured to reconstruct a motion trajectory based on the action data through positioning technology.

[0008] In an embodiment, the data perception layer further comprises: a nutrition analysis module configured to analyze nutritional intake based on the diet data through image recognition.

[0009] In an embodiment, the digital twin layer comprises: a fusion module configured to perform multi-modal data fusion on the physiological data, the psychological data, and the behavioral data, respectively; and a construction module configured to construct a virtual avatar of the patient based on the fused multi-modal data.

[0010] In an embodiment, the engine layer comprises: a prediction module comprising a time convolutional neural network and a pathology simulator; and a decision module configured to integrate a doctor artificial intelligence device, a pharmacist artificial intelligence device, and a nutritionist artificial intelligence device.

[0011] In an embodiment, the time convolutional neural network is configured to monitor the physiological data at a preset frequency within a preset time period to generate a physiological data set, and predict blood glucose fluctuation based on the physiological data set.

[0012] In an embodiment, the pathology simulator is configured to analyze correlation features of the physiological data and the psychological data based on a deep learning model of self-attention mechanism to obtain a data correlation degree, and predict a disease evolution path based on lesion data of a preset disease, the predicted blood glucose fluctuation, and the data correlation degree.

[0013] In an embodiment, the decision module is further configured to: input the disease evolution path and the virtual avatar into an optimization model, demonstrate drug metabolism effects and simulate exercise intervention effects on the virtual avatar; wherein an objective function of the optimization model includes maximizing therapeutic effect, minimizing side effects, and optimizing cost; and generate an intervention scheme based on the drug metabolism effects and the exercise intervention effects on the virtual avatar.

[0014] The virtual avatar-based physical data analysis system provided by the present application can construct a virtual avatar of a patient through physiological data, psychological data, and behavior data, integrate the physiological data, the psychological data, and the behavior data to form a digital twin, simulate a disease evolution path on the virtual avatar, predict a disease development risk, and generate a corresponding intervention scheme, thereby improving the accuracy and dynamics of physical data analysis, realizing advanced prediction of disease development, and generating an intervention scheme for reference by the patient. BRIEF DESCRIPTION OF DRAWINGS

[0015] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which:

[0016] Figure 1 FIG. 1 is a structural schematic diagram of a virtual avatar-based physical data analysis system according to an example embodiment of the present application.

[0017] Figure 2 FIG. 2 is a structural schematic diagram of a virtual avatar-based physical data analysis system according to another example embodiment of the present application.

[0018] Figure 3 FIG. 3 is a flow schematic diagram of a virtual avatar-based physical data analysis method according to an example embodiment of the present application.

[0019] FIG. 1 is a structural schematic diagram of a virtual avatar-based physical data analysis system according to an example embodiment of the present application. DETAILED DESCRIPTION

[0020] Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the described example embodiments.

[0021] Disease prediction is one of the important directions of modern medical research, and accurate disease prediction can help to take preventive measures in advance, reduce the risk and harm of disease. However, the traditional disease prediction model has obvious limitations, mainly manifested as static prediction. At present, the traditional model relies on historical data and cannot simulate the dynamic development of disease. The existing technology uses LSTM (Long Short-term Memory Networks) network to analyze historical data, but does not construct a dynamic digital twin, which cannot simulate organ-level pathological changes. In addition, the existing wearable devices (such as blood glucose meters) only alarm when the index is abnormal, and the patient is already in a pathological state, which has a lag. Mobile medical APPs (such as MyFitnessPal) provide general recommendations, but lack the ability to dynamically adjust individualization. And at present, the presentation of medical information is mainly in the form of text reports, two-dimensional charts and the like, lacking intuitive and visual three-dimensional visualization interface. For patients, these abstract information is difficult to understand, and it is difficult to intuitively feel the occurrence, development process of disease in the body and the mechanism of action of treatment measures. For example, in tumor treatment, patients often lack a clear understanding of the location, size, shape of the tumor and its relationship with the surrounding tissue, making it difficult to understand the selection of treatment plan and expected effect. For doctors, although they have professional medical knowledge, they also face difficulties in explaining complex pathological mechanisms to patients. The traditional information presentation method cannot fully display the three-dimensional structure and dynamic changes of the disease, making it difficult for doctors to convey key information to patients in simple and easy-to-understand language. This not only increases the difficulty of doctor-patient communication, but also easily leads to misunderstanding or mistrust of treatment plan by patients, affecting treatment compliance.

[0022] Therefore, the present application proposes a body data analysis system based on virtual avatar, which improves the management and utilization efficiency of body data, improves the accuracy and timeliness of disease prediction, and also improves the effect of doctor-patient communication.

[0023] Figure 1 is a structural schematic diagram of a body data analysis system based on virtual avatar provided by an exemplary embodiment of the present application, as Figure 1 shown, the body data analysis system based on virtual avatar 1 can include a data perception layer 11, a digital twin layer 12, and an engine layer 13. The data perception layer 11 is used to collect physiological data, psychological data and behavior data; the digital twin layer 12 is used to construct a virtual avatar of the patient based on the physiological data, psychological data and behavior data; the engine layer 13 is used to obtain a disease evolution path based on the lesion data of a preset disease and the virtual avatar, and generate an intervention scheme based on the disease evolution path.

[0024] The physiological data includes blood glucose data and blood pressure data, the psychological data includes voice spectrum data and facial expression data, and the behavioral data includes action data and diet data; the data perception layer includes: a first acquisition module, the first acquisition module is used for acquiring blood glucose data and blood pressure data; a second acquisition module, the second acquisition module is used for acquiring voice spectrum data and facial expression data; a third acquisition module, the third acquisition module is used for acquiring action data and diet data. For example, the first acquisition module can read blood glucose data and blood pressure data from blood glucose meters, smart bracelets and other devices, acquire voice spectrum data and facial expression data from the patient's voice and facial expressions, acquire the patient's action data according to the action sensor, and analyze the nutritional intake according to the patient's diet image recognition.

[0025] Figure 2 is the structure schematic diagram of the body data analysis system based on virtual avatar provided by another exemplary embodiment of the application, as Figure 2 shown, the data perception layer 11 includes: an integrated device module 111, the integrated device module 111 is used for integrating blood glucose data and blood pressure data; a psychological assessment module 112, the psychological assessment module 112 is used for extracting emotional features from voice spectrum data and facial expression data through voice frequency analysis and facial expression recognition; a behavior capture module 113, the behavior capture module 113 is used for reconstructing motion trajectory based on action data through positioning technology.

[0026] For example, the data perception layer can be used to integrate IoT (Internet of Things) devices (blood glucose meters, smart bracelets), psychological assessment APP (voice emotion analysis), and behavior capture systems (such as Kinect action sensors). The digital twin layer can construct a patient virtual avatar, and the digital twin layer can include: a physiological model: organ-level finite element simulation (such as pancreatic beta cell metabolism simulation); a psychological model: BERT (Bidirectional Encoder Representations from Transformers) based semantic analysis and micro-expression recognition; a behavior model: three-dimensional reconstruction of motion trajectory (SLAM algorithm).

[0027] Organ-level finite element simulation is a technique that uses the finite element method (FEM) to numerically simulate and analyze organs. It discretizes the complex organ structure into a combination of a finite number of simple elements, establishes a mechanical or physical model for each element, and considers the interaction between elements to simulate and predict the mechanical behavior, physiological function, and other aspects of the entire organ under different conditions. For example, the geometric structure of an organ is divided into numerous small elements such as tetrahedrons, hexahedrons, etc. The shape and size of the elements can be adjusted according to the complexity of the organ and the calculation requirements. A mechanical or physical equation is established for each element to describe the relationship between the physical quantities such as stress, strain, displacement, etc. inside the element. These equations are usually based on the basic theories of elasticity, fluid mechanics, etc. For example, when simulating the mechanical behavior of the heart, the elasticity equation is used to describe the stress-strain relationship of the myocardial tissue. The equations of each element are combined to form a global equation system, which describes the mechanical or physical state of the entire organ. Numerical calculation methods such as iterative method, direct method, etc. are used to solve the global equation system to obtain the distribution of physical quantities such as stress, strain, displacement, etc. of the organ under different conditions. Through organ-level finite element simulation, it can help to understand the mechanical and physiological changes of organs during disease development, such as in the study of atherosclerosis, through finite element simulation, the stress distribution changes of the blood vessel wall under conditions such as high blood pressure, high blood fat, and the formation and development of plaques can be simulated, thereby revealing the mechanism of disease occurrence. In addition, it can also evaluate the impact of drugs on organ function and structure. For example, in the development of new anti-tumor drugs, the mechanical and physiological effects of drugs on tumor tissue and its surrounding normal organs can be simulated to predict the efficacy and potential side effects of drugs. In the auxiliary diagnosis of diseases, by comparing the simulation results of normal organs and diseased organs, doctors can assist in disease diagnosis. For example, in the diagnosis of orthopedic diseases, the stress distribution of the skeleton under different stress conditions is simulated, combined with the actual symptoms and imaging examination results of the patient, to improve the accuracy of diagnosis.

[0028] BERT-based semantic analysis and micro-expression recognition is a solution that combines the powerful natural language processing capabilities of BERT with semantic analysis and micro-expression recognition technology, respectively, to improve the performance of related tasks. BERT uses a bidirectional Transformer encoder to perform deep representation learning on text, which can capture rich semantic information and accurately understand the meaning of text in semantic analysis. When applied to micro-expression recognition, it processes micro-expression-related text descriptions (such as emotion labels, feature descriptions, etc.) to assist in the extraction and classification of micro-expression features. Because BERT can capture rich semantic information, it can usually achieve high accuracy in semantic analysis tasks. Therefore, using BERT's semantic analysis and micro-expression recognition to establish a psychological model can determine the emotional category or other related attributes of micro-expression. For example, in micro-expression recognition, there are usually some text descriptions related to micro-expression, such as emotion labels (angry, sad, happy, etc.), micro-expression feature descriptions (eyebrow droop, mouth upturn, etc.). Input these text descriptions into the BERT model to get their semantic representations, and then fuse the text semantic representations generated by BERT with the image features of micro-expression (such as features extracted by convolutional neural networks). The fused features can more comprehensively represent the information of micro-expression, which helps to improve the accuracy of recognition. And in the psychological model, the micro-expression image and the corresponding text description (such as experimental scene description, subject psychological state description, etc.) are more closely related through BERT. For example, when studying human micro-expression responses in a specific situation, using BERT to analyze the semantics of the situation text, combined with micro-expression image features, can more deeply understand the relationship between micro-expression and the situation. In semantic analysis, not only the emotion of static text is focused on, but also the dynamic changes of micro-expression in the expression process of the speaker or text creator are captured by combining micro-expression recognition technology. The combination of the two achieves more accurate psychological evaluation.

[0029] The motion trajectory three-dimensional reconstruction aims to obtain the environment information through the sensor, estimate the motion trajectory of the sensor by using the SLAM (Simultaneous Localization and Mapping) algorithm, and construct a three-dimensional map of the surrounding environment. For example, first, the key points of the human skeleton, such as the head, shoulder, elbow, hand, hip, knee and foot, are identified through the Kinect motion sensor. Through tracking and analyzing these key points, the Kinect can capture the motion and posture of the human body in real time. Then, for visual SLAM, feature points are extracted from the image, and feature matching is performed between consecutive frames to estimate the relative motion between adjacent frames. According to the results of feature matching, the pose transformation of the camera between adjacent frames is calculated using geometric relationships. For laser SLAM, the relative motion between adjacent scans is estimated by a point cloud registration algorithm such as the ICP algorithm. The relative motion between consecutive frames is accumulated to construct a local motion trajectory and map.

[0030] The motion trajectory three-dimensional reconstruction process can be as follows: the sensor starts working, collects initial environment data, the SLAM algorithm is initialized, the initial pose and map are determined, the sensor data is continuously collected, the front-end odometer processes the data, estimates the relative motion between adjacent frames, constructs a local map, and the local map information is incorporated into the back-end optimization framework for global optimization. At the same time, loop detection is performed, and if a loop is detected, the loop information is added to the optimization process to eliminate cumulative errors. According to the optimized pose and map point information, the global map is updated, and the motion trajectory and three-dimensional map are output. Thus, a behavior model of a virtual avatar can be constructed to predict and reconstruct the behavior of the virtual avatar.

[0031] In some embodiments, the data perception layer further includes a nutrition analysis module, which analyzes nutritional intake based on dietary data through image recognition. For example, deep learning algorithms (such as convolutional neural networks, CNNs) are used to extract features and classify the patient's dietary images, identifying the types of food in the images (such as apples, rice, chicken, etc.), and estimating the portion size of food based on the proportions or reference objects in the images (such as the size of the plate). Next, a database is constructed, covering common foods and their nutritional components (such as protein, fat, carbohydrates, etc.), distinguishing the impact of different cooking methods on nutritional components (such as steaming and frying). The image recognition results are compared with the food features in the database to find the most matching food. Combined with the portion estimation results in the images, the nutritional component data in the database is adjusted, and finally, the user's dietary behavior is calculated based on the protein, fat, and carbohydrate content of the food. Based on the user's nutritional analysis, a more reasonable virtual avatar can be constructed, allowing the daily evolution path of the virtual avatar to synchronize with the patient's real life, thereby obtaining more accurate simulation results when simulating diseases and medications on the virtual identity.

[0032] In some embodiments, the digital twin layer may include: a fusion module for performing multimodal data fusion on physiological data, psychological data, and behavioral data respectively; and a construction module for constructing a virtual clone of the patient based on the fused multimodal data.

[0033] Multimodal data fusion refers to the integration and comprehensive analysis of information from different modalities to obtain a more comprehensive, accurate, and rich understanding. Single-modal data often has limitations; for example, text may lack spatial information, and images may lack semantic description. By fusing multimodal data, the shortcomings of single modalities can be compensated for, improving the completeness and accuracy of information. When creating a virtual avatar, combining physiological, psychological, and behavioral data allows for the establishment of correlation analysis between the patient's psychological state and behavioral data. By integrating the patient's dietary, exercise, and other behavioral data, and through multimodal fusion technology, a digital twin containing physiological, psychological, and behavioral data can be created. The constructed virtual avatar has a stronger correlation and more accurate representation of the patient.

[0034] like Figure 2 As shown, engine layer 13 includes a prediction module 121 and a decision module 122. The prediction module 121 includes a temporal convolutional neural network and a pathology simulator. The decision module 122 is used to integrate AI devices for doctors, pharmacists, and nutritionists. The pathology simulator can be a GAN (Generative Adversarial Network) pathology simulator.

[0035] The time convolutional neural network is configured to monitor physiological data at a preset frequency within a preset time period, generate a physiological data set, and predict blood glucose fluctuation based on the physiological data set. The pathology simulator is configured to analyze the correlation between physiological data and psychological data based on a deep learning model based on a self-attention mechanism, obtain a data correlation degree, and predict a disease evolution path based on pathological data of a preset disease, the predicted blood glucose fluctuation, and the data correlation degree.

[0036] The time convolutional neural network (TCN) and the pathology simulator belong to the fields of deep learning and medical simulation, respectively, but can be combined for medical time series data analysis and disease dynamic simulation. The time convolutional neural network is a convolutional neural network specially used for processing time series data, which can process sequence data in parallel and has higher training efficiency. The pathology simulator is a tool based on mathematical or physical models, which is used to simulate the occurrence, development and spread of diseases. The TCN can be used to extract features from medical time series data as input parameters of the pathology simulator, predict key parameters of disease progression through the TCN, update the parameters of the simulator in real time, and improve the accuracy of the simulation. Then, the output of the pathology simulator is compared with the real data, the accuracy of the simulation result is evaluated by the TCN, the model structure or parameters of the pathology simulator are adjusted based on the feedback of the TCN, and the reliability of the simulation is improved. Therefore, the TCN and the pathology simulator have unique advantages in time series analysis and disease dynamic simulation, respectively, and the combination of the two can realize data-driven pathology simulation and improve the accuracy of disease prediction and decision-making.

[0037] In some embodiments, the prediction engine can employ a pathology state generator (StyleGAN3 architecture) and a risk quantifier (Bayesian network probability calculation).

[0038] In some embodiments, the decision module can also be configured to input the disease evolution path and the virtual avatar into an optimization model, demonstrate the drug metabolism effect and simulate the exercise intervention effect on the virtual avatar, wherein the objective function of the optimization model includes maximizing the therapeutic effect, minimizing the side effects, and optimizing the cost, and generate an intervention scheme based on the drug metabolism effect and the exercise intervention effect on the virtual avatar.

[0039] To optimize the model, the objective function can be adjusted according to the actual situation of the patient. The default objective function includes maximizing the therapeutic effect, minimizing the side effects, and optimizing the cost. The three-dimensional virtual avatar established by fusing physiological monitoring data, psychological evaluation results, and behavior trajectory can conform to the real situation of the patient and synchronize the patient's various physical parameters in the real world. Therefore, the drug metabolism effect on the virtual avatar can realize organ-level disease evolution visualization, improve doctor decision-making efficiency, and assist patients and doctors in understanding the disease process. Simulating the effect of exercise intervention on the virtual avatar, combined with the drug metabolism effect, can quantify the interactive influence of drug metabolism and exercise intervention, thereby optimizing the exercise program to improve drug efficacy or reduce side effects, or adjusting the drug treatment program for poor exercise intervention. Multi-dimensional evaluation of the treatment program can improve the rationality of the generated intervention program. Moreover, virtual drug testing can reduce the cost of clinical verification.

[0040] As shown in Figure 2 , the body data analysis system based on virtual avatar 1 can also include an interaction layer 14 for supporting third-party devices to view the state of the virtual avatar and intervene in the state of the virtual avatar based on user needs. For example, supporting VR (Virtual Reality) / AR (Augmented Reality) devices to view the state of the virtual avatar and intervene in the state of the virtual avatar.

[0041] In some embodiments, the interaction layer can be implemented as a meta-universe interaction interface: supporting VR / AR mode to view the prediction results and intervention programs. For example, doctors view the patient's lesion simulation video through the meta-universe interface and select the appropriate intervention program. The system synchronously pushes personalized recipes and exercise plans to the patient's VR device.

[0042] Figure 3 is a flowchart of a body data analysis method based on a virtual avatar provided by an exemplary embodiment of the present application. The body data analysis method based on a virtual avatar can be applied to a body data analysis system based on a virtual avatar, as shown in Figure 3 , physiological data, psychological data, and behavior data are collected (see S310 of Figure 3 ); based on the physiological data, psychological data, and behavior data, a virtual avatar of the patient is constructed (see S320 of Figure 3 ); based on the lesion data of the preset disease and the virtual avatar, a disease evolution path is obtained (see S330 of Figure 3 ); and based on the disease evolution path, an intervention program is generated (see S340 of Figure 3 ).

[0043] In some embodiments, the physiological data includes blood glucose data and blood pressure data, the psychological data includes voice spectrum data and facial expression data, the behavioral data includes action data and diet data, and S310 can include collecting blood glucose data, blood pressure data, voice spectrum data, facial expression data, action data, and diet data.

[0044] For example, blood glucose data and blood pressure data are read from devices such as blood glucose meters, smart bracelets, etc., voice spectrum data and facial expression data are collected from the patient's voice and facial expressions, action data of the patient is collected according to action sensors, and nutritional intake is analyzed according to the patient's diet image recognition.

[0045] In some embodiments, S320 can include performing multi-modal data fusion on the physiological data, the psychological data, and the behavioral data, respectively; and constructing a virtual avatar of the patient based on the fused multi-modal data.

[0046] Multi-modal data fusion refers to the integration and comprehensive analysis of information from different modalities to obtain a more comprehensive, accurate, and rich understanding. Single-modal data often has limitations, such as text may lack spatial information, and images may lack semantic description. By fusing multi-modal data, the shortcomings of single modalities can be compensated for, improving information integrity and accuracy. When establishing a virtual avatar, combining physiological data, psychological data, and behavioral data can establish an association analysis of the patient's psychological state and behavioral data, integrate the patient's behavioral data such as diet and exercise, and through multi-modal fusion technology, a digital twin containing physiological, psychological, and behavioral data is established, and the virtual avatar constructed has stronger correlation and more accurate restoration with the patient himself.

[0047] For example, blood glucose data and blood pressure data can be integrated through a preset standard to establish an association between blood glucose data and blood pressure data, generating fused physiological detection data; emotional features can be extracted from voice spectrum data and facial expression data through voice frequency analysis and facial expression recognition; and through positioning technology, motion trajectories can be reconstructed based on action data, and nutritional intake can be analyzed based on diet data through image recognition.

[0048] In some embodiments, based on a time convolutional neural network, physiological data is monitored at a preset frequency within a preset time period to generate a physiological data set; based on the physiological data set, blood glucose fluctuation is predicted; based on a deep learning model of a self-attention mechanism, the association features of physiological data and psychological data are analyzed to obtain a data correlation degree; and based on the lesion data of a preset disease and the virtual avatar, a disease evolution path is obtained, including: based on the lesion data of the preset disease, the predicted blood glucose fluctuation, and the data correlation degree, the disease evolution path is predicted.

[0049] For example, using TCN network to process continuous monitoring data (blood glucose data and blood pressure data), predict future 72-hour blood glucose fluctuations. In addition to integrating relevant data by category, different categories of data can also be integrated, such as integrating psychological data and behavioral data, using a Transformer model to analyze the psychological-physiological data correlation.

[0050] In some embodiments, the disease evolution path and the virtual avatar are input into an optimization model, and the drug metabolism effect and the simulated exercise intervention effect are demonstrated on the virtual avatar; wherein the objective function of the optimization model includes maximizing efficacy, minimizing side effects, and optimizing cost; based on the drug metabolism effect and the exercise intervention effect of the virtual avatar, an intervention plan is generated.

[0051] For example, the objective function: maximize efficacy (HbA1c reduction), minimize side effects (hypoglycemia incidence), and optimize cost. Virtual verification can also be performed on the virtual avatar, such as pre-acting drug metabolism on the virtual avatar, simulating exercise intervention effects (based on musculoskeletal simulation) on the virtual avatar. Therefore, the intervention plan can include a virtual drug testing environment and a multi-agent negotiation mechanism (doctor AI + pharmacist AI + nutritionist AI).

[0052] The doctor AI can provide preliminary diagnosis and treatment plan based on patient's symptoms, medical history and examination results, predict disease risk by analyzing patient's health data (such as blood pressure and blood glucose), provide long-term monitoring and management suggestions for chronic disease patients (such as diabetes, hypertension). The pharmacist AI can analyze the patient's medication, avoid potential drug interactions, provide medication usage, dosage, precautions and possible side effects, adjust drug dosage according to patient's liver and kidney function, age, etc. The nutritionist AI can develop a diet plan based on the patient's health status, eating habits and nutritional needs, analyze the patient's diet records, assess whether the nutritional intake is balanced, and provide specific dietary recommendations such as low sugar, low salt, high fiber, etc. The doctor AI is responsible for disease diagnosis and treatment, the pharmacist AI ensures medication safety, and the nutritionist AI provides dietary support. The three of them work together to provide users with comprehensive services from prevention to treatment to rehabilitation. Multi-role AI can provide targeted intervention plans in combination with the patient's virtual avatar, improve medical efficiency, and reduce medical errors.

[0053] As a possible implementation, taking diabetes management as an example, the patient wears a smart device, the system automatically collects the fasting blood glucose (6.5 mmol / L), the number of steps (3500 steps), the voice emotion (anxiety index 0.72), the digital twin layer updates the pancreas model, and the beta cell activity is reduced to 68%. The AI engine predicts that the risk of retinopathy in 3 years is 42%, and generates two types of intervention schemes. Scheme A: metformin + 30 minutes of fast walking per day (expected risk reduced to 28%), scheme B: SGLT2 inhibitor + low-carbohydrate diet (expected risk reduced to 25%). The doctor views the vascular lesion simulation video through the interactive interface, selects scheme B and adjusts the dose, and the system synchronously pushes the personalized diet and exercise plan to the patient's VR device.

[0054] Therefore, the present application can quickly locate the lesion organ through the virtual avatar, improve the doctor's decision-making efficiency, intervene early to reduce acute complications, reduce the rate of emergency visits, provide an interactive interface through the interaction layer, make it more convenient and timely for patients and doctors to operate, reduce the cost of clinical verification through virtual drug testing, and reduce annual medical expenses through personalized schemes.

[0055] The embodiment of the present application provides a virtual avatar body data analysis device. The device embodiment can be realized by software, or realized by hardware or a combination of software and hardware. From the hardware aspect, in addition to CPU, memory, network interface and non-volatile memory, the device in the embodiment can also include other hardware, such as a forwarding chip responsible for processing packets, etc. Taking software realization as an example, as a device in logical sense, it is formed by reading the corresponding computer program instructions in the non-volatile memory to the memory for running by the CPU of the device.

[0056] According to another aspect of the present application, a computer readable storage medium is provided, the storage medium stores a computer program, and the computer program is used to execute the virtual avatar-based body data analysis method of any of the above embodiments.

[0057] In addition to the above method and device, the embodiment of the present application can also be a computer program product, which includes computer program instructions, and the computer program instructions make the processor execute the steps in the virtual avatar-based body data analysis method according to various embodiments of the present application described above when the processor runs.

[0058] According to another aspect of the present application, an electronic device is provided, which includes: a processor; a memory for storing processor-executable instructions; and the processor, which is used to execute the virtual avatar-based body data analysis method of any of the above embodiments.

[0059] In addition, an embodiment of the present application can also be a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the steps described above in the virtual avatar based body data analysis method according to various embodiments of the present application.

[0060] The above description is merely illustrative of the application, and not in limitation of the application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the scope of protection of the application.

Claims

1. A virtual avatar-based body data analysis system, characterized by, The virtual avatar-based physical data analysis system comprises: a data perception layer for collecting physiological data, psychological data and behavioral data; a digital twin layer for constructing a virtual avatar of a patient based on the physiological data, the psychological data and the behavioral data; an engine layer for obtaining a disease evolution path based on lesion data of a preset disease and the virtual avatar, and generating an intervention scheme based on the disease evolution path.

2. The virtual avatar-based body data analysis system of claim 1, wherein, The virtual avatar-based physical data analysis system further comprises: an interaction layer for supporting a third-party device to view a state of the virtual avatar, and intervening in a pre-performance of the state of the virtual avatar based on user demand.

3. The virtual avatar-based body data analysis system of claim 1, wherein, The physiological data comprises blood glucose data and blood pressure data, the psychological data comprises voice spectrum data and facial expression data, and the behavioral data comprises action data and dietary data; the data perception layer comprises: a first collection module for collecting blood glucose data and blood pressure data; a second collection module for collecting voice spectrum data and facial expression data; a third collection module for collecting action data and dietary data.

4. The virtual avatar-based body data analysis system of claim 3, wherein, The data perception layer further comprises: an integration device module for integrating blood glucose data and blood pressure data; a psychological assessment module for extracting emotional features from the voice spectrum data and facial expression data through voice frequency analysis and facial expression recognition; a behavior capture module for reconstructing a motion trajectory based on the action data through positioning technology.

5. The virtual avatar-based body data analysis system of claim 3, wherein, The data perception layer further comprises: a nutrition analysis module for analyzing nutritional intake based on the dietary data through image recognition.

6. The virtual avatar-based body data analysis system of claim 1, wherein, The digital twin layer comprises: a fusion module for performing multi-modal data fusion on the physiological data, the psychological data and the behavioral data respectively; a construction module for constructing a virtual avatar of a patient based on the fused multi-modal data.

7. The virtual avatar-based body data analysis system of claim 1, wherein, The engine layer comprises: a prediction module comprising a time convolutional neural network and a pathology simulator; a decision module for integrating a doctor artificial intelligence device, a pharmacist artificial intelligence device and a nutritionist artificial intelligence device.

8. The virtual avatar-based body data analysis system of claim 7, wherein, The time convolutional neural network is used to monitor the physiological data at a preset frequency within a preset time period to generate a physiological data set, and predict blood glucose fluctuation based on the physiological data set.

9. The virtual avatar-based body data analysis system of claim 7, wherein, The pathology simulator is used to analyze the correlation features of the physiological data and the psychological data based on a deep learning model of a self-attention mechanism to obtain a data correlation degree, and predict a disease evolution path based on lesion data of a preset disease, predicted blood glucose fluctuation and the data correlation degree.

10. The virtual avatar-based body data analysis system of claim 7, wherein, The decision module is further configured to input the disease evolution path and the virtual avatar into an optimization model, demonstrate drug metabolism effects and simulate exercise intervention effects on the virtual avatar; wherein, an objective function of the optimization model includes maximizing therapeutic effects, minimizing side effects, and optimizing costs; and generate an intervention scheme based on the drug metabolism effects and the exercise intervention effects on the virtual avatar.

Images