Multimodal health management program generation method and system based on deep learning
Through deep learning technology, analyzing multimodal health data is generated to generate personalized health management solutions, solving the problems of high labor costs and low service efficiency of experts in the existing health management solutions, and achieving more efficient and accurate health management.
Patent Information
- Application Number
- CN202311356531.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-19
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2043-10-19
AI Technical Summary
The existing health management plan requires detailed analysis based on individual differences and sign data, resulting in high labor costs for experts, low service efficiency and long waiting time.
Using a multimodal health management scheme generation method based on deep learning, the blood pressure, blood glucose, heart rate, blood oxygen saturation, body temperature and respiration rate data of client objects were obtained and preprocessed, and correlation analysis was performed to generate global sign semantic understanding feature vectors. , and then determine personalized non-pharmaceutical intervention health management plans.
It reduces the labor costs of experts, improves service efficiency, provides more accurate and personalized health management advice, and reduces the risk of chronic diseases.
Smart Images

Figure CN117612732B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent health management technology, and in particular to a method and system for generating a multimodal health management solution based on deep learning. Background Art
[0002] With the aging of the population and the increase of chronic diseases, the demand for health management is increasing. Health management programs are an important part of health management. They are a series of specific, operational and evaluable health promotion measures formulated based on the health risks and needs of individuals or groups. Health management programs usually include two types of interventions: drug intervention and non-drug intervention. Non-drug intervention mainly refers to improving or maintaining health status through adjustments in lifestyle, psychology, behavior, etc.
[0003] There are many problems with existing health management, one of which is the high cost of expert labor. Since health management plans need to be formulated based on individual differences and physical data of clients, professionals are required to conduct detailed analysis and evaluation of each client, and then recommend appropriate non-drug intervention plans based on their health status and goals. This approach not only consumes the time and energy of professionals, but also increases the waiting time and service costs of clients. Therefore, an optimized solution is expected. Summary of the invention
[0004] The embodiment of the present invention provides a multimodal health management program generation method based on deep learning and a system thereof, which obtains the blood pressure value, blood sugar value, heart rate value, blood oxygen saturation, body temperature value and respiratory rate of the analyzed customer object at multiple predetermined time points within a predetermined time period; performs data preprocessing to obtain a blood pressure time series input vector, a blood sugar time series input vector, a heart rate time series input vector, a blood oxygen saturation time series input vector, a body temperature time series input vector and a respiratory rate time series input vector; performs association analysis on the blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector and the respiratory rate time series input vector to obtain a global physical sign semantic understanding feature vector of the customer object; and, based on the global physical sign semantic understanding feature vector of the customer object, determines a health management program for non-drug intervention. In this way, the manpower cost of experts can be reduced and service efficiency can be improved.
[0005] The embodiment of the present invention also provides a method for generating a multimodal health management plan based on deep learning, which includes:
[0006] Obtaining blood pressure values, blood sugar values, heart rate values, blood oxygen saturation, body temperature values and respiratory rate of the analyzed client at multiple predetermined time points within a predetermined time period;
[0007] Preprocessing the blood pressure values, blood glucose values, heart rate values, blood oxygen saturation, body temperature values and respiratory rates at the plurality of predetermined time points to obtain a blood pressure time series input vector, a blood glucose time series input vector, a heart rate time series input vector, a blood oxygen saturation time series input vector, a body temperature time series input vector and a respiratory rate time series input vector;
[0008] Performing association analysis on the blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector, and the respiratory rate time series input vector to obtain a global vital sign semantic understanding feature vector of the client object; and
[0009] Based on the global vital sign semantic understanding feature vector of the client object, a health management plan with non-drug intervention is determined.
[0010] The embodiment of the present invention also provides a multimodal health management program generation system based on deep learning, which includes:
[0011] A data acquisition module, used to acquire blood pressure values, blood sugar values, heart rate values, blood oxygen saturation, body temperature values and respiratory rate of the analyzed customer object at multiple predetermined time points within a predetermined time period;
[0012] a data preprocessing module, configured to perform data preprocessing on the blood pressure values, blood glucose values, heart rate values, blood oxygen saturation, body temperature values and respiratory rates at the plurality of predetermined time points to obtain a blood pressure time series input vector, a blood glucose time series input vector, a heart rate time series input vector, a blood oxygen saturation time series input vector, a body temperature time series input vector and a respiratory rate time series input vector;
[0013] an association analysis module, configured to perform association analysis on the blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector, and the respiratory rate time series input vector to obtain a global vital sign semantic understanding feature vector of the client object; and
[0014] The health management program determination module is used to determine a health management program of non-drug intervention based on the semantic understanding feature vector of the global vital signs of the client object. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:
[0016] Figure 1Schematic diagram of the structure of the multimodal health management system provided in an embodiment of the present invention.
[0017] Figure 2 This is a flowchart of a method for generating a multimodal health management solution based on deep learning provided in an embodiment of the present invention.
[0018] Figure 3 A schematic diagram of the system architecture of a method for generating a multimodal health management solution based on deep learning provided in an embodiment of the present invention.
[0019] Figure 4 A block diagram of a multimodal health management solution generation system based on deep learning provided in an embodiment of the present invention.
[0020] Figure 5 This is an application scenario diagram of a multimodal health management solution generation method based on deep learning provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0021] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.
[0022] Unless otherwise specified, all technical and scientific terms used in the embodiments of the present invention have the same meanings as those commonly understood by those skilled in the art of the present invention. The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the scope of the present invention.
[0023] In the description of the embodiments of the present invention, it should be noted that, unless otherwise specified and limited, the term "connection" should be understood in a broad sense. For example, it can be an electrical connection or a connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.
[0024] It should be noted that the terms "first\second\third" involved in the embodiments of the present invention are only used to distinguish similar objects, and do not represent a specific order for the objects. It is understandable that the specific order or sequence of "first\second\third" can be interchanged where permitted. It should be understood that the objects distinguished by "first\second\third" can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.
[0025] In modern society, people are paying more and more attention to personal health and quality of life. However, due to changes in people's lifestyles and environments, traditional health management methods can no longer meet people's needs. Therefore, it is very important to develop a method and system that can generate health management plans using multimodal data.
[0026] The present invention provides a multimodal health management program generation method and system based on deep learning, which generates a personalized health management program by integrating multiple data sources, including but not limited to biosensor data, medical record data, health questionnaire data, etc.
[0027] First, the present invention uses deep learning technology to analyze and process multimodal data. The deep learning model can automatically extract and learn high-level features in the data, thereby achieving modeling and prediction of complex data patterns. By training the deep learning model, the correlation and features between different data sources can be learned and extracted.
[0028] Secondly, the present invention uses generative models to generate personalized health management plans. Generative models based on deep learning can learn data distribution to generate plans that match individual health conditions and needs. Generative models can combine individual physiological indicators, medical history, lifestyle and other factors to generate targeted health management suggestions.
[0029] Further, Figure 1 It is a structural diagram of the multimodal health management system. Figure 1 As shown, the present invention provides a multimodal health management system for implementing the above method. The system includes a data acquisition module, a deep learning module, a generation model and a solution generation module. The data acquisition module is used to collect multimodal data, the deep learning module is used to analyze and process the data, the generation model is used to generate a personalized health management plan, and the solution generation module is used to generate a final health management plan based on the output of the generation model.
[0030] The advantage of the present invention is that it can comprehensively utilize multiple data sources to achieve a more comprehensive and accurate health management plan generation. Through the application of deep learning technology, it is possible to better mine the information and potential rules in the data and provide personalized health management suggestions for individuals.
[0031] In one embodiment of the present application, the process of the multimodal health management program generation method based on deep learning is:
[0032] Client information input: symptom questionnaire;
[0033] Automatic information collection: blood pressure, blood sugar, body composition, smart wearable devices (heart rate, blood oxygen, body temperature, HRV, simulated lead ECG, sleep, respiratory rate);
[0034] Auxiliary testing equipment data: facial diagnosis, hand diagnosis, psychological stress, organ function status, and traditional Chinese medicine meridian detection data;
[0035] Medical indicators: Western medicine and functional medicine physical examination indicators;
[0036] Through comprehensive, cross-matrix verification and calculation of the above data, the health status and the root cause of the health problem can be finally determined. Through model matching, non-drug interventions are directly given, including diet, nutrition, exercise, psychology, sleep and other aspects of life. Health management plans.
[0037] The present invention realizes the generation of personalized health management plans by using deep learning technology and multimodal data. Compared with traditional health management methods, the present invention can more accurately analyze and predict the health status of individuals and provide more targeted health management suggestions, which has a positive significance for promoting people's health and quality of life.
[0038] Specifically, in one embodiment of the present invention, Figure 2 This is a flowchart of a method for generating a multimodal health management solution based on deep learning provided in an embodiment of the present invention. Figure 3 Schematic diagram of the system architecture of a method for generating a multi-modal health management solution based on deep learning provided in an embodiment of the present invention. Figure 2 and Figure 3 As shown, according to an embodiment of the present invention, a multimodal health management program generation method based on deep learning includes: 110, obtaining blood pressure values, blood sugar values, heart rate values, blood oxygen saturation, body temperature values and respiratory rate of the analyzed customer object at multiple predetermined time points within a predetermined time period; 120, performing data preprocessing on the blood pressure values, blood sugar values, heart rate values, blood oxygen saturation, body temperature values and respiratory rate at the multiple predetermined time points to obtain a blood pressure time series input vector, a blood sugar time series input vector, a heart rate time series input vector, a blood oxygen saturation time series input vector, a body temperature time series input vector and a respiratory rate time series input vector; 130, performing association analysis on the blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector and the respiratory rate time series input vector to obtain a global vital sign semantic understanding feature vector of the customer object; and, 140, determining a health management program for non-drug intervention based on the global vital sign semantic understanding feature vector of the customer object.
[0039] In step 110, ensure that the physiological indicator data of the client at multiple predetermined time points within a predetermined time period are obtained. This can be collected by using sensor devices, health monitoring devices or mobile applications. Pay attention to ensuring the accuracy and completeness of the data, and properly handle possible data missing or abnormal situations. By obtaining the physiological indicator data at multiple predetermined time points, the health status of the client at different time points can be fully understood, providing basic data for subsequent analysis and recommendations.
[0040] In step 120, the acquired physiological index data is preprocessed to obtain a time series input vector. The preprocessing method may include data cleaning, denoising, interpolation, smoothing or standardization to ensure the quality and availability of the data. Through data preprocessing, noise and outliers can be removed, data fluctuations can be smoothed, and the data can be converted into a time series input vector suitable for subsequent analysis, providing preparation for further association analysis.
[0041] In step 130, the preprocessed time series input vector is subjected to association analysis to extract the global vital sign semantic understanding feature vector of the client object. The time series data can be modeled and analyzed using a deep learning algorithm (such as a recurrent neural network or an attention mechanism) to capture associations and trends in the data. Through association analysis, the time series data of multiple physiological indicators of the client object can be comprehensively considered to extract the global vital sign semantic understanding feature vector, so as to better understand the health status and changing trends of the client object.
[0042] In step 140, the global sign semantic understanding feature vector of the client object can be used to determine a non-drug intervention health management program suitable for the client object based on a pre-established health management program database or expert knowledge. Machine learning algorithms, recommendation system algorithms, or rule engines and other methods can be used to match and recommend the feature vector of the client object with the health management program. By determining the health management program based on the global sign semantic understanding feature vector, a personalized non-drug intervention program can be provided based on the individual differences and health needs of the client object to help the client object improve or maintain health conditions and prevent the occurrence of chronic diseases. At the same time, through automated recommendations, the expert labor costs can be reduced and service efficiency can be improved.
[0043] In response to the above technical problems, the technical concept of the present invention is to comprehensively utilize the customer's blood pressure, blood sugar, heart rate, blood oxygen saturation, body temperature and respiratory rate, and combine them with deep learning algorithms to automatically match and recommend appropriate non-drug intervention health management plans. In this way, the expert labor cost can be reduced and service efficiency can be improved.
[0044] Based on this, in the technical solution of the present invention, first, the blood pressure value, blood sugar value, heart rate value, blood oxygen saturation, body temperature value and respiratory rate of the analyzed client object at multiple predetermined time points within a predetermined time period are obtained. It should be understood that the important physiological index data such as blood pressure value, blood sugar value, heart rate value, blood oxygen saturation, body temperature value and respiratory rate at multiple predetermined time points can reflect the health status and change trend of the client object.
[0045] In one embodiment of the present invention, data preprocessing is performed on the blood pressure values, blood sugar values, heart rate values, blood oxygen saturation, body temperature values and respiratory rates at the multiple predetermined time points to obtain a blood pressure timing input vector, a blood sugar timing input vector, a heart rate timing input vector, a blood oxygen saturation timing input vector, a body temperature timing input vector and a respiratory rate timing input vector, including: arranging the blood pressure values, blood sugar values, heart rate values, blood oxygen saturation, body temperature values and respiratory rates at the multiple predetermined time points according to the time dimension as the blood pressure timing input vector, the blood sugar timing input vector, the heart rate timing input vector, the blood oxygen saturation timing input vector, the body temperature timing input vector and the respiratory rate timing input vector.
[0046] Next, the blood pressure values, blood sugar values, heart rate values, blood oxygen saturation, body temperature values and respiratory rates at the plurality of predetermined time points are arranged according to the time dimension into a blood pressure time series input vector, a blood sugar time series input vector, a heart rate time series input vector, a blood oxygen saturation time series input vector, a body temperature time series input vector and a respiratory rate time series input vector. That is, the discrete time series distribution of each vital sign data is converted into a structured vector representation, so as to facilitate the reading and analysis of the subsequent model.
[0047] Then, the blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector and the respiratory rate time series input vector are subjected to association analysis to obtain a global vital sign semantic understanding feature vector of the client object. That is, the time series features of each vital sign data are comprehensively considered, and the time series association features between each vital sign data are constructed to enhance the understanding of the vital sign features of the client object.
[0048] In one embodiment of the present invention, an association analysis is performed on the blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector and the respiratory rate time series input vector to obtain a global vital sign semantic understanding feature vector for the customer object, including: extracting the vital sign time series features of the blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector and the respiratory rate time series input vector respectively to obtain a blood pressure time series feature vector, a blood sugar time series feature vector, a heart rate time series feature vector, a blood oxygen saturation time series feature vector, a body temperature time series feature vector and a respiratory rate time series feature vector; and extracting the association features between the blood pressure time series feature vector, the blood sugar time series feature vector, the heart rate time series feature vector, the blood oxygen saturation time series feature vector, the body temperature time series feature vector and the respiratory rate time series feature vector to obtain a global vital sign semantic understanding feature vector for the customer object.
[0049] Among them, the vital sign time series features of the blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector and the respiratory rate time series input vector are extracted respectively to obtain the blood pressure time series feature vector, the blood sugar time series feature vector, the heart rate time series feature vector, the blood oxygen saturation time series feature vector, the body temperature time series feature vector and the respiratory rate time series feature vector, including: passing the blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector and the respiratory rate time series input vector through a vital sign data time series feature extractor based on a one-dimensional convolutional neural network model to obtain the blood pressure time series feature vector, the blood sugar time series feature vector, the heart rate time series feature vector, the blood oxygen saturation time series feature vector, the body temperature time series feature vector and the respiratory rate time series feature vector.
[0050] Furthermore, the correlation features among the blood pressure time series feature vector, the blood sugar time series feature vector, the heart rate time series feature vector, the blood oxygen saturation time series feature vector, the body temperature time series feature vector and the respiratory rate time series feature vector are extracted to obtain a global vital sign semantic understanding feature vector of the customer object, including: passing the blood pressure time series feature vector, the blood sugar time series feature vector, the heart rate time series feature vector, the blood oxygen saturation time series feature vector, the body temperature time series feature vector and the respiratory rate time series feature vector through a vital sign data context encoder based on a converter module to obtain the global vital sign semantic understanding feature vector of the customer object.
[0051] In a specific example of the present invention, the blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector and the respiratory rate time series input vector are subjected to correlation analysis to obtain an encoding process of a global vital sign semantic understanding feature vector of a customer object, including: firstly, the blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector and the respiratory rate time series input vector are subjected to correlation analysis to obtain ... A vital sign data time series feature extractor of a dimensional convolutional neural network model is used to obtain a blood pressure time series feature vector, a blood glucose time series feature vector, a heart rate time series feature vector, a blood oxygen saturation time series feature vector, a body temperature time series feature vector and a respiratory rate time series feature vector; and then the blood pressure time series feature vector, the blood glucose time series feature vector, the heart rate time series feature vector, the blood oxygen saturation time series feature vector, the body temperature time series feature vector and the respiratory rate time series feature vector are passed through a vital sign data context encoder based on a converter module to obtain a global vital sign semantic understanding feature vector of the customer object.
[0052] That is, firstly, the vital sign time series features of the blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector and the respiratory rate time series input vector are extracted respectively; then, the correlation features between the various vital sign data are extracted. In this way, the time series correlation features between the various vital sign data are constructed, so as to obtain the global vital sign semantic understanding feature vector of the customer object.
[0053] In one embodiment of the present invention, based on the global vital signs semantic understanding feature vector of the client object, a health management plan with non-drug intervention is determined, including: passing the global vital signs semantic understanding feature vector of the client object through a classifier to obtain a classification result, and the classification result is used to represent a health management plan label with non-drug intervention.
[0054] Furthermore, the global sign semantic understanding feature vector of the client object is passed through a classifier to obtain a classification result, and the classification result is used to represent the label of the health management plan for non-drug intervention. The health management plan for non-drug intervention is established in advance by professionals after considering the health risks, conditions and goals of various situations. That is, through the classifier, the health management plan for non-drug intervention suitable for the client object can be automatically matched and recommended, thereby providing accurate and effective non-drug intervention measures.
[0055] In one embodiment of the present invention, the multimodal health management program generation method based on deep learning also includes a training step: training the vital sign data time series feature extractor based on the one-dimensional convolutional neural network model, the vital sign data context encoder based on the converter module and the classifier; wherein the training step includes: obtaining training data, the training data including training blood pressure values, training blood sugar values, training heart rate values, training blood oxygen saturation values, training body temperature values and training respiratory rates of the analyzed client object at multiple predetermined time points within a predetermined time period, as well as a health management program for non-drug intervention The true value of the label; the training blood pressure values, training blood sugar values, training heart rate values, training blood oxygen saturation, training body temperature values and training respiratory rate at the plurality of predetermined time points are arranged according to the time dimension as a training blood pressure timing input vector, a training blood sugar timing input vector, a training heart rate timing input vector, a training blood oxygen saturation timing input vector, a training body temperature timing input vector and a training respiratory rate timing input vector; the training blood pressure timing input vector, the training blood sugar timing input vector, the training heart rate timing input vector, the training blood oxygen saturation timing input vector, the training body temperature timing input vector and The training respiratory rate time series input vector is passed through the vital sign data time series feature extractor based on the one-dimensional convolutional neural network model to obtain a training blood pressure time series feature vector, a training blood sugar time series feature vector, a training heart rate time series feature vector, a training blood oxygen saturation time series feature vector, a training body temperature time series feature vector and a training respiratory rate time series feature vector; the training blood pressure time series feature vector, the training blood sugar time series feature vector, the training heart rate time series feature vector, the training blood oxygen saturation time series feature vector, the training body temperature time series feature vector and the training respiratory rate time series feature vector are passed through the converter based on The module's vital sign data context encoder is used to obtain a training customer object global vital sign semantic understanding feature vector; the training customer object global vital sign semantic understanding feature vector is passed through a classifier to obtain a classification loss function value; and the classification loss function value is used to train the vital sign data time series feature extractor based on the one-dimensional convolutional neural network model, the vital sign data context encoder based on the converter module, and the classifier, wherein in each round of iteration of the training, the training customer object global vital sign semantic understanding feature vector is subjected to weight space exploration constraint iteration based on class degree matrix regularization.
[0056] In the technical solution of the present invention, when the blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector and the respiratory rate time series input vector pass through a vital sign data context encoder based on a converter module, the intra-sample time series correlation features and the inter-sample full-time domain context correlation features of the time series distribution of the blood pressure value, blood sugar value, heart rate value, blood oxygen saturation, body temperature value and respiratory rate in the full time domain can be extracted, so that the global vital sign semantic understanding feature vector of the customer object includes both the single-sample time series correlation features and the cross-sample time series correlation features. Furthermore, considering the unbalanced distribution of the blood pressure value, blood sugar value, heart rate value, blood oxygen saturation, body temperature value and respiratory rate in the time series direction, the corresponding local feature distributions of the feature vector of the global vital signs semantic understanding of the customer object will also have unbalanced feature representations, which will lead to the enrichment of the label distribution corresponding to the diversified feature distribution across sample dimensions in the probability distribution domain of the classification result when the feature vector of the global vital signs semantic understanding of the customer object is classified and regressed through the classifier, thereby affecting the convergence effect of the weight matrix of the classifier during the classification process.
[0057] Based on this, when the applicant of the present invention classifies the global physical sign semantic understanding feature vector of the training customer object through a classifier, at each iteration of the weight matrix, the global physical sign semantic understanding feature vector of the training customer object is subjected to weight space exploration constraint based on class degree matrix regularization, which is specifically expressed as follows: the global physical sign semantic understanding feature vector of the training customer object is subjected to weight space exploration constraint iteration based on class degree matrix regularization according to the following optimization formula;
[0058] Wherein, the optimization formula is:
[0059]
[0060]
[0061] Wherein, V is the global physical sign semantic understanding feature vector of the training client object, specifically expressed as a column vector, V′ is a row vector, and M t ∈R L×L is a learnable domain transfer matrix, for example, it can be initially set to a diagonal matrix composed of the diagonal elements of the weight matrix M of the previous iteration, where M represents the weight matrix of the previous iteration, (·) T represents the transposed vector or transposed matrix of a vector or matrix, M′ is the weight matrix after iteration, Represents matrix multiplication.
[0062] Here, considering the domain gap between the weight space domain of the weight matrix and the probability distribution domain of the classification result of the training customer object global vital sign semantic understanding feature vector V, the weight matrix M is used as an inter-domain transferring agent relative to the class matrix regularization representation of the training customer object global vital sign semantic understanding feature vector V to transfer the probability distribution of valuable label constraints to the weight space, thereby avoiding the over-exploitation of the weight distribution in the weight space by the rich labeled probability distribution domain in the classification process based on the weight space, improving the convergence effect of the weight matrix, and also improving the training effect of the customer object global vital sign semantic understanding feature vector when classification and regression are performed through the classifier.
[0063] In summary, the deep learning-based multimodal health management program generation method based on the embodiment of the present invention is explained, which comprehensively utilizes the customer's blood pressure value, blood sugar value, heart rate value, blood oxygen saturation, body temperature value and respiratory rate, and combines the deep learning algorithm to automatically match and recommend appropriate non-drug intervention health management programs. In this way, the expert labor cost is reduced and the service efficiency is improved.
[0064] Figure 4 1 is a block diagram of a multi-modal health management solution generation system based on deep learning provided in an embodiment of the present invention. Figure 4 As shown, the multimodal health management program generation system 200 based on deep learning includes: a data acquisition module 210, which is used to obtain the blood pressure values, blood sugar values, heart rate values, blood oxygen saturation, body temperature values and respiratory rate of the analyzed customer object at multiple predetermined time points within a predetermined time period; a data preprocessing module 220, which is used to perform data preprocessing on the blood pressure values, blood sugar values, heart rate values, blood oxygen saturation, body temperature values and respiratory rate at the multiple predetermined time points to obtain a blood pressure time series input vector, a blood sugar time series input vector, a heart rate time series input vector, a blood oxygen saturation time series input vector, time series input vector, body temperature time series input vector and respiratory rate time series input vector; an association analysis module 230, used to perform association analysis on the blood pressure time series input vector, the blood glucose time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector and the respiratory rate time series input vector to obtain a global vital sign semantic understanding feature vector of the customer object; and a health management plan determination module 240, used to determine a non-drug intervention health management plan based on the global vital sign semantic understanding feature vector of the customer object.
[0065] In the multimodal health management program generation system based on deep learning, the data preprocessing module is used to arrange the blood pressure values, blood sugar values, heart rate values, blood oxygen saturation, body temperature values and respiratory rate at the multiple predetermined time points according to the time dimension as the blood pressure timing input vector, the blood sugar timing input vector, the heart rate timing input vector, the blood oxygen saturation timing input vector, the body temperature timing input vector and the respiratory rate timing input vector.
[0066] Those skilled in the art will appreciate that the specific operations of each step in the above-mentioned multimodal health management solution generation system based on deep learning have been described in detail above. Figures 1 to 3 The description of the deep learning-based multimodal health management solution generation method has been introduced in detail, and therefore, its repeated description will be omitted.
[0067] As described above, the multimodal health management solution generation system 200 based on deep learning according to an embodiment of the present invention can be implemented in various terminal devices, such as a server for generating a multimodal health management solution based on deep learning, etc. In one example, the multimodal health management solution generation system 200 based on deep learning according to an embodiment of the present invention can be integrated into the terminal device as a software module and / or a hardware module. For example, the multimodal health management solution generation system 200 based on deep learning can be a software module in the operating system of the terminal device, or it can be an application developed for the terminal device; of course, the multimodal health management solution generation system 200 based on deep learning can also be one of the many hardware modules of the terminal device.
[0068] Alternatively, in another example, the deep learning-based multimodal health management solution generation system 200 and the terminal device may also be separate devices, and the deep learning-based multimodal health management solution generation system 200 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.
[0069] Figure 5 FIG. 1 is an application scenario diagram of a method for generating a multimodal health management solution based on deep learning provided in an embodiment of the present invention. Figure 5 As shown, in this application scenario, first, the blood pressure values of the analyzed client at multiple predetermined time points within a predetermined time period are obtained (for example, Figure 5 C1 as shown in the figure), blood sugar level (e.g. Figure 5 C2 as shown in the figure), heart rate value (for example, Figure 5 C3 as shown in the figure), blood oxygen saturation (e.g. Figure 5 C4 as indicated in the figure), body temperature values (e.g. Figure 5C5 as shown in Figure ) and respiratory rate (e.g. Figure 5 Then, the obtained blood pressure value, blood sugar value, heart rate value, blood oxygen saturation, body temperature value and respiratory rate are input to a server (for example, Figure 5 In S) shown in , the server is capable of processing blood pressure values, blood sugar values, heart rate values, blood oxygen saturation, body temperature values and respiratory rate based on a multimodal health management solution generation algorithm based on deep learning to determine a health management solution for non-drug intervention.
[0070] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A method for generating a multimodal health management plan based on deep learning, characterized in that: include: Obtaining blood pressure values, blood sugar values, heart rate values, blood oxygen saturation, body temperature values and respiratory rate of the analyzed client at multiple predetermined time points within a predetermined time period; Preprocessing the blood pressure values, blood glucose values, heart rate values, blood oxygen saturation, body temperature values and respiratory rates at the plurality of predetermined time points to obtain a blood pressure time series input vector, a blood glucose time series input vector, a heart rate time series input vector, a blood oxygen saturation time series input vector, a body temperature time series input vector and a respiratory rate time series input vector; Performing association analysis on the blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector, and the respiratory rate time series input vector to obtain a global vital sign semantic understanding feature vector of the client object; and Determine a health management plan for non-drug intervention based on the global physical sign semantic understanding feature vector of the client object; The method further includes a training step: training a vital sign data temporal feature extractor based on a one-dimensional convolutional neural network model, a vital sign data context encoder based on a converter module, and a classifier; Wherein, the training step includes: Acquire training data, wherein the training data includes training blood pressure values, training blood sugar values, training heart rate values, training blood oxygen saturation values, training body temperature values, and training respiratory rates of the analyzed client at multiple predetermined time points within a predetermined time period, as well as true values of non-drug intervention health management program labels; Arrange the training blood pressure values, training blood sugar values, training heart rate values, training blood oxygen saturation values, training body temperature values and training respiratory rates at the plurality of predetermined time points into a training blood pressure timing input vector, a training blood sugar timing input vector, a training heart rate timing input vector, a training blood oxygen saturation timing input vector, a training body temperature timing input vector and a training respiratory rate timing input vector according to the time dimension respectively; The training blood pressure time series input vector, the training blood sugar time series input vector, the training heart rate time series input vector, the training blood oxygen saturation time series input vector, the training body temperature time series input vector and the training respiratory rate time series input vector are passed through the vital sign data time series feature extractor based on the one-dimensional convolutional neural network model to obtain a training blood pressure time series feature vector, a training blood sugar time series feature vector, a training heart rate time series feature vector, a training blood oxygen saturation time series feature vector, a training body temperature time series feature vector and a training respiratory rate time series feature vector; Passing the training blood pressure time series feature vector, the training blood sugar time series feature vector, the training heart rate time series feature vector, the training blood oxygen saturation time series feature vector, the training body temperature time series feature vector and the training respiratory rate time series feature vector through the vital sign data context encoder based on the converter module to obtain a global vital sign semantic understanding feature vector for the training client object; Passing the training client object global physical sign semantic understanding feature vector through a classifier to obtain a classification loss function value; and The classification loss function value is used to train the vital sign data temporal feature extractor based on the one-dimensional convolutional neural network model, the vital sign data context encoder based on the converter module, and the classifier, wherein in each iteration of the training, the weight space exploration constraint iteration based on the class degree matrix regularization is performed on the global vital sign semantic understanding feature vector of the training customer object; Wherein, in each round of iteration of the training, the weight space exploration constraint iteration based on the regularization of the class degree matrix is performed on the global physical sign semantic understanding feature vector of the training customer object, including: performing the weight space exploration constraint iteration based on the regularization of the class degree matrix on the global physical sign semantic understanding feature vector of the training customer object according to the following optimization formula; Wherein, the optimization formula is: ; ; in, is the global physical sign semantic understanding feature vector of the training customer object, specifically expressed as a column vector, is a row vector, is the learnable domain transfer matrix, represents the weight matrix of the previous iteration, represents the transposed vector or matrix of a vector or matrix, is the weight matrix after iteration, Represents matrix multiplication.
2. The method for generating a multimodal health management program based on deep learning according to claim 1, characterized in that: The blood pressure values, blood sugar values, heart rate values, blood oxygen saturation values, body temperature values and respiratory rates at the plurality of predetermined time points are preprocessed to obtain a blood pressure time series input vector, a blood sugar time series input vector, a heart rate time series input vector, a blood oxygen saturation time series input vector, a body temperature time series input vector and a respiratory rate time series input vector, including: The blood pressure values, blood sugar values, heart rate values, blood oxygen saturation, body temperature values and respiratory rates at the multiple predetermined time points are arranged according to the time dimension as the blood pressure timing input vector, the blood sugar timing input vector, the heart rate timing input vector, the blood oxygen saturation timing input vector, the body temperature timing input vector and the respiratory rate timing input vector.
3. The method for generating a multimodal health management program based on deep learning according to claim 2, characterized in that: The blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector and the respiratory rate time series input vector are subjected to association analysis to obtain a global vital sign semantic understanding feature vector of the client object, including: Respectively extracting the vital sign time series features of the blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector and the respiratory rate time series input vector to obtain a blood pressure time series feature vector, a blood sugar time series feature vector, a heart rate time series feature vector, a blood oxygen saturation time series feature vector, a body temperature time series feature vector and a respiratory rate time series feature vector; and The correlation features among the blood pressure time series feature vector, the blood sugar time series feature vector, the heart rate time series feature vector, the blood oxygen saturation time series feature vector, the body temperature time series feature vector and the respiratory rate time series feature vector are extracted to obtain a global vital sign semantic understanding feature vector of the client object.
4. The method for generating a multimodal health management program based on deep learning according to claim 3, characterized in that: Extracting the vital sign time series features of the blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector and the respiratory rate time series input vector respectively to obtain a blood pressure time series feature vector, a blood sugar time series feature vector, a heart rate time series feature vector, a blood oxygen saturation time series feature vector, a body temperature time series feature vector and a respiratory rate time series feature vector, including: The blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector and the respiratory rate time series input vector are passed through a vital sign data time series feature extractor based on a one-dimensional convolutional neural network model to obtain the blood pressure time series feature vector, the blood sugar time series feature vector, the heart rate time series feature vector, the blood oxygen saturation time series feature vector, the body temperature time series feature vector and the respiratory rate time series feature vector.
5. The method for generating a multimodal health management program based on deep learning according to claim 4, characterized in that: Extracting correlation features among the blood pressure time series feature vector, the blood sugar time series feature vector, the heart rate time series feature vector, the blood oxygen saturation time series feature vector, the body temperature time series feature vector, and the respiratory rate time series feature vector to obtain a global vital sign semantic understanding feature vector of the client object, including: The blood pressure time series feature vector, the blood sugar time series feature vector, the heart rate time series feature vector, the blood oxygen saturation time series feature vector, the body temperature time series feature vector and the respiratory rate time series feature vector are passed through a vital sign data context encoder based on a converter module to obtain a global vital sign semantic understanding feature vector of the customer object.
6. The method for generating a multimodal health management program based on deep learning according to claim 5, characterized in that: Based on the global physical sign semantic understanding feature vector of the client object, a health management plan for non-drug intervention is determined, including: The global physical sign semantic understanding feature vector of the client object is passed through a classifier to obtain a classification result, and the classification result is used to represent a health management program label of non-drug intervention.
7. A multimodal health management program generation system based on deep learning, using the multimodal health management program generation method based on deep learning according to claim 1, characterized in that: include: A data acquisition module, used to acquire blood pressure values, blood sugar values, heart rate values, blood oxygen saturation, body temperature values and respiratory rate of the analyzed customer object at multiple predetermined time points within a predetermined time period; a data preprocessing module, configured to perform data preprocessing on the blood pressure values, blood glucose values, heart rate values, blood oxygen saturation, body temperature values and respiratory rates at the plurality of predetermined time points to obtain a blood pressure time series input vector, a blood glucose time series input vector, a heart rate time series input vector, a blood oxygen saturation time series input vector, a body temperature time series input vector and a respiratory rate time series input vector; an association analysis module, configured to perform association analysis on the blood pressure time series input vector, the blood sugar time series input vector, the heart rate time series input vector, the blood oxygen saturation time series input vector, the body temperature time series input vector, and the respiratory rate time series input vector to obtain a global vital sign semantic understanding feature vector of the client object; and The health management program determination module is used to determine a health management program of non-drug intervention based on the semantic understanding feature vector of the global physical signs of the client object.
8. The multimodal health management program generation system based on deep learning according to claim 7, characterized in that: The data preprocessing module is used to: The blood pressure values, blood sugar values, heart rate values, blood oxygen saturation, body temperature values and respiratory rates at the multiple predetermined time points are arranged according to the time dimension as the blood pressure timing input vector, the blood sugar timing input vector, the heart rate timing input vector, the blood oxygen saturation timing input vector, the body temperature timing input vector and the respiratory rate timing input vector.
