Multifunctional health monitoring wearable device based on Internet of Things

Abstract

The invention discloses a multifunctional health monitoring wearable device based on the Internet of Things, and provides more comprehensive health state analysis by integrating different types of data (physiology, environment, social contact and the like). Meanwhile, an adaptive fusion mechanism is adopted, the importance of different data sources is dynamically adjusted, and the adaptability and accuracy of the model are improved; real-time health risk early warning is provided in combination with real-time data of the wearable device; and through a user feedback mechanism, the model has self-learning and adaptive capabilities, and a personalized health management scheme is formed.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a multifunctional health monitoring wearable device based on the Internet of Things. Background Art

[0002] With the continuous advancement of science and technology and the improvement of people's living standards, health management has received increasing attention. Wearable technology, as an important means of modern health monitoring, is gradually being integrated into people's daily lives. Wearable devices integrate a variety of sensors to monitor users' physiological data in real time, such as heart rate, blood pressure, blood oxygen, body temperature, exercise volume, and sleep quality, providing users with a convenient means of health monitoring.

[0003] There are many wearable health monitoring devices on the market, such as smart watches, fitness trackers, and health wristbands. These devices usually have the following problems: 1. Single health indicator monitoring: Most existing devices can only monitor a single health parameter. For example, a heart rate monitor can only provide heart rate data, but lacks comprehensive monitoring of other health indicators. This singleness limits users' comprehensive understanding of their health status.

[0004] 2. Insufficient data processing and analysis capabilities: Although some wearable devices can collect a variety of health data, their built-in data processing capabilities are relatively limited and cannot perform in-depth analysis and trend prediction. Users can often only obtain raw data and lack in-depth health insights and personalized recommendations.

[0005] 3. Lack of intelligent health management: Traditional wearable devices generally lack intelligent health management functions. Users need to manually record and analyze their health data, making it difficult to achieve truly intelligent health management. In addition, existing devices do not adequately consider individual differences among users and are unable to provide targeted health advice.

[0006] 4. Information island problem: The data of many wearable devices cannot be seamlessly connected with other health management systems (such as electronic health records, medical service providers, etc.), resulting in the isolation of user health information and affecting the efficiency of overall health management. This also makes medical service providers lack comprehensive data support when formulating personalized treatment plans.

[0007] 5. Privacy and security issues: With the increase in health data collection, users are increasingly concerned about the privacy protection of their personal health information. Existing devices may have security risks during data transmission and storage, and users' health information may be easily leaked or abused.

[0008] Therefore, based on the existing technology, there is an urgent need for a wearable device that integrates multiple health monitoring functions, which can realize real-time transmission and analysis of data through Internet of Things technology, and combine deep learning algorithms to provide users with personalized health management solutions. It not only improves the monitoring accuracy and reliability of user health data, but also can meet the user's diverse needs for health monitoring, enhance the user's health management experience, and help users achieve more efficient health management goals. Summary of the invention

[0009] The purpose of the present invention is to provide a multifunctional health monitoring wearable device based on the Internet of Things to solve the above-mentioned problems existing in the prior art.

[0010] An embodiment of the present invention provides a multifunctional health monitoring wearable device based on the Internet of Things, wherein the multifunctional health monitoring wearable device based on the Internet of Things includes a processor; the processor is used to perform the following steps: Obtain wearable device data, questionnaire data, social media data and environmental data; wherein the wearable device data is used to represent the user's physiological state, the wearable device data includes multiple data, the questionnaire data is used to represent the user's living status; the social media data is used to represent the user's social status, and the environmental data is used to represent the user's environmental conditions; The user's health data features are jointly extracted based on wearable device data, questionnaire data, social media data and environmental data through a joint feature extraction model; wherein the joint feature extraction model includes a joint long short-term memory network, a fully connected neural network, a sentiment analysis network and a graph neural network, the input of the joint long short-term memory network is the wearable device data, the input of the fully connected neural network is the questionnaire data, the input of the sentiment analysis network is the social media data, and the input of the graph neural network is the environmental data; in the joint feature extraction model, the internal state value of the joint long short-term memory network is adjusted through the loss functions of the fully connected neural network, the sentiment analysis network and the graph neural network, so that the output of the joint long short-term memory network integrates the implicit features of the questionnaire data, the social media data and the environmental data; the output of the joint long short-term memory network is used as the user's health data feature; the health data feature is used to characterize the user's health status.

[0011] Optionally, the processor is further used to: evaluate the user's health status based on health data characteristics.

[0012] Optionally, the joint long short-term memory network includes multiple layers of long short-term memory networks; each layer of the long short-term memory network includes multiple neurons, each neuron is an LSTM network, and each neuron of each layer of the long short-term memory network is used to process different items of data in the wearable device data; there are two neurons corresponding to each other in two adjacent layers of the long short-term memory network; the corresponding neurons are strung together into a neuron chain; In the joint feature extraction model, the internal state value of the joint long short-term memory network is adjusted through the loss functions of the fully connected neural network, the sentiment analysis network and the graph neural network, which is specifically achieved in the following way: The input of the first layer of the long short-term memory network of the joint long short-term memory network is the wearable device data. Multiple neurons correspond one-to-one to multiple items of data. The internal state value of each neuron in the first layer of the long short-term memory network is determined by the loss function of the fully connected neural network, the sentiment analysis network, and the graph neural network. The g-th neuron input of the k-th layer LSTM of the joint LSTM is the fused input information, and the joint input information is obtained by fusing the output of the g-th neuron of the k-1-th layer LSTM and the outputs of other neurons in the k-1-th layer LSTM; the internal state value of each neuron in the k-th layer LSTM is determined by the loss functions of the fully connected neural network, the sentiment analysis network, and the graph neural network, as well as the loss function of the k-1-th layer LSTM. It is determined that the value of k is 2, 3, ..., N; N represents the number of layers of the long short-term memory network included in the joint long short-term memory network.

[0013] Optionally, the input of the first layer of the long short-term memory network of the joint long short-term memory network is the wearable device data, and the internal state value of each neuron in the first layer of the long short-term memory network is determined by the loss function of the fully connected neural network, the sentiment analysis network and the graph neural network, specifically: Obtain the loss function L1 of the fully connected neural network, the loss function L2 of the sentiment analysis network, and the loss function L3 of the graph neural network; The weighted items of L1, L2 and L3 are adjusted respectively through the attention mechanism; Based on the adjusted weighted items, L1, L2 and L3 are weighted and summed to obtain the influence index; The internal state value of the g-th neuron in the first layer of LSTM is determined by the following formula: in, represents the internal state value of the g-th neuron in the first layer of LSTM. represents the element-wise product of vectors, The g-th item of data in the wearable device data; The candidate state of the g-th neuron in the first layer of LSTM, represents the value of the input gate of the g-th neuron in the first layer of LSTM. represents the value of the forget gate of the gth neuron in the first layer of the long short-term memory network; the value of g is 1, 2, 3, ..., M; M is the number of multiple data included in the wearable device data, represents the learning rate, Represents the impact index.

[0014] Optionally, the input of the k-th layer of the long short-term memory network of the joint long short-term memory network is the output of the k-1-th layer of the long short-term memory network, and the internal state value of each neuron in the k-th layer of the long short-term memory network is determined by the loss function of the fully connected neural network, the sentiment analysis network and the graph neural network and the loss function of the k-1-th layer of the long short-term memory network. Determine, including: The internal state value of the g-th neuron in the k-th layer of the long short-term memory network is determined by the following formula: in, represents the internal state value of the g-th neuron in the k-th layer of the long short-term memory network, represents the element-wise product of vectors, is the output of the g-th neuron in the k-1-th layer of the long short-term memory network; is the influence index of the g-th neuron in the k-th layer LSTM network, which is used to represent the candidate state of the g-th neuron in the k-th layer LSTM network. represents the value of the input gate of the g-th neuron in the k-th layer of the long short-term memory network, Represents the value of the forget gate of the g-th neuron in the first layer of long short-term memory network.

[0015] Optionally, adjusting the weighted item of L1, the weighted item of L2, and the weighted item of L3 respectively through the attention mechanism includes: Calculate the attention score for each loss function: , is the i-th attention score, is the weight matrix, is the bias vector; Calculate the attention weights: , represents the jth attention score, Indicates the i-th Attention weight; the value of j is 1, 2, 3; With the first attention weight As the weighted term of L1, with the second attention weight As the weighted term of L2, with the third attention weight As the weighted term of L3.

[0016] Optionally, a weighted sum of L1, L2 and L3 is performed based on the adjusted weighted items to obtain an influence index, which is specifically obtained by the following formula: .

[0017] Optional, weight matrix and the bias vector Determined by: Initialize the weight matrix and bias vector: Sampling from a uniform distribution to obtain an initialized weight matrix , that is, the initialized weight matrix The value of is in the range of [−0.01, 0.01]; sample from the normal distribution to obtain the initialized bias vector , that is, the initialized bias vector The value of is in the range of [0, 0.01]; Update and train the weight matrix using the following formula and the bias vector : in, Indicates The weight matrix of the iteration, Indicates The weight matrix of the iteration is When , the weight matrix is , that is, the initialized weight matrix; is the weight gradient; No. The bias vector for the iteration, Indicates The bias vector of the iteration, when When , the bias vector is , that is, the initialization bias vector, is the bias gradient; where in, is the loss function vector The transposed vector of is the derivative gradient of the loss function; , is the loss function vector The cross entropy loss.

[0018] Optionally, the joint input information is obtained in the following manner: The output of the g-th neuron in the k-1-th layer of the LSTM is , the outputs of other neurons in the k-1th layer of the long short-term memory network are recorded as , The value of is 1, 2, ..., M-1; Calculate each k-1 layer of long short-term memory network Attention score: , is the sth in the k-1th layer of the LSTM network The attention score, is the weight matrix, is the bias vector; Calculate the attention weights: , represents the zth attention score in the k-1th layer of the long short-term memory network, represents the sth attention weight in the k-1th layer of the long short-term memory network; The joint input information is obtained by weighted fusion through the following formula: in, Represents joint input information.

[0019] Optionally, the training method of the joint feature extraction model is: Obtain a training data set, the training data set includes a wearable device data training set, a questionnaire data training set, a social media data training set and an environment data training set; the wearable device data training set includes a plurality of training wearable device data, the questionnaire data training set includes a plurality of training questionnaire data, the social media data training set includes a plurality of social media data; the environment data training set includes a plurality of training environment data; The wearable device data training set, questionnaire data training set, social media data training set and environmental data training set are respectively input into the joint long short-term memory network, fully connected neural network, sentiment analysis network and graph neural network; The features of the questionnaire survey data training set are extracted through a fully connected neural network to obtain the loss function L1 of the fully connected neural network; The feature extraction of social media data training set is performed through the sentiment analysis network to obtain the loss function L2 of the sentiment analysis network; The environmental data is feature extracted through the graph neural network to obtain the loss function L3 of the graph neural network; The training wearable device data in the wearable device data training set is input into the joint long short-term memory network, and the internal state value of each neuron in the joint long short-term memory network is calculated. The wearable device data training set is feature extracted by the joint long short-term memory network to obtain the overall loss function of the joint long short-term memory network. The overall loss function is calculated by the following formula: in, represents the overall loss function, Represents the cross entropy loss value of the output of the joint long short-term memory network; When the overall loss function When convergence occurs, stop training the joint long short-term memory network, fully connected neural network, sentiment analysis network, and graph neural network to obtain the trained joint long short-term memory network, fully connected neural network, sentiment analysis network, and graph neural network.

[0020] Optional, learning rate The method of determining is: Get the square of the norm of all weight matrices and the square of the norm of all bias vectors; Get the sum of the squares of the norms of all weight matrices and the squares of the norms of all bias vectors, denoted as ; The learning rate is calculated by the following formula : in, represents the sum of the loss function of the fully connected neural network, the loss function of the sentiment analysis network, and the loss function of the graph neural network at the current time t; represents the sum of the loss function of the fully connected neural network, the loss function of the sentiment analysis network, and the loss function L3 of the graph neural network at the previous time t-1; is the gradient of the loss function of the fully connected neural network, the loss function of the sentiment analysis network, and the loss function of the graph neural network before the current time t; is a small constant with a value of 0.001; is the attenuation factor, The value of is 0.9.

[0021] Compared with the prior art, the embodiments of the present invention achieve the following beneficial effects: The embodiment of the present invention also provides a multifunctional health monitoring wearable device based on the Internet of Things, which extracts the health data features of the user based on the wearable device data, questionnaire data, social media data and environmental data through a joint feature extraction model. In the joint feature extraction model, the internal state value of the joint long short-term memory network is adjusted through the loss function of the fully connected neural network, the sentiment analysis network and the graph neural network, so that the joint long short-term memory network can mine the influence of the questionnaire data, social media data and environmental data on the wearable device data, and mine the implicit information of the wearable device data, questionnaire data, social media data and environmental data, thereby improving the accuracy of the joint long short-term memory network feature extraction. The output of the joint long short-term memory network integrates the implicit features of the questionnaire data, social media data and environmental data. The health data features can integrate various influencing factors to comprehensively and comprehensively represent the health status of the user, thereby improving the accuracy of the health data features in representing the health status of the user. On this basis, the health status of the user is evaluated based on the health data features, which can improve the accuracy and reliability of the user health status data evaluation. In detail, through multimodal fusion, multiple data sources are integrated to provide a more comprehensive health analysis. It integrates the adaptive attention mechanism to dynamically adjust the importance of the data source and optimize the prediction according to the user's health status and environmental changes. Combined with the emotional state analysis, the user's mental health status is captured through the sentiment analysis of social media data, so as to more comprehensively evaluate health. At the same time, the learning ability of the model is increased. The model can continuously learn from user feedback and gradually improve the accuracy and personalization of the prediction. In addition, the dynamic adjustment of the importance of the data source involves the combined use of multiple methods, including attention mechanism, user feedback, incremental learning, correlation analysis, performance monitoring and environmental adaptability, which can help the model respond to changes in the user's health status more flexibly and intelligently, thereby improving the accuracy and personalization of health assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flow chart of a health data monitoring and evaluation method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The present invention will be described in detail below in conjunction with the accompanying drawings.

[0024] Example 1 In order to solve the above technical problems, an embodiment of the present invention provides a multifunctional health monitoring wearable device based on the Internet of Things, which can also be called a multifunctional health data monitoring wearable device based on the Internet of Things.

[0025] The wearable device of the present invention adopts a wristband design. After the user wears it, the device automatically starts and starts monitoring. Taking heart rate and blood oxygen monitoring as an example, the device records heart rate and blood oxygen data every second and transmits the data to the cloud in encrypted form. In the cloud, the data is pre-processed and input into the deep learning model for analysis. If a continuous increase in heart rate is detected and accompanied by low blood oxygen saturation, the system will automatically notify the user and suggest that he or she rest immediately or consult a doctor. The user's mobile application interface is updated at the same time, showing changes in health status and suggestions.

[0026] Optionally, an embodiment of the present invention provides a multifunctional health monitoring wearable device based on the Internet of Things, including a data acquisition module and a processor. The data acquisition module is used to obtain wearable device data, questionnaire data, social media data and environmental data. In an embodiment of the present invention, the wearable device data is used to represent the physiological state of the user, the wearable device data includes multiple data, the questionnaire data is used to represent the user's living status; the social media data is used to represent the user's social status, and the environmental data is used to represent the user's environmental conditions. Specifically, the multiple data included in the wearable device data may include blood pressure, heart rate, number of steps and sleep quality score, etc. The wearable device data is time series data, and each wearable device data has a marked timestamp. The questionnaire data includes living habit data and psychological state data, etc. The questionnaire data is structured data, and the questionnaire data includes multiple feature fields. The social media data includes emotional state data and social interaction data, and the social media data is unstructured text data. The environmental data includes air quality data, weather information data and noise level data, etc., and the social media data is structured data.

[0027] Wearable device data can be collected through sensors, such as blood pressure monitors, pedometers, etc., and then transmitted to the data collection module. Questionnaire survey data can be input through the touch screen and then transmitted to the data collection module. Social media data can be input through the touch screen and then transmitted to the data collection module, or it can be connected to the user's social media account through wireless communication, and the social media data can be transmitted to the data collection module through the social media account. Environmental data can be collected through air quality collection equipment, weather monitoring equipment, decibel meters and other equipment, or it can be input on the touch screen and then transmitted to the data collection module.

[0028] After the data collection module collects the wearable device data, questionnaire data, social media data and environmental data, the wearable device data, questionnaire data, social media data and environmental data are sent to the processor. The processor implements its functions by executing the steps of the following health data monitoring and evaluation method, such as Figure 1 As shown, the processor is used to perform the following steps: S101: Obtain wearable device data, questionnaire data, social media data, and environmental data.

[0029] S102: Extracting health data features of the user based on the wearable device data, the questionnaire data, the social media data and the environmental data through a joint feature extraction model. The health data features are used to characterize the health status of the user.

[0030] In step S102, the joint feature extraction model includes a joint long short-term memory network, a fully connected neural network, a sentiment analysis network and a graph neural network, the input of the joint long short-term memory network is wearable device data, the input of the fully connected neural network is questionnaire data, the input of the sentiment analysis network is social media data, and the input of the graph neural network is environmental data. In the joint feature extraction model, the internal state value of the joint long short-term memory network is adjusted through the loss functions of the fully connected neural network, the sentiment analysis network and the graph neural network, so that the output of the joint long short-term memory network integrates the implicit features of the questionnaire data, social media data and environmental data. The output of the joint long short-term memory network is used as the health data feature of the user.

[0031] As an optional implementation manner, the processor is further configured to perform the following step S103: S103: Evaluate the user's health status based on the health data characteristics.

[0032] In the embodiment of the present invention, the internal state value of the joint long short-term memory network is adjusted through the loss function of the fully connected neural network, the sentiment analysis network and the graph neural network, so that the output of the joint long short-term memory network integrates the implicit features of the questionnaire data, social media data and environmental data to improve the accuracy and reliability of feature extraction. The main scheme is: the joint long short-term memory network includes multiple layers of long short-term memory networks, each layer of long short-term memory networks includes multiple neurons, each neuron is a long short-term memory network, recorded as LSTM network (Long Short-Term Memory Network, LSTM), each neuron of each layer of long short-term memory network is used to process different items of data in the wearable device data, and there are two neurons corresponding to each other in two adjacent layers of long short-term memory networks, and the corresponding neurons are strung into a neuron chain. On the basis of this structure, according to the step, in the joint feature extraction model, the internal state value of the joint long short-term memory network is adjusted through the loss function of the fully connected neural network, the sentiment analysis network and the graph neural network, which is specifically achieved in the following way: The input of the first layer of the joint long short-term memory network is the wearable device data. Multiple neurons correspond one-to-one to multiple items of data. The internal state value of each neuron in the first layer of the long short-term memory network is determined by the loss function of the fully connected neural network, sentiment analysis network and graph neural network.

[0033] The g-th neuron input of the k-th layer LSTM of the joint LSTM is the fused input information, and the joint input information is obtained by fusing the output of the g-th neuron of the k-1-th layer LSTM and the outputs of other neurons in the k-1-th layer LSTM; the internal state value of each neuron in the k-th layer LSTM is determined by the loss functions of the fully connected neural network, the sentiment analysis network, and the graph neural network, as well as the loss function of the k-1-th layer LSTM. It is determined that the value of k is 2, 3, ..., N. N represents the number of layers of the long short-term memory network included in the joint long short-term memory network. The value of g is 1, 2, 3, ..., M; M is the number of multiple data included in the wearable device data.

[0034] The input of the first layer of the LSTM network of the joint LSTM network is the wearable device data. The internal state value of each neuron in the first layer of the LSTM network is determined by the loss function of the fully connected neural network, the sentiment analysis network, and the graph neural network, specifically: Obtain the loss function L1 of the fully connected neural network, the loss function L2 of the sentiment analysis network, and the loss function L3 of the graph neural network.

[0035] The attention mechanism is used to adjust the weighted items of L1, L2 and L3 respectively. Specifically: Calculate the attention score for each loss function: , is the i-th attention score, is the weight matrix, is the bias vector.

[0036] Calculate the attention weights: , represents the jth attention score, Indicates the i-th Attention weight; the value of j is 1, 2, 3.

[0037] With the first attention weight As the weighted term of L1, with the second attention weight As the weighted term of L2, with the third attention weight As the weighted term of L3.

[0038] In this way, the model's weights can be effectively adjusted based on the characteristics of new data and user feedback to improve its predictive performance and user satisfaction.

[0039] Then, based on the adjusted weighted items, L1, L2 and L3 are weighted and summed to obtain the influence index, which is specifically calculated by the following formula: . Represents the impact index.

[0040] The internal state value of the g-th neuron in the first layer of LSTM is determined by the following formula: in, represents the internal state value of the g-th neuron in the first layer of LSTM. represents the element-wise product of vectors, The g-th item of data in the wearable device data; The candidate state of the g-th neuron in the first layer of LSTM, represents the value of the input gate of the g-th neuron in the first layer of LSTM. Represents the value of the forget gate of the g-th neuron in the first layer of long short-term memory network; Represents the learning rate.

[0041] Based on the above scheme, the input of the k-th layer LSTM of the joint LSTM is the output of the k-1-th layer LSTM, and the internal state value of each neuron in the k-th layer LSTM is determined by the loss functions of the fully connected neural network, the sentiment analysis network, and the graph neural network, as well as the loss function of the k-1-th layer LSTM. Determine, including: The internal state value of the g-th neuron in the k-th layer of the long short-term memory network is determined by the following formula: in, represents the internal state value of the g-th neuron in the k-th layer of the long short-term memory network, represents the element-wise product of vectors, is the output of the g-th neuron in the k-1-th layer of the long short-term memory network; is the candidate state of the g-th neuron in the k-th layer of the long short-term memory network, represents the value of the input gate of the g-th neuron in the k-th layer of the long short-term memory network, Represents the value of the forget gate of the g-th neuron in the first layer of long short-term memory network.

[0042] In the embodiment of the present invention, the weight matrix and the bias vector Determined by: Initialize the weight matrix and bias vector: Sampling from a uniform distribution to obtain an initialized weight matrix , that is, the initialized weight matrix The value of is in the range of [−0.01, 0.01]. Sampling from a normal distribution to obtain the initialized bias vector , that is, the initialized bias vector The value of is in the range of [0, 0.01].

[0043] Update and train the weight matrix using the following formula and the bias vector : in, Indicates The weight matrix of the iteration, Indicates The weight matrix of the iteration is When , the weight matrix is , which is the initialized weight matrix. is the weight gradient; No. The bias vector for the iteration, Indicates The bias vector of the iteration, when When , the bias vector is , that is, the initialization bias vector, is the bias gradient; where in, is the loss function vector The transposed vector of is the derivative gradient of the loss function; , is the loss function vector The cross entropy loss value of .

[0044] By adopting the above scheme, the initialization of the weight matrix and bias vector is very important. Reasonable initialization can accelerate convergence and improve model performance.

[0045] Optionally, the joint input information is obtained in the following manner: The output of the g-th neuron in the k-1-th layer of the LSTM is , the outputs of other neurons in the k-1th layer of the long short-term memory network are recorded as , The value of is 1, 2,…, M-1.

[0046] Calculate each k-1 layer of long short-term memory network Attention score: , is the sth in the k-1th layer of the LSTM network The attention score, is the weight matrix, is the bias vector.

[0047] Calculate the attention weights: , represents the zth attention score in the k-1th layer of the long short-term memory network, Represents the sth attention weight in the k-1th layer of the long short-term memory network.

[0048] The joint input information is obtained by weighted fusion through the following formula: in, Represents joint input information.

[0049] By introducing the attention mechanism into LSTM, fully connected neural networks, and graph neural networks, we can improve the expressiveness of the model, enabling it to focus more accurately on the key information in the input data. In the design of the loss function, the regularization term combined with the attention weight can effectively control the complexity of the model and improve the generalization ability of the model.

[0050] In this embodiment of the present invention, the learning rate The method of determining is: Obtain the square of the norm of all weight matrices and the square of the norm of all bias vectors. In the embodiment of the present invention, the 1-norm (L1 Norm) is used. The 1-norm is a norm of a vector or a matrix.

[0051] Get the sum of the squares of the norms of all weight matrices and the squares of the norms of all bias vectors, denoted as .

[0052] The learning rate is calculated by the following formula : in, represents the sum of the loss function of the fully connected neural network, the loss function of the sentiment analysis network, and the loss function of the graph neural network at the current time t; represents the sum of the loss function of the fully connected neural network, the loss function of the sentiment analysis network, and the loss function L3 of the graph neural network at the previous time t-1; is the gradient of the loss function of the fully connected neural network, the loss function of the sentiment analysis network, and the loss function of the graph neural network before the current time t. is a small constant with a value of 0.001; is the attenuation factor, The value of is 0.9.

[0053] By adopting the above technical solutions, multiple data sources are integrated through multimodal fusion to provide more comprehensive health analysis. The adaptive attention mechanism is integrated to dynamically adjust the importance of the data source and optimize the prediction according to the user's health status and environmental changes. Combined with emotional state analysis, the user's mental health status is captured through sentiment analysis of social media data, so as to more comprehensively evaluate health, while increasing the learning ability of the model. The model can continuously learn from user feedback and gradually improve the accuracy and personalization of predictions. In addition, dynamically adjusting the importance of data sources involves the combined use of multiple methods, including attention mechanisms, user feedback, incremental learning, correlation analysis, performance monitoring, and environmental adaptability, which can help the model respond to changes in user health status more flexibly and intelligently, thereby improving the accuracy and personalization of health detection.

[0054] Through the above method, the joint long short-term memory network and the working mode of the joint long short-term memory network are introduced. Next, the training method of the joint feature extraction model is introduced.

[0055] In an embodiment of the present invention, the joint feature extraction model method is: A training data set is obtained, which includes a wearable device data training set, a questionnaire data training set, a social media data training set and an environmental data training set; the wearable device data training set includes multiple training wearable device data, the questionnaire data training set includes multiple training questionnaire data, the social media data training set includes multiple social media data; the environmental data training set includes multiple training environment data; the training data in the training data set is labeled.

[0056] The wearable device data training set, questionnaire data training set, social media data training set and environmental data training set are respectively input into the joint long short-term memory network, fully connected neural network, sentiment analysis network and graph neural network.

[0057] The features of the questionnaire survey data training set are extracted through a fully connected neural network to obtain the loss function L1 of the fully connected neural network.

[0058] The sentiment analysis network is used to extract features from the social media data training set and the loss function L2 of the sentiment analysis network is obtained.

[0059] The environmental data is feature extracted through the graph neural network to obtain the loss function L3 of the graph neural network; The training wearable device data in the wearable device data training set is input into the joint long short-term memory network, and the internal state value of each neuron in the joint long short-term memory network is calculated. The wearable device data training set is feature extracted by the joint long short-term memory network to obtain the overall loss function of the joint long short-term memory network. The overall loss function is calculated by the following formula: in, represents the overall loss function, Represents the cross entropy loss value of the output of the joint long short-term memory network.

[0060] When the overall loss function When convergence occurs, stop training the joint long short-term memory network, fully connected neural network, sentiment analysis network, and graph neural network to obtain the trained joint long short-term memory network, fully connected neural network, sentiment analysis network, and graph neural network.

[0061] In this way, the joint long short-term memory network, fully connected neural network, sentiment analysis network and graph neural network are jointly trained, which improves the accuracy, reliability and robustness of the feature extraction of each of the joint long short-term memory network, fully connected neural network, sentiment analysis network and graph neural network.

[0062] After the joint feature extraction model is trained, the joint features of wearable device data, questionnaire data, social media data and environmental data can be extracted through the joint feature extraction model. Specifically, the joint feature extraction model is used to jointly extract the user's health data features based on wearable device data, questionnaire data, social media data and environmental data, including: The wearable device data, questionnaire data, social media data and environmental data are respectively input into the joint long short-term memory network, the fully connected neural network, the sentiment analysis network and the graph neural network. The features of the questionnaire data, social media data and environmental data are extracted by the fully connected neural network, the sentiment analysis network and the graph neural network, and the loss functions L1, L2 and L3 of the fully connected neural network, the sentiment analysis network and the graph neural network are obtained in real time.

[0063] Based on the loss functions L1, L2 and L3 of the fully connected neural network, sentiment analysis network and graph neural network, the internal state value of each neuron in the joint long short-term memory network is obtained.

[0064] The joint long short-term memory network extracts the user's health data features based on the internal state value and the wearable device data. That is, the internal state value replaces the internal state of each neuron in the joint long short-term memory network, and then the features of the wearable device data are extracted through the joint long short-term memory network to obtain the user's health data features.

[0065] In order to more clearly introduce how the joint long short-term memory network extracts the user's health data features based on the internal state value and wearable device data, the working process of each neuron LSTM network in the joint long short-term memory network is as follows: The LSTM network structure includes three gating mechanisms: input gate , output gate and the forget gate The calculation formulas for the three gates of the LSTM network structure are as follows: (1) (2) (3) in, represents the Logistic function, t represents the current time, is the state output at the previous moment, is the input at the current moment. In the embodiment of the present invention, for the neurons in the first layer of the long short-term memory network, Represents one of the multiple data of the wearable device, such as blood pressure, heart rate, number of steps, or sleep quality score. For the neurons in the k-1th layer of the long short-term memory network, This is the combined input information mentioned above. That is to say, for different neurons, the The data represented are different, but they are all one of the multiple items of wearable device data. Indicates the input gate The weight matrix of Indicates the input gate The weight matrix of Represents the bias vector of the input gate. Indicates the output gate The weight matrix of Indicates the output gate The weight matrix of Represents the bias vector of the output gate. Indicates the forgotten gate The weight matrix of Indicates the forgotten gate The weight matrix of Represents the bias vector of the forget gate. In the embodiment of the present invention, the weight matrix can also be obtained by the above method: and the bias vector Get , , , , , , , , . Optional, , , , , , It can also be randomly generated in the range of [−0.01, 0.01], , , It can also be randomly generated in the range [0, 0.01].

[0066] In the LSTM network, candidate states are introduced , that is, convert the input, and the calculation method is as follows: Mid-term, represents the candidate state of the LSTM network at time t, Table candidate status The weight matrix of Indicates candidate status The weight matrix of Represents the bias vector under the candidate state. , can be randomly generated in the range [−0.01, 0.01], Can be randomly generated in the range [0, 0.01].

[0067] The candidate state is obtained through a nonlinear activation function.

[0068] Then, the internal state value of the gth neuron in the first layer of LSTM is determined by the following formula: The internal state value of the g-th neuron in the k-th layer of the long short-term memory network is determined by the following formula: The output state of each neuron LSTM network for: ,in, for The hyperbolic tangent function of .

[0069] In the embodiment of the present invention, for S103, the health status of the user is evaluated based on the health data features, and specifically, a pre-trained convolutional neural network is used to evaluate the health status of the user based on the health data features, specifically: The training data is collected. The training data includes multiple training health data features. Each health data feature carries a health status label, indicating the type of health status, such as healthy, sub-healthy, and with health risks. Different health statuses can also be represented by different numbers.

[0070] Multiple training health data features are input into the convolutional neural network to train the convolutional neural network's health assessment capability for the training health data features.

[0071] When the loss function of the convolutional neural network converges, it is determined that the convolutional neural network is well trained, and the training of the convolutional neural network is stopped to obtain a trained convolutional neural network. At this time, the convolutional neural network can accurately evaluate the user's health status based on the health data features, that is, output the health status type corresponding to the health data features.

[0072] In summary, the technical solution adopted in the embodiment of the present invention provides a more comprehensive health status analysis by integrating different types of data (physiological, environmental, social, etc.). At the same time, an adaptive fusion mechanism is adopted to dynamically adjust the importance of different data sources to improve the adaptability and accuracy of the model. Combined with the real-time data of wearable devices, it provides instant health risk warnings. Through the user feedback mechanism, the model has the ability to self-learn and adapt, forming a personalized health management plan.

[0073] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description of the above specific languages ​​is for disclosing the best mode of the present invention.

[0074] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

Claims

1. A multifunctional health monitoring wearable device based on the Internet of Things, characterized in that: The multifunctional health monitoring wearable device based on the Internet of Things includes a processor; the processor is used to perform the following steps: Obtain wearable device data, questionnaire data, social media data and environmental data; wherein the wearable device data is used to represent the user's physiological state, the wearable device data includes multiple data, the questionnaire data is used to represent the user's living status; the social media data is used to represent the user's social status, and the environmental data is used to represent the user's environmental conditions; The user's health data features are jointly extracted based on wearable device data, questionnaire data, social media data and environmental data through a joint feature extraction model; wherein the joint feature extraction model includes a joint long short-term memory network, a fully connected neural network, a sentiment analysis network and a graph neural network, the input of the joint long short-term memory network is the wearable device data, the input of the fully connected neural network is the questionnaire data, the input of the sentiment analysis network is the social media data, and the input of the graph neural network is the environmental data; in the joint feature extraction model, the internal state value of the joint long short-term memory network is adjusted through the loss functions of the fully connected neural network, the sentiment analysis network and the graph neural network, so that the output of the joint long short-term memory network integrates the implicit features of the questionnaire data, the social media data and the environmental data; the output of the joint long short-term memory network is used as the user's health data feature; the health data feature is used to characterize the user's health status.

2. The multifunctional health monitoring wearable device based on the Internet of Things according to claim 1 is characterized in that: The joint long short-term memory network includes multiple layers of long short-term memory networks; each layer of the long short-term memory network includes multiple neurons, each neuron is an LSTM network, and each neuron of each layer of the long short-term memory network is used to process different items of data in the wearable device data; there are two neurons corresponding to each other in two adjacent layers of the long short-term memory network; the corresponding neurons are strung together into a neuron chain; In the joint feature extraction model, the internal state value of the joint long short-term memory network is adjusted through the loss functions of the fully connected neural network, the sentiment analysis network and the graph neural network, which is specifically achieved in the following way: The input of the first layer of the long short-term memory network of the joint long short-term memory network is the wearable device data. Multiple neurons correspond one-to-one to multiple items of data. The internal state value of each neuron in the first layer of the long short-term memory network is determined by the loss function of the fully connected neural network, the sentiment analysis network, and the graph neural network. The g-th neuron input of the k-th layer LSTM of the joint LSTM is the fused input information, and the joint input information is obtained by fusing the output of the g-th neuron of the k-1-th layer LSTM and the outputs of other neurons in the k-1-th layer LSTM; the internal state value of each neuron in the k-th layer LSTM is determined by the loss functions of the fully connected neural network, the sentiment analysis network, and the graph neural network, as well as the loss function of the k-1-th layer LSTM. It is determined that the value of k is 2, 3, ..., N; N represents the number of layers of the long short-term memory network included in the joint long short-term memory network.

3. The multifunctional health monitoring wearable device based on the Internet of Things according to claim 2 is characterized in that: The input of the first layer of the long short-term memory network of the joint long short-term memory network is the wearable device data. The internal state value of each neuron in the first layer of the long short-term memory network is determined by the loss function of the fully connected neural network, the sentiment analysis network and the graph neural network, specifically: Obtain the loss function L1 of the fully connected neural network, the loss function L2 of the sentiment analysis network, and the loss function L3 of the graph neural network; The weighted items of L1, L2 and L3 are adjusted respectively through the attention mechanism; Based on the adjusted weighted items, L1, L2 and L3 are weighted and summed to obtain the influence index; The internal state value of the g-th neuron in the first layer of LSTM is determined by the following formula: in, represents the internal state value of the g-th neuron in the first layer of LSTM. represents the element-wise product of vectors, The g-th item of data in the wearable device data; The candidate state of the g-th neuron in the first layer of LSTM, represents the value of the input gate of the g-th neuron in the first layer of LSTM. represents the value of the forget gate of the gth neuron in the first layer of the long short-term memory network; the value of g is 1, 2, 3, ..., M; M is the number of multiple data included in the wearable device data, represents the learning rate, Represents the impact index.

4. The multifunctional health monitoring wearable device based on the Internet of Things according to claim 3 is characterized in that: The input of the k-th layer of the joint long short-term memory network is the output of the k-1-th layer of the long short-term memory network, and the internal state value of each neuron in the k-th layer of the long short-term memory network is determined by the loss function of the fully connected neural network, the sentiment analysis network and the graph neural network and the loss function of the k-1-th layer of the long short-term memory network. Determine, including: The internal state value of the g-th neuron in the k-th layer of the long short-term memory network is determined by the following formula: in, represents the internal state value of the g-th neuron in the k-th layer of the long short-term memory network, represents the element-wise product of vectors, is the output of the g-th neuron in the k-1-th layer of the long short-term memory network; is the influence index of the g-th neuron in the k-th layer LSTM network, which is used to represent the candidate state of the g-th neuron in the k-th layer LSTM network. represents the value of the input gate of the g-th neuron in the k-th layer of the long short-term memory network, Represents the value of the forget gate of the g-th neuron in the first layer of long short-term memory network.

5. The multifunctional health monitoring wearable device based on the Internet of Things according to claim 4 is characterized in that: The weighted items of L1, L2 and L3 are adjusted respectively by the attention mechanism, including: Calculate the attention score for each loss function: , is the i-th attention score, is the weight matrix, is the bias vector; Calculate the attention weights: , represents the jth attention score, Indicates the i-th Attention weight; the value of j is 1, 2, 3; With the first attention weight As the weighted term of L1, with the second attention weight As the weighted term of L2, with the third attention weight As the weighted term of L3.

6. The multifunctional health monitoring wearable device based on the Internet of Things according to claim 5 is characterized in that: Based on the adjusted weighted items, L1, L2 and L3 are weighted and summed to obtain the influence index, which is specifically obtained by the following formula: 。 7. The multifunctional health monitoring wearable device based on the Internet of Things according to claim 5, characterized in that: Weight Matrix and the bias vector Determined by: Initialize the weight matrix and bias vector: Sampling from a uniform distribution to obtain an initialized weight matrix , that is, the initialized weight matrix The value of is in the range of [−0.01, 0.01]; sample from the normal distribution to obtain the initialized bias vector , that is, the initialized bias vector The value of is in the range of [0, 0.01]; Update and train the weight matrix using the following formula and the bias vector : in, Indicates The weight matrix of the iteration, Indicates The weight matrix of the iteration is When , the weight matrix is , that is, the initialized weight matrix; is the weight gradient; No. The bias vector for the iteration, Indicates The bias vector of the iteration, when When , the bias vector is , that is, the initialization bias vector, is the bias gradient; where in, is the loss function vector The transposed vector of is the derivative gradient of the loss function; , is the loss function vector The cross entropy loss.

8. The multifunctional wearable health monitoring device based on the Internet of Things according to claim 7 is characterized in that: The method for obtaining the joint input information includes: The output of the g-th neuron in the k-1-th layer of the LSTM is , the outputs of other neurons in the k-1th layer of the long short-term memory network are recorded as , The value of is 1, 2, ..., M-1; Calculate each k-1 layer of long short-term memory network Attention score: ， is the sth in the k-1th layer of the LSTM network The attention score, is the weight matrix, is the bias vector; Calculate the attention weights: , represents the zth attention score in the k-1th layer of the long short-term memory network, represents the sth attention weight in the k-1th layer of the long short-term memory network; The joint input information is obtained by weighted fusion through the following formula: in, Represents joint input information.

9. The multifunctional wearable health monitoring device based on the Internet of Things according to claim 8, characterized in that: The training method of the joint feature extraction model is: Obtain a training data set, the training data set includes a wearable device data training set, a questionnaire data training set, a social media data training set and an environment data training set; the wearable device data training set includes a plurality of training wearable device data, the questionnaire data training set includes a plurality of training questionnaire data, the social media data training set includes a plurality of social media data; the environment data training set includes a plurality of training environment data; The wearable device data training set, questionnaire data training set, social media data training set and environmental data training set are respectively input into the joint long short-term memory network, fully connected neural network, sentiment analysis network and graph neural network; The features of the questionnaire survey data training set are extracted through a fully connected neural network to obtain the loss function L1 of the fully connected neural network; The feature extraction of social media data training set is performed through the sentiment analysis network to obtain the loss function L2 of the sentiment analysis network; The environmental data is feature extracted through the graph neural network to obtain the loss function L3 of the graph neural network; The training wearable device data in the wearable device data training set is input into the joint long short-term memory network, and the internal state value of each neuron in the joint long short-term memory network is calculated. The wearable device data training set is feature extracted by the joint long short-term memory network to obtain the overall loss function of the joint long short-term memory network. The overall loss function is calculated by the following formula: in, represents the overall loss function, Represents the cross entropy loss value of the output of the joint long short-term memory network; When the overall loss function When convergence occurs, stop training the joint long short-term memory network, fully connected neural network, sentiment analysis network, and graph neural network to obtain the trained joint long short-term memory network, fully connected neural network, sentiment analysis network, and graph neural network.

10. The multifunctional wearable health monitoring device based on the Internet of Things according to claim 9, characterized in that: Learning Rate The method of determining is: Get the square of the norm of all weight matrices and the square of the norm of all bias vectors; Get the sum of the squares of the norms of all weight matrices and the squares of the norms of all bias vectors, denoted as ; The learning rate is calculated by the following formula : in, represents the sum of the loss function of the fully connected neural network, the loss function of the sentiment analysis network, and the loss function of the graph neural network at the current time t; represents the sum of the loss function of the fully connected neural network, the loss function of the sentiment analysis network, and the loss function L3 of the graph neural network at the previous time t-1; is the gradient of the loss function of the fully connected neural network, the loss function of the sentiment analysis network, and the loss function of the graph neural network before the current time t; is a small constant with a value of 0.001; is the attenuation factor, The value of is 0.9.

Images