Life safety monitoring system and method based on intelligent wearable device

Abstract

The present application relates to the field of intelligent monitoring, and specifically discloses a life safety monitoring system and method based on intelligent wearable devices. The system mines the correlation feature distribution information between the temporal dynamic correlation features of each physiological data item of the wearer's physiological parameter data and the temporal dynamic correlation features of each environmental parameter item of the environmental parameter data by adopting a neural network model based on deep learning, thereby comprehensively monitoring human life safety by utilizing the temporal correlation features between in vitro environmental parameter information and human physiological parameter information, thereby improving the accuracy of life safety monitoring of intelligent wearable devices.

Description

Technical Field

[0001] The present application relates to the field of intelligent monitoring, and more specifically, to a life safety monitoring system and method based on intelligent wearable devices. Background Art

[0002] With the improvement of living standards, people are paying more and more attention to life safety and health. Common physiological parameters of the human body, such as blood pressure, heart rate, and pulse, can directly or indirectly reflect the health status of the human body. At the same time, the environmental conditions in which people live also have a certain impact on their life safety and health.

[0003] Currently, smart bracelets on the market can use three-axis accelerometers to determine basic data from human movement. This data is then combined with the wearer's personal vital signs and, using specific algorithms, produces personalized monitoring data such as steps, distance, and calories burned. They can also monitor sleep using the three-axis accelerometer and heart rate using optical sensors. However, these existing smart bracelets fail to consider the impact of the surrounding environment on human health when monitoring vital signs, and are unable to monitor the wearer's surroundings and condition in real time.

[0004] Therefore, an optimized life safety monitoring system based on smart wearable devices is desired. Summary of the Invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a life safety monitoring system and method based on smart wearable devices, which uses a neural network model based on deep learning to mine the correlation feature distribution information between the temporal dynamic correlation features of each physiological data item of the wearer's physiological parameter data and the temporal dynamic correlation features of each environmental data item of environmental parameter data, thereby comprehensively monitoring human life safety by utilizing the temporal correlation features between in vitro environmental parameter information and human physiological parameter information, thereby improving the accuracy of life safety monitoring of smart wearable devices.

[0006] According to one aspect of the present application, a life safety monitoring system based on a smart wearable device is provided, comprising:

[0007] a data acquisition module, configured to acquire physiological parameter data and environmental parameter data at multiple predetermined time points within a predetermined time period, collected by a smart wearable device worn on a subject to be monitored, wherein the physiological parameter data includes body temperature, movement status, movement speed, heart rate, pulse, and blood pressure; and the environmental parameter data includes temperature, humidity, air pressure, ultraviolet intensity, and harmful gas concentration;

[0008] a data association module, configured to pass the physiological parameter data and the environmental parameter data at each predetermined time point through a deep neural network model comprising a plurality of fully connected layers to obtain a plurality of physiological depth feature vectors and a plurality of environmental depth feature vectors;

[0009] a data timing change module, configured to arrange the multiple physiological depth feature vectors and the multiple environmental depth feature vectors into a physiological full-time-series feature matrix and an environmental full-time-series feature matrix, and then respectively pass them through a dual-branch network structure including a first convolutional neural network model and a second convolutional neural network model to obtain a physiological timing association feature matrix and an environmental timing association feature matrix, wherein the first convolutional neural network model and the second convolutional neural network model use two-dimensional convolution kernels with different scales;

[0010] a data feature fusion module, configured to fuse the physiological time series correlation feature matrix and the environmental time series correlation feature matrix to obtain a parameter correlation feature matrix;

[0011] an association coding module for passing the parameter association feature matrix through a convolutional neural network model as a filter to obtain a classification feature vector;

[0012] The health monitoring module is used to pass the classification feature vector through a classifier to obtain a classification result, and the classification result is used to indicate whether the life and health status of the monitored object is good.

[0013] In the above-mentioned life safety monitoring system based on smart wearable devices, the data association module is used to: use the fully connected layer of the deep neural network model to fully connect the physiological parameter data of each predetermined time point using the following formula to obtain the multiple physiological depth feature vectors, wherein the formula is: ,in is the physiological parameter data at each predetermined time point, is the output vector, is the weight matrix, is the bias vector, represents matrix multiplication; and, using the fully connected layer of the deep neural network model to fully connect encode the environmental parameter data using the following formula to obtain the multiple environmental depth feature vectors, wherein the formula is: ,in is the environmental parameter data, is the output vector, is the weight matrix, is the bias vector, Represents matrix multiplication.

[0014] In the above-mentioned life safety monitoring system based on smart wearable devices, the data timing change module is used to: use each layer of the dual-branch network structure including the first convolutional neural network model and the second convolutional neural network model to perform the following on the input data in the forward pass of the layer: convolution processing on the input data to obtain a convolution feature map; pooling the convolution feature map along the channel dimension to obtain a pooled feature map; and, nonlinear activation on the pooled feature map to obtain an activation feature map; wherein, the output of the last layer of the dual-branch network structure including the first convolutional neural network model and the second convolutional neural network model is the physiological timing association feature matrix and the environmental timing association feature matrix, and the input of the first layer of the dual-branch network structure including the first convolutional neural network model and the second convolutional neural network model is the physiological full timing feature matrix and the environmental full timing feature matrix.

[0015] In the above-mentioned life safety monitoring system based on smart wearable devices, the data feature fusion module is used to fuse the physiological time series correlation feature matrix and the environmental time series correlation feature matrix using the following formula to obtain a parameter correlation feature matrix; wherein the formula is:

[0016]

[0017] in, represents the physiological time series correlation feature matrix, represents the environmental temporal correlation feature matrix, Represents a cascade function, Represents the parameter association feature matrix.

[0018] In the above-mentioned life safety monitoring system based on smart wearable devices, the association coding module is used to: use the layers of the convolutional neural network model as a filter to perform the following on the input data in the forward pass of the layer: convolution processing on the input data to obtain a convolution feature map; pooling based on the feature matrix on the convolution feature map to obtain a pooled feature map; and, nonlinear activation on the pooled feature map to obtain an activation feature map; wherein the output of the last layer of the convolutional neural network as a filter is the classification feature vector, and the input of the first layer of the convolutional neural network as a filter is the parameter association feature matrix.

[0019] In the above-mentioned life safety monitoring system based on smart wearable devices, it also includes a training module for training the deep neural network model including multiple fully connected layers, the dual-branch network structure including the first convolutional neural network model and the second convolutional neural network model, the convolutional neural network model as a filter, and the classifier.

[0020] In the above-mentioned life safety monitoring system based on smart wearable devices, the training module includes: a training data acquisition module for acquiring training data, wherein the training data includes training physiological parameter data at multiple predetermined time points within a predetermined time period and training environmental parameter data at the multiple predetermined time points, as well as the true value of whether the life and health status of the monitored object is good; a training data association module for respectively passing the training physiological parameter data and training environmental parameter data at each predetermined time point through the deep neural network model comprising multiple fully connected layers to obtain multiple training physiological depth feature vectors and multiple training environment depth feature vectors; a training data time series change module for arranging the multiple training physiological depth feature vectors and the multiple training environment depth feature vectors into a training physiological full time series feature matrix and a training environment full time series feature matrix, and then passing them through the dual-branch network structure comprising the first convolutional neural network model and the second convolutional neural network model to obtain a training physiological time series association feature matrix and a training environment time series association feature matrix. , wherein the first convolutional neural network model and the second convolutional neural network model use two-dimensional convolution kernels with different scales; a training data feature fusion module is used to fuse the training physiological time series association feature matrix and the training environment time series association feature matrix to obtain a training parameter association feature matrix; an optimization module is used to optimize the feature distribution of the training parameter association feature matrix to obtain an optimized training parameter association feature matrix; a training association coding module is used to pass the optimized training parameter association feature matrix through the convolutional neural network model as a filter to obtain a training classification feature vector; a classification loss module is used to pass the training classification feature vector through the classifier to obtain a classification loss function value; a training module is used to train the deep neural network model including multiple fully connected layers, the dual-branch network structure including the first convolutional neural network model and the second convolutional neural network model, the convolutional neural network model as a filter and the classifier based on the classification loss function value and through back propagation of gradient descent.

[0021] In the above-mentioned life safety monitoring system based on smart wearable devices, the optimization module includes: a matrix diagonalization unit, which is used to convert the training parameter association feature matrix into a diagonal matrix to obtain a parameter association diagonal matrix; and a matrix optimization unit, which is used to perform bit-by-bit shift association matching optimization of the parameter association diagonal matrix to obtain an optimized training parameter association feature matrix.

[0022] In the above-mentioned life safety monitoring system based on smart wearable devices, the matrix optimization unit is used to perform bit-by-bit shift correlation matching optimization of the parameter correlation diagonal matrix in eigennormalization using the following enhanced formula to obtain the optimized training parameter correlation feature matrix; wherein the enhanced formula is:

[0023]

[0024]

[0025] in is the parameter correlation diagonal matrix, arrive is obtained by eigendecomposing the parameter-related diagonal matrix eigenvalues, For the The eigenvalues ​​are arranged along the diagonal to obtain the eigennormalized matrix, and and are all diagonal matrices, is the distance between the intrinsic unitization matrix and the parameter association diagonal matrix, represents matrix multiplication, represents matrix addition, Indicates point multiplication by position, The optimized training parameter is associated with a feature matrix.

[0026] According to another aspect of the present application, a life safety monitoring method based on a smart wearable device is provided, which includes:

[0027] Acquiring physiological parameter data and environmental parameter data at multiple predetermined time points within a predetermined time period collected by a smart wearable device worn on the subject to be monitored, wherein the physiological parameter data includes body temperature, movement state, movement speed, heart rate, pulse and blood pressure, and the environmental parameter data includes temperature, humidity, air pressure, ultraviolet intensity and harmful gas concentration;

[0028] Passing the physiological parameter data and environmental parameter data at each predetermined time point through a deep neural network model comprising multiple fully connected layers to obtain multiple physiological depth feature vectors and multiple environmental depth feature vectors;

[0029] Arranging the multiple physiological depth feature vectors and the multiple environmental depth feature vectors into a physiological full temporal feature matrix and an environmental full temporal feature matrix, respectively, through a dual-branch network structure including a first convolutional neural network model and a second convolutional neural network model to obtain a physiological temporal correlation feature matrix and an environmental temporal correlation feature matrix, wherein the first convolutional neural network model and the second convolutional neural network model use two-dimensional convolution kernels with different scales;

[0030] fusing the physiological time series correlation feature matrix and the environmental time series correlation feature matrix to obtain a parameter correlation feature matrix;

[0031] Passing the parameter association feature matrix through a convolutional neural network model as a filter to obtain a classification feature vector;

[0032] The classification feature vector is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the life and health status of the subject to be monitored is good.

[0033] According to another aspect of the present application, an electronic device is provided, comprising: a processor; and a memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor executes the life safety monitoring method based on the smart wearable device as described above.

[0034] According to another aspect of the present application, a computer-readable medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the life safety monitoring method based on the smart wearable device as described above.

[0035] Compared with the existing technology, the present application provides a life safety monitoring system and method based on smart wearable devices. It uses a neural network model based on deep learning to mine the correlation feature distribution information between the temporal dynamic correlation features of each physiological data item of the wearer's physiological parameter data and the temporal dynamic correlation features of each environmental data item of the environmental parameter data, so as to comprehensively monitor human life safety by utilizing the temporal correlation features between in vitro environmental parameter information and human physiological parameter information, thereby improving the accuracy of life safety monitoring of smart wearable devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0037] Figure 1 is a block diagram of a life safety monitoring system based on a smart wearable device according to an embodiment of the present application;

[0038] Figure 2 is a block diagram of a life safety monitoring system based on a smart wearable device according to an embodiment of the present application;

[0039] Figure 3 This is a system architecture diagram of an inference module in a life safety monitoring system based on a smart wearable device according to an embodiment of the present application;

[0040] Figure 4This is a system architecture diagram of a training module in a life safety monitoring system based on a smart wearable device according to an embodiment of the present application;

[0041] Figure 5 A flowchart of convolutional neural network encoding in a life safety monitoring system based on a smart wearable device according to an embodiment of the present application;

[0042] Figure 6 A block diagram of an optimization module in a life safety monitoring system based on a smart wearable device according to an embodiment of the present application;

[0043] Figure 7 Flowchart of a life safety monitoring method based on a smart wearable device according to an embodiment of the present application;

[0044] Figure 8 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0046] Scenario Overview

[0047] As mentioned in the previous background technology, currently available smart bracelets can use a three-axis accelerometer to determine a person's movements, obtain basic data, and then combine it with the wearer's personal vital signs. Using a specific algorithm, they can generate personalized monitoring data, such as steps, distance, and calories. They can also monitor sleep using the three-axis accelerometer and heart rate using an optical sensor. However, existing smart bracelets fail to consider the impact of the surrounding environment on human health when monitoring vital signs, and are unable to monitor the wearer's environment and condition in real time. Therefore, an optimized life safety monitoring system based on smart wearable devices is desired.

[0048] Accordingly, in order to improve the accuracy of life safety monitoring for the wearer during actual human vital signs monitoring, it is necessary to comprehensively monitor the vital signs of the human body based on physiological parameter data and environmental parameter data. However, since physiological parameter information includes body temperature, movement state, movement speed, heart rate, pulse and blood pressure, and environmental parameter information includes temperature, humidity, air pressure, ultraviolet intensity and concentration of harmful gases such as NH3, H2S and CO, etc., these physiological parameter information and environmental parameter information not only have dynamic change patterns in the time dimension, but also have correlation relationships between the various environmental data items in the environmental parameter information, and the various physiological data items in the physiological parameter information are also correlated, and there is also an implicit overall correlation between the environmental parameter information and the physiological parameter information. Therefore, in this process, the difficulty lies in how to dig out the correlation feature distribution information between the temporal dynamic correlation features of each physiological data item of the physiological parameter data and the temporal dynamic correlation features of each environmental data item of the environmental parameter data, so as to utilize the temporal correlation features between the in vitro environmental parameter information and the human physiological parameter information to comprehensively perform human life safety monitoring, so as to improve the accuracy of life safety monitoring of smart wearable devices.

[0049] In recent years, deep learning and neural networks have been widely used in computer vision, natural language processing, text signal processing and other fields. The development of deep learning and neural networks provides a new solution and scheme for mining the correlation feature distribution information between the temporal dynamic correlation features of each physiological data item of the physiological parameter data and the temporal dynamic correlation features of each environmental data item of the environmental parameter data. Those skilled in the art should know that the deep neural network model based on deep learning can be trained through appropriate strategies, such as the back-propagation algorithm of gradient descent to adjust the parameters of the deep neural network model so that it can simulate the complex nonlinear associations between things, and this is obviously suitable for simulating and mining the correlation feature distribution information between the temporal dynamic correlation features of each physiological data item of the physiological parameter data and the temporal dynamic correlation features of each environmental data item of the environmental parameter data.

[0050] Specifically, in the technical solution of the present application, first, physiological parameter data and environmental parameter data at multiple predetermined time points within a predetermined time period are acquired by a smart wearable device worn on the subject to be monitored. The physiological parameter data include body temperature, movement state, movement speed, heart rate, pulse, and blood pressure, and the environmental parameter data include temperature, humidity, air pressure, ultraviolet intensity, and harmful gas concentration. In particular, the harmful gas concentration here includes but is not limited to NH3, H2S, CO, NO2, and SO2.

[0051] Then, considering that there is a correlation between each physiological data item in the physiological parameter data, there is also a correlation between each environmental data item in the environmental parameter data, therefore, in order to improve the accuracy of monitoring the vital health status of the monitored object, it is necessary to extract this correlation feature information to integrate the correlation features between these data to accurately detect vital signs. Specifically, in the technical solution of the present application, the physiological parameter data and environmental parameter data at each predetermined time point are respectively subjected to feature extraction in a deep neural network model containing multiple fully connected layers to extract the correlation feature distribution information between each physiological data item in the physiological parameter data and the correlation feature distribution information between each environmental data item in the environmental parameter data, so as to obtain multiple physiological depth feature vectors and multiple environmental depth feature vectors.

[0052] Furthermore, for the physiological parameter data, each physiological data item therein not only has an association relationship, but also has dynamic change characteristic information in the time dimension. Therefore, in the technical solution of the present application, it is necessary to fully express the temporal dynamic association characteristics of each physiological data item of the physiological parameter data, so as to improve the accuracy of physiological sign monitoring of the monitored object. In particular, it is also taken into account that the collaborative characteristics of each physiological data item of the physiological parameter data have different temporal dynamic change characteristic information under different time period spans. Therefore, in the technical solution of the present application, the multiple physiological depth feature vectors are arranged into a physiological full temporal feature matrix and then a dual-branch network structure including a first convolutional neural network model and a second convolutional neural network model is used to obtain a physiological temporal association feature matrix. It is worth mentioning that here, the first convolutional neural network model and the second convolutional neural network model use two-dimensional convolution kernels with different scales, so that the collaborative characteristics of each physiological data item of the physiological parameter data under different time spans can be extracted. Multi-scale dynamic change characteristic information of the temporal sequence.

[0053] Similarly, for the environmental parameter data, the multiple environmental depth feature vectors are arranged into an environmental full temporal feature matrix and then passed through a dual-branch network structure including a first convolutional neural network model and a second convolutional neural network model to obtain an environmental temporal correlation feature matrix. In particular, here, the first convolutional neural network model and the second convolutional neural network model use two-dimensional convolution kernels with different scales to extract the collaborative features of each environmental data item of the environmental parameter data and the temporal multi-scale dynamic change feature information under different time spans.

[0054] Then, the physiological time series correlation feature matrix and the environmental time series correlation feature matrix are further fused to fuse the time series multi-scale dynamic correlation features of the collaborative features of each physiological data item of the physiological parameter data and the time series multi-scale dynamic correlation features of the collaborative features of each environmental data item of the environmental parameter data, thereby obtaining a parameter correlation feature matrix.

[0055] Then, it is also considered that for the physiological parameter data as a whole and the environmental parameter data as a whole, there is also a temporal dynamic correlation relationship between the two in the time dimension, that is, the life and health status of the subject to be monitored is jointly affected by the temporal change information of its physiological signs and the temporal change information of the environment. Therefore, in the technical solution of the present application, in order to accurately detect and analyze the life and health status of the subject to be monitored, the parameter correlation feature matrix is ​​further subjected to feature mining in a convolutional neural network model as a filter to extract the correlation feature distribution information of the temporal multi-scale dynamic change characteristics of the physiological parameter data and the temporal multi-scale dynamic change characteristics of the environmental parameter data, thereby obtaining a classification feature vector.

[0056] Furthermore, the classification feature vector is classified in a classifier to obtain a classification result indicating whether the vital health status of the subject to be monitored is good.

[0057] That is, in the technical solution of the present application, the labels of the classifier include the vital health status of the object to be monitored being good (first label), and the vital health status of the object to be monitored being poor (second label), wherein the classifier determines to which classification label the classification feature vector belongs through a soft maximum function. It is worth noting that the first label p1 and the second label p2 here do not contain artificially set concepts. In fact, during the training process, the computer model does not have the concept of "whether the vital health status of the object to be monitored is good". It only has two classification labels and the probability of the output feature under these two classification labels, that is, the sum of p1 and p2 is one. Therefore, the classification result of whether the vital health status of the object to be monitored is good is actually converted into a binary class probability distribution that conforms to natural laws through the classification label. In essence, what is used is the physical meaning of the natural probability distribution of the label, rather than the linguistic text meaning of "whether the vital health status of the object to be monitored is good". It should be understood that in the technical solution of the present application, the classification label of the classifier is a detection and evaluation label for whether the life and health status of the monitored object is good. Therefore, after obtaining the classification result, the life and health status of the monitored object can be accurately analyzed and judged based on the classification result to improve the accuracy of life safety monitoring of smart wearable devices.

[0058] In particular, in the technical solution of the present application, first, here, when fusing the physiological time series association feature matrix and the environmental time series association feature matrix to obtain the parameter association feature matrix, taking into account the feature extraction of heterogeneous data represented by physiological parameter data and environmental parameter data under convolution kernels of different scales, the global correlation between the overall feature distributions of the physiological time series association feature matrix and the environmental time series association feature matrix is ​​poor, so that the correlation between the various eigenvalues ​​of the fused parameter association feature matrix is ​​low. In this way, during the training process of the model, when the gradient is back-propagated to the parameter association feature matrix, some of its eigenvalues ​​located at the edge of the class target domain will be mismatched and constrained in the opposite optimization direction, thereby affecting the training effect of the model.

[0059] Therefore, in the technical solution of the present application, it is preferred to first convert the parameter association characteristic matrix into a diagonal matrix, for example, , and then the parameter associated diagonal matrix Perform bit-by-bit shift correlation matching optimization of intrinsic unitization, and the optimized parameter correlation feature matrix Expressed as:

[0060]

[0061]

[0062] arrive is the parameter correlation diagonal matrix After eigendecomposition, we get eigenvalues, For the The eigenvalues ​​are arranged along the diagonal to obtain the eigennormalized matrix, which is also a diagonal matrix. is the intrinsic unit matrix The diagonal matrix associated with the parameters The distance between them.

[0063] That is, by correlating the diagonal matrix based on the parameters The eigennormalized matrix obtained by the eigendecomposition To associate the diagonal matrix with the parameters Perform bitwise shift association and associate the diagonal matrix with the parameters Matching feature correlations based on the projection distance in the relative intrinsic unit space can solve the mismatch problem of optimization direction caused by weak correlation distribution of features during back propagation of model parameters, and avoid the correlation feature matrix of optimized parameters. Eigenvalues ​​at the edge of the target domain are mismatched and constrained in opposite optimization directions, resulting in poor training results. This allows for comprehensive human life safety monitoring based on the external environmental conditions and physiological signs of the subject to be monitored, thereby improving the accuracy of life safety monitoring for smart wearable devices.

[0064] Based on this, the present application proposes a life safety monitoring system based on smart wearable devices, which includes: a data acquisition module for acquiring physiological parameter data and environmental parameter data of multiple predetermined time points within a predetermined time period collected by the smart wearable device worn on the monitored object, wherein the physiological parameter data include body temperature, movement state, movement speed, heart rate, pulse and blood pressure, and the environmental parameter data include temperature, humidity, air pressure, ultraviolet intensity and harmful gas concentration; a data association module for respectively passing the physiological parameter data and environmental parameter data of each predetermined time point through a deep neural network model including multiple fully connected layers to obtain multiple physiological depth feature vectors and multiple environmental depth feature vectors; a data time series change module for combining the multiple physiological depth feature vectors and the multiple environmental depth feature vectors. After the eigenvectors are arranged into a physiological full-time series feature matrix and an environmental full-time series feature matrix, they are respectively passed through a dual-branch network structure including a first convolutional neural network model and a second convolutional neural network model to obtain a physiological time series association feature matrix and an environmental time series association feature matrix, wherein the first convolutional neural network model and the second convolutional neural network model use two-dimensional convolution kernels with different scales; a data feature fusion module is used to fuse the physiological time series association feature matrix and the environmental time series association feature matrix to obtain a parameter association feature matrix; an association coding module is used to pass the parameter association feature matrix through a convolutional neural network model as a filter to obtain a classification feature vector; a health monitoring module is used to pass the classification feature vector through a classifier to obtain a classification result, and the classification result is used to indicate whether the vital health status of the monitored object is good.

[0065] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0066] Exemplary Systems

[0067] Figure 1 FIG is a block diagram of a life safety monitoring system based on a smart wearable device according to an embodiment of the present application. Figure 1 As shown, according to an embodiment of the present application, the life safety monitoring system 300 based on smart wearable devices includes an inference module, wherein the inference module includes: a data acquisition module 310; a data association module 320; a data time series change module 330; a data feature fusion module 340; an association coding module 350; and a health monitoring module 360.

[0068] Among them, the data acquisition module 310 is used to obtain physiological parameter data at multiple predetermined time points within a predetermined time period and environmental parameter data at the multiple predetermined time points collected by the smart wearable device worn on the monitored object, wherein the physiological parameter data includes body temperature, movement state, movement speed, heart rate, pulse and blood pressure, and the environmental parameter data includes temperature, humidity, air pressure, ultraviolet intensity and harmful gas concentration; the data association module 320 is used to pass the physiological parameter data and environmental parameter data of each predetermined time point through a deep neural network model containing multiple fully connected layers to obtain multiple physiological depth feature vectors and multiple environmental depth feature vectors; the data time series change module 330 is used to arrange the multiple physiological depth feature vectors and the multiple environmental depth feature vectors into a physiological full time series feature matrix and the environmental full-time series feature matrix, respectively, through a dual-branch network structure including a first convolutional neural network model and a second convolutional neural network model to obtain a physiological time series association feature matrix and an environmental time series association feature matrix, wherein the first convolutional neural network model and the second convolutional neural network model use two-dimensional convolution kernels with different scales; the data feature fusion module 340 is used to fuse the physiological time series association feature matrix and the environmental time series association feature matrix to obtain a parameter association feature matrix; the association coding module 350 is used to pass the parameter association feature matrix through a convolutional neural network model as a filter to obtain a classification feature vector; the health monitoring module 360 ​​is used to pass the classification feature vector through a classifier to obtain a classification result, and the classification result is used to indicate whether the vital health status of the monitored object is good.

[0069] Figure 3 This is a system architecture diagram of the inference module in the life safety monitoring system based on smart wearable devices according to an embodiment of the present application. Figure 3As shown, in the system architecture of the life safety monitoring system 300 based on the smart wearable device, in the inference process, the data acquisition module 310 first acquires the physiological parameter data of multiple predetermined time points within a predetermined time period and the environmental parameter data of the multiple predetermined time points acquired by the smart wearable device worn on the monitored object, wherein the physiological parameter data include body temperature, movement state, movement speed, heart rate, pulse and blood pressure, and the environmental parameter data include temperature, humidity, air pressure, ultraviolet intensity and harmful gas concentration; then, the data association module 320 respectively passes the physiological parameter data and environmental parameter data of each predetermined time point acquired by the data acquisition module 310 through a deep neural network model including multiple fully connected layers to obtain multiple physiological depth feature vectors and multiple environmental depth feature vectors; the data time series change module 330 arranges the multiple physiological depth feature vectors and the multiple environmental depth feature vectors acquired by the data association module 320 After the physiological full-time series feature matrix and the environmental full-time series feature matrix are obtained, they are respectively passed through a dual-branch network structure including a first convolutional neural network model and a second convolutional neural network model to obtain a physiological time series association feature matrix and an environmental time series association feature matrix, wherein the first convolutional neural network model and the second convolutional neural network model use two-dimensional convolution kernels with different scales; then, the data feature fusion module 340 fuses the physiological time series association feature matrix and the environmental time series association feature matrix obtained by the data time series change module 330 to obtain a parameter association feature matrix; the association coding module 350 passes the parameter association feature matrix fused by the data feature fusion module 340 through a convolutional neural network model as a filter to obtain a classification feature vector; further, the health monitoring module 360 ​​passes the classification feature vector obtained by the association coding module 350 through a classifier to obtain a classification result, and the classification result is used to indicate whether the vital health status of the monitored object is good.

[0070] Specifically, during the operation of the life safety monitoring system 300 based on the smart wearable device, the data acquisition module 310 is used to obtain physiological parameter data at multiple predetermined time points within a predetermined time period and environmental parameter data at the multiple predetermined time points collected by the smart wearable device worn on the monitored object, wherein the physiological parameter data includes body temperature, movement state, movement speed, heart rate, pulse and blood pressure, and the environmental parameter data includes temperature, humidity, air pressure, ultraviolet intensity and harmful gas concentration. It should be understood that when actually performing human vital sign monitoring, in order to improve the accuracy of life safety monitoring for the wearer, it is necessary to comprehensively monitor the vital signs of the human body based on physiological parameter data and environmental parameter data. In one example, the physiological parameter information includes body temperature, movement state, movement speed, heart rate, pulse and blood pressure, and the environmental parameter information includes temperature, humidity, air pressure, ultraviolet intensity and harmful gas concentration. In particular, here, the harmful gas concentration includes but is not limited to NH3, H2S, CO, NO2 and SO2.

[0071] Specifically, during the operation of the life safety monitoring system 300 based on the smart wearable device, the data association module 320 is used to pass the physiological parameter data and environmental parameter data at each predetermined time point through a deep neural network model containing multiple fully connected layers to obtain multiple physiological depth feature vectors and multiple environmental depth feature vectors. Considering that there is a correlation relationship between each physiological data item in the physiological parameter data and there is also a correlation relationship between each environmental data item in the environmental parameter data, in order to improve the accuracy of monitoring the life health status of the monitored object, it is necessary to extract such correlation feature information to comprehensively analyze the correlation features between these data to accurately detect vital signs. Specifically, in the technical solution of the present application, the physiological parameter data and environmental parameter data at each predetermined time point are respectively subjected to feature extraction in a deep neural network model containing multiple fully connected layers to extract the correlation feature distribution information between each physiological data item in the physiological parameter data and the correlation feature distribution information between each environmental data item in the environmental parameter data to obtain multiple physiological depth feature vectors and multiple environmental depth feature vectors. More specifically, in a specific example of the present application, the physiological parameter data at each predetermined time point is fully connected encoded using the fully connected layer of the deep neural network model using the following formula to obtain the multiple physiological depth feature vectors, wherein the formula is: ,in is the physiological parameter data at each predetermined time point, is the output vector, is the weight matrix, is the bias vector, represents matrix multiplication; and, using the fully connected layer of the deep neural network model to fully connect encode the environmental parameter data using the following formula to obtain the multiple environmental depth feature vectors, wherein the formula is: ,in is the environmental parameter data, is the output vector, is the weight matrix, is the bias vector, Represents matrix multiplication.

[0072] Specifically, during the operation of the life safety monitoring system 300 based on the smart wearable device, the data time series change module 330 is used to arrange the multiple physiological depth feature vectors and the multiple environmental depth feature vectors into a physiological full time series feature matrix and an environmental full time series feature matrix, and then respectively pass them through a dual-branch network structure including a first convolutional neural network model and a second convolutional neural network model to obtain a physiological time series correlation feature matrix and an environmental time series correlation feature matrix, wherein the first convolutional neural network model and the second convolutional neural network model use two-dimensional convolution kernels with different scales. Considering that each physiological data item in the environmental parameter data and the physiological parameter data not only has a correlation relationship, but also has dynamic change characteristic information in the time dimension, therefore, in the technical solution of the present application, it is necessary to fully express the time series dynamic correlation characteristics of each physiological data item of the physiological parameter data, so as to improve the accuracy of physiological sign monitoring of the monitored object. In particular, it is also taken into account that the collaborative features of each physiological data item of the physiological parameter data have different temporal dynamic change feature information under different time period spans. Therefore, in the technical solution of the present application, the multiple physiological depth feature vectors are arranged into a physiological full temporal feature matrix and then a dual-branch network structure including a first convolutional neural network model and a second convolutional neural network model is used to obtain a physiological temporal correlation feature matrix, and the multiple environmental depth feature vectors are arranged into an environmental full temporal feature matrix and then a dual-branch network structure including a first convolutional neural network model and a second convolutional neural network model is used to obtain an environmental temporal correlation feature matrix. It is worth mentioning that here, the first convolutional neural network model and the second convolutional neural network model use two-dimensional convolution kernels with different scales. In this way, the collaborative features of each physiological data item of the physiological parameter data under different time spans and the collaborative features of each environmental data item of the environmental parameter data under different time spans can be extracted. Multi-scale dynamic change feature information. More specifically, each layer of the dual-branch network structure including the first convolutional neural network model and the second convolutional neural network model is used to perform the following operations on the input data in the forward pass of the layer: convolution processing is performed on the input data to obtain a convolution feature map; pooling is performed on the convolution feature map along the channel dimension to obtain a pooled feature map; and nonlinear activation is performed on the pooled feature map to obtain an activation feature map; wherein the output of the last layer of the dual-branch network structure including the first convolutional neural network model and the second convolutional neural network model is the physiological temporal association feature matrix and the environmental temporal association feature matrix, and the input of the first layer of the dual-branch network structure including the first convolutional neural network model and the second convolutional neural network model is the physiological full temporal feature matrix and the environmental full temporal feature matrix.

[0073] Specifically, during the operation of the life safety monitoring system 300 based on the smart wearable device, the data feature fusion module 340 is used to fuse the physiological time series correlation feature matrix and the environmental time series correlation feature matrix to obtain a parameter correlation feature matrix. That is, after obtaining the physiological time series correlation feature matrix and the environmental time series correlation feature matrix, the two are further fused to fuse the time series multi-scale dynamic correlation features of the collaborative features of each physiological data item of the physiological parameter data and the time series multi-scale dynamic correlation features of the collaborative features of each environmental data item of the environmental parameter data, thereby obtaining a parameter correlation feature matrix. In a specific example of the present application, fusion can be performed in a cascade manner. More specifically, the physiological time series correlation feature matrix and the environmental time series correlation feature matrix are fused using the following formula to obtain a parameter correlation feature matrix; wherein, the formula is: ,in, represents the physiological time series correlation feature matrix, represents the environmental temporal correlation feature matrix, Represents a cascade function, Represents the parameter association feature matrix.

[0074] Specifically, during the operation of the life safety monitoring system 300 based on the smart wearable device, the association coding module 350 is used to pass the parameter association feature matrix through the convolutional neural network model as a filter to obtain a classification feature vector. It should be understood that the physiological parameter data as a whole and the environmental parameter data also have a dynamic correlation relationship in the time dimension. That is, the life health status of the object to be monitored is jointly affected by the temporal change information of its physiological signs and the temporal change information of the environment. Therefore, in the technical solution of the present application, in order to accurately detect and analyze the life health status of the object to be monitored, the parameter association feature matrix is ​​further subjected to feature mining in the convolutional neural network model as a filter to extract the correlation feature distribution information of the temporal multi-scale dynamic change characteristics of the physiological parameter data and the temporal multi-scale dynamic change characteristics of the environmental parameter data, thereby obtaining a classification feature vector. In a specific example, the convolutional neural network includes a plurality of neural network layers cascaded with each other, wherein each neural network layer includes a convolution layer, a pooling layer and an activation layer. In which, during the encoding process of the convolutional neural network, each layer of the convolutional neural network uses the convolution layer to perform convolution processing based on the convolution kernel on the input data during the forward transmission process of the layer, uses the pooling layer to perform pooling processing on the convolution feature map output by the convolution layer, and uses the activation layer to perform activation processing on the pooling feature map output by the pooling layer.

[0075] Figure 5This is a flow chart of convolutional neural network coding in a life safety monitoring system based on smart wearable devices according to an embodiment of the present application. Figure 5 As shown, in the encoding process of the convolutional neural network, it includes: using each layer of the convolutional neural network model as a filter to perform the following on the input data in the forward pass of the layer: S210, convolution processing on the input data to obtain a convolution feature map; S220, pooling the convolution feature map based on the feature matrix to obtain a pooled feature map; and, S230, nonlinear activation on the pooled feature map to obtain an activation feature map; wherein, the output of the last layer of the convolutional neural network as a filter is the classification feature vector, and the input of the first layer of the convolutional neural network as a filter is the parameter association feature matrix.

[0076] Specifically, during the operation of the smart wearable device-based life safety monitoring system 300, the health monitoring module 360 ​​is used to pass the classification feature vector through a classifier to obtain a classification result, and the classification result is used to indicate whether the life and health status of the monitored subject is good. That is, in the technical solution of the present application, the classification feature vector is passed through a classifier to obtain a classification result indicating whether the life and health status of the monitored subject is good. Specifically, the classifier is used to process the classification feature vector using the following formula to obtain the classification result, wherein the formula is: ,in, arrive is the weight matrix, arrive is the bias vector, is a classification feature vector. Specifically, the classifier includes multiple fully connected layers and a Softmax layer cascaded with the last fully connected layer of the multiple fully connected layers. In the classification process of the classifier, the classification feature vector is fully connected multiple times using the multiple fully connected layers of the classifier to obtain an encoded classification feature vector; then, the encoded classification feature vector is input into the Softmax layer of the classifier, that is, the encoded classification feature vector is classified using the Softmax classification function to obtain a classification label. In the technical solution of the present application, the labels of the classifier include the vital health status of the monitored object is good (first label) and the vital health status of the monitored object is poor (second label), wherein the classifier determines which classification label the classification feature vector belongs to by using a soft maximum function. It is worth noting that the first label p1 and the second label p2 here do not contain artificially set concepts. In fact, during the training process, the computer model does not have the concept of "whether the vital health status of the monitored object is good". It only has two classification labels and the probability of the output feature under these two classification labels, that is, the sum of p1 and p2 is one. Therefore, the classification result of whether the life and health status of the monitored subject is good is actually converted from the classification label into a binary class probability distribution that conforms to natural laws. In essence, the physical meaning of the natural probability distribution of the label is used, rather than the linguistic meaning of "whether the life and health status of the monitored subject is good." It should be understood that in the technical solution of this application, the classification label of the classifier is a detection and evaluation label of whether the life and health status of the monitored subject is good. Therefore, after obtaining the classification result, the life and health status of the monitored subject can be accurately analyzed and judged based on the classification result, thereby improving the accuracy of life safety monitoring of smart wearable devices.

[0077] It should be understood that before using the above-mentioned neural network model for inference, it is necessary to train the deep neural network model comprising multiple fully connected layers, the dual-branch network structure comprising the first convolutional neural network model and the second convolutional neural network model, the convolutional neural network model serving as a filter, and the classifier. That is, in the life safety monitoring system based on smart wearable devices of the present application, a training module is also included for training the deep neural network model comprising multiple fully connected layers, the dual-branch network structure comprising the first convolutional neural network model and the second convolutional neural network model, the convolutional neural network model serving as a filter, and the classifier. The training of deep neural networks mostly uses the backpropagation algorithm, which uses the chain rule to update the parameters of the current layer through the error transmitted from the next layer. This will encounter the problem of vanishing gradients when the network is very deep, or more broadly referred to as the unstable gradient problem.

[0078] Figure 2 FIG is a block diagram of a life safety monitoring system based on a smart wearable device according to an embodiment of the present application. Figure 2 As shown, according to the embodiment of the present application, the life safety monitoring system 300 based on the smart wearable device also includes a training module 400, which includes: a training data acquisition module 410; a training data association module 420; a training data time series change module 430; a training data feature fusion module 440; an optimization module 450; a training association encoding module 460; a classification loss module 470; and a training module 480.

[0079] Among them, the training data acquisition module 410 is used to obtain training data, and the training data includes training physiological parameter data of multiple predetermined time points within a predetermined time period and training environmental parameter data of the multiple predetermined time points, as well as the real value of whether the vital health status of the monitored object is good; the training data association module 420 is used to pass the training physiological parameter data and training environmental parameter data of each predetermined time point through the deep neural network model containing multiple fully connected layers to obtain multiple training physiological depth feature vectors and multiple training environment depth feature vectors; the training data time series change module 430 is used to arrange the multiple training physiological depth feature vectors and the multiple training environment depth feature vectors into a training physiological full time series feature matrix and a training environment full time series feature matrix, and then pass them through the dual-branch network structure containing the first convolutional neural network model and the second convolutional neural network model to obtain a training physiological time series association feature matrix and a training environment time series association feature matrix, wherein the first convolutional neural network model and the second convolutional neural network model The two convolutional neural network models use two-dimensional convolution kernels with different scales; the training data feature fusion module 440 is used to fuse the training physiological time series association feature matrix and the training environment time series association feature matrix to obtain a training parameter association feature matrix; the optimization module 450 is used to optimize the feature distribution of the training parameter association feature matrix to obtain an optimized training parameter association feature matrix; the training association encoding module 460 is used to pass the optimized training parameter association feature matrix through the convolutional neural network model as a filter to obtain a training classification feature vector; the classification loss module 470 is used to pass the training classification feature vector through the classifier to obtain a classification loss function value; the training module 480 is used to train the deep neural network model comprising multiple fully connected layers, the dual-branch network structure comprising the first convolutional neural network model and the second convolutional neural network model, the convolutional neural network model as a filter and the classifier based on the classification loss function value and through back propagation of gradient descent.

[0080] Figure 4This is a system architecture diagram of a training module in a life safety monitoring system based on a smart wearable device according to an embodiment of the present application. Figure 4 As shown, in the system architecture of the life safety monitoring system 300 based on the smart wearable device, in the training module 400, the training data is first obtained by the training data acquisition module 410, and the training data includes training physiological parameter data of multiple predetermined time points within a predetermined time period and training environmental parameter data of the multiple predetermined time points, as well as the real value of whether the life health status of the monitored object is good; then, the training data association module 420 passes the training physiological parameter data and training environmental parameter data of each predetermined time point obtained by the training data acquisition module 410 through the deep neural network model comprising multiple fully connected layers to obtain multiple training physiological depth feature vectors and multiple training environment depth feature vectors; the training data time series change module 430 arranges the multiple training physiological depth feature vectors and the multiple training environment depth feature vectors obtained by the training data association module 420 into a training physiological full time series feature matrix and a training environment full time series feature matrix, and then passes them through the dual-branch network structure comprising the first convolutional neural network model and the second convolutional neural network model to obtain a training physiological time series association feature matrix and a training environment time series association feature matrix, wherein the first convolutional neural network model is a plurality of fully connected layers. The network model and the second convolutional neural network model use two-dimensional convolution kernels with different scales; the training data feature fusion module 440 fuses the training physiological timing association feature matrix and the training environment timing association feature matrix obtained by the training data timing change module 430 to obtain a training parameter association feature matrix; then, the optimization module 450 performs feature distribution optimization on the training parameter association feature matrix fused by the training data feature fusion module 440 to obtain an optimized training parameter association feature matrix; the training association encoding module 460 passes the optimized training parameter association feature matrix obtained by the optimization module 450 through the convolutional neural network model as a filter to obtain a training classification feature vector; the classification loss module 470 passes the training classification feature vector obtained by the training association encoding module 460 through the classifier to obtain a classification loss function value; further, the training module 480 trains the deep neural network model comprising multiple fully connected layers, the dual-branch network structure comprising the first convolutional neural network model and the second convolutional neural network model, the convolutional neural network model as a filter, and the classifier based on the classification loss function value and through back propagation of gradient descent.

[0081] In particular, in the technical solution of the present application, first, here, when fusing the physiological time series association feature matrix and the environmental time series association feature matrix to obtain the parameter association feature matrix, considering the feature extraction of heterogeneous data represented by physiological parameter data and environmental parameter data under convolution kernels of different scales, the global correlation between the overall feature distributions of the physiological time series association feature matrix and the environmental time series association feature matrix is ​​relatively poor, so that the correlation between the various eigenvalues ​​of the fused parameter association feature matrix is ​​low. In this way, during the training process of the model, when the gradient is back-propagated to the parameter association feature matrix, some of its eigenvalues ​​located at the edge of the class target domain will be mismatched and constrained in the opposite optimization direction, thereby affecting the training effect of the model. Therefore, in the technical solution of the present application, it is preferred to first convert the parameter association feature matrix into a diagonal matrix, for example, denoted as , and then the parameter associated diagonal matrix Perform bit-by-bit shift correlation matching optimization of intrinsic unitization, and the optimized parameter correlation feature matrix Expressed as:

[0082]

[0083]

[0084] in is the parameter correlation diagonal matrix, arrive is obtained by eigendecomposing the parameter-related diagonal matrix eigenvalues, For the The eigenvalues ​​are arranged along the diagonal to obtain the eigennormalized matrix, and and are all diagonal matrices, is the distance between the intrinsic unitization matrix and the parameter association diagonal matrix, represents matrix multiplication, represents matrix addition, Indicates point multiplication by position, The parameter-associated feature matrix is ​​optimized. That is, the parameter-associated diagonal matrix The eigennormalized matrix obtained by the eigendecomposition To associate the diagonal matrix with the parameters Perform bitwise shift association and associate the diagonal matrix with the parameters Matching feature correlations based on the projection distance in the relative intrinsic unit space can solve the mismatch problem of optimization direction caused by weak correlation distribution of features during back propagation of model parameters, and avoid the correlation feature matrix of optimized parameters. Eigenvalues ​​at the edge of the target domain are mismatched and constrained in opposite optimization directions, resulting in poor training results. This allows for comprehensive human life safety monitoring based on the external environmental conditions and physiological signs of the subject to be monitored, thereby improving the accuracy of life safety monitoring for smart wearable devices.

[0085] Figure 6 FIG. 1 is a block diagram of an optimization module in a life safety monitoring system based on a smart wearable device according to an embodiment of the present application. Figure 6 As shown, the optimization module 450 includes: a matrix diagonalization unit 451, which is used to convert the training parameter association feature matrix into a diagonal matrix to obtain a parameter association diagonal matrix; and a matrix optimization unit 452, which is used to perform bit-by-bit shift association matching optimization of the parameter association diagonal matrix to obtain an optimized training parameter association feature matrix.

[0086] In summary, according to the embodiment of the present application, a life safety monitoring system 300 based on a smart wearable device is explained, which uses a neural network model based on deep learning to mine the correlation feature distribution information between the temporal dynamic correlation features of each physiological data item of the wearer's physiological parameter data and the temporal dynamic correlation features of each environmental data item of the environmental parameter data, so as to comprehensively perform human life safety monitoring by utilizing the temporal correlation features between in vitro environmental parameter information and human physiological parameter information, so as to improve the accuracy of life safety monitoring of smart wearable devices.

[0087] As described above, the life safety monitoring system based on the smart wearable device according to the embodiment of the present application can be implemented in various terminal devices. In one example, the life safety monitoring system 300 based on the smart wearable device according to the embodiment of the present application can be integrated into the terminal device as a software module and / or hardware module. For example, the life safety monitoring system 300 based on the smart wearable device can be a software module in the operating system of the terminal device, or it can be an application developed for the terminal device; of course, the life safety monitoring system 300 based on the smart wearable device can also be one of the many hardware modules of the terminal device.

[0088] Alternatively, in another example, the life safety monitoring system 300 based on the smart wearable device and the terminal device may also be separate devices, and the life safety monitoring system 300 based on the smart wearable device may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0089] Exemplary Methods

[0090] Figure 7Flowchart of the life safety monitoring method based on smart wearable device according to the embodiment of the present application. Figure 7 As shown, according to the embodiment of the present application, the life safety monitoring method based on the smart wearable device includes the following steps: S110, obtaining physiological parameter data of multiple predetermined time points within a predetermined time period and environmental parameter data of the multiple predetermined time points collected by the smart wearable device worn on the monitored object, wherein the physiological parameter data include body temperature, movement state, movement speed, heart rate, pulse and blood pressure, and the environmental parameter data include temperature, humidity, air pressure, ultraviolet intensity and harmful gas concentration; S120, respectively passing the physiological parameter data and environmental parameter data of each predetermined time point through a deep neural network model including multiple fully connected layers to obtain multiple physiological depth feature vectors and multiple environmental depth feature vectors; S130, combining the multiple physiological depth feature vectors and the multiple environmental depth feature vectors. After the feature vectors are arranged into a physiological full-time series feature matrix and an environmental full-time series feature matrix, they are respectively passed through a dual-branch network structure including a first convolutional neural network model and a second convolutional neural network model to obtain a physiological time series association feature matrix and an environmental time series association feature matrix, wherein the first convolutional neural network model and the second convolutional neural network model use two-dimensional convolution kernels with different scales; S140, the physiological time series association feature matrix and the environmental time series association feature matrix are fused to obtain a parameter association feature matrix; S150, the parameter association feature matrix is ​​passed through a convolutional neural network model as a filter to obtain a classification feature vector; S160, the classification feature vector is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the vital health status of the monitored object is good.

[0091] In one example, in the above-mentioned life safety monitoring method based on a smart wearable device, step S120 includes: using the fully connected layer of the deep neural network model to fully connect the physiological parameter data of each predetermined time point using the following formula to obtain the multiple physiological depth feature vectors, wherein the formula is: ,in is the physiological parameter data at each predetermined time point, is the output vector, is the weight matrix, is the bias vector, represents matrix multiplication; and, using the fully connected layer of the deep neural network model to fully connect encode the environmental parameter data using the following formula to obtain the multiple environmental depth feature vectors, wherein the formula is: ,in is the environmental parameter data, is the output vector, is the weight matrix, is the bias vector, Represents matrix multiplication.

[0092] In one example, in the above-mentioned life safety monitoring method based on smart wearable devices, the step S130 includes: using each layer of the dual-branch network structure including the first convolutional neural network model and the second convolutional neural network model to perform the following on the input data in the forward pass of the layer: convolution processing on the input data to obtain a convolution feature map; pooling the convolution feature map along the channel dimension to obtain a pooled feature map; and, nonlinear activation on the pooled feature map to obtain an activation feature map; wherein the output of the last layer of the dual-branch network structure including the first convolutional neural network model and the second convolutional neural network model is the physiological temporal association feature matrix and the environmental temporal association feature matrix, and the input of the first layer of the dual-branch network structure including the first convolutional neural network model and the second convolutional neural network model is the physiological full temporal feature matrix and the environmental full temporal feature matrix.

[0093] In one example, in the above-mentioned life safety monitoring method based on smart wearable devices, step S140 includes: fusing the physiological time series correlation feature matrix and the environmental time series correlation feature matrix to obtain a parameter correlation feature matrix using the following formula; wherein the formula is:

[0094]

[0095] in, represents the physiological time series correlation feature matrix, represents the environmental temporal correlation feature matrix, Represents a cascade function, Represents the parameter association feature matrix.

[0096] In one example, in the above-mentioned life safety monitoring method based on smart wearable devices, the step S150 includes: using each layer of the convolutional neural network model as a filter to perform the following on the input data in the forward pass of the layer: convolution processing on the input data to obtain a convolution feature map; pooling the convolution feature map based on a feature matrix to obtain a pooled feature map; and, nonlinear activation on the pooled feature map to obtain an activation feature map; wherein the output of the last layer of the convolutional neural network as a filter is the classification feature vector, and the input of the first layer of the convolutional neural network as a filter is the parameter association feature matrix.

[0097] In summary, according to the embodiment of the present application, a life safety monitoring method based on a smart wearable device is explained, which uses a neural network model based on deep learning to mine the correlation feature distribution information between the temporal dynamic correlation features of each physiological data item of the wearer's physiological parameter data and the temporal dynamic correlation features of each environmental data item of the environmental parameter data, so as to comprehensively monitor human life safety by utilizing the temporal correlation features between in vitro environmental parameter information and human physiological parameter information, so as to improve the accuracy of life safety monitoring of smart wearable devices.

[0098] Exemplary electronic devices

[0099] Below, reference Figure 8 To describe the electronic device according to the embodiment of the present application.

[0100] Figure 8 The figure shows a block diagram of an electronic device according to an embodiment of the present application.

[0101] like Figure 8 As shown, the electronic device 10 includes one or more processors 11 and a memory 12 .

[0102] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0103] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the functions of the life safety monitoring system based on smart wearable devices described in the various embodiments of the present application as described above and / or other desired functions. Various contents such as parameter association feature matrices may also be stored in the computer-readable storage medium.

[0104] In one example, the electronic device 10 may further include an input device 13 and an output device 14 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0105] The input device 13 may include, for example, a keyboard, a mouse, and the like.

[0106] The output device 14 can output various information to the outside, including classification results, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0107] Of course, to simplify, Figure 8 Only some of the components related to the present application in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device 10 may further include any other appropriate components according to specific application scenarios.

[0108] Exemplary computer program products and computer-readable storage media

[0109] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to perform the steps in the functions of the life safety monitoring method based on smart wearable devices according to various embodiments of the present application described in the above "Exemplary System" section of this specification.

[0110] The computer program product may be written in any combination of one or more programming languages ​​to implement the program code for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0111] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps in the functions of the life safety monitoring method based on a smart wearable device according to various embodiments of the present application described in the above "Exemplary System" section of this specification.

[0112] The computer-readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0113] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this application are merely illustrative and not restrictive, and it should not be assumed that these advantages, strengths, and effects are required of each embodiment of this application. In addition, the specific details disclosed above are merely illustrative and facilitating understanding, and are not restrictive. The above details do not limit this application to necessarily being implemented using the above specific details.

[0114] The block diagrams of the devices, devices, equipment, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0115] It should also be noted that in the apparatus, device, and method of the present application, each component or each step can be decomposed and / or recombined, and such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0116] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0117] The above description has been provided for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A life safety monitoring system based on smart wearable devices, characterized in that: include: a data acquisition module, configured to acquire physiological parameter data and environmental parameter data at multiple predetermined time points within a predetermined time period, collected by a smart wearable device worn on a subject to be monitored, wherein the physiological parameter data includes body temperature, movement status, movement speed, heart rate, pulse, and blood pressure; and the environmental parameter data includes temperature, humidity, air pressure, ultraviolet intensity, and harmful gas concentration; a data association module, configured to pass the physiological parameter data and the environmental parameter data at each predetermined time point through a deep neural network model comprising multiple fully connected layers to obtain multiple physiological depth feature vectors and multiple environmental depth feature vectors; a data timing change module, configured to arrange the multiple physiological depth feature vectors and the multiple environmental depth feature vectors into a physiological full-time-series feature matrix and an environmental full-time-series feature matrix, and then respectively pass them through a dual-branch network structure including a first convolutional neural network model and a second convolutional neural network model to obtain a physiological timing association feature matrix and an environmental timing association feature matrix, wherein the first convolutional neural network model and the second convolutional neural network model use two-dimensional convolution kernels with different scales; a data feature fusion module, configured to fuse the physiological time series correlation feature matrix and the environmental time series correlation feature matrix to obtain a parameter correlation feature matrix; an association coding module for passing the parameter association feature matrix through a convolutional neural network model as a filter to obtain a classification feature vector; A health monitoring module, configured to pass the classification feature vector through a classifier to obtain a classification result, wherein the classification result is used to indicate whether the life and health status of the subject to be monitored is good; Wherein, the data association module is used to: The physiological parameter data at each predetermined time point is fully connected encoded using the fully connected layer of the deep neural network model using the following formula to obtain the multiple physiological depth feature vectors, wherein the formula is: ,in It is the physiological parameter data at each predetermined time point, is the output vector, is the weight matrix, is the bias vector, represents matrix multiplication; and The environmental parameter data is fully connected encoded using the fully connected layer of the deep neural network model using the following formula to obtain the multiple environmental depth feature vectors, wherein the formula is: ,in is the environmental parameter data, is the output vector, is the weight matrix, is the bias vector, Represents matrix multiplication.

2. The life safety monitoring system based on smart wearable devices according to claim 1, characterized in that: The data timing change module is used to: use each layer of the dual-branch network structure including the first convolutional neural network model and the second convolutional neural network model to respectively perform the following on the input data in the forward pass of the layer: Perform convolution on the input data to obtain a convolution feature map; Pooling the convolutional feature map along the channel dimension to obtain a pooled feature map; as well as Performing nonlinear activation on the pooled feature map to obtain an activated feature map; Among them, the output of the last layer of the dual-branch network structure including the first convolutional neural network model and the second convolutional neural network model is the physiological timing association feature matrix and the environmental timing association feature matrix, and the input of the first layer of the dual-branch network structure including the first convolutional neural network model and the second convolutional neural network model is the physiological full timing feature matrix and the environmental full timing feature matrix.

3. The life safety monitoring system based on smart wearable devices according to claim 2, characterized in that: The data feature fusion module is used to fuse the physiological time series correlation feature matrix and the environmental time series correlation feature matrix using the following formula to obtain a parameter correlation feature matrix; Wherein, the formula is: ； in, represents the physiological time series correlation feature matrix, represents the environmental temporal correlation feature matrix, Represents a cascade function, Represents the parameter association feature matrix.

4. The life safety monitoring system based on smart wearable devices according to claim 3 is characterized in that: The association coding module is used to: use each layer of the convolutional neural network model as a filter to respectively perform the following on the input data in the forward pass of the layer: Perform convolution on the input data to obtain a convolution feature map; Performing pooling on the convolutional feature map based on a feature matrix to obtain a pooled feature map; as well as Performing nonlinear activation on the pooled feature map to obtain an activated feature map; The output of the last layer of the convolutional neural network serving as a filter is the classification feature vector, and the input of the first layer of the convolutional neural network serving as a filter is the parameter association feature matrix.

5. The life safety monitoring system based on smart wearable devices according to claim 4 is characterized in that: It also includes a training module for training the deep neural network model including multiple fully connected layers, the dual-branch network structure including the first convolutional neural network model and the second convolutional neural network model, the convolutional neural network model as a filter, and the classifier.

6. The life safety monitoring system based on smart wearable devices according to claim 5, characterized in that: The training module includes: A training data acquisition module is used to obtain training data, wherein the training data includes training physiological parameter data at multiple predetermined time points within a predetermined time period and training environmental parameter data at the multiple predetermined time points, as well as a true value of whether the life and health status of the subject to be monitored is good; a training data association module, configured to pass the training physiological parameter data and the training environment parameter data at each predetermined time point through the deep neural network model comprising a plurality of fully connected layers to obtain a plurality of training physiological deep feature vectors and a plurality of training environment deep feature vectors; A training data temporal change module, configured to arrange the multiple training physiological depth feature vectors and the multiple training environment depth feature vectors into a training physiological full temporal feature matrix and a training environment full temporal feature matrix, and then pass them through the dual-branch network structure comprising the first convolutional neural network model and the second convolutional neural network model to obtain a training physiological temporal correlation feature matrix and a training environment temporal correlation feature matrix, respectively, wherein the first convolutional neural network model and the second convolutional neural network model use two-dimensional convolution kernels with different scales; A training data feature fusion module, configured to fuse the training physiological time series correlation feature matrix and the training environment time series correlation feature matrix to obtain a training parameter correlation feature matrix; an optimization module, configured to optimize the feature distribution of the training parameter associated feature matrix to obtain an optimized training parameter associated feature matrix; A training association coding module is used to pass the optimized training parameter association feature matrix through the convolutional neural network model as a filter to obtain a training classification feature vector; A classification loss module, configured to pass the training classification feature vector through the classifier to obtain a classification loss function value; A training module is used to train the deep neural network model including multiple fully connected layers, the two-branch network structure including the first convolutional neural network model and the second convolutional neural network model, the convolutional neural network model as a filter, and the classifier based on the classification loss function value and through back propagation of gradient descent.

7. The life safety monitoring system based on smart wearable devices according to claim 6, characterized in that: The optimization module includes: a matrix diagonalization unit, configured to convert the training parameter-related feature matrix into a diagonal matrix to obtain a parameter-related diagonal matrix; and The matrix optimization unit is used to perform bit-by-bit shift correlation matching optimization of the parameter correlation diagonal matrix in eigen-unitization to obtain an optimized training parameter correlation feature matrix.

8. The life safety monitoring system based on smart wearable devices according to claim 7 is characterized in that: The matrix optimization unit is used to perform bit-by-bit shift correlation matching optimization of the parameter correlation diagonal matrix in eigen-unitization using the following enhanced formula to obtain the optimized training parameter correlation feature matrix; Wherein, the strengthening formula is: ； in is the parameter correlation diagonal matrix, arrive is obtained by eigendecomposing the parameter-related diagonal matrix eigenvalues, For the The eigenvalues ​​are arranged along the diagonal to obtain the eigennormalized matrix, and and are all diagonal matrices, is the distance between the intrinsic unitization matrix and the parameter association diagonal matrix, represents matrix multiplication, represents matrix addition, Indicates point multiplication by position, The optimized training parameter is associated with a feature matrix.

9. A life safety monitoring method based on a smart wearable device, using the life safety monitoring system based on a smart wearable device according to claim 1, characterized in that: include: Acquiring physiological parameter data and environmental parameter data at multiple predetermined time points within a predetermined time period collected by a smart wearable device worn on the subject to be monitored, wherein the physiological parameter data includes body temperature, movement state, movement speed, heart rate, pulse and blood pressure, and the environmental parameter data includes temperature, humidity, air pressure, ultraviolet intensity and harmful gas concentration; Passing the physiological parameter data and the environmental parameter data at each predetermined time point through a deep neural network model including multiple fully connected layers to obtain multiple physiological depth feature vectors and multiple environmental depth feature vectors; Arranging the multiple physiological depth feature vectors and the multiple environmental depth feature vectors into a physiological full temporal feature matrix and an environmental full temporal feature matrix, respectively, through a dual-branch network structure including a first convolutional neural network model and a second convolutional neural network model to obtain a physiological temporal correlation feature matrix and an environmental temporal correlation feature matrix, wherein the first convolutional neural network model and the second convolutional neural network model use two-dimensional convolution kernels with different scales; fusing the physiological time series correlation feature matrix and the environmental time series correlation feature matrix to obtain a parameter correlation feature matrix; Passing the parameter association feature matrix through a convolutional neural network model as a filter to obtain a classification feature vector; The classification feature vector is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the life and health status of the subject to be monitored is good.

Images