Wearable device and system for human activity recognition using topological data analysis methods

Abstract

A system for human activity recognition using persistent homology and machine learning includes a wearable device having a sensor configured to detect motion of a subject wearing the wearable device, and a processor configured to receive data from the sensor, generate persistence images of the data using persistent homology, input the persistence images to a machine learning model that classifies the persistence images according to a classification of activity, compare the classified activity with a baseline model, and produce a recognition of the activity of the subject based on the comparison.

Description

WEARABLE DEVICE AND SYSTEM FOR HUMAN ACTIVITY RECOGNITIONUSING TOPOLOGICAL DATA ANALYSIS METHODSCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] The present application claims the benefit of U.S. Provisional Patent Application No. 63 / 593,739 filed October 27, 2023, the entirety of which is incorporated hereinBACKGROUND

[0002] Sensor data has become increasingly prevalent in the field of machine learning. One task involving sensor data is Human Activity Recognition (HAR), which aims to decipher specific actions that a person is doing, based on accelerometer and gyroscope data. HAR is a classification task that involves predicting which activity a person is performing from biometric sensor time-series data. These activities can involve common acts such as walking, jogging, typing and eating. Collection of data for use in HAR can come from, for example, healthcare monitoring, surveillance systems, and assisted living facilities. Further, wearable devices such as smart watches, handheld devices, headsets, headphones and the like can be used to collect such data. Wearable sensor data can then be collected from measurement devices such as accelerometers, gyroscopes and other motion-sensing instruments within the wearable devices.

[0003] For improving the accuracy of HAR programs, deep learning can be applied to sensor data that is obtained, for example, from these wearable devices. Examples of deep learning methods include Conventional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM) and a multi-channel CNN Bi-Directional LSTM (MCBLSTM). However, such machine learning methods can have difficulty making sense of sensor datasets, such as those obtained from wearable devices.BRIEF SUMMARY

[0004] According to one example of the present disclosure, a system for recognizing human activity, comprises: a wearable device, the wearable device having a sensor configured to detect motion of a subject wearing the wearable device; and one or more processors and memory collectively configured to: receive data from the sensor; generate persistence images of the data using persistent homology; input the persistence images to a machine learning model trained to classify the persistence images according to an activity; compare the classified activity with a baseline model; and produce a recognition of the activity of the subject based on the comparison.

[0005] In various embodiments of the above example, the one or more processors is further configured to determine an abnormality of the activity based on the sensor data; the one or more processors is further configured to generate an alert when the abnormality is determined; the one or more processors is further configured to predict a neurodegenerative medical condition based on the abnormality; the neurodegenerative medical condition is an early onset of cancer reoccurrence, the system further comprises a display configured to display an analysis of the sensor data; the wearable device comprises the display; the wearable device is a smartwatch; the wearable device comprises at least one of the one or more processors; the one or more processors is configured to develop the baseline model by applying predetermined classifications to the motion of the subject; the machine learning model is a bi-directional LSTM; the sensor is an accelerometer; and / or the accelerometer is configured to produce values of acceleration in three orthogonal dimensions.

[0006] According to another example of the present disclosure, a method for recognizing human activity comprises: receiving data of the human activity from a sensor worn by a person performing the human activity; generating persistence images of the data using persistent homology; inputting the persistence images to a machine learning model trained to classify the persistence images according to an activity; comparing the classified activity with a baseline model; and producing a recognition of the activity of the subject based on the comparison.

[0007] In various embodiments of the above example, the method further comprises developing the baseline model according to predetermined classifications of the human activity for predetermined time intervals; the method further comprises determining an abnormality of the activity based on the data; and / or the method further comprises generating an alert when the abnormality is determined.BRIEF DESCRIPTIONS OF THE SEVERAL VIEWS OF THE DRAWING

[0008] FIG. 1 illustrates a block diagram of an example HAR system.

[0009] FIG. 2 illustrates a flowchart for implementing an HAR system.

[0010] FIG. 3 is a simplified overview of an HAR system.

[0011] FIG. 4 is a flowchart for applying topology and machine learning to sensor data.

[0012] FIG. 5 is a flowchart for topological data analysis.

[0013] FIG. 6 is a flowchart for applying a machine learning model to sensor data.

[0014] FIG. 7 illustrates a Rips complex with varying parameters.

[0015] FIGS. 8A and 8B illustrates point clouds produced in persistent homology.

[0016] FIGS. 9A and 9B illustrate persistence diagrams, and FIG. 9C illustrates a persistence image.

[0017] FIG. 10 illustrates a persistence diagram generated by signal filtration.

[0018] FIG. 11 illustrates further examples of persistence images.

[0019] FIG. 12 is an example of an LSTM cell.DETAILED DESCRIPTIONS OF THE DRAWING

[0020] In view of the above, there is a desire for easier techniques for extracting features from the sensor data. This disclosure thus relates to systems and methods for implementing HAR from wearable device data. Throughout the disclosure, a system will be described that implements topological data analysis (TDA) and persistent homology methods for analysis of sensor data. The TDA allows for the extraction of topological features occurring within the abstract representation of a dataset, the reduction of high-dimensionality of the dataset, and the ability to analyze noisy data. More specifically, the classification of real-world activities is enhanced when derivative statistics and representative persistence images are used as input features for activity classification.

[0021] A block drawing of an embodiment of the wearable device and system is shown in FIG. 1. The system includes a wearable device 100 which may be any of a variety of wearable devices, such as a smartwatch. Alternative to the smartwatch, the wearable device may be other medical devices known in the art, such as heart rate monitors, respiratory monitors, etc. The wearable device 100 can have a sensor, such as an inertial sensor or accelerometer, that can detect activity, such as motion, of the person wearing the device. The sensor 102 may record acceleration or inertial data in X, Y, and Z components (orthogonal directions), for example.

[0022] It is further contemplated that the wearable device 100 can have internal computing and hardware components such as an internal processor 104, display 106, a transmitter 108, receiver 110, I / O device 112, battery 114, and the like. The internal processor 104 can be configured to perform functions such as receiving data from the sensor, generating persistence images, inputting data such as persistence images into machine learning models, classifying persistence images according to machine learning models, comparing classified images and activities with baseline activity models, and producing activity recognition or activity variation recognition based on baseline models. The processor 104 may additionally include or be operatively connected to memory or other storage. The communication components (e g., transmitter 108 and receiver 110) can additionally be configured to communicate with a database 116 and an external processor or processors 118 for storing the sensor data andperforming computations that may be involved, for example, in the described machine learning methods. Further, the database 116 may include recorded data of the wearer, and may include publicly available or shared external data libraries, that contain information such as test data.

[0023] The wearable device 100 along with the systems and methods set forth in this disclosure improve the accuracy at which human activity can be identified and further, can allow for abnormalities in such activities to be identified. The detection of abnormalities can indicate an early onset of a medical condition. For example, in post-surgery management of brain cancer procedures, the reoccurrence of certain cancers such as Glioblastoma can present and be identified early based on abnormalities in certain physical activities. Neurodegenerative conditions such as Parkinson’s disease can similarly be detected early by abnormalities. These abnormalities may be, for example, tremors, weakness in movement, imbalance, etc.

[0024] The abnormalities in patient activities or movements may be detected with the aid of supervised machine learning models. As will be described, persistence images may be the input layers into said machine learning models. The machine learning models may be trained using labeled data, using publicly available datasets which include information on tremors and seizures, and / or using data collected from patient trials. In patient trials, awearable device such as the wearable device 100 may be worn by a patient, to continuously collect training data from the sensor data. At the same time, patients may log symptoms or medical conditions through an app, for example. The patient-logged entries may be compared with the collected sensor data, and labeled at specific time stamps. Additionally, abnormalities may be detected based on activity frequency and likelihood spectra across time periods, comparing the periodicity of an activity (whether the activity is performed faster or slower than expected) or comparing direct persistence diagrams for distance (e.g., a Wasserstein metric).

[0025] FIG. 2 provides a flowchart of an exemplary method of using such a device. First, a user of the device performs a certain physical activity 200. The type of physical activity performed can be open-ended. While certain activities such as running, walking, exercising, etc. are contemplated, more sedentary activities such as sitting, eating chips, sleeping, etc. can also be monitored. The activity performed is tracked by a sensor in step 202. The sensor may be the accelerometer or inertial sensor described in relation to the wearable device 100, but other types of sensors for tracking the physical activity are contemplated. In addition to or instead of motion data, data from the sensor or other input can include vital signs, physiological signals, sleep information, head movement, and other data related to the subject activity or health condition.

[0026] Here, the recording of sensor data may continuously be performed throughout the user’s day, for example. Additionally, the recording of sensor data in step 202 can be a trial period where baseline levels are established. In this case, it is contemplated that the user could be prompted to perform certain activities or confirm that certain activities have been performed. The trial period may further be based on predetermined time intervals of activity. After the sensor data has been recorded or as the data is being recorded, and is stored in a memory, TDA methods and machine learning LSTM models can be applied to the recorded data in step 204. If the recorded sensor data is part of a trial period for establishing baseline activity levels, the data manipulation in step 204 may yield a classification matrix, which relates sensor data to certain activities. Alternatively, this may be an optional step, as baseline data may be collected beforehand or sample baseline data from a pre-established dataset may be used. The manipulated data from step 204 may then yield a determination of a physical activity in step 208, based on a comparison with determined physical activities of the baseline model.

[0027] The select sensor data with the accompanying label of the determined physical activity may then be compared to the baseline activity in step 210. For example, this can be a comparison of persistence images with baseline models, as will be explained in greater detail in the foregoing. Based on the comparison between the sensor data, the determined activity and the baseline data for the type of activity determined, an abnormality can be detected and output. Upon detection of an abnormality, the processor 104 may generate an alert such as a notification to the wearable device 100, a notification to a physician’s electronic device, an alarm, etc. The alert may further contain specific information pertaining to the abnormality, such as whether the abnormality indicates a predicted early onset of a medical condition, such as cancer reoccurrence, as described above. As noted in the above, the abnormality or information about the abnormality may be determined by the supervised machine learning network.

[0028] In sum, these variations can be useful for recognizing human activities and detecting abnormalities that can be developed into a diagnosis for a subject or be used to track the early onset or recurrence of a medical condition. While the above-described methods can be used to develop a personalized baseline model for the activity of a single individual, additionally or alternatively, the above-described methods can be used to develop a broad model of activity classification for a general population.

[0029] FIG. 3 provides a diagram briefly summarizing the flow of signals throughout an exemplary system. As noted with resect to FIG. 1, the wearable device 100 may be a smartwatch as shown and may contain a sensor 102 contained therein. It should be understoodthat this embodiment is merely one example and other devices and techniques for sensing subject data may be suitable. The sensor 102 may collect a continuous steam of sensor data 300 having sliding window embeddings 302. The sliding window embeddings 302 may be batches of data in predetermined sizes. For example, a dim of 200 frames of data which corresponds to 10 seconds of recording time. The sliding window 302 may be a Takens’ embedding with an optimized window size. Each sliding window embedding 302 is analyzed at the processor 104. The processor 104 may be the internal processor locally stored on the wearable device 100, or alternatively the processor could be a remote processor external to the wearable device 100, in which case the sensor data is transmitted to the processor 104 via a transmitter 108 of the wearable device 100. The processor 104 may include or be in communication with the database 116, which may include a training dataset.

[0030] For example the database 116 may include a Wireless Sensor Data Mining (WISDM) dataset, which can be mined from a sample group of subjects and may include accelerometer and gyroscopic data for various activities. The data can be measured through time and include X, Y and Z components for each sensor measure. In an example dataset, 51 subjects were asked to perform activities for 3 minutes each while wearing a smartwatch. The activities of the dataset can include real-world activities such as eating chips, folding clothes, and typing, among others. Further, the data may be grouped into larger classifications of activities, such as hand-oriented activities and eating activities. Many similar classifications of physical activities are contemplated. It is contemplated that methods of resampling and rebalancing may be implemented to correct potential imbalances in the training dataset, such as overrepresentation of an activity.

[0031] Additionally, time-series augmentation may be used to bolster the dataset (e.g., the sensor data stream 300) with synthetically generated training data. The training data can be generated using a variety of augmentation techniques applied to the original data to increase the volume of data severalfold. Augmentation techniques for time-series data such as Gaussian noise, Spherical Coordinate Transformation, Permutation, Amplitude Scaling, and axis swapping may be suitable to improve the robustness of the data.

[0032] In the present example, augmenting the time-series data may help prevent overfitting and underfitting the data, which is caused by insufficient variability in the training data compared to the size of the model. Examples of augmenting the data using Gaussian Noise, Amplitude Scaling and Axis Swapping are described below.

[0033] In an implementation of Gaussian Noise, noise is added to the model to augment the data and reduce overfitting behavior. In such a case, the noise may be added according to the following: x 't= xt+ St (Formula 1)In Formula 1, xtis a given Takens’ embedding and x’t is the noisy version of the Takens’ embedding. The values stare drawn from a Gaussian distribution of st~ (N(p, o)), where p-0 and G=n*var( activity). In other words, in Gaussian Noise augmentation the spread of the noise distribution is determined by a hyperparameter n scaled by the value of the noiseless data.

[0034] In an implementation of Amplitude Scaling, each dimension of data is multiplied, separated by activity label, by a scalar multiple which is drawn from a Gaussian distribution. The Amplitude Scaling method may include: x ’t = axt (Formula 2)

[0035] Finally, in an implementation of Axis Swapping, the method involves rotating the three-dimensional signal acquired from the sensor 102 (e.g., accelerometer and / or gyroscope data). Below is a representation of the un-rotated sensor data: x = {x, y, z} (Formula 3)The data from Formula 3 is rotated and represented as: x’ = {y, x, z] (Formula 4)

[0036] Using the TDA and machine learning systems which will be described in the foregoing, and optionally using training data from the database 116, the processor 104 can determine baseline and / or abnormal physical activity levels. The determinations made at the processor 104 may then be transmitted to and displayed via the display 106. The display 106 may be the local display of the wearable device 100, or may be a remote display for use by a physician, for example. The display 106 may display, for example, an analysis of the subject’s activity including the activities performed and any abnormalities, the persistence images of the sensor data, or an alert or diagnosis of an early-onset condition identified based on abnormalities. However, these uses are not limiting and it is contemplated that other presentations of the sensor data or activity determinations of the subject may be shown on the display 106.

[0037] FIG. 4 provides a detailed flowchart of the analysis and processing of the sensor data (of step 204) for activity classification. First, a step 400 of retrieving the sensor data is performed. This may refer to a sliding window embedding 302 from a continuous stream of sensor data 300 at a predetermined interval. The sensor data of the sliding window embedding 302 may be an input to both a TDA model (step 402) with, for example a 1-D persistenthomology and may also be an input to a deep learning algorithm, such as a Bi-directional LSTM model (step 404) where persistent images (such as the persistence images which are output from the TDA model in step 402) are passed through a pre-trained model While steps 402 and 404 are illustrated as parallel steps in FIG. 4, they are not necessarily parallel in time and the outputs from step 402 may be inputs to step 404. The pre-trained model may be, for example, a frozen pre-trained model having two dense layers. Results from studies conducted by the inventors show that a model using features obtained from TDA, both with and without other deep learning algorithms such as a bi-directional LSTM, performed significantly better than comparative models not implementing TDA

[0038] Persistence images are input to a machine learning model that classifies the persistence images according to a classification of activity. Classification of real-world activities from this dataset can be enhanced when generated persistence images and landscapes are used as input features to a machine learning model for classification. Inputs to a machine learning model for HAR classification may include the accelerometer segments, the accelerometer persistence images, the gyroscope segments, and the gyroscope persistence images. It is contemplated, however, that additional inputs may be used and conversely, a model may be created without all of said inputs. The resulting output may be a one-hot encoded vector of a size corresponding to the number of possible activities, which describes the likelihoods of the input corresponding to one of the activities.

[0039] The persistence images generated from step 402 and the outputs of the bi-directional LSTM from step 404 are passed to the concatenation layer (step 406) where the outputs are concatenated together. The concatenated outputs are then put into fully connected layers (step 408), for example having 128 and 64 nodes with a rectified linear unit (ReLU) activation function, and then to an output layer. For example, a baseline model of 18 activities can have a final output vector size of 18. The baseline model vector is a probability model for evaluating the likelihood that measured sensor data corresponds to an activity. The baseline model vector may further be trained on a labeled training dataset such as a publicly available library (e.g., WISDM dataset). That is, the sliding window embedding 302 of the sensor data 300 is compared with each activity of the baseline model and a probability that the sliding window embedding matches the activity is determined. Thus, the final output layer may then be used to determine a type of activity performed based on activity classification (step 410), from which abnormalities can be detected. Kernel regularizes can be applied to each dense layer to avoid overfitting the model. An LI regularization penalty may be implemented with a regularization factor of 0.001, for example.

[0040] The models and methods used to classify patient data as activities may be similarly used to develop symptom models, which may classify patient data as having symptoms or abnormalities. It is contemplated that a training dataset may be developed by labeling sensor data based on identified or patient-reported symptoms. This dataset may supplement the TDA model such that abnormalities in patient data can be more precisely attributed to patient symptoms. For example, both activity likelihood and symptom presence or absence may be reported for the same data.

[0041] Next, FIG. 5 provides an exemplary flowchart of passing the sensor data segments through a TDA model (step 402 from FIG. 4). In the present embodiment, TDA can be used on wearable sensor data to extract topological features from the data. Briefly, TDA is an approach to quantifying topological shapes that appear within data that provide new insight into underlying features of the source of the data.

[0042] TDA has utility in summarizing high-dimensional data and fitting the data into more useful low-dimensional models without losing important information. In machine learning classification models, the quality of a model is quantified by its Fl score, which is a measure of the model’s precision and recall of correct positive predictions by the model. HAR models that implemented TDA, with and without other deep learning algorithms, achieved Fl scores of 98.13% and 99.15%, respectively, compared with a highest published accuracy of 97% for a bi-directional based LSTM method alone. Thus, there are additional features hidden in the high dimensional shapes of accelerometer and gyroscope data that can aid in predictions of task label from the data. This intrinsic shape of the data can be studied with TDA.

[0043] Within the TDA, persistent homology may be a core algorithm utilized to extract the topological features of a dataset. Briefly, persistent homology is a TDA method developed in algebraic topology for processing and understanding the shape and space of data point clouds, and can be used to describe the birth and death of complex geometric structures such as loops, voids, etc. Loops are formed by expanding the connections between points with increasing distance. Loops within data are born, and die at certain expansion distances. As shown in FIG. 5, initially, a point cloud is generated (step 500) from the acquired sensor data segment. A method of persistent homology (step 502) is applied in analyzing the shape and space of the point cloud from step 500.

[0044] Persistent homology may function as an improved Vietoris-Rips (Rips) complex, which is formed out of a point cloud at a given filtration level, for example, as shown in FIG. 7. A Rips complex is created using a parameter E, where if a distance between two points is less than or equal to E, the two points are connected with an edge. A disadvantage of a Ripscomplex is that the parameter e must be carefully selected. FIG. 7 shows a comparison between a Rips complex where a chosen e is too big 700, a Rips complex where a chosen e is too small 702, and a Rips complex where a chosen e is optimal 704. When a chosen e is too big, it is harder to obtain meaningful features from the complex and if the chosen c is too small, features in the data may be missed as certain points are not connected.

[0045] Persistent homology provides an advantage over this shortcoming of a Rips complex, as all possible values of c are tested. Persistent homology can be described as using slowly enlarging circles around all points in a point cloud, shown for example in FIGS. 8 A and 8B. When these circles are enlarged to the point of overlapping, or put another way, the distance between two points is less than or equal to c, the two circles are connected with a line, such as in FIG. 8A. When the lines form a loop, persistent homology records at which value of e the loop is formed at, known as the “birth” of a loop, such as in birth loop 800. When the loop is closed at another value of c, this is as known as the “death” of the loop, shown for example as death loop 802 in FIG. 8B. This process is known as 1-D persistent homology, as opposed to 2-D persistent homology which is run using expanding three-dimensional objects such as spheres. When persistent homology is run, a high dimensional dataset, such as that obtained from wearable devices, can be transformed to a set of birth (bi) and (di) death coordinates that contain the birth and death of these loops and summarize the topology of the dataset.

[0046] Turning back to FIG. 5, in step 504, persistence images and persistent landscapes using the persistent homology may be generated to enable further visualization of persistence within TDA. The persistence images and persistence landscapes are available to make TDA compatible with machine learning techniques. Persistence diagrams are tools that can aid in the visualization of birth-death pairs. Persistence diagrams may take the form of x-y graphs where the x-axis represents birth coordinates and the y-axis represents death coordinates. If the birth and death coordinates of a simplex z are b, and di, then the persistence of the simplex is d, - bi. Persistence diagrams are not easily input to machine learning algorithms due to their nonvectorized format. Persistence images can address this problem by creating vectorized image representations of persistence diagrams. A persistence image may be a vectorized transformation of the persistence diagram. Effectively, a persistence image is the graph of these coordinate pairs with a smoothing kernel passed over it. Thus, in step 506, a persistence image (such as the persistence images derived from the sliding window embedding 302) is then fed into a pre-trained machine learning algorithm such as a CNN for classification. The persistence image may also be fed into 2D convolution layers, and the fully connected model can developweights that predict activity from a combination of the persistence image and raw sensor data 300.

[0047] An example of birth-death and birth-persistence diagrams for the same data are shown in FIGS. 9 A, 9B, and 9C. As explained above, the x-axes of the coordinate graphs in FIGS. 9A-9C represent the e at which a certain loop is born and the y-axes represent the e at which the death of the loop occurs. FIG. 9A shows an example persistence diagram generated from 200 random points at 1-D persistent homology. FIG. 9B shows the persistence diagram of FIG. 9 A transformed from (bi, di) coordinates to (bi, di-bi) coordinates. FIG. 9C shows a persistence image generated from a kernel being passed over the (bi, di-bi) coordinates. The persistence image in FIG. 9C, for example, as a vectorized image in a 100x100 format, which is suited for machine learning models. FIG. 10 shows a persistence diagram generated from complexes generated by signal filtration. Persistent features are connected as the filtration passes from the lowest to the highest signal value. Further example persistence images from sensor data of various activities are shown in FIG. 11.

[0048] In addition to above-described TDA process 402, the sensor data segments may be applied to a deep learning model such as a bi-directional LSTM 404. While LSTMs allow for local information and past information to be incorporated, bi-directional LSTMS may further improve the performance of LSTMs by using two LSTMs, one going forward and the other going backwards. As a result, a network, such as the neural network, can consider sequence information in both directions, future and past, allowing for more complex local information to improve the learning of sequential data and patterns. LSTMs can provide advantages over one-directional networks, such as RNNs, by having trainable parameters that can determine whether a given input should be forgotten or passed on to the next input in a sequence. An example method for applying the sensor data to a machine learning model is shown in FIG. 6.

[0049] First, in step 600, the data segments are passed to a convolutional layer (or layers, for example, constituting a CNN) with a fixed number of output filters and kernel size, for example 64 output filters and a kernel size of 5. However, other numbers of filters and kernel sizes may be used. The resulting vector from the convolutional layer of step 600 may further be applied to a MaxPooling layer in step 602. A MaxPooling layer can be used to develop an abstracted version of the vector from the convolutional layer in step 600, which is useful in extracting the vector’s most pronounced features. In a MaxPooling layer, n-dimensional vectors are aggregated by taking the maximum value of a window with a certain pool size to decrease the number of trainable parameters. The model resulting from step 602 may then be fed into abidirectional LSTM in step 604. The bidirectional LSTM in step 604 may have, for example, 128 units.

[0050] An example of an LSTM cell, such as from step 604, is shown in FIG. 12. In this example, it is an input gate, is a forget gate, otis an output gate, c is a sigmoid function, wxis a weight used for neurons at a gate x, ht-i is the output of a previous LSTM cell, xtis an input at a time t, bxare biases for a gate x, Ct is a cell state and C ’t are candidates at time t. The LSTM can take in an input JO at time t, and have various gates that are calculated. The gates it, ft, and otare calculated using JLc, a vector containing weights for the neurons in gate x. The following equations can be used for calculating each gate: it= a Wtht^ + UtXt+ bf) (Formula 5)(Formula 6) ot= (r(Woht-1+ U0X0+ bf) (Formula 7)C't— tanh Whf^ + Uxt+ b~) (Formula 8)Ct= ft1Ct-i + L ■ C't (Formula 9) ht= ot■ cr(Cf) (Formula 10)

[0051] Time-series data, for example, from the WISDM dataset of database 116, can be recorded at a fixed frequency such as 20 Hz, and labeled according to a corresponding activity. For each activity, a sliding window embedding may be calculated for sensor data corresponding to that activity. A sliding window embedding allows for the computation of local topological features, rather than purely global features. A sliding window embedding can also improve signal periodicity and the signal -to-noise ratio when an extraneous signal is present, which is useful in computationally heavy methods, such as from TDA. Given an embedding dimension, a time delay or step T, and a sliding window length dim, the sliding window embedding can preserve underlying topological features in data from the sensor data. Additionally given a time series f(t), and an output vector S, a sliding window embedding can be defined as:(dim - 1)T] (Formula 11)

[0052] As a constant sliding window embedding size aids in compatibility with CNNs, a dim of 175 frames (8.75 seconds) may be employed, optimizing for computational time while retaining sufficient information from the sensor data about any possible activity. However, other dim sizes may be alternatively used. Mutual information can in turn be used to calculate an optimal T. When the available data is binned, pt can be defined as a probability that an element is in bin i and pj,k can be defined as a probability that an element x is in the bin j and an element Xj+Tis in bin k. Mutual information for the optimization of r can be defined as:(F ormula 12)

[0053] Therein, r can be optimized by minimizing I(T). Sliding window embeddings and the optimal parameters can be found using standard tools from the Giotto-TDA library, for example. A 1-D persistent homology can be calculated for each sliding window embedding using models such as Giotto-TDA’ s VietorisRipsPersistance and persistence images can be created using Giotto’s Persistenceimage functions. In one example, input features may have segments of size 175 and the persistence images can correspond to each segment.

[0054] As noted above, at the conclusion of running the bi-directional LSTM 604, the resulting output vector is input to the concatenate layer 606.

[0055] While various features are presented above, it should be understood that the features may be used singly or in any combination thereof. Further, it should be understood that variations and modifications may occur to those skilled in the art to which the claimed examples pertain.

Claims

CLAIMS1. A system for recognizing human activity, comprising: a wearable device, the wearable device having a sensor configured to detect motion of a subject wearing the wearable device; and one or more processors and memory collectively configured to: receive data from the sensor; generate persistence images of the data using persistent homology; input the persistence images to a machine learning model trained to classify the persistence images according to an activity; compare the classified activity with a baseline model; and produce a recognition of the activity of the subject based on the comparison.

2. The system of claim 1, wherein the one or more processors is further configured to determine an abnormality of the activity based on the sensor data.

3. The system of claim 2, wherein the one or more processors is further configured to generate an alert when the abnormality is determined.

4. The system of claim 2, wherein the one or more processors is further configured to predict a neurodegenerative medical condition based on the abnormality.

5. The system of claim 4, wherein the neurodegenerative medical condition is an early onset of cancer reoccurrence.

6. The system of claim 1, further comprising a display configured to display an analysis of the sensor data.

7. The system of claim 6, wherein the wearable device comprises the display.

8. The system of claim 1, wherein the wearable device is a smartwatch.

9. The system of claim 1, wherein the wearable device comprises at least one of the one or more processors.

10. The system of claim 1, wherein the one or more processors is configured to develop the baseline model by applying predetermined classifications to the motion of the subject.

11. The system of claim 1, wherein the machine learning model is a bi-directional LSTM.

12. The system of claim 1, wherein the sensor is an accelerometer.

13. The system of claim 12, wherein the accelerometer is configured to produce values of acceleration in three orthogonal dimensions.

14. A method for recognizing human activity, the method comprising: receiving data of the human activity from a sensor worn by a person performing the human activity; generating persistence images of the data using persistent homology; inputting the persistence images to a machine learning model trained to classify the persistence images according to an activity; comparing the classified activity with a baseline model; and producing a recognition of the activity of the subject based on the comparison.

15. The method of claim 14, further comprising: developing the baseline model according to predetermined classifications of the human activity for predetermined time intervals.

16. The method of claim 14, further comprising: determining an abnormality of the activity based on the data.

17. The method of claim 16, further comprising: generating an alert when the abnormality is determined.

Images