Subsequence classification for time series data

Abstract

Methods and systems for classifying subsequences of time series data from a sequence of time series data are disclosed. For a given "focal subsequence" derived from a sequence of time series data, a computer system can construct an "ego-network," which can identify the k-nearest neighbor subsequences for that focal subsequence. These k-nearest neighbor subsequences can be derived from a training sequence of time series data, and may be labeled with corresponding classifications. A computer system can process the focal subsequence and the k-nearest neighbor subsequences using a backbone sub-model, thereby generating a subsequence embedding and nearest neighbor embeddings. Label embeddings can optionally be combined with corresponding nearest neighbor embeddings. These embeddings can be combined and processed using an output sub-model in order to generate a classification corresponding to the focal subsequence.

Description

PATENT Attorney Docket No.: 079900-1440902 Client Reference No.: 7599WO01 SUBSEQUENCE CLASSIFICATION FOR TIME SERIES DATA CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is a PCT application, which claims priority to U.S. Provisional Application No.63 / 510,328, filed on June 26, 2023, which is herein incorporated by reference in its entirety. BACKGROUND

[0002] A time series can comprise a sequence (or “segment”) of data values or observations associated with time values, time stamps, or indices, thereby enabling the time series data values to be organized, displayed, and / or analyzed based on their chronological ordering. Many data sets produces from real-world observations can be organized into time series. An electrocardiogram or ECG (i.e., a graph of heart data) is an example of a time series. From such a time series, a doctor or nurse can deduce useful information about the health of a patient, such as the regularity of the patient’s heartbeat or if the patient has a heart condition such as myocardial ischemia. Another example of a time series is stock ticker data, which may track the price of a stock or commodity over time. Yet another example of time series data is time-based credit and debit card transaction data.

[0003] A time series can comprise subsequences (also referred to as “subsegments”) of time series data. Each of these subsequences may have some meaning within the context of the time series or in the context of real-world observations corresponding to the time series. For example, a typical heartbeat waveform may comprise a P wave, a PR interval, a QRS complex, an ST segment, and a T wave, which may be proceeded and followed by a relatively flat baseline. As such, an ECG time series could comprise a sequence comprising repeating heartbeat subsequences comprising the parts of the heartbeat waveform mentioned above. Alternatively, a subsequence from an ECG time series could comprise part of a heartbeat waveform, e.g., the P wave, the QRS complex, the ST segment, etc., or some combination of these parts. 78601649V.1

[0004] Classifying subsequences of time series data is a time series data mining task that may be performed in various applications. In the context of ECG data, classifying subsequences of time series data corresponding to heartbeats can be used to diagnose various heart conditions. For example, in a normal heartbeat, the ST segment generally matches the baseline ECG signal. However, ST segment depression, in which the ST segment is below the baseline, can be indicative of myocardial ischemia. As such, classifying a subsequence of ECG heartrate data corresponding to an ST segment as “normal” or “anomalous” (e.g., indicating ST depression down-sloping) can be useful for diagnosing heart diseases.

[0005] As another example, classifying subsequences of time series seismographic data can be useful for the early detection of earthquakes, which may be useful as part of early earthquake warning systems. An earthquake often comprises both “pressure waves” and “shear waves.” While pressure waves typically propagate roughly twice as fast as shear waves (and are therefore observed prior to shear waves), they are typically less destructive. By classifying and differentiating between seismographic time series subsequences corresponding to primary waves and ground vibrations corresponding to harmless local activity (such as construction equipment, trains and large trucks, etc.), it may be possible to identify whether an earthquake is about to occur and it may also be possible to estimate its severity.

[0006] However, existing research within the field of time series classification has predominately focused on scenarios where relevant “foreground” subsequences (or foreground subsegments) have already been extracted from a sequence of time series data. In the context of ECG data, this could mean that some processing has already been performed on the ECG data to identify heartbeat waveforms and isolate those heartbeat waveforms for time series classification. In the context of seismographic data, this could mean that some processing has already been performed on the seismographic data to extract potential pressure wave subsequences from the seismographic time series.

[0007] Unfortunately, real-world time series data often contain both foreground subsequences and “background” subsequences (e.g., subsequences that may not be of interest). In such cases, it may be difficult to identify and extract foreground 2 78601649V.1subsequences. In addition, in many real-world applications, it may not be possible to pre-process data and extract foreground subsequences. For example, an earthquake warning system may need to classify subsequences in real-time or close to real-time, and extra time spent extracting subsequences of time series data may limit the effectiveness of such a system.

[0008] As such, conventional time series classification methods that assume prior extraction of foreground subsequences are often inadequate for classifying subsequences of real-world time series data. Embodiments of the present disclosure address these and other problems individually and collectively. SUMMARY

[0009] Embodiments of the present disclosure are directed to novel time series subsequence classification method and systems. Using such methods, subsequences of time series data from an input sequence of time series data can be classified. Methods according to embodiments do not require that foreground subsequences of time series data be identified and extracted from the sequence of time series data prior to subsequence classification. Additionally, methods and systems according to embodiments use novel machine learning and data processing techniques that provide for more accurate subsequence classification with reduced classification noise.

[0010] One such novel aspect of methods according to embodiments are k- nearest neighbor graphs, also known as “ego networks”, which can be used to improve classification accuracy of methods according to embodiments. In a conventional machine learning based subsequence classification system (which do not use k-nearest neighbor graphs), a classifier can take a subsequence as an input and return a classification of that subsequence, e.g., indicating a likelihood that the subsequence belongs to one or more classes.

[0011] By contrast, in methods according to embodiments, for a given subsequence of interest (e.g., a “focal subsequence”), a computer system can identify one or more (i.e., k) nearest neighbor subsequences from a k-nearest neighbor graph. Such nearest neighbor subsequences can be derived from labeled training data. The computer system can create embeddings corresponding to the focal subsequence and each nearest neighbor subsequence, along with label 3 78601649V.1embeddings corresponding to the nearest neighbor subsequences. These embeddings can then be input into a classifier model, which can return a classification corresponding to the focal subsequence.

[0012] In general terms, including these nearest neighbor subsequences and labels as an input to a classification model provides the classification model with additional information that may be useful for classifying the focal subsequence. For example, if the five most similar subsequences (e.g., the nearest neighbors) to the focal subsequences are all labeled as “anomalous” subsequences (e.g., corresponding to anomalous heart data, anomalous seismographic data, etc.), that might imply that the focal subsequence is also an anomalous subsequence. By using this information, a computer system may produce a classification that more accurately classifies the focal subsequence than e.g., classifying the focal subsequence without using nearest neighbor information. As such k-nearest neighbor graphs (also referred to as “ego networks”) can enable more accurate classification of time series subsequences. Embodiments of the present disclosure also relate to novel methods to effectively construct ego networks and time series subsequence graphs (e.g., comprising the ego networks of all subsequences in a given time series), so that they can be used to improve subsequence classification accuracy, as described above.

[0013] Further, some embodiments of the present disclosure relate to novel methods of enforcing temporal consistency in classifications of adjacent subsequences. In general, if each subsequence in a sequence of time series data is classified, the resulting classifications may themselves form a “classification sequence.” Such a classification sequence may exhibit variations in classifications and erroneous classifications. As described in more detail further below, in order to address this “classification noise,” a computer system performing methods according to embodiments can detect “onset subsequences” (e.g., indicative of a new classification regime) and majority subsequence classifications within subsequence classification subsets. The computer system can replace or overwrite subsequence classifications with their corresponding majority subsequence classifications, thereby enforcing temporal consistency. As shown in table 1 of FIG.20, these novel temporal post-processing methods improve the accuracy of time series subsequence classification methods according to embodiments. 4 78601649V.1

[0014] Embodiments of the present disclosure can be used to accurately classify subsequences of time series data in a variety of useful applications. For example, methods according to embodiments can be used to classify subsequences of ECG data as normal or anomalous for the purpose of detecting heart disease. As another example, embodiments of the present disclosure can be used to classify time series seismography data as corresponding to pressure waves or other sources of vibration (e.g., construction or heavy machinery) for the purpose of early earthquake warning systems. Embodiments of the present disclosure can also be used to implement “subsequence classification as a service,” e.g., enabling a server computer to perform subsequence classification or other time series data analysis (which may be based on subsequence classification) on behalf of client computers.

[0015] However, it should be understood that embodiments of the present disclosure are directed to methods and systems for performing time series data analysis and subsequence classification. While these methods and systems can be applied to a variety of real-world application, several examples of which are provided herein, it should be understood that embodiments of the present disclosure are not directed to e.g., the diagnosis of heart disease, earthquake prediction, radar pulse detection, or any of the other exemplary applications described herein.

[0016] In more detail, one embodiment is directed to a method for classifying subsequences of time series data using a machine learning model, which can comprise at least an output sub-model. A computer system can receive a sequence of time series data comprising one or more subsequences. The computer system can construct a k-nearest neighbor graph based on the one or more subsequences. The computer system can produce one or more subsequence classifications corresponding to the one or more subsequences. For each subsequence of the one or more subsequences, the computer system can determine one or more nearest neighbor subsequences corresponding to that subsequence using the k-nearest neighbor graph. The computer system can produce a subsequence embedding and one or more nearest neighbor embeddings based on the subsequence and the one or more nearest neighbor subsequences. The computer system can produce a combined embedding by combining the subsequence embedding and the one or more nearest neighbor embeddings. The computer system can then produce a subsequence classification using the output sub-model and the combined 5 78601649V.1embedding, thereby producing one or more subsequence classifications corresponding to the one or more subsequences.

[0017] Some embodiments are directed to novel methods of post-processing time series classification data. For example, in some embodiments, the one or more subsequence classifications described above could comprise a plurality of subsequence classifications. The computer system can then identify a plurality of onset subsequence classifications from the plurality of subsequence classifications. The computer system can identify a plurality of subsequence classification subsets based on the plurality of onset subsequence classifications and a predetermined window size value. Each subsequence classification of the plurality of subsequence classification subsets can comprise an onset subsequence classification and a number of subsequence classifications proportional to the predetermined window size value. The computer system can determine a majority subsequence classification for each subsequence classification subset. The computer system can replace each subsequence classification of the plurality of subsequence classifications with a corresponding majority subsequence classification corresponding to one or more subsequence classification subsets to which the subsequence classification belongs. In this way, the computer system can produce a plurality of post-processed subsequence classifications.

[0018] Another embodiment is directed to a method for training a machine learning model comprising at least an output sub-model to classify subsequences of time series data. A computer system can receive a training sequence of time series data comprising a plurality of training subsequences. The computer system can construct a k-nearest neighbor graph based on the plurality of training subsequences. The computer system can perform an iterative training process until a terminating condition has been met. The iterative training process can comprise a plurality of training rounds. In each training round, the computer system can sample a batch of training subsequences. Each batch of training subsequences can comprise one or more training subsequences from the plurality of training subsequences. The computer system can produce one or more subsequence classifications corresponding to the batch of training subsequences. For each training subsequence of the batch of training subsequences, the computer system can determine one or more nearest neighbor subsequences corresponding to the 6 78601649V.1training subsequence using the k-nearest neighbor graph and a training index corresponding to the training subsequence. The computer system can produce a subsequence embedding and one or more nearest neighbor embeddings based on the training subsequence and the one or more nearest neighbor subsequences. The computer system can produce a combined embedding by combining the subsequence embedding and the one or more nearest neighbor embeddings. The computer system can produce a subsequence classification using the output sub- model and the combined embedding. In this way, the computer system can produce one or more subsequence classifications corresponding to the batch of training subsequences. The computer system can generate one or more loss values based on the one or more subsequence classifications and one or more training labels corresponding to the batch of training subsequences. The computer system can update the machine learning model based on the one or more loss values. The computer system can perform this iterative training process until the terminating condition has been met.

[0019] Another embodiment is directed to a computer system comprising a processor and a non-transitory computer readable medium coupled to the processor. The non-transitory computer readable medium can comprise code or instructions, executable by the processor for implementing methods according to embodiments, including the methods described above. TERMS

[0020] A “server computer” may include a powerful computer or cluster of computers. For example, a server computer can include a large mainframe, a minicomputer cluster, or a group of servers functioning as a unit. In one example, a server computer can include a database server coupled to a web server. A server computer may comprise one or more computational apparatuses and may use any of a variety of computing structures, arrangements, and compilations for servicing the requests for one or more client computers.

[0021] A “memory” may include any suitable device or devices that may store electronic data. A suitable memory may comprise a non-transitory computer readable medium that stores instructions that can be executed by a processor to implement a desired method. Examples of memories include one or more memory 7 78601649V.1chips, disk drives, etc. Such memories may operate using any suitable electrical, optical, and / or magnetic mode of operation. A “memory buffer” can include a region of memory used to temporarily store data.

[0022] A “processor” may include any suitable data computation device or devices. A processor may comprise one or more microprocessors working together to accomplish a desired function. The processor may include a CPU that comprises at least one high-speed data processor adequate to execute program components for executing user and / or system generated requests. The CPU may be a microprocessor such as AMD’s Athlon, Duron and / or Opteron; IBM and / or Motorola’s PowerPC; IBM’s and Sony’s Cell processor; Intel’s Celeron, Itanium, Pentium, Xenon, and / or XScale; and / or the like processor(s).

[0023] A “data set” may include any set of one or more “observations” or “data values.” A “data value” can include any data element. A data value can comprise a “data vector,” one or more values (represented in vector form) corresponding to a data element or observation. “Time series data” may comprise data for which each data value has an associated “time value,” “time stamp,” or “time indicator.” Such values can represent or correspond to the time at which the corresponding data value was observed or collected. A “time series data value” may include any data value corresponding to time series data. Time series data is often ordered chronologically based on time values. A “segment” (sometimes referred to as a “sequence”) of time series data can comprise a set of (usually sequential) time series data observations. A “subsegment” or “subsequence” of time series data can comprise a subset of (usually sequential) time series data observations derived from a segment.

[0024] “Sampling” may include any process or method used to collect data values. Sampling can be used to collect data values from an existing data set. The act of sampling may result in a “sample,” one or more data values collected from the data set during sampling. Data sets can be sampled via a variety of means. For example, “random sampling” can involve sampling data values from a data set randomly.

[0025] A “window” or “window of data” may include any number of contiguous data elements from a data set. A “window” may be defined by a starting data value 8 78601649V.1and an ending data value, such that the window contains all data values between the starting data value and ending data value (and optionally the starting data value and ending data values themselves). “Window sampling” can be used to sample data values contained within a window of data.

[0026] A “motif” or “pattern” may refer to some data within a dataset that repeats throughout the dataset. By contrast, an “anomaly” or “discord” may refer to some data within a dataset that is different from the rest of the data in the dataset.

[0027] A “Matrix Profile” can comprise a vector that stores the Euclidean distance (or e.g., the z-normalized Euclidean distance) between each subsequence in a time series and its nearest neighbor, i.e., the most similar subsequence based on Euclidean distance between the two subsequences. The Matrix Profile can be used to identify motifs and discords in time series, e.g., by identifying subsequences with nearest neighbor distances that are either unusually high or unusually low.

[0028] "Machine learning" may refer to artificial intelligence processes in which software applications may be trained to make accurate predictions through learning. The predictions can be generated by applying input data to a predictive model formed by performing statistical analyses on aggregated data. A model can be trained using training data, such that the model may be used to make accurate predictions. A prediction can comprise, for example, a classification of the subject of an image, a recommendation of a book, movie, or television show, etc.

[0029] A “machine learning model” can refer to a software module configured to be executed by one or more processors to provide a classification or numerical value of a property of one or more samples. One type of machine learning model is a “supervised learning model.” Supervised learning models may use different machine learning approaches and techniques including analytical learning, artificial neural network, backpropagation, boosting (meta-algorithm), Bayesian statistics, case- based reasoning, decision tree learning, inductive logic programming, Gaussian process regression, genetic programming, group method of data handling, kernel estimators, learning automata, learning classifier systems, minimum message length (decision trees, decision graphs, etc.), multilinear subspace learning, naive Bayes classifiers, maximum entropy classifiers, conditional random fields, probably approximately correct learning (PAC) learning, ripple down rules, knowledge 9 78601649V.1acquisition methodologies, symbolic machine learning algorithms, subsymbolic machine learning algorithms, minimum complexity machines (MCM), random forests, ensembles of classifiers, ordinal classification, data pre-processing, handling imbalanced datasets, statistical relational learning, or Proaftn, a multicriteria classification algorithm. A machine learning model may include linear regression, logistic regression, deep recurrent neural network (e.g., long short term memory, LSTM), hidden Markov model (HMM), linear discriminant analysis (LDA), k-means clustering, density-based spatial clustering of applications with noise (DBSCAN), random forest algorithm, support vector machine (SVM), or any other machine learning model described herein. Supervised learning models can be trained in various ways using various cost / loss functions that derive error from known labels (using e.g., least squares and absolute difference from known classification) and various optimization techniques, such as backpropagation, steepest descent, conjugate gradient, and Newton and quasi-Newton techniques.

[0030] The process of “training” a machine learning model may include any steps used to prepare a machine learning model to perform some task. Often training involves determining or optimizing a set of “parameters” (which characterize the machine learning model) that result in acceptable model performance. Training can be performed in a series of “training rounds” during which training data is used to update the parameters of the machine learning model, for example, based on a loss value.

[0031] A “loss value” or “error value” may include any value that indicates the deviation between a result of some process, method, or function and an expected, desired, or correct result. For example, if a machine learning model can detect anomalies in a data set of data values, 17 of which are anomalous, if the machine learning model only detects 15 the 17 anomalous data values, the loss value could comprise, e.g., 2 (17 – 15). Loss values can be used to train and evaluate the training of machine learning models, e.g., by optimizing machine learning model parameters by minimizing the loss value, and using processes such as stochastic gradient descent or backpropagation.

[0032] A “hyperparameter” can include any value used to configure a machine learning model that is external to the machine learning model. Typically, a 10 78601649V.1hyperparameter is set, and is not estimated or determined from the training data that is used to train the machine learning model.

[0033] A machine learning model may comprise multiple “sub-models” or “layers,” which may refer to parts of a larger machine learning system. For example, a machine learning model could comprise a long short-term memory layer (which itself can comprise multiple layers), in addition to an attention layer and a linear layer. Layers can sometimes be organized in series, such that the input to a machine learning system is processed by a first set of layers, which produces an output that is then processed by a subsequent set of layers, and so forth until the output of the machine learning model is produced by the final layer in the series.

[0034] “Outlier detection” may include any process or method used to detect outliers. For example, outlier detection can be used to detect outlier data values in a data set. An “outlier” may include anything that deviates from what is standard, normal, or expected, particularly when it deviates to an unusual or extreme degree. An outlier data value can correspond to a particular outlier event. An “outlier score” may include any value generated during an outlier detection process. An outlier score can indicate the likelihood that a data value comprises an outlier. For example, an outlier score of “0.81” can indicate that there is an 81% chance that a data value is an outlier. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] FIG.1 shows an exemplary time series classification problem.

[0036] FIG.2 shows an exemplary time series semantic segmentation problem.

[0037] FIG.3 shows an exemplary time series subsequence classification problem.

[0038] FIG.4 shows an exemplary system for classifying subsequences of time series data according to some embodiments.

[0039] FIG.5 shows an exemplary machine learning model that can be used to classify subsequences of time series data according to some embodiments. 11 78601649V.1

[0040] FIG.6 shows some exemplary nearest neighbor relationships between subsequences of time series data.

[0041] FIG.7 shows an exemplary k-nearest neighbor STOMP method according to embodiments.

[0042] FIG.8 shows an exemplary recurrent neural network backbone sub- model according to some embodiments.

[0043] FIG.9 shows an exemplary residual neural network backbone sub- model according to some embodiments.

[0044] FIG.10 shows an exemplary transformer backbone sub-model according to some embodiments.

[0045] FIG.11 shows an exemplary residual block according to some embodiments.

[0046] FIG.12 shows an exemplary transformer block according to some embodiments.

[0047] FIG.13 shows an exemplary subsequence classification model according to some embodiments.

[0048] FIG.14 shows a flowchart corresponding to an exemplary subsequence classification inference method according to some embodiments.

[0049] FIG.15 shows pseudocode corresponding to an exemplary subsequence inference method according to some embodiments.

[0050] FIG.16 shows a flowchart corresponding to an exemplary subsequence classification training method according to some embodiments.

[0051] FIG.17 shows pseudocode corresponding to an exemplary subsequence classification training method according to some embodiments.

[0052] FIG.18 shows a flowchart corresponding to an exemplary temporal consistency post-processing method according to some embodiments.

[0053] FIG.19 shows a visual example of an exemplary classification post- processing method according to some embodiments. 12 78601649V.1

[0054] FIG.20 shows two tables corresponding to experimental results.

[0055] FIG.21 shows an exemplary computer system according to some embodiments. DETAILED DESCRIPTION

[0056] As described above, some embodiments of the present disclosure are directed to methods and systems for performing time series classification. As such, in order to facilitate a better understanding of embodiments of the present disclosure, various time series classification problems are described below with reference to FIGs.1-3. More information about various time series classification problems can be found in references [1-4].

[0057] FIG.1 depicts on exemplary time series classification problem 102. As described above, a commonly studied time series classification problem relates to the identification of subsequences of time series data that have been extracted from a larger time series [2, 3]. The exemplary subsequences of time series data depicted in FIG.1 have been extracted from time series accelerometer data, e.g., from a pedometer or a wearable fitness device. Such accelerometer data may reflect physical activities performed by a person wearing the personal fitness device or in possession of the pedometer. For example, one time series subsequence may correspond to a period of time during which the person was walking, and a different time series subsequence may correspond to a period of time during which the person was running. The exemplary time series classification problem 102 can involve determining whether a test time series subsequence 106 corresponds to one class (e.g., “walking”) or another class (e.g., “running”).

[0058] Such a problem can be solved using machine learning. For example, if a data scientist has access to training time series subsequences 104 with known labels (e.g., walking labels and running labels), the data scientist could train a machine learning module using the training time series subsequences 104 during a training phase 108. After training, the machine learning model can be used to classify the test time series subsequence 106 during a testing phase 110, at which time the machine learning model can return a predicted class label corresponding to the test time series subsequence 106, e.g., “running”, indicating that the test time 13 78601649V.1series subsequence 106 likely corresponds to a period of time during which a person was running.

[0059] As discussed briefly above, this time series classification problem is widely studied, but assumes that subsequences of interest have already been identified and extracted from larger time series sequences. However, in many real- world time series data sets, subsequences of interest may not have been pre- extracted, and additional processing may be necessary to extract such subsequences, which may be difficult to implement and / or inefficient, for example, involving a data scientist manually identify and extracting subsequences of interest. By contrast, in subsequence classification methods according to embodiments, test subsequences of time series data do not need to be extracted from a test sequence of time series data in order to be classified, which may be preferable in some applications.

[0060] FIG.2 depicts an exemplary time series semantic segmentation problem 202 [4], which is another related type of time series classification problem. The semantic segmentation problem generally involves identifying boundaries between “regimes” of a time series. Each regime may comprise subsequences of time series data corresponding to different classes. The objective or solution to the semantic segmentation problem is generally the identification of the border between regimes, along with the classification of time series subsequences on either side of the border.

[0061] The exemplary time series segmentation problem 202, like the exemplary time series classification problem 102 from FIG.1, can be solved using applications of machine learning. A training time series 204 can be segmented into multiple regimes, including a walking regime 208 and a running regime 210. The ground truth labels for the training data can comprise the location of the boundary between the regimes and the corresponding class of each regime. The trained machine learning model can identify boundaries separating regimes and can classify each regime (e.g., as a walking regime or a running regime). Unlike the exemplary time series classification problem 102 from FIG.1, the exemplary semantic segmentation problem 202 does not require the identification and extraction of time series subsequences. 14 78601649V.1

[0062] Some embodiments of the present disclosure can address the “subsequence classification problem”, and an exemplary subsequence classification problem 302 is depicted in FIG.3. Similar to semantic segmentation, the subsequence classification problem can involve evaluating a segment of time series data, rather than individual subsequences of time series data from that segment. In FIG.3, a testing time series 310 can comprise multiple test subsequences, including test subsequences 306. Exemplary subsequence classification problem 302 can involve classifying these test subsequences 306. However, the testing time series 310 can also comprise “background subsequences” such as background subsequence 314, which can comprise a subsequence of time series data that is unrelated to classes of interest.

[0063] As in the examples of FIGs.1 and 2, the exemplary testing time series 310 (and a training time series 308) can comprise fitness accelerometer data, and subsequences of time series data, including the test subsequences 306, can correspond to periods of running or walking by a person wearing a wearable fitness device. Subsequences of time series data that do not correspond to walking or running classes can be considered background subsequences corresponding to “background classes.”

[0064] Like the examples of FIGs.1 and 2, machine learning can be used to address the exemplary time series subsequence classification problem 302 depicted in FIG.3. Training subsequences of time series data 304 from a training time series 308 can be labeled as walking or running subsequences, and their location and length in the training time series 308 can be recorded. The training time series, labels, and the locations and lengths of training subsequences 304 can be used as training data to train a machine learning model. After training, the trained machine learning model can be used to classify test subsequences from the testing time series 310, e.g., by predicting the label corresponding to each test subsequence 306, thereby determining whether each test subsequence 306 comprises a walking subsequence, a running subsequence, or a background subsequence.

[0065] The testing time series 310 of FIG.3 specifically identifies three test subsequences 306 among background subsequences, i.e., test subsequences 306 corresponding to classes of interest. However, a time series such as the testing time 15 78601649V.1series 310 can comprise a large number of test subsequences, which can be defined over the entire testing time series 310 using a sliding window 312. In testing time series 310, most of these test subsequences comprise background subsequences, such as background subsequence 314. Some of these test subsequences can comprise “mixed” subsequences, such as mixed subsequence 316, which can contain time series data corresponding to a class of interest as well as background time series data.

[0066] As such, a machine learning model (or other subsequence classification system) can classify all test subsequences in testing time series 310 (instead of just the three indicated test subsequences 306) by moving a sliding window 312 across the testing time series 310 and classifying each test subsequence contained in the sliding window 312. The result of this operation can comprise a sequence of classifications spanning the testing time series 310. Some of these classifications, corresponding to sliding window positions containing background subsequences, may comprise background classifications. Other classifications, corresponding to sliding window positions containing running subsequences or walking subsequences (e.g., test subsequences 306) may comprise running or walking classifications. Other classifications, corresponding to, e.g., sliding windows that partially contain background subsequences and partially contain walking or running sequences may indicate some degree of classification ambiguity, or may indicate multiple classifications. For example, a classification could indicate e.g., “75% chance of walking, 15% chance of running, 10% chance of background”, indicating the predicted probability that the subsequence belongs to either of the classes of interest or comprises a background subsequences.

[0067] As stated above, some methods according to embodiments are directed to addressing time series subsequence classification problems, such as the exemplary time series subsequence classification problem 302 depicted in FIG.3. Methods according to embodiments can also use machine learning for this purpose. However, unlike the general machine learning solution described above, embodiments of the present disclosure can use novel k-nearest neighbor techniques and a novel machine learning architectures to enhance classification accuracy, as described in more detail further below. 16 78601649V.1

[0068] It should be understood that while FIGs.1-3 depict univariate time series, each of the exemplary time series classification problems can be formulated and extended to multivariate time series data. Similarly, embodiments of the present disclosure can be used to classify multivariate subsequences derived from multivariate time series. It should likewise be understood that although reference is made to the classification of fitness accelerometer data, these time series classification problems are applicable to any variety of time series data (e.g., radio frequency data, ECG data, seismographic data, etc.,), and similarly, methods according to embodiments can be used to classify any variety of subsequences of time series data.

[0069] A system for performing time series subsequence classification methods according to embodiments is summarized with reference to FIG.4. Such a system can include a computer system 404, which can generally perform time series subsequence classification methods described herein. The computer system 404 can instantiate, store, train, and use a machine learning model 408 to perform methods according to embodiments.

[0070] Machine learning models according to embodiments are described in more detail below with reference to FIGs.5 and 13. Generally however, machine learning model 408 can comprise at least an output sub-model 410, which can be used to produce subsequence classifications corresponding to one or more subsequences of time series data. Each subsequence classification can indicate one or more likelihoods that a corresponding subsequence is a member of one or more classes. For example, for time series fitness data (as described above with reference to FIGs.1-3), the output sub-model 410 can be used to produce subsequence classifications indicating one or more likelihoods (e.g., probabilities) that corresponding subsequences comprise walking subsequences or running subsequences.

[0071] The machine learning model 408 can also comprise one or more backbone sub-models 412. The computer system 404 can use the one or more backbone sub-models 412 to generate subsequence embeddings and nearest neighbor embeddings, which, in general terms, can be used by the output sub-model 410 in order to produce the subsequence classifications. Various types of backbone 17 78601649V.1sub-models and backbone sub-model architectures can be used, such as Long Short-Term Memory Network (LSTM) architectures, recurrent neural network (RNN) architectures, Gated Recurrent Unit Network (GRU) architectures, Residual Network (ResNet) architectures, and transformer architectures. Exemplary backbone sub- models are described in more detail below with reference to FIGs.8-10.

[0072] Methods according to embodiments can be used for a variety of purposes, in a variety of contexts, and using a variety of different types of time series data, and a non-limiting list of examples is provided below. However, it should be understood that embodiments of the present disclosure are directed to methods and systems for performing time series subsequence classification, and not for the use of time series subsequence classification within a particular context. For example, while methods according to embodiments could be used to classify subsequences of ECG data and aid in the diagnosis of heart disease based on such classifications, methods according to embodiments comprise methods for classifying subsequences of time series data, not methods for diagnosing heart diseases.

[0073] For example, as implied above, classifying subsequences of time series heartbeat data from ECG data can be used to diagnose different forms of heart disease. Computer system 404 can perform methods according to embodiments using machine learning model 408 to classify subsequences of ECG data, e.g., in order to determine if subsequence of ECG data are anomalous (indicating abnormal function of a patient’s heart, e.g., myocardial ischemia) or normal (indicating normal heart function). Such classifications could be used to diagnose a patient. An action could then be taken to medically treat the patient account to the diagnosis.

[0074] As another example, classification of wave information in time series seismology data could be used to predict the severity and estimated onset of earthquakes. Earthquakes often comprise both pressure waves and slower moving (but more destructive) shear waves. Because of the difference in speed, pressure waves can be detected before shear waves, and may be used to predict future shear waves. In such an application, the computer system 404 could classify subsequences of seismographic data using methods according to embodiments and machine learning model 408, in order to determine if subsequences of 18 78601649V.1seismographic data correspond to pressure waves or benign ground vibrations (e.g., due to construction, heavy machinery, trains, etc.), in order to predict shear waves as part of an earthquake warning system. An action could then be taken to provide electronic messages that warn the affected population regarding a possible earthquake.

[0075] Methods according to embodiments can be used to implement “time series classification as a service”, in which the computer system 404 classifies subsequences of time series data (or performs other time series data analysis based on such classification) on behalf of requestor(s) 402. In general terms, one or more subsequences in a time series could be representative of one or more events (e.g., one or more heartbeats, one or more radar pulses, one or more seismographic waves, etc.), and the computer system 404 could perform methods according to embodiments in order to qualify or quantify these events for requestor(s) 402. In such a case, the computer system 404 can receive request(s) to qualify or quantify the one or more events represented by the sequence of time series data from the requestor(s) 402. After performing subsequence classification, the computer system 404 can provide one or more subsequence classifications to the requestor(s) 402 responsive to the request(s).

[0076] The requestor(s) can then take any appropriate action (e.g., a manual or automated action) according to any received subsequence classifications. For example, communications may be sent out in response to determining that time series data has a particular subsequence of a classification of interest, or automated actions (e.g., automatically initiate a defense mechanism upon detection that time series data has a particular subsequence of interest (e.g., data indicating a type of cyberattack).

[0077] In some cases, requests may include any relevant sequences of time series data needed to service those requests, e.g., sequences of time series data that can be classified by the computer system 404 using machine learning model 408. In other embodiments, the computer system 404 may retrieve relevant sequences of time series data from a database 406 (or other data structure) in order to service requests from requestor(s) 402. 19 78601649V.1

[0078] As an example, in the context of ECG data, a requestor 402 could comprise a computer system associated with a medical research center, which submits a request to classify ECG data for the purpose of diagnosing heart diseases. The computer system 404 could classify subsequences of ECG data using machine learning model 408 and return those subsequences to the requestor 402. A doctor working at the medical research center could use the subsequence classifications to diagnose a patient.

[0079] As another example, in the context of seismographic wave classification, the requestor 402 could comprise a computer system associated with a central geological station and the computer system 404 could comprise a computer system associated with a seismographic monitoring station. The requestor 402 could transmit requests to classify subsequences of time series seismographic data for the purpose of an earthquake warning system. The computer system 404 can retrieve a relevant sequence of time series seismographic data, e.g., from a seismographic connected to the computer system 404, classify subsequences of seismographic data, and transmit the classifications to the requestor 402 responsive to the request. Based on these classifications, the geological station requestor 402 can issue an early earthquake warning, if applicable.

[0080] As described above, requestor(s) 402 can comprise requestor computer system(s). Such requestor computer system(s) can comprise client computer(s) and the computer system 404 may comprise a server computer, which requestor(s) 402 may communicate with over a communication network 414. A communication network such as communication network 414 can take any suitable form, and may include any one and / or the combination of the following: a direct interconnection; the Internet; a Local Area Network (LAN); a Metropolitan Area Network (MAN); an Operating Missions as Nodes on the Internet (OMNI); a secured custom connection; a Wide Area Network (WAN); a wireless network (e.g., employing protocols such as, but not limited to a Wireless Application Protocol (WAP), I-mode, and / or the like); and / or the like. Messages between computers and devices in the system of FIG.4 and / or over communication network 414 may be transmitted using a secure communication protocol, such as, but not limited to, File Transfer Protocol (FTP); HyperText Transfer Protocol (HTTP); Secure HyperText Transfer Protocol (HTTPS); Secure Socket Layer (SSL), ISO (e.g., ISO 8583) and / or 20 78601649V.1the like. Any suitable communication protocol can be used to communicate over the communication network 414, e.g., for the purpose of creating one or more communication channels. A communication channel may, in some instances, comprise a secure communication channel, which may be established in any known manner, such as through the use of mutual authentication, a session key, and establishment of a Secure Socket Layer (SSL) session.

[0081] As described in more detail below with reference to FIG.21, a computer system such as computer system 404 can comprise a processor and a non-transitory computer readable medium (e.g., a hard drive) coupled to the processor. The non-transitory computer readable medium can comprise code or instructions, executable by the processor for performing methods according to embodiments described herein.

[0082] For simplicity of illustration and exposition, a certain number of devices, entities, and components are shown in FIG.4. It should be understood however that systems according to embodiments of the present disclosure can include more than one of each device, entity, component, computer system, etc. In addition, some systems according to embodiments may include a lesser number of devices, entities, and / or components or a greater number of devices, entities, and / or components than those shown in FIG.4. For example, computer system 404 may comprise a distributed computing system comprising several computers collectively performing methods according to embodiments. Likewise, there may be multiple databases 406 from which computer system 404 retrieves relevant sequences of time series data and multiple communication networks 414 over which communication system 404 communicates with requestor(s) 402.

[0083] Some methods according to embodiments are described below with reference to FIG.5, which shows an exemplary machine learning model 502 comprising an output sub-model 524 and one or more backbone sub-model(s) 514. As described above, a computer system can use a machine learning model such as machine learning model 502 to perform methods according to embodiments. As depicted in FIG.5, machine learning model 502 can be used to classify a “focal subsequence” 504, thereby producing a focal subsequence classification 526. 21 78601649V.1

[0084] In order to orient the reader and facilitate a better understanding of embodiments of the present disclosure, a basic framework of machine learning is presented herein.

[0085] Many machine learning models use a data set as an input in order to produce an output, which may also comprise a data set. Inputs to a machine learning model are sometimes referred to as “features,” which can be organized in “feature vectors.” The nature of the inputs and outputs generally depend on the task performed or problem solved by the machine learning model. For example, an English to Spanish machine learning translation system may receive input data comprising English text and generate output data comprising a Spanish text translation of the input English text.

[0086] Machine learning models are typically governed by parameters, which often relate to the mapping between inputs and outputs. In a neural network, for example, parameters can comprise “weights,” which can correspond to the relative weight given to the output of one neuron on the calculation of the output of another neuron. Typically, changing the parameters of the machine learning model will change how the machine learning model performs its particular task. For example, for a given set of parameters and an input “A,” a machine learning model may produce an output “B” (e.g., a classification of the input “A”), however, for a different set of parameters and the same input “A,” the machine learning model may produce a different output “C” (e.g., a different classification of the input “A”). Generally, some sets of parameters will typically result in better performance at a given task or problem than other sets of parameters for a given machine learning model. For example, for a machine learning translation system, some sets of parameters may result in more accurate translations than other sets of parameters.

[0087] While it is often computationally infeasible to identify the “best” set of parameters for given task, the process of “training” a machine learning model typically involves determining a satisfactory set of parameters for a machine learning model for its given task. For example, for a machine learning translation system, training may involve determining a set of parameters that enables a machine learning translation system to translate e.g., a page of text with few grammatical or translation errors. The performance of a machine learning model can be quantified 22 78601649V.1using statistics such as “loss terms” or “loss values” (also referred to as “error terms” or “error values”), scores, or other numerical quantifiers. The process of training can involve determining parameters that minimize (or maximize) these statistics, thereby identifying parameters that cause the machine learning model to achieve satisfactory performance at its given task.

[0088] There are many ways in which training based on error terms can be performed, and an exhaustive list will not be provided. Some methods involve the use of optimization techniques such as “gradient descent” or “stochastic gradient descent.” The gradient of an error term with respect to model parameters can be used to determine the change in model parameters that results in the greatest immediate reduction of the error term. By performing numerous “training rounds” and updating the model parameters in view of gradients in each training round, corresponding error terms can be reduced. Since the error terms generally relates to the model’s performance (in that a smaller error term typically corresponds to a better performing machine learning model), iteratively updating model parameters by reducing error terms results in better performing machine learning models.

[0089] There are a variety of forms of machine learning, including unsupervised, supervised, and semi-supervised machine learning. They will not be described in detail, however supervised learning is summarized briefly below. In supervised learning, a “training data set” can be used to train a machine learning model. Often, this training data set can comprise input data and corresponding desired or expected output data. For example, in the context of a machine learning English to Spanish translation system, the training data set could comprise English language sentences and their corresponding Spanish language translations. As another example, in the context of a classifier, “labels” corresponding to input data can be paired with that input data and used as a training data set.

[0090] Such training data sets enable the derivation of statistics (such as error terms or loss values) that can be used to train the machine learning model. For example, input data from a training data set can be input into an untrained or partially trained machine learning model, producing a model output. Because the ideal or expected model output is already contained in the training data set, the ideal or expected model output can be compared against the actual model output in order to 23 78601649V.1derive a loss or error term. For example, if an untrained or partially trained translation system produces a translation that is very similar to the ideal or expected translation, then the resulting error term may be small. By contrast, if the untrained or partially trained translation system produces a translation that is very dissimilar to the ideal or expected translation, then the resulting error term may be large.

[0091] Regardless, model parameters can be updated in view of any error terms (e.g., based on gradients as described above) during training, in order to reduce the error terms across the entire training data set and thereby train the machine learning model. Often, training is performed until some terminating condition has been met. Such a terminating condition can comprise a set number of training rounds or “epochs,” which can comprise passes through the entire training data set. Another example terminating condition is convergence, in which the training ends when model parameters or error terms become unchanging (or only change to a small degree) in each successive training round. Provided that the training data set is relatively representative of data the model might encounter in the real world, using supervised training can result in machine learning models that are effective or accurately perform their respective tasks.

[0092] However, it should be understood that the term “machine learning model” can generally refer to any model with parameters that can be tuned, automatically or otherwise, to better fit the model to a particular dataset or task, and that techniques such as the use of multiple training rounds or stochastic gradient descent are not necessary for something to be considered machine learning. Even well-established statistical techniques such as linear, polynomial, exponential, or logistic regression can be considered machine learning in some contexts.

[0093] Referring back to FIG.5, as described above, a computer system can use a machine learning model 502 in order to classify a focal subsequence 504 thereby generating a focal subsequence classification 526. As discussed above with reference to FIGs.1-3, in a conventional application of machine learning for subsequence classification, a subsequence of time series data (such as focal subsequence 504) can be the sole input to a machine learning model, which can produce an output classification based on this input. 24 78601649V.1

[0094] By contrast, in methods according to embodiments, there can be multiple inputs to the machine learning model 502 including both the focal subsequence 504 and k-nearest neighbor subsequences to the focal subsequence 504, i.e., nearest neighbor subsequences 510-512 in FIG.5. The inclusion of both the focal subsequence 504 and the nearest neighbor subsequences 510-512 in the input set can improve focal subsequence classification accuracy. As such, in order to facilitate a better understanding of embodiments of the present disclosure, it may be helpful to describe subsequence similarity and the concept of nearest neighbor subsequences in more detail.

[0095] Many sequences of time series data derived from real-world data or observations, can exhibit two general classes of subsequences. The first, called “patterns” or “motifs,” comprise subsequences of time series data that generally repeat within a larger time series. While there may be some variation in these motifs, they may appreciably appear to correspond to a repeating event or phenomenon. In the context of ECG data, a heartbeat waveform (in, the absence of arrhythmia such as ventricular fibrillation) is an example of a motif. The second general class, called “anomalies” or “discords,” comprise subsequences of time series data that are generally unique or unusual and do not repeat within the larger time series.

[0096] Within or across time series, the most similar subsequences (measured using a distance metric such as Euclidean distance or z-normalized Euclidean distance) may be referred to as “nearest neighbor subsequences”. For example, the five most similar subsequences to a “focal subsequence” may be referred to as the five nearest neighbors (sometimes presented herein as the “5- nearest neighbors”) to that focal subsequence. A subsequence with very similar nearest neighbors may comprise a motif, while a subsequence with very dissimilar nearest neighbors may comprise a discord. Hence, nearest neighbor information can be used in the identification of motifs and discords in sequences of time series data.

[0097] The concept of nearest neighbor subsequences may be better understood with reference to FIG.6, which depicts three nearest neighbor relationships between six subsequences 602-612 within a time series. As subsequence 604 is most similar to subsequence 602, subsequence 604 is the nearest neighbor to subsequence 602. Likewise, subsequence 610 is the nearest 25 78601649V.1neighbor to subsequence 606, and subsequence 612 is the nearest neighbor to subsequence 608. Notably, these nearest neighbor relationships are not dependent on temporal distance, but rather on similarity in shape between subsequences.

[0098] Embodiments of the present disclosure use novel methods of utilizing nearest neighbor information to enhance subsequence classification accuracy. Referring back to FIG.5, as state above, in methods according to embodiments, a machine learning model 502 may take both a focal subsequence 504 and k (e.g., 5, 10, etc.) labeled nearest neighbor subsequences (e.g., nearest neighbor subsequences 510-512) as an input and return a focal subsequence classification 526. As described further below, and shown in Table 2 of FIG.20, the use of the five-nearest neighbor subsequences or 10-nearest neighbor subsequences improves classification accuracy.

[0099] In general terms, these labeled nearest neighbor subsequences 510- 512 can provide context for the machine learning model 512, which may be absent from conventional machine learning approaches to subsequence classification. If focal subsequence 504 is very similar to its nearest neighbors subsequences 510- 512, and all of those nearest neighbor subsequences 510-512 share the same classification (e.g., a “normal” or non-anomalous classification), that may suggest that focal subsequence 502 may also have the same classification, which may be “considered” (among other factors resulting from model training) by the machine learning model 502 when producing focal subsequence classification 526. By contrast, if focal subsequence 502 is very dissimilar to nearest neighbors subsequence 510-512, and all of those nearest neighbor subsequences share the same classification, then it may suggest that focal subsequence 502 does not share that classification, which may be “considered” by machine learning model 502 when producing focal subsequence classification 526.

[0100] In some embodiments, a computer system can acquire the nearest neighbor subsequences 510-512 from a training sequence of time series data 506 using a k-nearest neighbor finding method 508. In such embodiments, nearest neighbor subsequences 510-512 can comprise the k most similar subsequences to focal subsequence 504 contained within the training sequence of time series data 506. In some embodiments, the k-nearest neighbor finding method 508 may be 26 78601649V.1based on the Matrix Profile [5, 6] and related Matrix Profile generation methods such as STOMP [5, 8] and SCAMP

[0011] . As such, description of the Matrix Profile and related concepts may facilitate a better understanding of some embodiments of the present disclosure.

[0101] The Matrix Profile [5, 6] comprises a vector that indicates the nearest neighbor subsequence for each subsequence in a time series. In the case of a “self- join,” the nearest neighbor subsequence can comprise another subsequence in the same time series. In the case of an “AB-join,” the nearest neighbor subsequence can comprise a subsequence in a different time series. By analyzing the most similar pairs of subsequences, pattens and anomalies can be quickly identified, and as such the Matrix Profile can be useful for accomplishing various time series data mining tasks, including motif discovery, anomaly detection, and semantic segmentation [5- 7].

[0102] The Matrix Profile can, in some cases, be considers analogous to a 1- nearest neighbor subsequence graph. In order to identify k-nearest neighbors for the purpose of classifying subsequences of time series data, existing methods of generating a Matrix Profile (e.g., STOMP, SCAMP, etc.) can be adapted to produce k-nearest neighbor Matrix Profiles in some embodiments of the present disclosure. Such k-nearest neighbor Matrix profiles can indicate the 2, 3, 5, 10, 20, etc. nearest neighbors, rather than only the first nearest neighbor. As described above, such k- nearest neighbor Matrix Profiles (sometimes referred to herein as “ego networks,” “k- nearest neighbor graphs,” or other similar terms) may enhance the classification accuracy of a subsequence classification machine learning model. As such, Matrix Profile generating methods such as STOMP and techniques that can be used to adapt such Matrix Profile generation methods to embodiments of the present disclosure are described below.

[0103] The problem of constructing the k-nearest neighbor subsequence graph could be naively solved by extracting all subsequences from relevant sequences of time series data and computing the pairwise distances between them. However, this approach has a time complexity of ^^^ ^^ଶ^^^, where ^^ is the length of the sequences of time series data and ^^ is the subsequence length. This time 27 78601649V.1complexity may be unacceptably slow, particularly for long sequences of time series data or large subsequence lengths.

[0104] By contrast, a computer system performing methods according to embodiments can construct the k-nearest neighbor subsequence graph more efficiently by leveraging a novel modification of the STOMP method [5, 8], originally designed for computing the Matrix Profile. The conventional Matrix Profile generally represents the 1-nearest neighbor graph for all subsequences in a time series [5, 9], and as such, it can be extended to represent the k-nearest neighbor graph. Pseudocode for an extended STOMP method (Method 1) is presented in FIG.7.

[0105] Method 1 generally comprises a method for generating a k-nearest neighbor Matrix Profile (which may be equivalent to a k-nearest neighbor subsequence graph) from a time series ^^ ∈ ℝ^. Method 1 of FIG.7 shows a method for generating a “self-join” k-nearest neighbor Matrix Profile, However, it is possible to adapt Method 1 in order to implement “AB-join” k-nearest neighbor Matrix Profile. In some embodiments of the present disclosure, a computer system can generate a k-nearest neighbor matrix profile that identifies, for each subsequence in a sequence of time series data, the k-nearest neighbor subsequences in a training sequence of time series data. As such, the description below also includes modifications to some of lines 702-720, which can be used to adapt Method 1 for use in some methods according to embodiments.

[0106] Generally, the inputs of Method 1 can comprise a time series ^^ ∈ ℝ^, (sometimes referred to as a “test time series” ^^௧^^௧), a subsequence length ^^ ∈ ℕ, and a number of nearest neighbors ^^ ∈ ℕ as inputs. In some embodiments, the inputs may also comprise a “training time series” ^^௧^^^^. Some of these inputs (e.g., the number of nearest neighbors, the subsequence length, etc.) may comprise hyperparameters that can be set by practitioners of methods according to embodiments. The length of the time series ^^ can be denoted by ^^. Method 1 can output a matrix ^^ that can store the index of the k-nearest neighbors of each subsequence in ^^. The matrix ^^ may comprise a k-nearest neighbor Matrix Profile.

[0107] At line 704, a computer system can initialize the matrix ^^ such that it can store the index of the k-nearest neighbors of each subsequence in ^^ (or in the case of an AB-join, the k-nearest neighbors from another subsequence, e.g., a 28 78601649V.1training sequence of time series data ^^௧^^^^). The dimensions of the matrix ^^ may depend on the number of nearest neighbors ^^, and the total number of distinct subsequences in the time series ^^, i.e., ^^ െ ^^ ^ 1.

[0108] As stated above, Method 1 can be modified to implement an AB-join k- nearest neighbor Matrix Profile, i.e., when a computer system is using methods according to embodiments to classify subsequences using a trained subsequence classification model. In such a case, there can be two time series: a training time series ^^௧^^^^∈and a test time series ^^௧^^௧∈ ℝ^^^ೞ^, and a k-nearest neighbor Matrix Profile (sometimes referred to as an ego network) can be constructed for each subsequence in ^^௧^^௧with respect to the subsequences in ^^௧^^^^. To accommodate this change, line 704 in Method 1 can be modified such that output matrix ^^ has size ^ ^^௧^^௧െ ^^ ^ 1^ ൈ ^^.

[0109] Generally, the “for loop” of lines 706 to 718 can involve a computer system iterating through each subsequence to identify the k-nearest neighbors of that subsequence. In each iteration, the iterator value ^^ can point to the start of one of the ^^ െ ^^ ^ 1 subsequences in the time series. When adapting Method 1 to implement an AB-join k-nearest neighbor matrix profile between query subsequences ^^ in a test time series ^^௧^^௧and subsequences in a training time series ^^௧^^^^, the range of the iterator value ^^ can be adjusted to ^0, … , ^^௧^^௧െ ^^ ^ 1^, where ^^௧^^௧is the length of the test time series ^^௧^^௧.

[0110] At line 708, the computer system can extract a query subsequence ^^, i.e., by determining ^^ contiguous values from ^^ starting at index ^^, i.e., ^^ ← ^^^ ^^ ∶ ^^ ^ ^^^. The k-nearest neighbors for that query subsequence ^^ can be determined by the computer system by performing steps associated with lines 710-718 of FIG.7. When adapting Method 1 to implement an AB-join k-nearest neighbor Matrix Profile between query subsequences ^^ in a test time series ^^௧^^௧and subsequences in a training time series ^^௧^^^^, line 708 can be replaced with ^^ ← ^^௧^^௧^ ^^ ∶ ^^ ^ ^^^.

[0111] At line 710 the computer system can generate a distance profile ^^ between ^^ and each subsequence in ^^. In some embodiments, the distance profile ^^ ∈ ℝ^ି^ା^can stores the z-normalized Euclidean distance between ^^ and each subsequence in ^^. For example, ^^^1^ can store the distance between ^^ and the first subsequence in ^^, i.e., ^^^1 ∶ 1 ^ ^^^. A naive implementation of this step has a time 29 78601649V.1complexity of ^^^ ^^ ^^^. However, utilizing techniques presented in [5, 8], the time complexity of the operations of line 710 can be reduced to ^^^ ^^^. When adapting Method 1 to implement an AB-join k-nearest neighbor Matrix Profile between query subsequences ^^ in a test time series ^^௧^^௧and subsequences in a training time series ^^௧^^^^, line 710 can be replaced with ^^ ← ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^^ ^^, ^^௧^^^^^.

[0112] When generating the distance profile ^^ at line 710, the computer system can determine the distance (e.g., the z-normalized Euclidean distance) between time series subsequences using either all of dimensions, some of the dimensions, or using sub-dimensional calculations. Following a similar approach as

[0012] , for a given pair of multidimensional subsequences, a computer system can compute the z-normalized Euclidean distance between them for each dimension and then aggregate the distances across different dimensions by summing them. As such, in some methods according to embodiments, each dimension can contribute equally to the distance between the subsequences. Other embodiments can use the concept of sub-dimensional nearest neighbors, as presented in

[0012] , as sub- dimensional nearest neighbors may hold more meaningful comparisons between subsequences than all-dimensional nearest neighbors.

[0113] At line 712, the computer system can apply an exclusion zone to the distance profile ^^ to avoid “trivial matches”

[0010] with the query subsequence ^^. As a reminder, a Matrix Profile generally indicates the nearest neighbor to each subsequence in a time series, e.g., ^^. However, as each subsequence is identical to itself, each subsequence is technically its own nearest neighbor, an observation that is not particularly useful for time series data mining. Hence an exclusion zone can be used to prevent the computer system from matching subsequences to themselves during the generating of the k-nearest neighbor Matrix Profile. Wider exclusion zones can be used to prevent matching subsequences that are not strictly identical, but have significant overlap.

[0114] Expressed in other words, if ^^ is the ithsubsequence of ^^, ^^^ ^^^ is zero, and the values around the ithposition in ^^ may be very close to zero. To prevent these subsequences from being considered as nearest neighbors, a computer system performing the steps associated with line 712 can replace the values around 30 78601649V.1the ithposition in ^^ with infinity. For example, if the input index to the MASKINGWITHINF() function is the iterator value ^^, ^^^ ^^ െ^∶ ^^ ^ ^ ଶ ^ can be set to infinity.

[0115] When adapting Method 1 to implement an AB-join k-nearest neighbor Matrix Profile between query subsequences ^^ in a test time series ^^௧^^௧and subsequences in a training time series ^^௧^^^^, there is no trivial match problem. In this case, a computer system implementing Method 1 of FIG.7 can ignore line 712.

[0116] In the “for loop” of lines 714-718, the computer system can identify the k-nearest neighbors of the query subsequence ^^ in the time series ^^ using the distance profile ^^. For example, at line 716, the computer system can determine the minimum value in the distance profile ^^ and store it as a corresponding element of the matrix ^^. Afterwards, at line 718, the computer system can use “Masking with Infinity” function from line 712 to mask the identified minimum value from the distance profile ^^, preventing it from being identified in a subsequence iteration of the for loop. By repeating this process ^^ times, the computer system can identify the k-nearest neighbors for the query subsequence ^^ and store indices corresponding to those subsequences in the matrix ^^.

[0117] The output can be returned at line 720. The time complexity of Method 1 is ^^^ ^^ଶ^, as it involves computing ^^ െ ^^ ^ 1 distance profiles, and the space complexity for storing the output matrix ^^ is ^^^ ^^ ^^^. The modified version of Method 1 (i.e., in the case of an AB-join between a training time series and a testing time series) has a time complexity of ^^^ ^^௧^^^^^^௧^^௧^ and a space complexity of ^^^max^ ^^௧^^^^, ^^ ^^௧^^௧^^.

[0118] Although FIG.7 depicts pseudocode corresponding to a k-nearest neighbor STOMP method, other Matrix Profile generation methods, such as the SCAMP method

[0011] can be used or adapted in embodiments of the present disclosure. An adapted SCAMP methods can achieve higher efficiency by constructing approximate k-nearest neighbor subsequence graphs. However, the impact of using approximate k-nearest neighbor subsequence graphs on subsequence classification accuracy remains unknown. As such, it may be preferable to use STOMP-based methods (e.g., as described with reference to FIG. 7) over SCAMP methods. 31 78601649V.1

[0119] Referring back to FIG.5, the focal subsequence 504 and the nearest neighbor subsequences 510-512 (determined using k-nearest neighbor finding method 508) can be input into the machine learning model 502 and can be used to produce a focal subsequence classification 526. In some embodiments, the machine learning model 502 can comprise multiple sub-models, including an output sub- model 524 and one or more backbone sub-model(s) 514. In general terms, a computer system can use the one or more backbone sub-models 514 to produce “embeddings” corresponding to the focal subsequence 504 and the nearest neighbor subsequences 510-512, i.e., a focal subsequence embedding 516 and nearest neighbor embeddings 518-520. In general terms, these embeddings can be combined (producing a combined embedding 522) and input into the output sub- model 524, which can produce the focal subsequence classification 526.

[0120] In general terms, an embedding such as focal subsequence embedding 516 or nearest neighbor embeddings 518-520 can comprise alternative representations (e.g., latent space representations) of the information contained in the focal subsequence 504 and the nearest neighbor subsequences 510-512. Often, an embedding can comprise relevant information from data used to generate the embedding, but can be represented with a smaller amount of data. For example, a focal subsequence comprising 10 kilobytes of time series data could be represented by a 128 bit embedding. As smaller embeddings may be processed (i.e., classified using output sub-model 524) more quickly than larger subsequences of time series data, the use of embeddings may improve the speed and efficiency of methods according to embodiments.

[0121] In some embodiments, the backbone sub-model(s) 514 can comprise machine learning model(s) with Long Short-Term Memory Network (LSTM) architecture(s), recurrent neural network (RNN) architecture(s), gated recurrent unit network (GRU) architecture(s), residual network (ResNet), architecture(s), or transformer architecture(s). Three exemplary backbone sub-model architectures that can be used in some embodiments of the present disclosure are described below with reference to FIGs.8-10.

[0122] FIG.8 shows an exemplary recurrent neural network backbone sub- model 802, which can use an LSTM architecture. LSTM architectures may be useful 32 78601649V.1for modeling sequential data [13-15], such as subsequences of time series data in embodiment of the present disclosure. An input 804 can comprise a subsequence of time series data or a nearest neighbor subsequence of time series data. A computer system can initially process this input 804 using a one dimensional convolution layer 806 to extract local patterns. The one dimensional convolution layer 806 can reduce the input 804 to length 64, and can use a filter size of seven and a stride size of two.

[0123] Subsequently, the computer system can use two bi-directional recurrent neural network layers 808-810 (which can comprise LSTM layers) to summarize the processed input using LSTM recurrence mechanisms. The output of the last LSTM layer can comprise a vector or a sequence of vectors, e.g., each with length 64.

[0124] From this output sequence, at step 812 the computer system can extract the vector corresponding to the last time step. This selected vector can be passed through three linear layers (814, 818, and 822), with rectified linear unit ReLU activation layers in between (816 and 820). The linear layers 814, 818, and 822 can have various input dimensions and output dimensions. For example, linear layer 814 can have an input dimension of 128 and an output dimension of 128, whereas linear layer 818 can have an input dimension of 128 and an output dimension of 256, and linear layer 822 can have an input dimension of 256 and an output dimension of 128.

[0125] The last time step (acquired at step 812) can encapsulate the information from the entire input time series. However, it should be noted that the first time step can be acquired at step 812 instead of the last time step, as the RNN layers 808-810 are bidirectional. Regardless, the output 824 of the recurrent neural network backbone sub-model 802 (which may also be referred to as an LSTM backbone sub-model) can comprise a size 128 vector for each input subsequence.

[0126] It should be understood that FIG.8 depicts one exemplary configuration of a recurrent neural network backbone sub-model 802, and various other configurations are possible, using different numbers or configurations of layers and different layer parameters. For example, the one dimensional convolution layer 806 could have a different filter size, stride size, or output size. As another example, more or less than two bi-directional recurrent neural network layers could be used. 33 78601649V.1As yet another example, a longer or shorter sequence of linear layers and rectified linear unit activation functions could be used, and the dimensions of the inputs and outputs of these layers or activation functions could be changed. Likewise, the output 824 could comprise a vector or vectors of a different size, e.g., 256 instead of 128.

[0127] Gated Recurrent Unit Network (GRU) are another type of RNN architecture that can be used for modeling sequential data [14-16]. Some embodiments of the present disclosure can use a backbone sub-model with a GRU architecture, which may be similar to the recurrent neural network backbone sub- model 802, except bi-directional RNN layers 808-810 could be implemented as GRU layers instead of LSTM layers.

[0128] FIG.9 shows an exemplary residual network backbone sub-model 902, which can use a residual network (ResNet) architecture. ResNet is a time series classification model that has been successfully used in computer vision applications [17, 18]. Extensive evaluations reported in

[0019] have demonstrated that ResNet is one of the most effective methods for time series classification.

[0129] Similar to the recurrent neural network backbone sub-model 802 of FIG.8, the residual network backbone sub-model 902 can include a one dimensional convolution layer 906 that can capture local patterns from an input 904 (e.g., a subsequence of time series data or a nearest neighbor subsequence). The one dimensional convolutional layer 906 can reduce the input 904 to length 64, and could use a filter size of seven and a stride size of two.

[0130] Subsequently, the output of the one dimensional convolutional layer 906 can be passed through three residual blocks 908-912. Each residual block can have different input dimensions and output dimensions, e.g., residual block 908 can have an input dimension of 64 and an output dimension of 64, whereas residual block 910 can have an input dimension of 64 and an output dimension of 128, and residual block 912 can have an input dimension of 128 and an output dimension of 128.

[0131] An exemplary residual block 1102 is depicted in FIG.11. The residual block 1102 can comprise two passages: a main passage 1122 and a skip connection passage 1124. The main passage 1122 can process an input sequence 1104 using three pairs of one dimensional convolutional ReLU layers (i.e., one dimensional 34 78601649V.1convolutional layers 1108, 1114, and 1118 and ReLU layers 1112, 1116, and 1120). Exemplary filter sizes and stride sizes are presented in FIG.11.

[0132] The skip connection passage 1124 can include an optional one dimensional convolutional layer 1110, which can be introduced to the skip connection passage 1124 when the dimensions of the input 1104 and the output 1106 of residual block 1102 differ. The output of the main passage 1122 and the skip connection passage 1124 can be combined to form the output 1106 of the residual block 1102, e.g., through element-wise addition, through the use of a linear combination layer, etc.

[0133] It should be understood that FIG.11 depicts one exemplary configuration of a residual block 1102, and various other configurations are possible, using different numbers or configurations of layers and different layer parameters. For example, the residual block 1102 could comprise a longer or shorter sequence of one dimensional convolutional ReLU layers, or the parameters of the one dimensional convolutional layers, such as the filter size and stride size could be different.

[0134] Referring back to FIG.9, if a sequence length of an output of residual block 912 is greater than one, the computer system can employ a global average pooling function 914 to generate a global intermediate representation of the input 904. This global intermediate representation can then be processed by three linear layers (916, 920, and 924) with ReLU activation functions (918 and 922). Each linear layer can have different input dimensions and output dimensions, e.g., linear layer 916 can have an input dimension of 128 and an output dimension of 128, whereas linear layer 920 can have an input dimension of 128 and an output dimension of 256, and linear layer 924 can have an input dimension of 256 and an output dimension of 128. The output 926 of exemplary residual network backbone sub-model 902 can comprise an embedding vector that represents an input subsequence of time series data or an input nearest neighbor subsequence, i.e., input 904.

[0135] It should be understood that FIG.9 depicts one exemplary configuration of a residual network backbone sub-model 902, and various other configurations are possible, using different numbers or configurations of layers and different layer parameters. For example, the one dimensional convolution layer 906 35 78601649V.1could have a different filter size, stride size, or output size. As another example, more or less than three residual blocks could be used. As yet another example, a longer or shorter sequence of linear layers and rectified linear unit activation functions could be used, and the dimensions of the inputs and outputs of these layers or activation functions could be changed. Likewise, the output 926 could comprise a vector of a different size, e.g., 256 instead of 128.

[0136] FIG.10 shows an exemplary transformer backbone sub-model 1002, which can use a transformer model architecture. Transformer models can be used as alternatives to recurrent neural networks for sequence modelling [14, 15, 20-22]. Similar to the recurrent neural network backbone sub-model 802 of FIG.8 and the residual network backbone sub-model 902 of FIG.9, the transformer backbone sub- model 1002 can include a one dimensional convolutional layer 1006 which can capture local patterns from an input 1004 (e.g., a subsequence of time series data or a nearest neighbor subsequence).

[0137] At step 1008, a computer system can use fixed position encodings to incorporate positional information into the output of one dimensional convolutional layer 1006, e.g., as described in

[0020] ). Further, at step 1010, the computer system can prepend a special start token to the beginning of the sequence, in order to enable effective learning of global representations of the input 1004.

[0138] Afterwards, the sequence can pass through four consecutive transformer blocks 1012-1018. In some embodiments, transformer blocks such as transformer blocks 1012-1018 can comprise multi-head self-attention transformer blocks. FIG.10 depicts transformer blocks 1012-1018 comprising eight heads. An exemplary transformer block 1202 is depicted in FIG.12. Transformer blocks, such as transformer block 1202, can model complex dependencies and enhance subsequence representations through self-attention and position-wise transformations.

[0139] The transformer block 1202 of FIG.12 can comprise two stages: a multi-head self-attention stage 1216 and a feed-forward stage 1218. A multi-head self-attention module 1208 can be employed in the multi-head self-attention stage 1216. The transformer block 1202 can apply two linear layers (1210 and 1214) with a ReLU activation function (1212) to each position of the sequence obtained from the 36 78601649V.1multi-head self-attention module 1208. The multi-head self-attention module 1208 can enable the transformer block 1202 to capture dependencies between different positions of the input 1204. The feed-forward stage 1218 can include a position-wise feed-forward neural network, which can comprise a ReLU activation function 1212 and a linear layer 1214. The feed-forward stage 1218 can enable the transformer block to incorporate non-linear transformations and enhance the representation of each position.

[0140] Both the multi-head self-attention stage 1216 and the feed-forward stage 1218 can incorporate skip connections (i.e., skip connections 1220 and 1222), such that the input 1204 is combined with (e.g., added to) the output of each stage. This mechanism can facilitate the flow of information from the input 1204 to the output 1206, enabling the retainment of information throughout transformer block 1202.

[0141] It should be understood that FIG.12 depicts one exemplary configuration of a transformer block 1202, and various other configurations are possible, using different numbers or configurations of layers and different layer parameters.

[0142] Referring back to FIG.10, at step 1020, the computer system can extract an intermediate representation associated with the start token (appended at step 1010) from the output sequence generated by the transformer blocks 1012- 1018. This extracted representation can serve as a global representation of the input 1004 (e.g., a subsequence of time series data or a nearest neighbor subsequence), and can capture information corresponding to the entire input 1004. This mechanism shares similarities with the [CLS] token used in transformer models like BERT

[0023] .

[0143] Subsequently, the computer system can process the global intermediate representation using three linear layers (1022, 1026, and 1030) and two ReLU activation functions 1024 and 1028. The output 1032 of the transformer backbone sub-model 1002 can comprise an embedding representative of an input subsequence of time series data.

[0144] It should be understood that FIG.10 depicts one exemplary configuration of a transformer backbone sub-model 1002, and various other configurations are possible, using different numbers or configurations of layers and 37 78601649V.1different layer parameters. For example, the one dimensional convolution layer 1006 could have a different filter size, stride size, or output size. As another example, more or less than four transformer blocks could be used. As yet another example, a longer or shorter sequence of linear layers and rectified linear unit activation functions could be used, and the dimensions of the inputs and outputs of these layers or activation functions could be changed. Likewise, the output 1032 could comprise a vector of a different size, e.g., 256 instead of 128.

[0145] Further, it should be understood that layer normalization

[0024] can be applied for all normalization layers in models and sub-model according to embodiments. Layer normalization has proven to be effective when used with sequential data [20, 24]. By applying layer normalization, embodiments of the present disclosure can ensure stable and consistent normalization across different layers.

[0146] Referring back to FIG.5, the outputs of backbone sub-model(s) 514 can comprise a focal subsequence embedding 516 and one or more nearest neighbor embeddings 518-520. The computer system can combine these embeddings in some manner (e.g., via concatenation) to produce a combined embedding 522. This combined embedding 522 can then be input into output sub- model 524, thereby producing focal subsequence classification 526, which may indicate a likelihood that the focal subsequence 504 belongs to one or more classes. For example, in the case of wearable fitness data, the focal subsequence classification 526 can indicate the likelihood that the focal subsequence 504 comprises a “running” subsequence or a “walking” subsequence.

[0147] Methods and machine learning models according to embodiments are described in more detail below with reference to FIGs.13-17. FIG.13 shows an exemplary machine learning model according to embodiments (sometimes referred to as a “subsequence classification model”, an “ego network transformer model”, or more rarely, just a “transformer model”). FIG.14 shows a flowchart corresponding to an inference method according to some embodiments, i.e., a method by which a computer system can use a trained machine learning model (such as the machine learning model depicted in FIG.5 or FIG.13) in order to classify subsequences of 38 78601649V.1time series data. FIG.15 shows pseudocode corresponding to this inference method.

[0148] FIG.16 shows a flowchart corresponding to a training method according to some embodiments, by which a computer system can train a machine learning model to classify subsequences of time series data. FIG.17 shows pseudocode corresponding to this training method. Because training methods may be better understood in view of inference methods, inference methods according to embodiments are described first with reference to FIGs.13-15, followed by training methods described with reference to FIGs.13, 16, and 17.

[0149] Referring now to FIG.14, which shows a flowchart of a method for classifying subsequences of time series data using a machine learning model comprising at least an output sub-model (e.g., a machine learning model similar to the machine learning models depicted in FIGs.5 and 13). Such a method can be performed by a computer system (such as computer system 404 from FIG.4), which may instantiate, train, and use the machine learning model in order to perform the method of FIG.14. As described with reference to FIG.4, methods according to embodiments can be used to implement subsequence classification as a service, by which a computer system can classify subsequences of time series data for one or more requestors. In such cases, subsequences of time series data may be representative of one or more events (e.g., measured heartbeats, detected seismograph pressure waves, detected radar pulses, etc.), and the computer system may perform the method of FIG.14 in order to qualify or quantify these events for a requestor.

[0150] As such, if methods according to embodiments are used to provide subsequence classification as a service, at optional step 1402 the computer system can receive a request from a requestor to quantify or qualify one or more events represented by a sequence of time series data. The requestor could be a computer or an individual operating a computer. The one or more events can correspond to one or more subsequences of time series data within the sequence of time series data. After performing other steps associated with methods according to embodiments (e.g., steps 1404-1416), the computer system can provide a response to the requestor at step 1418. Such a response may comprise one or more 39 78601649V.1subsequence classifications, which may quantify or qualify the one or more events corresponding to the one or more subsequences of time series data. The computer system may receive the request (and transmit the response) via a communication network, such as communication network 414 in FIG.4.

[0151] At step 1404, the computer system can receive the sequence of time series data comprising the one or more subsequences. In some embodiments, the computer system can receive the sequence of time series data in the request from the requestor. In other embodiments, the computer system can receive the sequence of time series data by retrieving it from a database (such as database 406 in FIG.4), or from some other data structure or memory element. The computer system may also receive the sequence of time series data from a sensor or data stream. For example, a computer system associated with a seismographic measuring station may interface with a seismograph, which may stream seismographic time series data to the computer system. As stated above, the sequence of time series data may comprise one or more subsequences of time series data.

[0152] At step 1406, the computer system can construct a k-nearest neighbor graph based on the one or more subsequences. The k-nearest neighbor graph can indicate, for each subsequence of the one or more subsequences, one or more (i.e., k) nearest neighbor subsequences corresponding to each of those subsequences. In some embodiments, the one or more nearest neighbor subsequences can comprise training subsequences from a training sequence of time series data, which may be denoted by ^^^, ..., ^^^ି^, and which may correspond to training labels ^^^, ..., ^^^ି^. The computer system may retrieve this training sequence of time series data at step 1404, or at any other appropriate time.

[0153] Step 1406 generally corresponds to line 1504 in the pseudocode of FIG.15. As indicated at line 1504, the computer system can construct the k-nearest neighbor graph ^^ by finding the k-nearest neighbors from a training time series ^^௧^^^^for the one or more subsequence in the sequence of time series data ^^௧^^௧. In some embodiments, the k-nearest neighbor graph can comprise (or can be derived from) a k-nearest neighbor Matrix Profile. The computer system can construct the k-nearest neighbor graph using a variety of methods, including the modified k-nearest neighbor 40 78601649V.1STOMP method described above with reference to FIG.7. As such, to construct the k-nearest neighbor graph, in some embodiments the computer system can generate a distance profile by computing a distance (e.g., a z-normalized Euclidean distance) between each subsequence of the one or more subsequences and one or more training subsequences from the training sequence of time series data, e.g., as described above with reference to line 710 of FIG.7.

[0154] Using the distance profile, the computer system can determine, for each subsequence of the one or more subsequences, k training subsequences or k training indices (which could be used to identify the k training subsequences) corresponding to k minimum distance training subsequences of the one or more training subsequences. In this way, the computer system can determine one or more training subsequences or one or more training indices corresponding to the one or more subsequences. The k-nearest neighbor graph can comprise the one or more training subsequences or one or more training indices for each subsequence of the one or more subsequences. This process can generally correspond to lines 714-718 of FIG.7, described above.

[0155] Expressed in other words, for each subsequence of the one or more subsequences, the computer system can find the k most similar training subsequences from the training sequence, then record those similar training subsequences in the k-nearest neighbor graph.

[0156] As described above, the k-nearest neighbor graph can comprise a k- nearest neighbor Matrix Profile. As such, the k-nearest neighbor graph can comprise a k by n matrix, where n is a number of subsequences in the one or more subsequences (distinct from n in e.g., the description of FIG.7, which refers to a length of a sequence of time series data), and k is a number of minimum distance training subsequences of the one or more training subsequences. In such a case, each element of the n by k matrix can comprise a training index corresponding to a minimum training subsequence. For example, the (2, 3) element of the n by k matrix could comprise an index corresponding to the third most similar training subsequence corresponding to the second subsequence in the sequence of time series data. 41 78601649V.1

[0157] At step 1408, the computer system can produce one or more subsequence classifications corresponding to the one or more subsequences. The computer system can produce these subsequence classifications by performing steps 1410-1416 for each subsequence of the one or more subsequences. In some embodiments, the one or more subsequences may comprise all of the subsequences contained in the sequence of time series data, and as such, steps 1410-1416 may be performed for the entire sequence of time series data. Steps 1408-1416 generally correspond to the “for loop” of lines 1506-1512 and the return statement of line 1514 in the pseudocode of FIG.15.

[0158] At step 1410, the computer system can use the k-nearest neighbor graph to determine one or more nearest neighbor subsequences corresponding to a subsequence of the one or more subsequences, i.e., a subsequence of the one or more subsequences that the computer system is classifying via an iteration of steps 1410-1416. Step 1410 can generally correspond to lines 1508 and 1510 in the pseudocode of FIG.15. For a k-nearest neighbor graph comprising an n by k matrix, the computer system can identify the one or more nearest neighbor subsequences by identifying one of the n rows corresponding to the subsequence. The k columns of that particular row can comprise indices identifying the one or more nearest neighbor subsequences.

[0159] A subsequence that the computer system is in the process of classifying may be referred to as a “focal subsequence”. FIG.13 shows a subsequence classification model (sometimes referred to as an “ego network transformer model”). FIG.13 also shows a block representation of a focal subsequence 1302 and its one or more nearest neighbor subsequence 1304-1306, which can comprise inputs to the subsequence classification model. FIG.13 also shows a network representation 1334 of the k-nearest neighbor graph, in which the focal subsequence 1302 is represented by a focal node 1336, and one or more nearest neighbor subsequences 1304-1306 (i.e., the k-nearest neighbors) are represented by neighbor nodes 1338-1342.

[0160] Referring back to FIG.14, at step 1412, the computer system can produce a subsequence embedding and one or more nearest neighbor embeddings based on the subsequences and the one or more nearest neighbor subsequences. 42 78601649V.1In some embodiments, the subsequence embedding may comprise a subsequence embedding vector and the one or more nearest neighbor embeddings may comprise one or more nearest neighbor embedding vectors. This may enable the computer system to produce a combined embedding in step 1414, e.g., via vector concatenation. Herein, a subsequence embedding (also referred to as a “focal node representation,” such as focal representation 1318 in FIG.13) may be denoted by ^^^^^^^and may corresponding to a “focal subsequence” ^^^^^^^. Likewise, the one or more (i.e., k) nearest neighbor embeddings may be denoted by ^^^, ^^^, ... ^^^ି^and may correspond the one or more nearest neighbor subsequences ^^^, ^^^, ... ^^^ି^.

[0161] As depicted in FIG.13, in some embodiments, the machine learning model can comprise a backbone sub-model, or multiple backbone sub-models 1308- 1312, which can comprise either multiple instances of the same backbone sub- model or different backbone sub-models, or e.g., a single backbone sub-model applied independently to each subsequence. The computer system can produce the subsequence embedding and the one or more nearest neighbor embeddings using the backbone sub-model(s). In general terms, the computer system can produce the subsequence embedding and the one or more nearest neighbor embeddings by inputting the subsequence and the one or more nearest neighbor subsequences into one or more backbone sub-models, the outputs of which can comprise the subsequence embedding and the one or more nearest neighbor embeddings.

[0162] As described above with reference to FIGs.8-10, in some embodiments the backbone sub-model(s) can comprise machine learning model(s) with Long Short-Term Memory Network (LSTM) architecture(s), recurrent neural network (RNN) architecture(s), Gated Recurrent Unit Network (GRU) architecture(s), residual network (ResNet) architectures, transformer architecture(s) or any other appropriate architecture.

[0163] Referring back to FIG.14, at step 1414, the computer system can produce a combined embedding by combining the subsequence embedding and the one or more nearest neighbor embeddings. As described above, in some embodiments the subsequence embedding can comprise a subsequence embedding vector and the one or more nearest neighbor embeddings can comprise one or more nearest neighbor embedding vectors. In such cases, the combined 43 78601649V.1embedding can comprise a combined embedding matrix, and combining the subsequence embedding and the one or more nearest neighbor embeddings can comprise concatenating the subsequence embedding vector and the one or more nearest neighbor embedding vectors, thereby producing the combined embedding matrix.

[0164] As described above, the one or more nearest neighbor subsequences can comprise one or more training subsequences from a training sequence of time series data. Such training subsequences may have corresponding labels, which may indicate one or more classifications corresponding to the one or more training subsequences. In some embodiments, the computer system can enhance the one or more nearest neighbor embeddings using this label information, thereby improving classification accuracy. As such, producing the combined embedding in step 1414 can further comprise combining each nearest neighbor embedding of the one or more nearest neighbor embeddings with a corresponding label embedding of the one or more label embeddings. Herein, ^^^can represent a learnable label embedding for ^^^, the label corresponding to the ithnearest neighbor subsequence. In some embodiments, the computer system can combine a nearest neighbor embedding with a corresponding label embedding additively, i.e., ^^^^← ^^^^ ^^^. The computer system can generate the one or more label embeddings using any appropriate means, e.g., using a backbone sub-model, another machine learning model, etc.

[0165] FIG.13 shows a focal node representation 1318 (a subsequence embedding) and a ^^ െ 1 node representation 1320 (a nearest neighbor embedding, corresponding to the ^^ െ 1௧^nearest neighbor subsequence). As depicted in FIG.13, these embeddings can comprise 128 by 1 vectors, however it should be understood that methods according to embodiments can be practiced with various sized vectors. FIG.13 also shows the inclusion of label embedding information at steps 1314-1316, and the combination of the subsequence embedding and one or more nearest neighbor embeddings at step 1322, as described with reference to FIG.14. In FIG. 13, as the subsequence embedding comprises a 128 by 1 vector and the k nearest neighbor embeddings each comprise 128 by 1 vectors, the combined embedding matrix can comprise a 128 by (k + 1) matrix. Such a combined embedding matrix can 44 78601649V.1thereby contain the subsequence embedding ^^^^^^^and the one or more nearestneighbor embeddings ^^^^, ..., ^^^^ି^.

[0166] Referring back to FIG.14, at step 1416 the computer system can produce a subsequence classification using the output sub-model and the combined embedding. By repeating steps 1410-1416 for each subsequence in the one or more subsequences, the computer system can produce one or more subsequence classifications corresponding to the one or more subsequences. In some embodiments, each subsequence classification of the one or more subsequence classifications can indicate one or more likelihoods that a corresponding subsequence is a member of one or more classes. For example, in the context of wearable fitness data, a subsequence classification could comprise “90% walking, 9% running, 1% background”, indicating that there is a 90% probability (likelihood) that a subsequence corresponds to a walking class, a 9% probability that the subsequence corresponds to a running class, and a 1% probability that the subsequence corresponds to a background class.

[0167] Lines 1512-1514 of the pseudocode of FIG.15 generally corresponds to steps 1412-1416 of FIG.14. At line 1512, the computer system can produce the one or more subsequence classifications (also referred to as predicted labels) for the one or more subsequences ^^^and store it in a vector of one or more subsequence classifications ^^^௧^^௧^ ^^^. At line 1514, the computer system can return or output theone or more subsequence classifications ^^^௧^^௧.

[0168] The process of producing a subsequence classification using an output sub-model may be better understood with reference to FIG.13, which shows a detailed view of an exemplary output sub-model 1344. In some embodiments, an output sub-model can comprise a transformer sub-model and a classifier sub-model. In some embodiments, the transformer sub-model can comprise one or more multi- head self-attention transformer blocks. FIG.13 depicts a transformer sub-model 1346 comprising two transformer blocks 1324 and 1326. Transformer blocks 1324 and 1326 may be better understood with reference to FIG.12 and the transformer block description above. In FIG.13, transformer blocks 1324 and 1326 each have eight self-attention heads, an input dimension of 128, and an output dimension of 128. However, it should be understood that methods according to embodiments can 45 78601649V.1be practiced using different input dimensions, output dimensions, number of attention heads, number of transformer blocks, etc. In some embodiments, the computer system can use the transformer sub-model 1346 to produce an enhanced combined embedding based on the combined embedding produced at step 1322 (or step 1414 of FIG.14), i.e., by inputting the combined embedding into the transformer sub-model 1346, which can, in general terms, aggregate information from the subsequence embedding and one or more nearest neighbor embeddings.

[0169] In some embodiments, the computer system can produce a subsequence classification using the classifier sub-model 1348 and the enhanced combined embedding. At step 1328, the computer system can determine a focal representation (also referred to as an “enhanced subsequence embedding”) by extracting an embedding representation of the focal subsequence 1302 from the enhanced combined embedding. The focal representation can comprise data from the enhanced combined embedding representative of the focal subsequence 1302. The computer system can then produce the subsequence classification 1332 by inputting the focal representation into the classifier sub-model 1348. In some embodiments, the classifier sub-model 1348 can comprise a linear layer 1330, which can be used to produce logit values for each subsequence class, indicating the likelihood (probability) that the focal subsequence 1302 belongs to each class. The subsequence classification 1332 can comprise these logit values, or any appropriate data or alternative class representations derived from these logit values (e.g., a single data value indicative of the most likely class based on the logit values).

[0170] Referring back to FIG.14, after classifying each of the one or more subsequences in the sequence of time series data, at optional step 1418 the computer system can optionally provide a response to a requestor in response to a request for time series subsequence classification services at step 1402. As described above, in some embodiments, the one or more subsequence classifications can correspond to one or more events, and by performing time series subsequence classification and providing the one or more subsequence classifications to the requestor, the computer system can qualify or quantify the one or more events for the requestor, thereby servicing the request. 46 78601649V.1

[0171] By using k-nearest neighbor subsequence graphs, the inference method described above with reference to FIGs.13-15 is a powerful and versatile solution to the subsequence classification problem. Methods according to embodiments offer notable advantage over attending to all subsequences in the training data, as opposed to the k-nearest neighbor subsequences. By focusing on the top k-nearest neighbors, methods according to embodiments achieve improved efficiency. This is particularly useful given the space complexity of transformer blocks, which grow quadratically with the number of inputs. For example, if the training data comprised one million subsequences, and all subsequences were attended to (as opposed to the k-nearest neighbors), storing the attention matrix alone would require over seven terabytes of memory.

[0172] Having described methods of using a machine learning model to perform inference, e.g., for the purpose of generating one or more subsequence classifications, it may be useful to describe methods of training the machine learning model for this purpose. In practice, there are a variety of methods that can be used to train a machine learning model. For ease of exposition, the following description assumes a “round based” framework of machine learning model training, e.g., as described further above. However, methods according to embodiments could conceivably be adapted to other training frameworks.

[0173] FIG.16 shows a flowchart of a method for training a machine learning model to classify subsequences of time series data. A machine learning model according to embodiments can comprise at least an output sub-model (e.g., as depicted in FIGs.5 and 13), which can be defined by a set of output sub-model parameters. The process of training can generally involve updating these output sub- model parameters as well as other parameters associated with the machine learning model. The method of FIG.16 may be performed by a computer system.

[0174] At step 1602, the computer system can receive a training sequence of time series data comprising a plurality of training subsequences. The computer system can receive the training sequence of time series data in any appropriate manner, e.g., by retrieving it from a database (such as database 406 in FIG.4), from a data stream, or from any other data structure or memory element. The plurality of training subsequences in the training sequence of time series data may be labeled 47 78601649V.1by an associated plurality of labels. Each label may indicate a class to which an associated training subsequence belongs. For example, in the context of wearable fitness device data, a label may indicate whether a subsequence corresponds to a period of running or walking. The computer system can receive any applicable labels along with the training sequence of time series data at step 1602.

[0175] At step 1604, the computer system can construct a k-nearest neighbor graph based on the plurality of training subsequences. The k-nearest neighbor graph can indicate, for each training subsequence of the plurality of training subsequences, one or more (i.e., k) nearest neighbor subsequences corresponding to each of those training subsequences.

[0176] Step 1604 generally corresponds to line 1704 in the pseudocode of FIG.17. As indicated at line 1704, the computer system can construct the k-nearest neighbor graph ^^ by finding the k-nearest neighbors for each training subsequence in the training time series ^^. In some embodiments, the k-nearest neighbor graph can comprise (or can be derived from) a k-nearest neighbor Matrix Profile. The computer system can construct the k-nearest neighbor graph using a variety of methods, including the modified k-nearest neighbor STOMP method described above with reference to FIG.7. As such, to construct the k-nearest neighbor graph, in some embodiments the computer system can generate a distance profile by computing a distance (e.g., a z-normalized Euclidean distance) between each subsequence of the one or more subsequences and one or more training subsequences from the training sequence of time series data, e.g., as described above with reference to line 710 of FIG.7.

[0177] Using the distance profile, the computer system can determine, for each training subsequence of the plurality of training subsequences, k training subsequences or k training indices (which could be used to identify the k training subsequences) corresponding to k minimum distance training subsequences of the plurality of training subsequences. In this way, the computer system can determine one or more training subsequences or one or more training indices corresponding to the one or more training subsequences. The k-nearest neighbor graph can comprise the one or more training subsequences or one or more training indices for each 48 78601649V.1training subsequence of the plurality of training subsequences. This process can generally correspond to lines 714-718 of FIG.7, described above.

[0178] Expressed in other words, for each training subsequence of the plurality of training subsequences, the computer system can find the k most similar training subsequences from the training sequence, then record those similar training subsequences in the k-nearest neighbor graph.

[0179] As described above, the k-nearest neighbor graph can comprise a k- nearest neighbor Matrix Profile. As such, the k-nearest neighbor graph can comprise a k by n matrix, where n is a number of training subsequences in the plurality of training subsequences (distinct from n in e.g., the description of FIG.7, which refers to a length of a sequence of time series data), and k is a number of minimum distance training subsequences of the one or more training subsequences. In such a case, each element of the n by k matrix can comprise a training index corresponding to a minimum training subsequence. For example, the (2, 3) element of the n by k matrix could comprise an index corresponding to the third most similar training subsequence corresponding to the second training subsequence in the training sequence of time series data.

[0180] Referring back to FIG.16, at step 1606, the computer system can begin performing an iterative training process used to train the machine learning model. The iterative training process can comprise a plurality of training rounds and can be performed until a terminating condition (e.g., a predefined number of training rounds or epochs, a convergence condition, etc.) has been met. Each training round can comprise steps 1608-1624 of FIG.16. If the terminating condition has not been met, the computer system can return to step 1606 and perform an additional training round. The computer system can repeat this process until the terminating condition has been met.

[0181] At step 1608, the computer system can sample a batch of training subsequences. The batch of training subsequences can comprise one or more training subsequences from the plurality of training subsequences. In some embodiments, the plurality of training subsequences can each comprise a predefined number of time series data values. For example, each training subsequence can comprise 100 sequential time series data values, e.g., corresponding to a 49 78601649V.1subsequence length ^^ of 100. In such embodiments, the batch of training subsequences can comprise a number of training subsequences based on the predefined number of time series data values and a number of time series data values in the training sequence of time series data.

[0182] As an example, if the training sequence of time series data comprises, e.g., 1000 time series data values, and each training subsequence comprises 100time series data values, then a batch of training subsequences could comprise, e.g.,10 ൌ 1000 / 100 non-overlapping training subsequences, which can comprise theminimum number of training subsequences needed to span the entire training time series, and can comprise the number of subsequences used for training in each epoch. Using all of the subsequences in the training sequence of time series data may lead to redundancy due to overlap between training subsequences. As such, sampling training subsequences in the manner described above may be more efficient.

[0183] Step 1608 generally corresponds to lines 1708 and 1710 in FIG.17. Line 1708 generally corresponds to a process for determining a total number of sample subsequences ^^^^^^^^, as described above. In line 1710, the computer system can randomly sample (or use any other appropriate sampling technique) the ^^^^^^^^subsequences from the training sequence of time series data without replacement. The sampling process can yield the sampled subsequences ^^^^^^^^, their associated ground truth labels ^^^^^^^^, and the indices of the subsequences ^^^^^^^^. This random sampling approach can enable the model to train on different shifts of essentially the same subsequences, thereby enhancing its robustness.

[0184] At step 1610, the computer system can produce one or more subsequence classifications corresponding to the batch of training subsequences by performing steps 1612-1618 for each training subsequence of the batch of training subsequences. Afterwards, at steps 1620 and 1622, the computer system can generate loss values based on these subsequence classifications and update the machine learning model based on these loss values. Steps 1610-1622 generally correspond to the “for loop” of lines 1712-1720 of the pseudocode of FIG.17.

[0185] At step 1612, the computer system can use the k-nearest neighbor graph and a training index corresponding to a training subsequence to determine 50 78601649V.1one or more nearest neighbor subsequences corresponding to the training subsequence. Step 1612 can generally correspond to lines 1716 and 1718 of the pseudocode of FIG.17. This training subsequence can comprise a training subsequence of the plurality of training subsequences that the computer system is classifying via an iteration of steps 1610-1618. For a k-nearest neighbor graph comprising an n by k matrix, the computer system can identify the one or more nearest neighbor subsequences by identifying one of the n rows corresponding to the training subsequence. The k columns of that particular row can comprise indices identifying the one or more nearest neighbor subsequences.

[0186] At step 1614, the computer system can produce a subsequence embedding and one or more nearest neighbor embeddings based on the training subsequence and the one or more nearest neighbor subsequences. In some embodiments, the subsequence embedding may comprise a subsequence embedding vector and the one or more nearest neighbor embeddings may comprise one or more nearest neighbor embedding vectors. This may enable the computer system to produce a combined embedding at step 1616, e.g., via vector concatenation.

[0187] As described above, in some embodiments the machine learning model can comprise a backbone sub-model. Such a backbone sub-model can comprise a machine learning model with a Long Short-Term Memory Network (LSTM) architecture, a recurrent neural network (RNN) architecture, a Gated Recurrent Unit Network (GRU) architecture, a Residual Network (ResNet) architecture, or a transformer architecture. The computer system can produce the subsequence embedding and the one or more nearest neighbor embeddings using the backbone sub-model. In general terms, the computer system can produce the subsequence embedding and the one or more nearest neighbor embeddings by inputting the training subsequence and the one or more nearest neighbor subsequences into the backbone sub-model, the output of which can comprise the subsequence embedding and the one or more nearest neighbor embeddings.

[0188] At step 1616, the computer system can produce a combined embedding by combining the subsequence embedding and the one or more nearest neighbor embeddings. As described above, in some embodiments the subsequence 51 78601649V.1embedding can comprise a subsequence embedding vector and the one or more nearest neighbor embeddings can comprise one or more nearest neighbor embedding vectors. In such cases, the combined embedding can comprise a combined embedding matrix, and combining the subsequence embedding and the one or more nearest neighbor embeddings can comprise concatenating the subsequence embedding vector and the one or more nearest neighbor embedding vectors, thereby producing the combined embedding matrix.

[0189] As described above, the one or more nearest neighbor subsequences can comprise one or more training subsequences from the training sequence of time series data. Such training sequences can have corresponding labels, which may indicate one or more classifications corresponding to the one or more training subsequences. In some embodiments, the computer system can enhance the one or more nearest neighbor embeddings using this label information, thereby improving classification accuracy. As such, producing the combined embedding in step 1616 can further comprise combining each nearest neighbor embedding of the one or more nearest neighbor embeddings with a corresponding label embedding of the one or more label embeddings. In some embodiments, the computer system can combine a nearest neighbor embedding with a corresponding label embedding additively. The computer system can generate the one or more label embeddings using any appropriate means, e.g., using a backbone sub-model, another machine learning model, etc.

[0190] At step 1618, the computer system can produce a subsequence classification using the output sub-model and the combined embedding. By repeating steps 1612-1618 for each training subsequence in the batch of training subsequences, the computer system can produce one or more subsequence classifications corresponding to the batch of training subsequences. In some embodiments, each subsequence classification of the one or more subsequence classifications can indicate one or more likelihoods that a corresponding subsequence is a member of one or more classes. For example, a subsequence classification could comprise a vector (0.9., 0.09, 0.01), indicating that there is a 90% probability (likelihood) that a subsequence corresponds to a first class (e.g., a walking class), a 9% probability that the subsequence corresponds to a second class 52 78601649V.1(e.g., a running class), and a 1% probability that the subsequence corresponds to a third class (e.g., a background class).

[0191] The process of producing a subsequence classification using an output sub-model can be better understood with reference to FIG.13 and the description further above. In some embodiments, an output sub-model can comprise a transformer sub-model and a classifier sub-model. The computer system can use the transformer sub-model to produce an enhanced combined embedding based on the combined embedding produced at step 1616, e.g., by inputting the combined embedding into the transformer sub-model.

[0192] Afterwards, in some embodiments, the computer system can produce a subsequence classification using the classification sub-model and the enhanced combined embedding. The computer system can optionally determine an enhanced subsequence embedding by extracting an embedding representation of a corresponding training subsequence from the enhanced combined embedding. The enhanced subsequence embedding can comprise data from the enhanced combined embedding representative of the training subsequence. The computer system can then produce the subsequence classification by inputting the enhanced combined embedding into the classifier sub-model. In some embodiments, the classifier sub- model can comprise a linear layer, which can be used to produce logit values for each subsequence class, indicating the likelihood (probability) that the training subsequence belongs to each class. The subsequence classification can comprise these logit values, or any other appropriate data or alternative class representations derived from these logit values.

[0193] At step 1620, the computer system can generate one or more loss values based on the one or more subsequence classifications and one or more training labels corresponding to the batch of training subsequences. These loss values may be related to the difference between the subsequence classifications and the known training labels associated with the training subsequences. For example, for a subsequence classification (0.90, 0.09, 0.01) corresponding to, e.g., a 90% probability that a subsequence corresponds to a walking class, a 9% probability that the subsequence corresponds to a running class, and a 1% probability that the subsequence corresponds to a background class, the corresponding label may 53 78601649V.1comprise the vector (1.00, 0.00, 0.00), indicating that the subsequence corresponds to a walking class. A difference can be computed between the subsequenceclassification and the label, e.g., ^^0.90 െ 1.00^ଶ ^ ^0.09 െ 0.00^ଶ ^ ^0.01 െ 0.00^ଶ ^0.135. Such differences could be computed for each training subsequence in thebatch of training subsequences, and used to determine loss values, e.g., via some form of aggregation.

[0194] At step 1622, the computer system can update the machine learning model based on the one or more loss values, thereby training the machine learning model. Step 1622 generally corresponds to line 1720 of the pseudocode of FIG.17, in which the computer system can update the parameters ^^ of the machine learning model ^^ఏ. As described above, in some embodiments the machine learning model can comprise an output sub-model and a backbone sub-model. The output sub- model may be defined by a set out output sub-model parameters, and the backbone sub-model may be defined by a set of backbone sub-model parameters. In such cases, updating the machine learning model based on the one or more loss values can comprise updating the set of output sub-model parameters based on the one or more loss values and updating the set of backbone sub-model parameters based on the one or more loss values.

[0195] As described above, in some embodiments the output sub-model may further comprise a transformer sub-model and a classifier sub-model. The transformer sub-model can be defined by a set of transformer sub-model parameters and the classifier sub-model can be defined by a set of classifier sub-model parameters. In such cases, the set of output sub-model parameters can comprise the set of transformer sub-model parameters and the set of classifier sub-model parameters, which can be updated based on the one or more loss values in order to train the machine learning model.

[0196] At step 1624, the computer system can determine if the terminating condition has been met. As described above, in some embodiments, the terminating condition can comprise a defined number of training rounds or training epochs, and the terminating condition can be met if a total number of training rounds or training epochs equals or exceeds the defined number of training rounds or training epochs, e.g., as show in line 1706 of FIG.17. In other embodiments, the terminating 54 78601649V.1condition can comprise a convergence condition. This terminating condition can be met if the plurality of machine learning model parameters (e.g., a plurality of output sub-model parameters and a plurality of backbone sub-model parameters) converge, e.g., exhibit little to no change in consecutive training rounds.

[0197] If the terminating condition has not been met, at step 1624, the computer system can repeat the iterative training process and return to step 1606. The computer system can sample a new batch of training subsequences and perform an additional round of training. This process can be repeated until the terminating condition has been met.

[0198] Otherwise at step 1624, the computer system can complete the iterative training process. At this point, the parameters of the machine learning model can be output (e.g., as in line 1722 of FIG.17) and fixed, and the machine learning model can be used to classify subsequences of time series data in production, e.g., as described above.

[0199] As described above, some embodiments are directed to methods for training and using a machine learning model to classify one or more subsequences of time series data, thereby generating one or more subsequence classifications. However, some embodiments are also directed to temporal consistency post- processing methods, which can be used to improve classification accuracy. Such temporal consistency post-processing methods are described in more detail below.

[0200] Consider a sequence of time series data comprising a plurality of subsequences each of a predetermined subsequence length m. Methods according to embodiments can be used to classify each subsequence of the plurality of subsequences, thereby producing a plurality of subsequence classifications. In this case, the plurality of subsequence classifications can be ordered based on the position of the corresponding subsequences in the sequence of time series data, effectively creating a “sequence of subsequence classifications,” itself a type of time series.

[0201] Adjacent subsequences in a sequence of time series data may have considerable overlap between the time series data values contained in such subsequences. For example, for a subsequence length of 100 time series data values, and a sliding window stride of one, two adjacent subsequences may have 99 55 78601649V.1time series data values in common. Consequently, it may be likely that these adjacent subsequences have the same classification. More generally, it may be more likely that “temporally close” subsequences may have similar classifications.

[0202] As such, in some embodiments the computer system can perform temporal consistency post-processing methods to improve subsequence classification accuracy by improving the temporal consistency of the plurality of subsequence classifications. In general terms, the computer system can accomplish this by adjusting subsequence classifications based on the classifications of temporally close subsequences, based on the theory that temporally close subsequences are more likely to share the same classification. In this way, the computer system can correct potentially erroneous subsequence classifications.

[0203] Classification accuracy improvements resulting from temporal consistency post-processing methods have been demonstrated through experimentation, and Table 1 of FIG.20 (described in more detail further below) summarizes these classification accuracy improvements. A temporal consistency post-processing method according to embodiments is described in more detail below with reference to FIGs.18 and 19. FIG.18 shows a flowchart corresponding to a temporal consistency post-processing method, and FIG.19 shows a visual example of such a method.

[0204] FIG.19 shows a plurality of subsequence classifications 1902 ( ^^^^^^^^), which a computer system could generate using a machine learning model and methods according to embodiments, as described above. The plurality of subsequence classifications 1902 can be organized into a sequence, each data value corresponding to a different subsequence in a sequence of time series data. As an example, in the context of wearable fitness data, a classification “0” could indicate a background subsequence, a classification “1” could comprise a walking classification, and a classification “2” could comprise a running classification.

[0205] Referring to FIGs.18 and 19, at step 1802, the computer system can identify a plurality of onset subsequence classifications from the plurality of subsequence classifications. In the visual example of FIG.19, there are three onset subsequence classifications 1906, 1908, and 1910. In general, an onset subsequence classification can correspond to the appearance of a non-background 56 78601649V.1classification that is different from a previous subsequence classification. In FIG.19, onset subsequence classification 1906 corresponds to the first visible “1” classification in the plurality of subsequences 1902, following a background subsequence classification. Onset subsequence classification 1908 can correspond to the “2” classification following a “0” classification and “1” classification. Likewise, onset subsequence classification 1910 can correspond to a “1” classification following three “0” classifications and a “2” classification. By iterating through the plurality of subsequence classifications and comparing each subsequence classification to preceding subsequence classifications, the computer system can identify the plurality of onset subsequence classifications. The computer system can track or otherwise store indices corresponding to the plurality of onset subsequence classifications, e.g., for the purpose of performing temporal consistency post- processing methods according to embodiments.

[0206] At step 1804, the computer system can identify a plurality of subsequence classification subsets based on the plurality of onset subsequence classifications and a predetermined window size value. Each subsequence classification subset of the plurality of subsequence classification subsets can comprise an onset subsequence classification and a number of subsequence classifications proportional to the window size value. Step 1804 may be better understood with reference to the visual example of FIG.19.

[0207] FIG.19 assumes a predetermined window size value of three subsequence classifications. There are three offset subsequence classifications 1906-1910. As such, there are three subsequence classification subsets 1912-1916, each containing a corresponding offset classification. Each subsequence classification subset 1912-1916 can contain a total number of subsequence classifications corresponding to the predetermined window size (three). As such, each subsequence classification subset 1912-1914 can comprise a number of (non- offset) subsequence classifications proportional to the predetermined window size value, i.e., two, such that the total number of subsequence classifications in each subsequence classification subset is equal to predetermined window size value. Such subsequence classifications can comprise the subsequence classifications immediately following corresponding onset subsequence classifications. 57 78601649V.1

[0208] At step 1806, the computer system can determine a majority subsequence classification for each subsequence classification subset. For example, referring to the visual example of FIG.19, subsequence classification subset 1912 can comprise classifications (1, 1, 0), and the computer system could determine a majority subsequence classification of “1”, e.g., by counting the number of subsequence classifications corresponding to each class in the subset classification subset 1912, and determining the greatest number among the counts. Likewise, for subsequence classification subset 1914 (comprising classifications (2, 2, 0)), the computer system could determine a majority subsequence classification of “2”. For subsequence classification subset 1916 (comprising classifications (1, 0, 0)), the computer system could determine a majority subsequence classification of “0”.

[0209] At step 1808, the computer system can replace each subsequence classification in the plurality of subsequence classification with a corresponding majority subsequence classification. Each majority subsequence classification can correspond to one or more classification subsets to which the subsequence classification belongs. In this way, the computer system can produce a plurality of post-processed subsequence classifications. FIG.19 shows a plurality of post- processed subsequence classifications 1904 ( ^^^^௧^^). In some embodiments, replacing each subsequence classification with a majority subsequence classification can comprise only replacing the non-majority classifications with their respective majority classification.

[0210] In the visual example of FIG.19, the majority classification of subsequence classification subset 1912 is “1”, and as such, in the plurality of post- processed subsequence classifications 1904, the potentially erroneous “0” classification has been replaced by a “1” classification. Similarly, the potentially erroneous “0” classification of subsequence classification subset 1914 has been replaced by a “2” classification in the plurality of post-processed subsequence classifications 1904. Additionally, the potentially erroneous “1” classification of subsequence classification subset 1916 has been replaced by a “0” classification in the plurality of post-processed subsequence classifications 1904.

[0211] Steps 1802-1808 of FIG.18 can be implemented using a sliding window approach. In some embodiments, the length of the sliding window can be 58 78601649V.1determined using a validation data set. The computer system can slide a window along a classification sequence and perform these steps for each subset contained in the sliding window. These steps can be repeated until no more onsets are found. In this way the computer system can produce a plurality of post-processed subsequence classifications.

[0212] Experiments were conducted using methods and systems according to embodiments and 128 univariate time series datasets sourced from the UCR Archive [3]. The experimental results demonstrate the efficiency and superior performance of methods and systems according to embodiments compared to existing machine learning approaches.

[0213] In the experiments, various methods and machine learning models, including methods according to embodiments, were used to classify subsequences of time series data from the 128 datasets. Methods according to embodiments were performed using various numbers of nearest neighbor subsequences, including one- nearest neighbor (1NN), five-nearest neighbor (5NN), and ten-nearest neighbor (10NN). These methods were compared against established machine learning models, including LSTM, GRU, ResNet, and Transformer models. Scores (e.g., ^^^scores) were calculated for each method and each dataset then aggregated across all datasets to arrive at average ranking scores presented in Tables 1 and 2 of FIG. 20. Generally, a lower score indicates better performance.

[0214] Table 1 demonstrates the effectiveness of post-processing methods according to embodiments described above, indicated by post-processing improvements 2002. The post-processing methods according to embodiments were applied to the classifications produced by all methods, including the established machine learning models, and ranking scores were calculated for before and after post-processing. Post-processing improvements 2002 show that the average ranking score after post-processing is 6.93, which is less than the average ranking score before post-processing (10.07), indicating improvements resulting from post- processing methods according to embodiments.

[0215] Table 2 demonstrates performance improvements from using methods according to embodiments over established machine learning models. In the experiments of Table 2, individual LSTN, GRU, ResNet, and Transformer backbone 59 78601649V.1sub-models were used to classify time series subsequences from the 128 datasets. Scores were calculated and aggregated, resulting in the scores depicted in the “Base” column of Table 2. Additionally, methods and machine learning models according to embodiments (e.g., those depicted in FIGs.5 and 13) were implemented using various numbers of nearest neighbors, i.e., five-nearest neighbors (5NN) and ten-nearest neighbors (10NN), using backbone sub-models corresponding to the rows of Table 2. Scores were likewise calculated and aggregated, and are depicted in the 5NN and 10NN columns of Table 2.

[0216] As indicated by subsequence classification model improvements 2004 in Table 2, methods and machine learning models according to embodiments outperformed the base models in both the five-nearest neighbor case and the ten nearest neighbor case, indicating that methods according to embodiments and proposed ego-network transformer models according to embodiments improve classification performance.

[0217] In conclusion, the experimental findings on the 128 datasets from the UCR Archive demonstrate that by incorporating novel temporal consistency post- processing methods and leveraging the novel ego-network transformer model, embodiments of the present disclosure achieve superior performance in subsequence classification over established machine learning based methods of time series subsequence classification.

[0218] Any of the computer systems mentioned herein may utilize any suitable number of subsystems. Examples of such subsystems are shown in FIG.21 in computer system 2100. In some embodiments, a computer system includes a single computer apparatus, where the subsystems can be the components of the computer apparatus. In other embodiments, a computer system can include multiple computer apparatuses, each being a subsystem, with internal components. Examples of computer systems include desktop and laptop computers, tablets, mobile phones and other mobile devices.

[0219] The subsystems shown in FIG.21 are interconnected via a system bus 2112. Additional subsystems such as a printer 2108, keyboard 2118, storage device(s) 2120, monitor 2124 (e.g., a display screen, such as an LED display), which is coupled to display adapter 2114, and others are shown. Peripherals and 60 78601649V.1input / output (I / O) devices, which couple to I / O controller 2102, can be connected to the computer system by any number of means known in the art such as input / output (I / O) port 2116 (e.g., USB, FireWire®). For example, I / O port 2116 or external interface 2122 (e.g., Ethernet, Wi-Fi, etc.) can be used to connect computer system 2100 to a wide area network such as the Internet, a mouse input device, or a scanner. The interconnection via system bus 2112 allows the central processor 2106 to communicate with each subsystem and to control the execution of a plurality of instructions from system memory 2104 or the storage device(s) 2120 (e.g., a fixed disk, such as a hard drive, or optical disk), as well as the exchange of information between subsystems. The system memory 2104 and / or the storage device(s) 2120 may embody a computer readable medium. Such a computer readable medium may be coupled to the central processor 2106 via the system bus 2112. Additionally, such a computer readable medium may comprise code or instructions, executable by the central processor 2106 for performing methods according to embodiments, e.g., those described above. Another subsystem is a data collection device 2110, such as a camera, microphone, accelerometer, and the like. Any of the data mentioned herein can be output from one component to another component and can be output to the user.

[0220] A computer system can include a plurality of the same components or subsystems, e.g., connected together by external interface 2122, by an internal interface, or via removable storage devices that can be connected and removed from one component to another component. In some embodiments, computer systems, subsystem, or apparatuses can communicate over a network. In such instances, one computer can be considered a client and another computer a server, where each can be part of a same computer system. A client and a server can each include multiple systems, subsystems, or components.

[0221] Any of the computer systems mentioned herein may utilize any suitable number of subsystems. In some embodiments, a computer system includes a single computer apparatus, where the subsystems can be components of the computer apparatus. In other embodiments, a computer system can include multiple computer apparatuses, each being a subsystem, with internal components. 61 78601649V.1

[0222] A computer system can include a plurality of the components or subsystems, e.g., connected together by external interface or by an internal interface. In some embodiments, computer systems, subsystems, or apparatuses can communicate over a network. In such instances, one computer can be considered a client and another computer a server, where each can be part of a same computer system. A client and a server can each include multiple systems, subsystems, or components.

[0223] It should be understood that any of the embodiments of the present invention can be implemented in the form of control logic using hardware (e.g., an application specific integrated circuit or field programmable gate array) and / or using computer software with a generally programmable processor in a modular or integrated manner. As used herein a processor includes a single-core processor, multi-core processor on a same integrated chip, or multiple processing units on a single circuit board or networked. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will know and appreciate other ways and / or methods to implement embodiments of the present invention using hardware and a combination of hardware and software.

[0224] Any of the software components or functions described in this application may be implemented as software code to be executed by a processor using any suitable computer language such as, for example, Java, C, C++, C#, Objective-C, Swift, or scripting language such as Perl or Python using, for example, conventional or object-oriented techniques. The software code may be stored as a series of instructions or commands on a computer readable medium for storage and / or transmission, suitable media include random access memory (RAM), a read only memory (ROM), a magnetic medium such as a hard-drive or a floppy disk, or an optical medium such as a compact disk (CD) or DVD (digital versatile disk), flash memory, and the like. The computer readable medium may be any combination of such storage or transmission devices.

[0225] Such programs may also be encoded and transmitted using carrier signals adapted for transmission via wired, optical, and / or wireless networks conforming to a variety of protocols, including the Internet. As such, a computer readable medium according to an embodiment of the present invention may be 62 78601649V.1created using a data signal encoded with such programs. Computer readable media encoded with the program code may be packaged with a compatible device or provided separately from other devices (e.g., via Internet download). Any such computer readable medium may reside on or within a single computer product (e.g., a hard drive, a CD, or an entire computer system), and may be present on or within different computer products within a system or network. A computer system may include a monitor, printer or other suitable display for providing any of the results mentioned herein to a user.

[0226] Any of the methods described herein may be totally or partially performed with a computer system including one or more processors, which can be configured to perform the steps. Thus, embodiments can involve computer systems configured to perform the steps of any of the methods described herein, potentially with different components performing a respective steps or a respective group of steps. Although presented as numbered steps, steps of methods herein can be performed at a same time or in a different order. Additionally, portions of these steps may be used with portions of other steps from other methods. Also, all or portions of a step may be optional. Additionally, and of the steps of any of the methods can be performed with modules, circuits, or other means for performing these steps.

[0227] The specific details of particular embodiments may be combined in any suitable manner without departing from the spirit and scope of embodiments of the invention. However, other embodiments of the invention may involve specific embodiments relating to each individual aspect, or specific combinations of these individual aspects. The above description of exemplary embodiments of the invention has been presented for the purpose of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form described, and many modifications and variations are possible in light of the teaching above. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications to thereby enable others skilled in the art to best utilize the invention in various embodiments and with various modifications as are suited to the particular use contemplated.

[0228] The above description is illustrative and is not restrictive. Many variations of the invention will become apparent to those skilled in the art upon 63 78601649V.1review of the disclosure. The scope of the invention should, therefore, be determined not with reference to the above description, but instead should be determined with reference to the pending claims along with their full scope or equivalents.

[0229] One or more features from any embodiment may be combined with one or more features of any other embodiment without departing from the scope of the invention.

[0230] A recitation of “a”, “an” or “the” is intended to mean “one or more” unless specifically indicated to the contrary. The use of “or” is intended to mean an “inclusive or,” and not an “exclusive or” unless specifically indicated to the contrary.

[0231] All patents, patent applications, publications and description mentioned herein are incorporated by reference in their entirety for all purposes. None is admitted to be prior art. 64 78601649V.1REFERENCES [1] Abdoli, A. C. Murillo, C.-C. M. Yeh, A. C. Gerry, and E. J. Keogh, “Time series classification to improve poultry welfare,” in 2018 17TH IEEE International conference on machine learning and applications (ICMLA). IEEE, 2018, pp. 635–642. [2] A. Bagnall, H. A. Dau, J. Lines, M. Flynn, J. Large, A. Bostrom, P. Southam, and E. Keogh, “The UEA multivariate time series classification archive, 2018,” arXiv preprint arXiv:1811.00075, 2018. [3] H. A. Dau, A. Bagnall, K. Kamgar, C.-C. M. Yeh, Y. Zhu, S. Gharghabi, C. A. Ratanamahatana, and E. Keogh, “The UCR time series archive,” IEEE / CAA Journal of Automatica Sinica, vol.6, no.6, pp.1293–1305, 2019. [4] J.-H. Li, L. Tian, H. Wang, Y. An, K. Wang, and L. Yu, “Segmentation and recognition of basic and transitional activities for continuous physical human activity,” IEEE access, vol.7, pp.42565–42576, 2019. [5] C.-C. M. Yeh, Y. Zhu, L. Ulanova, N. Begum, Y. Ding, H. A. Dau, Z. Zimmerman, D. F. Silva, A. Mueen, and E. Keogh, “Time series joins, motifs, discords and shapelets: a unifying view that exploits the matrix profile,” Data Mining and Knowledge Discovery, vol.32, pp.83–123, 2018. [6] Y. Zhu, S. Gharghabi, D. F. Silva, H. A. Dau, C.-C. M. Yeh, N. Shakibay Senobari, A. Almaslukh, K. Kamgar, Z. Zimmerman, G. Funning et al., “The swiss army knife of time series data mining: ten useful things you can do with the matrix profile and ten lines of code,” Data Mining and Knowledge Discovery, vol.34, pp.949–979, 2020. [7] C.-C. M. Yeh, Y. Zheng, J. Wang, H. Chen, Z. Zhuang, W. Zhang, and E. Keogh, “Error-bounded approximate time series joins using compact dictionary representations of time series,” in Proceedings of the 2022 SIAM International Conference on Data Mining (SDM). SIAM, 2022, pp.181–189. [8] Y. Zhu, Z. Zimmerman, N. S. Senobari, C.-C. M. Yeh, G. Funning, A. Mueen, P. Brisk, and E. Keogh, “Matrix Profile II: Exploiting a novel algorithm and gpus to break the one hundred million barrier for time series motifs and joins,” 65 78601649V.1in 2016 IEEE 16th international conference on data mining (ICDM). IEEE, 2016, pp.739–748. [9] C.-C. M. Yeh, Y. Zhu, L. Ulanova, N. Begum, Y. Ding, H. A. Dau, D. F. Silva, A. Mueen, and E. Keogh, “Matrix Profile I: all pairs similarity joins for time series: a unifying view that includes motifs, discords and shapelets,” in 2016 IEEE 16th international conference on data mining (ICDM). IEEE, 2016, pp. 1317–1322.

[0010] A. Mueen, E. Keogh, Q. Zhu, S. Cash, and B. Westover, “Exact discovery of time series motifs,” in Proceedings of the 2009 SIAM international conference on data mining. SIAM, 2009, pp.473–484.

[0011] Z. Zimmerman, K. Kamgar, N. S. Senobari, B. Crites, G. Funning, P. Brisk, and E. Keogh, “Matrix profile XIV: scaling time series motif discovery with gpus to break a quintillion pairwise comparisons a day and beyond,” in Proceedings of the ACM Symposium on Cloud Computing, 2019, pp.74–86.

[0012] C.-C. M. Yeh, N. Kavantzas, and E. Keogh, “Matrix Profile VI: Meaningful multidimensional motif discovery,” in 2017 IEEE international conference on data mining (ICDM). IEEE, 2017, pp.565–574.

[0013] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, vol.9, no.8, pp.1735–1780, 1997.

[0014] B. Lim and S. Zohren, “Time series forecasting with deep learning: a survey,” Philosophical Transactions of the Royal Society A, vol. 379, no. 2194, p. 20200209, 2021.

[0015] H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, “Informer: Beyond efficient transformer for long sequence time series forecasting,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol.35, no.12, 2021, pp.11106 – 11115.

[0016] K. Cho, B. Van Merrienboer, D. Bahdanau, and Y. Bengio, “On the properties of neural machine translation: Encoder-decoder approaches,” arXiv preprint arXiv:1409.1259, 2014. 66 78601649V.1

[0017] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp.770–778.

[0018] Z. Wang, W. Yan, and T. Oates, “Time series classification from scratch with deep neural networks: A strong baseline,” in 2017 International joint conference on neural networks (IJCNN). IEEE, 2017, pp.1578– 1585.

[0019] H. Ismail Fawaz, G. Forestier, J. Weber, L. Idoumghar, and P.-A. Muller, “Deep learning for time series classification: a review,” Data mining and knowledge discovery, vol.33, no.4, pp.917–963, 2019.

[0020] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol.30, 2017.

[0021] S. Li, X. Jin, Y. Xuan, X. Zhou, W. Chen, Y.-X. Wang, and X. Yan, “Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting,” Advances in neural information processing systems, vol. 32, 2019.

[0022] H. Chen, Y. Lin, M. Pan, L. Wang, C.-C. M. Yeh, X. Li, Y. Zheng, F. Wang, and H. Yang, “Denoising self-attentive sequential recommendation,” in Proceedings of the 16th ACM Conference on Recommender Systems, 2022, pp.92–101.

[0023] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.

[0024] J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint arXiv:1607.06450, 2016. 67 78601649V.1

Claims

WHAT IS CLAIMED IS:

1. A method for classifying subsequences of time series data using a machine learning model comprising at least an output sub-model, the method comprising: receiving, by a computer system, a sequence of time series data comprising one or more subsequences; constructing, by the computer system, a k-nearest neighbor graph based on the one or more subsequences; and producing, by the computer system, one or more subsequence classifications corresponding to the one or more subsequences by performing, for each subsequence of the one or more subsequences: (1) determining, by the computer system, one or more nearest neighbor subsequences corresponding to the subsequence using the k-nearest neighbor graph, (2) producing, by the computer system, a subsequence embedding and one or more nearest neighbor embeddings based on the subsequence and the one or more nearest neighbor subsequences, (3) producing, by the computer system, a combined embedding by combining the subsequence embedding and the one or more nearest neighbor embeddings, and (4) producing, by the computer system, a subsequence classification using the output sub-model and the combined embedding, thereby producing the one or more subsequence classifications.

2. The method of claim 1, wherein the machine learning model comprises a backbone sub-model, and wherein the computer system produces the subsequence embedding and the one or more nearest neighbor embeddings using the backbone sub-model.

3. The method of claim 2, wherein the backbone sub-model comprises a machine learning model with a Long Short-Term Memory Network (LSTM) architecture, a recurrent neural network (RNN) architecture, a Gated 68 78601649V.1Recurrent Unit Network (GRU) architecture, a Residual Network (ResNet) architecture, or a transformer architecture.

4. The method of claim 1, wherein the one or more nearest neighbor subsequences comprise one or more training subsequences from a training sequence of time series data, and wherein producing the combined embedding by combining the subsequence embedding and the one or more nearest neighbor embeddings further comprises: combining, by the computer system, each nearest neighbor embedding of the one or more nearest neighbor embeddings with a corresponding label embedding of one or more label embeddings, wherein the one or more label embeddings correspond to one or more training labels of the one or more training subsequences.

5. The method of claim 1, wherein the subsequence embedding comprises a subsequence embedding vector, wherein the one or more nearest neighbor embeddings comprise one or more nearest neighbor embedding vectors, wherein the combined embedding comprises a combined embedding matrix, and wherein combining the subsequence embedding and the one or more nearest neighbor embeddings comprises concatenating the subsequence embedding vector and the one or more nearest neighbor embedding vectors, thereby producing the combined embedding matrix.

6. The method of claim 1, wherein the output sub-model comprises a transformer sub-model and a classifier sub-model, and wherein producing a subsequence classification using the output sub-model and the combined embedding comprises: producing, by the computer system, an enhanced combined embedding using the transformer sub-model and the combined embedding; and producing, by the computer system, a subsequence classification using the classifier sub-model and the enhanced combined embedding.

7. The method of claim 6, wherein producing a subsequence classification using the classifier sub-model and the enhanced combined embedding comprises: 69 78601649V.1determining, by the computer system, a focal representation based on the enhanced combined embedding; and producing, by the computer system, the subsequence classification by inputting the focal representation into the classifier sub-model.

8. The method of claim 6, wherein the transformer sub-model comprises one or more multi-head self-attention transformer blocks.

9. The method of claim 6, automatically performing an action based upon the one or more sub-sequence classifications.

10. The method of claim 1, wherein each subsequence classification of the one or more subsequence classifications indicates one or more likelihoods that a corresponding subsequence is a member of one or more classes.

11. The method of claim 1, wherein the one or more subsequence classifications comprise a plurality of subsequence classifications, and wherein the method further comprises: identifying, by the computer system, a plurality of onset subsequence classifications from the plurality of subsequence classifications; identifying, by the computer system, a plurality of subsequence classification subsets based on the plurality of onset subsequence classifications and a predetermined window size value, wherein each subsequence classification subset of the plurality of subsequence classification subsets comprises an onset subsequence classification and a number of subsequence classifications proportional to the predetermined window size value; determining, by the computer system, for each subsequence classification subset, a majority subsequence classification; and replacing, by the computer system, each subsequence classification of the plurality of subsequence classifications with a corresponding majority subsequence classification corresponding to one or more subsequence classification subsets to which the subsequence classification belongs, thereby producing a plurality of post-processed subsequence classifications. 70 78601649V.

112. The method of claim 1, wherein the one or more subsequences are representative of one or more events, wherein the method is performed to qualify or quantify these events for a requestor, and wherein the method further comprises: receiving, by the computer system, from the requestor, a request to qualify or quantify one or more events represented by the sequence of time series data; and providing, by the computer system, to the requestor, the one or more subsequence classifications responsive to the request, thereby qualifying or quantifying the one or more events.

13. The method of claim 1, wherein constructing the k-nearest neighbor graph based on the one or more subsequences comprises: generating, by the computer system, a distance profile by computing a distance between each subsequence of the one or more subsequences and one or more training subsequences from a training sequence of time series data; and determining, by the computer system, using the distance profile, for each subsequence of the one or more subsequences, k training subsequences or k training indices corresponding to k minimum distance training subsequences of the one or more training subsequences, thereby determining one or more training subsequences or one or more training indices corresponding to the one or more subsequences, wherein the k-nearest neighbor graph comprises the one or more training subsequences or one or more training indices for each subsequence of the one or more subsequences.

14. The method of claim 13, wherein the k-nearest neighbor graph comprises an n by k matrix, where n is a number of subsequences in the one or more subsequences and k is a number of minimum distance training subsequences of the one or more training subsequences, and wherein each element of the n by k matrix comprises a training index corresponding to a minimum distance training subsequence.

15. A method for training a machine learning model comprising at least an output sub-model to classify subsequences of time series data, the method comprising: 71 78601649V.1receiving, by a computer system, a training sequence of time series data comprising a plurality of training subsequences; constructing, by the computer system, based on the plurality of training subsequences, a k-nearest neighbor graph; and performing an iterative training process until a terminating condition has been met, the iterative training process comprising a plurality of training rounds, each training round comprising: sampling, by the computer system, a batch of training subsequences comprising one or more training subsequences from the plurality of training subsequences, producing, by the computer system, one or more subsequence classifications corresponding to the batch of training subsequences by performing, for each training subsequence of the batch of training subsequences: (1) determining, by the computer system, one or more nearest neighbor subsequences corresponding to the training subsequence using the k-nearest neighbor graph and a training index corresponding to the training subsequence, (2) producing, by the computer system, a subsequence embedding and one or more nearest neighbor embeddings based on the training subsequence and the one or more nearest neighbor subsequences, (3) producing, by the computer system, a combined embedding by combining the subsequence embedding and the one or more nearest neighbor embeddings, and (4) producing, by the computer system, a subsequence classification using the output sub-model and the combined embedding, thereby producing the one or more subsequence classifications, and generating, by the computer system, one or more loss values based on the one or more subsequence classifications and one or more training labels corresponding to the batch of training subsequences, 72 78601649V.1updating, by the computer system, the machine learning model based on the one or more loss values, and performing, by the computer system, the iterative training process until the terminating condition has been met.

16. The method of claim 15, wherein the plurality of training subsequences each comprise a predefined number of time series data values, and wherein the batch of training subsequences comprises a number of training subsequences based on the predefined number of time series data values and a number of time series data values in the training sequence of time series data.

17. The method of claim 15, wherein the machine learning model comprises a backbone sub-model, wherein the computer system produces the subsequence embedding and the one or more nearest neighbor embeddings using the backbone sub-model, wherein the backbone sub-model is defined by a set of backbone sub-model parameters, and wherein updating the machine learning model based on the one or more loss values comprises updating the set of backbone sub- model parameters based on the one or more loss values.

18. The method of claim 15, wherein the output sub-model is defined by a set of output sub-model parameters, and wherein updating the machine learning model based on the one or more loss values comprises updating the set of output sub-model parameters based on the one or more loss values.

19. The method of claim 18, wherein the output sub-model comprises a transformer sub-model defined by a set of transformer sub-model parameters and a classifier sub-model defined by a set of classifier sub-model parameters, and wherein the set of output sub-model parameters comprises the set of transformer sub-model parameters and the set of classifier sub-model parameters.

20. A computer system comprising: a processor; and a non-transitory computer readable medium coupled to the processor, the non-transitory computer readable medium comprising code or instructions, executable by the processor for performing a method for classifying subsequences 73 78601649V.1of time series data using a machine learning model comprising at least an output sub-model, the method comprising: receiving a sequence of time series data comprising one or more subsequences; constructing a k-nearest neighbor graph based on the one or more subsequences; and producing one or more subsequence classifications corresponding to the one or more subsequences by performing, for each subsequence of the one or more subsequences: (1) determining one or more nearest neighbor subsequences corresponding to the subsequence using the k-nearest neighbor graph, (2) producing a subsequence embedding and one or more nearest neighbor embeddings based on the subsequence and the one or more nearest neighbor subsequences, (3) producing, by the computer system, a combined embedding by combining the subsequence embedding and the one or more nearest neighbor embeddings, and (4) producing, by the computer system, a subsequence classification using the output sub-model and the combined embedding, thereby producing the one or more subsequence classifications. 74 78601649V.1

Images