Energy saving system and method for iteratively analyzing time series data using attention power

Abstract

A computer-implemented method for analyzing time series data is provided. The method includes providing a series of time series batches of time series data to an attention power iteration (API) model. The method further includes batch generating, by the API model, a series of time series sketches based on the series of time series of time series data. The method also includes assigning a weight to a new time series batch of the series of time series batches based at least in part on a previous time series sketch of the series of time series sketches. The method also includes generating an output for each time series sketch of the series of time series sketches.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is a non-provisional application claiming the benefit of U.S. Provisional Application No. 63 / 555,190, filed on February 19, 2024, under 35 U.S.C. §119(e), which is hereby incorporated by reference in its entirety. Technical Field

[0003] The present disclosure generally relates to systems and methods for analyzing time series data. In particular, the present disclosure relates to systems and methods for analyzing time series data by implementing attention weighting. Background Art

[0004] Resource-constrained devices, characterized by limited computing power and memory, are essential components of modern infrastructure. Devices such as edge devices (e.g., routers, smartphones) and distributed energy resources (DERs) (such as solar PV units, turbines, and gas units) fall into this category. These devices accumulate large amounts of high-frequency time series data streams over time, making manual real-time analysis impractical. Artificial intelligence (AI) models, particularly those operating on time series data, have gained attention due to their ability to autonomously identify underlying patterns. However, existing AI models, particularly popular neural networks, often require large amounts of memory and computing energy, making them unsuitable for direct deployment on resource-constrained devices. While a common workaround is to transmit the collected data to a cloud server for analysis, this approach consumes bandwidth, time, and raises concerns about data privacy and security during transmission. In addition, many AI models rely on batch learning, which processes the entire time series simultaneously, rather than the incremental learning required for real-time analysis.

[0005] To address these challenges, streaming and cost-effective models that can analyze time series data streams in real time directly on resource-constrained devices are urgently needed. Summary of the Invention

[0006] Aspects and advantages of computer-implemented methods and computing systems according to the present disclosure will be set forth in part in the following description, or may be obvious from the description, or may be learned through practice of the technology.

[0007] According to one embodiment, a computer-implemented method for analyzing time series data is provided. The method includes providing a series of time series batches of time series data to an attention power iteration (API) model. The method also includes generating, by the API model, a series of time series sketches based on the series of time series batches of time series data. The method also includes assigning weights to new time series batches in the series of time series batches based at least in part on previous time series sketches in the series of time series sketches. The method also includes generating an output for each time series sketch in the series of time series sketches.

[0008] According to another embodiment, a computing system is provided. The computing system includes one or more processors. The computing system also includes one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform an operation. The operation includes providing a series of time series batches of time series data to an attention power iteration (API) model. The operation also includes generating a series of time series sketches based on the series of time series batches of time series data by the API model. The operation also includes assigning weights to new time series batches in the series of time series batches based at least in part on previous time series sketches of the series of time series sketches. The operation also includes generating an output for each time series sketch in the series of time series sketches.

[0009] These and other features, aspects, and advantages of the computer-implemented method and computing system of the present invention will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present invention and, together with the description, serve to explain the principles of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] This specification, with reference to the accompanying drawings, sets forth a complete and enabling disclosure of the computer-implemented method and computing system of the present invention, including the best mode of making and using the system and method of the present invention, to those skilled in the art, wherein:

[0011] Figure 1 illustrates a block diagram of an example computing system that performs real-time analysis of time series data according to example embodiments of the present disclosure;

[0012] Figure 2 The embodiment according to the present disclosure is illustrated by Figure 1 A block diagram of an example streaming time series classification model implemented by the computing system shown;

[0013] Figure 3 The embodiment according to the present disclosure is illustrated by Figure 1A block diagram of an example streaming time series classification model implemented by the presented computing system;

[0014] Figure 4 A flowchart illustrating a computer-implemented method for analyzing time series data according to an embodiment of the present disclosure;

[0015] Figure 5 illustrates a process according to an embodiment of the present disclosure; and

[0016] Figure 6 A flow chart illustrating a computer-implemented method for analyzing time series data according to an embodiment of the present disclosure is illustrated. DETAILED DESCRIPTION

[0017] Reference will now be made in detail to embodiments of the present computer-implemented method and computing system, one or more examples of which are illustrated in the accompanying drawings. Each example is provided by way of explanation of the present invention, not limitation of the present invention. Indeed, it will be apparent to those skilled in the art that modifications and variations can be made in the present invention without departing from the scope or spirit of the present invention as protected by the claims. For example, features illustrated or described as part of one embodiment can be used in another embodiment to produce yet another embodiment. Therefore, this disclosure is intended to cover such modifications and variations as fall within the scope of the appended claims and their equivalents.

[0018] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations. Additionally, all embodiments described herein should be considered exemplary unless specifically stated otherwise.

[0019] The detailed description uses numerical and letter designations to refer to features in the drawings. Like or similar designations in the drawings and description have been used to refer to like or similar components of the present invention. As used herein, the terms "first," "second," and "third" are used interchangeably to distinguish one component from another and are not intended to indicate the position or importance of each component.

[0020] Terms with approximate meanings (such as "about," "approximately," "substantially," and "substantially") are not limited to the precise values specified. In at least some cases, approximate language may correspond to the precision of an instrument used to measure a value, or the precision of a method or machine used to construct or manufacture a component and / or system. In at least some cases, approximate language may correspond to the precision of an instrument used to measure a value, or the precision of a method or machine used to construct or manufacture a component and / or system. For example, approximate language may refer to within a tolerance of 1%, 2%, 4%, 5%, 10%, 15%, or 20% of an individual value, a range of values, and / or an end value of a range of values. When used in the context of an angle or direction, such terms include within ten degrees greater than or less than the angle or direction. For example, "substantially vertical" includes directions within ten degrees of vertical in any direction (e.g., clockwise or counterclockwise).

[0021] As used herein, the terms "comprises," "includes," "has," or any other variations thereof are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a list of features is not necessarily limited to only those features but may include other features not expressly listed or inherent to such process, method, article, or apparatus. Furthermore, unless expressly stated to the contrary, "or" refers to an inclusive or and not an exclusive or. For example, condition A or B is satisfied by any one of the following: A is true (or exists) and B is false (or does not exist); A is false (or does not exist) and B is true (or exists); and both A and B are true (or exist).

[0022] Here and throughout the specification and claims, unless context or language indicates otherwise, range limitations are combined and interchangeable, and such ranges are identified and include all sub-ranges contained therein. For example, all ranges disclosed herein are inclusive of the endpoints, and the endpoints are independently combinable with each other.

[0023] The present application relates to systems and methods for solving the problem of streaming time series classification, in which a classification class label is predicted for each time series consisting of an ordered set of real-valued attributes (often multivariate). Streaming time series classification operates on a stream of time series, generating real-time labels while processing the data only once and utilizing limited storage. The present disclosure includes a neural network for streaming time series classification that is designed to operate with limited computational memory and energy resources while ensuring high processing speed.

[0024] For example, when data from the same time series are realized sequentially, the system can use incremental weighted principal component analysis (PCA) on the latent space of supervised neural network learning to construct a real-time representation. This representation is then used for time series classification. Although traditional PCA is an unsupervised method for dimensionality reduction of non-time series data, the system and method of the present disclosure applies weighted PCA to the latent space of time and supervised learning to adapt to the nature of time series data. Since time series samples are not equally important, the system and method of the present disclosure introduces weighted PCA, in which a temporal attention model is used to assign weights to each time series sample. Using such weighted time series streams, the model can gradually update the main direction, thereby forming different real-time representations for classification. The complete system architecture includes latent space learning and incremental weighted PCA, which is trained end-to-end in a supervised learning manner. Specifically, the model can use streaming attention power iteration (strAPI) to gradually improve the main direction within the supervised latent space. The model can generate real-time representations and labels for each time series without observing the entire sequence. When compared to other similar systems, the systems and methods of the present disclosure have been shown to consistently achieve superior classification accuracy with minimal energy consumption and faster computation speeds on average.

[0025] The systems and methods of the present disclosure provide multiple technical effects and benefits. As an example, the system and method can be used to utilize machine learning models and attention power iteration models, thereby enabling real-time processing of time series data on resource-constrained devices. For example, the model continuously updates a compact representation of the entire time series, thereby enhancing classification (e.g., output) accuracy while saving energy and processing time. It should be noted that the model performs well in streaming scenarios where access to the complete time series is not required, thereby enabling rapid decisions. The model performs well in classification accuracy and energy efficiency, with consumption exceeding 70% less than the baseline and task completion three times faster than the baseline. This work improves real-time responsiveness, energy conservation, and operational efficiency for constrained devices, thereby helping to optimize various applications.

[0026] Referring now to the drawings, example embodiments of the present disclosure will be discussed in greater detail.

[0027] Figure 1 A block diagram of an example computing system 100 that performs real-time analysis of time series data is depicted, according to an example embodiment of the present disclosure. System 100 includes a user computing device 102, a server computing system 130, and a training computing system 150 communicatively coupled via a network 180.

[0028] The user computing device 102 can be any type of computing device, such as, for example, a control computing device for a machine (e.g., a control computing device for a wind turbine, a water turbine, a crane, etc.), a personal computing device (e.g., a laptop or desktop computer), a mobile computing device (e.g., a smart phone or tablet computer), a diagnostic computing device for diagnosing machine anomalies, a wearable computing device, an embedded computing device, an edge computing device, or any other type of computing device.

[0029] The user computing device 102 includes one or more processors 112 and memory 114. The one or more processors 112 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one or more processors that are operatively connected. The memory 114 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 114 can store data 116 and instructions 118 that are executed by the processor 112 to cause the user computing device 102 to perform operations.

[0030] In some implementations, the user computing device 102 can store or include one or more machine learning models 120. For example, the machine learning model 120 can be a variety of machine learning models, or can otherwise include a variety of machine learning models, such as a neural network (e.g., a deep neural network) or other types of machine learning models, including nonlinear models and / or linear models. The neural network can include a feedforward neural network, a recurrent neural network (e.g., a long short-term memory recurrent neural network), a convolutional neural network, a temporal convolutional network (TCN), or other forms of neural networks. Some example machine learning models can utilize attention mechanisms, such as self-attention. For example, some example machine learning models can include a multi-head self-attention model (e.g., a transformer model). Reference Figures 2 to 3 An example machine learning model 120 is discussed.

[0031] In some implementations, one or more machine learning models 120 can be received from the server computing system 130 over the network 180, stored in the user computing device memory 114, and then used or otherwise implemented by the one or more processors 112. In some implementations, the user computing device 102 can implement multiple parallel instances of a single machine learning model 120.

[0032] Additionally or alternatively, one or more machine learning models 140 may be included in, or otherwise stored and implemented by, a server computing system 130 that communicates with the user computing device 102 according to a client-server relationship. For example, the machine learning models 140 may be implemented as part of a web service by the server computing system 130. Thus, one or more models 120 may be stored and implemented at the user computing device 102, and / or one or more models 140 may be stored and implemented at the server computing system 130.

[0033] The user computing device 102 may also include one or more user input components 122 for receiving user input. For example, the user input component 122 may be a touch-sensitive component (e.g., a touch-sensitive display or touchpad), a microphone, a traditional keyboard, or other means by which a user can provide user input.

[0034] The user computing device 102 may include one or more sensors and / or may be communicatively coupled to one or more sensors. The one or more sensors may be configured to generate time series data. The time series data may describe the performance of a machine and / or environmental signal over a period of time.

[0035] The server computing system 130 includes one or more processors 132 and memory 134. The one or more processors 132 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one or more processors that are operatively connected. The memory 134 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 134 can store data 136 and instructions 138 that are executed by the processor 132 to cause the server computing system 130 to perform operations.

[0036] In some implementations, the server computing system 130 includes, or is otherwise implemented by, one or more server computing devices. Where the server computing system 130 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0037] As described above, the server computing system 130 may store one or more machine learning models 140, or otherwise include the one or more machine learning models. For example, the model 140 may be a variety of machine learning models, or may otherwise include a variety of machine learning models. Example machine learning models include neural networks or other multi-layer nonlinear models. Example neural networks include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Some example machine learning models may utilize attention mechanisms, such as self-attention. For example, some example machine learning models may include a multi-head self-attention model (e.g., a transformer model). Reference Figures 2 to 3 An example model 140 is discussed.

[0038] User computing device 102 and / or server computing system 130 may train models 120 and / or 140 via interaction with a training computing system 150 communicatively coupled via network 180. Training computing system 150 may be separate from server computing system 130, may be part of server computing system 130, or may be part of user computing system 102.

[0039] The training computing system 150 includes one or more processors 152 and a memory 154. The one or more processors 152 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.), and can be one or more processors that are operatively connected. The memory 154 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 154 can store data 156 and instructions 158 that are executed by the processor 152 to cause the training computing system 150 to perform operations. In some specific implementations, the training computing system 150 includes one or more server computing devices, or is otherwise implemented by one or more server computing devices.

[0040] The training computing system 150 may include a model trainer 160 that trains the machine learning models 120 and / or 140 stored at the user computing device 102 and / or the server computing system 130 using various training or learning techniques (e.g., such as backpropagation of error). For example, a loss function may be backpropagated through the model to update one or more parameters of the model (e.g., based on the gradient of the loss function). Various loss functions may be used, such as mean squared error, likelihood loss, cross entropy loss, hinge loss, and / or various other loss functions. Gradient descent techniques may be used to iteratively update the parameters over multiple training iterations.

[0041] In some implementations, performing backpropagation of errors may include performing truncated backpropagation through time. Model trainer 160 may perform various generalization techniques (e.g., weight decay, dropout, etc.) to improve the generalization ability of the model being trained. Model trainer 160 may include one or more teacher models for performing distillation training to generate a compressed model that can be implemented via computing device 102.

[0042] The model trainer 160 includes computer logic for providing the desired functionality. The model trainer 160 can be implemented in hardware, firmware, and / or software that controls a general-purpose processor. For example, in some implementations, the model trainer 160 includes a program file stored on a storage device, loaded into memory, and executed by one or more processors. In other implementations, the model trainer 160 includes one or more sets of computer-executable instructions stored in a tangible computer-readable storage medium, such as RAM, a hard disk, or optical or magnetic media.

[0043] The network 180 can be any type of communication network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and can include any number of wired or wireless links. In general, communications on the network 180 can be performed via any type of wired and / or wireless connection using a variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, secure HTTP, SSL).

[0044] The machine learning models described in this specification can be used in a variety of tasks, applications, and / or use cases.

[0045] In some implementations, the input to the machine learning model of the present disclosure can be, for example, sensor data from an environmental sensor 182. The environment can be operatively connected to (or coupled to) a power generation system 184, such as a gas turbine engine, a water turbine, a wind turbine, or other power generation system.

[0046] A machine learning model may process sensor data to generate an output. As an example, a machine learning model may process sensor data to generate a recognition output. As another example, a machine learning model may process sensor data to generate a prediction output. As another example, a machine learning model may process sensor data to generate a classification output. As another example, a machine learning model may process sensor data to generate a segmentation output. As another example, a machine learning model may process sensor data to generate a visualization output. As another example, a machine learning model may process sensor data to generate a diagnostic output. As another example, a machine learning model may process sensor data to generate a detection output.

[0047] Figure 1 An example computing system that can be used to implement the present disclosure is illustrated. Other computing systems may also be used. For example, in some implementations, the user computing device 102 may include a model trainer 160 and a training dataset 162. In such implementations, the model 120 may be trained and used locally at the user computing device 102. In some such implementations, the user computing device 102 may implement the model trainer 160 to personalize the model 120 based on user-specific data.

[0048] In many embodiments, computing system 100 may be a resource-constrained device (e.g., having limited memory and / or processing power). In other embodiments, computing system 100 may be an edge computing device.

[0049] Figure 2 A block diagram depicts an example streaming time series classification model 200 according to an example embodiment of the present disclosure, which may be referenced above. Figure 1 The computing system 100 described herein is implemented. In some specific implementations, the model 200 is trained to receive a series of time series streams or batches 202. The model 200 can also be configured to perform weighted principal component analysis (e.g., weighted PCA or WPCA) on the time series batches 202 via an attention power iteration (API) model (or weighting block) 204 to generate a series of time series sketches 206.

[0050] The API model 204 can gradually improve the main direction in the supervised latent space. The API model 204 can find the main eigenvalues and eigenvectors of the large time series batch 202 and can incorporate an attention mechanism. The attention mechanism enables the API model 204 to weight certain features or dimensions more heavily, thereby improving performance. The API model 204 can advantageously enable the temporal patterns of the time series batch 202 to be effectively captured by iteratively updating the reduced representation of the time series batch 202 (i.e., by iteratively updating the series of time series sketches 206). The API model 204 can ensure that significant temporal patterns are highlighted by assigning corresponding weights to significant temporal patterns, thereby achieving accurate classification. For example, significant temporal patterns may include (but are not limited to) trends (e.g., long-term directional movements or trends), seasonality (e.g., periodic fluctuations or cycles), cyclical patterns (e.g., non-seasonal fluctuations), or other patterns.

[0051] The model 200 can also be configured to provide real-time analysis of the series of time series sketches (such as prediction, classification, or other output) via a classification model 208. In some embodiments, as shown, the classification model 208 can generate a classification label 210 for each time series sketch 206A, 206B, 206C in the series of time series sketches 206.

[0052] As shown, the model 200 may be provided with a series of time series batches 202, such as a first time series batch 202A, a second time series batch 202B, and so on, up to an Nth time series batch 202C. The series of time series batches 202 may each be a portion of a time series (or time series data) that may be reduced by the model 200 before being assigned weights in the API model 204. As a non-limiting example, given a time series of data collected over a 10-minute period, the first time series batch 102A may be the first minute of data (e.g., data from the time series collected from minutes 0 to 1), the second time series batch 102B may be the second minute of data (e.g., data from the time series collected from minutes 1 to 2), and so on. The time series from which the time series batches 202 are formed may include any suitable time series of data. For example, the time series may be sensor data, weather data, supply chain data, energy consumption data, stock price data, or other data collected over a period of time (e.g., periodically).

[0053] The API model 204 can advantageously enable the temporal patterns of the time series batch 202 to be effectively captured by iteratively updating the reduced representation of the time series batch 202 (i.e., by iteratively updating the series of time series sketches 206). The API model 204 can ensure that significant temporal patterns are highlighted by assigning corresponding weights to significant temporal patterns, thereby achieving accurate classification. For example, significant temporal patterns may include (but are not limited to) trends (e.g., long-term directional movements or trends), seasonality (e.g., periodic fluctuations or cycles), cyclical patterns (e.g., non-seasonal fluctuations), or other patterns.

[0054] In the API model 204, weighted PCA can be applied to each of the time series batches 202. Weighted PCA can be an extension of the traditional PCA method, in which the importance or significance of each time series batch 202A, 202B, 202C in the set of time series batches 202 is weighted. For example, in traditional PCA, all data points are treated equally (e.g., given equal weights). In contrast, in weighted PCA (or WPCA), each time series batch 202A, 202B, 202C is assigned (or adjusted by) a weight that reflects its relative importance or contribution to the analysis. Weighted PCA can include multiple steps, such as (but not limited to) a data processing step, a weighting step, a covariance matrix calculation, an eigendecomposition step, a principal component selection step, and a dimensionality reduction step.

[0055] The weighting of the time series batch by the API model 204 can be based on the importance of the temporal pattern. For example, the model 200 can be configured to weight certain trends identified in the time series batch 202 more heavily than other trends.

[0056] Model 200 integrates matrix sketching technology specifically tailored for real-time time series classification within a neural network architecture. Figure 2 As shown, the method relates to unsupervised matrix sketches being combined in a supervised context. That is, matrix sketches can relate to creating smaller approximate representations (e.g., sketches or matrix sketches). Matrix sketches can be used to reduce data dimensions while retaining important structural information. In a supervised context, this integration of matrix sketches can enable the sketches linked to a series of weighted time series streams to be continuously updated. The dynamically updated sketch can be used as a compact representation of an input time series, thereby being convenient to be classified into known time series categories in real time at any given timestamp. This fusion of unsupervised matrix sketches and supervised neural networks solves the demand for real-time processing while maintaining accuracy and efficiency of classification tasks.

[0057] Figure 3 A block diagram of a streaming time series classification model 300 according to an example embodiment of the present disclosure is illustrated, which may be referred to above. Figure 1 The computing system 100 described herein is implemented. As shown, the model 300 may be provided with time series data 302 as input, and the model 300 may generate an output 304 (such as a classification label, a prediction, or other output). The time series data 302 may be, for example, raw time series data captured from a sensor. In an exemplary embodiment, the time series data 302 may be provided from an environmental sensor connected to a turbine system (such as a gas turbine system or a wind turbine system). The time series data may be a series of data indexed in chronological order, such as temperature data, pressure data, velocity data (e.g., rotational velocity data, translational velocity data, and / or flow velocity data), or other data.

[0058] The time series data 302 may be stored in a series of time series batches 306 (e.g. Figure 3 306 are provided to the model 300. Each of the time series batches 306 can be a portion of the total time series data 302. In some implementations, the time series batches 306 can each be an aggregation or compilation of a portion of the original time series data 302. A series of time series batches 306 can be provided to the model 300 in sequence, with each new time series batch following where the last time series batch left off. For example, a first time series batch can be provided to the model 300, followed by a second time series batch, followed by a third time series batch, and so on. For example, for a time series with more than ten minutes of data, the first time series batch can include time series data starting from the first minute, the second time series batch can include data starting from the second minute, and the third time series batch can include time series data starting from the third minute.

[0059] In some implementations, a series of time series batches 306 can be provided to the API model 308. The API model 308 can generate a series of time series sketches 316 (eg, U1, U2, ..., U2) based on the series of time series batches 306 of the time series data 302. N ). A series of time series batches 306 may be positionally encoded, reduced, and weighted by the API model 308 to generate a series of time series sketches 316. Each time series sketch 316 (e.g., U1, U2, ... U N ) may be a compact summary of the corresponding time series batch 306 (e.g., a compact summary of data aggregation, such as a compact summary of the time series batch 306, which may be an aggregation of a portion of the original time series data 302). In such a specific implementation, necessary information from each time series batch 306 may be captured, while other information may be extracted. In this way, using a series of time series sketches 316 (e.g., U1, U2, ... U N ) Calculating the metric can advantageously be less expensive than calculating the exact value. In an exemplary embodiment, each time series sketch can be a matrix sketch. However, in other embodiments, each time series sketch can be a set of vectors or a graphical representation of the time series data. For example, in an embodiment where the time series data describes the internal temperature of a gas turbine engine (e.g., the temperature in one of the compressor section, the combustion section, or the turbine section), the time series sketch can be a compact representation of the temperature data (e.g., a compact matrix or a compact graphical representation).

[0060] For example, in various implementations, the API model 308 may include a positional encoding model 310 of the API model 308. The positional encoding model 310 may be provided with a series of time series batches 306, and the positional encoding model 310 may generate a series of encoded time batches 312 as output. Subsequently, the series of encoded time batches 312 may be provided to a projection model 314, which may project the series of encoded time batches 312 into a sketch space. In particular, the projection model 314 may implement a matrix sketch to project the series of encoded time batches 312 into a sketch space. The projection model 314 may generate a series of time series sketches from the series of encoded time batches 312.

[0061] In many embodiments, the API model 312 may include an incremental update model 318. The new time series batch 306 may be assigned a weight (i.e., the new time series batch 306 may be modified or adjusted by the weight). The weight assigned to the new time series batch 306 may be based at least in part on the previous time series sketch in the series of time series sketches 316. The weight assigned to the new time series batch 306 may be a weighted sum of two elements: the first element approximates the observed data covariance, and the second element represents the covariance of the new batch.

[0062] In many embodiments, the model 300 may generate an output 304 (such as a classification label, prediction, or other output) for each time series sketch 316 in the series of time series sketches 316. For example, the API model may provide the series of time series sketches 316 to a classification block 320 to predict a class probability distribution. As shown, the classification block 320 may include a fully connected layer and a softmax function. In various implementations, the model 300 may generate a label 322 as output 304 for each time series sketch 316 (in real time). In other implementations, the model 300 may generate a prediction as output 304 for each time series sketch 316 (in real time).

[0063] As a non-limiting example, model 300 may be provided with sensor data (e.g., time series data) indicating one or more parameters associated with a power generation system (such as a wind turbine or a gas turbine engine) indexed in a temporal order. The sensor data may be pressure data (e.g., from a pressure sensor), temperature data (e.g., from a temperature sensor), vibration data (e.g., from a vibration sensor or accelerometer), and / or velocity data. Output 304 of model 300 may be a classification label indicating a type of failure mode of the power generation system. For example, in an embodiment where the power generation system is a gas turbine engine, the failure mode classification labels that may be output from model 300 may include: a bearing vibration failure event (e.g., bearing vibration exceeds a threshold for a predetermined period of time); a temperature failure event (e.g., a temperature distribution in the exhaust section of the gas turbine engine exceeds a threshold for a predetermined period of time, thereby indicating a leak in the combustion section); and an overspeed failure event (e.g., a rotor of the gas turbine engine rotates at a speed that exceeds a failure threshold for a predetermined period of time). In an embodiment where the power generation system is a wind turbine system, the failure mode classification labels that may be output from model 300 may include a blade breakage event or other failure events.

[0064] Time series data can be analyzed in real time by model 300. For example, sensor data can be provided to model 300 in real time, so that model 300 can generate classification label output 304 in real time. This allows model 300 to identify fault events as they occur (or shortly after they occur). Rapid identification of fault events can advantageously allow for rapid resolution of the fault events, thereby minimizing downtime of the power generation system.

[0065] In some embodiments, the model 300 may include a temporal convolutional network or model (TCN) 324. A series of time series batches 306 may be provided to the TCN 324 to generate a series of TCN-generated embeddings (e.g., as output of the TCN 324). In various implementations, the TCN 324 may employ a sliding window to extract nonlinear features from the series of time series batches 306 before providing them to the API model 308. For example, the model 300 may provide the series of TCN-generated embeddings as input to the API model 308. The TCN-generated embeddings may be learned representations of the input data (e.g., the time series data 302) in which certain nonlinear features have been extracted from the time series data by the TCN 324.

[0066] Figure 4 A flowchart illustrating a computer-implemented method 400 for analyzing time series data is shown. The method 400 may be implemented by, for example, the method described above with reference to Figure 1 The method 400 may be performed by the computing system 100 shown and described. At (402), the method 400 may include providing a series of time series batches of time series data to an attention power iteration (API) model. At (404), the method 400 may include generating, by the API model, a series of time series sketches based on the series of time series batches of time series data. The method 400 may also include: at (406), assigning weights to new time series batches in the series of time series batches based at least in part on previous time series sketches in the series of time series sketches. At (408), the method 400 may also include generating an output for each time series sketch in the series of time series sketches.

[0067] Figure 5 exemplifies a representable API model 308 ( Figure 3 ) process 500. In particular, the API model 308 may follow Figure 5 The process shown is used to generate this series of time series sketches.

[0068] In line 1, a process is initiated to process each batch (e.g., X1, ...X N ) for loop.

[0069] In line 2, the positional encoding follows the time series order, unlike permutation-invariant samples in regular data streams. The positional information within the time series is critical. Process 500 achieves this by using Continuously Enhanced Positional Embedding (CAPE), which is advantageous because CAPE is computationally efficient and robust in handling input lengths.

[0070] In line 3, the encoded batch is passed through the fully connected layer f vIt undergoes a transformation and is subsequently activated by a rectified linear function (ReLU). This introduces nonlinearity, which improves the expressiveness of the learned sketch.

[0071] In lines 4 to 8, for the first batch (e.g., X1), the initial sketch vector U1 is updated until the initial sketch vectors converge. In lines 9 to 18, for any subsequent batch (i.e., not the first batch), the sketch vector U is updated. i , until convergence. In line 10, the sketch U is initialized with the previous sketch U i .

[0072] Lines 11 to 15 incorporate an attention model that assigns weights to the new batch relative to the most recent sketch. The update on line 15 is a weighted sum of two elements: the first approximates the observed data covariance, while the second represents the covariance of the new batch. These weights are derived from the attention model in lines 11 to 13.

[0073] It is important to note that the normalization steps in lines 7 and 16 of Algorithm 2 provide two benefits: 1) preventing the sketch magnitude from becoming too large and enhancing convergence stability; and 2) ensuring classification robustness to time series length. Without normalization, the sketch magnitude would be correlated with the time series length, thus destroying the similarity between shorter and longer time series of the same class.

[0074] Now refer to Figure 6 , provides a flow chart of a computer-implemented method 600 for analyzing time series data. The method 600 may be implemented by, for example, the method described above with reference to Figure 1 The method 600 may include obtaining or receiving time series data as input at (602). The time series data may describe a series of time-based data points. The method 600 may also include processing the time series data with a temporal convolutional model to generate one or more time series embeddings at (604). The temporal convolutional model may be the one described above. Figure 3TCN 324 described. Method 600 may also include processing one or more time series embeddings with a representation generation model to generate an output representation. The output representation may describe a graphical representation of the series of time-based data points. The representation generation model may convert the time series embedding into a compact information representation of the embedding (e.g., via dimensionality reduction). In many embodiments, the output representation may be a matrix sketch. The matrix sketch may reduce the size of the time series embedding while maintaining key characteristics. Method 600 may also include processing the output representation with a classification model at (608) to generate classification labels for the time series data. In many embodiments, the time series data is generated using one or more sensors associated with a power generation system (such as a gas turbine engine). Additionally, in various embodiments, the classification labels include anomaly detection classification. Anomaly detection classification may identify trends or data points that significantly deviate from the norm for the time series data. For example, using time series data associated with one or more parameters of a gas turbine engine, anomaly detection classification may identify various trip (or fault) events of the gas turbine engine, such as overspeed, high temperature in the exhaust section, etc.

[0075] The above reference Figures 1 to 6 The described computing system 100, models 200, 300, methods 400, processes 500, and methods 600 provide many advantages over known models. For example, known artificial intelligence (AI) models, particularly popular neural networks, often require large amounts of memory and computing energy, making them unsuitable for deployment on resource-constrained devices. The above-mentioned models 200, 300 are designed to operate with limited computing memory and energy resources while ensuring high processing speeds. Configurations including the models described herein can perform complex data processing tasks on devices with limited computing resources, and can perform tasks with reduced latency when compared to traditional techniques. Experimental results emphasize that models 200, 300 have exceptional performance, exhibiting not only excellent accuracy, but also significantly reduced energy consumption, increased operating speed, and reduced computational complexity.

[0076] The above-described models 200, 300 advantageously utilize the API models 204, 308, thereby enabling real-time processing on resource-constrained devices. The models 200, 300 continuously update a compact representation of the entire time series, thereby enhancing classification (e.g., output) accuracy while saving energy and processing time. It is noted that the models 200, 300 perform well in streaming scenarios where access to the complete time series is not required, thereby enabling rapid decisions. The models 200, 300 excel in classification accuracy and energy efficiency, consuming more than 70% less than the baseline and completing tasks three times faster than the baseline. This work improves real-time responsiveness, energy conservation, and operational efficiency for constrained devices, thereby helping to optimize various applications.

[0077] This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims and may include other examples that occur to those skilled in the art. If such other examples include structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal language of the claims, such other examples are intended to be within the scope of the claims.

[0078] Further aspects of the invention are provided by the subject matter of the following clauses:

[0079] A computer-implemented method for analyzing time series data, the method comprising: obtaining, by a computing system comprising one or more processors, a series of time series batches of the time series data; generating, by the computing system and utilizing an attention power iteration (API) model, a series of time series sketches based on the series of time series batches of the time series data; assigning, by the computing system, weights to new time series batches in the series of time series batches based at least in part on previous time series sketches of the series of time series sketches; and generating, by the computing system, an output for each time series sketch in the series of time series sketches.

[0080] A computer-implemented method according to any preceding clause, further comprising providing the series of time series batches to a temporal convolutional network (TCN) to generate a series of TCN-generated embeddings, wherein the TCN extracts non-linear features from the series of time series batches by performing a sliding window process on the time series data.

[0081] A computer-implemented method according to any preceding clause, further comprising providing the series of TCN-generated embeddings as input to the API model.

[0082] A computer-implemented method according to any preceding clause, further comprising providing the series of time-series batches to a positional encoding model of the API model, the positional encoding model generating a series of encoded time batches from the series of time-series batches.

[0083] The computer-implemented method of any preceding clause, further comprising providing the series-encoded time series batches to a projection model, the projection model generating the series time series sketch from the series-encoded time series batches.

[0084] A computer-implemented method according to any preceding clause, further comprising providing the series of time series sketches to a classification block to predict a class probability distribution.

[0085] A computer-implemented method as described in any preceding clause, wherein generating an output further comprises: generating a classification label for each time series sketch in the series of time series sketches.

[0086] A computer-implemented method as described in any preceding clause, wherein generating an output further comprises generating a forecast for each time series sketch in the series of time series sketches.

[0087] A computer-implemented method according to any preceding clause, wherein the API model complies with Figure 5 The process shown is to generate the series of time series sketches.

[0088] A computing system for analyzing time series data, the system comprising: one or more processors; and one or more non-transitory computer-readable media that collectively store instructions, wherein the instructions, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising: obtaining a series of time series batches of the time series data; generating a series of time series sketches based on the series of time series batches of the time series data through an attention power iteration (API) model; assigning weights to new time series batches in the series of time series batches based at least in part on previous time series sketches of the series of time series sketches; and generating output for each time series sketch in the series of time series sketches.

[0089] The computing system of any preceding clause, further comprising providing the series of time series batches to a temporal convolutional network (TCN) to generate a series of TCN-generated embeddings, wherein the TCN employs a sliding window to extract nonlinear features from the series of time series batches.

[0090] A computing system according to any preceding clause, further comprising providing the series of TCN-generated embeddings as input to the API model.

[0091] A computing system according to any preceding clause, further comprising providing the series of time-series batches to a positional encoding model of the API, the positional encoding model generating a series of encoded time batches from the series of time-series batches.

[0092] The computing system of any preceding clause, further comprising providing the series of encoded time series batches to a projection model, the projection model generating the series of time series sketches from the series of encoded time series batches.

[0093] A computing system according to any preceding clause, further comprising providing the series of time series sketches to a classification block to predict a class probability distribution, the classification block comprising a fully connected layer and a softmax function.

[0094] A computing system as described in any preceding clause, wherein generating an output further comprises: generating a classification label for each time series sketch in the series of time series sketches.

[0095] A computing system as described in any preceding clause, wherein generating an output further comprises generating a forecast for each time series sketch in the series of time series sketches.

[0096] A computing system as described in any preceding clause, wherein the API model complies with Figure 5 The process shown is to generate the series of time series sketches.

[0097] A method according to the embodiments as shown and described herein.

[0098] A computing system according to the embodiments as shown and described herein.

[0099] A computing system for time-based data classification, the system comprising: one or more processors; and one or more non-transitory computer-readable media that collectively store instructions, wherein the instructions, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising: obtaining time series data, wherein the time series data describes a series of time-based data points; processing the time series data using a temporal convolutional model to generate one or more time series embeddings; processing the one or more time series embeddings using a representation generation model to generate an output representation, wherein the output representation describes a graphical representation of the series of time-based data points; and processing the output representation using a classification model to generate a classification label for the time series data.

[0100] A system according to any preceding clause, wherein the output representation comprises a matrix sketch.

[0101] A system according to any preceding clause, wherein the representation generation model includes one or more multi-head self-attention models and one or more power iteration models, wherein the one or more power iteration models are configured to process input data and generate one or more feature vectors.

[0102] A system as described in any preceding clause, wherein the time series data is generated using one or more sensors associated with a turbine, and wherein the classification label comprises an anomaly detection classification.

Claims

1. A computer-implemented method for analyzing time series data, the method comprising: Obtaining, by a computing system comprising one or more processors, a series of time series batches of the time series data; generating, by the computing system and using an attention power iteration (API) model, a series of time series sketches based on the series of time series batches of the time series data; assigning, by the computing system, a weight to a new time series batch in the series of time series batches based at least in part on a previous time series sketch in the series of time series sketches; as well as An output is generated by the computing system for each time series sketch in the series of time series sketches.

2. The computer-implemented method of claim 1 , further comprising providing the series of time series batches to a temporal convolutional network (TCN) to generate a series of TCN-generated embeddings, wherein the TCN extracts nonlinear features from the series of time series batches by performing a sliding window process on the time series data.

3. The computer-implemented method of claim 2, further comprising providing the embeddings generated by the series of TCNs as input to the API model.

4. The computer-implemented method of claim 1 , further comprising providing the series of time-series batches to a positional encoding model of the API model, the positional encoding model generating a series of encoded time batches from the series of time-series batches.

5. The computer-implemented method of claim 4, further comprising providing the series-encoded time series batches to a projection model, the projection model generating the series time series sketch from the series-encoded time series batches.

6. The computer-implemented method of claim 1, further comprising providing the series of time series sketches to a classification block to predict a class probability distribution.

7. The computer-implemented method of claim 1 , wherein generating an output further comprises: Generate a classification label for each time series sketch in the series of time series sketches.

8. The computer-implemented method of claim 1 , wherein generating an output further comprises: Generates forecasts for each time series sketch in the series of time series sketches.

9. A computing system for analyzing time series data, the system comprising: one or more processors; and One or more non-transitory computer-readable media collectively storing instructions that, when executed by the one or more processors, cause the computing system to perform operations comprising: Obtaining a series of time series batches of the time series data; Generating a series of time series sketches in batches based on the series of time series of the time series data through an attention power iteration (API) model; assigning weights to new time series batches in the series of time series batches based at least in part on previous time series sketches in the series of time series sketches; and Generates output for each time series sketch in the series of time series sketches.

10. The computing system of claim 9, further comprising providing the series of time series batches to a temporal convolutional network (TCN) to generate a series of TCN-generated embeddings, wherein the TCN employs a sliding window to extract nonlinear features from the series of time series batches.

11. The computing system of claim 9, further comprising providing the embeddings generated by the series of TCNs as input to the API model.

12. The computing system of claim 9, further comprising providing the series of time series batches to a positional encoding model of the API, the positional encoding model generating a series of encoded time batches from the series of time series batches.

13. The computing system of claim 12, further comprising providing the series-encoded time series batches to a projection model, the projection model generating the series time series sketch from the series-encoded time series batches.

14. The computing system of claim 9, further comprising providing the series of time series sketches to a classification block to predict a category probability distribution, the classification block comprising a fully connected layer and a softmax function.

15. The computing system of claim 9, wherein generating an output further comprises: Generate a classification label for each time series sketch in the series of time series sketches.

Images