Extension of functional neural network for dimensionality reduction and multi-class classification of time series data background

Abstract

The system and method described in the description of the present application extend a functional neural network (FNN) for time-series dimensionality reduction and multi-class classification. The present invention uses a functional encoder and decoder, and a bi-functional auto-encoder (BFAE) uses base expansion to reduce the number of both features and points in time (two directions). The FNN also facilitates time-series multi-class classification, thereby enabling three or more classes of data to be detected. The functional encoder derives a low-dimensional latent representation of the data by using the continuous neurons of a continuous hidden layer. The representation is then processed by the functional decoder in order to reconstruct the original information. In the case of multi-class classification, the system utilizes cross entropy loss and a softmax activation function to effectively handle three or more classes and improve classification performance.

Description

Extending Functional Neural Networks for Multi-Class Classification and Dimensionality Reduction on Time Series Data Backgrounds

[0001] The present disclosure is generally directed to bi-functional autoencoders (BFAEs), and more specifically to BFAEs that perform two-way dimensionality reduction using functional neural networks (FNNs), and to using FNNs to perform multi-class classification.

[0002] Time series data, which are data collected over time, are ubiquitous in today's world. Time series analysis provides many organizations with valuable insights into trends, patterns, and anomalies in a wide range of applications, including finance, healthcare, manufacturing, and environmental monitoring. Time series classification, the process of categorizing time series data into different classes or categories, is essential for many industrial and real-world tasks. For example, in finance, it helps classify financial transactions as fraudulent or legitimate; in healthcare, it assists in diagnosing diseases based on patient monitoring data; and in manufacturing, it predicts equipment failures from sensor data to optimize maintenance schedules and reduce downtime. However, the amount of time series data grows exponentially, making it difficult to store, transmit, and analyze. Although numerous research papers have studied dimensionality reduction, there are no products or services yet available that simultaneously reduce the number of features and time points in time series data, which would be particularly useful for high-frequency data such as vibration information from sensors, audio signals, weather data, and so on.

[0003] Existing approaches to dimensionality reduction have been developed for multivariate problems, but they provide inadequate linear representations and are unable to reduce the number of time points. For example, techniques such as PCA (Principal Component Analysis) and AE (Autoencoder) are unable to efficiently capture and model the temporal information of time series data. Similarly, neural networks for classification methods such as long short-term memory (LSTM), random forests, and rockets are limited by their structure or completely ignore temporal information.

[0004] Functional data analysis (FDA) has proven to be a robust statistical approach for analyzing patterned time series data. Function-on-function models can be used to construct mathematical mappings of time series data for dimensionality reduction. Compared to deep learning (DL) approaches, functional data modeling techniques increase the efficiency of capturing the rich information of time series data with fewer parameters using function-on-function and function-on-scalar models. Functional data modeling places fewer constraints on the data format (i.e., data can have different resolutions across samples) and the underlying mapping (i.e., parameters can be different at different times within the considered time horizon). However, methods such as functional linear models (FLM), functional principal component analysis (FPCA), and functional autoencoders (FAE) have limitations. For classification, FLM takes linear relationships and is constrained by binary classification. On the other hand, for dimensionality reduction, FPCA only provides a linear representation and cannot capture complex relationships, and FAE only provides a limited scalar representation of the data. Dimensionality reduction based on these methods tends to produce inaccurate results when dealing with complex real-world data.

[0005] Embodiments herein extend functional neural networks (FNNs) for time-series dimensionality reduction and multi-class classification. Using a functional encoder and decoder, a bifunctional autoencoder (BFAE) reduces the number of both features and time points (two-way) using basis expansion. To facilitate time-series multi-class classification, FNNs are extended with appropriate loss and output layer activation functions, allowing them to detect more than two classes of data.

[0006] A function encoder uses successive neurons in successive hidden layers to derive a low-dimensional latent representation of the data, which is then processed by a function decoder to reconstruct the original information.

[0007] For multi-class classification, the system utilizes cross-entropy loss and softmax activation functions to effectively handle more than two classes and improve classification performance.

[0008] In some aspects of the present disclosure, a bifunctional autoencoder (BFAE) includes a function encoder that performs steps including: performing two-way dimensionality reduction using a function neural network (FNN), and in response to receiving time series information at an input layer of the FNN, processing the time series information through successive hidden layers including successive neurons to learn a low-dimensional representation of the time series information; and performing two-way dimensionality reduction using basis expansion to reduce both the number of features and the number of time points in the time series information to reduce at least one of computation time, storage requirements, or data transfer time; obtaining reconstructed time series information using the learned low-dimensional representation; a function decoder that uses the learned low-dimensional representation of the time series information; and performing an analysis task using the low-dimensional representation of the time series information.

[0009] In some aspects, temporal relationships in a successive hidden layer may be preserved, utilizing continuous neurons to identify at least one of a function or a pattern in the time series information. The continuous neurons in the hidden layer may be defined using a parameter function and a bivariate parameter function.

[0010] In some aspects, the function encoder selects basis functions from at least one of B-splines, wavelets, and Fourier functions to increase data capture efficiency. The function encoder may determine the number of features and the number of time points observed in the latent representation layer. The function encoder may reduce the dimensionality of the time series information data with minimal information loss. The function decoder may further reconstruct the original time series information from the reduced dimensional representation.

[0011] In some aspects of the present disclosure, a multi-class classification system using an FNN includes: a model training module that, in a training phase, constructs an FNN model that learns a mapping of the time series data to classes associated with the time series data; and a model deployment module that, in an inference phase, applies the trained FNN model to the time series data using the mapping to detect three or more classes in the time series data, where the FNN model is trained by using at least one of a direct method (FDNN) or a basis deployment, each including successive neurons forming successive hidden layers, and outputs the three or more classes.

[0012] In some aspects, the loss is a cross-entropy loss and the activation function is a softmax activation function.

[0013] In some aspects, the techniques described herein relate to systems in which a function encoder captures non-linear relationships in time-series data.

[0014] In some aspects, the techniques described herein relate to systems in which a successive hidden layer utilizes successive neurons to identify at least one of a function or a pattern in time series data.

[0015] In some aspects, the techniques described herein relate to a non-transitory computer-readable medium storing instructions for executing a process, the instructions including: performing, at a function encoder, steps including: in response to receiving time series information at an input layer of a function neural network (FNN), processing the time series information through successive hidden layers including successive neurons to learn a low-dimensional representation of the time series information; and performing a two-way dimensionality reduction using basis expansion to reduce both the number of features and the number of time points in the time series information to reduce at least one of computation time, storage requirements, or data transfer time; and at a function decoder, the function decoder uses the learned low-dimensional representation of the time series information; and performing an analysis task using the low-dimensional representation of the time series information.

[0016] In some aspects, techniques described herein relate to a non-transitory computer-readable medium storing instructions for performing a process, the instructions including: constructing a functional neural network (FNN) model that learns a mapping of time series data to classes associated with the time series data in a training phase; and applying the trained FNN model to the time series data in an inference phase using the mapping to detect three or more classes in the time series data, wherein the FNN model is trained by using at least one of a first-order method (FDNN) or basis expansion, each including successive neurons forming successive hidden layers; and outputting the three or more classes.

[0017] Aspects of the present disclosure may involve a system that may include: means for receiving time series information at an input layer of an FNN and processing the time series information through successive hidden layers including successive neurons to learn a low-dimensional representation of the time series information; and means for performing a two-way dimensionality reduction that reduces both the number of features and the number of time points in the time series information using a basis expansion to reduce at least one of computation time, storage requirements, or data transfer time. Aspects of the present disclosure may involve a system that may include: a function decoder that obtains reconstructed time series information using the learned low-dimensional representation, the function decoder using the learned low-dimensional representation of the time series information; and means for performing an analysis task using the low-dimensional representation of the time series information.

[0018] Aspects of the present disclosure may involve a system that may include: means for constructing an FNN model that learns a mapping of time series data to classes associated with the time series data in a training phase; and means for applying the trained FNN model to the time series data using the mapping to detect three or more classes in the time series data in an inference phase, wherein the FNN model is trained by using at least one of an FDNN or a basis expansion, each including successive neurons forming successive hidden layers; and means for outputting the three or more classes.

[0019]

[0014] Figure 1 illustrates an exemplary data-driven system for dimensionality reduction of time series data according to various embodiments of the present disclosure.

[0015] Figure 2 illustrates a general BFAE architecture for dimensionality reduction according to various embodiments of the present disclosure.

[0016] Figure 3 illustrates a flow diagram illustrating application of a trained model for dimensionality reduction of time series data according to various embodiments of the present disclosure.

[0017] Figure 4 illustrates an exemplary data-driven system for multi-class classification of time series data according to various embodiments of the present disclosure.

[0018] Figure 5 illustrates a general FNN architecture for multi-class classification according to various embodiments of the present disclosure.

[0019] Figure 6 illustrates a flowchart illustrating an exemplary two-way dimensionality reduction process according to various embodiments of the present disclosure.

[0020] Figure 7 illustrates an exemplary process for multi-class classification according to various embodiments of the present disclosure.

[0021] Figure 8 illustrates an example computing environment including an example computer device.

[0020] The following detailed description provides details of the drawings and exemplary implementations of the present application. Reference numbers and descriptions of elements that are duplicated between drawings are omitted for clarity. Terms used throughout the description are provided by way of example and are not intended to be limiting. For example, use of the term "automatic" can include a fully automatic implementation or a semi-automatic implementation with user or administrator control over certain aspects of the implementation, depending on the desired implementation of those skilled in the art practicing the implementations of the present application. Selection can be performed by a user through a user interface or other input means, or can be achieved through a desired algorithm. The exemplary implementations as described herein can be utilized alone or in combination, and the functionality of the exemplary implementations can be achieved through any means depending on the desired implementation.

[0021] Rapid developments in technology require ever-increasing data storage capabilities. The exponential growth of information, especially in the form of time series data, necessitates systems and methods for efficiently storing large amounts of information with exponential and minimal signal loss. Often, this information is not independent but exhibits complex relationships, making the problem more challenging. Dimensionality reduction plays an important role in addressing this challenge by meaningfully reducing information.

[0022] Mathematically, the main goal of dimensionality reduction is to reduce the dimensionality by projecting data into a lower dimensional subspace with minimal information loss. This helps to identify important variables and their interactions, address the curse of dimensionality, ease information transfer, and reduce both storage requirements and computation time.

[0023] Dimensionality reduction has become an active area of ​​research due to its usefulness. In recent years, the development of dimensionality reduction models has proven beneficial in many fields, including time series, text, image, and video analysis. Conversely, time series classification plays a vital role in the modern world, enabling predictive analysis, anomaly detection, and informed decision-making across various industries. It enables businesses to forecast future trends, anticipate equipment failures, and detect anomalies in data, leading to improved efficiency and cost savings. In healthcare, time series classification aids in diagnosing diseases and monitoring patient health. In environmental monitoring, it helps predict weather patterns and natural disasters. In finance, time series classification aids in fraud detection and stock market forecasting, providing valuable insights for risk management and decision-making processes.

[0024] Additionally, time series classification is useful in smart manufacturing, optimizing production processes and enabling predictive maintenance strategies. It also plays a vital role in energy management, forecasting energy consumption and promoting sustainable practices. Overall, time series classification is a versatile and essential tool that continues to drive developments in various industries, improving efficiency and enabling informed decision-making in the face of complex and dynamic data.

[0025] Multi-class classification is essential in scenarios where data needs to be categorized into more than two classes to provide a nuanced understanding of complex real-world problems. It provides a richer view into the relationships between different classes and their properties, aiding in more detailed data analysis. In real-world applications such as email classification or image recognition, multi-class classification allows data to be categorized into multiple categories, such as spam, advertising, and the main inbox of emails, or various objects or animals in an image. This detailed classification is important for making informed decisions and gaining deeper insights into the data.

[0026] Compared to binary classification, multi-class classification presents a greater challenge due to the increasing number of classes and the complexity of distinguishing between them, which is amplified for time series data. However, this complexity has become an important research area in machine learning, driving the development of classification algorithms. Multi-class classification provides a comprehensive assessment of classifier performance, considering not only the overall accuracy but also the effectiveness of the classifier for each individual class. This comprehensive assessment is essential for understanding the strengths and limitations of a classifier, leading to more robust and reliable classification systems.

[0027] Embodiments provide a data-driven system and method that effectively learns a mathematical mapping of time series to a low-dimensional latent space without loss of information. This is advantageous for use cases such as audio signals or vibration information measured with ultra-high frequency waves, and temperature readings recorded with very granular resolution. In these scenarios, accurate and compact information is crucial for performing various downstream tasks.

[0028] Some data-driven embodiments effectively learn a mathematical mapping of time series to their labels or classes (multi-class) for use cases such as anomaly detection, weather classification, etc. In these applications, learning temporal information leads to improved performance.

[0029] In brief, various embodiments extend the Functional Neural Network (FNN) approach so that it can be used (with minor modifications) for both classification and dimensionality reduction tasks by effectively exploiting the temporal nature of the data.

[0030] Unlike existing approaches, the multi-class classification for time series data embodiments herein offer several advantages. They automatically capture nonlinear relationships present in the data and can efficiently classify more than two classes by using either first-order methods or basis expansion. The superiority of these systems and methods has been demonstrated through real-world data analysis. This forecasting approach is beneficial in a variety of scenarios, including industries where time series information containing multiple classes is available and where multi-class classification is common in real-world problems.

[0031] Additionally, the approach of the two-way dimensionality reduction embodiments herein offers significant benefits compared to existing methods. These embodiments can automatically capture nonlinear relationships in the data and reduce both the number of features and the time points at which time series are observed. By using basis functions, the system can more efficiently capture data using appropriate functions. The advantages of these systems and methods have also been confirmed through real-world data analysis. These embodiments are particularly beneficial in industries where information is stored at high frequency, addressing the global challenge of data storage and transfer costs. Using accurately dimensionality-reduced data allows for faster and more accurate analysis.

[0032] 1 illustrates an exemplary data-driven system for dimensionality reduction of time-series data in accordance with various embodiments of the present disclosure. In an embodiment, the system 100 includes a data check and data preprocessing module 104 and a BFAE module 108. The BFAE module 108 may include a function encoder 110, a dimensionality reduction form 112, and a function decoder 114.

[0033] In operation, the data check and data preprocessing module 104 aims to ensure that the time series data 102 (e.g., weather data) is observed regularly over time (e.g., hourly) without significant time gaps between adjacent observations.

[0034] The data checking and data preprocessing module 104 may perform steps before the data is used as input to machine learning (ML) or deep learning (DL) algorithms. Such data preparation steps may be performed on the input data before it is input to these algorithms. It is noted that a specific data preparation method is not required. Examples of data checking and data preprocessing steps include noise / outlier identification and removal, missing data imputation, etc. Once the data has been preprocessed in this manner, it may be split into training and test sets. The training set is used during the model training phase, and the test set is used to evaluate the model.

[0035] The BFAE module 108 performs a training phase for dimensionality reduction. In an embodiment, this is accomplished using a function encoder 110, which obtains a compact latent representation 112 of the time series information. A function decoder 114 recovers the original time series information from the trained latent representation. A BFAE application module (not shown) may apply the trained dimensionality reduction model to new data, for example, in the inference phase.

[0036] Using the following mathematical notation, for each data sample, time series data is observed within a time range T, assuming the number of samples is N. The observed data is a set of data X, where s∈S (compact internal) for j=1,...,M, i=1,...,N, r=1,...,R. （ｉ，ｒ） (s ｉ，ｊ ) The time series data may be fed into the neural network architecture shown in FIG.

[0037] 2 illustrates a general BFAE architecture for dimensionality reduction in accordance with various embodiments of the present disclosure. As shown, the BFAE architecture 200 includes an encoder 202 and a decoder 204 operating on a neural network 206. The neural network 206 includes an input layer 208, successive hidden layers 210-216, a latent representation 220, and a successive output layer 230. The BFAE architecture 200 identifies underlying patterns in the data and creates compact latent representations. In embodiments, the BFAE architecture 200 accomplishes this by leveraging the neural network 206 to discover complex relationships within the patterns. The BFAE includes successive neurons (e.g., 218) that make up the successive hidden layers 210-216. The lth successive hidden layer and the Rth successive neuron may be defined as follows:

[0038] where σ is the activation function, is a parameter function, is a bivariate parameter function. The continuous neuron is extended using basis functions as follows:

[0039] Also, the backpropagation of continuous neurons may be extended using basis functions as follows:

[0040] Also,

[0041] During the ceremony, is the unknown matrix, are known basis functions (eg, B-splines, wavelets, or Fourier).

[0042] The defined successive neurons and derivatives can be used to complete forward and backpropagation. This process may be iterated between forward and backpropagation until a stopping criterion is reached. The BFAE performs dimensionality reduction through a function encoder, thereby passing information from the input layer 208 through successive hidden layers 210-216 until it reaches layer 220, which The function decoder then constructs the original time series information 230 from the learned latent representations. The number of features and the number of time points at which features are observed in the latent representation layer 220 may be chosen freely.

[0043] 3 is a flow diagram illustrating application of a trained model for dimensionality reduction of time-series data, according to various embodiments of the present disclosure. In embodiments, the trained model for time-series data may be applied by a BFAE model application module (not shown) to perform steps including: (1) using the dimensionality-reduced form 302 output by the trained model by a function decoder 306 to retrieve or reconstruct the original time-series data 308; and (2) using the dimensionality-reduced form 302 output by the trained model by a function decoder 306 to obtain a compact form of the data that may be used for various analytical tasks 310.

[0044] FIG. 4 illustrates an exemplary data-driven system for multi-class classification of time-series data in accordance with various embodiments of the present disclosure. In an embodiment, the system 400 includes a data checking and data preprocessing module 104 and a multi-class FNN 402. In operation, the data checking and data preprocessing module 104 aims to ensure that time-series data to be used in subsequent calculations are observed regularly over time, without significant time gaps between adjacent observations. The data checking and data preprocessing module 104 may perform steps before the data is used as input to an ML or DL ​​algorithm. Such data preparation steps may be performed on input data before it is input to these algorithms. It is noted that no specific data preparation method is required. Examples of data checking and data preprocessing steps include noise / outlier identification and removal, missing data imputation, etc. Once the data has been preprocessed in this manner, it may be divided into training and test sets. The training set is used during the model training phase, and the test set is used to evaluate the model.

[0045] In an embodiment, a multi-class model module (not shown) may perform a training phase to develop a model from the FNN for multi-class classification using the processed time series data 106. Based on the results obtained from the multi-class model module, a multi-class model application module (not shown) may apply the trained model. For the multi-class model module, the following mathematical notation is used: For each data sample, time series data is observed within a time range T, assuming the number of samples is N. The observed data is summed over t for j=1,...,M,i=1,...,N. ｉ，ｊ ∈T, X ｉ，ｊ (t ｉ， j). Since the modeling task is multi-class classification, there are more than two labels or classes. The time series data may be fed into the neural network architecture shown in Figure 5 to obtain the output classes.

[0046] 5 illustrates a general FNN architecture for multi-class classification in accordance with various embodiments of the present disclosure. The FNN architecture 500 comprises an input layer 502, successive hidden layers 504-506, and a successive output layer 510. The FNN architecture 500 identifies underlying patterns in data and optimizes the model to discover complex relationships within the patterns.

[0047] The FNN 500 comprises successive neurons (eg, 508) that make up successive hidden layers 504-506. The lth successive hidden layer and the rth successive neuron may be defined as follows: where σ is the activation function,

[0048] is a parameter function, is a bivariate parameter function. Using the defined successive neurons, forward and back propagation can be completed, and partial derivatives may be calculated to update the parameter function in the back propagation step. This process iterates between forward and back propagation until a stopping criterion is reached. The FNN 500 has the flexibility to consider other functional features that allow it to improve the current results. The number of successive hidden layers, the number of successive neurons in each successive hidden layer, and the activation function may be chosen freely. This approach to optimizing the network 500 is called a Functional Direct Neural Network (FDNN). The loss used is the cross-entropy loss, and the output activation function is softmax.

[0049] Another approach to solving a network is through a basis function neural network (FBNN). In such an embodiment, continuous neurons are augmented with basis functions as follows:

[0050] Also, the backpropagation of a continuous neuron is as follows:

[0051] but,

[0052] During the ceremony, is the unknown matrix, are known basis functions (eg, B-splines, wavelets, or Fourier).

[0053] Although not explicitly illustrated, it is understood that an embodiment for multi-class classification of time series models utilizes a data collection and storage module that collects historical data, a model training unit that uses FNN multi-class classification to build a model utilizing the time series data, and a model deployment unit that deploys the trained model to streaming data to create and communicate real-time data-driven information.

[0054] 6 is a flowchart illustrating an exemplary two-way dimensionality reduction process according to various embodiments of the present disclosure. Process 600 can begin when the function encoder at the input layer of the FNN receives time series information.

[0055] In step 604, the function encoder may process the time series information through successive hidden layers to learn a low-dimensional representation of the time series information. The hidden layers may include successive neurons.

[0056] In step 606, the function encoder may perform a two-way dimensionality reduction using basis expansion to reduce both the number of features and the number of time points in the time series information in order to reduce at least one of computation time, storage requirements, or data transfer time.

[0057] In step 608, the function decoder may use the learned low-dimensional representation to obtain reconstructed time series information. In an embodiment, the function decoder may use the learned low-dimensional representation of the time series information.

[0058] Finally, in step 610, analytical tasks may be performed using the reduced dimensional representation of the time series information.

[0059] 7 is a flowchart illustrating an example process for performing multi-class classification according to various embodiments of the present disclosure. In a training phase, process 700 can begin when a model training module builds an FNN model that learns the mapping of time series data to classes associated with the time series data.

[0060] In step 704, in the inference phase, the model expansion module may apply the trained FNN model to the time series data using the mapping to detect three or more classes in the time series data, where the FNN model is trained by using at least one of FDNN or basis expansion, each including successive neurons forming successive hidden layers.

[0061] At step 706, three or more classes may then be output.

[0062] Those skilled in the art will recognize that (1) certain steps may be performed arbitrarily, (2) steps may not be limited to the particular order described herein, (3) certain steps may be performed in different orders, and (4) certain steps may occur simultaneously.

[0063] 8 illustrates an example computing environment having example computing devices suitable for use in some illustrative implementations. The computing device 805 of the computing environment 800 can include one or more processing units, cores, or processors 810, memory 815 (e.g., RAM, ROM, and / or other), internal storage 820 (e.g., magnetic, optical, solid-state storage, and / or organic), and / or I / O interface 825, any of which can be coupled by a communication mechanism or bus 830 for communicating information or embedded in the computing device 805. The I / O interface 825 can also be configured to receive images from a camera or provide images to a projector or display, depending on the desired implementation.

[0064] Computing device 805 can be communicatively coupled to input / user interface 835 and output device / interface 840. Either or both of input / user interface 835 and output device / interface 840 can be wired or wireless interfaces and can be detachable. Input / user interface 835 may include any device, component, sensor, or interface, physical or virtual, that can provide input (e.g., buttons, touchscreen interface, keyboard, pointing / cursor control, microphone, camera, Braille, motion sensor, optical reader, and / or the like). Output device / interface 840 may include a display, television, monitor, printer, speakers, Braille, etc. In some illustrative implementations, input / user interface 835 and output device / interface 840 can be embedded in or physically coupled to computing device 805. In other illustrative implementations, other computing devices may function as or provide the functionality of input / user interface 835 and output device / interface 840 of computing device 805.

[0065] Examples of computing devices 805 may include highly mobile devices (e.g., smartphones, devices in automobiles and other machines, devices carried by people and animals, etc.), mobile devices (e.g., tablets, notebooks, laptops, personal computers, portable televisions, radios, etc.), and devices not designed for mobility (e.g., desktop computers, other computers, information kiosks, televisions with and / or combined with one or more processors, radios, etc.).

[0066] Computing device 805 may be communicatively coupled (e.g., via I / O interface 825) to external storage 845 and network 850 for communicating with any number of networked components, devices, and systems, including one or more computing devices of the same or different configurations. Computing device 805, or any connected computing device, may function as, provide services for, or be referred to as a server, client, thin server, general-purpose machine, special-purpose machine, or another label.

[0067] I / O interface 825 can include wired and / or wireless interfaces using any communication or I / O protocol or standard (e.g., Ethernet, 802.11x, Universal System Bus, WiMax, modem, cellular network protocols, etc.) for communicating information to and from at least all connected components, devices, and networks of computing environment 800. Network 850 can be any network or combination of networks (e.g., the Internet, a local area network, a wide area network, a telephone network, a cellular network, a satellite network, etc.).

[0068] The computing device 805 may use and / or communicate using computer-usable or computer-readable media, including transitory and non-transitory media. Transitory media include transmission media (e.g., metallic cables, fiber optics), signals, carrier waves, etc. Non-transitory media include magnetic media (e.g., disks and tape), optical media (e.g., CD ROM, digital video disks, Blu-ray disks), solid-state media (e.g., RAM, ROM, flash memory, solid-state storage), and other non-volatile storage or memory.

[0069] The computing device 805 can be used to implement techniques, methods, applications, processes, or computer-executable instructions in some example computing environments. The computer-executable instructions can be retrieved from transitory media and stored on and retrieved from non-transitory media. The executable instructions can be in one or more of any programming, scripting, and machine language (e.g., C, C++, C#, Java, Visual Basic, Python, Perl, JavaScript, etc.).

[0070] The processor 810 can run under any operating system (OS) (not shown) in a native or virtual environment. One or more applications can be deployed, including a logic unit 860, an application programming interface (API) unit 865, an input unit 870, an output unit 875, and an inter-unit communication mechanism 895 through which different units communicate with each other, the OS, and other applications (not shown). The described units and elements may vary in design, function, configuration, or implementation and are not limited to the provided description. The processor 810 can be in the form of a hardware processor, such as a central processing unit (CPU), or a combination of hardware and software units.

[0071] In some example implementations, once information or instructions for execution are received by API unit 865, they may be communicated to one or more other units (e.g., logic unit 860, input unit 870, output unit 875). In some examples, logic unit 860 may be configured to control the flow of information between units and, in some example implementations described above, direct the services provided by API unit 865, input unit 870, and output unit 875. For example, the flow of one or more processes or implementations may be controlled solely by logic unit 860 or in combination with API unit 865. Input unit 870 may be configured to obtain inputs for calculations described in example implementations, and output unit 875 may be configured to provide outputs based on the calculations described in example implementations.

[0072] The processor 810 may be configured to execute a method or computer instructions that may involve receiving time series information at an input layer of an FNN and processing the time series information through successive hidden layers including successive neurons to learn a low-dimensional representation of the time series information, for example, as described with respect to FIGS. 2 and 5 .

[0073] The processor 810 may be configured to execute a method or computer instructions that may involve performing a two-way dimensionality reduction using basis expansion to reduce both the number of features and the number of time points in the time series information to reduce at least one of computation time, storage requirements, or data transfer time, for example, as described with respect to FIG. 1 .

[0074] The processor 810 may be configured to execute a method or computer instructions, for example, as described with respect to FIG. 3 , to obtain reconstructed time series information using the learned low-dimensional representation, which may involve a function decoder using the learned low-dimensional representation of the time series information and performing an analysis task using the low-dimensional representation of the time series information.

[0075] The processor 810 may be configured to execute a method or computer instructions that may involve, for example, constructing an FNN model in a training phase that learns a mapping of the time series data to classes associated with the time series data, and applying the trained FNN model to the time series data in an inference phase using the mapping to detect three or more classes in the time series data, where the FNN model is trained by using at least one of an FDNN or basis expansion, each including successive neurons forming successive hidden layers, and thereafter outputting the three or more classes, as described with respect to FIGS. 4 and 5 .

[0076] Some portions of the detailed descriptions are presented in terms of algorithms and symbolic representations of operations within a computer. These algorithmic descriptions and symbolic representations are the means used by those skilled in the data processing arts to convey the essence of their innovations to others skilled in the art. An algorithm is a prescribed sequence of steps leading to a desired end state or result. In exemplary implementations, the steps performed require physical manipulations of tangible quantities to achieve a tangible result.

[0077] Unless otherwise specifically indicated, as will be apparent from the discussion, it will be recognized that throughout the description, discussion utilizing terms such as "processing," "computing," "calculating," "determining," "displaying," and the like can include operations and processes of a computer system or other information processing device that manipulate and convert data represented as physical (electronic) quantities in the computer system's registers and memory into other data that is similarly represented as physical quantities in the computer system's memory or registers or other information storage, transmission, or display device.

[0078] Example implementations may also relate to apparatuses for performing the operations herein. This apparatus may be specially constructed for the required purposes, or may include one or more general-purpose computers selectively activated or reconfigured by one or more computer programs. Such computer programs may be stored on computer-readable media, such as computer-readable storage media or computer-readable signal media. Computer-readable storage media may include tangible media, such as optical disks, magnetic disks, read-only memory, random-access memory, solid-state devices, drives, or any other type of tangible or non-transitory medium suitable for storing electronic information. Computer-readable signal media may include media such as carrier waves. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. A computer program may include a pure software implementation containing instructions that perform the operations of a desired implementation.

[0079] Various general-purpose systems may be used with the programs and modules according to the examples herein, or it may prove convenient to construct more specialized apparatus to perform the desired method steps. Additionally, the example implementations are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages ​​may be used to implement the techniques of the example implementations as described herein. Instructions in the programming language may be executed by one or more processing devices, such as a central processing unit (CPU), processor, or controller.

[0080] As is known in the art, the operations described above can be implemented by hardware, software, or some combination of software and hardware. Various aspects of exemplary implementations may be implemented using circuits and logic devices (hardware), while other aspects may be implemented using instructions stored on a machine-readable medium (software) that, when executed by a processor, cause the processor to perform methods that implement implementations of the present application. Furthermore, some implementations of the present application may be implemented solely by hardware, while other exemplary implementations may be implemented solely by software. Furthermore, the various functions described may be implemented in a single unit or may be spread across multiple components in various ways. When implemented by software, the methods may be executed by a processor, such as a general-purpose computer, based on instructions stored on a machine-readable medium. If desired, the instructions may be stored on the medium in compressed and / or encrypted format.

[0081] Additionally, other implementations of the present application will be apparent to those skilled in the art from consideration of this specification and practice of the techniques of this application. Various aspects and / or components of the described example implementations may be used alone or in any combination. It is intended that the specification and example implementations be considered as examples only, with the true scope and spirit of the present application being indicated by the following claims.

Claims

1. A bifunctional autoencoder (BFAE) that performs two-way dimensionality reduction using a functional neural network (FNN), comprising: a functional encoder that, in response to receiving time series information at an input layer of the FNN, performs the steps of: processing the time series information through successive hidden layers including successive neurons to learn a low-dimensional representation of the time series information; and performing two-way dimensionality reduction using basis expansion to reduce both the number of features and the number of time points in the time series information to reduce at least one of computation time, storage requirements, or data transfer time; a functional decoder that uses the learned low-dimensional representation to obtain reconstructed time series information, the function decoder using the learned low-dimensional representation of the time series information; and performing an analysis task using the low-dimensional representation of the time series information.

2. The BFAE of claim 1, wherein temporal relationships in the successive hidden layers are preserved.

3. The BFAE of claim 1, wherein the function encoder selects basis functions from at least one of B-splines, wavelets, and Fourier functions to increase data capture efficiency.

4. The BFAE of claim 1, wherein the successive hidden layers utilize the successive neurons to identify at least one of a function or a pattern in the time series information.

5. The BFAE of claim 1, wherein the function encoder reduces the dimensionality of the time series information data with minimal information loss.

6. The BFAE of claim 1, wherein the function decoder reconstructs the original time series information from the reduced dimensional representation.

7. The BFAE of claim 1, wherein the combination of the function encoder and the function decoder performs dimensionality reduction.

8. The BFAE of claim 1, wherein the function encoder determines the number of features and the number of time points observed in the latent representation layer.

9. The BFAE of claim 1, wherein the continuous neurons of the hidden layer are defined using a parametric function and a bivariate parametric function.

10. A multi-class classification system using a functional neural network (FNN), comprising: a model training module that, in a training phase, constructs an FNN model that learns a mapping of time series data to classes associated with the time series data; a model deployment module that, in an inference phase, applies the trained FNN model to time series data using the mapping to detect three or more classes in the time series data, wherein the FNN model is trained by using at least one of a first-order method (FDNN) or basis deployment, each comprising successive neurons forming successive hidden layers; and outputting the three or more classes.

11. The system of claim 10, wherein temporal relationships in the successive hidden layers are preserved.

12. The system of claim 10, wherein the loss is a cross-entropy loss and the activation function is a softmax activation function.

13. The system of claim 10, wherein the continuous neurons of the hidden layer are defined using a parametric function and a bivariate parametric function.

14. A non-transitory computer-readable medium storing instructions for executing a process, the instructions including: at a function encoder, in response to receiving time series information at an input layer of a function neural network (FNN), performing steps including: processing the time series information through successive hidden layers comprising successive neurons to learn a low-dimensional representation of the time series information; and performing a two-way dimensionality reduction using basis expansion to reduce both the number of features and the number of time points in the time series information to reduce at least one of computation time, storage requirements, or data transfer time; at a function decoder, using the learned low-dimensional representation to obtain reconstructed time series information, the function decoder using the learned low-dimensional representation of the time series information; and performing an analysis task using the low-dimensional representation of the time series information.

15. A non-transitory computer-readable medium storing instructions for carrying out a process, the instructions including: in a training phase, constructing a functional neural network (FNN) model that learns a mapping of time series data to classes associated with the time series data; in an inference phase, applying the trained FNN model to the time series data using the mapping to detect three or more classes in the time series data, wherein the FNN models are trained by using at least one of a first-order method (FDNN) or basis expansion, each including successive neurons forming successive hidden layers; and outputting the three or more classes.

Images