Ensemble of grouped two-stage attention-based recurrent neural networks for multivariate time series prediction

Abstract

A computer-implemented method for multivariate time series forecasting, comprising the following: Decomposing (320) each time series from a set of several source time series and a target time series into a raw component containing an original time segment, a shape component containing the original time segment minus the values ​​of each time series in time step T, and a trend component containing a value difference of two successive time steps, wherein the target time series comprises time series data from at least one hardware sensor for anomaly detection of a hardware system; for each decomposed component, selecting a drive time series relevant to it from the set and obtaining hidden features of the selected drive time series by applying (330) the set to an attention-based input encoder of an ensemble of grouped two-stage attention-based recurrent neural networks (EC-DARNNS); automatic grouping (330) by a hardware processor of the hidden features in a hidden space using an attention-based temporal decoder of the EC-DARNNS, wherein each grouped two-stage attention-based RNN in the ensemble is fixedly assigned to and applied to a respective decomposed component; and Predictions (330) of a respective value of one or more future time steps for the target series based on respective prediction outputs for each decomposed component by the EC-DARNNS; and the hardware system is controlled in response to the respective value of one or more future time steps.

Description

BACKGROUND Technical area

[0001] The present invention relates to data processing and in particular to an ensemble of grouped two-stage attention-based recurrent neural networks (RNNs) for multivariate time series prediction. Description of the related area

[0002] Many real-world applications, such as financial market forecasting, medical diagnostics, weather forecasting, communication network traffic forecasting, predictive maintenance, and others, can benefit from multivariate time series forecasting.

[0003] QIN, Yao [et al.]: A dual-stage attention-based recurrent neural network for time series prediction. arXiv preprint arXiv:1704.02971v4, 2017 discloses a dual-stage attention-based recurrent neural network (DA-RNN) for predicting time series. The encoder employs a novel input attention mechanism that adaptively selects the relevant drive sequences. The decoder uses a temporal attention mechanism to automatically select relevant hidden states of the encoder across all time steps. With these two attention mechanisms, the DA-RNN can adaptively select the most relevant input features and capture the long-term temporal dependencies of a time series.

[0004] CN 1 09 919 364A discloses multivariate time series forecasting techniques based on adaptive noise suppression and integrated LSTM.

[0005] CN 1 09 325 875 A reveals a type of implicit group discovery method based on the hidden characteristics of social network users.

[0006] CAO, Jian; LI, Zhi; LI, Jian: Financial time series forecasting model based on CEEMDAN and LSTM. Physica A: Statistical mechanics and its applications, 2019, Vol. 519, pp. 127-139. This paper reveals two hybrid forecasting models that combine the two types of empirical mode decomposition (EMD) with long-term memory (LSTM). The financial time series is a type of nonlinear and nonstationary random signal that can be decomposed into several intrinsic mode functions with different time scales by the original EMD and the complete empirical mode decomposition with adaptive noise (CEEMDAN).

[0007] Mixture model. In: Wikipedia, The Free Encyclopedia. Revision as of 18.07.2019. URL: https: / / en.wikipedia.org / w / index.php?title=Mixture_model&ol-did=906800500 [accessed on 05.03.2024] reveals a definition of the mixture model, where the mixture model is a probabilistic model for representing the presence of subpopulations within a total population, without requiring an observed data set to identify the subpopulation to which a single observation belongs.

[0008] Several types of studies have been conducted for the multivariate time series forecasting problem. Three key types include the following: (1) linear techniques such as an autoregressive moving average model (ARMA model) or an autoregressive integrated moving average model (ARIMA model); (2) probabilistic models such as Gaussian processes (GPs); and (3) deep neural networks such as long short-term memories (LSTMs).

[0009] However, these methods are limited for the following reasons: (1) Linear methods cannot capture the underlying nonlinear relationships; (2) Probability models employ a predefined nonlinear form and do not need to be flexible to capture complex relationships; and (3) existing deep neural networks, e.g., LSTM, do not consider the structural information of hidden features (input data) and cannot process multiple types of historical data. SUMMARY

[0010] The present invention is defined in the independent claims. The dependent claims define embodiments of the invention. According to aspects of the present disclosure, a computer-implemented method for multivariate time series forecasting is provided. The method comprises decomposing each time series from a set of several drive time series and a target time series into a raw component, a shape component, and a trend component. The method further comprises, for each decomposed component, selecting a drive time series relevant to it from the set and obtaining hidden features of the selected drive time series by applying the set to an attention-based input encoder of an ensemble of clustered two-stage attention-based recurrent neural networks (EC-DARNNS).The procedure also includes automatic grouping of the hidden features in a hidden space by a hardware processor using an attention-based temporal decoder of the EC-DARNNS. Each of the grouped two-stage attention-based RNNs in the ensemble is fixed to a specific decomposed component and applied to it. The procedure further includes predicting the value of one or more future time steps for the target series based on the respective prediction outputs for each decomposed component by the EC-DARNNS.

[0011] According to further aspects of the present disclosure, a computer program product for multivariate time series forecasting is created. The computer program product includes a non-transitory, computer-readable storage medium containing program instructions embodied therein. The program instructions are executable by a computer to cause the computer to perform a procedure. The procedure comprises decomposing each time series from a set of multiple drive time series and a target time series into a raw component, a shape component, and a trend component. The procedure further comprises, for each decomposed component, selecting a drive time series relevant to it from the set and obtaining hidden features of the selected drive time series by applying the set to an attention-based input encoder of an ensemble of clustered two-stage attention-based recurrent neural networks (EC-DARNNS).The procedure also includes automatically grouping the hidden features in a hidden space using an attention-based temporal decoder of the EC-DARNNS. Each of the grouped two-stage attention-based RNNs in the ensemble is fixed to a specific decomposed component and applied to it. The procedure further includes predicting the value of one or more future time steps for the target series based on the respective prediction outputs for each decomposed component by the EC-DARNNS.

[0012] According to further aspects of the present disclosure, a computer processing system for multivariate time series forecasting is created. The computer processing system includes a working memory for storing program code. The computer processing system further includes a processor that is functionally coupled to the working memory to execute the program code to decompose each time series from a set of multiple drive time series and a target time series into a raw component, a shape component, and a trend component. The processor further executes the program code to select, for each decomposed component, a drive time series relevant to it from the set and to obtain hidden features of the selected drive time series by applying the set to an attention-based input encoder of an ensemble of clustered two-stage attention-based recurrent neural networks (EC-DARNNS).The processor also executes the program code to automatically group the hidden features in a hidden space using an attention-based temporal decoder of the EC-DARNNS. Each of the grouped two-stage attention-based RNNs in the ensemble is fixed to and applied to a specific decomposed component. The processor further executes the program code to predict a specific value for one or more future time steps of the target sequence based on the respective prediction outputs for each decomposed component by the EC-DARNNS.

[0013] These and other features and advantages will become apparent from the following detailed description of their illustrative embodiments, which should be read in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The disclosure provides details in the following description of preferred embodiments with reference to the following figures; they show: Fig. 1 a block diagram showing an exemplary computing device in accordance with an embodiment of the present invention; Fig. 2 a block diagram showing an exemplary method for multivariate time series forecasting using an ensemble of grouped attention-based dual-state RNNS, in accordance with an embodiment of the present invention; Fig. 3 a block diagram, which further shows various blocks of Fig. 2 shows, in accordance with an embodiment of the present invention; Fig. 4 a block diagram showing an exemplary basic structure of an EC-DARNN model in accordance with an embodiment of the present invention; Fig. 5 a block diagram showing an exemplary basic structure of an input attention mechanism of a grouped DARNN model, in accordance with an embodiment of the present invention; Fig. 6 a block diagram showing an exemplary basic structure of a temporal attention mechanism of a grouped DARNN model, in accordance with an embodiment of the present invention; Fig. 7 a flowchart illustrating an exemplary method for using a grouped DARNN model in accordance with an embodiment of the present invention; Fig. 8 a flowchart illustrating an exemplary method for feature regularization in accordance with an embodiment of the present invention; and Fig. 9 a block diagram showing an exemplary computing environment in accordance with an embodiment of the present invention. DETAILED DESCRIPTION OF PREFERRED EXECUTION FORMS

[0015] In accordance with embodiments of the present invention, systems and methods for multivariate time series forecasting are created using an ensemble of grouped two-stage attention-based recurrent neural networks (RNNS).

[0016] In particular, the present invention, with respect to an illustrative embodiment, predicts the future target time series values ​​from time T + 1 to T + N based on the historical data of several input time series and the target time series from time step 1 to T. As a non-limiting example, the present invention can be applied to perform an indoor temperature forecast for energy efficiency purposes. In this case, the indoor temperature is the target series, and the input series contains the relative humidity, carbon dioxide in ppm, whether it is raining, the outdoor temperature, etc.

[0017] To address this problem, one or more embodiments of the present invention propose a continuous deep encoder / decoder basic structure, i.e., an ensemble of grouped two-level attention-based recurrent neural networks (EC-DARNNs), to perform multivariate time series prediction.

[0018] In one embodiment, the present invention performs multivariate time series forecasting as follows. First, the raw time series data is decomposed into three different components: raw, shape, and trend. Then, for each component, a set of several drive sequences is fed into attention-based input encoders to select relevant drive sequences and obtain hidden features. The hidden features are then automatically grouped in the hidden space based on a Gaussian mixture model. Finally, the grouped hidden features and the target sequence are fed into decoders, and the output of the decoders from different components is used to predict the values ​​of future time steps.

[0019] Fig. Figure 1 is a block diagram showing an exemplary computing device 100 in accordance with an embodiment of the present invention. The computing device 100 is configured to perform a multivariate time series forecast.

[0020] The Computing Unit 100 can be embodied as any type of computing or computer equipment capable of performing the functions described herein, including, without limitation, a computer, a server, a rack-based server, a blade server, a workstation, a desktop computer, a laptop computer, a notebook computer, a tablet computer, a mobile computing unit, a portable computing unit, a network device, an internet device, a distributed computing system, a processor-based system, and / or a consumer electronics device. Additionally or alternatively, the Computing Unit 100 can be embodied as one or more compute sleds, memory sleds, or additional racks, sleds, compute enclosures, or other components of a physically dispersed Computing Unit. As described in Fig. As shown in Figure 1, the computing device 100 includes, for illustrative purposes, the processor 110, an input / output subsystem 120, a main memory 130, a data storage device 140, and a communication subsystem 150, and / or other components and devices commonly found in a server or similar computing device. Of course, in other embodiments, the computing device 100 may include further or additional components, such as those commonly found in a server computer (e.g., various input / output devices). Additionally, in some embodiments, one or more of the illustrative components may be incorporated into another component or otherwise form a section thereof. For example, in some embodiments, the main memory 130, or sections thereof, may be incorporated into the processor 110.

[0021] The Processor 110 can be embodied as any type of processor capable of performing the functions described herein. The Processor 110 can be embodied as a single processor, multiple processors, one or more central processing units (one or more CPUs), one or more graphics processing units (one or more GPUs), one or more single-core or multi-core processors, one or more digital signal processors, one or more microcontrollers, one or more other processors, or one or more processing / control circuits.

[0022] The main memory 130 can be any type of volatile or non-volatile memory or data storage capable of performing the functions described herein. During operation, the main memory 130 can store various data and software used during the operation of the computing device 100, such as operating systems, applications, programs, libraries, and drivers. The main memory 130 is communicatively coupled to the processor 110 by means of the input / output subsystem 120, which can be implemented as a circuit arrangement and / or components, to facilitate input / output operations with the processor 110, the main memory 130, and other components of the computing device 100.For example, the input / output subsystem 120 can be embodied or otherwise incorporated as memory control unit hubs, input / output control hubs, platform control unit hubs, an integrated control circuit arrangement, firmware facilities, communication links (e.g., point-to-point links, bus links, wires, cables, optical fibers, printed circuit board traces, etc.), and / or other components and subsystems to facilitate the input / output operations. In some embodiments, the input / output subsystem 120 can form a section of a system-on-a-chip (SoC) and be included on a single integrated circuit chip together with the processor 110, the memory 130, and other components of the computing device 100.

[0023] The data storage device 140 can be embodied as any type of device or devices configured for short-term or long-term data storage, such as storage devices and circuits, memory cards, hard disk drives, solid-state drives, or other data storage devices. The data storage device 140 can store program code 140A for multivariate time series forecasting. The communication subsystem 150 of the computing device 100 can be embodied as any network interface control unit or other communication circuit, device, or collection thereof capable of enabling communications between the computing device 100 and other remote devices over a network. The communication subsystem 150 can be configured to support one or more communication technologies (e.g., wired or wireless communications) and associated protocols (e.g.,to use Ethernet, InfiniBand®, Bluetooth®, Wi-Fi®, WiMAX, etc.) to effect such communication.

[0024] As shown, the computing device 100 can also include one or more peripheral devices 160. The peripheral devices 160 can include any number of additional input / output devices, interface devices, and / or other peripheral devices. For example, in some embodiments, the peripheral devices 160 can include a display device, a touchscreen, a graphics circuit arrangement, a keyboard, a mouse, a speaker system, a microphone, a network interface, and / or other input / output devices, interface devices, and / or peripheral devices.

[0025] Of course, the computing device 100 can also include other elements (not shown), as can be readily deduced by a person skilled in the art, as well as omit certain elements. For example, depending on its specific implementation, the computing device 100 can include various other input and / or output devices, as can be readily understood by a person skilled in the art. For example, different types of wireless and / or wired input and / or output devices can be used. Furthermore, additional processors, control units, memory, etc., can be used in several configurations. These and other variants of the processing system 100 are readily deduced by a person skilled in the art based on the instructions of the present invention provided herein.

[0026] The term “hardware processor subsystem” or “hardware processor,” as used here, can refer to a processor, a working memory (which includes RAM, one or more caches, etc.), software (which includes memory management software), or combinations thereof, working together to perform one or more specific tasks. In useful embodiments, the hardware processor subsystem may include one or more data processing elements (e.g., logic circuits, processing circuits, instruction execution units, etc.). The one or more data processing elements may be contained in a central processing unit, a graphics processing unit, and / or a separate control unit based on processor or computational elements (e.g., logic gates, etc.). The hardware processor subsystem may include one or more integrated working memories (e.g.,This may include temporary storage, dedicated memory arrangements, read-only memory, etc. In some embodiments, the hardware processor subsystem may include one or more working memories, which may be integrated or non-integrated, or which may be dedicated for use by the hardware processor subsystem (e.g., ROM, RAM, basic input / output system (BIOS), etc.).

[0027] In some embodiments, the hardware processor subsystem can contain and execute one or more software elements. These one or more software elements can include an operating system and / or one or more applications and / or specific code to achieve a defined result.

[0028] In further embodiments, the hardware processor subsystem can include a dedicated, specialized circuit arrangement that performs one or more electronic processing functions to achieve a defined result. Such a circuit arrangement can include one or more application-specific integrated circuits (ASICs), FPGAs, and / or PLAs.

[0029] These and other variants of a hardware processor subsystem are also provided in accordance with embodiments of the present invention.

[0030] Fig. Figure 2 is a block diagram showing an exemplary method 200 for multivariate time series forecasting using an ensemble of grouped attention-based dual-state RNNS, in accordance with an embodiment of the present invention.

[0031] In Block 210, a set of sensors is installed, for example in a power plant, an IT system, etc., to monitor the status of the power plant, the IT system, etc.

[0032] Block 220 collects multivariate time series data (containing historical source and target series) in these systems.

[0033] In block 230, the multivariate time series data are entered into the EC-DARNN model.

[0034] Block 240 provides the output of the EC-DARNN model as the predicted values ​​from time steps T + 1 to T + N. The output can be in the form of individual predictions for each of the decomposed components, including raw, shape, and trend, or it can be a single prediction derived from the preceding multiple predictions.

[0035] Block 250 performs an action in response to the search results.

[0036] Examples of actions include, but are not limited to, the detection of anomalies in computer processing systems and the control of the system in which an anomaly is detected. For example, a query in the form of time-series data from a hardware sensor or sensor network (e.g., grid) indicating anomalous behavior (dangerous or otherwise excessive operating speed (e.g., motor, transmission linkage), dangerous or otherwise excessive operating heat (e.g., motor, transmission linkage), dangerous or otherwise out-of-tolerance orientation (e.g., motor, transmission linkage, etc.)) can be labeled using a text message.In a processing pipeline, an initial input time series can be processed in multiple text messages and then recombined to include a subset of the text messages for a more focused resulting output time series with respect to a given topic (e.g., an anomaly type). Accordingly, based on the implementation, a facility can be switched off, its operating speed can be reduced, an alignment procedure (e.g., hardware-based) can be performed, and so on.

[0037] Another exemplary action could be operational parameter tracking, where a history of parameter changes over time can be logged because it is used to perform further functions such as hardware machine control functions, which include switching on or off, slowing down, speeding up, adjusting positions, etc., based on the detection of a given operating state, which is equated to a given output time series and / or a given text comment relating to historical data.

[0038] Exemplary environments in which the present invention can be used include, but are not limited to, power plants, information technology systems, manufacturing plants, etc. These and other environments can be readily considered by a person skilled in the art based on the instructions for the present invention provided herein.

[0039] Fig. Figure 3 is a block diagram, which also includes blocks 230 and 240 of Fig. Figure 2 shows, in accordance with an embodiment of the present invention.

[0040] In block 310, the multivariate time series data are entered at different time steps.

[0041] In block 320, the time series is decomposed into different types, e.g., raw, form, and trend. The raw type contains the original time series segment. The form type contains the original time series segment minus the values ​​of each time series at time step T. The trend type contains the value difference between two consecutive time steps.

[0042] In block 330, different types of time series data are fed into an EC-DARNN to obtain prediction results.

[0043] Fig. Figure 4 is a block diagram showing an exemplary basic structure 400 of an EC-DARNN model in accordance with an embodiment of the present invention.

[0044] The basic structure 400 contains an input 410, from which various types of input time series, including scale 411, form 422 and trend 413, are obtained.

[0045] A grouped DARNN 1 421 processes the scale input time series 411 to learn the hidden representation (the features) of this type in order to generate an initial prediction 431.

[0046] A grouped DARNN 2 422 processes the shape input time series 412 to learn the hidden representation (the features) of this type in order to generate a second prediction 432.

[0047] A grouped DARNN 3 423 processes the trend input time series 413 to learn the hidden representation (the features) of this type in order to generate a third prediction 433.

[0048] From the first prediction 431, the second prediction 432 and the third prediction 443, a final prediction 440 is generated based on an attention mechanism that can adaptively select the important factors.

[0049] Fig. Figure 5 is a block diagram showing an exemplary basic structure 500 of an input attention mechanism of a grouped DARNN model, in accordance with an embodiment of the present invention.

[0050] The basic structure 500 comprises a control row 510 of length T, an attention input layer 520, a softmax layer 530 and a new input 540 at time t.

[0051] Fig. Figure 6 is a block diagram showing an exemplary basic structure 600 of a temporal attention mechanism of a grouped DARNN model, in accordance with an embodiment of the present invention.

[0052] The basic structure 600 comprises an encoder 610, which is formed from a set of LSTMs 611, a temporal attention layer 620, a softmax layer 630 and a decoder 640, which is formed from another set of LSTMs 641.

[0053] Fig. Figure 7 is a flowchart showing an exemplary method 700 for using a grouped DARNN model in accordance with an embodiment of the present invention.

[0054] In block 710, the target time series of a type (e.g. scale, shape or trend) is entered into the grouped DARNN.

[0055] In block 720, the input is encoded by the attention input layer 520 to obtain hidden representations / features based on the relevant time steps.

[0056] In block 730, the hidden representations / features are regularized by a loss of Gaussian mixture models (GMM loss).

[0057] In block 740, the hidden features and target sequences of the same type (as the input control sequence) are used in the temporal attention layer 620 of the decoder 640 to perform a prediction based on the relevant hidden temporal features.

[0058] Fig. Figure 8 is a flowchart showing an exemplary method 800 for feature regularization in accordance with an embodiment of the present invention.

[0059] In block 810, the number of clusters is predefined based on a set of hidden features.

[0060] In block 820, a GMM is applied to the hidden features and a GMM loss is used as a regularization expression to regularize the prediction.

[0061] Fig.Figure 9 is a block diagram showing an exemplary computing environment 900 in accordance with an embodiment of the present invention.

[0062] Environment 900 contains a server 910, several client facilities (which are collectively designated by the figure reference 920), a controlled system A 941 and a controlled system B 942.

[0063] Communication between units in an environment 700 can be carried out via one or more networks 930. A wireless network 930 is shown for illustration. In further embodiments, a wired, a wireless, and / or a combination thereof can be used to facilitate communication between the units.

[0064] Server 910 receives time-series data from client devices 920. Server 910 can control one of the systems 941 and / or 942 based on a prediction generated by this data. In one embodiment, the time-series data can include data related to the controlled systems 941 and / or 942, such as, but not limited to, sensor data.

[0065] Some of the many accompanying benefits of an EC-DARNN over previous work include, but are not limited to, the following: (1) The preceding time series forecasting techniques based on deep neural networks do not explicitly take into account the structural information of hidden features (input data), whereas EC-DARNNs address this problem by using a Gaussian mixture model over the hidden features. (2) Conventional time series forecasting techniques based on deep neural networks cannot process multiple types of historical time series data, whereas EC-DARNNs address this problem by means of the ensemble of several basic grouped DARNNs with a unified task.

[0066] EC-DARNNs are the first model to incorporate the structural information of hidden features (input data) and to use multiple types of historical time series data to make time series predictions. These and other concomitant advantages are readily apparent to a person skilled in the art based on the instructions of the present invention provided herein.

[0067] The embodiments described herein can consist entirely of hardware, entirely of software, or of both hardware and software elements. In a preferred embodiment, the present invention is implemented in software, which includes, but is not limited to, firmware, resident software, microcode, etc.

[0068] Embodiments may include a computer program product accessible from a computer-usable or computer-readable medium, providing program code for use by or in conjunction with a computer or any command execution system. A computer-usable or computer-readable medium may include a device that stores, communicates, disseminates, or transports the program for use by or in conjunction with the command execution system, command execution device, or command execution unit. The medium may be a magnetic, optical, electronic, electromagnetic, infrared, or semiconductor system (or a corresponding device or unit), or a dissemination medium. The medium may be a computer-readable storage medium such as...a semiconductor or solid-state working memory, a magnetic tape, a removable computer floppy disk, a read / write memory (RAM), a read-only memory (ROM), a rigid magnetic data carrier, and an optical disk, etc.

[0069] Any computer program can be materially stored in a machine-readable storage medium or storage device (e.g., program memory or a magnetic storage medium) that is readable by a general-purpose programmable computer or a programmable computer for a specific purpose, in order to configure and control the operation of a computer and to perform the procedures described herein when the storage medium or storage device is read by the computer. The invention system can also be considered to be embodied in a computer-readable storage medium configured with a computer program, wherein the storage medium configured in this way causes a computer to operate in a specific and predefined manner to perform the functions described herein.

[0070] A data processing system capable of storing and / or executing program code may contain at least one processor, which is directly or indirectly coupled to memory elements via a system bus. The memory elements may include local working memory, used during actual program code execution; bulk memory; and intermediate memory, which provides temporary storage of at least a portion of the program code to reduce the number of code retrievals from bulk memory during execution. Input / output or I / O devices (including, but not limited to, keyboards, displays, pointers, etc.) may be coupled to the system either directly or via intermediary input / output control units.

[0071] Network adapters can also be connected to the system to allow the data processing system to be connected to other data processing systems or remote printers or storage devices via intermediary private or public networks. Modems, cable modems, and Ethernet cards are just a few of the currently available types of network adapters.

[0072] A reference in the specification to "a particular embodiment" or "an embodiment" of the present invention, as well as further variants thereof, means that a particular feature, structure, property, etc., described in connection with the embodiment, is included in at least one embodiment of the present invention. Thus, the appearances of the phrase "in a particular embodiment" or "in an embodiment," as well as further variants appearing at various points in the specification, do not necessarily all refer to the same embodiment. However, it is understood that features of one or more embodiments can be combined based on the instructions for the present invention provided herein.

[0073] It is understood that the use of any of the following “ / ”, “and / or” and “at least one of” e.g. in the cases of “A / B”, “A and / or B” and “at least one of A and B” is intended to include the selection of only the first listed option (A) or the selection of only the second listed option (B) or the selection of both options (A and B).As a further example, in the cases of "A, B and / or C" and "at least one of A, B and C", such a phrase is intended to include the selection of only the first listed option (A), or the selection of only the second listed option (B), or the selection of only the third listed option (C), or the selection of only the first and second listed options (A and B), or the selection of only the first and third listed options (A and C), or the selection of only the second and third listed options (B and C), or the selection of all three options (A, B, and C). This can be extended to all listed elements.

[0074] The foregoing is to be understood as illustrative and exemplary in every respect, but not as limiting, and the scope of the invention disclosed herein is not to be determined from the precise description, but rather from the claims, which are to be interpreted in accordance with the full scope permitted by patent law. It is understood that the embodiments shown and described herein are merely illustrative of the present invention and that those skilled in the art may implement various modifications without departing from the scope and spirit of the invention. Skilled in the art may implement various further combinations of features without departing from the scope and spirit of the invention.Having thus described aspects of the invention with the details and comprehensiveness required by patent law, the attached claims set out what is claimed and what is sought to be protected by the patent.

Claims

[1] Computer-implemented method for multivariate time series forecasting, comprising the following: Decomposing (320) each time series from a set of several source time series and a target time series into a raw component containing an original time segment, a shape component containing the original time segment minus the values ​​of each time series in time step T, and a trend component containing a value difference of two successive time steps, wherein the target time series comprises time series data from at least one hardware sensor for anomaly detection of a hardware system; for each decomposed component, selecting a drive time series relevant to it from the set and obtaining hidden features of the selected drive time series by applying (330) the set to an attention-based input encoder of an ensemble of grouped two-stage attention-based recurrent neural networks (EC-DARNNS); automatic grouping (330) by a hardware processor of the hidden features in a hidden space using an attention-based temporal decoder of the EC-DARNNS, wherein each grouped two-stage attention-based RNN in the ensemble is fixedly assigned to and applied to a respective decomposed component; and Predictions (330) of a respective value of one or more future time steps for the target series based on respective prediction outputs for each decomposed component by the EC-DARNNS; and the hardware system is controlled in response to the respective value of one or more future time steps. [2] Computer-implemented method according to claim 1, wherein the automatic grouping step automatically groups the hidden features in the hidden space on the basis of a Gaussian mixture model (GMM). [3] Computer-implemented method according to claim 1, wherein the Gaussian mixture model is used for regularizing hidden features. [4] Computer-implemented method according to claim 3, wherein the hidden features are regularized with respect to a GMM loss implemented as a regularization expression. [5] Computer-implemented method according to claim 1, wherein the set of several control time series represents historical data. [6] Computer-implemented method according to claim 1, wherein the attention-based input encoder comprises multiple long short-term memories (LTSMs). [7] Computer-implemented method according to claim 6, wherein the attention-based input encoder further comprises one or more attention input layers followed by one or more softmax layers. [8] Computer-implemented method according to claim 7, wherein the multiple LSTMs precede the one or more attention input layers. [9] Computer-implemented method according to claim 1, wherein the attention-based temporal decoder comprises multiple long short-term memories (LTSMs). [10] Computer-implemented method according to claim 9, wherein the attention-based temporal decoder further comprises one or more temporal attention layers followed by one or more softmax layers. [11] Computer-implemented method according to claim 10, wherein the multiple LSTMs precede the one or more temporal attention layers. [12] Computer program product for multivariate time series forecasting, wherein the computer program product comprises a non-transitory computer-readable storage medium embodying program instructions, wherein the program instructions are executable by a computer to cause the computer to perform a procedure comprising: Decomposing (320) each time series from a set of several source time series and a target time series into a raw component containing an original time segment, a shape component containing the original time segment minus the values ​​of each time series in time step T, and a trend component containing a value difference of two successive time steps, wherein the target time series comprises time series data from at least one hardware sensor for anomaly detection of a hardware system; for each decomposed component, selecting a drive time series relevant to it from the set and obtaining hidden features of the selected drive time series by applying (330) the set to an attention-based input encoder of an ensemble of grouped two-stage attention-based recurrent neural networks (EC-DARNNS); automatic grouping (330) of the hidden features in a hidden space using an attention-based temporal decoder of the EC-DARNNS, wherein each grouped two-stage attention-based RNN in the ensemble is fixedly assigned to and applied to a respective decomposed component; and Predictions (330) of a respective value of one or more future time steps for the target series based on respective prediction outputs for each decomposed component by the EC-DARNNS; and the hardware system is controlled in response to the respective value of one or more future time steps. [13] Computer program product according to claim 12, wherein the automatic grouping step automatically groups the hidden features in the hidden space on the basis of a Gaussian mixture model (GMM). [14] Computer program product according to claim 12, wherein the Gaussian mixture model is used for the regularization of hidden features. [15] Computer program product according to claim 14, wherein the hidden features are regularized with respect to a GMM loss implemented as a regularization expression. [16] Computer program product according to claim 12, wherein the set of several control time series represents historical data. [17] Computer program product according to claim 12, wherein the attention-based input encoder comprises multiple long short-term memories (LTSMs). [18] Computer processing system for multivariate time series forecasting, comprising the following: a working memory (130) to store program code; and a processor (110) that is functionally linked to the main memory in order to execute the program code Decomposing each time series from a set of multiple source time series and a target time series into a raw component containing an original time segment, a shape component containing the original time segment minus the values ​​of each time series in time step T, and a trend component containing a value difference of two successive time steps, wherein the target time series comprises time series data from at least one hardware sensor for anomaly detection of a hardware system; For each of the decomposed components, select a drive time series relevant to it from the set and obtain hidden features of the selected drive time series by applying the set to an attention-based input encoder of an ensemble of grouped two-stage attention-based recurrent neural networks (EC-DARNNS); Automatic grouping of the hidden features in a hidden space using an attention-based temporal decoder of the EC-DARNNS, wherein each grouped two-stage attention-based RNN in the ensemble is fixedly assigned to and applied to a respective decomposed component; and Predictions of a respective value of one or more future time steps for the target series based on respective prediction outputs for each decomposed Component through the EC-DARNNS; and the hardware system is controlled in response to the respective value of one or more future time steps.

Images