Dual-Channel Networks for Retrieving Multivariate Time Series with Static States

Abstract

A computer-implemented method is provided. The method includes jointly encoding a current time series segment and a corresponding static state into compact features by a dual channel feature extractor. The method further includes converting the compact features into binary codes by a binary code extractor. The method also includes calculating distances between the binary codes and all binary codes stored in a binary code database. The method further includes searching for top relevant multivariate time series segments based on the distances.

Description

[Technical Field]

[0001] The present invention relates generally to time series processing, and more particularly to dual channel networks for multivariate time series retrieval with stationary states. [Background technology]

[0002] 2. Description of Related Art Multivariate time series search is the task of finding the most relevant multivariate time series segments from a vast amount of historical data by querying them with current observations. A realistic way to perform multivariate time series search is to obtain a compact representation of the historical data in binary code that preserves the relative similarity relationships in the raw input space. However, the state of a system is not always determined solely by the time series that describe the dynamic system behavior, but sometimes also by the static state of the system. Summary of the Invention

[0003] According to an aspect of the present invention, there is provided a computer-implemented method. The method includes jointly encoding a current time series segment and a corresponding static state into compact features by a dual-channel feature extractor. The method further includes converting the compact features into binary codes by a binary code extractor. The method also includes calculating distances between the binary codes and all binary codes stored in a binary code database. The method further includes retrieving top-relevant multivariate time series segments based on the distances.

[0004] According to another aspect of the present invention, a computer program product is provided. The computer program product includes a non-transitory computer-readable storage medium having program instructions embodied thereon. The program instructions are computer-executable to cause the computer to perform a method. The method includes jointly encoding a current time series segment and a corresponding static state into compact features by a dual-channel feature extractor embedded in one or more hardware processors of the computer. The method further includes converting the compact features into binary codes by a binary code extractor embedded in the one or more hardware processors. The method also includes calculating, by the one or more hardware processors, distances between the binary codes and all binary codes stored in a binary code database. The method further includes searching, by the one or more hardware processors, for top-relevant multivariate time series segments based on the distances.

[0005] According to yet another aspect of the present invention, there is provided a computer processing system. The computer processing system includes a memory device for storing program code. The computer processing system further includes a processor device for executing program code for jointly encoding a current time series segment and a corresponding static state into compact features by a dual-channel feature extractor embedded in the one or more hardware processors. The one or more hardware processors further execute program code for converting the compact features into binary codes by a binary code extractor embedded in the one or more hardware processors. The one or more hardware processors also execute program code for calculating distances between the binary code and all binary codes stored in a binary code database. The one or more hardware processors further execute program code for searching for highly relevant multivariate time series segments based on the distances.

[0006] These and other features and advantages will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings. [Brief explanation of the drawings]

[0007] The present disclosure provides details in the following description of preferred embodiments with reference to the following figures.

[0008] [Figure 1] FIG. 2 is a block diagram illustrating an exemplary computing device, according to an embodiment of the present invention.

[0009] [Figure 2] FIG. 2 is a block diagram illustrating an exemplary dual-channel feature extractor, in accordance with an embodiment of the present invention.

[0010] [Figure 3] 3 is a block diagram illustrating the dual-channel feature extractor of FIG. 2 in accordance with an embodiment of the present invention.

[0011] [Figure 4] FIG. 1 is a flow diagram illustrating an exemplary training method, according to an embodiment of the present invention.

[0012] [Figure 5] FIG. 4 is a flow diagram illustrating an exemplary hashing stage, according to an embodiment of the present invention.

[0013] [Figure 6] FIG. 4 is a flow diagram illustrating an exemplary search stage according to an embodiment of the present invention.

[0014] [Figure 7] 1 is a block diagram illustrating an exemplary environment in which the present invention may be applied, in accordance with an embodiment of the present invention.

[0015] [Figure 8]FIG. 2 illustrates an exemplary time series, time series segments, and static states according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0016] Embodiments of the present invention are directed to a dual-channel network for searching multivariate time series with static states.

[0017] One or more embodiments of the present invention provide an end-to-end neural network model architecture that considers not only time series inputs but also static states for more accurate multivariate time series search.

[0018] As mentioned above, conventional state-of-the-art techniques for time series retrieval assume that the system state is determined solely by dynamic behavior expressed as a time series. However, in reality, the system state is also influenced by static observations such as driving mode and day of the week. Recurrent neural network-based time series coders, commonly used for time series retrieval, cannot handle such static inputs because they assume temporal dependencies between timestamps.

[0019] One or more embodiments of the present invention address the task of time series retrieval for systems whose state is determined not only by dynamic behavior but also by static profiles. Embodiments of the present invention incorporate a multilayer perceptron-based static encoder and a recurrent neural network-based temporal encoder. These encoders are jointly trained end-to-end based on a metric learning loss, such as triplet loss.

[0020] Thus, in one embodiment, at least one of the following two inventive features may be involved: (1) a multi-layer perceptron (MLP)-based static coder is jointly trained with a recurrent neural network (RNN)-based time series coder; and (2) a metric learning loss allows training a binary code that preserves the relative similarity between the input raw time series as well as the static state.

[0021] The task of multivariate time series search can be applied to many tasks in complex systems, including system state identification, fault detection, fault prediction, etc., to name a few example tasks to which embodiments of the present invention can be applied.

[0022] As used herein, a "static state" refers to a system state that does not change within the same contiguous time period as a time series segment. The difference between a static state and a time series is the frequency of change: a static state changes only at regular time intervals, whereas a time series changes from moment to moment.

[0023] 1 is a block diagram illustrating an exemplary computing device 100 according to one embodiment of the present invention, configured to perform multivariate time series searches with static status of a dual channel network.

[0024] Computing device 100 may be embodied as any type of computing or computing device capable of performing the functions described herein, including, but not limited to, a computer, a server, a rack-based server, a blade server, a workstation, a desktop computer, a laptop computer, a notebook computer, a tablet computer, a mobile computing device, a wearable computing device, a network appliance, a web appliance, a distributed computing system, a processor-based system, and / or a user electronic device. Additionally or alternatively, computing device 100 may be embodied as one or more compute sleds, memory sleds, or other racks, sleds, computing chassis, or other components of a physically separated computing device. As shown in FIG. 1 , computing device 100 illustratively includes processor 110, an input / output subsystem 120, memory 130, data storage device 140, and a communications subsystem 150, and / or other components and devices typically found in a server or similar computing device. Of course, computing device 100 may, in other embodiments, include other or additional components (e.g., various input / output devices) typically found in a server computer. Additionally, in some embodiments, one or more of the illustrated components may be incorporated into or otherwise form part of another component. For example, memory 130, or portions thereof, may be incorporated into processor 110 in some embodiments.

[0025] Processor 110 may be embodied as any type of processor capable of performing the functions described herein, including a single processor, multiple processors, a central processing unit (CPU), a graphics processing unit (GPU), a single or multi-core processor, a digital signal processor, a microcontroller, or other processor or processing / control circuitry.

[0026] Memory 130 may be embodied as any type of volatile or non-volatile memory or data storage capable of performing the functions described herein. During operation, memory 130 may store various data and software used during operation of computing device 100, such as an operating system, applications, programs, libraries, and drivers. Memory 130 is communicatively coupled to processor 110 via I / O subsystem 120 and may be embodied as circuits and / or components for facilitating input / output operations with processor 110, memory 130, and other components of computing device 100. For example, I / O subsystem 120 may be embodied as or otherwise include a memory controller hub, an input / output control hub, a platform controller hub, integrated control circuitry, firmware devices, communication links (e.g., point-to-point links, bus links, wires, cables, light guides, printed circuit board traces, etc.), and / or other components and subsystems for facilitating input / output operations. In some embodiments, I / O subsystem 120 may form part of a system-on-chip (SOC) and be integrated with processor 110, memory 130, and other components of computing device 100 on a single integrated circuit chip.

[0027] Data storage 140 may be embodied as any type of device or apparatus configured for short-term or long-term storage of data, such as, for example, memory devices and circuits, memory cards, hard disk drives, solid-state drives, or other data storage devices. Data storage 140 may store program code for multivariate time series searches with static states in a dual-channel network. Communications subsystem 150 of computing device 100 may be embodied as any network interface controller or other communications circuitry, device, or collection thereof capable of enabling communications between computing device 100 and other remote devices over a network. Communications subsystem 150 may be configured to achieve such communications using any one or more communications technologies (e.g., wired or wireless communications) and associated protocols (e.g., Ethernet, InfiniBand, Bluetooth, Wi-Fi, WiMAX, etc.).

[0028] As shown, computing device 100 may also include one or more peripheral devices 160. Peripheral devices 160 may include any number of additional input / output devices, interface devices, and / or other peripheral devices. For example, in some embodiments, peripheral devices 160 may include a display, a touchscreen, graphics circuitry, a keyboard, a mouse, a speaker system, a microphone, a network interface, and / or other input / output devices, interface devices, and / or peripheral devices. Peripherals may also include automotive systems, including steering, braking, acceleration, lighting, stability, etc., as described herein.

[0029] Of course, computing device 100 may include other elements (not shown) or omit certain elements, as would readily occur to one skilled in the art. For example, various other input and / or output devices may be included in computing device 100, depending on the particular implementation of the same, as would be readily understood by one skilled in the art. For example, various types of wireless and / or wired input and / or output devices may be used. Furthermore, additional processors, controllers, memory, etc. may be utilized in a variety of configurations. These and other variations of processing system 100 will be readily contemplated by one skilled in the art in view of the teachings of the present invention provided herein.

[0030] As employed herein, the terms “hardware processor subsystem” or “hardware processor” or “processor” can refer to a processor, memory (including RAM, cache, etc.), software (including memory management software), or combinations thereof, working together to perform one or more specific tasks. In useful embodiments, the hardware processor subsystem can include one or more data processing elements (e.g., logic circuits, processing circuits, instruction execution devices, etc.). The one or more data processing elements can be included in a central processing unit, an image processing unit, and / or a separate processor or computing element-based controller (e.g., logic gates, etc.). The hardware processor subsystem can include one or more on-board memories (e.g., cache, dedicated memory array, read-only memory, etc.). In some embodiments, the hardware processor subsystem can include one or more memories that can be on-board or off-board or dedicated for use by the hardware processor subsystem (e.g., ROM, RAM, basic input / output system (BIOS), etc.).

[0031] In some embodiments, the hardware processor subsystem may include and execute one or more software elements, which may include an operating system and / or one or more applications and / or specific code to achieve a particular result.

[0032] In other embodiments, the hardware processor subsystem may include dedicated, specialized circuitry that performs one or more electronic processing functions to achieve a specified result. Such circuitry may include one or more application-specific integrated circuits (ASICs), FPGAs, and / or PLAs.

[0033] These and other variations of the hardware processor subsystem are also contemplated in accordance with embodiments of the present invention.

[0034] Figure 2 is a block diagram illustrating an exemplary dual-channel feature extractor 200, according to an embodiment of the present invention. Figure 3 is a block diagram illustrating the dual-channel feature extractor 220 of Figure 2, according to an embodiment of the present invention.

[0035] The dual-channel feature extractor 200 includes a data preprocessor 210, a dual-channel feature extractor 220, a binary code extractor 230, and a binary code database 240. In Figures 2 and 3, "f" denotes a feature vector, "b" denotes a binary code, and "h" denotes the output of a hidden layer.

[0036] The data preprocessor 210 preprocesses the raw data to extract the static state and time series for each time step. Specifically, for T consecutive time points, at each time point t=0, 1,..., L, the data preprocessor 210 extracts the time series segments t from the raw time series.

number

number

[0037] The dual-channel feature extractor 220 jointly encodes the static states and time series preprocessed by the data preprocessor 210 into a compact representation. The dual-channel feature extractor 220 includes a network architecture based on a multi-layer perceptron (MLP) 221 and a recurrent neural network (RNN) 222. The dual-channel feature extractor 220 further includes a binary prediction layer 232 that reshapes the size of the features from the concatenation of the outputs of the MLP 221 and the RNN 222 and predicts the binary code.

[0038] The binary code extractor 230 converts the compact representation encoded by the dual channel feature extractor 220 into a binary code by checking the signs of all entries in the feature vector.

[0039] The binary code database 240 stores all previous binary codes extracted by the binary code extractor 230 .

[0040] In one embodiment, the present invention has three phases: a training phase, a hashing phase, and a search phase. In the training phase, the network parameters of the dual-channel feature extractor 220 are trained on all past time series and their static states. After the training phase, all past time series and static states are encoded into binary codes based on the dual-channel feature extractor 220 trained in the training phase and stored in the binary code database 240. In the search phase, for each input time series and static state, the extracted binary code is compared with all past ones in the binary code database 240 to search for the most relevant time series and static state.

[0041] FIG. 4 is a flow diagram illustrating an exemplary training method 400, according to an embodiment of the present invention.

[0042] At block 410, the data preprocessor 210 extracts multivariate time series segments and corresponding static states from the historical data.

[0043] In block 420, the multivariate time series segments and the corresponding static states are jointly encoded into a compact representation by the dual-channel feature extractor 220. It should be understood that the terms "compact feature" and "compact representation" are used interchangeably herein. A compact feature is a feature with a dimension that is much smaller than the original dimension multiplied by the length of the time series segment.

[0044] In block 430, the encoded representation is evaluated by a supervised metric learning loss.

[0045] In block 440, the network parameters of the dual channel feature extractor 220 are updated.

[0046] Block 450 determines whether a stopping condition has been met. If so, the method ends. If not, the method returns to step 420.

[0047] The training phase further extracts multivariate time series segments (slices of the multivariate time series lasting a time step) and corresponding static states, as per block 410. The multivariate time series segments are extracted from the entire multivariate time series by a sliding window.

[0048] Further with respect to block 420, the time series segments with static states are encoded into a compact representation by the dual channel feature extractor 220.

[0049] Specifically, the static state at time t

number

number

number

number

number

number

number

number

[0050] Further with respect to block 430, the encoded representation is subjected to a supervised metric learning loss, e.g., triplet loss

number

number

number

[0051] Anchor samples are randomly selected from all data segments, positive samples are randomly selected from data samples belonging to the same class as the anchor, and negative samples are randomly selected from data samples belonging to a different class than the anchor.

[0052] Further, with respect to block 440, the model parameters of the dual channel feature extractor 220 are updated to reduce the loss function based on stochastic gradient descent.

[0053] Further, with respect to block 450, if the stopping condition is not met, the training loop is repeated from block 320, and if the stopping condition is met, the training phase ends.

[0054] FIG. 5 is a flow diagram illustrating an exemplary hashing step 500, according to an embodiment of the present invention.

[0055] At block 510, the data preprocessor 210 extracts multivariate time series segments and corresponding static states from the historical data.

[0056] In block 520, the dual-channel feature extractor 220 jointly encodes all of the multivariate time series segments and the corresponding static states into a compact representation.

[0057] In block 530, all compact representations are converted into binary codes by the binary code extractor 230.

[0058] At block 540, the binary code is stored in the binary code database 240.

[0059] Further with respect to block 510, multivariate time series segments and corresponding static states are extracted.

[0060] Further with respect to block 520, all of the multivariate time series segments with corresponding static states are encoded into a compact representation by the dual-channel feature extractor 220 trained in the training phase.

[0061] Further with respect to block 530, all compact representations obtained in block 220 are converted into binary vectors by checking the signs of all entries of the compact representations.

[0062] FIG. 6 is a flow diagram illustrating an exemplary search stage 600 according to an embodiment of the present invention.

[0063] In block 610, the current time series segment with static states is jointly encoded into a compact representation by the dual channel feature extractor 220.

[0064] In block 620, the compact representation is converted into binary code by the binary code extractor 230.

[0065] At block 630, the Hamming (or other) distance between the binary code and all binary codes in the binary code database 240 is calculated.

[0066] At block 640, the top relevant multivariate time series segments are retrieved based on the distance, where the shorter the distance, the more relevant the multivariate time series segment.

[0067] At block 650, actions are performed corresponding to at least the most relevant multivariate time series segments. For example, the most relevant multivariate time series segments may indicate an imminent automobile collision. In such a case, an accident avoidance strategy may be required, including controlling one or more of the automobile's steering, braking, acceleration, stability, lighting, etc.

[0068] Further with respect to block 610, for the currently observed time series segment having a static state, a compact representation (feature) is extracted based on the dual-channel feature extractor 220 learned in the training phase.

[0069] Further with respect to block 620, the compact representation extracted in block 610 is converted into a binary code by checking the signs of all entries of the feature vector that embodies the compact representation.

[0070] Further, with respect to block 630, the Hamming distance between the binary code converted in block 320 and all binary codes in the binary code database 230 is calculated.

[0071] FIG. 7 is a block diagram illustrating an exemplary environment 700 in which the present invention may be applied, according to an embodiment of the present invention.

[0072] In environment 700, a user 788 is located in a scene with multiple objects 799, each with its own position and trajectory. The user 788 is driving a vehicle 772 (e.g., a car, truck, motorcycle, etc.) that has an ADAS 777.

[0073] The ADAS777 receives one or more high-order multivariate time series segments.

[0074] Vehicle control decisions are made in response to one or more of the top multivariate time series segments, such that the ADAS 777 may take action in response to the decisions, such as, but not limited to, controlling the steering, braking, and acceleration systems.

[0075] Thus, in an ADAS context, steering, acceleration / braking, friction (or lack thereof), yaw rate, lighting (hazards, high beam flashing, etc.), tire pressure, turn signaling, etc. can all be efficiently utilized in optimization decisions according to the present invention.

[0076] A system of the present invention (e.g., system 777) may interface with a user through one or more systems of a vehicle 772 that the user is operating. For example, a system of the present invention may provide user information via system 772A of the vehicle 772 (e.g., a display system, a speaker system, and / or some other system). Additionally, a system of the present invention (e.g., system 777) may interface with the vehicle 772 itself (e.g., through one or more systems of the vehicle 772, including but not limited to a steering system, a braking system, an acceleration system, a steering system, a lighting (turn signals, headlights) system, etc.) to control the vehicle and cause the vehicle 772 to perform one or more actions. In this manner, the user or the vehicle 772 itself can navigate around these objects 799 and avoid potential collisions therebetween. Providing information and / or controlling the vehicle may be considered actions determined in accordance with embodiments of the present invention.

[0077] Although described with respect to ADAS, the present invention is applicable to a myriad of applications, including, for example, trajectory, navigation, including autonomous agents, robots, assistive technology for the visually impaired, etc., may be utilized by embodiments of the present invention.

[0078] FIG. 8 is a diagram illustrating an exemplary timeline 801, timeline segments 802, and static states 803, in accordance with an embodiment of the present invention.

[0079] Time series 801 represents the overall time series. Time series segments 802 represent various portions of time series 801. Static state 803 is the state of the system that does not change at the same successive points in time as time series segments 802. The difference between static state 803 and time series 801 is the frequency of change: time series 801 changes from moment to moment, while static state 803 changes only at regular time intervals.

[0080] The present invention may be a system, method, and / or computer program product at any possible level of technical detail of integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to perform aspects of the present invention.

[0081] A computer-readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge structures in grooves having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as ephemeral signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted over wires.

[0082] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or can be downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can be comprised of copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing / processing device.

[0083] Computer-readable program instructions for carrying out operations of the present invention may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or object-oriented programming languages ​​such as SMALLTALK®, C++, or similar programming languages, or conventional procedural programming languages ​​such as the “C” programming language. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), and the connection may be to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can utilize state information of computer-readable program instructions to personalize the electronic circuitry and execute the computer-readable program instructions to carry out aspects of the present invention.

[0084] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0085] These computer-readable program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing device to manufacture a machine such that the instructions, executed by the processor of the computer or other programmable data processing device, create means for performing the functions / acts specified in the block or blocks of the flowcharts and / or block diagrams. These computer-readable program instructions can also be stored on a computer-readable storage medium that can direct a computer, programmable data processing device, and / or other device to function in a particular manner, such that the instructions stored on the computer-readable storage medium constitute an article of manufacture having stored instructions that implement aspects of the functions / acts specified in the block or blocks of the flowcharts and / or block diagrams.

[0086] The computer-readable program instructions may be loaded into a computer, other programmable data processing apparatus, or other device to cause a sequence of operational steps to be executed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, such that the instructions executing on the computer, other programmable apparatus, or other device perform the function / operation specified in the flowchart or block diagram block or blocks.

[0087] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, which contains one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may in fact be executed substantially concurrently, or the blocks may be executed in the reverse order, depending on the functionality involved. It is also noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, can be implemented by a special-purpose hardware-based system that performs the specified functions or operations or executes a combination of special-purpose hardware and computer instructions.

[0088] In the specification, references to "one embodiment" or "one embodiment" of the present invention, as well as other variations thereof, mean that a particular feature, structure, characteristic, etc. described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrase "in one embodiment" or "in one embodiment" in various places throughout this specification, as well as any other variations thereof, are not necessarily all referring to the same embodiment.

[0089] For example, in the case of "A / B," the use of any of the following terms " / ," "and / or," "at least one of," such as "A and / or B" or "at least one of A and B" will be understood to be intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), the selection of only the first and third listed alternatives (A and C), the selection of only the second and third listed alternatives (B and C), or the selection of all three alternatives (A, B, and C). This can be expanded as many times as the number of listed items, as can be readily understood by one of ordinary skill in this and related arts.

[0090] The foregoing is understood in all respects to be illustrative and exemplary, but not restrictive, and the scope of the invention disclosed herein is to be determined not from the detailed description, but from the claims which are interpreted in accordance with the full breadth permitted by the patent laws. It will be understood that the embodiments shown and described herein are merely exemplary of the invention, and that those skilled in the art could make various modifications without departing from the scope and spirit of the invention. Various other feature combinations could be implemented by those skilled in the art without departing from the scope and spirit of the invention. Having thus described aspects of the invention with the detail and particularity required by the patent laws, what is desired to be claimed and protected by Letters Patent is set forth in the appended claims.

Claims

1. 1. A computer-implemented method comprising: Jointly encoding the current time series segment and the corresponding static state into compact features by a dual-channel feature extractor; converting the compact features into a binary code by a binary code extractor; calculating the distance between said binary code and all binary codes stored in a binary code database; and retrieving top relevant multivariate time series segments based on the distances.

2. extracting multivariate time series segments and corresponding static states from historical data by a data preprocessor; jointly encoding the multivariate time series segments and the corresponding static states into compact features with dimensions much smaller than the original dimensions multiplied by the length of the multivariate time series segments by a dual-channel feature extractor; performing evaluation of the jointly encoded compact features by a supervised metric learning loss to provide compact features that preserve local similarity between the multivariate time series segments and the corresponding static states in an input space; updating network parameters of the dual-channel feature extractor based on the evaluation results to reduce a loss of a loss function based on stochastic gradient descent; 10. The computer-implemented method of claim 1, further comprising: repeating the joint encoding, evaluating, and updating until a stopping condition is reached to provide a trained dual-channel feature extractor.

3. converting the compact signature into a binary code; 3. The computer-implemented method of claim 2, further comprising: storing the binary code in a binary code database.

4. The computer-implemented method of claim 2 , further comprising extracting the multivariate time series segments from the overall time series using a sliding window.

5. The computer-implemented method of claim 2 , wherein the evaluation includes identifying and extracting features from anchor samples, positive samples, and negative samples corresponding to the time series segment.

6. 6. The computer-implemented method of claim 5, wherein the anchor sample is randomly selected from the time series segment, the positive sample is randomly selected from data samples belonging to the same class as the anchor sample, and the negative sample is randomly selected from data samples belonging to a different class than the anchor sample.

7. 2. The computer-implemented method of claim 1, wherein the dual-channel feature extractor is composed of a multi-layer perceptron and a recurrent neural network, the outputs of which are coupled to a binary prediction layer by a coupling element.

8. The computer-implemented method of claim 7 , wherein the binary prediction layer reshapes feature sizes of the compact features and predicts binary codes.

9. 8. The computer-implemented method of claim 7, wherein the output of the multilayer perceptron is retrieved from the final hidden representation of the multilayer perceptron.

10. The computer-implemented method of claim 7 , wherein the binary prediction layer is a fully connected layer.

11. A computer program, the computer program comprising: Jointly encoding the current time series segment and the corresponding static state into compact features by a dual-channel feature extractor; converting the compact features into a binary code by a binary code extractor; calculating the distance between said binary code and all binary codes stored in a binary code database; and retrieving top relevant multivariate time series segments based on said distances.

12. The method comprises: extracting multivariate time series segments and corresponding static states from historical data by a data preprocessor; jointly encoding the multivariate time series segments and the corresponding static states by the dual-channel feature extractor into compact features having dimensions much smaller than the original dimensions multiplied by the length of the multivariate time series segments; performing evaluation of the jointly encoded compact features by a supervised metric learning loss to provide compact features that preserve local similarity between the multivariate time series segments and the corresponding static states in an input space; updating network parameters of the dual-channel feature extractor based on the evaluation results to reduce a loss of a loss function based on stochastic gradient descent; 12. The computer program of claim 11, further comprising: repeating the joint encoding, evaluating, and updating until a stopping condition is reached to provide a trained dual-channel feature extractor.

13. The method comprises: converting the compact signature into a binary code; 13. The computer program of claim 12, further comprising: storing the binary code in a binary code database.

14. The method comprises: The computer program of claim 12 , further comprising extracting the multivariate time series segments from the overall time series using a sliding window.

15. The computer program of claim 12 , wherein the evaluation comprises identifying and extracting features from anchor samples, positive samples, and negative samples corresponding to the time series segment.

16. 16. The computer program of claim 15, wherein the anchor sample is randomly selected from the time series segment, the positive sample is randomly selected from data samples belonging to the same class as the anchor sample, and the negative sample is randomly selected from data samples belonging to a different class than the anchor sample.

17. The computer program product of claim 11 , wherein the dual-channel feature extractor is composed of a multi-layer perceptron and a recurrent neural network, the outputs of which are coupled to a binary prediction layer by a coupling element.

18. The computer program product of claim 17 , wherein the binary prediction layer reshapes feature sizes of the compact features and predicts binary codes.

19. 18. The computer program of claim 17, wherein the output of the multilayer perceptron is retrieved from a final hidden representation of the multilayer perceptron.

20. 1. A computer processing system comprising: a memory device for storing program code; a processor unit for executing the program code, the program code comprising: jointly encoding the current time series segment and the corresponding static state into compact features by a dual-channel feature extractor embedded in one or more hardware processors; converting the compact features into binary code by a binary code extractor embedded in the one or more hardware processors; Calculating the distance between the binary code and all binary codes stored in a binary code database; and a computer processing system, program code for searching for top relevant multivariate time series segments based on said distances.

Images