Methods, systems, and readable media for distance measurements for time series

Abstract

A time series distance estimation system can receive two time series and estimate a distance or dissimilarity between the two time series. The system can compute time warp functions of the two time series and perform trend filtering alternately in a multi-level framework to further speed up the computation.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure claims the benefit of priority to U.S. application No. 16 / 952,005, filed on November 18, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates generally to data processing techniques, and more particularly to methods, systems, and readable media for distance measurement of time series. Background Art

[0004] Calculating the difference between two time series has been widely used in many applications involving the calculation of distance between time series, such as time series similarity search, outlier detection, clustering and classification, etc. This calculation of the difference between two time series can also be used in many different technical fields, for example, it can include but is not limited to: speech recognition, speaker recognition, machine learning, signal processing, robotics, economics and finance, bioinformatics.

[0005] However, real-world time series data often contains noise and outliers. This noise and outliers can skew the distances between different time series and may lead to singularity problems where a single point in one time series is mapped to a segment (i.e., multiple points) in another time series. Furthermore, existing methods for mapping corresponding points between two time series (or determining the difference or variance between two time series) have time and space complexities that are at least quadratic in the length of the time series, making them difficult to apply to the analysis of long time series. Furthermore, some existing methods cannot handle the situation where some data or points in one time series are missing when compared to another time series. Consequently, these methods cannot find matching portions in one time series relative to another or fill in missing portions in one time series relative to another. Without addressing these issues, existing methods for mapping two time series (or determining the difference or variance between two time series) will inevitably result in high computational and storage workloads, limiting their application in technical fields such as speech recognition, machine learning, signal processing, and robotics, as described above. Summary of the Invention

[0006] One object of the present disclosure is to provide a method, system, and readable medium for distance measurement of a time series. According to a first aspect of an embodiment of the present disclosure, a method for distance measurement of a time series is provided, the method comprising:

[0007] Detrend the first and second time series;

[0008] iteratively downsampling the first detrended time series and the second detrended time series to obtain a representation of the first detrended time series and a representation of the second detrended time series at multiple levels, respectively;

[0009] iteratively performing a plurality of operations on a respective representation of the first detrended time series and a respective representation of the second detrended time series successively at each of the plurality of levels from a highest level to a lowest level; and

[0010] In returning an estimated distance between the first time series and the second time series, the estimated distance is based at least in part on a lowest level representation of the first detrended time series and a lowest level representation of the second detrended time series.

[0011] In some embodiments, further comprising:

[0012] Before detrending the first time series and the second time series, normalization is performed on the first time series and the second time series.

[0013] In some embodiments, the current level is a highest level, the plurality of operations include prediction and upsampling operations, and performing the prediction and upsampling operations on each level of the plurality of levels from the highest level to the lowest level includes:

[0014] Dynamic time warping is performed on the representation of the first detrended time series and the representation of the second detrended time series at a current level to obtain a warped path between the representation of the first detrended time series and the representation of the second detrended time series.

[0015] In some embodiments, the current level is a level lower than a highest level, the plurality of operations include prediction and upsampling operations, and performing the prediction and upsampling operations on each level of the plurality of levels from the highest level to the lowest level includes:

[0016] upsampling the warped path between the representation of the first detrended time series and the representation of the second detrended time series at a previous level higher than a current level to obtain a new warped path at the current level, and generating a search constraint for the current level; and

[0017] The trend estimate for the previous level is upsampled as the trend estimate for the current level.

[0018] In some embodiments, the plurality of operations further include a time alignment operation, and performing the time alignment operation at each level of the plurality of levels from the highest level to the lowest level includes:

[0019] The new warping path is refined by performing dynamic time warping with search constraints at the current level.

[0020] In some embodiments, the plurality of operations further comprises a time graph detrending operation, and performing the time graph detrending operation on each level of the plurality of levels from the highest level to the lowest level comprises:

[0021] generating a graph using the upsampled trend estimate and the improved regularized path;

[0022] updating the trend estimate at a current level based at least in part on the generated graph; and

[0023] A distance between the representation of the first detrended time series and the representation of the second detrended time series is calculated at a current level.

[0024] In some embodiments, further comprising:

[0025] Missing data segments of the first time series and aligned corresponding data segments of the second time series are removed.

[0026] In some embodiments, the first time series and the second time series include two signals that are acquired during different time periods of an application and are compared to determine whether the two signals are considered to be the same or from the same source.

[0027] In some embodiments, the application includes at least one of speech recognition, speaker recognition, machine learning, signal processing, robotics, and bioinformatics.

[0028] According to a second aspect of an embodiment of the present disclosure, a computer-readable medium for time series distance measurement is provided, wherein the computer-readable medium stores executable instructions. When the executable instructions are executed by one or more processors, the one or more processors perform actions, the actions including:

[0029] Detrend the first and second time series;

[0030] iteratively downsampling the first detrended time series and the second detrended time series to obtain a representation of the first detrended time series and a representation of the second detrended time series at multiple levels, respectively;

[0031] iteratively performing a plurality of operations on a respective representation of the first detrended time series and a respective representation of the second detrended time series successively at each of the plurality of levels from a highest level to a lowest level; and

[0032] In returning an estimated distance between the first time series and the second time series, the estimated distance is based at least in part on a lowest level representation of the first detrended time series and a lowest level representation of the second detrended time series.

[0033] In some embodiments, the actions further include:

[0034] Before detrending the first time series and the second time series, normalization is performed on the first time series and the second time series.

[0035] In some embodiments, the current level is a highest level, the plurality of operations include prediction and upsampling operations, and performing the prediction and upsampling operations on each level of the plurality of levels from the highest level to the lowest level includes:

[0036] Dynamic time warping is performed on the representation of the first detrended time series and the representation of the second detrended time series at a current level to obtain a warped path between the representation of the first detrended time series and the representation of the second detrended time series.

[0037] In some embodiments, the current level is a level lower than a highest level, the plurality of operations include prediction and upsampling operations, and performing the prediction and upsampling operations on each level of the plurality of levels from the highest level to the lowest level includes:

[0038] upsampling the warped path between the representation of the first detrended time series and the representation of the second detrended time series at a previous level higher than a current level to obtain a new warped path at the current level, and generating a search constraint for the current level; and

[0039] The trend estimate for the previous level is upsampled as the trend estimate for the current level.

[0040] In some embodiments, the plurality of operations include a time alignment operation, and performing the time alignment operation at each level of the plurality of levels from a highest level to a lowest level includes:

[0041] The new warping path is refined by performing dynamic time warping with search constraints at the current level.

[0042] In some embodiments, the plurality of operations includes a time graph detrending operation, and performing the time graph detrending operation at each level of the plurality of levels from a highest level to a lowest level includes:

[0043] generating a graph using the upsampled trend estimate and the improved regularized path;

[0044] updating the trend estimate at a current level based at least in part on the generated graph; and

[0045] A distance between the representation of the first detrended time series and the representation of the second detrended time series is calculated at a current level.

[0046] According to a third aspect of an embodiment of the present disclosure, a system for distance measurement of a time series is provided, comprising:

[0047] one or more processors; and

[0048] a memory for storing executable instructions that, when executed by the one or more processors, cause the one or more processors to perform actions, the actions comprising:

[0049] Detrend the first and second time series;

[0050] iteratively downsampling the first detrended time series and the second detrended time series to obtain a representation of the first detrended time series and a representation of the second detrended time series at multiple levels, respectively;

[0051] iteratively performing a plurality of operations on a respective representation of the first detrended time series and a respective representation of the second detrended time series successively at each of the plurality of levels from a highest level to a lowest level; and

[0052] In returning an estimated distance between the first time series and the second time series, the estimated distance is based at least in part on a lowest level representation of the first detrended time series and a lowest level representation of the second detrended time series.

[0053] In some embodiments, the actions further include:

[0054] Before detrending the first time series and the second time series, normalization is performed on the first time series and the second time series.

[0055] In some embodiments, the current level is a highest level, the plurality of operations include prediction and upsampling operations, and performing the prediction and upsampling operations on each level of the plurality of levels from the highest level to the lowest level includes:

[0056] Dynamic time warping is performed on the representation of the first detrended time series and the representation of the second detrended time series at a current level to obtain a warped path between the representation of the first detrended time series and the representation of the second detrended time series.

[0057] In some embodiments, the current level is a level lower than a highest level, and the plurality of operations include prediction and upsampling operations, performing the prediction and upsampling operations on each level of the plurality of levels from the highest level to the lowest level includes:

[0058] upsampling the warped path between the representation of the first detrended time series and the representation of the second detrended time series at a previous level higher than a current level to obtain a new warped path at the current level, and generating a search constraint for the current level; and

[0059] The trend estimate for the previous level is upsampled as the trend estimate for the current level.

[0060] In some embodiments, the plurality of operations include a time alignment operation, and performing the time alignment operation at each level of the plurality of levels from a highest level to a lowest level includes:

[0061] The new warping path is refined by performing dynamic time warping with search constraints at the current level.

[0062] Through the above scheme, it is possible to calculate the time warping functions of two time series and perform trend filtering alternately in a multi-level framework to speed up the calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The detailed description is set forth with reference to the accompanying drawings. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in different figures indicates similar or identical items.

[0064] Figure 1 An example environment is shown in which a time series distance estimation system may be used.

[0065] Figure 2 A more detailed example time series distance estimation system is shown.

[0066] Figure 3 Showing an exemplary graph structure between two time series.

[0067] Figure 4 Shows an example approach for handling missing values ​​in time series in a multilevel framework.

[0068] Figure 5 Shows an example method for time series distance estimation. Specific embodiments

[0069] As mentioned above, existing methods for mapping two time series (or determining the difference or variance between two time series) have many technical problems or shortcomings, such as distance deviation due to the presence of noise and outliers, relatively high time and space complexity, missing data problems, etc., which limit their application in technical fields such as speech recognition, speaker recognition, machine learning, signal processing, robotics, economics and finance, bioinformatics, etc.

[0070] The present disclosure describes an exemplary time series distance estimation system. In some embodiments, the time series distance estimation system can perform filtering on a multivariate time series, such as trend filtering (which may include but is not limited to general time graph trend filtering, etc.) to estimate a real signal (or time series). In some embodiments, the time series distance estimation system can estimate or calculate a time warp function and perform trend filtering simultaneously or alternately. In some embodiments, the time series distance estimation system can learn a time warp function and perform trend filtering simultaneously or alternately in a multi-level framework to further speed up the calculation.

[0071] In some embodiments, a time series distance estimation system can receive two time series, and the two time series can include two signals that are obtained over different time periods of an application and are compared to determine whether the two signals are considered to be the same or from the same source. In some embodiments, the time series distance estimation system can recursively perform downsampling by a predetermined factor to obtain different levels of representation of the time series. In some embodiments, starting from the highest level of representation (i.e., lowest resolution), the time series distance estimation system can estimate or calculate a time warping function for the two time series. Then, based on the estimated or calculated time warping function, the time series distance estimation system can construct a weighted graph to map or combine the two time series in pairs.

[0072] In some embodiments, after constructing a weighted graph for mapping or pairwise combining two time series, the time series distance estimation system can perform time graph trend filtering using the weighted graph to update the denoised estimate.

[0073] In some embodiments, based on the time warping function calculated and obtained in the representation of the previous level, the time series distance estimation system can iteratively or recursively calculate or estimate the time warping function in the representation of the next level, thereby reducing computational complexity and improving computational speed. Based on the newly calculated time warping function, the time series distance estimation system can construct a new weighted graph to map or pairwise combine the two time series of the representation of the next level, and use the newly constructed weighted graph to perform time graph trend filtering to update the denoising estimate of the representation of the next level. The time series distance estimation system can iteratively or recursively perform the above operations until the lowest level of representation (i.e., the original highest resolution).

[0074] In some embodiments, the time series distance estimation system can further employ a unified framework within the multi-level framework to handle missing data points in the time series. In some embodiments, when calculating or estimating the time warping function, the time series distance estimation system can adaptively exclude missing blocks (i.e., missing data points) in one time series and corresponding aligned counter blocks in another time series.

[0075] After performing the above operations, the time series distance estimation system can obtain the final time warping function of the two time series at the lowest level of representation (i.e., the original highest resolution). The time series distance estimation system can then estimate or calculate the distance or dissimilarity between the two time series. Depending on the type of the two sequences (e.g., speech signals, robot motion, etc.), the time series distance estimation system or another system (to which the calculated distance or dissimilarity between the two time series is sent) can perform subsequent operations for the intended application (e.g., speech recognition, robot error detection, etc.).

[0076] In some embodiments, the functions performed by the time series distance estimation system described herein can be performed by multiple separate units or services. For example, a receiving service can receive data for a time series, and a downsampling service can perform downsampling to obtain representations of different levels of the time series. In addition, a computing service can generate a time warping function for two time series at a particular level, and a construction service can construct a weighted graph to map or pairwise combine the two time series based on the time warping function computed at that particular level. In addition, a filtering service can perform time graph trend filtering using the weighted graph to update the denoised estimate.

[0077] Furthermore, although in the examples described herein the time series distance estimation system may be implemented as a combination of software and hardware implemented and distributed in multiple devices, in other examples the time series distance estimation system may be implemented and distributed as a service provided over a network and / or in one or more computing nodes in a cloud computing architecture.

[0078] This application describes multiple, different embodiments and implementations. The following sections describe example frameworks suitable for practicing various implementations. Next, this application describes example systems, devices, and processes for implementing a time series distance estimation system.

[0079] Sample Environment

[0080] Figure 1 An example environment 100 is shown that can be used to implement a time series distance estimation system. Environment 100 can include a time series distance estimation system 102. In this example, time series distance estimation system 102 is described as existing as a single entity or device. In some examples, some or all of the functionality of time series distance estimation system 102 can be included in or provided by multiple computing nodes 104-1, 104-2, ..., 104-N (collectively referred to as computing nodes 104), which are connected and communicate via a network 106, where N is a positive integer. In other examples, time series distance estimation system 102 can communicate data with one or more computing nodes 104 via network 106.

[0081] In some embodiments, multiple computing nodes 104 may form a computer system 108 (e.g., a cloud computing architecture or system), or may form part of a computer system 108. In some embodiments, the computer system 108 may provide various services to multiple client devices (for simplicity, the computer system 108 may be described in detail in the following sections). Figure 1 Only one client device 110 is shown. In this example, the time series distance estimation system 102 may be depicted as part of the computer system 108. In other examples, the time series distance estimation system 102 may be a separate entity that provides support to the computer system 108 (e.g., provides time series distance estimation).

[0082] In some embodiments, each of the computing nodes 104 can be implemented as any of a variety of devices having computing capabilities, and can include, but is not limited to: a processor (which may include a single-core processor or a multi-core processor), a desktop computer, a notebook or portable computer, a handheld device, a netbook, an Internet appliance, a tablet or slate computer, a mobile device (e.g., a mobile phone, a personal digital assistant, a smart phone, etc.), a server computer, etc., or a combination thereof.

[0083] The network 106 may be a wireless or wired network, or a combination thereof. The network 106 may be a collection of independent networks that are interconnected and function as a single large network (e.g., the Internet or an intranet). Examples of such independent networks include, but are not limited to, telephone networks, cable networks, local area networks (LANs), wide area networks (WANs), and metropolitan area networks (MANs). Furthermore, the independent networks may be wireless or wired networks, or a combination thereof. A wired network may include electrical carrier connections (e.g., communication cables, etc.) and / or optical carriers or connections (e.g., fiber optic connections, etc.). A wireless network may include, for example, a WiFi network, other radio frequency networks (e.g., Bluetooth Zigbee, etc.

[0084] In some embodiments, the time series distance estimation system 102 may receive two time series (i.e., corresponding data segments of the two time series) from the client device 110 of the user 112 or the computing node 104 of the computer system 108. In some embodiments, each time series may include data obtained or sampled over a period of time, and the type of data for each time series depends on the type of application for which the time series is used or obtained. By way of example and not limitation, for speech recognition, the two time series may include sound signals collected or captured by a sound receiving device (e.g., a microphone) for the same set of phrases or the same sentence, and the time series distance estimation system 102 may calculate or estimate the distance or dissimilarity between the two time series (i.e., sound signals) to determine whether the sound signals are emitted by the same user or by different users.

[0085] Example time series distance estimation system

[0086] Figure 2 1 shows a more detailed time series distance estimation system 102. In some embodiments, the time series distance estimation system 102 may include, but is not limited to, one or more processors 202, a memory 204, an input / output (I / O) interface 206, and / or a network interface 208. In some embodiments, some functions or components of the time series distance estimation system 102 (e.g., one or more processors 202) may be implemented using hardware, such as an ASIC (i.e., an application-specific integrated circuit), an FPGA (i.e., a field-programmable gate array), and / or other hardware. In this example, the time series distance estimation system 102 may exist as a separate entity that may or may not be associated with a device such as a computing node 104. In some examples, some functions of the time series distance estimation system 102 may be included in a device such as a computing node 104.

[0087] In some embodiments, the processor 202 may be configured to execute instructions stored in the memory 204 and / or instructions received from the input / output interface 206 and / or from the network interface 208. In some embodiments, the processor 202 may be implemented as one or more hardware processors, including, for example, a microprocessor, a special instruction set processor, a physical processing unit (PPU), a central processing unit (CPU), a graphics processing unit, a digital signal processor, a tensor processing unit, etc. Additionally or alternatively, the functions described herein may be performed at least in part by one or more hardware logic components. For example, illustrative types of hardware logic components that may be used include, but are not limited to, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc.

[0088] Memory 204 may include computer-readable media in the form of volatile memory, such as random access memory (RAM), and / or nonvolatile memory, such as read-only memory (ROM), or flash RAM. Memory 204 is an example of a computer-readable medium.

[0089] Computer readable media may include volatile or non-volatile types, removable or non-removable media, which may use any method or technology to implement information storage. Information may include computer-readable instructions, data structures, program modules, or other data. Examples of computer readable media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other internal storage technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical memory, cassette tape, magnetic disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include any transient media, such as modulated data signals and carrier waves.

[0090] Although exemplary hardware components are described in the time series distance estimation system 102 in this example, in other examples, the time series distance estimation system 102 may further include other hardware components and / or other software components, such as a program unit 210 that executes instructions stored in the memory 204 for performing various operations. In some embodiments, the time series distance estimation system 102 may also include program data 212 that stores data for performing time series distance estimation.

[0091] Example Robust Dynamic Time Warping Method

[0092] In order to achieve robustness to noise and outliers while maintaining the flexibility of dynamic index matching or alignment, the time series distance estimation system 102 can use filtering, such as trend filtering, to effectively filter out noise and outliers included in the time series being compared. In some embodiments, the time series distance estimation system 102 can alternately estimate the time warping function and detrend the time series. In some embodiments, dynamic index matching or alignment may include determining an index of a time series x (e.g., x i The index of the data point of ) and the index of another time series y (for example, as y j The mapping between the indices of the data points.

[0093] In some embodiments, the time series distance estimation system 102 may iteratively or recursively perform operations including, but not limited to: (1) maintaining a time warping function (e.g., (1) estimating a detrended time series (e.g., the time series represented by u and v) while keeping the estimated detrended time series u and v fixed; and (2) estimating a time warping function based on the time series u and v while keeping the estimated detrended time series u and v fixed. Table 1 below shows the process of this alternating method, the steps of which are described in further detail below.

[0094] Table 1

[0095]

[0096]

[0097] Robust Self-Detrending

[0098] In some embodiments, before performing time warping, the time series distance estimation system 102 may perform robust trend filtering on each time series to be compared to remove the effects of outliers and noise. In some embodiments, if a time series of length n (i.e., n time points) is represented as y = [y1, y2, ..., y n ] T , then the time series distance estimation system 102 can decompose the time series into a trend component and a residual component as shown below:

[0099] y t =τ t +r t Or y = τ + r (1)

[0100] Where τ=[τ1,τ2,…,τ n ] T, which indicates the trend component, and r=[r1,r2,…,r n ] T , which indicates the residual component.

[0101] In some embodiments, the time series distance estimation system 102 can apply a detrending filter to detrend the time series. As an example and not a limitation, for illustrative purposes, an example detrending filter that can capture slow and sudden trend changes while being robust to outliers is used herein. However, other detrending filters can also be used, as long as they can detect or estimate (and therefore filter) one or more trends included in the time series. In some embodiments, the example detrending filter can extract trends by minimizing the following objective function, which is:

[0102] g γ (y-τ)+λ1||D (1) τ||1+λ2||D (2) τ||2 (2)

[0103] Among them, g γ (y-τ)=∑ i g γ (x i ), which is the sum of the element-wise Huber loss functions, each element of which is:

[0104]

[0105] and, and are the first-order difference matrix and the second-order difference matrix, respectively, and are expressed as follows:

[0106]

[0107] In some embodiments, the use of Huber loss makes the example detrending filter more robust to outliers in the time series compared to the sum of squares loss (although the sum of squares loss can also be used instead of Huber loss for simple calculations), while the first-order and second-order difference operators in the regularization term can capture sudden and slow trend changes. After performing the above detrending operation, the two input time series x and y may be written as:

[0108]

[0109] Time Warping Alignment

[0110] In some embodiments, after performing a self-detrending step or operation on the time series, the time series distance estimation system 102 may apply a dynamic time warping method (e.g., FastDTW) to the detrended time series to obtain a time warping function (which may correspond to an index mapping from one time series x (an index of a time series x) to another time series y (an index of a time series y)), or a time warping path (which is a set of matrix elements that define an index mapping, and is used to determine how to stretch two time series x and y under certain constraints, and for measuring the degree of difference between the two time series x and y after time warping alignment). In some embodiments, the dynamic time warping method may include any conventional or future dynamic time warping method that can determine a time warping function or an index mapping between two time series (such as time series x and time series y in this example). Examples of dynamic time warping methods may include, but are not limited to, the FastDTW method. In some embodiments, these operations or steps of time warping alignment can be extended in a multi-level manner for efficient computation and missing data operations, which will be described in detail in later sections.

[0111] Temporal Graph Detrending

[0112] In some embodiments, in order to more accurately estimate the detrended time series u and v, when performing the detrending operation on the time series u and v, the time series distance estimation system 102 may consider not only the data points in one time series but also the data points in the other time series. For example, in the graph detrending operation, if G = (V, E) is a graph with vertices V = {1, ..., n} and undirected edges E = {e1, ..., e s}, y=[y1,…,y n ] T ∈R n Across each node, the k-order graph trend filter estimates τ = [τ1,…,τ n ] T can be obtained by solving:

[0113]

[0114] Where Δ(k+1) is the k+1-order graph difference operator. When k=0, the corresponding first-order graph difference operator Δ(1) can make all local differences on all edges disadvantageous, as shown below:

[0115] ||Δ (k+1) τ||1=∑ (i,j)∈E |τ i -τ j | (5)

[0116] In some embodiments, Δ(1) may be expressed in matrix form as Δ (1) ∈{-1,0,1} s×n , s = |E|, i.e., the number of edges. In some embodiments, if Then Δ(1) has the OK:

[0117]

[0118] in, It has -1 at the i-th position and 1 at the j-th position.

[0119] In some embodiments, similar to trend filtering for a univariate time series, a high-order graph difference operator can be recursively defined. For example, the graph difference operator defined above can be simplified to a graph difference operator defined on a univariate time series, where V = {1, 2, ..., n} and E {(i, i+1): i = 1, 2, ..., n-1}.

[0120] In some embodiments, a general time graph detrending method for multivariate time series can be designed by extending the graph-based detrending method in Equation (4). In some embodiments, the relationship between time series can be incorporated during detrending, which can also adaptively handle lagging effects. In addition, weights can be introduced for each edge in the graph (i.e., binary weights (0 or 1), or decimal weights with any value in the range [0, 1]).

[0121] In some embodiments, the time series distance estimation system 102 can construct a graph G of two time series x and y, wherein the two time series x and y are compared based on the time warping alignment in step 2. As an example and not a limitation, the constructed graph is described as having m+n vertices, each vertex corresponding to x∈R m and y∈R n For simplicity, in this example, the vertex set is represented as E = {x1, x2, ..., x m ,y1,y2,…,y n In some embodiments, each vertex x t (a time point in time series x) can be connected to its left neighbor x t-1 and right neighbor x t+1 (They are neighboring time points in the time series x). In addition, vertex x t It can also connect to its peer vertex (the peer time point in the time series y, ).

[0122] In some embodiments, in order to avoid errors that may be introduced in the time warping alignment of step 2, the time series distance estimation system 102 may construct additional or extra edges to improve robustness. For example, given that there is an edge in the graph G The time series distance estimation system 102 may add a predetermined number (2, 4, 6, 10, etc.) of additional edges. In some embodiments, the predetermined number (or x) of additional edges t The size of the surrounding neighborhood may depend on the noise and outliers included or expected in the time series. Figure 3 An exemplary graph structure 300 between two time series x and y is shown. In this example, Figure 3 As shown, the time series distance estimation system 102 can add 4 additional edges, namely and Each data point in a time series can be connected not only to its neighbors in the same time series, but also to its peers in the other time series. Through the dynamic time warping operation in step 2, the time series x and y are aligned. Figure 3 As shown, the timestamps on the time series x are t-1, t, and t+1. is an index mapping from time series x to time series y. In this case, the data points with timestamps t-1, t, and t+1 on time series x are mapped to the data points with timestamps t-1, t, and t+1 on time series y. The data points are aligned or corresponded.

[0123] In some embodiments, the time series distance estimation system 102 may assign weights to the edges of the constructed graph G. In some embodiments, the time series distance estimation system 102 may assign different weights to the edges of the constructed graph G based at least in part on the type and / or length of the edge. By way of example and not limitation, the time series distance estimation system 102 may assign higher weights to edges connecting neighboring vertices on the same time series than to edges connecting peer vertices on different time series. In addition, the weights of edges between peer vertices on different time series decrease as the length of the edge between the peer vertices increases (i.e., the peer vertices move further and further away from the DTW alignment).

[0124] In some embodiments, after constructing a graph G for a time series (e.g., time series x and y), the time series distance estimation system 102 can calculate a general time graph detrending based on the following formula:

[0125]

[0126] Where w = [u; v] is the concatenation of inputs u and v. and They are respectively the first-order graph difference operator and the second-order graph difference operator defined on a constructed or weighted graph G for capturing sudden or slow trend changes.

[0127] Iterative processing

[0128] In some embodiments, to achieve better performance, the time series distance estimation system 102 may repeat steps 2 and 3 of the alternating method as described in Table 1 to update the time warping function and the trend estimate until convergence or a predetermined number of iterations (e.g., three iterations, four iterations, ..., ten iterations, etc.) is reached. In some embodiments, after convergence or reaching the predetermined number of iterations, the two time series x and y and their alignment may achieve a desired compromise, which may not only maintain a close proximity to the original time series (i.e., a time series not contaminated by noise and outliers), but also have a final alignment with a lower DTW distance.

[0129] Robust dynamic time warping in a multi-level framework

[0130] In some embodiments, the time series distance estimation system 102 can utilize the similarity of the shape and alignment of the time series between different resolutions, and perform the adjustment of the DTW alignment and the determination of the trend estimate alternately from low resolution to high resolution to further speed up the calculation speed of the robust dynamic time warping method. In some embodiments, the time series distance estimation system 102 can use the results obtained at a lower resolution as a starting value to perform an estimation for a higher resolution, so that the DTW alignment results obtained at a lower resolution can be used as a constraint to limit the path search space for a higher resolution. Table 2 below shows detailed information on the robust DTW method of the multi-level framework (or simply referred to as the multi-level robust DTW method).

[0131] Table 2

[0132]

[0133]

[0134] In some embodiments, as shown in Table 2, the multi-level robust DTW method may include multiple steps, including but not limited to: (1) robust self-detrending; (2) multi-level representation; (3) prediction and upsampling; (4) time-warping alignment using DTW; (5) time map detrending; (6) iterative processing; and (7) normalized distance output.

[0135] In some embodiments, in step (1) (i.e., the robust self-detrending step), the time series distance estimation system 102 may perform detrending on the two time series at the highest resolution (i.e., the original resolution without downsampling). In some embodiments, in step (2) (i.e., the multi-level representation step), the time series distance estimation system 102 may obtain a multi-level representation of the time series by downsampling the detrended time series obtained in step (1) (i.e., the robust self-detrending step) by a predetermined factor. For example, the detrended time series at the original highest resolution corresponds to a level 1 representation, and may be downsampled by a predetermined factor (e.g., 2 in this example) to generate a level 2 representation. The time series distance estimation system 102 may continue to perform downsampling until n is obtained. t Level represents, where n t is the total number of different levels in the multi-level framework (or the number of iterations in Table 2).

[0136] In some embodiments, in step (3) (ie, the step of prediction and upsampling), if the current level is the highest level (ie, the nth level), t level), the time series distance estimation system 102 can perform a DTW method (e.g., FastDTW) on the time series to obtain a warped index alignment and use the current level representation as the basis for trend estimation. t For a current level (level 1) of the time series distance estimation system 102, the time series distance estimation system 102 may upsample the regularized path obtained for the previous level (i.e., level (1+1)) by a predetermined factor (i.e., 2 in this example) and add an additional search width defined by a parameter (i.e., a radius as shown in Table 2) to generate a search constraint for the current level called a projection. In some embodiments, the time series distance estimation system 102 may further upsample the trend estimate obtained at the previous level (i.e., level (1+1)) for the current level.

[0137] In some embodiments, in step (4) (i.e., the step of performing time warping alignment by DTW), the time series distance estimation system 102 may improve the warped path obtained in the previous step of prediction and upsampling by performing the DTW method at the current level using the predicted warping constraints (i.e., the search constraints generated in the previous step of prediction and upsampling). In some embodiments, in step (5) (i.e., the step of detrending the time graph), the time series distance estimation system 102 may generate a graph using the upsampled trend estimate obtained in the prediction and upsampling step and the improved DTW alignment obtained in step 4 (i.e., the step of performing time warping alignment by DTW). In some embodiments, the time series distance estimation system 102 may further update the trend estimate based at least in part on the generated graph and calculate the distance between the two time series at the current level.

[0138] In some embodiments, in step (6) (i.e., the iterative processing step), the time series distance estimation system 102 may repeat steps (3)-(5) (i.e., the steps of prediction and upsampling, the step of time warping alignment by DTW, and the step of detrending the time graph). t times, and decreasing the level with each iteration until the lowest level representation (i.e., the original highest resolution) is reached.

[0139] Missing data handling

[0140] In some embodiments, the time series distance estimation system 102 may remove all missing values ​​(or data points) having the same time range during the DTW calculation. However, a data segment including missing values ​​(or data points) in one time series may correspond to another data segment in another time series. Figure 4 An example method 400 for handling missing values ​​in a time series in a multi-level framework is shown. Figure 4 In the figure, the solid lines represent the corresponding data segments of the two time series x and y at a higher resolution; the dashed lines represent the corresponding data segments of the two time series x and y at a lower resolution. In this example, two data points are missing from the time series y at a higher resolution and are marked with a cross. If these two data points were not missing, the best alignment learned from DTW could align these two data points to their corresponding data points on the time series x at a higher resolution, as given by Figure 4 In some embodiments, it is desirable to exclude these missing data points in a time series (e.g. Figure 4 The time series x at medium and high resolution ), which is estimated to align with NaNs (i.e., “Not-a-Number,” or undefined or unrepresented values) in another time series (i.e., the time series y at the higher resolution).

[0141] In some embodiments, the time series distance estimation system 102 may recursively downsample the two time series by a predetermined number (e.g., 2) and calculate the average of a predetermined number of adjacent or neighboring data points (in this example, two adjacent or neighboring data points) to determine the distance or similarity between the two time series until the missing data blocks disappear (or no data points are missing). As an example and not a limitation, if both adjacent or neighboring data points have values, the time series distance estimation system 102 may calculate the average of the predetermined number of adjacent or neighboring data points by calculating the arithmetic mean of the two adjacent or neighboring data points. Alternatively, if one of the adjacent or neighboring data points has a value val and the other data points have no value (i.e., a missing value), the time series distance estimation system 102 may use val as the average of the two adjacent or neighboring data points. Alternatively, if both adjacent or neighboring data points have no value (i.e., both have missing values), the time series distance estimation system 102 may use NaN as the average of the two adjacent or neighboring data points.

[0142] In some embodiments, at a lower resolution without missing values, the time series distance estimation system 102 can calculate DTW and obtain alignment. In some embodiments, to estimate alignment at a higher resolution, the time series distance estimation system 102 can consider the boundaries of values ​​and NaNs. For example, in Figure 4 In the example shown, at lower resolutions, and is described as a pair of alignment points. In this example, There is value, is NaN, and and are described as having values. In this case, the time series distance estimation system 102 can be performed by finding and The alignment between two corresponding data points in the time series x and y is heuristically estimated by minimizing the value of . In this example, if is smaller, the time series distance estimation system 102 can use higher resolution to and Alignment. Following this rule, the time series distance estimation system 102 can estimate the alignment in the boundaries of NaN blocks at a higher resolution, and thus can remove missing data segments of one time series and their aligned counterparts on the other time series. The time series distance estimation system 102 can apply this process to multiple missing data segments or value blocks on two time series.

[0143] Example Method

[0144] Figure 5 Schematic diagram showing an example method for time series distance estimation. Figure 5 The method can but does not have to be Figure 1 In an environment and using Figure 2 For the sake of convenience, refer to Figure 1 and Figure 2 Method 500 is described. However, method 500 may alternatively be implemented in other environments and / or using other systems.

[0145] Method 500 is described in the general context of computer-executable instructions. Generally, computer-executable instructions can include routines, programs, objects, components, data structures, processes, modules, functions, etc. that perform specific functions or implement specific abstract data types. In addition, each example method is illustrated as a collection of blocks in a logical flow diagram, which represents a sequence of operations that can be implemented in hardware, software, firmware, or a combination thereof. The order in which the method is described is not intended to be interpreted as a limitation, and any number of the described method blocks can be combined in any order to implement the method or an alternative method. In addition, individual blocks can be omitted from the method without departing from the spirit and scope of the subject matter described herein. In the context of software, a block represents a computer instruction that, when executed by one or more processors, performs the described operation. In the context of hardware, some or all of the blocks can represent an application-specific integrated circuit (ASIC) or other physical component that performs the described operation.

[0146] Return Reference Figure 5 At block 502 , the time series distance estimation system 102 may receive a plurality of time series to be compared.

[0147] In some embodiments, the time series distance estimation system 102 may receive a plurality of time series to be compared, such as a first time series and a second time series. The first time series and the second time series may include or correspond to two signals, which are obtained during different time periods of an application and are compared to determine whether the two signals are considered to be the same or from the same source. In some embodiments, examples of applications may include, but are not limited to, speech recognition, speaker recognition, machine learning, signal processing, robotics, and bioinformatics. For example, the first time series and the second time series may include or correspond to two sound signals, which are obtained during different time periods of an application and are compared to determine whether the two signals are spoken by the same speaker (i.e., from the same source).

[0148] At block 504 , the time series distance estimation system 102 may normalize and detrend the first time series and the second time series in the plurality of time series.

[0149] In some embodiments, the time series distance estimation system 102 may perform normalization and detrending on the first time series and the second time series in the plurality of time series to obtain a first detrended time series and a second detrended time series. In some embodiments, the time series distance estimation system 102 may perform trend filtering (e.g., applying a detrending filter with a Huber loss function) to remove corresponding trend components and noise components from the first time series and the second time series.

[0150] At block 506 , the time series distance estimation system 102 may iteratively downsample the first detrended time series and the second detrended time series to obtain a representation of multiple levels of the first detrended time series and a representation of multiple levels of the second detrended time series, respectively.

[0151] In some embodiments, the time series distance estimation system 102 can recursively perform downsampling by a predetermined factor (e.g., 2, etc.) to obtain representations of different levels for each time series of the first detrended time series and representations of different levels for each time series of the second detrended time series. In some embodiments, the time series distance estimation system 102 can predetermine a number of levels (i.e., the number of downsampling applications) for the first detrended time series column and the second detrended time series. In some embodiments, the time series distance estimation system 102 can predetermine the number of levels based on, for example, the original lengths of the first detrended time series column and the second detrended time series. By way of example and not limitation, the number of levels can be greater when the original lengths of the first detrended time series and the second detrended time series are longer.

[0152] At block 508 , the time series distance estimation system 102 may iteratively perform prediction and upsampling operations, time warping alignment operations, and time graph detrending operations on corresponding representations of the first detrended time series and corresponding representations of the second detrended time series continuously at each level in the plurality of levels from the highest level to the lowest level.

[0153] In some embodiments, after downsampling the first detrended time series and the second detrended time series to obtain representations of the first detrended time series at multiple levels and representations of the second detrended time series at multiple levels, the time series distance estimation system 102 may iteratively perform a plurality of different operations on the corresponding representations of the first detrended time series and the corresponding representations of the second detrended time series at each level of the plurality of levels, from the highest level to the lowest level. In some embodiments, the plurality of operations may include, but are not limited to, forecasting and upsampling operations, time warping and alignment operations, and time graph detrending operations.

[0154] For example, for a current level among the multiple levels, the time series distance estimation system 102 can continuously perform different operations (e.g., prediction and upsampling operations, time warping alignment operations, and time graph detrending operations) on the corresponding representation of the first detrended time series and the corresponding representation of the second detrended time series at the current level.

[0155] In some embodiments, the time series distance estimation system 102 may perform prediction and upsampling operations on the representation of the first detrended time series and the representation of the second detrended time series at the current level. For example, if the current level is the highest level, the time series distance estimation system 102 may perform dynamic time warping on the representation of the first detrended time series and the representation of the second detrended time series at the current level to obtain a warped path between the representation of the first detrended time series and the representation of the second detrended time series.

[0156] In some embodiments, if the current level is a level lower than the highest level, the time series distance estimation system 102 may perform upsampling on the warped path between the representation of the first detrended time series and the representation of the second detrended time series at a previous level higher than the current level to obtain a new warped path for the current level and generate a search constraint for the current level. Alternatively, the time series distance estimation system 102 may perform upsampling on the trend estimate for the previous level as the trend estimate for the current level.

[0157] In some embodiments, after performing the prediction and upsampling operations at the current level, the time series distance estimation system 102 can perform a time warping alignment operation. By way of example and not limitation, the time series distance estimation system 102 can refine the new warping path by performing dynamic time warping with search constraints on the current level.

[0158] In some embodiments, after performing the time warping alignment operation at the current level, the time series distance estimation system 102 can perform a time graph detrending operation. In some embodiments, the time series distance estimation system 102 can generate a graph using the upsampled trend estimate and the improved warping path. Additionally, the time series distance estimation system 102 can update the trend estimate at the current level based at least in part on the generated graph and calculate a distance between a representation of the first detrended time series and a representation of the second detrended time series at the current level.

[0159] In some embodiments, the time series distance estimation system 102 can further perform a missing data processing operation on the first detrended time series and the second detrended time series. In some embodiments, the time series distance estimation system 102 can remove missing data segments of the first time series and aligned corresponding data segments of the second time series, or vice versa.

[0160] At block 510 , the time series distance estimation system 102 may base the estimated distance between the first and second time series at least in part on the lowest level representation of the first and second detrended time series.

[0161] In some embodiments, after iteratively performing a plurality of different operations on the corresponding representations of the first detrended time series and the corresponding representations of the second detrended time series at each level from the highest level to the lowest level in the plurality of levels, the time series distance estimation system 102 may return the final distance between the representations of the first detrended time series and the representations of the second detrended time series obtained at the lowest level as the estimated distance or dissimilarity between the first time series and the second time series. In some embodiments, the time series distance estimation system 102 may also send the estimated distance or dissimilarity to an application that obtains and uses the first time series and the second time series (such as speech recognition, speaker recognition, machine learning, signal processing, robotics, and bioinformatics), so that the application can perform subsequent processing based at least in part on the estimated distance or dissimilarity.

[0162] Since the robust dynamic time warping method has been described in detail in the aforementioned description, for additional details of the operations performed by the method 500 described herein (such as prediction and upsampling operations, time warping alignment operations, time graph detrending operations, missing data processing operations, etc.), reference may be made to the aforementioned description.

[0163] Although the above method blocks are described as being performed in a particular order, in some embodiments, some or all of the method blocks may be performed in other orders, or in parallel.

[0164] Summarize

[0165] Although the embodiments have been described using language specific to structural features and / or methodological acts, it should be understood that the claims are not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claimed subject matter. Additionally or alternatively, some or all operations may be implemented by one or more ASICs, FPGAs, or other hardware.

Claims

1. A method for distance measurement of a time series, the method comprising: detrending a first time series and a second time series, wherein the first time series and the second time series comprise two signals, the two signals being obtained during different time periods of an application and being compared to determine whether the two signals are considered to be the same or originate from the same source, the two time series comprise sound signals collected or captured by a sound receiving device for the same set of phrases or the same sentence, and the time series distance estimation system calculates or estimates a distance or dissimilarity between the two time series to determine whether the sound signals are emitted by the same user or by different users; iteratively downsampling the first detrended time series and the second detrended time series to obtain a representation of the first detrended time series and a representation of the second detrended time series at a plurality of levels, respectively; iteratively performing a plurality of operations on the respective representations of the first detrended time series and the respective representations of the second detrended time series successively at each of the plurality of levels from a highest level to a lowest level, the plurality of operations comprising a forecasting and upsampling operation, a time warping alignment operation, and a time map detrending operation; and In returning an estimated distance between the first time series and the second time series, the estimated distance is based at least in part on a lowest level representation of the first detrended time series and a lowest level representation of the second detrended time series.

2. The method according to claim 1, further comprising: Before detrending the first time series and the second time series, normalization is performed on the first time series and the second time series.

3. The method according to claim 1, wherein The current level is a highest level, and performing the prediction and upsampling operations on each level from the highest level to the lowest level in the multiple levels includes: Dynamic time warping is performed on the representation of the first detrended time series and the representation of the second detrended time series at a current level to obtain a warped path between the representation of the first detrended time series and the representation of the second detrended time series.

4. The method according to claim 1, wherein The current level is a level lower than a highest level, and performing the prediction and upsampling operations on each level from the highest level to the lowest level in the plurality of levels includes: upsampling the warped path between the representation of the first detrended time series and the representation of the second detrended time series at a previous level higher than a current level to obtain a new warped path at the current level, and generating a search constraint for the current level; and The trend estimate for the previous level is upsampled as the trend estimate for the current level.

5. The method according to claim 4, wherein Performing the time alignment operation at each level from the highest level to the lowest level in the plurality of levels includes: The new warping path is refined by performing dynamic time warping with search constraints at the current level.

6. The method according to claim 5, wherein: Performing the time chart detrending operation at each of the plurality of levels from a highest level to a lowest level includes: generating a graph using the upsampled trend estimate and the improved regularized path; updating the trend estimate at a current level based at least in part on the generated graph; and A distance between the representation of the first detrended time series and the representation of the second detrended time series is calculated at a current level.

7. The method according to claim 1, further comprising: Missing data segments of the first time series and aligned corresponding data segments of the second time series are removed.

8. The method according to claim 1, wherein The application includes at least one of speech recognition, speaker recognition, machine learning, signal processing, robotics, and bioinformatics.

9. A computer-readable medium for distance measurement of a time series, the computer-readable medium storing executable instructions that, when executed by one or more processors, cause the one or more processors to perform actions comprising: detrending a first time series and a second time series, wherein the first time series and the second time series comprise two signals, the two signals being obtained during different time periods of an application and being compared to determine whether the two signals are considered to be the same or originate from the same source, the two time series comprise sound signals collected or captured by a sound receiving device for the same set of phrases or the same sentence, and the time series distance estimation system calculates or estimates a distance or dissimilarity between the two time series to determine whether the sound signals are emitted by the same user or by different users; iteratively downsampling the first detrended time series and the second detrended time series to obtain a representation of the first detrended time series and a representation of the second detrended time series at a plurality of levels, respectively; iteratively performing a plurality of operations on the respective representations of the first detrended time series and the respective representations of the second detrended time series successively at each of the plurality of levels from a highest level to a lowest level, the plurality of operations comprising a forecasting and upsampling operation, a time warping alignment operation, and a time map detrending operation; and In returning an estimated distance between the first time series and the second time series, the estimated distance is based at least in part on a lowest level representation of the first detrended time series and a lowest level representation of the second detrended time series.

10. The computer-readable medium of claim 9, the actions further comprising: Before detrending the first time series and the second time series, normalization is performed on the first time series and the second time series.

11. The computer-readable medium of claim 9, wherein: The current level is a highest level, and performing the prediction and upsampling operations on each level from the highest level to the lowest level in the multiple levels includes: Dynamic time warping is performed on the representation of the first detrended time series and the representation of the second detrended time series at a current level to obtain a warped path between the representation of the first detrended time series and the representation of the second detrended time series.

12. A system for distance measurement of a time series, comprising: one or more processors; as well as a memory for storing executable instructions that, when executed by the one or more processors, cause the one or more processors to perform actions, the actions comprising: detrending a first time series and a second time series, wherein the first time series and the second time series comprise two signals, the two signals being obtained during different time periods of an application and being compared to determine whether the two signals are considered to be the same or originate from the same source, the two time series comprise sound signals collected or captured by a sound receiving device for the same set of phrases or the same sentence, and the time series distance estimation system calculates or estimates a distance or dissimilarity between the two time series to determine whether the sound signals are emitted by the same user or by different users; iteratively downsampling the first detrended time series and the second detrended time series to obtain a representation of the first detrended time series and a representation of the second detrended time series at a plurality of levels, respectively; iteratively performing a plurality of operations on the respective representations of the first detrended time series and the respective representations of the second detrended time series successively at each of the plurality of levels from a highest level to a lowest level, the plurality of operations comprising a forecasting and upsampling operation, a time warping alignment operation, and a time map detrending operation; and In returning an estimated distance between the first time series and the second time series, the estimated distance is based at least in part on a lowest level representation of the first detrended time series and a lowest level representation of the second detrended time series.

Images