Multi-element dynamic time bending distance measurement learning method and system

Abstract

The invention discloses a multivariate dynamic time bending distance measurement learning method and system, and the method comprises the steps: firstly, carrying out the calculation of a multivariate weighted dynamic time bending distance, taking a weighted Euclidean distance as a point pair distance, and calculating a dynamic time bending distance for a multivariate time sequence; then, through point pair distance weight learning, the dynamic time bending distance is introduced into a large-interval nearest neighbor classification method to construct an objective function, and non-convex optimization is achieved through a coordinate descent method; according to the method, the non-convex optimization problem can be effectively solved, the multivariate dynamic time bending measurement accuracy is remarkably improved, the learning efficiency is high, and the method can play an important role in the fields of big data, data mining, artificial intelligence, databases and the like.

Description

Technical Field

[0001] The present invention belongs to the fields of big data, database, data mining, etc., and specifically relates to a multivariate dynamic time warp distance metric learning method and system. Background Art

[0002] Time series are widely present in people's daily life and industrial production, such as real-time transaction data of funds or stocks, daily sales data of retail markets, sensor monitoring data of process industries, astronomical observation data, aerospace radar, satellite monitoring data, real-time weather temperature and air quality index, etc. The industry has proposed many time series analysis methods, including similarity query methods, classification methods, clustering methods, prediction methods, anomaly detection methods, etc. Among them, many methods require distance measurement of time series, such as kNN classifier, k-means clustering method, etc. Therefore, time series distance measurement methods have a wide range of application needs in the industry.

[0003] The most commonly used time series distance measurement methods can be divided into lockstep measurement methods and elastic measurement methods. The former uses a point-to-point measurement method, that is, the distance between two time series is obtained by strictly comparing the distance between the sampling values ​​of the two at each moment and then accumulating them. The most common ones are Manhattan distance, Euclidean distance and Chebyshev distance, which are all L p A special case of norm distance when p takes different values. This type of method has the advantages of easy implementation, low computational complexity, satisfying the distance triangle inequality, and no parameters. However, the measurement accuracy is very sensitive to factors such as noise, amplitude scaling and drift, phase shift, and can only be used to measure equal time series. The elastic measurement method adopts a point-to-multipoint measurement method, that is, a sampling value of a time series can correspond to multiple sampling values ​​of another series, and all sampling values ​​of the two series are traversed through the dynamic programming method. The most common methods of this type are dynamic time warping distance and edit distance. Compared with the lock-step measurement, the elastic measurement can achieve the best matching alignment of the sampling values ​​of the two time series, can effectively handle basic morphological changes such as time warping, phase shift, amplitude scaling and drift, and is robust to noise and outliers, so it has higher measurement accuracy. Summary of the invention

[0004] In order to efficiently learn multivariate dynamic time warping distance, the present invention adopts the following technical solutions:

[0005] A multivariate dynamic time warping distance metric learning method comprises the following steps:

[0006] (1) Calculation of multivariate weighted dynamic time warping distance: Obtain a pair of multidimensional environmental monitoring data as a multivariate time series, use weighted Euclidean distance as a measurement method, calculate the point-to-point distance matrix of the corresponding elements of the two multivariate time series, and use the dynamic programming algorithm to find the optimal curved path on the matrix so that the cumulative sum of the element values ​​on the path is minimized. The cumulative sum of the path is used as the final distance measurement value to calculate the dynamic time warping distance;

[0007] (2) Point-to-point distance weight learning: Using the multivariate weighted dynamic time warping distance as the kernel distance metric, the objective function is constructed based on the large-margin nearest neighbor classification method. The coordinate descent method is used to iteratively optimize the optimal curving path and point-to-point distance weight parameters until the objective function converges. The trained weight parameters are used to perform a similarity comparison of one-to-multidimensional environmental monitoring data.

[0008] Furthermore, the step (2) of learning the point-to-point distance weights includes the following sub-steps:

[0009] (2.1) Realign the multivariate time series according to the optimal curved path and calculate the weighted Euclidean distance between the two based on the corresponding elements;

[0010] (2.2) Substitute the weighted Euclidean distance into the loss function of the large-margin nearest neighbor classification method to obtain the objective function and its constraints;

[0011] (2.3) Keeping the optimal curved path unchanged, the objective function is optimized using the gradient descent method until convergence;

[0012] (2.4) Keep the weight coefficient unchanged and find the optimal curved path between multivariate time series again;

[0013] (2.5) Repeat steps (2.3) to (2.4) until the optimal curved path no longer changes and the objective function converges.

[0014] Furthermore, in step (1), the elements q1, q2, ..., q m As the horizontal axis, the elements k1, k2, ..., k of the multivariate time series K of the second multidimensional environmental monitoring data are n As the vertical axis, a point-to-point distance matrix is ​​constructed, and each unit in the path is calculated using the weighted Euclidean distance, that is, WED(q1,k1)=w1(a1-b1) 2 +w2(a2-b2) 2 +…+w p (a p -b p ) 2 , where a1, a2, …, a pRepresents the environmental data of different dimensions of sequence element q1, b1, b2, ..., b p Represents the environmental data of different dimensions of sequence element k1, w1, w2, ..., w p Indicates the corresponding weight coefficient; in the point-to-point distance matrix, the first cell is used as the starting point, the last cell of the diagonal is used as the ending point, the optimal curved path is found through the dynamic programming algorithm, and the accumulated value of the path cells is used as the dynamic time bending distance;

[0015] Furthermore, in step (2), the multivariate time series is realigned according to the optimal curved path, as follows:

[0016]

[0017] According to the matching element index of the optimal curved path, the elements q and k of the two time series are matched one by one, and the weighted Euclidean distance between them is calculated.

[0018] Furthermore, in the step (2.1), the multivariate dynamic time warping distance is converted into a calculation form of the L2 norm:

[0019]

[0020] Where X and Y represent the given multivariate time series, and Respectively represent the sequence and the elements in the sequence realigned according to the optimal curved path of X and Y, w i represents the weighted Euclidean distance weight coefficient, ||·|| 2 represents L2 norm calculation, d(·) represents the distance function, and L represents the number of corresponding elements.

[0021] Furthermore, in step (2.1), formula (1) is substituted into the loss function of the large-margin nearest neighbor classification method to obtain the objective function (2) and its constraints:

[0022]

[0023]

[0024] in, represents a data set containing n samples, represents the i-th multivariate time series sample, represents its category label, Φ i is the sample x i The target neighbor set, W represents the set of all weight coefficients, M and N are the samples x i The nearest neighbor with different label x lmThe nearest neighbor with the same label as x z The length of the realignment sequence is and Respectively represent x i The corresponding realignment element The sth and rth dimension values ​​of Represents x lm The corresponding realignment element The sth dimension value of Realignment element The rth dimension value of , w represents the weight vector, w s and w r Represent the sth and rth weight coefficients respectively.

[0025] A multivariate dynamic time warping distance metric learning system is applied to the comparison of environmental monitoring data, comprising: a multivariate weighted dynamic time warping distance generation module and a point pair distance weight learning module, which are respectively used for the multivariate weighted dynamic time warping distance calculation and the point pair distance weight learning.

[0026] The advantages and beneficial effects of the present invention are:

[0027] 1. The weight coefficient is introduced into the traditional multivariate dynamic time warping distance, which significantly improves the data adaptability of the distance measurement, effectively distinguishes the importance of different dimensions of the multivariate time series, and then constrains the contribution of each dimension in the distance measurement, thereby improving the measurement accuracy;

[0028] 2. The learning method based on large-margin nearest neighbors has lower computational complexity and stronger interpretability, and is more resistant to noise samples;

[0029] 3. Through iterative optimization of the optimal curved path and weight coefficients, the optimization problem of non-convex objective functions is effectively solved, and the efficiency of distance metric learning is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flow chart of a method in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0032] like Figure 1As shown in FIG, a multivariate dynamic time-warping distance metric learning method is used to convert the daily environmental data (including temperature, air pressure, wind speed, and light intensity) collected by a certain IoT environmental monitoring system within a year into the same order of magnitude after data normalization to achieve distance metric learning. Specifically, the method includes the following steps:

[0033] (1) Multivariate weighted dynamic time warping distance calculation: Using weighted Euclidean distance as the measurement method, the point-to-point distance matrix of the multivariate time series is calculated. The dynamic programming algorithm is used to find the optimal curved path on the matrix so that the cumulative sum of the element values ​​on the path is minimized. The cumulative sum of the path is used as the final distance measurement value to calculate the dynamic time warping distance; the details are as follows:

[0034] Take the elements q1,q2,…,q of the first multivariate time series Q m As the horizontal axis, the elements k1, k2, ..., k of the second multivariate time series K n As the vertical axis, a point-to-point distance matrix is ​​constructed. Each cell in the path is calculated using the weighted Euclidean distance to obtain WED(q1,k1)=w1(a1-b1) 2 +w2(a2-b2) 2 +…+w p (a p -b p ) 2 , where a1, a2, …, a p Represents sensor data of different dimensions of sequence element q1, b1, b2, …, b p Represents sensor data of different dimensions of sequence element k1, w1,w2,…,w p Represents the corresponding weight coefficient, and the weight at this time is the initial value;

[0035] In the point-to-point distance matrix, the first cell is used as the starting point and the last cell of the diagonal is used as the ending point. The optimal curved path is found through the dynamic programming algorithm, and the accumulated value of the path cells is used as the dynamic time bending distance.

[0036] (2) Point-to-point distance weight learning: Using multivariate weighted dynamic time warping distance as the kernel distance metric, constructing the objective function based on the large-margin nearest neighbor classification method, and iteratively optimizing the optimal curving path and point-to-point distance weight parameters using the coordinate descent method until the objective function converges; specifically, the following sub-steps are included:

[0037] (2.1) Realign the multivariate time series according to the optimal curved path, as follows:

[0038]

[0039] Based on the curved path, the corresponding elements q and k on the horizontal and vertical axes are matched, and the weighted Euclidean distance between the two is calculated based on the corresponding elements;

[0040] Convert the multivariate dynamic time warping distance into the calculation form of L2 norm:

[0041]

[0042] Where X and Y represent the given multivariate time series, and Respectively represent the sequence and the elements in the sequence realigned according to the optimal curved path of X and Y, w i represents the weighted Euclidean distance weight coefficient, ||·|| 2 represents L2 norm calculation, d(·) represents the distance function, and L represents the number of corresponding elements.

[0043] In the embodiment of the present invention, assuming that the environmental monitoring sequence X = {(3.5, 4.0, 3.2, 3.7), (3.6, 4.5, 3.1, 4.3), (4.2, 4.2, 3.6, 3.9), ..., (3.0, 3.4, 2.9, 3.1)} and Y = {(1.3, 1.5, 1.3, 1.4), (1.2, 1.8, 1.2, 1.3), (2.0, 2.3, 1.8, 2.0), ..., (2.4, 2.7, 2.2, 2.5)}, the optimal curved path is {(1, 1), (1, 2), (2, 2), (2, 3), (3, 4), ...}, realign X and Y to obtain

[0044] and Then the dynamic time warping distance between X and Y can be converted into the L2 norm distance for calculation according to formula (1).

[0045] (2.2) Substitute the L2 norm calculation formula (1) of X and Y into the loss function of the large margin nearest neighbor classification method to obtain the objective function (2) with the weight coefficient as the variable and its constraints:

[0046]

[0047]

[0048] in, represents a data set containing n samples, Represents a sample, represents the label, Φ i is the sample x i The target neighbor set, W represents the set of all weight coefficients;

[0049] (2.3) Keeping the optimal curved path unchanged, the objective function (2) is optimized using the gradient descent method until convergence;

[0050] (2.4) Keep the weight coefficient W unchanged and find the optimal curved path between multivariate time series again;

[0051] (2.5) Repeat steps (2.3) to (2.4) until the optimal curved path no longer changes and formula (2) converges.

[0052] In addition to environmental monitoring data, multivariate time series data also includes retail market sales data, industrial process sensor monitoring data, astronomical observation data, aerospace radar data, etc. In addition, the present invention can also be generalized to distance measurement application scenarios of other types of data.

[0053] Aiming at the problem of multivariate time series distance metric learning, the present invention proposes a multivariate dynamic time warping distance learning method. By introducing weight coefficients in the point pair distance, the importance of each dimension of the multivariate time series can be effectively distinguished and constrained, and the metric accuracy of the multivariate dynamic time warping distance can be significantly improved. At the same time, based on large-margin nearest neighbor learning and coordinate descent methods, non-convex optimization problems can be effectively solved, and the learning efficiency is relatively high. The present invention can be widely used in time series data mining tasks in industrial production and daily life.

[0054] A multivariate dynamic time warping distance metric learning system is applied to the comparison of environmental monitoring data, comprising: a multivariate weighted dynamic time warping distance generation module and a point-to-point distance weight learning module, which are used for multivariate weighted dynamic time warping distance calculation and point-to-point distance weight learning respectively.

[0055] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multivariate dynamic time warping distance metric learning method, characterized in that The steps include: (1) Calculation of multivariate weighted dynamic time warping distance: Obtain a pair of multidimensional environmental monitoring data as a multivariate time series, use weighted Euclidean distance as a measurement method, calculate the point-to-point distance matrix of the corresponding elements of the two multivariate time series, and use the dynamic programming algorithm to find the optimal curved path on the matrix so that the cumulative sum of the element values ​​on the path is minimized. The cumulative sum of the path is used as the final distance measurement value to calculate the dynamic time warping distance; (2) Point-to-point distance weight learning; The multivariate weighted dynamic time warp distance is used as the kernel distance metric, and the objective function is constructed based on the large-margin nearest neighbor classification method. The coordinate descent method is used to iteratively optimize the optimal curvature path and the point pair distance weight parameters until the objective function converges. The trained weight parameters are used to perform similarity comparison of one-to-multidimensional environmental monitoring data.

2. A multivariate dynamic time warping distance metric learning method according to claim 1, characterized in that: The step (2) of point pair distance weight learning includes the following sub-steps: (2.1) Realign the multivariate time series according to the optimal curved path and calculate the weighted Euclidean distance between the two based on the corresponding elements; (2.2) Substitute the weighted Euclidean distance into the loss function of the large-margin nearest neighbor classification method to obtain the objective function and its constraints; (2.3) Keeping the optimal curved path unchanged, the objective function is optimized using the gradient descent method until convergence; (2.4) Keep the weights unchanged and re-find the optimal curved path between multivariate time series; (2.5) Repeat steps (2.3) to (2.4) until the optimal curved path no longer changes and the objective function converges.

3. A multivariate dynamic time warping distance metric learning method according to claim 1 or 2, characterized in that: In the step (1), the elements q1, q2, ..., q m As the horizontal axis, the elements k1, k2, ..., k of the multivariate time series K of the second multidimensional environmental monitoring data are n As the vertical axis, a point-to-point distance matrix is ​​constructed. Each cell in the path is calculated using the weighted Euclidean distance to obtain WED(q1,k1)=w1(a1-b1) 2 +w2(a2-b2) 2 +…+w p (a p -b p ) 2 , where a1, a2, …, a p Represents the environmental data of different dimensions of sequence element q1, b1, b2, ..., b p Represents the environmental data of different dimensions of sequence element k1, w1, w2, ..., w p Indicates the corresponding weight; in the point-to-point distance matrix, the first cell is taken as the starting point, the last cell of the diagonal is taken as the ending point, the optimal curved path is found through the dynamic programming algorithm, and the accumulated value of the path cells is taken as the dynamic time bending distance.

4. The multivariate dynamic time warping distance metric learning method according to claim 3, characterized in that: In step (2), the multivariate time series is realigned according to the optimal curved path, as follows: According to the matching element index of the optimal curved path, the elements q and k of the two time series are matched one by one, and the weighted Euclidean distance between them is calculated.

5. The multivariate dynamic time warping distance metric learning method according to claim 1, characterized in that: In the step (2.1), the multivariate dynamic time warping distance is converted into the calculation form of the L2 norm: Where X and Y represent the given multivariate time series, and Respectively represent the sequence and the elements in the sequence realigned according to the optimal curved path of X and Y, w i represents the weighted Euclidean distance weight coefficient, ||·|| 2 represents L2 norm calculation, d(·) represents the distance function, and L represents the number of corresponding elements.

6. A multivariate dynamic time warping distance metric learning method according to claim 5, characterized in that: In the step (2.1), the multivariate dynamic time warping distance is substituted into the loss function of the large-margin nearest neighbor classification method to obtain the objective function and its constraints: in, represents a data set containing n samples, represents the i-th multivariate time series sample, represents its category label, Φ i is the sample x i The target neighbor set, W represents the set of all weight coefficients, M and N are the samples x i The nearest neighbor with different label x lm The nearest neighbor with the same label as x z The length of the realignment sequence is and Respectively represent x i The corresponding realignment element The sth and rth dimension values ​​of Represents x lm The corresponding realignment element The sth dimension value of Realignment element The rth dimension value of , w represents the weight vector, w s and w r represent the sth and rth weight coefficients respectively.

7. A system according to claim 1 for learning a multivariate dynamic time warping distance metric, applied to comparison of environmental monitoring data, characterized in that: include: The multivariate weighted dynamic time warping distance generation module and the point pair distance weight learning module are used for the multivariate weighted dynamic time warping distance calculation and the point pair distance weight learning respectively.

Images