Time series data compression and reconstruction method based on graph signal processing

Abstract

The present invention discloses a time series data compression and reconstruction method based on graph signal processing. The existing methods have poor migration capabilities and do not consider scenarios with limited computing resources. In the method of the present invention, the computing center uses the characteristics of small differences between adjacent data points in the time series data to construct a low-frequency graph signal, and sends the graph structure to the sensor node; the sensor node calculates the frequency domain value of the data sequence according to the graph structure, and selects to transmit the frequency domain data of the low-frequency part to achieve data compression; when the ratio of the frequency domain signal energy after compression to the total energy of the current data sequence is lower than the threshold, the sensor determines that the current graph structure needs to be updated, and retransmits the complete data to the computing center. The computing center completes the update of the graph structure based on the new data to adapt to environmental changes. The method of the present invention uses the low-frequency characteristics of the graph signal to achieve efficient compression of time series data. The algorithm complexity is low and it is suitable for sensor nodes. At the same time, the rapid update of the graph structure is used to ensure its adaptability in different environments.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, in particular to the field of data processing technology, and specifically relates to a time series data compression and reconstruction method based on graph signal processing. Background Art

[0002] With the rapid development of communication and information processing technology, the demand for information and the amount of data to be processed have also increased dramatically. The data of various applications are mainly obtained through sensors in edge systems. Long-term and large-scale data collection will bring huge challenges to the storage, communication and computing resources of edge data collection systems. Considering the different requirements for data freshness when used for subsequent decision-making and application, data can be divided into data with high freshness requirements and data with low freshness requirements. For data with low freshness requirements, in order to reduce energy consumption, sensors often obtain a batch of data before transmitting it to the data center. The same batch of data obtained over a long period of time has time redundancy. In recent years, with the development of cloud computing and fog computing, sensors as edge nodes have participated in edge computing. Sensor nodes need to add edge intelligent processing modules, that is, to give sensor nodes computing capabilities. Therefore, data can be pre-processed at the sensor to reduce the amount of data transmission, thereby reducing communication overhead.

[0003] The existing time series data compression methods mainly include compressed sensing, function approximation, data compression methods based on autoencoders, and serialization algorithms. The compressed sensing method needs to assume that the data is compressible, and the noise reduction performance depends on the prior information of the noise. In addition, the computational cost and design cost are high, and the compression rate has room for improvement. The main idea of ​​the function approximation method is that the time series can be expressed as a function of time. Since there are new values ​​that cannot be processed, it is impossible to find a function that approximates the entire time series. Therefore, the time series can be divided into multiple segments, and an approximation function is found for each segment. It is not feasible to explore all possible functions, so only one function family is considered in the implementation, and an attempt is made to find parameters that are closer to each segment, which makes the compression lossy. The advantage of this algorithm is that it does not depend on the data domain, so no training phase is required, and the regression algorithm only considers isolated single segments. Considering that the serialization algorithm for the IoT application scenario is to sequentially combine multiple simple compression methods to achieve compression of time series data, the algorithm is simple to implement, but the compression effect needs to be improved. In recent years, with the development of machine learning, some scholars have applied autoencoders to data compression and reconstruction, which can achieve a high compression rate, but the migration ability is poor.

[0004] The existing time series data compression methods based on serialization algorithms, such as the Chinese invention patent application with application number 202210913155.9, disclose a method and device for compressing and storing time series data of industrial equipment. The limitation of this method is that it uses a coding compression method based on information entropy, and the compression performance stability is poor.

[0005] There are many deficiencies in the existing technology. First, most algorithms do not consider scenarios with limited computing resources, and the algorithm calculation complexity is too high, resulting in poor practicality. Second, when the algorithm application scenario changes, such as the data collection environment, the algorithm needs to be reset, which takes a long time to adapt to new data and the algorithm migration capability is poor. In response to the above problems, the present invention aims to design a time series data compression and reconstruction method that responds to changes in collection scenarios and combines limited computing resources. Summary of the invention

[0006] The purpose of the present invention is to provide a time series data compression and reconstruction method based on graph signal processing. On the data acquisition side (i.e., sensor or edge computing node), the time redundancy characteristics of the perception data obtained by the fixed position sensor are used to achieve data compression and reduce the amount of data transmission; on the data collection side (i.e., computing center), the perception data is reconstructed according to the received compressed data for subsequent information processing. Compared with the existing time series data compression and reconstruction method based on serialization algorithm, the time series data compression and reconstruction method provided achieves a high compression rate while ensuring the low complexity and migration capability of the method.

[0007] The data collection system of the present invention is composed of two layers: a computing center and an edge node. The edge node is a sensor that can sense data and has a certain computing capability, and can achieve low computing cost processing. The present invention focuses on data compression of sensor nodes and data reconstruction of the computing center.

[0008] The method of the present invention specifically comprises:

[0009] Step (1) The sensor generates a sensor data sequence x1=[x 1,1 ,…,x 1,i ,…,x 1,I ] T , and send it to the computing center; where x 1,i represents the perceived data value of the i-th time series data point, i = 1, 2, ..., I, I represents the length of the data time series sequence, [·] T Represents a transpose operation.

[0010] Step (2) After the computing center receives the perception data sequence x1, it constructs the data graph represents the node set of the data graph, and ε represents the edge set of the data graph; the computing center calculates the Laplacian matrix and feature information of the data graph; the specific process is as follows:

[0011] (2-1) The time series data points in the perception data sequence are abstracted as nodes in the data graph, and the perception data value is used as the node value. Calculate the relative difference between any two data points in the data sequence to determine whether there is a connecting edge between the two data points: the relative difference between data point i and data point j Among them, max(x1) and min(x1) represent the maximum and minimum values ​​of the perception data sequence x1 respectively; when δ i,j When it is less than the set threshold γ, it means that there is a connecting edge between data point i and data point j, otherwise there is no connecting edge; for a data point that has no connecting edge with any other data point in the perception data sequence x1, a connecting edge is constructed between the data point and its adjacent previous data point to ensure that each data point in the perception data sequence x1 has at least one connecting edge.

[0012] (2-2) When there is an edge connecting data point i and data point j, the weight of the edge is 1, that is, w i,j =1, otherwise w i,j =0; Data graph The weight matrix Computational Graph The combined Laplace matrix L = DW; where the data graph The weighted degree matrix D is the weighted degree of each node {d1, d2, …, d I} is a diagonal matrix with diagonal elements, and the weighted degree of data point i

[0013] (2-3) Computing center data graph The combined Laplace matrix L is decomposed into features, that is, L = UΛU T , the superscript T indicates the transpose, and Λ is the eigenvalue {λ1,…,λ k ,…,λ I} is a diagonal matrix with diagonal elements, λ k represents the kth Laplace matrix eigenvalue, that is, the data graph kth frequency; the calculation center stores and updates the Fourier basis matrix U = [u1, ..., u k ,…,u I ],u k Represents the eigenvector corresponding to the kth Laplace matrix eigenvalue, that is, the graph Fourier basis, k = 1, 2, …, I.

[0014] Step (3) The computing center sends the graph Fourier basis matrix U to the sensor node.

[0015] Step (4) The sensor node receives the graph Fourier basis matrix U from the computing center, and updates and stores the received graph Fourier basis matrix U.

[0016] Every time the sensor senses I data, it regards it as a data sequence; for each data sequence, execute steps (5) to (8).

[0017] Step (5) The sensor node calculates the frequency domain value of the subsequent sensing data sequence to be transmitted.

[0018] According to the perception data sequence x m With the stored graph Fourier basis matrix U, calculate the perception data sequence x m The frequency domain components of the signal s m =[s m，1 ,…,s m，i ,…,s m，I ] T ,s m =U T x m ,s m，i is the i-th frequency component of the m-th perception data sequence.

[0019] Step (6) The sensor node performs frequency domain compression of the data sequence.

[0020] The sensor node first calculates the perception data sequence x m Energy spectrum of |s m | 2 Then select the first n components with large energy in the frequency domain components of the image signal, n = I × r, r represents the data compression rate. The energy of n components N represents a set of n frequency components. Thus, the perception data sequence x is obtained. m The total energy E of the signal in the frequency domain.

[0021] The sensor node calculates the ratio E of the energy of n frequency components to the total energy of all frequency components n / E; if the ratio E n / E≥η, then these frequency component sets N plus the identifier b are the data sequence x m After the compression, c={b,N}, execute step (7) and set the threshold η∈[0,1]; if the ratio E n / E<η, then determine the data sequence x m After the existing data graph is converted to the frequency domain, the frequency domain components are scattered and the data compression effect is poor. The computing center and the sensor need to update the data graph. The sensor will sense the data sequence x m Transmitted to the computing center, execute steps (2) to (4), that is, use the perception data sequence x m Construct a new data graph.

[0022] The identifier b is used to indicate which frequency component the currently compressed data represents, and is obtained using a bit compression algorithm, as follows:

[0023] First, set the initial value of b to 0, and then perform the following operation on all frequency components: If the i-th frequency component s i ∈N, then b=b|1<<(i-1), otherwise no operation is performed.

[0024] Step (7) The sensor node transmits the data c to the computing center.

[0025] Step (8) The computing center recovers the data sequence perceived by the sensor based on the compressed data c received. For each data sequence, the computing center first calculates the frequency domain component estimate based on the identifier b and the compressed frequency component set N. Then, according to the inverse Fourier transform of the graph, the reconstructed data sequence is calculated

[0026] Calculate the center of the frequency domain component estimate The specific method is as follows:

[0027] Set frequency domain component estimates The initial value is 0. For i=1, 2, ..., I, if b&1<<(i-1)=1, it means that the frequency domain data received by the computing center contains the i-th frequency component, then is equal to the first frequency component in set N, and then the first frequency component in set N is removed from set N, otherwise no operation is performed; the operation stops when set N is an empty set.

[0028] Compared with the existing time series data compression and reconstruction methods, the method of the present invention has the following beneficial effects:

[0029] (1) The present invention considers that the computing center is responsible for implementing the data graph construction algorithm and data reconstruction algorithm with high computational complexity. The computing center uses the small difference between adjacent data points in the time series data to construct a graph signal with low-frequency characteristics. The sensor transforms the graph signal in the frequency domain and then transmits the frequency domain data of the low-frequency part, which reduces the amount of data transmitted by the sensor and realizes data compression with low computational complexity. Compared with the existing time series data compression method, the method designed by the present invention improves the compression efficiency of time series data under the condition of lower computational complexity.

[0030] (2) The present invention takes into account that when the environment where the sensor is located undergoes significant changes, the data sequence can be retransmitted through the sensor, and the computing center reconstructs the data graph to adapt to the environmental changes. Good compression performance can still be guaranteed when the external environment changes. That is, the method designed by the present invention has better migration capabilities and can adapt to environmental changes in a shorter time. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a flow chart of the method of the present invention;

[0032] Figure 2 A schematic diagram of time series data sensed by a sensor in an embodiment;

[0033] Figure 3 A diagram showing the connection relationship between data points in the embodiment;

[0034] Figure 4 It is a schematic diagram of frequency domain components in the embodiment;

[0035] Figure 5 This is a comparison chart between the reconstructed data results and the original data in the embodiment. DETAILED DESCRIPTION

[0036] The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0037] This embodiment takes the temperature data sensed by a sensor fixed in shallow water as an example to illustrate a time series data compression and reconstruction method proposed by the present invention. The specific process is as follows: Figure 1 shown.

[0038] Step (1) The sensor generates a sensor data sequence x1=[x 1，1 , …, x 1，i , …, x 1，I ] T , and send it to the computing center; where x 1,i represents the perceived data value of the i-th time series data point, i = 1, 2, ..., I, I represents the length of the data time series sequence, [·] T In this embodiment, the data sensed by the sensor node is as follows: Figure 2 As shown in the figure, there are 100 data points in total, of which the first 50 data points represent the first data sequence and will be used to construct the initial data graph, and the last 50 data points represent the second data sequence. The first data sequence sensed by the sensor node is x1=[x 1，1 , x 1，2 , …, x 1，50 ] T , in this embodiment, I=50.

[0039] Step (2) After the computing center receives the sensor data sequence x1, it constructs a data graph, which is expressed as represents the node set of the graph, and ε represents the edge set of the graph; the computing center calculates the Laplacian matrix and feature information of the constructed graph. The specific process is as follows:

[0040] (2-1) That is, calculate the relative difference between data point i and data point j Among them, max(x1) and min(x1) represent the maximum and minimum values ​​of the perception data sequence x1 respectively. i,j When γ is less than the set threshold value, a connection edge is constructed between data point i and data point j, otherwise there is no connection edge. In addition, if a data point in the perception data sequence x1 does not have a connection edge with other data points, the current data point and its adjacent previous node are connected to ensure that each node in the perception data sequence x1 has at least one connection edge. This embodiment considers γ = 0.1, and constructs the connection relationship between nodes as follows Figure 3 shown.

[0041] (2-2) When there is an edge connecting data point i and data point j, the weight of the edge is 1, that is, w i,j =1, otherwise w i,j =0;Fig. The weight matrix picture The weighted degree matrix D is the weighted degree of each node {d1, d2, ..., d I} is a diagonal matrix with diagonal elements, and the weighted degree of data point i Final calculation graph The combined Laplace matrix L = DW.

[0042] (2-3) By combining the eigendecomposition of the Laplace matrix L, that is, L = UΛU T , Λ is the eigenvalue {λ1,…,λ k , ..., λ 50} is a diagonal matrix with diagonal elements, λ k represents the kth Laplace matrix eigenvalue, that is, the kth frequency of the data graph, U = [u1, ..., u k ,…,u 50 ],u k Represents the eigenvector corresponding to the k-th Laplace matrix eigenvalue, that is, the graph Fourier basis, k = 1, 2, …, 50.

[0043] Step (3) The computing center sends the graph Fourier basis matrix U to the sensor node.

[0044] Step (4) The sensor node receives the graph Fourier basis matrix U from the computing center, and updates and stores the received graph Fourier basis matrix U.

[0045] Every time the sensor senses I data, it regards it as a data sequence; for each data sequence, execute steps (5) to (8). (In this embodiment, there are only two sequences, so only the second sequence is compressed, that is, only the second sequence is executed. Steps (5) to (8)).

[0046] Step (5) The sensor calculates the frequency domain value of the subsequent sensing data sequence to be transmitted.

[0047] According to the perception data sequence x2=[x 2，1 , x 2，2 , ..., x 2，50 ] T ( Figure 2 The last 50 data) and the stored Fourier basis matrix U of the image are used to calculate the image signal frequency domain component s2 of the perception data sequence x2 = [s 2，1 ,s 2，2 , ..., s 2，50 ] T , s2=U T x2. The frequency domain component of the signal is shown in the figure below: Figure 4 shown.

[0048] Step (6) The sensor performs frequency domain compression of the data sequence.

[0049] The sensor node first calculates the energy spectrum of the sensed data sequence x2 |s2| 2 Then select the first n=I×r=10 components with large energy in the frequency domain components of the image signal. In this embodiment, the compression rate is r=0.2. These n frequency domain components are Figure 4 The frequency components of the 1st, 2nd, 3rd, 6th, 7th, 9th, 18th, 40th, 47th, and 50th frequencies, that is, N = {s 2，1 ,s 2，2 ,s 2，3 ,s 2，6 ,s 2，7 ,s 2，9 ,s 2，18 ,s 2，40 ,s 2，47 ,s 2，50}, the energy of these n components is Therefore, the total energy of the image signal in the frequency domain of the perception data sequence x2 is E=500.

[0050] The sensor node calculates the ratio E of the energy of n frequency components to the total energy of all frequency components N / E. Ratio E N / E≥η, in this embodiment, η=0.95, so these frequency component sets N plus identifier b are selected as the compressed data c={b, N} of the data sequence x2, and step (7) is executed.

[0051] The identifier b is used to indicate which frequency component the currently compressed data represents, and is obtained using a bit compression algorithm, as follows:

[0052] First, b is set to an initial value of 0. Then the following operation is performed on all frequency components. If the i-th frequency component si ∈N, i=1, 2, …, I, then b=b|1<<(i-1), otherwise no operation is performed. We obtain b=10010000001000000000000000000000100000000101100111.

[0053] Step (7) The sensor node transmits the data c to the computing center.

[0054] Step (8) After receiving data c, the computing center reconstructs the data sequence.

[0055] The computing center recovers the data sequence perceived by the sensor based on the received compressed data c. For each data sequence (there are only two sequences in the embodiment, so the second data sequence is reconstructed), the computing center first calculates the frequency domain component estimation value based on the identifier b and the compressed frequency component set N Then, according to the inverse Fourier transform of the graph, the reconstructed data sequence is calculated as follows: The reconstructed time series data results are compared with the original data. Figure 5 shown.

[0056] Calculate the center of the frequency domain component estimate The specific method is as follows:

[0057] Set frequency domain component estimates The initial value is 0, for all i=1,2,…,50,that is, i is executed from small to large. If b&1<<(i-1)=1, it means that the frequency domain data received by the computing center contains the i-th frequency component, then j = j + 1, where s m,j ∈N represents the jth value in set N, and the initial value of j is 1; otherwise, no operation is performed. represents the estimated value of the frequency domain component in the second data sequence after compression and transmission to the calculation center, that is, Figure 4 The frequency components of the 1st, 2nd, 3rd, 6th, 7th, 9th, 18th, 40th, 47th, and 50th frequencies are selected, and the values ​​of the remaining frequency components are 0.

[0058] The contents described in the above embodiments are merely an enumeration of implementation forms of the present invention. The protection scope of the present invention should not be limited to the specific forms described in the embodiments, and the protection scope of the present invention should also include similar invention methods conceived on the basis of the present invention.

Claims

1. A time series data compression and reconstruction method based on graph signal processing. The data collection system consists of two layers: a computing center and an edge node. The edge node is a sensor that can sense data. It is characterized by: The method is as follows: Step (1) The sensor generates a sensor data sequence x1=[x 1,1 ,...,x 1,i ,...,x 1,I ] T , and send it to the computing center; x 1,i represents the perceived data value of the i-th time series data point, i = 1, 2, ..., I, I represents the length of the data time series sequence, [·] Τ Represents the transpose operation; Step (2) After the computing center receives the perception data sequence x1, it constructs the data graph represents the node set of the data graph, and ε represents the edge set of the data graph. The computing center calculates the Laplace matrix L and the graph Fourier basis matrix U of the data graph. The specific process is as follows: (2-1) Abstract the time series data points in the perception data sequence as nodes in the data graph, and use the perception data value as the node value; calculate the relative difference between any two data points in the data sequence to determine whether there is a connecting edge between the two data points: the relative difference between data point i and data point j max(x1) and min(x1) represent the maximum and minimum values ​​of the perception data sequence x1 respectively; when δ i,j When it is less than the set threshold γ, it means that there is a connection edge between data point i and data point j, otherwise there is no connection edge; for a data point that has no connection edge with any other data point in the perception data sequence x1, a connection edge is constructed between the data point and its adjacent previous data point to ensure that each data point in the perception data sequence x1 has at least one connection edge; (2-2) When there is an edge connecting data point i and data point j, the weight of the edge is 1, that is, w i,j =1, otherwise w i,j =0; Data graph The weight matrix Computational Graph The combined Laplace matrix L = DW; where the data graph The weighted degree matrix D is the weighted degree of each node {d1, d2, …, d I } is a diagonal matrix with diagonal elements, and the weighted degree of data point i (2-3) Computing center data graph The combined Laplace matrix L is decomposed into features, that is, L = UΛU T , the superscript T indicates the transpose, and Λ is the eigenvalue {λ1,…,λ k , …, λ I } is a diagonal matrix with diagonal elements, λ k represents the kth Laplace matrix eigenvalue, that is, the data graph kth frequency; the calculation center stores and updates the Fourier basis matrix U = [u1, ..., u k ,…,u I ],u k represents the eigenvector corresponding to the k-th Laplace matrix eigenvalue, i.e., the graph Fourier basis, k = 1, 2, ..., I; Step (3) The computing center sends the graph Fourier basis matrix U to the sensor node; Step (4) The sensor node receives the graph Fourier basis matrix U from the computing center, updates and stores the received graph Fourier basis matrix U; each time the sensor senses I data, it regards it as a data sequence; for each data sequence, execute steps (5) to (8); Step (5) the sensor node calculates the frequency domain value of the subsequent sensing data sequence to be transmitted; According to the perception data sequence x m With the stored graph Fourier basis matrix U, calculate the perception data sequence x m The frequency domain components of the signal s m =[s m，1 ,…,s m，i ,…,s m，I ] T ,s m =U T x m ,s m，i is the i-th frequency component of the m-th perception data sequence; Step (6) the sensor node performs frequency domain compression of the data sequence; The sensor node first calculates the perception data sequence x m Energy spectrum of |s m | 2 Then select the first n components with large energy in the frequency domain components of the image signal, n = I × r, r represents the data compression rate; the energy of the n components N represents a set of n frequency components; thus, the perception data sequence x is obtained m The total energy E of the signal in the frequency domain; The sensor node calculates the ratio E of the energy of n frequency components to the total energy of all frequency components n / E; if the ratio is E n / E≥η, then these frequency component sets N plus the identifier b are the data sequence x m After the compression, c={b,N}, execute step (7) and set the threshold η∈[0,1]; if the ratio E n / E<η, then determine the data sequence x m After the existing data graph is converted to the frequency domain, the frequency domain components are scattered and the data compression effect is poor. The computing center and the sensor need to update the data graph. The sensor will sense the data sequence x m Transmitted to the computing center, execute steps (2) to (4), that is, use the perception data sequence x m Construct a new data graph; the identifier b is used to indicate which frequency component the current compressed data represents; The identifier b is obtained by using a bit compression algorithm: first, the initial value of b is set to 0, and then the following operation is performed on all frequency components: if the i-th frequency component s i ∈N, then b=b|1<<(i-1), otherwise no operation is performed; Step (7) The sensor node transmits the data c to the computing center; Step (8) The computing center recovers the data sequence perceived by the sensor based on the received compressed data c. For each data sequence, the computing center first calculates the frequency domain component estimation value based on the identifier b and the compressed frequency component set N. Then, according to the inverse Fourier transform of the graph, the reconstructed data sequence is calculated Calculate the center of the frequency domain component estimate The specific method is: set the frequency domain component estimation value The initial value is 0. For i=1, 2, ..., I, if b&1<<(i-1)=1, it means that the frequency domain data received by the computing center contains the i-th frequency component, then is equal to the first frequency component in set N, and then the first frequency component in set N is removed from set N, otherwise no operation is performed; the operation stops when set N is an empty set.

Images