Compression of multi-modal sensing signals

Abstract

A data transmission and compression method is provided that is arranged to compress and transmit data between an edge device and a remote server. The method includes: collecting data at the edge device, wherein the data is due to a plurality of signal features; generating a data matrix at the edge device; transforming the data matrix, wherein transforming the data includes using a wavelet transform; compressing the data, wherein compressing the data includes utilizing an autoencoder; and transmitting the encoded compressed data to the remote server via a communication channel. The method further includes: at the remote server, decompressing the data utilizing an autoencoder; reconstructing the signal features using an inverse wavelet transform; and storing the reconstructed data features in a data repository on the remote server.

Description

TECHNICAL FIELD

[0001] The field of the invention relates to data compression techniques for transmitting large amounts of information over one or more networks, where the information comprises measured multi-peak signals of one or more devices (such as in the PowerGenome architecture). BACKGROUND

[0002] In recent years, optimizing power systems has become a prominent focus in order to improve the efficiency, cost and power requirements of such systems. Determining relationships between different signals from different devices within a power system network allows for the optimization of such systems to be achieved. Various benefits can be obtained by collating all signal information of a power system, and depending on the application can lead to higher level control of certain devices. An example of such a system is the PowerGenome architecture, where the compilation of all measured data within the system is compiled into a single database. However, in order to optimize such systems (such as PowerGenome), large amounts of data need to be collected from a number of different sources. Typically, in PowerGenome type applications, a number of different signals are measured at each of a number of devices connected within the system. These signals are typically recorded as high resolution raw data from the measurements obtained by, for example, sensor devices, and are collated together in order to establish relationships between the devices within the system. Therefore, these types of applications require large amounts of computational power, storage and high transmission bandwidth capabilities for efficient operation. The higher the complexity of the system, the more data needs to be recorded and therefore higher computational power and transmission means are required to facilitate this type of power system.

[0003] However, there are problems when trying to transmit this large amount of data over a network (such as wirelessly streaming data from a device to a remote server), as the data bandwidth cannot meet the demands for efficient data transfer unless equipped with large amounts of resources. This can not be feasible due to economic and / or technical reasons, and therefore limits the application, optimization and efficiency of certain power systems. In particular, streaming high resolution raw sensor data (such as in the PowerGenome architecture) becomes unrealistic for existing technology communication techniques.

[0004] The present invention aims to provide a method and system of compressing raw data signals in order to reduce the amount of data transmitted over a network. In particular, compressing a number of multi-peak signals (such as current and voltage waveforms) from a number of electronic devices to allow for large data set transmission from the electronic devices to a remote server or cloud. SUMMARY

[0005] As noted above, the present invention solves the problem of transmitting a substantial amount of data over a network. The present invention utilizes data featureization and compression techniques to reduce the amount of data transmitted between an edge device (e.g., a router, multiplexer, routing switch) or an embedded device (e.g., a computer system) and the cloud. This equates to reducing the size of the data to be transmitted over the network by 3 or more orders of magnitude. In a preferred embodiment, the present invention uses a combination of sparse wavelet transforms and artificial neural networks (such as autoencoders) to accomplish this. Furthermore, the disclosed methods and systems provide the ability to record data related to the difference between subsequent signals rather than the raw signal data. This allows more information to be stored at the edge or embedded device prior to performing featureization and compression, and reduces the computational resources required at the edge or embedded platform. For example, in a common PowerGenome-type system, the device measures raw current and voltage waveforms (i.e., periodic AC signals) from the intelligent circuit breakers within the device and stores this data prior to transmission to the server. Collating and transmitting this raw data requires an unrealistic amount of storage and communication resources. However, in the present invention, only the difference between the currently measured signal and the previously measured signal (i.e., Δx, where x = V or I) is recorded and stored, thus reducing the need for substantial computational resources. For example, in a series of measurements x1, x2, x3, x4, etc., only the Δx values are recorded, such that x2 = x1 + Δx1, x3 = x2 + Δx2, x4 = x3 + Δx3. In some embodiments, the Δx values can be different, i.e., Δx1 ≠ Δx2 ≠ Δx3. In any case, Δx1 ≠ Δx2 ≠ Δx3 takes up less storage space than x1, x2, x3, x4. Additionally, the algorithms implementing the data featureization and compression techniques of the disclosed methods use low computational power, thus facilitating applications utilizing edge or embedded devices.

[0006] In a preferred embodiment of the present invention, there is provided a transmission and compression method arranged to compress and transmit data between an edge device and a remote server, the transmission and compression method comprising the steps of: collecting data at the edge device, wherein the data is attributed to a plurality of signal features; generating a data matrix at the edge device; transforming the data matrix, wherein transforming the data comprises using a wavelet transform; compressing the data, wherein compressing the data comprises utilizing an autoencoder; transmitting the encoded compressed data to the remote server via a communication channel; the method further comprising: decompressing the data at the remote server using the autoencoder; reconstructing the signal features using an inverse wavelet transform; and storing the reconstructed data features in a data repository on the remote server.

[0007] In some aspects of the present invention, the collected data features can comprise measurement data from a plurality of N related sensors at the edge device. Collecting data from a plurality of related sensors provides a deeper understanding of the system operation.

[0008] The data features can be periodic signals taken as a time series. This provides an improved overview of the devices within the system architecture and the performance of such devices over time.

[0009] In some cases, the data taken can be raw data of measurements, allowing common methods to be implemented for small data sets if required.

[0010] In some aspects of the invention, the data recorded in the time series is a difference (Ax) for subsequent measurements, such that x2 = xi + Ax1, x3 = x2 + Ax2, x4 = x3 + Ax3, where Ax can be a positive or negative integer. This reduces the total amount of data taken, and therefore required to be transmitted.

[0011] In some cases, the edge device can be an embedded device. Embedding the edge device in the appliance or device in use removes the need to add additional edge devices to the architecture.

[0012] The transmission can be by a network protocol. It will be appreciated that other types of communication capable of transmitting data and information can be used.

[0013] The data can be taken from a PowerGenome architecture, where a large amount of data belonging to a large number of devices within the architecture can then be transmitted and analysed remotely.

[0014] The step of transforming the data can be arranged to sparsify the matrix. This results in a sparse matrix containing very few non-zero elements, reducing the density of the data to be compressed and transmitted.

[0015] The autoencoder on the edge device can be a neural network with an input layer. This type of autoencoder has the purpose of providing a learned representation for a set of data input into the autoencoder. The learned representation allows for dimensionality reduction.

[0016] The autoencoder on the remote server can be a pre-trained neural network. Having a set of pre-trained neural networks allows for improved and more efficient training of the autoencoder to be used depending on the application.

[0017] In one aspect of the application, there is provided a data transmission and compression system arranged to compress and transmit data between an edge device and a remote server, the data transmission and compression system comprising: an edge device, wherein the edge device comprises: a data collector, wherein the data collector collects a plurality of data features; a matrix generator, wherein the data matrix generator generates a matrix of the data; a transformer, wherein the transformer transforms the data using a wavelet transform; a compressor, wherein the compressor compresses the data using an autoencoder; and a transmitter; wherein the transmitter transmits the encoded compressed data to a remote server via a communication channel; a remote server, wherein the remote server comprises: a receiver; a decompressor, wherein the decompressor decompresses the data using an autoencoder; a reconstructor, wherein the reconstructor reconstructs the data into initial signal features using an autoencoder; and a data repository, wherein the reconstructed data features are stored in the data repository. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a schematic diagram of an exemplary system architecture, wherein the exemplary device is a microwave appliance;

[0019] Figure 2 is a flowchart of the process occurring within the edge device according to the application;

[0020] Figure 3 is the compression of the current-voltage features of an exemplary microwave appliance as measured by current and voltage sensors at the edge device; Figure 1

[0021] Figure 4 is a flowchart of the method steps for sensing, compressing and transmitting data on an edge device;

[0022] Figure 5 is the process occurring at the remote server;

[0023] Figure 6 is the decompression of the compressed current-voltage features of an exemplary microwave appliance in the cloud;

[0024] Figure 7 is a flowchart of the method steps for decompressing data in a cloud server; and

[0025] Figure 8 is a flowchart of the method steps for autoencoder training, customization for this particular application. DETAILED DESCRIPTION

[0026] Figure 1 is an exemplary schematic diagram showing the system architecture 100 comprising both sides of the communication channel, wherein the communication channel is a wireless network between an edge device 101 and a remote server 102. Figure 1 ​The application shown is one of transmitting data from a microwave appliance 105 to a cloud server 102. The microwave oven 105 has N sensors 104 for measuring power characteristics (at least two of which are current and voltage sensors 103), and an on-premises edge device 101 for transmitting data to the cloud 102. Figure 1 The current and voltage sensors 103 in are represented as smart breakers. Other sensors can be used or included in the device to gather additional information about the operation of the appliance. The edge device 101 is responsible for data acquisition and compression, while the remote server (i.e. the cloud) 102 is responsible for data recovery and storage.

[0027] In use, Figure 2 is a flowchart of the processes and operations that occur at the edge device. The first step 201 is to acquire raw data from the sensors on the edge device. For example, current-voltage data as measured by the sensors 103 of a smart breaker on a microwave appliance 105, as shown in Figure 1 Different characteristics of different appliances can be measured concurrently from N sensors 104 on N devices that make up the system architecture 100, as in PowerGenome. In a home environment, these can include but are not limited to appliances such as microwave ovens, washing machines, refrigerators, cookers, etc. In a manufacturing environment, the devices can include but are not limited to devices such as motors, pumps, mixers, compressors, generators, etc. The method can be applied in other environments where a system architecture arises and there is a need for remote data processing.

[0028] First, the edge device 101 acquires data and creates a data matrix 202. The edge device 101 acquires time series data features from N relevant sensors 104 and creates an N-dimensional matrix representation of the data (initialized to zero), as shown in Figure 2 If the data obtained at each time instant (x t ) is the acquired data (x1, x2, x3,..., x N ) of each of the sensors (1, 2, 3,..., N), then 1 is added to the corresponding position of the matrix (x1, x2, x3,..., x N ), i.e. up to a maximum value. Data acquisition is performed for a predefined time period, for example 10 seconds, which includes N time instants. After the full window of samples (i.e. Nx (x t )), the values of the matrix are scaled to between 0 and 1 (i.e. normalized) depending on the maximum value recorded.

[0029] The edge device 101 then performs two operations on the N-dimensional matrix, transformation and compression. This is shown in Figure 2The transform operation separates the high and low frequency components of the matrix by using an N-dimensional wavelet transform, such as a Haar transform. As will be known to the skilled person, other wavelet transforms can be used. The component values are resolved by applying the wavelet transform to a particular depth, which is determined according to the application, for example signal and image processing. Applying the wavelet transform to the N-dimensional matrix 203 makes signal compression 204 in the second of the two operations performed at the edge device 101 easier. This is because the high frequency signal components are easily identified and can be compressed more or less than the low frequency components based on the application needs.

[0030] The transformed matrix 204 is then compressed using an autoencoder. Autoencoders use neural networks to compress signals by finding features that contain the most important information. These will typically be the largest scaled values of the matrix, i.e. the highest number of time constant samples of the measurement that have the same value. This can be depicted as the highest intensity values within a 3D mapping image plot of the features of the measurement, where the third dimension data is the intensity value for the corresponding x-y values. The autoencoder also forces the neural network to use fewer features in the hidden nodes, i.e. the low intensity regions of the plot. The number of nodes generated at the output of the edge device 101 (the output nodes) is the same as the number of input nodes received at the remote server 102. Thus, one half of the autoencoder compresses the signal at the edge device 101, while the other half of the autoencoder decompresses the signal at the remote server 102.

[0031] In an exemplary implementation of the present application, as shown in Figure 3 the process is applied to the current and voltage signals of a microwave appliance 105. The current and voltage are measured by corresponding sensors 301 and a 2D matrix is constructed from the periodic information inherent in the AC current and voltage signals, as shown in 302 of Figure 3 Figure 3 The intensity values within the plot of 302 correspond to the number of times a particular value has been captured and the x-y coordinates represent the current and voltage values. After generating these current-voltage feature plots, a 2D Haar transform is applied, as shown in 303 of Figure 3 ​As shown in 303, a Haar transform is applied to a specific depth, as determined by an algorithm. This results in a sparse 2D signal, which is fed into a 2D neural network at a transmitter within the microwave appliance or at an edge device connected to the microwave appliance. The 2D neural network is a compressed half of a pre-trained autoencoder and can be part of a high-power edge computing unit. Thus, the portion of the autoencoder located on edge device 101 is a neural network with an input layer (i.e., input data size) that is much larger than the output layer (i.e., output data size). The resulting encoded data of the measured properties of microwave appliance 105 can then be transmitted to a remote server 102. This process can also be applied to signals within electric vehicles (EVs), where data is acquired from sensors within the EV and subsequently transmitted from the vehicle to a gateway, server, or cloud. For system architectures connecting multiple devices, analysis can be performed at the gateway of the connected devices and subsequently transmitted from the gateway to a server or cloud.

[0032] Figure 4 The desired method steps for implementing one aspect of the invention on an edge device 101 are described. This is by Figure 4 The flowchart 400 is shown in the figure. Figure 4 Steps 401 to 405 represent the acquisition, compression, and transmission of data at the edge device. In 401, current and voltage data are read by edge device 101, where values ​​are acquired at a certain sampling rate or at a certain moment from N sensors 104 (at least two of which are current and voltage sensors 103). These current and voltage (IV) values ​​are timestamped, thus giving the IV data a time value. These timestamped IV values ​​are stored in a data repository. Depending on the device, application, and computing power, this storage of data can be temporary or long-term. The processed timestamped data is treated as a series of N one-dimensional vectors, which are then used to generate an N-dimensional matrix, as in 402. This significantly reduces the amount of data because any periodic information is compressed into a single value or point. Since AC current and voltage signals are periodic multi-peak signals, large compression can be achieved. After acquiring a full window of samples (i.e., a set of N timestamped data within a specific time series), the samples are normalized relative to the number of occurrences. The most frequently occurring value will be the maximum value of 1, with other values ​​scaled accordingly. This allows for determining correlation based on the number of samples appearing at a particular point.

[0033] exist Figure 4 At position 403, in other words, step 3 of the method involves (e.g., using a Haar transform) performing a wavelet transform to sparsify the matrix. The wavelet transform is applied to compute approximations of the horizontal, vertical, and diagonal results of the matrix (see [link to documentation]). Figure 3(303). Repeat the wavelet transform several times (i.e., multiple levels of the transform) until a sufficiently sparse output is achieved. This is as follows: Figure 4 The iterative process shown in 403 involves applying a deeper level of transformation until the percentage of zeros measured is sufficient (e.g., non-zero < 10%). The depth level of the transformation depends on the application type and the type of data to be measured (e.g., data obtained from current and voltage signals), and is known through training and / or experience. Both edge device 101 and remote server 102 use the same transformation depth to match the reconstructed data features with the initial data features. This transformation depth information is shared by both edge device 101 and remote server 102, so the depth is known to both.

[0034] like Figure 4 Step 4 of the method described in 404 includes: compressing sparse data using a pre-trained neural network (e.g., an autoencoder). The pre-trained neural network implemented at this step is a first half of a pre-trained autoencoder and can be part of the edge device 101 as described above. Details of the pre-trained autoencoder are provided below. Compression of the sparse data signal also helps remove unwanted information because the data passes through a neural network with a large number of inputs and a small number of outputs (see [link to documentation]). Figure 3 The 304 error message is then passed. Finally, the data is transmitted. The resulting compressed data is sent to a remote server via a network protocol, such as... Figure 4 As shown in 405. Examples of suitable network protocols are WiFi, 4G, LTE-A, Ethernet, etc. Other network protocols can be considered. Steps 1 to 5 can be repeated until the process stops, times out, or, for example, a finite set of data features has been transmitted.

[0035] Figure 5 The process 500 occurring at a remote server (or cloud) is illustrated via a flowchart. As in 501, the first step is to receive a compressed data signal, such as that transmitted from edge device 101. This can be performed, for example, by a receiver at remote server 102. Once received, remote server 102 stores the compressed data in a data repository before applying any of the techniques used to recover the data back to its initial signal characteristics. An autoencoder is then used to decompress the compressed data matrix (as in 502, the inverse of the process performed at edge device 101) to retrieve less important information contained in the signal. The number of input nodes received by remote server 102 is the same as the number of output nodes generated at edge device 101. This decompression recovers a portion of the initial data within the data matrix, such that the data output of the autoencoder at remote server 102, to be further decompressed, mirrors the data input of the autoencoder at edge device 101. Thus, hidden nodes representing lower values ​​within the data matrix are recovered.

[0036] In a third step of the method, the data is then transformed by performing an inverse N-dimensional wavelet transform, such as a Haar transform, as indicated by 503 in the flowchart of Figure 5 This is performed to a certain depth, which is determined according to the application, but should match the depth of the transform applied at the edge device 101. The resulting data matrix is a normalized data matrix of the data, which represents the collected data (xl, x2, x3,..., xN) of each of the sensors (1, 2, 3,..., N) 104 connected to the edge device 101 at each time instant (xt) within the initial predefined time period. Finally, as indicated by 504 of Figure 5 The decompressed and transformed data, which represents the initial time series data features measured at the device or appliance 105, is stored within the data repository of the remote server 102.

[0037] In one exemplary implementation of the present application, as indicated by the schematic diagram 600 of Figure 6 The process is applied to the current and voltage signals of a microwave appliance 105, and the remote server 102 is in this case a cloud, such as a public or private enterprise cloud. This allows for remote monitoring and analysis of properties of the microwave appliance 105. Figure 6 The decompression step of the method occurring in the cloud 102 for exemplary microwave oven 105 current and voltage signals is depicted. The receiver part 601 of the autoencoder, which can be part of the cloud computing unit 102, is shown with an input node 602 of the received compressed data as transmitted from the edge device 101. This is the inverse of the autoencoder on the edge device 101 (see 304 of Figure 3 The neural network located on the cloud 102 has a much larger output layer (i.e. output data size) than the input layer (i.e. input data size). Using the output data, an inverse 2D Haar transform is applied, as indicated by 603 of Figure 6 The transform is performed to a certain depth as determined by the algorithm, which is equivalent to the depth of the 2D Haar transform used at the edge device 101. The transform results in an initial current-voltage data matrix and feature plot (as indicated by 302 of Figure 3 The exemplary current-voltage plot is shown in 603 of Figure 6 which corresponds to the determined current and voltage values as measured from the AC signals of the microwave appliance 105. The exemplary current-voltage plot is shown in 603 of Figure 3 The resulting values are then stored in a data repository on the cloud 102 for subsequent analysis, which has the ability to be performed at any location as long as the user has access to the data repository of the cloud 102.

[0038] Figure 7 A flowchart depicting the required method steps to be implemented on the remote server 102 of an aspect of the present application. This is shown by Figure 7 in Figure 7 of the drawings. Figure 7 Method steps 1 to 4 (referred to as 701 to 704) of Figure 7 represent the process for decompression of the current-voltage characteristic in the cloud server 102. In step 1 (i.e. 701), the receiver element of the cloud 102 receives the compressed current-voltage characteristic as transmitted from the edge device 101 using a network protocol. Once the cloud 102 obtains the data sent by the edge 101, the data is stored before it is decompressed. The data can be stored temporarily in a data store located at the cloud server 102 or can be stored in the data store for an extended amount of time. The temporarily stored data is stored at least until sufficient data is received for a full window, which is equivalent to the window sample measured at the edge device 101. In the decompression process 702 at the cloud server 102, the second half of the pre-trained autoencoder is used to decompress the data. The data is passed through the trained neural network which has a small number of inputs and a large number of outputs. This is shown by 702 of Figure 7. The resultant output data values are stored in the data store. Again, the data can be stored temporarily or for an extended amount of time. Figure 6

[0039] In Figure 7 step 3 (i.e. 703) of the method involves performing an inverse wavelet transform, for example using an inverse Haar transform, to densify the matrix. The inverse wavelet transform is performed a number of times, i.e. to a number of levels equal to the depth used at the edge device 101. This is applied to calculate approximations of the horizontal, vertical and diagonal results of the matrix (see Figure 6 ​The decompressed N-dimensional (i.e. current-voltage) features of the N-dimensional matrix are stored in the data repository in the fourth step 704 of the method performed at the remote server 102. The features represent unique events or states of some real-world appliance (e.g. microwave 105). If multiple appliances or devices are connected within a system architecture, such as PowerGenome, several features can be compressed, transmitted, decompressed and then stored for later analysis. For example, decompressing the signal to get the current-voltage features allows for studying the characteristics of the machine, appliance or device 105, thereby providing an understanding of the type of events that occur at the machine, appliance or device 105. An event can be considered as the device 105 turning on or off or changing state. Having remote access to multiple events occurring from multiple devices 105 connected in a system architecture provides a deeper understanding of the operation and evaluation of the network of devices 105 without the need to be on-site. This saves time, energy, and can contribute to improving the efficiency within the system. Other benefits of implementing such a method can be realized depending on the application.

[0040] To implement the compression and transmission method and system as described above, a pre-trained autoencoder is utilized and can be selected from a variety of known autoencoders. The autoencoder can be a generally commonly used autoencoder and then customized for a specific application. As Figure 8 An exemplary training of the autoencoder used in the present invention is provided in method steps 1 to 6 as represented by 801 to 806 of the flowchart. Training is required for both halves of the autoencoder (edge 101 and cloud 102), where method steps 801 to 806 include training each side together. The output layer nodes of the edge neural network are connected one-to-one to the input layer nodes of the neural network in the cloud. A gradient descent algorithm is used to update the weights of the nodes on both the edge 101 and cloud 102 sides of the autoencoder with the goal of minimizing the difference between the original (input) data at the edge 101 and the reconstructed (output) data at the cloud 102. This allows the reconstructed data to be comparable to the original data with minimal error.

[0041] Starting from step 1 (801) of the flowchart of FIG. 8, the autoencoder receives training data at the edge 101 side, where the data used for autoencoder training is the sparsified data matrix. This is obtained by performing the steps of the compression part of the method as described above (see 301 to 303 and 401 to 403 of the flowchart of FIG. 3). Figure 8 Figure 3 Starting from step 1 (801) of the flowchart of FIG. 8, the autoencoder receives training data at the edge 101 side, where the data used for autoencoder training is the sparsified data matrix. This is obtained by performing the steps of the compression part of the method as described above (see 301 to 303 and 401 to 403 of the flowchart of FIG. 3). Figure 4 ​of 401 to 404). Once compression is applied, step 2 of the autoencoder training (i.e. 802) requires the data to be forward propagated in the neural network by performing a forward pass. This forward pass through the neural network is performed with the same number of input nodes as output nodes and a small number of hidden nodes. Thus, the neural network is a coupling of the autoencoder at the edge device 101 with the autoencoder at the cloud 102.

[0042] Gradient descent or backpropagation is then applied as shown by 803 of the flowchart of Figure 8 Backpropagation is performed to enable adjustment of the neural network weights. This is achieved by setting the input to be the target output of the neural network and applying backpropagation. If the desired level of accuracy is achieved (e.g. accuracy > 95%), step 4 of the method can be performed as shown by 804 of Figure 8 However, if the predetermined maximum number of iterations has not been reached, the steps represented by 802 and 803 are performed again and the training continues. The number of iterations can be defined by the user. If the desired accuracy has not been obtained and the maximum number of iterations has been reached, the algorithm proceeds to 806.

[0043] In step 5 (805 of Figure 8 of 804), the trained autoencoder is saved in the database along with the autoencoder weights and structure. This is then stored in the data repository. Step 5 (805 of Figure 8 of 804) involves further compressing the data of the autoencoder by reducing the number of nodes in the intermediate layers of the autoencoder. The intermediate layer data corresponds to the output values of the compression algorithm. The number of intermediate nodes is successively reduced by 1 and the autoencoder is retrained at each reduction iteration. The retraining involves performing the second, third and fourth method steps again, i.e. 802, 803 and 804. Once the autoencoder cannot be further compressed, the algorithm moves from 804 of the flowchart to 806. Step 6 of the method (as shown in 806 of Figure 8 of 806) involves selecting the best encoder for a particular application. If previous autoencoders have not been stored in the data repository, it is not possible to create an autoencoder that compresses the particular input data with the desired accuracy, i.e. it is best to store multiple autoencoders. In the case where no previous autoencoders are stored and the newly compressed autoencoder does not achieve satisfactory accuracy, the training method described above cannot be used. In this case, the accuracy must be lowered or a different compression method needs to be used. If there are multiple autoencoders, the autoencoder with the highest accuracy (e.g. > 95% accuracy) and the smallest number of nodes will be selected. Thus, this pre-trained autoencoder will be used in the transmission of data (e.g. current-voltage features) between the edge device 101 and the remote server or cloud 102.

Claims

1. A data transmission and compression method arranged to compress and transmit data between an edge device and a remote server, the data transmission and compression method comprising the steps of: acquiring data at the edge device, wherein the data is attributed to a plurality of signal characteristics, wherein the acquired data characteristics comprise measured data from a plurality of N related sensors at the edge device, wherein the data characteristics are periodic signals acquired as a time series; generating an N-dimensional data matrix at the edge device comprising: incrementing a value in the matrix by one if the data value acquired from the N related sensors at each time instant corresponds to a position in the matrix for a predefined time period; and scaling the values of the matrix to between 0 and 1 according to the maximum value recorded in the matrix; transforming the data matrix, wherein transforming the data comprises using a wavelet transform, wherein the step of transforming the data is arranged to sparsify the matrix; compressing the data, wherein compressing the data comprises utilizing an autoencoder; transmitting the encoded compressed data to the remote server via a communication channel; the method further comprising: decompressing the data at the remote server utilizing an autoencoder; reconstructing the signal characteristics using an inverse wavelet transform; and storing the reconstructed data characteristics in a data repository on the remote server.

2. The method according to claim 1, wherein the acquired data characteristics comprise current and voltage signals.

3. The method according to claim 1 or 2, wherein the acquired data is raw data of measurements.

4. The method according to any one of claims 1 to 3, wherein the data recorded in the time series is a difference value Ax for subsequent measurements such that x2 = xi + Ax1, x3 = x2 + Ax2, x4 = x3 + Ax3, wherein Ax can be a positive or negative integer.

5. The method according to any one of claims 1 to 4, wherein the edge device comprises an embedded device.

6. The method according to any one of claims 1 to 5, wherein the transmission is made over a network protocol.

7. The method according to any one of claims 1 to 6, wherein the autoencoder on the edge device comprises a neural network with an input layer.

8. The method according to any one of claims 1 to 7, wherein the autoencoder on the remote server comprises a pre-trained neural network.

9. A data transmission and compression system arranged to compress and transmit data between an edge device and a remote server, the data transmission and compression system comprising: an edge device, wherein the edge device comprises: a data acquirer; wherein the data acquirer acquires a plurality of data characteristics, wherein the acquired data characteristics comprise measured data from a plurality of N related sensors at the edge device, wherein the data characteristics are periodic signals acquired as a time series; a data matrix generator; wherein the data matrix generator generates a matrix of N dimensions of data, the data matrix generator configured to: increment a value of a location in the matrix by one if the data value collected from each of the N related sensors at each time instant corresponds to the location in the matrix within a predefined time period; and scale the values of the matrix to between 0 and 1 according to a maximum value recorded in the matrix; a transformer, wherein the transformer transforms the data using a wavelet transform, wherein the transformer is configured to sparsify the matrix; a compressor, wherein the compressor compresses the data using an autoencoder; and a transmitter; wherein the transmitter transmits the encoded compressed data to the remote server via a communication channel; a remote server, wherein the remote server comprises: a receiver; a decompressor, wherein the decompressor decompresses the data using an autoencoder; a reconstructor, wherein the reconstructor reconstructs the data into the initial signal features using an autoencoder; and a data repository, wherein the reconstructed data features are stored in the data repository.

Images