Method and apparatus for facilitating storage of data from an industrial automation control system or power system

Abstract

To facilitate storage of data from multiple data sources (31-33) of an industrial automation control system, a power distribution system, or a power generation system, a decision device (20) executes a machine learning algorithm to determine a compression technique based on the data source (31-33) from which the data originates.

Description

Technical Field

[0001] The present invention relates to methods and apparatus for facilitating the storage of data from multiple data sources. In particular, the present invention relates to methods and apparatus that can improve the storage of data provided by data sources of an industrial automation control system IACS, a power distribution system or a power generation system. Background Art

[0002] Modern power utilities (such as power generation systems, power grids or substations) and modern industrial systems include a large number of components, sensors and intelligent electronic devices (one or more IEDs). Analyzing the data provided by these components is a difficult task.

[0003] Analyzing and using data related to the operation of an Industrial Automation Control System (IACS), distribution system, or generation system is important for improving the ongoing control of these systems and subsequent failure analysis. Traditionally, transmission system operators (TSOs) and distribution system operators (DSOs) retain fine-grained measurement data sets over a period of time (e.g., last year). In most cases, old data is aggregated and stored at a granularity of 15 minutes, which is too coarse to perform meaningful analysis and queries on the aggregated data later to extract meaningful patterns and relationships from past data. As a result, valuable information is lost due to current long-term data storage strategies.

[0004] It would be beneficial to store data relating to the operation of an IACS, power distribution system or power generation system in a manner that would allow a more thorough analysis to be performed at a later time.

[0005] For example, data may need to be stored in a way that allows operators to retrospectively analyze whether and how the existing grid can be used to meet growing customer demand in order to understand and further optimize their systems for future use.

[0006] Integration of unstable renewable energy sources requires more analysis to ensure grid stability. This requires storing detailed renewable energy data and system loads for longer durations (such as several years) to ensure stable optimization and control systems.

[0007] To further illustrate, a more thorough analysis of previously collected data may be required in order to determine grid expansion strategies, if any.

[0008] In these and other cases, detailed data must be available over time, for example, to cover long-term changes such as weekly, monthly, quarterly, and annual patterns. Under current information aggregation strategies, such analyses are either not possible or would require excessive storage requirements.

[0009] US8156232B2 discloses an industrial automation device including a compression component for compressing XML data.

[0010] “Data Compression and Prediction Using Machine Learning for Industrial IoT” by J. Park et al., presented at the 2018 International Conference on Information Networks (ICOIN), describes techniques for compressing industrial data into representative vectors using lossy compression using neural network regression. Summary of the invention

[0011] The object of the present invention is to provide improved methods and apparatus to facilitate the storage of data originating from sensors, merging units, intelligent electronic devices (IEDs), or other components of an industrial automation control system (IACS), a power distribution system, a power generation system, or other system. In particular, the object is to provide methods and apparatus to facilitate the storage of data from data sources of an IACS, a power distribution system, a power generation system, or other system to reduce storage space requirements, thereby facilitating subsequent analysis of the data that may fit into weekly, monthly, quarterly, and / or annual patterns.

[0012] According to an embodiment, machine learning methods are used to compress data in a manner that can support detailed long-term analysis while reducing the physical storage required. This allows operators to collect more data (e.g., by reducing the communication bandwidth required to transmit data from sensors to storage systems) and store that data more efficiently for longer time intervals. Thus, data compressed using machine learning techniques enables the provision of new services and products.

[0013] Machine learning methods can optimize compression by automatically selecting from a variety of candidate compression techniques and / or determining compression parameters for each of several data sets / streams. In this way, the system can optimize itself, which also reduces engineering effort.

[0014] A method according to an embodiment facilitates storage of data from multiple data sources of a system. The system is an IACS, a power distribution system, or a power generation system. The method includes determining, using at least one integrated circuit of a decision device, a compression technique to be applied to the data. The decision device executes a machine learning algorithm to determine a compression technique based on a data source from which the data originates. The method includes applying, by the decision device, the compression technique determined for a data source to data from the data source.

[0015] Information about changes in the compression technology to be adopted may be provided by the decision device to the corresponding data source and / or storage device via a push mechanism.

[0016] Determining the compression technique may include executing a machine learning algorithm to determine which of several candidate compression techniques to apply.

[0017] The several candidate compression techniques may be predetermined compression techniques.

[0018] At least some of these candidate compression techniques may include correlating time series data from several data sources.

[0019] Depending on the determined correlation (eg, based on a threshold comparison of the determined correlation), information indicative of the presence of the correlation may be transmitted by the data source and / or stored at the storage device.

[0020] Depending on the determined correlation, time series data or frequency domain data may be transmitted and / or stored for only one data source of the set of at least two data sources, the time series data of which is determined to be correlated.

[0021] At least some of these candidate compression techniques may include performing classification and / or clustering.

[0022] Depending on the result of classification and / or clustering, information indicating that data of a set including at least two data sources are determined to belong to the same category and / or cluster may be transmitted and / or stored.

[0023] Depending on the results of classification and / or clustering, time series data or frequency domain data may be transmitted and / or stored for only one data source in a set of at least two data sources whose time series data is determined to belong to the same category and / or cluster.

[0024] Determining the compression technique may include executing a machine learning algorithm to determine whether to apply compression in the time domain or in the frequency domain.

[0025] Determining the compression technique may include executing a machine learning algorithm to determine at least one parameter of the compression technique.

[0026] The method may further include automatically repeating the steps of determining the compression technique and applying the compression technique.

[0027] The steps of determining a compression technique and applying the compression technique may be repeated in a regular manner.

[0028] The steps of determining a compression technique and applying the compression technique may be repeated periodically or in an event-driven manner.

[0029] Determining the compression technique may include executing a machine learning algorithm to generate an update to a data model or profile associated with the data source.

[0030] The method may further include transmitting, by the decision device, update information related to the update.

[0031] The decision device may transmit update information to updated data sources for which data models or profiles have been determined.

[0032] The update information may include information about changes in one or several parameters of the compression technique.

[0033] The method may further include the data source updating the compression model or configuration file stored locally at the data source based on the received update information, thereby generating an updated compression model or configuration file.

[0034] The method may include performing compression by the data source based on the updated compression model or profile.

[0035] The data source to which the update information is transmitted may include a sensor device or a merging unit.

[0036] The method may further include transmitting, by the decision device, the update information to at least one storage device that stores data from the data source.

[0037] The at least one storage device may be provided separately from the plurality of data sources.

[0038] Machine learning algorithms can determine compression techniques within constraints associated with the data source.

[0039] The data source related constraints may include enforcing lossless storage.

[0040] The data source related constraints may include enforcing lossy storage.

[0041] The data source related constraints may include enforcing temporal data storage.

[0042] The data source related constraints may include enforcing frequency domain data storage.

[0043] The method may further include training the machine learning algorithm during operation of the IACS, the power distribution system, or the power generation system.

[0044] Training a machine learning algorithm may include learning whether compression applied in the time domain or in the frequency domain is more beneficial.

[0045] The method may further include using historical data to pre-train the machine learning algorithm.

[0046] The data output by the plurality of data sources may include data streams. The data streams may include non-packetized data.

[0047] The decision device may determine a first compression technique to be used for data originating from a first data source and a second compression technique to be used for data originating from a second data source, the second compression technique being different from the first compression technique and the second data source being different from the first data source.

[0048] The plurality of data sources may include at least one intelligent electronic device (IED).

[0049] The plurality of data sources may include sensor devices operable to output digital data, in particular data streams.

[0050] The plurality of data sources may include a merging unit operable to output digital data.

[0051] The plurality of data sources are operable to output data relating to physical values ​​of the IACS, power generation system, or power distribution system.

[0052] The data related to physical values ​​may include current, voltage, switch parameters (such as the switching state or operating state of a switch motor), transformer condition parameters (such as parameters related to the bushing or insulation of the transformer), or other data.

[0053] The method may further include compressing data using the determined compression technique by the plurality of data sources before outputting the data, and storing the compressed data by at least one storage device.

[0054] The method may further include analyzing the compressed data stored in the at least one storage device.

[0055] Analyzing the stored compressed data may include analyzing the suitability of the power distribution system or the power generation system in accommodating increasing customer demand.

[0056] Analyzing the stored compressed data may include performing a stability analysis on the power distribution system or the power generation system.

[0057] Analyzing the stored compressed data may include determining a grid expansion strategy.

[0058] The compressed data being analyzed may represent a time period exceeding 24 hours.

[0059] The compressed data being analyzed may represent a time period greater than 7 days.

[0060] The compressed data analyzed may represent a time period exceeding 28 days.

[0061] The compressed data analyzed may represent a time period exceeding 3 months.

[0062] The compressed data analyzed may represent a time period of more than 1 year.

[0063] The compressed data analyzed may represent a time period of at least two years.

[0064] The compressed data analyzed may represent a time period of at least three years.

[0065] A decision device according to an embodiment is adapted to facilitate storage of data from multiple data sources of an IACS, a power distribution system, or a power generation system. The decision device includes at least one interface adapted to be communicatively coupled to the multiple data sources, and at least one integrated circuit operable to determine a compression technique to be applied to the data. The decision device is adapted to execute a machine learning algorithm to determine a compression technique based on the data source from which the data originates.

[0066] The decision device may be operable to provide information regarding changes in compression techniques to be employed to respective data sources and / or storage devices via a push mechanism.

[0067] The at least one integrated circuit is operable to generate control information that applies a compression technique determined for a data source prior to storing the data originating from the data source.

[0068] The at least one integrated circuit is operable to execute a machine learning algorithm to determine which of several candidate compression techniques to apply.

[0069] The at least one integrated circuit is operable to execute a machine learning algorithm to determine at least one parameter of a compression technique.

[0070] The at least one integrated circuit is operable to automatically repeat a determination compression technique.

[0071] The at least one integrated circuit is operable to repeatedly determine the compression technique in a regular manner.

[0072] The at least one integrated circuit is operable to repeatedly determine the compression technique periodically or in an event driven manner.

[0073] The at least one integrated circuit is operable to execute a machine learning algorithm to generate updates to a data model or profile associated with the data source.

[0074] The decision device is operable to transmit update information related to the update.

[0075] The update information may include information about changes in one or several parameters of the compression technique.

[0076] The decision device may transmit update information to updated data sources for which data models or profiles have been determined.

[0077] The data source to which the update information is transmitted may include a sensor device or a merging unit.

[0078] The decision device is operable to transmit update information to at least one storage device storing data from a data source.

[0079] The at least one storage device may be provided separately from the plurality of data sources.

[0080] The at least one integrated circuit is operable to execute a machine learning algorithm to determine a compression technique under constraints associated with a data source.

[0081] The data source related constraints may include enforcing lossless storage.

[0082] The data source related constraints may include enforcing lossy storage.

[0083] The data source related constraints may include enforcing temporal data storage.

[0084] The data source related constraints may include enforcing frequency domain data storage.

[0085] The at least one integrated circuit is operable to train a machine learning algorithm during operation of the IACS, power distribution system, or power generation system.

[0086] The at least one integrated circuit is operable such that training the machine learning algorithm includes learning whether compression applied in the time domain or compression applied in the frequency domain is more beneficial.

[0087] Machine learning algorithms can be pre-trained using historical data.

[0088] The decision device is operable to determine a first compression technique to be used for data originating from a first data source and a second compression technique to be used for data originating from a second data source, the second compression technique being different from the first compression technique and the second data source being different from the first data source.

[0089] A system according to an embodiment, the system is an industrial automation control system IACS, a power distribution system or a power generation system, and the system includes: multiple data sources and at least one storage device for storing compressed data from the multiple data sources; and a decision device of the embodiment.

[0090] The system is operable to perform the method of an embodiment.

[0091] The decision device is operable to transmit update information related to an update of a data model or configuration file associated with the data source. The decision device is operable to transmit the update information to the data source for which the update of the data model or configuration file has been determined. The data source may update the compression model or configuration file stored locally in the data source based on the received update information, thereby generating an updated compression model or configuration file.

[0092] The update information may include information about changes in one or several parameters of the compression technique.

[0093] The data source may perform compression based on an updated compression model or profile.

[0094] The plurality of data sources are operable to output data streams. The data streams may include non-packetized data.

[0095] The plurality of data sources may include at least one intelligent electronic device (IED).

[0096] The plurality of data sources may include sensor devices operable to output digital data, in particular data streams.

[0097] The plurality of data sources may include a merging unit operable to output digital data.

[0098] The plurality of data sources are operable to output data relating to physical values ​​of the IACS, power generation system, or power distribution system.

[0099] The data related to physical values ​​may include current, voltage, switch parameters (such as the switching state or operating state of a switch motor), transformer condition parameters (such as parameters related to the bushing or insulation of the transformer), or other data.

[0100] The plurality of data sources are operable to compress the data using the determined compression technique prior to outputting the data.The at least one storage device may store the compressed data.

[0101] The system may further comprise an analysis device operable to analyze the stored compressed data.

[0102] The analysis device is operable to analyze the suitability of an electrical distribution system or a power generation system in accommodating increasing customer demand.

[0103] The analysis device is operable to perform a stability analysis on a power distribution system or a power generation system.

[0104] The analyzing device is operable to determine a grid expansion strategy.

[0105] The analysis apparatus is operable to analyze compressed data representing a time period exceeding 24 hours.

[0106] The analysis apparatus is operable to analyze compressed data representing a period of time exceeding seven days.

[0107] The analysis apparatus is operable to analyze compressed data representing a period of time exceeding 28 days.

[0108] The analysis apparatus is operable to analyze compressed data representing a time period exceeding 3 months.

[0109] The analysis apparatus is operable to analyze compressed data representing a time period in excess of 1 year.

[0110] The analysis apparatus is operable to analyze compressed data representing a time period of at least two years.

[0111] The analysis apparatus is operable to analyze compressed data representing a time period of at least three years.

[0112] A method according to an embodiment facilitates processing of data from multiple data sources of a system. The system may be an industrial automation control system (IACS), a power distribution system, or a power generation system. The method includes using at least one integrated circuit of a decision device to determine a processing technique to be applied to the data. The decision device executes a machine learning algorithm to determine a processing technique based on a data source from which the data originates. The method includes applying, by the decision device, a processing technique determined for a data source to data from the data source.

[0113] According to another embodiment, a decision device is adapted to facilitate processing of data from multiple data sources. The decision device includes at least one interface adapted to be communicatively coupled to the multiple data sources, and at least one integrated circuit operable to determine a processing technique to be applied to the data. The decision device may execute a machine learning algorithm to determine the processing technique based on the data source from which the data originated.

[0114] The compression technique may be selected from Lempel-Ziv type compression algorithms, run-length encoding and similar methods.

[0115] Methods and devices according to embodiments allow for automatic generation and / or updating of lossy and lossless compression profiles. The compression profile may depend on the data owner. The compression profile may represent the data to be stored. Statistical methods, neural networks, reinforcement learning, or other techniques may be used to generate or update the compression profile.

[0116] Such a process provides a number of parameters that can be modified or updated. The generation and / or updating of the configuration file can be optimized for the frequency and type of queries made during the data analysis task to use the stored data. Alternatively or additionally, the data owner can specify preferences in advance. This may be meaningful in situations where data loss must be avoided to the greatest extent possible.

[0117] Different compression profile information from different data owners and / or different data sources can be combined to make up for missing information in certain profiles. Alternatively or additionally, clustering can be performed on the compression profiles to ensure that the system can use the compression profiles for the data without actually having a priori knowledge of the profiles of the data in detail. A machine learning engine can leverage system knowledge (e.g., about the behavior and properties of the power grid) to reason about the extracted profiles and provide corrections when necessary. Alternatively or additionally, statistical features such as correlations and / or physical interdependencies in system components and measurements can be used to improve models and profiles.

[0118] By means of the method, apparatus and system according to the embodiments, various effects and advantages are obtained.

[0119] Storage space requirements and communication bandwidth required (eg, for transferring compressed data to a storage device) are reduced. Automatic selection of compression algorithms reduces engineering effort.

[0120] Since significantly more data is available during the assessment period, existing data analysis methods can be expanded and improved. By ensuring secure energy supply, the enhanced analysis options enable transmission system operators (TSOs) and distribution system operators (DSOs) to operate their systems closer to their capacity limits.

[0121] Higher prediction accuracy can be obtained based on the compressed data, which makes the operation of the power grid or IACS safer and also has advantages in energy trading and resource investigation (e.g., coal, natural gas).

[0122] The enhanced data analysis options allow for more reliable recommendations of maintenance and expansion options.Alternatively or additionally, the enhanced data analysis options allow for dynamic trading and delivery of energy at various voltage levels. BRIEF DESCRIPTION OF THE DRAWINGS

[0123] The subject matter of the invention will be explained in more detail with reference to preferred exemplary embodiments illustrated in the accompanying drawings, in which:

[0124] Figure 1 is a schematic diagram of a system including a decision making device according to an embodiment.

[0125] Figure 2is a schematic diagram of a system including a decision making device according to an embodiment.

[0126] Figure 3 is a flow chart of a method according to an embodiment.

[0127] Figure 4 is a schematic diagram of a system including a decision making device according to an embodiment.

[0128] Figure 5 is a flow chart of a method according to an embodiment.

[0129] Figure 6 is a schematic diagram illustrating the use of machine learning to determine compression techniques.

[0130] Figure 7 is a flow chart of a method according to an embodiment.

[0131] Figure 8 is a schematic diagram illustrating the operation of the embodiment. DETAILED DESCRIPTION

[0132] Exemplary embodiments of the present invention will be described with reference to the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements. Although some embodiments will be described in the context of a power generation system or a power distribution system, the methods and apparatus described in detail below can be used in a variety of systems.

[0133] Unless specifically stated otherwise, the features of the embodiments may be combined with each other.

[0134] According to an embodiment of the present invention, a machine learning method is used to compress data provided by multiple data sources of an industrial automation control system (IACS), power generation system, or power distribution system in a manner that supports detailed long-term analysis while reducing the required physical storage. This reduces the communication bandwidth required to transfer data from the data source to the storage device. Data can be stored more efficiently. For a given available storage capacity, optimized storage achieved by automatically determined compression allows data to be stored for much longer time intervals for analysis.

[0135] Figure 1 is a schematic diagram of a system according to an embodiment.

[0136] The system includes a plurality of data sources 31, 32, 33, collectively referred to as data sources 30. The data sources 31, 32, 33 may be sensors, merging units, intelligent electronic devices (IEDs), or other devices that provide data related to the operation of an IACS, a power generation system, or a power distribution system, respectively. The data sources 31, 32, 33 may output data in digital form. The data sources 31, 32, 33 may output data in a time domain and / or in a frequency domain. At least some of the data sources 31, 32, 33 may output data streams, including non-packetized data.

[0137] As will be explained in more detail below, the data sources 31, 32, 33 are each operable to compress data prior to transmission using a compression technique determined using a machine learning approach.

[0138] The system includes one or several storage devices 41, 42, 43. These storage devices are collectively referred to as at least one storage device 40. One or more storage devices 41, 42, 43 are operable to store compressed data from data sources 31, 32, 33 for subsequent analysis. When storing compressed data, one or more storage devices 41, 42, 43 may store information related to a data model or profile associated with the data source 31, 32, 33.

[0139] One or more storage devices 41, 42, 43 may include a non-volatile storage medium having the ability to store compressed data representing a time period of greater than 24 hours, a time period of greater than 7 days, a time period of greater than 28 days, a time period of greater than 3 months, a time period of greater than 1 year, a time period of at least two years, or a time period of at least three years.

[0140] The one or more storage devices 41, 42, 43 may also store information about changes in compression techniques and / or changes in parameters of compression algorithms applied to data from various sources. The one or more storage devices 41, 42, 43 may store such information about changes in compression profiles and associated time information indicating the time when the changes took effect.

[0141] The system may comprise an analysis device 50. The analysis device 50 is operable to retrieve stored data from one or more storage devices 41, 42, 43 for further analysis.

[0142] The analysis device 50 is operable to analyze the suitability of the power distribution system or power generation system in accommodating increasing customer demands of the power distribution system or power generation system based on the compressed data provided by the data sources 31 , 32 , 33 .

[0143] Alternatively or additionally, the analysis device 50 may be operable to perform a stability analysis of the electrical distribution system or the electrical power generation system based on the compressed data provided by the data sources 31 , 32 , 33 .

[0144] Alternatively or additionally, the analysis device 50 may be operable to determine a grid extension strategy for the power distribution system or the power generation system based on the compressed data provided by the data sources 31 , 32 , 33 .

[0145] The analysis device 50 is operable to perform long-term analysis on the compressed data. The long-term analysis may include, for example, extracting trends that are only visible on a longer time scale based on the compressed data, the compressed data representing a time period of more than 24 hours, a time period of more than 7 days, a time period of more than 28 days, a time period of more than 3 months, a time period of more than 1 year, a time period of at least two years, or a time period of at least three years.

[0146] Importantly, as will be explained in detail below, the manner in which the data is processed at the data source 31, 32, 33 before being output is determined by the decision device 20. The decision device 20 employs a machine learning algorithm to determine which compression technique is to be applied to the data at the data source 31, 32, 33 before being output and / or to set one or more parameters of the compression technique applied to the data at the data source 31, 32, 33 before being output.

[0147] In this way, the decision device 20 can generate a profile or model for data compression. This can be done separately for each data source 31, 32, 33 or for groups of data sources, depending on, for example, the type of data, the amount of data or the ownership of the data.

[0148] The decision device 20 typically comprises an interface 22 operable to output information to the data source 31, 32, 33. The decision device 20 may provide information about changes in compression techniques to be employed by the data sources 31, 32, 33 to the respective data sources 31, 32, 33 via a push mechanism. The decision device 20 may provide information about changes in compression techniques to be employed by the data sources 31, 32, 33 to one or more storage devices 41, 42, 43, for example, via a push mechanism. The information about changes in compression techniques may be transmitted as an incremental update, indicating changes in the compression profile or data model.

[0149] The interface 22 may optionally be operable to receive information from the analysis device 50 and / or from the data sources 31, 32, 33 and / or from one or more storage devices 40. The received information may allow the decision device 20 to determine whether and how to modify the machine learning algorithm for selecting a compression technique based on the compression applied by the data source using an appropriate quality metric, such as compression and / or loss incurred during compression and / or the quality of the results generated by the analysis device 50. During ongoing operation of the IACS, power generation system or distribution system in which the data sources 31, 32, 33 are installed, the decision device 20 may use the received information to train the machine learning algorithm 24.

[0150] Alternatively or additionally, the user interface 26 may allow a transmission system operator (TSO) or a distribution system operator (DSO) to provide input that may be used, for example, during ongoing operation of the IACS, generation system, or distribution system to train the machine learning algorithm 24 executed by the decision device 20. The input received at the user interface 26 may indicate the quality (as perceived by the TSO or DSO) of the compression performed by the data sources 31, 32, 33.

[0151] The decision device 20 includes at least one integrated circuit 21. The at least one integrated circuit may include a processor, a microprocessor, a controller, a microcontroller, and an application-specific integrated circuit (ASIC) or any combination thereof. The at least one integrated circuit is operable to execute a machine learning module 24. The machine learning module 24 may include statistical methods, reinforcement learning, neural networks, convolutional neural networks, adversarial neural networks, classifier techniques, or other suitable machine learning implementations.

[0152] The machine learning module 24 may be initialized by training it using historical data. Alternatively or additionally, system specific expertise related to the IACS, power generation system or distribution system may be used. For example, the machine learning module 24 may be operable to use system knowledge (e.g., about the behavior and properties of the power grid) to reason about the extracted compression profiles and provide corrections where necessary. Alternatively or additionally, the machine learning module 24 may use statistical features (such as correlations and (physical) interdependencies in systems such as IACS and power systems) to improve models and profiles.

[0153] The decision device 20 includes at least one data storage device or memory 24. The data storage device or memory 24 is operable to store information 25 about a data model or profile for compression in a non-volatile manner. For a data source 31, 32, 33, the information 25 may include information about a compression technique to be employed by the data source 31, 32, 33 and information about parameters of the corresponding compression technique. For example, for run-length encoding (RLE), the parameters may indicate the minimum and maximum lengths of a repeating sequence for which RLE is performed. To further illustrate, for lossy compression, the parameters may define the degree of information loss that may result from the compression.

[0154] The data model or profile for compression may, but need not, be provided separately for each data source 31, 32, 33. For example, if the profiles of certain data sources 31, 32, 33 lack information, the available profile information may be used in combination to make up for the missing information in certain profiles. Alternatively or additionally, clustering may be performed on the profiles to ensure that the profiles can be used for the data without actually having detailed knowledge of the a priori profile of the data. Known clustering techniques may be used for this purpose. The machine learning module 24 may utilize system knowledge (e.g., about the behavior and properties of the power grid or IACS) to infer the missing information. The at least one integrated circuit 21 may use statistical features such as correlations and physical interdependencies to improve the model and profile 25.

[0155] The decision device 20 , the data sources 31 , 32 , 33 , one or more storage devices 41 , 42 , 43 and optionally the analysis device 50 may communicate via a suitable communication bus or network 19 .

[0156] During operation, the decision device 20 may assign signals to categories and models that are stored to represent various raw data collected at the data sources 31, 32, 33. The data may represent current, voltage, switch parameters (such as the switch state or operating state of a switch motor), transformer condition parameters (such as parameters related to the bushing or insulation of the transformer), or other data related to the operation of a power generation system, a distribution system, or an industrial system.

[0157] The decision device 20 is able to optimize compression by automatically selecting from a variety of compression techniques and determining compression parameters for each data source 31, 32, 33. The system can be self-optimizing, which also reduces engineering effort. Compression is not limited to Lempel-Ziv type compression algorithms and similar methods, and can exploit patterns / similarities between data in the same data set or even between different data sets from different sources. Compression is implemented using a machine learning based decision device 20 that can determine when to use which compression.

[0158] The decision device 20 may assign the signal to a category and a model 25. The model 25 may rely on various raw data of the data sources 31, 32, 33. Machine learning methods are used to update the model 25 with new data. This may form the basis for building or updating lossy and lossless profiles representing the data to be stored using, for example, statistical methods, neural networks or reinforcement learning. User preferences may be taken into account, for example, to enforce lossless compression for certain data sources.

[0159] Information about the frequency and type of queries output by analysis device 50 during data analysis in order to use the stored data may be taken into account when determining the best compression technique or compression parameters.

[0160] For example, for data that is more frequently queried by analysis device 50, a compression technique that allows the data to be decompressed more quickly may be preferred, even though this may result in a less noticeable reduction in storage space for the compressed data. For data that is less frequently queried by analysis device 50, a compression technique that provides a more noticeable reduction in data size during compression may be preferred.

[0161] Alternatively or additionally, the data owner may specify preferences in advance. This may be very useful in situations where data loss must be avoided to the greatest extent possible. For example, a TSO or DSO may wish to ensure that only data representing system quantities in a power grid or power generation system that could cause a catastrophic failure is compressed in a lossless manner.

[0162] In order to minimize the amount of control data related to the compression technique that needs to be exchanged between the multiple data sources 30 and the decision device 20 and / or between one or more storage devices 40 and the decision device 20, the model / profile can be pushed to the data source 31, 32, 33 responsible for the sensing or first digital processing step. In this case, only the update information related to the update of the model / profile needs to be transmitted to the processing and storage devices 41, 42, 43, 50. Similarly, the storage devices 41, 42, 43 only need to store changes in the profile and / or model. The update information can be transmitted as an incremental update, which includes, for example, information about parameter changes of the model / profile of the data source 31, 32, 33.

[0163] Compression and optionally encryption may be used to ensure privacy protection and confidentiality. For example, privacy protection may be ensured using the techniques described in "Processing Encrypted and Compressed Time Series Data" by M. Harvan, S. Kimoto, T. Locher, Y.-A. Pignolet, and J. Schneider, 37th IEEE International Conference on Distributed Computing Systems (ICDCS), 2017. Compression and (if applicable) encryption methods may be customized by the decision device 20 for data originating from the data sources 31, 32, 33 and for queries by the analysis device 50 during analysis.

[0164] The candidate compression techniques selected from the compression techniques used by the data sources 31, 32, 33 are not particularly limited. Examples of efficient compression techniques that can be used include lossy compression such as via piecewise regression (using techniques described, for example, in F. Eichinger, P. Efros, S. Kamouskos, and K. Bohm, "A Time-Series Compression Technique and its Application to the Smart Grid," VLDB Journal, 24(2), 193-218, 2015) or lossless compression using run-length encoding (using techniques described, for example, in F. Fusco, M. Vlachos, and X. Dimitropoulos, "RasterZip: Compressing Network Monitoring Data with Support for Partial Decompression," Proc. ACM Internet Measurement Conference (IMC), 2012).

[0165] Figure 2 is a schematic diagram illustrating a decision device 20, one or more storage devices 41, 42, 43, and an analysis device 50 used in conjunction with components of a power distribution system or a power generation system. For example, a bay of a substation may include: switches QB1, QB2, QC2, AQ1, QB9, and QC9, transformers BC1, BC2 for current sensing, and transformer BA1 for voltage sensing. The sensor devices 31, 32, 33 are the first components that provide digital data outputs and apply a compression technique determined based on the machine learning method of the decision device 20.

[0166] The decision device 20 is adapted to automatically select and improve the compression applied by the data sources 31, 32, 33. The decision device 20 may employ machine learning methods to select and improve the compression.

[0167] Figure 3 is a flow chart of a method 60 according to an embodiment. The method 60 may be executed by the decision device 20.

[0168] At step 61, information about the data sources 31, 32, 33 is received. This information may be obtained by user input or by analyzing a configuration file. The configuration file may be a standardized configuration description or a proprietary configuration description of an IACS, a power distribution system or a power generation system.

[0169] At step 62, a machine learning algorithm is executed to determine appropriate compression techniques and / or compression parameters for the data sources 31, 32, 33. The machine learning algorithm executed by the decision device 20 may continue to learn during operation of the IACS, power distribution system, or power generation system and / or during data analysis performed by the analysis device 50.

[0170] At step 63, update information is generated and transmitted. The update information may indicate changes in a configuration file or data model used by the data source 31, 32, 33 for compression. The update information may be transmitted to the data source 31, 32, 33 and one or more storage devices 41, 42, 43 via a push mechanism. The update information may include information about the time when the update of the compression technology takes effect.

[0171] The data sources 31 , 32 , 33 and the one or more storage devices 41 , 42 , 43 may use the update information to update the locally stored configuration files or models.

[0172] Figure 4 is a schematic block diagram of a system according to an embodiment. A data source 34 may provide data indicating at least one parameter of an IACS, a power generation system, or a power distribution system. The data source 34 may include at least one integrated circuit 35. The at least one integrated circuit may include a processor, a microprocessor, a controller, a microcontroller, and an application specific integrated circuit (ASIC), or any combination thereof.

[0173] The data source 34 may include a non-volatile storage device or memory device that locally stores information about a data model or profile 36 for data compression. The data model or profile may determine what kind of compression the data source 34 applies to the data before outputting the data. The data model or profile may determine the parameters of the corresponding compression algorithm determined by the decision device 20.

[0174] During operation, in response to update information from the decision device 20, at least one integrated circuit 35 can update the locally stored model or configuration file. For example, the parameters or even the type of compression technology can be changed according to the received update information. The updated model or configuration file can be effective and applied from the time specified by the decision device 20.

[0175] The storage device 44 may provide data indicative of at least one parameter of the IACS, power generation system, or power distribution system. The storage device 44 may include at least one integrated circuit 45. The at least one integrated circuit may include a processor, a microprocessor, a controller, a microcontroller, and an application specific integrated circuit (ASIC), or any combination thereof.

[0176] The storage device 44 may include a non-volatile storage device or memory device that locally stores information about a data model or profile 46 used by the data source 34 for data compression. The data model or profile may determine what kind of compression the data source 34 applies to the data before outputting the data. The data model or profile may determine the parameters of the corresponding compression algorithm determined by the decision device 20. The data model or profile may determine, for example, a decompression algorithm that must be applied to the stored compressed data during data analysis.

[0177] During operation, in response to update information from the decision device 20, the at least one integrated circuit 45 can update the locally stored model or configuration file. For example, the parameters or even the type of compression technique can be changed according to the received update information. The updated model or configuration file can be recorded in association with the time specified by the decision device 20.

[0178] The update information transmitted by the decision device 20 using, for example, a push mechanism may be incremental update information. For example, the decision device 20 may transmit only changes to the configuration file or model to the data source 34 and the storage device 44.

[0179] It should be understood that each of the plurality of data sources 31, 32, 33 and each of the several storage devices 41, 42, 43 (if present) may be as described with reference to Figure 4 Operate as described.

[0180] Figure 5 is a flow chart of a method 70 according to an embodiment. The method 70 may be executed by the decision device 20.

[0181] At step 71, feedback data regarding previous compression decisions may be received. The feedback data may be automatically provided by the analysis device 50, one or more storage devices 40, and / or data sources 31, 32, 33. The feedback data may include quality metrics or information that allows the decision device 20 to determine a numerical value indicating the quality of the compressed data for subsequent analysis. Alternatively or additionally, the feedback data may be input via the user interface 26.

[0182] The feedback data may be used to train a machine learning algorithm at step 72. The training may include neural network training, reinforcement learning, or other machine learning methods.

[0183] At step 73, it is determined whether a trigger condition is met for transmitting the updated compression decision to the data source 31, 32, 33, one or more storage devices 40, and / or the analysis device 50. If the trigger condition is not met, the method may return to step 71. If the trigger condition is met, then at step 74, update information related to the change in compression technology may be generated and pushed to the data source 31, 32, 33, one or more storage devices 40, and / or the analysis device 50.

[0184] Constraints may be used when determining compression techniques and parameters to be used by data sources 31, 32, 33. Constraints may be automatically imposed by decision device 20 using, for example, expert knowledge about the various data sources and / or information included in configuration files of the IACS, power generation system, or power distribution system. Constraints may be input by an operator.

[0185] Figure 6 The selection from candidate compression techniques 91 to 98 based on the imposed constraints is illustrated. For example, some compression techniques 91, 92 may be suitable for lossless compression of time domain data. Other compression techniques 93, 94 may be suitable for lossless compression of frequency domain data. Other compression techniques 95, 96 may be suitable for lossy compression of time domain data. Other compression techniques 97, 98 may be suitable for lossy compression of frequency domain data. Possible compression techniques may be selected from a subset of all candidate compression techniques 91 to 98, depending on whether lossless or lossy compression is to be performed and / or on whether the data is time domain data or frequency domain data. Groups 81, 82, 83, 84 of compression algorithms may be defined.

[0186] Devices and methods according to embodiments can use machine learning methods to compress data for transmission, storage, and / or analysis in a manner that can support detailed long-term analysis while minimizing the physical storage required. This allows operators to collect more data (by reducing the communication bandwidth required to transmit data from sensors to storage systems) and store that data more efficiently for much longer time intervals. Therefore, data compressed using machine learning techniques can enable new services and products.

[0187] Machine learning methods can optimize compression by automatically selecting from a variety of compression methods and / or determining compression parameters for each data set or data stream. In this way, the system can be self-optimizing, which also reduces engineering workload. Compression is not limited to Lempel-Ziv type compression algorithms and similar methods. Compression can exploit patterns / similarity between data in the same data set or even between different data sets. Methods and devices according to embodiments allow compression algorithms to be implemented using a machine learning based decision maker that determines when to use which compression to make the solution unique.

[0188] Long term storage and generation of profile management may include analysis of various features in the time and / or frequency domains. Different types of methods may be used to store data in an efficient manner, including compression, signal analysis and processing, and signal correlation for representative statistics, classification algorithms for assigning signals to categories and models that are stored to represent various raw data.

[0189] Machine learning methods can be used to update models trained on available past data with new data. This can form the basis for building lossy and lossless profiles (depending on the preferences of the data owner) representing the data to be stored (e.g., using statistical methods, or neural networks and reinforcement learning). Since such a process provides a variety of adjustable parameters, this step can be optimized for the frequency and type of queries made during data analysis tasks to use the stored data. Alternatively or additionally, the data owner can specify preferences in advance. This may be meaningful in situations where data loss must be avoided to the greatest extent possible.

[0190] The method and system according to the embodiment allow compression to be optimized. There are different compression algorithms, and the focus of these compression algorithms is to store data in a more compact manner. For example, a compression technique of the classic Lempel-Ziv type can be used, which performs lossless compression on digital data in a universal manner. However, there are also some compression techniques that actually lose information, but this is acceptable in many cases. Embodiments of the present invention provide methods or modules that can automatically select between several (predefined) methods for compression and parameterize these algorithms accordingly. Compression can then be performed on other devices in the system, for example, performed near the source of the data to ensure communication bandwidth and / or performed at a storage device.

[0191] The methods and systems may modify the decision about which compression algorithm to use and / or the parameterization of the algorithm in a regular manner to further optimize the system given the assumption that the data source may change over time.

[0192] Figure 7 is a flow chart of a method 65 that can be automatically performed. Steps 61 to 63 can be as described in reference Figure 3 The method is implemented as explained to automatically determine which of several compression algorithms is to be used and / or to parameterize the compression algorithm.

[0193] At step 64, it is determined whether a trigger for updating the compression decision is met. The trigger at step 64 may be the expiration of a timer, so that the determination of the compression algorithm to be used and its parameterization may be repeated in a regular manner (e.g., periodically). The trigger of step 64 may be a trigger event, so that the determination of the compression algorithm to be used and its parameterization may be repeated in an event-driven manner.

[0194] Therefore, decisions on which compression algorithm to use and / or parameterization of the algorithm are made in a regular manner in order to further optimize the system under the assumption that the data source may change over time.

[0195] The data from the data source may be time series data. An example of such time series data is, but is not limited to, power consumption as a function of time. For example, the time series data may be electricity consumption of a household.

[0196] One or more candidate compression algorithms may correlate time series data from different data sources (e.g., different households). One or more time series data sets may be formed by grouping time series data from different data sources based on correlation. The time series data may be grouped into one or more sets such that all time series data in a set are similar enough to satisfy a correlation criterion. For example, the cross-correlation between any pair of time series data grouped into a set may satisfy a threshold criterion (e.g., may exceed a threshold).

[0197] One or more candidate compression algorithms may be implemented in such a way that for any time series data set grouped as described above, only time series data originating from one of the data sources is transmitted and / or stored. For other data sources grouped as belonging to the same set, the compression algorithm may provide an indicator that identifies the time series data originating from these data sources as being associated with the time series data of one of the data sources in the group whose time series data is stored.

[0198] That is, the compression mechanism can associate time series from different data sources with each other and transmit only some of them (in a further compressed format) and specify that the consumption patterns of others are very similar (with some information loss). This avoids transmitting all the individual time series data curves.

[0199] The compression mechanism can additionally use clustering techniques to classify different time series data (e.g., consumption patterns) into multiple categories. These categories can vary as a function of time, such as as a function of the day of the week, as a function of the month, etc. For example, there can be X categories for Monday, Y categories for Tuesday, etc. The compression algorithm can then identify the time series data (e.g., consumption patterns) for a particular day and then transmit the category number instead of the time series data for transmission and / or storage. This results in compact storage.

[0200] The machine learning algorithm can learn criteria for grouping time series data into sets and / or for classifying signals. For example, the machine learning algorithm can receive quality feedback during ongoing operation of the system and / or during data analysis. The quality feedback can include professional feedback received via an interface and / or quantitative feedback automatically generated by the system. The machine learning algorithm can adjust the relevance criteria for associating numbers and types of data and / or categories during ongoing operation to learn the best way to select a compression algorithm and its parameterization.

[0201] Compression techniques involving information loss should be selectively employed only when the information loss is acceptable.

[0202] The machine learning algorithm can automatically decide whether to use lossy or lossless compression. For example, the machine learning algorithm can determine the granularity and / or format of the required information, which can be inferred from the data itself and / or from metadata. For example, if the data is image data or other data with a lot of redundant information that is not marked as critical, lossy compression can be selected. If the data is identified as including critical fields that cannot be lost, lossless compression is selected.

[0203] Figure 8 Exemplary data 91 to 94 from various data sources are illustrated. The data 91 to 94 may represent time series data, respectively. A candidate compression algorithm may calculate the cross-correlation between the time series data 91 to 94 and compare the result with a threshold.

[0204] The compression algorithm may cause only one of the time series data deemed to meet the relevance criteria (eg, time series data 91 from the first source) to be transmitted and / or stored (optionally after further processing).

[0205] The compression algorithm may cause the transmission and / or storage of an indicator instead of the transmission and / or storage of data 92, 93 from other sources. The indicator may indicate that the data 92, 93 is associated with the time series data 91.

[0206] The machine learning algorithm can adjust the parameterization of the compression algorithm based on the received feedback, as already referenced e.g. Figure 5 Explained.

[0207] For example, a threshold value for determining whether data 94 from a source meets a relevance criterion that allows data 94 to be grouped together with time series data 91 from another source (such that transmission and / or storage of one of the time series data 91, 94 may be replaced by transmission and / or storage of an indicator indicating similarity) may be learned during ongoing operation. This may be accomplished by increasing and / or decreasing a threshold value used in a relevance threshold comparison based on received feedback.

[0208] For signal processing, it may be beneficial to compress either the time domain or the frequency domain over a period of time. In these methods and systems, a machine learning algorithm may learn whether compression in the time domain or compression in the frequency domain is more beneficial. This may be done based on received feedback, as has been described with reference to e.g. Figure 5 Explained.

[0209] Some compression algorithms can be adapted to combine the signal with other signals in the system (as already mentioned in Figure 8 The correlation values ​​can be used to identify signals that are similar enough to meet the correlation threshold criteria. It is not necessary to transmit and / or store every signal in a set of signals that are considered similar. The machine learning algorithm learns over time and adjusts which method to use over time. This can be done based on the feedback received, as has been referenced, for example, Figure 5 Explained.

[0210] Not only signals, but also different profile information can be combined to make up for the missing information in certain profiles. Clustering of profiles can be used to ensure that the system can use profiles for the data without actually knowing the profile of the data in detail. To this end, machine learning algorithms can use system knowledge (e.g. about the behavior and properties of the power grid) to make inferences about the extracted profiles and provide corrections when necessary. Statistical features such as correlations and / or (physical) interdependencies can be used to improve models and profiles.

[0211] In order to minimize the amount of data to be exchanged, the model / profile can be pushed to the device responsible for sensing or the first digital processing step. In this case, only the data that updates the model / profile needs to be transmitted to further processing and storage facilities. The storage device only needs to store changes to the profile and / or model. By the method, device and system according to the embodiments, various effects and advantages are obtained. For example, the storage space requirements and the required communication bandwidth are reduced. The automatic selection of the compression algorithm reduces the engineering workload.

[0212] Although the invention has been described in detail in the drawings and the foregoing description, such description should be considered illustrative or exemplary rather than restrictive. Variations of the disclosed embodiments can be understood and implemented by those skilled in the art in practicing the claimed invention based on a study of the drawings, the present disclosure and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a or an" does not exclude a plurality. The simple fact that certain elements or steps are stated in different claims does not indicate that a combination of these elements or steps cannot be used to advantage, and in particular, apart from actual claim dependencies, any other meaningful claim combination should be considered to fall within the scope of the present disclosure.

Claims

1. A method for facilitating storage of data from a plurality of data sources (31-34) of a system, the system being an industrial automation control system (IACS), a power distribution system or a power generation system, the method comprising: determining, using at least one integrated circuit (21) of a decision device (20) to apply a compression technique to the data, wherein the decision device (20) executes a machine learning algorithm to determine the compression technique based on a data source (31-34) from which the data originates, wherein determining the compression technique includes executing the machine learning algorithm to generate an update to a data model or profile (25, 36, 46) associated with the data source (31-34); and Applying the compression technique determined for a data source (31-34) to data from the data source (31-34) by the decision device (20) by generating update information related to the update of the data model or configuration file (25, 36, 46), and transmitting the update information to the data source (31-34), for which the update of the data model or configuration file (25, 36, 46) has been determined; By means of the data source (31-34), the compression model or configuration file (36) stored locally in the data source (31-34) is updated based on the received update information, thereby generating an updated compression model or configuration file, and Compression is performed based on the updated compression model or configuration file via the data source (31-34).

2. The method of claim 1, wherein: Information about changes in the compression technique to be adopted is provided by the decision device (20) to corresponding data sources (31-33) and / or storage devices (41-44) via a push mechanism.

3. The method of claim 2, wherein: Information about changes in the compression technique is transmitted as incremental updates and indicates changes to the compression profile or data model.

4. The method according to any one of claims 1 to 3, wherein: Determining the compression technique includes executing the machine learning algorithm to determine which compression technique to apply from a plurality of candidate compression techniques (91-98).

5. The method according to any one of claims 1 to 3, wherein: Determining the compression technique includes executing the machine learning algorithm to determine at least one parameter of the compression technique.

6. The method of any one of claims 1 to 3, further comprising automatically repeating the steps of determining the compression technique and applying the compression technique.

7. The method according to claim 6, wherein: The steps of determining the compression technique and applying the compression technique are repeated in a regular manner.

8. The method of claim 1, wherein: The data sources (31-34) to which the update information is transmitted include sensor devices or merging units.

9. The method according to any one of claims 1 to 3, further comprising transmitting the update information to at least one storage device (41-44) through the decision device (20), and the at least one storage device stores data from the data source (31-34).

10. The method according to any one of claims 1 to 3, wherein: The machine learning algorithm determines the compression technique, subject to constraints that depend on the data source.

11. The method according to any one of claims 1 to 3, further comprising: The machine learning algorithm is trained during operation of the IACS, power distribution system, or power generation system.

12. The method of claim 11, wherein: Training the machine learning algorithm includes learning whether compression in the time domain or compression in the frequency domain is more beneficial.

13. The method according to any one of claims 1 to 3, wherein: The determined compression technique determines the correlation of time series data from different data sources, and / or, The determined compression technology includes classification or clustering technology.

14. The method of claim 13, wherein: The method further comprises transmitting and / or storing information depending on the determined correlation.

15. The method of claim 13, wherein: The method further comprises transmitting and / or storing information indicative of the categories or clusters, and / or wherein the classification or clustering is time dependent.

16. A decision device (20) adapted to facilitate storage of data from a plurality of data sources (31-34) from an industrial automation control system (IACS), a power distribution system or a power generation system, the decision device (20) comprising: at least one interface (22) adapted to be communicatively coupled to the plurality of data sources (31-34); as well as At least one integrated circuit (21) operable to determining a compression technique to be applied to the data, wherein the decision device (20) executes a machine learning algorithm to determine the compression technique based on a data source (31-34) from which the data originates, wherein determining the compression technique comprises executing the machine learning algorithm to generate an update to a data model or profile (25, 36, 46) associated with the data source (31-34), and The compression technique determined for the data source (31-34) is applied to the data from the data source (31-34) by generating update information related to the update of the data model or configuration file (25, 36, 46), and transmitting the update information to the data source (31-34), wherein the update of the data model or configuration file (25, 36, 46) has been determined for the data source, so that the data source (31-34) Based on the received update information, the compression model or configuration file (36) stored locally in the data source (31-34) is updated to generate an updated compression model or configuration file, and Compression is performed based on the updated compression model or profile.

17. The decision device (20) of claim 16, wherein: The decision device (20) is adapted to provide information about changes in compression techniques to be employed to corresponding data sources (31-33) and / or storage devices (41-44) via a push mechanism.

18. The decision device (20) of claim 17, wherein: The decision device (20) is adapted to transmit information of a change in compression technique as an incremental update, the information of the change in compression technique indicating a change in a compression profile or a data model.

19. The decision device (20) as claimed in any one of claims 16 to 18, wherein: The at least one integrated circuit (21) is operable to generate control information that applies a compression technique determined for a data source (31-34) to the data prior to storing the data.

20. A system, the system being an industrial automation control system (IACS), a power distribution system or a power generation system, the system comprising: Multiple data sources (31-34); at least one storage device (41-44) for storing compressed data originating from the plurality of data sources (31-34); as well as A decision device (20) as claimed in any one of claims 16 to 19.

21. The system of claim 20, wherein: The system is operable to perform the method of any one of claims 1 to 15.

Images