Data processing device for multi-layered data management with automated control and provenance tracking

Abstract

Data processing device for multi-layered data management, comprising a storage unit with a first data layer for receiving unprocessed raw data, a second data layer for storing transformed data, and a third data layer for storing validated data sets. a processing unit which is set up to transfer and process data between the data layers according to predefined transformation rules, as well as a control unit which is set up to monitor each data processing step based on defined test parameters and to generate structured origin information for each data record and store it in the storage unit, wherein the origin information includes at least information on source data, processing steps and target layer and enables complete traceability across all data layers.