8 results about "Extract, transform, load" patented technology

The invention discloses a scientific research project management system based on big data processing, and relates to the technical field of project management. The data acquisition module comprises a plurality of key modules, the data acquisition module is connected with multiple sources, acquires project data greater than or equal to 10Hz, cleans the project data, transmits the cleaned project data to a storage center adopting a Hadoop distributed architecture through ETL (Extract Transform Load), and establishes an index and compresses and backs up Snappy according to multiple dimensions; the data analysis module performs clustering and mining association based on a Spark framework; the project progress monitoring module is used for capturing key nodes, displaying the key nodes by using a Gantt chart, and lagging 20% early warning pushing; the personnel management module records personnel information, intelligently recommends and evaluates performance; the fund management module tracks the fund and gives out a report early warning when the fund deviates from the budget by 15%; and the achievement management module collects and arranges achievements, mines and compares to promote transformation. According to the system, scientific research projects are comprehensively and intelligently managed, data are accurately collected and efficiently analyzed, the progress is monitored in real time, personnel funds are reasonably managed and controlled, achievement transformation is assisted, and scientific research benefits are improved.

Provided are systems, methods, and apparatuses for data analytics based on zero extract transform load (ETL). In one or more examples, the systems, devices, and methods include generating a data request for at least a subset of data stored in a database; converting the data request to an object storage request; determining, based on parsing the object storage request, that the object storage request includes an extract transform load (ETL) request; creating an ETL command based on the object storage request; routing the ETL command to a storage device based on the storage device comprising at least one ETL function requested in the data request; and determining information from result data that is received from the storage device in response to routing the ETL command to the storage device.

Provided are systems, methods, and apparatuses for data analytics based on zero extract transform load (ETL). In one or more examples, the systems, devices, and methods include generating a data request for at least a subset of data stored in a database; converting the data request to an object storage request; determining, based on parsing the object storage request, that the object storage request includes an extract transform load (ETL) request; creating an ETL command based on the object storage request; routing the ETL command to a storage device based on the storage device comprising at least one ETL function requested in the data request; and determining information from result data that is received from the storage device in response to routing the ETL command to the storage device.

Various embodiments described herein relate to systems and methods for processing data received from various data sources in a facility. In this regard, an input is received corresponding to at least one change in one or more assets. Configuration data associated with the one or more assets is updated based on the at least one change. Further, the updated configuration data is persisted at one or more data loaders. At least one Extract Transform Load (ETL) process is triggered corresponding to the updated configuration data to generate one or more aggregates associated with the one or more assets. As a result, Key Performance Indicator (KPI) metrics associated with the one or more assets is calculated.

Systems and methods for graph-based Extract Transform Load (ETL) code generation are disclosed. A natural language request to create ETL code according to a description of the ETL code in the natural language request may be received. A graph representation of the ETL code may be generated that includes nodes corresponding to operations to obtain data from data sources, perform transformations to at least the portion of the data, and store the data in destinations. To create the graph representation, interactions with a generative machine learning model may identify the operations for the ETL code in the description in the natural language request and construct the graph representation that includes the nodes corresponding to the operations along with connections between the nodes identified according to the description in the natural language request. A response to the natural language request may be provided via the interface based on the graph representation.

Provided are systems, methods, and apparatuses for data analytics based on zero extract transform load (ETL). In one or more examples, the systems, devices, and methods include generating a data request for at least a subset of data stored in a database; converting the data request to an object storage request; determining, based on parsing the object storage request, that the object storage request includes an extract transform load (ETL) request; creating an ETL command based on the object storage request; routing the ETL command to a storage device based on the storage device comprising at least one ETL function requested in the data request; and determining information from result data that is received from the storage device in response to routing the ETL command to the storage device.

A system for data warehouse workload transformation is disclosed. A data extraction module receives job metadata from one or more extract transform load applications, extracts domain logic information from the job metadata. A canonical model generation module creates one or more relational models from the domain logic information, generates a canonical model for the job metadata by merging the one or more relational models. A model optimization module creates a query execution plan using an optimization technique, passes the canonical model through one or more optimization layers, and optimizes the query execution plan to execute common logic at a single point of time for removing one or more redundant operations. A code generation module generates an output code corresponding to the predefined target environment from an optimized canonical model, upon mapping of the one or more entities, for the workload transformation.

Provided are systems, methods, and apparatuses for data analytics based on zero extract transform load (ETL). In one or more examples, the systems, devices, and methods include generating a data request for at least a subset of data stored in a database; converting the data request to an object storage request; determining, based on parsing the object storage request, that the object storage request includes an extract transform load (ETL) request; creating an ETL command based on the object storage request; routing the ETL command to a storage device based on the storage device comprising at least one ETL function requested in the data request; and determining information from result data that is received from the storage device in response to routing the ETL command to the storage device.