Dynamically including metadata configuration into logical model

Abstract

Embodiments generate changes to the logical model. Embodiments receive changes in the configuration file, the changes including declarative configuration, extracting and loading the changes into the database and updating the corresponding database model. Embodiments generate a first logical model representing a database model and generate a second logical model including changes. Embodiments automatically generate a compiled visualization image in the container from the second logical model using the declarative configuration, wherein the visualization image is adapted to be used by the business intelligence system to provide a visualization of the data incorporating the changes.

Description

Technical Field

[0001] One embodiment is directed generally to a computer system, and more particularly to using metadata configuration to facilitate visualization of underlying data of the computer system. Background Art

[0002] A data warehouse tool or business intelligence platform, such as Oracle Business Intelligence Enterprise Edition ("OBIEE"), Oracle Analytics Cloud ("OAC"), or Oracle Analytics Server ("OAS"), allows for the definition of a logical database schema on top of the physical database tables. The logical database schema includes a schema that describes the relationships between the underlying physical database tables. For example, it is possible to define a star schema that describes the relationships between the tables and connects logically related tables together.

[0003] Understanding the relationships between underlying database tables is essential to efficiently retrieving data from the database. For example, in order to form database queries that provide meaningful results, it is necessary to specify in each query the tables to search and the filtering criteria used to narrow the results to be presented from the combination of tables. Without knowing the relationships between the tables and the structure of the data within the tables, the queries will produce results that are not meaningful to the end user. Summary of the invention

[0004] An embodiment generates changes to a logical model. An embodiment receives changes in a configuration file, the changes including a declarative configuration, extracts the changes and loads the changes into a database and updates a corresponding database model. An embodiment generates a first logical model representing the database model and generates a second logical model including the changes. An embodiment automatically generates a compiled visualization image from the second logical model in a container using the declarative configuration, wherein the visualization image is suitable for use by a business intelligence system to provide a visualization of the data incorporating the changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Further embodiments, details, advantages and modifications will become apparent from the following detailed description of embodiments in conjunction with the accompanying drawings.

[0006] Figure 1 is a block diagram of a BI system including BI functionality for an enterprise network and including dynamic artifacts including functionality according to an embodiment of the present invention.

[0007] Figure 2 is a computer server / system in the form of an embodiment of the present invention Figure 1 A block diagram of one or more components of a BI system.

[0008] Figure 3 Diagram showing the fact table for the daily sales fact.

[0009] Figure 4 Figure 1.2 shows a graph filled with measurement data. Figure 3 fact table.

[0010] Figure 5 Pictured Figure 3 The dimension table concept of the product dimension in the fact table of Figure 6 The diagram shows a dimension table populated with attributes of the product dimension.

[0011] Figure 7 The diagram shows Figure 3 Example of a star schema generated for the sales fact table and its associated dimension tables in .

[0012] Figure 8 According to the embodiment Figure 2 A flow chart of the functionality of the dynamic metadata configuration including module when performing the dynamic inclusion of metadata configuration into the logical model functionality.

[0013] Fig. 9 According to the embodiment Figure 2 A flowchart of the detailed functionality of the dynamic metadata configuration including module when performing dynamic inclusion of metadata configuration into logical model functionality. DETAILED DESCRIPTION

[0014] One embodiment automatically generates metadata configurations, which are then included in a logical model, such as a business intelligence repository database file ("RPD file"), in the form of a catalog file. The metadata configurations can be used to automatically create visualizations for many underlying different tenant data repositories.

[0015] Embodiments flexibly include metadata configurations (i.e., configured files) such as catalog files into RPDs for Oracle Analytics Cloud ("OAC"), or any other business intelligence system that implements RPD files. The Oracle BI repository (RPD file) stores BI server configuration metadata specific to a BI Server customer. The metadata defines the logical schema, physical schema, physical to logical mappings, aggregate table navigation, and other constructs. RPD files have historically been compressed files that are difficult to update or maintain, especially for multi-tenant applications. Therefore, embodiments generate and store the RPD separately from the OAC instance (e.g., in Figure 1 120 instead of 125). This improves the user's ability to upgrade OAC versions in a more modular manner.

[0016] In an embodiment, metadata configuration or configuration metadata is the rules that modify and tune the repository database to indicate updates, such as: new visualizations, new data elements to render, aggregations, etc.

[0017] Business Intelligence ("BI") platforms, such as Oracle Analytics or BI systems, can be extended Business Intelligence Enterprise Edition (“OBIEE”), Analytics Cloud (“OAC”) or The OAS Data Visualization (DV) graphical user interface (GUI) is a graphical user interface that allows you to create custom visualizations based on the logical model.

[0018] Embodiments flexibly include catalog files into the RPD of OAS or another BI application using RPD. RPDs are files in OBIEE, OAC, OAS that store metadata and rules for creating and presenting logical database schemas. RPDs have historically been compressed files that are difficult to update or maintain, especially for multi-tenant applications. For example, known processes for updating catalog files for each of many tenants in a multi-tenant instance are manual, and thus the workload increases as the number of tenants increases.

[0019] In response, an embodiment generates an RPD and stores it separately from the analysis instance (i.e., in Figure 1 120, rather than BI platform / tool / system 125). This improves the ability of users to upgrade versions of OAS or any other BI application in a more modular manner, since RPDs are typically tied to a single owner's analytics instance. In contrast, embodiments use a separate storage mechanism where RPDs are stored for use by different owners / users. Separate storage makes it possible for administrators to update a user's configuration before the user receives an instance to use. For example, one instance is a BI system (e.g., Figure 1 Server 125) and an enterprise network (for example, Figure 1 A connection to the enterprise network 115).

[0020] Reference will now be made in detail to embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to one of ordinary skill in the art that the present disclosure can be practiced without these specific details. In other cases, well-known method processes, components, and circuits are not described in detail to avoid unnecessarily obscuring aspects of the embodiments. Whenever possible, similar reference numerals will be used for similar elements.

[0021] Figure 1 is a block diagram of a BI system 100 according to an embodiment of the present invention, the system including BI functionality for an enterprise network and including dynamic artifacts including functionality. In one embodiment, the system 100 includes a business intelligence configuration system 105 connected to an enterprise network 115 via the Internet 110 (or another suitable communication network or combination of networks). In one embodiment, the business intelligence configuration system 105 can be an OBIEE, OAC or OAS service implementation, or any other BI system that implements RPD files. In one embodiment, the business intelligence configuration system 105 includes various systems and components, such as a dynamic metadata configuration including server / module 120, a business intelligence server 125, (one or more) data repositories 130, and a web interface server 135.

[0022] In one embodiment, the dynamic metadata configuration inclusion server 120 includes one or more components configured to implement the methods, functions, and features described herein associated with dynamically including metadata configuration into a logical model.

[0023] In one embodiment, the business intelligence server / system 125 may include business intelligence applications and functions for retrieving, analyzing, mining, visualizing, transforming, reporting, and otherwise using data associated with the operation of the business. In one embodiment, the business intelligence server 125 may include a data collection component that captures data associated with the operation of the business and records it in a data repository, such as (one or more) data repository 130. In one embodiment, other business intelligence servers 125 may also include a user management module for managing user access to the business intelligence configuration system 105.

[0024] Each component of the business intelligence configuration system 105 is logically configured to perform the functions that the component is described to perform. In one embodiment, the components of the business intelligence system 105 may each be implemented as a collection of one or more software modules executed by one or more computing devices specifically configured for such execution. In one embodiment, the components of the business intelligence configuration system 105 are implemented on one or more hardware computing devices. In one embodiment, the components of the business intelligence configuration system 105 are each implemented by a dedicated computing device. In one embodiment, the components of the business intelligence configuration system 105 are implemented by a common (or shared) computing device, even if Figure 1 In one embodiment, the business intelligence configuration system 105 may be hosted by a dedicated third party, such as in an Infrastructure as a Service ("IAAS"), Platform as a Service ("PAAS"), or Software as a Service ("SAAS") architecture. In one embodiment, the components of the business intelligence configuration system 105 communicate with each other via electronic messages or signals. These electronic messages or signals may be configured as calls to functions or procedures that access features or data of the components, such as, for example, application programming interface ("API") calls. Each component of the business intelligence configuration system 105 may parse the content of a received electronic message or signal to identify a command or request that the component can execute, and in response to identifying the command, the component will automatically execute the command or request.

[0025] The enterprise network 115 may be associated with an enterprise. For simplicity and clarity of explanation, the enterprise network 115 is represented by a field local area network 140 to which one or more personal computers 145 or servers 150 are operably connected, and one or more remote user computers 155 or mobile devices 160 connected to the enterprise network 115 via the Internet 110. Each personal computer 145, remote user computer 155, or mobile device 160 is generally dedicated to a specific end user, such as an employee or contractor associated with the enterprise, but such dedicatedness is not required. The personal computer 145 and the remote user computer 155 may be, for example, a desktop computer, a laptop computer, a tablet computer, or other devices capable of connecting to the local area network 140 or the Internet 110. The mobile device 160 may be, for example, a smart phone, a tablet computer, a mobile phone, or other devices capable of connecting to the local area network 140 or the Internet 110 via a wireless network (such as a cellular phone network or Wi-Fi). Users of the enterprise network 115 interface with the business intelligence configuration system 105 across the Internet 110 (or another suitable communication network or combination of networks).

[0026] In one embodiment, a remote computing system, such as a remote computing system of an enterprise network 115, can access information or applications provided by the business intelligence configuration system 105 through a web interface server 135. In one embodiment, the remote computing system can send a request to and receive a response from the web interface server 135. In one example, access to the information or application can be achieved by using a web browser on a personal computer 145, a remote user computer 155, or a mobile device 160. For example, these computing devices 145, 155, 160 of the enterprise network 115 can request and receive a web-based graphical user interface ("GUI") for dynamically displaying visualization data for use in the business intelligence configuration system 105. In one example, the web interface server 135 can present HTML code to the personal computer 145, the server 150, the remote user computer 155, or the mobile device 160 so that these computing devices render into the GUI of the business intelligence configuration system 105. In another example, messages may be exchanged between the web interface server 135 and the personal computer 145, server 150, remote user computer 155, or mobile device 160, and may take the form of Remote Representational State Transfer ("REST") requests, such as using JavaScript Object Notation ("JSON") as a data exchange format, or Simple Object Access Protocol ("SOAP") requests to and from an XML server. For example, the computers 145, 150, 155 of the enterprise network 110 may request information included in a custom column, or may request information derived at least in part from information included in a custom column (such as analysis results based at least in part on the custom column).

[0027] In one embodiment, data repository 160 includes one or more operational databases configured to store and provide a wide range of information related to the operation of the business, such as data about enterprise resource planning, customer relationship management (including customer data such as account numbers, demographic information, third-party data, etc.), finance and accounting, order processing, time and billing, inventory management and distribution, employee management and payroll, calendaring and collaboration, product information management, demand and material requirements planning, procurement, sales, sales force automation, marketing, e-commerce, supplier management, supply chain management, product lifecycle management, descriptions of hardware assets and their status, production output, shipping and tracking information, and any other information collected by the business. Such operational information may be stored in the operational databases in real time as the information is collected. In one embodiment, data repository 160 includes a mirror or replica database of each operational database, which may be used for disaster recovery, or to assist in supporting read-only operations. In one embodiment, the operational databases are In some example configurations, the data repository(s) 160 may use one or more It can be implemented using Exadata compute form factors, network attached storage ("NAS") devices, and / or other specialized server devices.

[0028] Figure 2 is a computer server / system 100 in the form of an embodiment of the present invention. Figure 1 1 is a block diagram of one or more components of the BI system 100. Although shown as a single system, the functionality of the system 100 can be implemented as a distributed system. In addition, the functionality disclosed herein can be implemented on separate servers or devices that can be coupled together through a network. In addition, one or more components of the system 100 may not be included. Figure 2 One or more components may also be used to implement Figure 1 any element of .

[0029] The system 100 includes a bus 12 or other communication mechanism for transmitting information, and a processor 22 coupled to the bus 12 to process the information. The processor 22 can be any type of general or special purpose processor. The system 10 also includes a memory 14 for storing information and instructions to be executed by the processor 22. The memory 14 can be composed of any combination of random access memory ("RAM"), read-only memory ("ROM"), static storage devices such as magnetic or optical disks, or any other type of computer-readable medium. The system 100 also includes a communication device 20, such as a network interface card, to provide access to a network. Therefore, a user can interface with the system 100 directly, or remotely through a network interface, or in any other way.

[0030] Computer readable media can be any available media that can be accessed by processor 22, including volatile and nonvolatile media, removable and non-removable media, and communication media. Communication media may include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and include any information delivery media.

[0031] Processor 22 is also coupled to a display 24, such as a liquid crystal display ("LCD"), via bus 12. A keyboard 26 and cursor control device 28, such as a computer mouse, are also coupled to bus 12 to enable a user to interface with system 10.

[0032] In one embodiment, memory 14 stores software modules that provide functionality when executed by processor 22. These modules include an operating system 15 that provides operating system functionality for system 10. These modules also include a dynamic metadata configuration inclusion module 16 that provides functionality for dynamically including metadata configuration into a logical model, as well as all other functionality disclosed herein. System 10 may be part of a larger system. Thus, system 10 may include one or more additional functional modules 18 to include additional functionality, such as any other BI functionality disclosed above (e.g., OBIEE, OAS, etc.). A file storage device or database 17 is coupled to bus 12 to provide centralized storage for modules 16 and 18, as well as to implement Figure 1 Data repository 130. In one embodiment, database 17 is a relational database management system ("RDBMS") that can use structured query language ("SQL") to manage stored data.

[0033] In one embodiment, database 17 is implemented as an in-memory database ("IMDB"). An IMDB is a database management system that relies primarily on main memory for computer data storage. This is in contrast to database management systems that employ disk storage mechanisms. Main memory databases are faster than disk-optimized databases because disk access is slower than memory access, the internal optimization algorithms are simpler, and fewer CPU instructions are executed. Accessing data in memory eliminates seek times when querying data, which provides faster and more predictable performance than disk.

[0034] In one embodiment, when database 17 is implemented as an IMDB, it is implemented based on a distributed data grid. A distributed data grid is a system in which a collection of computer servers work together in one or more clusters to manage information and related operations (such as calculations) in a distributed or clustered environment. A distributed data grid can be used to manage application objects and data shared across servers. A distributed data grid provides low response time, high throughput, predictable scalability, continuous availability, and information reliability. In a specific example, a distributed data grid (such as, for example, Oracle's "Oracle Coherence" data grid) stores information in memory to achieve higher performance, and uses redundancy to keep copies of the information synchronized across multiple servers, thereby ensuring the resilience of the system and the continuous availability of data in the event of a server failure.

[0035] As disclosed, many BI analysis tools (such as OAS) that use RPD are created as single-tenant applications for end users. Generally speaking, end users are responsible for their own repository database (RPD). The Oracle BI repository (RPD file) stores BI server metadata. The metadata defines the logical schema, physical schema, physical to logical mapping, aggregate table navigation and other constructs. It is expected that end users edit their Oracle BI repository using the BI administration tool.

[0036] However, multi-tenancy of OAS or other BI systems needs to be enabled for users / clients / developers who maintain RPDs for customers / users / clients. Users need to maintain data repositories for clients (i.e. Figure 1 data repository 130), in which case it is also more efficient for users / clients / developers to maintain the RPD because they are closer to the data model than a typical client that maintains its own data repository.

[0037] In business intelligence systems such as Figure 1 In the system 105), the dimensional model describes the data structure for the business intelligence system by associating facts with dimensions. The dimensional model is stored as a business intelligence platform repository file (also called an RPD file or a BI repository file). The BI repository file defines the logical model, the physical model, and the physical to logical mapping. The BI repository file represents the dimensional model as three layers: the physical layer, the business model and mapping layer, and the presentation layer. The physical layer defines the objects and relationships of the original data source. The business model and mapping layer defines the business or logical model of the data and specifies the mapping between the logical model and the physical model. The presentation layer controls the view of the logical model given to the user.

[0038] The process of automatically generating a logical database schema, such as an RPD, includes identifying physical fact tables, dimension tables, and relationships between tables, and using metadata associated with the identified tables to merge related tables and generate a logical schema that can be useful for subsequent data retrieval and database updates.

[0039] A fact table in a dimensional database model is a table that stores measurements of events. Figure 3 The figure shows the fact table for the daily sales fact. Figure 3 In the example shown, the fact table includes rows corresponding to the data items that would exist for the sales of a product. In the illustrated example, these items include product ID, store ID, date ID, quantity sold, and dollar sales. Even for a single store on a single day, there can be hundreds or thousands of entries in such a table.

[0040] Some rows in the fact table are foreign keys ("FK"). Foreign keys are links to dimension tables. Dimension tables are tables that store context associated with events that are referenced by one or more fact tables. In the daily sales fact table, the product ID, store ID, and date ID attributes reference one or more dimension tables.

[0041] As disclosed, a fact table may include measurements of each sales transaction that occurred during a certain time period. Figure 4 Illustration of a populated Figure 3 The fact table. Figure 4 In the fact table shown in , each row corresponds to a single transaction (e.g., a sale), and each column corresponds to a measurement, which is also a foreign key. In the real world, Figure 4 The product sales fact table shown in can contain billions of rows, each with many different links or keys to dimension tables. Such a fact table can be part of a data warehouse schema that is initially imported into a data warehouse tool such as OBIEE. In order to correctly format structured queries against a database, it is necessary to know the relationships between the tables, which is called the database schema.

[0042] To make queries and updates to database tables more efficient, you can define a logical layer on top of the physical database tables, and define a presentation layer on top of the logical layer. Defining a logical database schema includes defining a logical fact table from a physical fact table, defining a logical dimension table from a physical dimension table, and linking logical tables. An example of how to define and link a logical fact table and a dimension table is disclosed below.

[0043] A dimension table is a table that one or more fact tables point to and stores attributes that describe the dimension. A dimension table contains a single primary key that corresponds to a foreign key in the associated fact table. Figure 5 Pictured Figure 3 The dimension table concept of the product dimension in the fact table of Figure 6 The diagram shows a dimension table populated with attributes of the product dimension. Figure 5 As shown in , the Product dimension table can include a hierarchy of product identification identifiers (such as SKU, Name, Subcategory, Class, Department, etc.). The Product ID attribute is the primary key ("PK"). The primary key is a unique key that identifies a dimension table column. Figure 5 In the product ID, the primary key is used to identify Figure 6 The primary key Product ID is also embedded as a foreign key in Figure 3 A natural key is a key created based on business rules outside the control of the data warehouse system. In the example shown, the SKU number is a natural key because it is created by the retail store's inventory control system. Figure 6 It should be noted that each product ID 1-9 includes multiple attributes.

[0044] To determine sales of a specific product in a specific store, you must specify Figure 4 The fact table shown in Figure 6 and a filter on the product ID of interest. However, the query initiator may not be aware of the relationship between the tables and therefore may not format the query correctly. As the number of tables and interrelationships increases, the problem becomes more difficult. To avoid this problem, it is necessary to define a dimensional schema that shows the logical relationships between the tables.

[0045] The dimensional schema is a representation of the relationship between the logical dimension tables and the logical fact tables. This dimensional relationship of the database can be manually generated by the database developer based on the logical facts and the relationship with the dimensions. Figure 7 The diagram shows Figure 3 An example of a star schema generated by the sales fact table and its associated dimension tables in . In the example shown, the daily sales logical fact table consists of Figure 3 The logical fact table is a logically connected dimension table, product dimension table, date dimension table and store dimension table, each dimension table contains attributes of a specific dimension, where the dimensions are product, date and store.

[0046] Creating a complete logical dimensional database schema for an underlying physical database schema includes a logical database schema generator that transforms physical database tables and metadata into business intelligence (BI) tool metadata that describes the logical schema structure. The basic mechanisms of the logical database schema generator can be table, view and column metadata, primary key (PK) and FK relationships, database table and column comments, and dimension objects. The logical database schema generator can also utilize constraints, indexes, and other metadata to generate the logical database schema.

[0047] The logical database schema generator can augment or extend the physical schema metadata with metadata derived from the base schema using conventions such as naming conventions or default assumptions (measurement aggregation rules, target level rules). When deriving the logical database schema, the logical database schema generator can also use explicit overrides and definitions from external files if the information cannot be derived (e.g., target hierarchy level, derived measure definitions, or aggregation rules).

[0048] Additional details regarding the automatic generation of a logical database schema from physical database tables and metadata are disclosed in U.S. Patent No. 10,169,378, the disclosure of which is incorporated herein by reference.

[0049] Figure 8 According to the embodiment Figure 2The dynamic metadata configuration includes a flow chart of the functionality of the module 16 when performing the dynamic inclusion of metadata configuration into the logical model functionality. In one embodiment, Figure 8 and below Fig. 9 The functionality is implemented by software stored in a memory or other computer-readable or tangible medium and executed by a processor. In other embodiments, each functionality may be performed by hardware (e.g., by using an application specific integrated circuit (“ASIC”), a programmable gate array (“PGA”), a field programmable gate array (“FPGA”), etc.) or any combination of hardware and software.

[0050] In general, combined Figure 8 , in an embodiment, before adding any new data elements to an Oracle Analytics or BI system dataset, the following apply:

[0051] All tables must be either FACT or DIMENSION

[0052] type;

[0053] DIMENSION tables usually have only primary key and attribute fields;

[0054] The FACT table has both attribute columns, measurement columns, and a foreign key column that references the primary key of the DIMENSION table.

[0055] A measurement column is a numeric value to which an aggregate function (COUNT, SUM, AVG, etc.) will be applied.

[0056] An attribute is some discrete text or value, such as a name, location, or energy burden bucket. A non-bucketed energy burden value is not a good example of an attribute; the value should be a measurement.

[0057] Example: The presence of a DATE column in a table is an indicator that the table is a FACT table, and each date column becomes a reference to the DATE dimension and other references to other dimensions;

[0058] Each FACT table will be presented as a subject area in OAV. Although it is possible to build a subject area from multiple FACT tables, this functionality is not supported in the RPD generator. Therefore, one FACT table will have only one subject area;

[0059] Each DIMENSION table will be presented as a folder with attributes bound to one or more subject areas;

[0060] The main condition for attaching attributes from a dimension to a subject area is to have a foreign key in the FACT table pointing to the DIMENSION table;

[0061] In an embodiment, after defining a set of FACT and DIMENSION tables in a source or data warehouse of "big" data (such as Hadoop storage) and loading the data using an Exchange Transfer Load ("ETL") job or any other way, it is time to implement the physical part of it in the ETL job:

[0062] Define physical tables for Oracle database;

[0063] Define which tables will be loaded from the Hadoop Distributed File System into the Oracle Database;

[0064] 1:1 loading of tables is usually not sufficient, thus requiring additional tables and transformation scripts to define them and create the transformations;

[0065] Define views. There is a separate schema for views, i.e. the schema with OBIEE suffix. Only this schema is visible to OAS applications. It is a representation mart of data,

[0066] Contains a "clean" version of the schema, hiding temporary and intermediate tables and columns.

[0067] The schema ETF job will be read by the RPD generator which will try to figure out the logical data model from it and this is usually enough to provide a draft version of the data model for presentation and further development. In addition to the schema and changes, a further step is to override the default RPD generator behavior, such as renaming data elements or defining sort columns.

[0068] Figure 8 The functionality of allows many different instances of OAS (also known as "OAV" or Oracle Utilities Opower Analytics Visualization) or any BI system using RPD. Specifically, Figure 8 The functionality allows organizations to give their development teams the flexibility to update each client's unique metadata for their 10-100 clients without having to manually update each client.

[0069] For example, a data analytics company working with a utility company (rather than the company's clients) can pre-configure data that will be visualized by the BI system. In one example, the company receives data from clients every day and the company's development team is logically mapping this data to make it available to clients who might not otherwise have access because, as project managers, the data is only available through their own organization's IT team or marketing team, which can be very siloed. The utility company already gets daily data feeds for other products (such as home energy reports), but these insights are very useful to the project manager clients who are typically not technically savvy. By updating the RPD, the utility company's developers took the technical challenges of setting up the data out of the hands of the users and were able to set everything up so that it is immediately available when the users arrive.

[0070] Generally speaking, Figure 8 The functionality automatically integrates the output of the RPD generator with OAS or any other Oracle Analytics.

[0071] At 801, desired changes to the data model and / or configuration metadata are pushed to the configuration repository. In one embodiment, the configuration repository is GitHub. These changes are human-readable and can therefore be made manually by the end user. Examples of changes are incoming data changes or creating pre-built dashboards for data visualization. Examples of changes can be the addition / modification / deletion of tables, columns, views, or other database objects. The end user will see it as a change in the values ​​displayed in the UI, such as a changed folder name, a modified property list, etc. In addition, when the database model schema changes, prior to 801, the metadata configuration makes the RPD conform to the updated schema.

[0072] In an embodiment, changes (i.e., metadata configuration or "declarative" configuration) are implemented in a JSON file (.js file). Changes are part of the metadata configuration or catalog file and are dynamically included in the RPD in an embodiment. A partial example of the JSON file at 801 is as follows:

[0073]

[0074]

[0075]

[0076]

[0077]

[0078] At 802, an ETL job is built and run. For example, a data analytics company working with a utility company obtains data feeds of millions of customers per day for each utility client. The raw data received is run through a production analytics cluster to curate the information into a usable format. BI-ETL performs the functionality of processing to extract, transmit, and load data into a business intelligence database. ETL is a three-stage process in which data is extracted, transformed, and loaded into an output data container. Data can be collated from one or more sources and can also be output to one or more destinations.

[0079] In an embodiment, OAV-BI-ETL is a program written in Java and executed on a data warehouse (e.g., a Hadoop cluster). OAV-BI-ETL extracts data from the cluster, transforms it, and loads it into a database that is used as a source of data by Oracle Analytics or a BI system.

[0080] The latest code is used to update the database model to the latest version. In one embodiment, the database is a data source, such as an Oracle database. The database model is a database configuration, including tables, columns, and views. In one embodiment, the database model is the star schema described above.

[0081] At 803, an RPD generator is run to generate an RPD ("first RPD"). In an embodiment, an RPD is an XML file that represents a database model but is generally not human-readable. In an embodiment, at 803, an RPD generator disclosed in U.S. Patent No. 10,169,378 is used.

[0082] At 804, a business intelligence ("BI") server tool is run. The BI server tool uses the XML file from 803 as input and generates a revised RPD file ("second RPD") for use in OAS. The revised RPD includes the metadata configuration and catalog files from JSON files from 801 that describe what should be included in the RPD. The metadata configuration included in the revised RPD also includes the schema of the database used to generate the RPD at 803. The BI server tool or BI system can be any BI system that utilizes the RPD. In one embodiment, the BI system is Oracle Utilities OpowerAnalytics Visualization, which enables utilities to explore customer data and create customized data extracts and insights related to Opower projects. It includes pre-built analytical subject areas and visualizations that allow clients to derive strategic insights from data, such as the number of messages sent to customers and how well customers interact with the Opower product.

[0083] At 805, the RPD is published by uploading the RPD file to a shared location, such as uploading the RPD file to the cloud. The RPD is uploaded in a compiled form of a visual configuration, which can be downloaded to build an image and made available to multiple tenants / users.

[0084] At 806, the RPD file is downloaded by the developer and used to build an Oracle Analytics Docker image. A "Docker" image is a file used to execute code in a Docker container. A Docker image serves as an instruction set for building a Docker container (such as a template). A Docker image also serves as a starting point when using Docker. An image is equivalent to a snapshot in a virtual machine ("VM") environment. Docker is a collection of platform-as-a-service products that use OS-level virtualization to deliver software in packages called "containers." In other embodiments, other containerized functionalities may be used instead of Docker, an environment framework that implements virtualization. The compiled image can be used by the BI system to automatically create visualizations of data and simulate the results of changes received, and it can be received by the BI system as a container. Thus, at 806, processing runs in a container and uses declarative configuration to automatically generate visualizations that would otherwise be generated by a user via user input to a GUI (i.e., the latter is a known solution for generating visualizations customized for each tenant, which is labor intensive).

[0085] In an embodiment, upload / download is used because the first system hosts the revised RPD instance used by one or more enterprise users. In other embodiments, the revised RPD is directly used to build an image, rather than upload / download.

[0086] In an embodiment, Figure 8 The functionality is the result of a "Jenkins" job being triggered. A Jenkins job is generally a sequential collection of user-defined tasks. For example, a job can fetch source code from version control, compile code, run unit tests, etc. Note that in Jenkins, the word "job" is synonymous with "pipeline".

[0087] Fig. 9 According to the embodiment Figure 2 A flowchart of the detailed functionality of the dynamic metadata configuration including module 16 when performing the dynamic inclusion of metadata configuration into the logical model functionality.

[0088] At 901, Figure 8 In response to error 801, make changes to the JSON configuration file and commit the changes.

[0089] At 902, with Figure 8 802 correspondence, from a secure repository such as Figure 1 Oracle database or data repository 130) reads database credentials.

[0090] At 903, with Figure 8 802 corresponds to retrieving the job source code (BI-ETL) and building the source into a JAR file.

[0091] At 904, Figure 8 802 corresponds to retrieving an external configuration for a BI-ETL job and running the job to update a database.

[0092] At 905, Figure 8 Corresponding to 803, the RPD generator (as disclosed in U.S. Patent No. 10,169,378) is run against the updated database.

[0093] In 907, with Figure 8 Corresponding to 803, determine whether the XML file has been generated by the RPD generator. If not, then send a failure alert at 906. If the job code is incorrect, the database configuration is incorrect, the RPD generator verification fails, an infrastructure problem occurs, etc., then the generation of the XML file will fail.

[0094] At 908, Figure 8 Corresponding to 804 , download and run a BI client tool from, for example, Oracle Corporation to transform XML into RPD. The BI client translates the XML output of the RPD generator into an RPD binary file.

[0095] At 910, with Figure 8 Corresponding to 804, determine whether the RPD file has been generated. If not, then generate a failure alarm at 909.

[0096] At 911, Figure 8 For 805, the RPD file is published to a shared location.

[0097] At 912, with Figure 8 806 corresponds to triggering a job to build a custom OAS image using the RPD file.

[0098] At 913, with Figure 8 806 corresponds to building a new image with the updated RPD file.

[0099] Fig. 9 The command line commands for executing the process / method are illustrated. In one embodiment, these commands are automated and executed as part of the code, but for visibility, they are specified as command line commands.

[0100] In an embodiment, the RPD file is generated and stored outside the analysis instance. The analysis instance itself is the result of the disclosed process (ie, the OAS Docker image at 806 / 913).

[0101] use Fig. 9 The functionality of the automation is such that no access to credentials is required, so anyone can make changes without access to those credentials (i.e., no direct access to the credentials is required to trigger the process). The credentials themselves are required to access the database during the execution of the automation, but no one can read them.

[0102] The access control that triggers this process is controlled at the source version control system (e.g., Git) level. Git has its own credentials and process of peer reviewing changes before they are accepted.

[0103] Embodiments provide excellent security and are consistent with the infrastructure as code model. Infrastructure as code is the process of managing and provisioning computer data centers through machine-readable definition files rather than physical hardware configurations or interactive configuration tools.

[0104] The features, structures, or characteristics of the present disclosure described throughout this specification may be combined in any suitable manner in one or more embodiments. For example, throughout the specification, the use of "one embodiment", "some embodiments", "an embodiment", "certain embodiments", or other similar language means that a particular feature, structure, or characteristic described in relation to an embodiment may be included in at least one embodiment of the present disclosure. Therefore, throughout this specification, the appearance of the phrase "one embodiment", "some embodiments", "an embodiment", "certain embodiments", or other similar language does not necessarily refer to the same set of embodiments, and the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0105] Those of ordinary skill in the art will readily appreciate that the embodiments discussed above may be practiced with steps in a different order and / or with configuration elements that are different from the disclosed configurations. Therefore, while the present disclosure contemplates the embodiments outlined, it will be apparent to those skilled in the art that certain modifications, variations, and alternative configurations will be apparent while remaining within the spirit and scope of the present disclosure. Therefore, in order to determine the limits and scope of the present disclosure, reference should be made to the appended claims.

Claims

1. A method of generating changes to a logical model, the method comprising: receiving a change in a configuration file, the change comprising a declarative configuration; Extract and load changes into the database and update the corresponding database model; generating a first logical model representing a database model; generating a second logic model including the changes; as well as A compiled visualization image from the second logical model is automatically generated in the container using the declarative configuration, wherein the visualization image is suitable for use by a business intelligence system to provide a visualization incorporating the changed data. 2 . The method of claim 1 , wherein the first logic model includes a first RPD and the second logic model includes a second RPD. The method of claim 2 , wherein generating the first logical model comprises using an RPD generator. The method of claim 1 , wherein the changes include metadata. The method of claim 1 , wherein the changes include catalog files.

6. The method of claim 1, further comprising: The second logical model is uploaded, wherein the uploaded second logical model is suitable for downloading by one or more business users to generate visualizations of the data.

7. The method of claim 1, further comprising: The database model schema is changed, wherein the second logical model incorporates the changed database model schema.

8. The method of claim 1, wherein the configuration file comprises a JavaScript Object Notation (JSON) file.

9. A computer readable medium having stored thereon instructions which, when executed by one or more processors, cause the processors to generate changes to a logical model, the generating comprising: receiving a change in a configuration file, the change comprising a declarative configuration; Extract and load changes into the database and update the corresponding database model; generating a first logical model representing a database model; generating a second logic model including the changes; as well as A compiled visualization image from the second logical model is automatically generated in the container using the declarative configuration, wherein the visualization image is suitable for use by a business intelligence system to provide a visualization incorporating the changed data.

10. The computer-readable medium of claim 9, wherein the first logical model includes a first RPD and the second logical model includes a second RPD.

11. The computer-readable medium of claim 10, wherein generating the first logical model comprises using an RPD generator.

12. The computer-readable medium of claim 9, wherein the changes include metadata.

13. The computer readable medium of claim 9, wherein the change comprises a catalog file.

14. The computer readable medium of claim 9, generating further comprising: The second logical model is uploaded, wherein the uploaded second logical model is suitable for downloading by one or more business users to generate visualizations of the data.

15. The computer readable medium of claim 9, generating further comprising: The database model schema is changed, wherein the second logical model incorporates the changed database model schema.

16. The computer-readable medium of claim 9, wherein the configuration file comprises a JavaScript Object Notation (JSON) file.

17. A system for generating changes to a logical model, the system comprising: One or more processors configured to: receiving a change in a configuration file, the change comprising a declarative configuration; Extract and load changes into the database and update the corresponding database model; generating a first logical model representing a database model; generating a second logic model including the changes; as well as A compiled visualization image from the second logical model is automatically generated in the container using the declarative configuration, wherein the visualization image is suitable for use by a business intelligence system to provide a visualization incorporating the changed data.

18. The system of claim 17, wherein the first logical model includes a first RPD and the second logical model includes a second RPD.

19. The system of claim 18, wherein generating the first logical model comprises using an RPD generator.

20. The system of claim 17, wherein the changes include metadata.

Images