Dynamic incorporation of metadata configurations into logical models

Abstract

Embodiments generate changes to a logical model. Embodiments receive the changes in a configuration file, the changes including declarative configurations, and embodiments extract the changes, load the changes into a database, and update a corresponding database model. Embodiments generate a first logical model representing the database model and generate a second logical model including the changes. Embodiments use the declarative configurations to automatically generate, within a container, a visualization image compiled from the second logical model, the visualization image adapted for use by a business intelligence system to provide a visualization of the data incorporating the changes.

Description

[Technical Field]

[0001] One embodiment relates generally to computer systems, and more particularly to using metadata constructs to facilitate visualization of underlying data in a computer system. [Background technology]

[0002] Background information Data warehouse tools or business intelligence platforms such as Oracle Business Intelligence Enterprise Edition ("OBIEE"), Oracle Analytics Cloud ("OAC") or Oracle Analytics Server ("OAS") allow you to define logical database schemas on top of physical database tables. Logical database schemas contain patterns that describe the relationships between the underlying physical database tables. For example, you can define a star schema that describes the relationships between tables and joins logically related tables.

[0003] To efficiently retrieve data from a database, it is essential to know the relationships between the underlying database tables. For example, to form a database query that provides meaningful results, each query must specify the tables to be searched and specify filter criteria to narrow the results presented from the combination of tables. Without knowledge of the relationships between the tables and the data structure within the tables, the query will produce results that are meaningless to the end user. Summary of the Invention

[0004] overview Embodiments generate changes to a logical model. Embodiments receive the changes in a configuration file, the changes including declarative configurations, and embodiments extract the changes, load the changes into a database, and update a corresponding database model. Embodiments generate a first logical model representing the database model and generate a second logical model including the changes. Embodiments use the declarative configurations to automatically generate, within a container, a visualization image compiled from the second logical model, the visualization image adapted for use by a business intelligence system to provide a visualization of the data incorporating the changes.

[0005] Further embodiments, details, advantages and modifications will become apparent from the following detailed description of the embodiments, which is to be considered in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a block diagram of a BI system including BI functionality for an enterprise network and including dynamic artifact inclusion functionality according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram of one or more components of the BI system of FIG. 1 in the form of a computer server / system according to an embodiment of the present invention. [Figure 3] FIG. 10 illustrates a fact table for daily sales facts. [Figure 4] FIG. 4 is a diagram showing the fact table of FIG. 3 into which measurement data has been input. [Figure 5] FIG. 4 illustrates the dimension table concept for the product dimension in the fact table of FIG. 3. [Figure 6] FIG. 10 is a diagram showing a dimension table into which attributes of a product dimension have been entered. [Figure 7] FIG. 4 is a diagram illustrating an example of a star schema generated from the sales fact table and its associated dimension tables of FIG. 3. [Figure 8]3 is a flow diagram of the functionality of the dynamic metadata configuration inclusion module of FIG. 2 when performing dynamic inclusion of metadata configurations into logical model functionality, according to an embodiment. [Figure 9] 3 is a detailed functional flow diagram of the dynamic metadata configuration inclusion module of FIG. 2 when performing dynamic inclusion of metadata configurations into logical model functions, according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0007] Detailed Description In one embodiment, a logical model, such as a business intelligence repository database file ("RPD file"), includes automatically generated metadata configurations in the form of catalog files that can be used to automatically create visualizations of many different underlying tenant data stores.

[0008] Embodiments flexibly include metadata configurations (i.e., files of configuration), such as catalog files, in the RPD of Oracle Analytics Cloud (“OAC”), or any other business intelligence system that implements an RPD file. The Oracle BI repository (RPD file) stores BI Server configuration metadata specific to a BI Server customer. This metadata defines logical schemas, physical schemas, physical-to-logical mappings, aggregate table navigation, and other configurations. RPD files have historically been compressed files that are difficult to update or maintain, especially for multitenant applications. Therefore, embodiments generate and store the RPD separately from the OAC instance (e.g., at 120 in FIG. 1 instead of 125). This improves users' ability to upgrade their OAC version in a more modular manner.

[0009] In an embodiment, metadata configurations or configuration metadata are rules for modifying and adjusting the repository database to indicate updates such as new visualizations, new data elements to be rendered, aggregations, and the like.

[0010] Business intelligence ("BI") platforms (commonly referred to as "Oracle Analytics" or "BI systems") such as Oracle® Business Intelligence Enterprise Edition ("OBIEE"), Oracle® Analytics Cloud ("OAC"), or Oracle® Application Server ("OAS") can be extended to integrate catalog files (i.e., files of configuration or metadata structure) into one or more BI repository files (RPD files) that represent the logical model to be presented to customers. The catalog files can be used to create custom visualizations from the logical model and presented in the OBIEE, OAC, and OAS Data Visualization ("DV") graphical user interfaces ("GUIs").

[0011]

[0003] Embodiments provide the flexibility to include catalog files in the RPD of an OAS or another BI application that utilizes the RPD. The RPD is a file that stores metadata and rules for creating and presenting logical database schemas in OBIEE, OAC, and OAS. Historically, RPDs have been compressed files that are difficult to update or maintain, especially in multitenant applications. For example, known processes for updating catalog files for each of the many tenants in a multitenant instance are manual, resulting in an increasing level of effort as the number of tenants increases.

[0012] In contrast, embodiments generate and store the RPD separately from the analysis instance (i.e., in 120 in FIG. 1 instead of in BI platform / tool / system 125). This improves a user's ability to upgrade versions of OAS or any other BI application in a more modular manner, since the RPD is typically tied to the analysis instance for a single owner. In contrast, embodiments use a separate storage mechanism in which RPDs are stored for use by different owners / users. Separate storage allows an administrator to update configurations for users before they receive the instances for use. For example, one instance is one connection of a BI system (e.g., server 125 in FIG. 1 , described below) to one enterprise network (e.g., enterprise network 115 in FIG. 1 , described below).

[0013] Reference will now be made in detail to the embodiments of the present disclosure, as illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the embodiments. Wherever possible, like reference numerals are used to refer to like elements.

[0014] 1 is a block diagram of a BI system 100 that includes BI functionality for an enterprise network and that includes dynamic artifact inclusion functionality according to an embodiment of the present invention. In one embodiment, system 100 includes a business intelligence configuration system 105 connected to an enterprise network 115 by the Internet 110 (or another suitable communication network or combination of networks). In one embodiment, business intelligence configuration system 105 may be an OBIEE, OAC, or OAS service implementation, or any other BI system that implements an RPD file. In one embodiment, business intelligence configuration system 105 includes various systems and components, such as a dynamic metadata configuration inclusion server / module 120, a business intelligence server 125, a data store 130, and a web interface server 135.

[0015] In one embodiment, the dynamic metadata configuration inclusion server 120 includes one or more components configured to implement the methods, functions, and features described herein related to the dynamic inclusion of metadata configurations into logical models.

[0016] In one embodiment, the business intelligence server / system 125 may include business intelligence applications and functionality for searching, analyzing, mining, visualizing, transforming, reporting, and otherwise utilizing data related to the operation of a business. In one embodiment, the business intelligence server 125 may include a data collection component that captures and records data related to the operation of a business in a data repository, such as data store 130. In one embodiment, the other business intelligence server 125 may further include a user management module for managing user access to the business intelligence configuration system 105.

[0017] Each of the components of the business intelligence configuration system 105 is configured by logic to perform the functions that the component is described to perform. In one embodiment, the components of the business intelligence system may each be implemented as a set of one or more software modules executed by one or more computing devices specially configured for such execution. In one embodiment, the components of the business intelligence configuration system 105 are implemented on one or more hardware computing devices. In one embodiment, the components of the business intelligence configuration system 105 are each implemented by a dedicated computing device. In one embodiment, the components of the business intelligence configuration system 105, although depicted as separate units in FIG. 1 , are implemented by a common (or shared) computing device. In one embodiment, the business intelligence configuration system 105 may be hosted by a dedicated third party, for example, in an Infrastructure as a Service ("IAAS"), Platform as a Service ("PAAS"), or Software as a Service ("SAAS") architecture. In one embodiment, components of business intelligence configuration system 105 communicate with each other through electronic messages or signals. These electronic messages or signals may be configured as calls to functions or procedures that access features or data of the components, such as, for example, application programming interface ("API") calls. Each component of business intelligence configuration system 105 may analyze the content of a received electronic message or signal to identify a command or request that the component can perform, and in response to identifying the command, the component automatically performs the command or request.

[0018] The enterprise network 115 may be associated with a business. For simplicity and clarity, the enterprise network 115 is represented by an on-site local area network 140 to which one or more personal computers 145 or servers 150 are operatively connected, along with one or more remote user computers 155 or mobile devices 160 connected to the enterprise network 115 via the Internet 110. Each personal computer 145, remote user computer 155, or mobile device 160 is generally dedicated to a particular end user, such as an employee or contractor, associated with the business, although such dedication is not required. The personal computers 145 and remote user computers 155 may be, for example, desktop computers, laptop computers, tablet computers, or other devices capable of connecting to the local area network 140 or the Internet 110. The mobile devices 160 may be, for example, smartphones, tablet computers, mobile phones, or other devices capable of connecting to the local area network 140 or the Internet 110 via a wireless network, such as a cellular network or Wi-Fi. Users of the enterprise network 115 interface with the business intelligence configuration system 105 via the Internet 110 (or another suitable communications network or combination of networks).

[0019] In one embodiment, remote computing systems (such as those of the enterprise network 115) can access information or applications provided by the business intelligence configuration system 105 via the web interface server 135. In one embodiment, the remote computing systems can send requests to and receive responses from the web interface server 135. In one example, access to the information or applications can be through the use of a web browser on a personal computer 145, a remote user computer 155, or a mobile device 160. For example, these computing devices 145, 155, 160 of the enterprise network 115 can request and receive a web page-based graphical user interface (“GUI”) for dynamically displaying visualization data for use in the business intelligence configuration system 105. In one example, the web interface server 135 can present HTML code to the personal computer 145, the server 150, the remote user computer 155, or the mobile device 160 for these computing devices to render in the GUI of the business intelligence configuration system 105. In another example, communications may be exchanged between web interface server 135 and personal computer 145, server 150, remote user computer 155, or mobile device 160 and may take the form of, for example, remote representational state transfer ("REST") requests using JavaScript object notation ("JSON") as the data exchange format, or simple object access protocol ("SOAP") requests to or from an XML server. For example, computers 145, 150, 155 on enterprise network 110 may request information contained in custom columns or may request information derived at least in part from information contained in custom columns (such as analytical results based at least in part on custom columns).

[0020] In one embodiment, data store 160 includes one or more operational databases configured to store and provide a wide range of information related to the operation of the business, such as data related to enterprise resource planning, customer relationship management (including customer data such as account numbers, demographic information, and third-party data), financial accounting, order processing, time billing, inventory management and distribution, employee management and payroll, calendar collaboration, product information management, demand and material requirements planning, purchasing, sales, sales force automation, marketing, e-commerce, vendor management, supply chain management, product lifecycle management, descriptions of hardware assets and their status, production output, shipment tracking information, and any other information collected by the business. Such operational information can be stored in the operational databases in real time as the information is collected. In one embodiment, data store 160 includes a mirror or copy database of each operational database that can be used for disaster recovery or can be used secondary to support read-only operations. In one embodiment, the operational database is an Oracle® database. In some example configurations, data store 160 may be implemented using one or more Oracle® Exadata compute shapes, network-attached storage (“NAS”) devices, and / or other dedicated server devices.

[0021] 2 is a block diagram of one or more components of the BI system 100 of FIG. 1 in the form of a computer server / system 100 according to an embodiment of the present invention. Although shown as a single system, the functionality of system 100 may be implemented as a distributed system. Additionally, the functionality disclosed herein may be implemented on separate servers or devices that may be coupled via a network. Furthermore, one or more components of system 100 may not be included. One or more components of FIG. 2 may also be used to implement any of the elements of FIG. 1.

[0022] System 100 includes a bus 12 or other communication mechanism for communicating information and a processor 22 coupled to bus 12 for processing information. Processor 22 may be any type of general-purpose or special-purpose processor. System 10 further includes memory 14 for storing information and instructions executed by processor 22. Memory 14 may be any combination of random access memory (“RAM”), read-only memory (“ROM”), static storage such as a magnetic or optical disk, or any other type of computer-readable medium. System 100 further includes a communication device 20, such as a network interface card, to provide access to a network. Thus, a user may interface with system 100 directly, remotely over a network, or in any other manner.

[0023] Computer-readable media can be any available media that can be accessed by processor 22 and includes both volatile and nonvolatile media, removable and non-removable media, and communication media. Communication media can include computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

[0024] Processor 22 is further coupled via bus 12 to a display 24, such as a liquid crystal display ("LCD"), and a keyboard 26 and cursor control device 28, such as a computer mouse, are further coupled to bus 12 to enable a user to interface with system 10.

[0025] In one embodiment, memory 14 stores software modules that provide functionality when executed by processor 22. These modules include operating system 15, which provides operating system functionality for system 10. The modules further include dynamic metadata configuration inclusion module 16, which provides functionality for dynamic inclusion of metadata configurations into logical models and all other functionality disclosed herein. System 10 may be part of a larger system. Thus, system 10 may include one or more additional functionality modules 18 to include additional functionality, such as any other BI functionality disclosed above (e.g., OBIEE, OAS, etc.). A file storage device or database 17 is coupled to bus 12 to provide centralized storage for implementing modules 16 and 18, as well as data store 130 of FIG. 1 . In one embodiment, database 17 is a relational database management system (“RDBMS”) capable of using structured query language (“SQL”) to manage stored data.

[0026] In one embodiment, database 17 is implemented as an in-memory database ("IMDB"). An IMDB is a database management system that relies primarily on main memory for computer data storage, as opposed to database management systems that use a disk storage mechanism. Main-memory databases are faster than disk-optimized databases because disk access is slower than memory access, and the internal optimization algorithms are simpler and execute fewer CPU instructions. Accessing data in memory eliminates seek times when querying data, resulting in faster and more predictable performance than disk.

[0027] In one embodiment, when implemented as an IMDB, database 17 is implemented based on a distributed data grid. A distributed data grid is a system in which a collection of computer servers cooperate in one or more clusters to manage information and related operations, such as computation, in a distributed or clustered environment. Distributed data grids can be used to manage application objects and data shared among servers. Distributed data grids provide low response times, high throughput, predictable scalability, continuous availability, and reliability of information. In particular, distributed data grids, such as Oracle's "Oracle Coherence" data grid, store information in-memory to achieve higher performance and use redundancy to synchronize copies of that information across multiple servers, ensuring system resilience and continuous availability of data in the event of a server failure.

[0028] As disclosed, many BI analytics tools that use RPDs, such as OAS, are created as single-tenant applications for end users. Typically, end users are responsible for their own repository databases (RPDs). The Oracle BI repository (RPD file) stores the metadata for the BI server. This metadata defines the logical schema, physical schema, physical-to-logical mapping, aggregate table navigation, and other configurations. End users are expected to edit the Oracle BI repository using a BI administration tool.

[0029] However, there is a need to enable multi-tenant application of an OAS or other BI system to users / clients / developers who maintain the RPD for their customers / users / clients. There is a need for users who maintain a data store (i.e., data store 130 in FIG. 1) for their clients, where it is more efficient for the user / client / developer to also maintain the RPD, since they are closer to the data model than a typical client who maintains their own data store.

[0030] In a business intelligence system, such as system 105 in FIG. 1, a dimensional model describes the structure of the data for the business intelligence system by relating facts to dimensions. This dimensional model is stored as a business intelligence platform repository file (also called an RPD file or BI repository file). The BI repository file defines the logical schema, the physical schema, and the physical-to-logical mapping. The BI repository file represents the dimensional model in three layers: the physical layer, the business model and mapping layer, and the presentation layer. The physical layer defines the objects and relationships of the original data sources. The business model and mapping layer defines the business or logical model of the data and specifies the mapping between the logical model and the physical schema. The presentation layer controls the view of the logical model presented to users.

[0031] The process of automatically generating a logical database schema, such as an RPD, involves identifying physical fact tables, dimension tables, and the relationships between the tables, and using metadata associated with the identified tables to merge related tables and generate a logical schema that is useful for subsequent data retrieval and database updates.

[0032] A fact table in a dimensional database model is a table that stores measurements of events. Figure 3 shows a fact table for daily sales facts. In Figure 3, the fact table contains rows that correspond to data items present in the sales of a product. In the example shown, these items include product ID, store ID, date ID, quantity sold, and sales amount. There can be hundreds or thousands of items in such a table for even a single store in a single day.

[0033] Some of the rows in a fact table are foreign keys ("FKs"). Foreign keys are links to dimension tables. Dimension tables are tables that store the context associated with events referenced by one or more fact tables. In the daily sales fact table, the product ID, stored ID, and date ID attributes reference one or more dimension tables.

[0034] As disclosed, a fact table can contain measurements from each sales transaction that occurred over a period of time. Figure 4 shows the fact table from Figure 3 populated with measurement data. In the fact table illustrated in Figure 4, each row corresponds to a single transaction (e.g., a sale), and each column corresponds to a measure that is also a foreign key. In the real world, a product sales fact table like the one shown in Figure 4 could contain billions of rows, with each row containing many different links or keys to dimension tables. Such a fact table could be part of a data warehouse schema that is first imported into a data warehouse tool such as OBIEE. To properly format structured queries to the database, it is necessary to know the relationships between tables, known as the database schema.

[0035] To make queries and updates to the database tables more efficient, a logical layer can be defined on top of the physical database tables, and a presentation layer can be defined on top of the logical layer. Defining a logical database schema includes defining logical fact tables from physical fact tables, defining logical dimension tables from physical dimension tables, and linking the logical tables. Examples of how logical fact and dimension tables are defined and linked are disclosed below.

[0036] A dimension table is a table pointed to by one or more fact tables and stores attributes that describe the dimension. A dimension table contains a single primary key that corresponds to a foreign key in the associated fact table. Figure 5 illustrates the dimension table concept for the product dimension of the fact table in Figure 3, and Figure 6 illustrates the dimension table populated with product dimension attributes. For example, as shown in Figure 5, the product dimension table may contain a hierarchy of product identification identifiers, such as SKU, name, subclass, class, and department. The product ID attribute is the primary key ("PK"). A primary key is a unique key that identifies a column in a dimension table. In Figure 5, the primary key product ID identifies the first column in the physical dimension table shown in Figure 6. The primary key product ID is also embedded as a foreign key in the fact table shown in Figure 3. A natural key is a key created according to business rules outside the control of the data warehouse system. In the illustrated example, the SKU number is a natural key because it is created by the retailer's inventory management system. Note from the dimension table in Figure 6 that each product ID 1-9 contains multiple attributes.

[0037] To determine the sales of a particular product at a particular store, a query must specify the fact table shown in Figure 4, the dimension tables shown in Figure 6, and a filter on the product ID of interest. However, the query writer may not be aware of the relationships between the tables and may therefore be unable to format the query correctly. This problem becomes more difficult as the number of tables and their interrelationships increase. To avoid this problem, a dimensional schema must be defined that shows the logical relationships between the tables.

[0038] A dimensional schema represents the relationships between logical dimension tables and logical fact tables. Such dimensional relationships can be manually generated by a database developer based on the logical facts and the relationships between dimensions. Figure 7 shows an example of a star schema generated from the sales fact table and its associated dimension tables of Figure 3. In the illustrated example, the daily sales logical fact table contains the categories of measures shown in Figure 3. The logical fact tables are logically joined dimension tables, namely, a product dimension table, a date dimension table, and a store dimension table, where each dimension table contains attributes for a particular dimension, where the dimensions are product, date, and store.

[0039] Creating a complete logical dimensional database schema for an underlying physical database schema involves a logical database schema generator that converts physical database tables and metadata into business intelligence (BI) tool metadata that describes the logical schema structure. The basic mechanisms of a logical database schema generator can be table, view, and column metadata, primary key (PK) and FK relationships, database table and column comments, and dimension objects. The logical database schema generator can also utilize constraints, indexes, and other metadata to generate the logical database schema.

[0040] The logical database schema generator can extend or augment the metadata of the physical schema with metadata derived from the base schema using conventions, for example, naming conventions or default assumptions (measure aggregation rules, target level rules). When deriving the logical database schema, the logical database schema generator can also use explicit overrides and definitions from external files if it cannot derive information such as target hierarchy levels, derived measure definitions or aggregation rules.

[0041] Additional details regarding the automatic generation of logical database schemas from physical database tables and metadata are disclosed in US Pat. No. 10,169,378, the disclosure of which is incorporated herein by reference.

[0042] 8 is a flow diagram of the functionality of dynamic metadata configuration inclusion module 16 of FIG. 2 when performing dynamic inclusion of metadata configurations into logical model features, according to an embodiment. In one embodiment, the functionality of FIG. 8 and the following FIG. 9 is implemented by software stored in memory or other computer-readable or tangible medium and executed by a processor. In other embodiments, the functions may be performed by hardware (e.g., through the use of application-specific integrated circuits ("ASICs"), programmable gate arrays ("PGAs"), field-programmable gate arrays ("FPGAs"), etc.), or any combination of hardware and software.

[0043] Generally, with reference to FIG. 8, before adding any new data elements to a dataset in Oracle Analytics or a BI system in an embodiment, the following applies: All tables must be either FACT or DIMENSION type. ·DIMENSION tables usually have only a primary key and attribute fields. · A FACT table has both an attribute column, a measure column, and a foreign key column that references the primary key of a DIMENSION table. A measure column is any numeric value to which an aggregate function (such as COUNT, SUM, AVG, etc.) is applied. Attributes are discrete text or numbers, such as Name, Location, or buckets of Energy Burden. The non-bucket Energy Burden value is not a good example of an attribute; this number should be a measure. Example: The presence of a DATE column in a table is an indicator that the table is a FACT table, and each date column is a reference to the DATE dimension along with other references to other dimensions. Each FACT table is represented as a Subject Area in the OAV. It is possible to build a Subject Area from multiple FACT tables, but this is not supported by the RPD Generator. Therefore, there is exactly one Subject Area per FACT table. Each DIMENSION table is presented as a folder with attributes bound to one or more Subject Areas. The main condition for attaching an attribute from a dimension to a Subject Area is that it has a foreign key in the FACT table to the DIMENSION table.

[0044] In an embodiment, a set of FACT and DIMENSION tables is defined in a source of "big" data or a data warehouse such as Hadoop storage, and after the data has been loaded through an Exchange Transfer Load ("ETL") job or any other method, it is time to perform the physical part of the ETL job. Define physical tables for an Oracle database. Define which tables to load from the Hadoop Distributed File System into the Oracle database. Loading tables one-to-one is usually insufficient, so additional tables and transform scripts are required, define them, and create the transforms. Define a view. There is a separate schema for the view, suffixed with OBIEE. Only this schema is visualized by the OAS application. It is a presentation mart of the data, containing a "clean" version of the schema, hiding temporary and intermediate tables and columns.

[0045] The Schema ETF job is read by the RPD Generator and attempts to grasp the logical data model from it, presenting a draft version of the data model that is usually sufficient for further development. A further step beyond the schema and modifications is to override the default RPD Generator behavior, such as renaming data elements or defining sort columns.

[0046] The functionality in Figure 8 allows for many different instances of OAS (also known as "OAV" or Oracle Utilities Opower Analytics Visualization) or any BI system that uses RPD. Specifically, the functionality in Figure 8 allows organizations to have their development teams flexibly update each client's unique metadata for tens or hundreds of clients, rather than having to do it manually for each client.

[0047] For example, a data analytics company working with a utility that receives large amounts of source data rather than its own clients can pre-configure the data to be visualized by the BI system. In one example, the company receives data daily from its clients, and its development team logically maps this data for clients who, as program managers, might not have access because the data might only be available through their own organization's IT or marketing teams, which may be siloed. The utility already gets daily data feeds for other products, such as home energy reports, but the insights are extremely useful to the program manager's clients, who are typically non-technical. By updating the RPD, the utility's developers can take the technical challenges of configuring the data out of users' hands and configure everything for immediate use upon arrival.

[0048] The functionality in Figure 8 generally automatically integrates the output of the RPD Generator with OAS or any other Oracle Analytics.

[0049] At 801, desired changes to the data model and / or configuration metadata are pushed to a configuration store. In one embodiment, the configuration store is GitHub. Changes are human-readable and can be made manually by end users. Examples of changes include modifying incoming data or creating pre-built dashboards for data visualization. Examples of changes include adding / modifying / removing tables, columns, views, and other database objects. End users see this as changes to displayed values ​​in the UI, such as changed folder names, modified attribute lists, etc. Additionally, if the database model schema is changed before 801, the metadata configuration will make the RPD conform to the updated schema.

[0050] In an embodiment, the changes (i.e., metadata configuration or "declarative" configuration) are implemented in a JSON file (.js file). The changes are part of the metadata configuration or catalog file, and in an embodiment, are dynamically included in the RPD. An example portion of the JSON file for 801 is as follows:

[0051] { "schema":[ { "Name":"RPD_GENERATOR", "table":[ { "Name":"FACT_HOUSEHOLD", "Type":"FACT", "column":[ { "Name":"Household Count", "Description":"Household Count", "expression":"1", "post-aggregation":false } ] }, { "Name":"FACT_OUTBOUND_COMMUNICATIONS", "Type":"FACT", "column":[ { "Name":"Outbound Communication Count", "Description":"Outbound Communication Count", "expression":"1", "post-aggregation":false } ] }, { "Name":"FACT_WEB_PAGEVIEWS", "Type":"FACT", "column":[ { "Name":"Web Page View Count", "Description":"Web Page View Count", "expression":"1", "post-aggregation":false }, { "Name":"Unique Page View Count", "Business Name":"Unique Page View Count", "Description":"Unique Page View Count", "Aggregation rule":"Count Distinct", "expression":"\"ID\"", "post-aggregation":false } ] }, { "Name":"FACT_CUSTOMER_ENGAGEMENT", "Type":"FACT", "column":[ { "Name":"Engagement Count", "Description":"Engagement Count", "expression":"1", "post-aggregation":false }, {"name":"SMS Count", "Description":"SMS Count", "expression":"CASE WHEN \"CHANNEL\" = 'SMS' THEN 1 ELSE 0 END", "post-aggregation":false, "Aggregation rule":"Sum", }, {"name":"SMS Filter", "Description":"SMS Filter", "expression":"FILTER(\"Fact-Customer Engagement.Engagement Count\" USING (\"Fact-Customer Engagement.Channel\" = 'SMS'))", "after aggregation":true, "Aggregation rule":"Sum", } ] }, { "Name":"FACT_WEB_AUTHENTICATIONS", "Type":"FACT", "column":[ { "Name":"Web Authentications Count", "Description":"Web Authentication Count", "expression":"1", "post-aggregation":false } ] }, { "Name":"DIM_DATE", "Type":"DIMENSION" }, { "Name":"DIM_CUSTOMER_LOCATION", "Type":"DIMENSION" }, { "Name":"DIM_CUSTOMER_STUDY_GROUP", "Type":"DIMENSION" }, { "Name":"DIM_WEB_PAGEVIEWS", "Type":"DIMENSION" }, { "Name":"DIM_CUSTOMER", "Type":"DIMENSION" } ] } ] } At 802, ETL jobs are built and executed. For example, a data analytics company working with a utility receives a daily feed of data on millions of customers for each utility client. The received raw data is run through a production analytics cluster to curate the information into a usable format. BI-ETL performs the functions of the process of extracting, transferring, and loading data into a business intelligence database. ETL is a three-step process in which data is extracted, transformed, and loaded into an output data container. Data can be collated from one or multiple sources and can also be output to one or multiple destinations.

[0052] In an embodiment, OAV-BI-ETL is a program written in Java language and runs on a data warehouse (e.g., a Hadoop cluster). OAV-BI-ETL extracts data from the cluster, transforms it, and loads it into a database that is used as a data source by Oracle Analytics or a BI system.

[0053] The latest code is used to update the database model to the latest version. In one embodiment, the database is a data source such as an Oracle database. The database model is a database structure including tables, columns, and views. In one embodiment, the database model is the star schema described above.

[0054] At 803, an RPD generator is executed to generate an RPD ("first RPD"). In an embodiment, the RPD is an XML file that represents a database model, but is generally not human readable. In one embodiment, 803 uses the RPD generator disclosed in U.S. Patent No. 10,169,378.

[0055] At 804, a business intelligence (“BI”) server tool is executed. The BI server tool uses the XML file from 803 as input and generates a revised RPD file (“second RPD”) for use by the OAS. The revised RPD includes a metadata configuration and catalog file from the JSON file from 801 that describe what should be included in the RPD. The metadata configuration included in the revised RPD further includes the schema of the database used to generate the RPD at 803. The BI server tool or BI system can be any BI system that utilizes RPDs. In one embodiment, the BI system is Oracle Utilities Opower Analytics Visualization, which enables utilities to explore customer data and create custom data extracts and insights related to their Opower programs. This BI system includes a rich set of pre-built analytical focus areas and visualizations that enable clients to derive strategic insights from their data, such as the number of communications sent to customers and the extent to which customers are engaged with Opower products.

[0056] At 805, the RPD is published by uploading the RPD file to a shared location, such as uploading the RPD file to a cloud. The RPD is uploaded in a compiled form of a visual configuration that can be downloaded to build an image, making it available to multiple tenants / users.

[0057] At 806, the RPD file is downloaded by a developer and used to build an Oracle Analytics Docker image. A "Docker" image is a file used to run code inside a Docker container. A Docker image serves as a set of instructions for building a Docker container, such as a template. A Docker image also serves as a starting point when using Docker. An image is equivalent to a snapshot in a virtual machine ("VM") environment. Docker is a set of platform-as-a-service products that uses OS-level virtualization to deliver software in packages called "containers." In other embodiments, other containerization capabilities can be used instead of Docker enforcing a virtualization environment framework. The compiled image is used by the BI system to automatically create data visualizations and emulate the results of received changes that can be received by the BI system as containers. Thus, at 806, a process runs in the container and uses declarative configuration to automatically generate visualizations that would otherwise be generated by a user via user input to a GUI (i.e., user generation is a known solution for generating customized visualizations per tenant, which is heavy lifting).

[0058] In an embodiment, upload / download is used because the first system hosts instances of the revised RPD that are used by one or more enterprise users. In other embodiments, instead of upload / download, the revised RPD is used directly to build the image.

[0059] In an embodiment, the functionality in Figure 8 is the result of a triggered Jenkins job. A Jenkins job is generally a user-defined set of sequential tasks. For example, a job may fetch source code from version control, compile code, run unit tests, and so on. Note that in Jenkins, the term "job" is synonymous with "pipeline."

[0060] FIG. 9 is a flow diagram of the detailed functionality of the dynamic metadata configuration inclusion module 16 of FIG. 2 when performing dynamic inclusion of metadata configurations into logical model features, according to an embodiment.

[0061] At 901, which corresponds to 801 in Figure 8, changes are made to the JSON configuration file and the changes are committed.

[0062] At 902, which corresponds to 802 in FIG. 8, database credentials are retrieved from a secure store, such as an Oracle database or data store 130 in FIG.

[0063] At 903, which corresponds to 802 in FIG. 8, the job source code (BI-ETL) is retrieved and the source is built into a JAR file.

[0064] At 904, which corresponds to 802 in FIG. 8, the external configuration of the BI-ETL job is retrieved and the job is executed to update the database.

[0065] At 905, which corresponds to 803 in FIG. 8, the RPD generator disclosed in US Pat. No. 10,169,378 is run against the updated database.

[0066] At 907, which corresponds to 803 in Figure 8, it is determined whether an XML file was generated by the RPD generator. If not, a failure alert is sent at 906. XML file generation can fail if the job code is incorrect, the database configuration is incorrect, the RPD generator validation is not passed, or an infrastructure issue occurs.

[0067] At 908, which corresponds to 804 in Figure 8, a BI client tool, for example from Oracle, is downloaded and executed to convert the XML to RPD. The BI client converts the XML output of the RPD generator into an RPD binary file.

[0068] At 910, which corresponds to 804 in Figure 8, it is determined whether an RPD file has been generated. If not, a failure alert is generated at 909.

[0069] At 911, which corresponds to 805 in FIG. 8, the RPD file is published to a shared location. At 912, which corresponds to 806 in FIG. 8, a job is triggered to build a custom OAS image using the RPD file.

[0070] At 913, which corresponds to 806 in FIG. 8, a new image is built using the updated RPD file.

[0071] Figure 9 shows command line commands for executing a process / method. In one embodiment, the commands are automated and executed as part of the code, but are specified as command line commands for visibility.

[0072] In an embodiment, the RPD file is generated and stored outside of the analysis instance, which then obtains the results of the disclosed process (i.e., the 806 / 913 OAS Docker image).

[0073] The functionality in Figure 9 does not require access to credentials, so anyone can make changes without access to the credentials (i.e., direct access to the credentials is not required to trigger the process). The credentials themselves are required to access the database during automated execution, but no human will ever read them.

[0074] Access control for triggering the process is controlled at the level of the source version control system (e.g., Git), which has its own credentials as well as a process for peer review of changes before they are accepted.

[0075] Embodiments provide superior security and conform to the infrastructure-as-code pattern, which is the process of managing and provisioning computer data centers through machine-readable definition files rather than physical hardware configurations or interactive configuration tools.

[0076] The features, structures, or characteristics of the present disclosure described throughout this specification may be combined in any suitable manner in one or more embodiments. For example, the use of "one embodiment," "some embodiments," "an embodiment," "some embodiments," or other similar language throughout this specification refers to the fact that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the present disclosure. Thus, the appearances of "one embodiment," "some embodiments," "an embodiment," "some embodiments," or other similar phrases throughout this specification do not necessarily all refer to the same embodiments, and the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0077] Those skilled in the art will readily appreciate that embodiments such as those described above may be practiced using steps in a different order and / or with elements in a different configuration than that disclosed. Thus, while this disclosure discusses general embodiments, it will be apparent to those skilled in the art that certain modifications, variations, and alternative configurations will be apparent while remaining within the spirit and scope of the disclosure. Accordingly, reference should be made to the appended claims to determine the scope of the disclosure.

Claims

1. 1. A method for generating changes to a logical model, comprising: receiving the changes in a configuration file, the changes including declarative configurations, the method comprising: extracting the changes, loading the changes into a database, and updating a corresponding database model; generating a first logical model representing the database model; generating a second logical model including the changes; and automatically generating, within a container, a visualization image compiled from the second logical model using the declarative configuration, the visualization image adapted for use by a business intelligence system that provides a visualization of the data incorporating the changes.

2. The method of claim 1 , wherein the first logical model includes a first RPD and the second logical model includes a second RPD.

3. The method of claim 2 , wherein generating the first logical model comprises using an RPD generator.

4. The method of claim 1 , wherein the changes include metadata.

5. The method of claim 1 , wherein the modifications include a catalog file.

6. 10. The method of claim 1, further comprising uploading the second logical model, wherein the uploaded second logical model is adapted to be downloaded by one or more enterprise users to generate visualizations of the data.

7. The method of claim 1 , further comprising modifying a database model schema, wherein the second logical model incorporates the modified database model schema.

8. The method of claim 1 , wherein the configuration file comprises a JavaScript object notation (JSON) file.

9. 1. A computer-readable medium storing instructions that, when executed by one or more processors, cause the processors to generate changes to a logical model, the generating comprising: receiving the changes in a configuration file, the changes including declarative configuration; and generating the extracting the changes, loading the changes into a database, and updating a corresponding database model; generating a first logical model representing the database model; generating a second logical model including the changes; and automatically generating, within a container, a visualization image compiled from the second logical model using the declarative configuration, the visualization image adapted for use by a business intelligence system that provides a visualization of the data incorporating the changes.

10. 10. The computer-readable medium of claim 9, wherein the first logical model includes a first RPD and the second logical model includes a second RPD.

11. The computer-readable medium of claim 10 , wherein generating the first logical model comprises using an RPD generator.

12. The computer-readable medium of claim 9 , wherein the changes include metadata.

13. The computer-readable medium of claim 9 , wherein the modifications include a catalog file.

14. The generating step comprises:

10. The computer-readable medium of claim 9, further comprising uploading the second logical model, wherein the uploaded second logical model is adapted to be downloaded by one or more enterprise users to generate a visualization of the data.

15. The generating step comprises:

10. The computer-readable medium of claim 9, further comprising modifying a database model schema, wherein the second logical model incorporates the modified database model schema.

16. The computer-readable medium of claim 9 , wherein the configuration file comprises a JavaScript object notation (JSON) file.

17. 1. A system for generating changes to a logical model, comprising: one or more processors, the one or more processors and configured to receive the changes in a configuration file, the changes including declarative configuration, the one or more processors: Extracting the changes, loading the changes into a database, and updating a corresponding database model; generating a first logical model representing the database model; generating a second logical model that includes the changes; The system is configured to automatically generate, within a container, a visualization image compiled from the second logical model using the declarative configuration, the visualization image adapted for use by a business intelligence system that provides a visualization of the data incorporating the changes.

18. 20. The system of claim 17, wherein the first logical model includes a first RPD and the second logical model includes a second RPD.

19. 20. The system of claim 18, wherein generating the first logical model comprises using an RPD generator.

20. The system of claim 17 , wherein the changes include metadata.

Images