System and method for real-time data aggregation in a virtual cube in a multidimensional database environment

Abstract

According to an embodiment, a system and method for real-time data aggregation in a virtual cube in a multidimensional database is described herein. The method may provide a multidimensional database server executed on a computer at a computer including one or more microprocessors, wherein the multidimensional database server supports at least one multidimensional cube, a data source associated with the multidimensional cube; and at least one real-time data source accessible by the data source. The method may query the data source through the multidimensional cube, the query including an identification of at least one real-time data source. The method may provide on-demand access to the multidimensional cube to at least one real-time data source through the data source. The method may use target data from at least one real-time data source within the multidimensional cube without storing it therein.

Description

[0001] Copyright Notice

[0002] A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.

[0003] Priority claim:

[0004] This application claims priority to U.S. Patent Application No. 16 / 374,402, filed on April 3, 2019, entitled “SYSTEM AND METHOD FOR REAL TIME DATA AGGREGATION IN A VIRTUAL CUBE IN A MULTIDIMENSIONAL DATABASE ENVIRONMENT,” and U.S. Provisional Patent Application No. 62 / 700,762, filed on July 19, 2018, entitled “SYSTEM AND METHOD FOR REAL TIME DATAAGGREGATION IN A VIRTUAL CUBE IN A MULTIDIMENSIONAL DATABASE ENVIRONMENT,” which are incorporated herein by reference. Technical Field

[0005] Embodiments of the present invention relate generally to databases and data warehouses, and more particularly to systems and methods for root cause analysis and automatic generation of key metrics in a multidimensional database environment. Background Art

[0006] The multidimensional database computing environment enables companies to deliver critical business information to the right people when they need it, including the ability to leverage and integrate data from multiple existing data sources and distribute filtered information to the end-user community in a format that best meets the user's needs. Users can interact with and explore data along familiar business dimensions in real time, enabling analysis at the speed of thought. These are some examples of the types of environments in which embodiments of the present invention can be used. Summary of the invention

[0007] According to an embodiment, a system and method for real-time data aggregation in a virtual cube in a multidimensional database according to an embodiment is described herein. A method can provide a multidimensional database server executed on a computer at a computer including one or more microprocessors, wherein the multidimensional database server supports at least one multidimensional cube and a data source associated with the multidimensional cube. The method can query a data source through a multidimensional cube, and the query includes an identification of at least one real-time data source accessible to the data source. The method can provide on-demand access to a multidimensional cube to at least one real-time data source through a data source. The method can use target data from at least one real-time data source within a multidimensional cube without storing it therein. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 An example of a multidimensional database environment according to an embodiment is illustrated.

[0009] Figure 2 An exemplary multidimensional database environment according to an embodiment is illustrated.

[0010] Figure 3 An exemplary embodiment for dynamic calculation according to an embodiment is illustrated.

[0011] Figure 4 The dynamic calculation of external data in a multidimensional database environment according to an embodiment is illustrated.

[0012] Figure 5 The dynamic calculation of external data in a multidimensional database environment according to an embodiment is illustrated.

[0013] Figure 6 A dynamic computation continuum is illustrated according to an embodiment.

[0014] Figure 7 A system for real-time data aggregation in a virtual cube according to an embodiment is illustrated.

[0015] Figure 8 A system for real-time data aggregation in a virtual cube according to an embodiment is illustrated.

[0016] Fig. 9 It is a method flow chart of a method for real-time data aggregation in a virtual cube in a multidimensional database. DETAILED DESCRIPTION

[0017] The above, together with other features, will become apparent with reference to the attached description, claims and drawings. Specific details are set forth in order to provide an understanding of the various embodiments. However, it will be apparent that the various embodiments can be practiced without these specific details. The attached description and drawings are not intended to be limiting.

[0018] Multidimensional database environments, examples of which include Oracle Essbase, can be used to integrate large amounts of data (in some cases data from multiple data sources) in a manner that addresses the specific requirements of end users, and distribute filtered information to those users.

[0019] Figure 1 An example of a multidimensional database environment 100 according to an embodiment is illustrated.

[0020] like Figure 1 As shown, according to an embodiment, a multidimensional database environment operating as a database layer may include one or more multidimensional database server systems 102, each of which may include physical computer resources or components 104 (e.g., microprocessor / CPU, physical memory, network components), an operating system 106, and one or more multidimensional database servers 110 (e.g., Essbase servers).

[0021] According to an embodiment, the middle tier 120 may include one or more services, such as, for example, provider services 122 (e.g., Hyperion provider services), governance services 124 (e.g., Essbase governance services), or studio / integration services 126 (e.g., Essbase studio / Essbase integration services). The middle tier may provide access to a metadata catalog 129 and / or one or more data sources 130 (e.g., relational databases) via ODBC / JDBC 127, 128 or other types of interfaces for use with a multidimensional database environment.

[0022] According to an embodiment, one or more data sources may also be accessed by one or more multidimensional database servers via ODBC / JDBC 132 or other types of interfaces for use in providing a multidimensional database.

[0023] According to an embodiment, the client layer 140 may include one or more multidimensional database clients 142 (e.g., Essbase server clients) that enable access to a multidimensional database (such as, for example, Smart View, Spreadsheet Add-in, Smart Search, Management Services, MaxL, XMLA, CAPI or VB API applications, Oracle Business Intelligence Enterprise Edition Plus, or other types of multidimensional database clients). The client layer may also include a console for use with services in the middle layer, such as, for example, a management service console 144 or a studio / integration service console 146.

[0024] According to an embodiment, communication between the client, middle tier, and database tier may be provided by one or more of TCP / IP, HTTP, or other types of network communication protocols.

[0025] According to an embodiment, a multidimensional database server may integrate data from one or more data sources to provide a multidimensional database, data structure, or cube(s) 150 , which may then be accessed to provide filtered information to end users.

[0026] In general, each data value in a multidimensional database is stored in a cell of a cube; and a particular data value can be referenced by specifying its coordinates along a dimension of the cube. The intersection of a member from one dimension with a member from each of one or more other dimensions represents a data value.

[0027] For example, Figure 1 As shown, the figure illustrates a cube 162 that can be used in a sales-oriented business application. When a query indicates "Sales", the system can interpret this query as a slice or layer 164 of data values ​​within the database that contains all "Sales" data values, where "Sales" is intersected with "Actual" and "Budget". To reference a specific data value 166 in the multidimensional database, the query can specify members on each dimension, such as by specifying "Sales, Actual, January". Slicing the database in different ways provides different perspectives on the data; for example, a slice of data values ​​168 for "February" examines all those data values ​​whose time / year dimension is fixed to "February".

[0028] Database Overview

[0029] According to an embodiment, the development of a multidimensional database begins with the creation of a database schema that defines the structural relationships between members in the database; organizes data in the database; and defines integrations and mathematical relationships. Within the hierarchical tree or data structure of the database schema, each dimension includes one or more members, which in turn may include other members. The specification of a dimension guides the system how to integrate the values ​​of its various members. An integration is a group of members within a branch of the tree.

[0030] Dimensions and Members

[0031] According to an embodiment, a dimension represents the highest level of integration in a database profile. Standard dimensions may be selected to represent components of a business plan related to departmental functions (e.g., time, account, product line, market, region). Attribute dimensions associated with standard dimensions enable users to group and analyze members of standard dimensions based on member attributes or characteristics. Members (e.g., Product A, Product B, Product C) are individual components of a dimension.

[0032] Dimensions and Membership

[0033] According to an embodiment, a multidimensional database uses family (parents, children, siblings; descendants and ancestors); and hierarchy (generations and levels; roots and leaves) terms to describe the roles and relationships of members in a database schema.

[0034] According to an embodiment, a parent is a member that has branches below it. For example, "Margin" can be a parent of "Sales" and "Cost of Goods Sold" (COGS). A child is a member that has a parent above it. In the above example, "Sales" and "Cost of Goods Sold" are children of the parent "Margin". Siblings are children of the same direct parent within the same generation.

[0035] According to an embodiment, descendants are members in the branch below a parent. For example, "Profit", "Inventory", and "Ratios" can be descendants of Measures; in this case, the children of "Profit", "Inventory", and "Ratios" are also descendants of Measures. Ancestors are members in the branch above a member. In the above example, "Gross Profit", "Profit", and Measures can be ancestors of "Sales".

[0036] According to an embodiment, the root is the top member in a branch. For example, a metric can be the root of "Profit", "Inventory", and "Ratio"; and also the root of the children of "Profit", "Inventory", and "Ratio". Leaf (level 0) members have no children. For example, Open "Inventory", Add, and End "Inventory" can be leaf members.

[0037] According to an embodiment, a generation refers to a level of integration within a dimension. The root branch of the tree is considered "generation 1," and the numbering of generations increases from the root to the leaf members. Levels refer to branches within a dimension; and are numbered inversely to the numerical ordering used for generations, with level numbers decreasing from leaf members to their roots.

[0038] According to an embodiment, a user may assign a name to a generation or level and use that name as shorthand for all members in that generation or level.

[0039] Sparse and dense dimensions

[0040] Data sets within a multidimensional database often share two characteristics: the data is not smoothly and evenly distributed; and data does not exist for most member combinations.

[0041] According to an embodiment, to solve this problem, the system can identify two types of standard dimensions: sparse dimensions and dense dimensions. A sparse dimension is a dimension with a relatively low percentage of available data positions being filled; while a dense dimension is a dimension in which there is a relatively high probability that one or more cells are occupied in each combination of the dimension. Many multidimensional databases are inherently sparse because they lack data values ​​for most member combinations.

[0042] Data Block and Index System

[0043] According to an embodiment, a multidimensional database uses data blocks and indexes to store and access data. The system can create a multidimensional array or data block for each unique combination of sparse standard dimension members, where each data block represents dense dimension members for its combination of sparse dimension members. An index is created for each data block, where the index represents a combination of sparse standard dimension members and includes an entry or pointer for each unique combination of sparse standard dimension members for which at least one data value exists.

[0044] According to an embodiment, when the multidimensional database server searches for a data value, it may use the pointer provided by the index to locate the appropriate data block; and within that data block, locate the cell containing the data value.

[0045] Management and control services

[0046] According to an embodiment, a governance service (eg, Essbase governance service) provides a single point of access that enables users to design, develop, maintain, and manage servers, applications, and databases.

[0047] studio

[0048] According to an embodiment, a studio (eg, Essbase studio) provides a wizard-driven user interface for performing tasks related to data modeling, cube design, and analytical application construction.

[0049] Spreadsheet Add-ins

[0050] According to an embodiment, the spreadsheet add-in integrates a multidimensional database with a spreadsheet, providing support for enhanced commands such as Connect, Pivot, Drill-down, and Calculate.

[0051] Integration Services

[0052] According to an embodiment, an integration service (eg, Essbase integration service) provides a metadata-driven environment used in integrating between data stored in a multidimensional database and data stored in a relational database.

[0053] Provider Services

[0054] According to an embodiment, a provider service (eg, Hyperion provider service) operates as a data source provider for Java API, Smart Views, and XMLA clients.

[0055] Smart View

[0056] According to an embodiment, Smart Views provide a common interface to, for example, Hyperion Financial Management, Hyperion Planning, and Hyperion Enterprise Performance Management workspace data.

[0057] Developer Products

[0058] According to an embodiment, a developer product enables rapid creation, management, and deployment of customized enterprise analytics applications.

[0059] Lifecycle Management

[0060] According to an embodiment, lifecycle management (eg, Hyperion Enterprise Performance Management System Lifecycle Management) provides means for enabling an enterprise performance management product to migrate applications, repositories, or individual artifacts across product environments.

[0061] OLAP

[0062] According to an embodiment, online analytical processing (OLAP) provides an environment that enables users to analyze enterprise data. For example, the financial department can use OLAP for applications such as budgeting, activity-based costing, financial performance analysis, and financial modeling to provide "just-in-time" information.

[0063] According to an embodiment, an OLAP system can organize data into multiple dimensions, allowing a searcher / user of a data set to perform a directional search that traverses various dimensions to ultimately obtain results of interest. An OLAP system can view data as residing at the intersection of dimensions. In other words, data under an OLAP system can be organized and stored as a multidimensional database that is an instantiation of the cross product of all dimensions. This allows a user / searcher to traverse the hierarchy of details along the dimensions of interest in an ad hoc manner to obtain specific target data. Slowly changing data can be represented as metadata within the current data set.

[0064] Real-time data aggregation in virtual cubes

[0065] According to an embodiment, in a conventional system, one of the advantages of a multidimensional database cube is that all underlying data (e.g., source data) is loaded into the multidimensional database. Since the multidimensional database does not have to retrieve data for each requested analysis, this allows faster analysis, and since the data is loaded at the site of the multidimensional database, this can also speed up the calculation. However, as the trend towards big data continues, loading all source data into a multidimensional database becomes inconvenient and can lead to performance degradation in some cases. By supporting real-time data aggregation, a multidimensional database can have multiple (distributed) cubes that all reference the same relational database (e.g., source data). Instead of retrieving data from a loaded source, the multidimensional database can query one or more source data locations and aggregate the data in real time.

[0066] Figure 2 An exemplary multidimensional database environment according to an embodiment is illustrated.

[0067] According to an embodiment, within the multidimensional database server 200, there may be one or more database cubes 210. Based on each of these cubes, one or more partitions 211 may be generated. Partitions may be used to simplify data access across multiple cubes for users 220. Partitions allow users and administrators to select specific subsets of data and metadata for analysis using replicated, transparent, or linked partitioning schemes.

[0068] According to an embodiment, cube 210 may be populated with data from a plurality of external data sources, such as another cube 202, a database 203, data 204 (e.g., in the form of a comma-delimited list of variables), and / or a spreadsheet 205. (Although only certain external data sources are shown in the figure, one of ordinary skill in the art will recognize that additional and varying other data sources may be accessed and loaded into cube 210.) However, all external data must first be loaded into cube 210 in a conventional system, whether it is from another cube 202, a database 203, a CSV file 204, or from a spreadsheet 205.

[0069] According to an embodiment, in a traditional system where source data is relatively small, loading source data into a multidimensional database server can be considered an advantage of a multidimensional database environment because it allows data aggregation and calculation. Calculation and aggregation occur on data located within the multidimensional database environment. Data is loaded, aggregated, and calculated in its own repository.

[0070] Figure 3 An exemplary embodiment for dynamic calculation according to an embodiment is shown.

[0071] According to an embodiment, Figure 3 It shows an additional advantage gained when utilizing a multidimensional database environment. This is the idea of ​​using hybrid dynamic calculations and aggregations.

[0072] According to an embodiment, dynamic calculation 300 can include multiple loading values ​​and reduce the size of the database. In dynamic calculation, the loading values ​​A 301, B 302, C 303, D 304 and E 305 loaded into the multidimensional database of the managed dynamic calculation 300 from an external source 320 (or multiple external sources) can be used for real-time dynamic calculation. This enables users to calculate metrics on the loaded and stored values ​​in real time. These metrics can include, for example, F 306, G 307, H 308, I 309 and J 310. However, calculation is carried out on the stored values ​​(i.e., the values ​​loaded and stored in the cube from the external source). These metrics can be additionally stored in the multidimensional database, which further increases the occupied space of the multidimensional database.

[0073] According to an embodiment, the size of the data loaded into the cube may then be reduced based on dependency analysis such as shown in dynamic calculation 300. In this manner, less data is stored in the cube since a smaller number of data dimensions need to be loaded into the cube or provided by the user.

[0074] According to an embodiment, dynamic calculation of loaded values ​​can reduce the size of the database. In addition, it provides a real-time ability to calculate metrics on loaded and stored values. However, since the calculation is performed on the values ​​(e.g., leaf values) loaded and stored in the multidimensional database, the overall size of the database may still be large because all values ​​from external sources are loaded and stored, and the dynamic calculation will be run on these values ​​(but will not be stored itself).

[0075] Figure 4 The dynamic calculation of external data in a multidimensional database environment according to an embodiment is illustrated.

[0076] According to an embodiment, dynamic calculation 400 may not rely on Figure 3 400 , but rather (via a database interface, such as an SQL database interface) to an external source 420 (or other storage location type) where the values ​​A 401, B 402, C 403, D 404, and E 405 are located, and retrieved on demand (dynamically). For example, such a retrieval operation may occur when a dynamic calculation metric is executed. For example, if a metric F 406 is desired or called to execute, then the system and method may retrieve the values ​​A 401 and B 402 from the external source 420 on demand, which are then used to determine the metric F, which may be used in the dynamic calculation 400. However, the values ​​of A and B will not be stored, but rather discarded after use, thereby minimizing the storage footprint utilized by the dynamic calculation in a multidimensional database server environment.

[0077] According to an embodiment, the values ​​extracted from the external source 420 are determined based on a user query or based on a query from a cube (where dynamically calculated). In this way, only the requested / target data is accessed, while other data that may be located at the external source (e.g., unloaded data) is ignored until a call or request is made that relies on data at that source.

[0078] According to an embodiment, in this manner, where all values ​​within a cube are dynamically calculated, and all data is extracted directly from the data source, then the cube has a zero footprint - all data comes directly from the table in real time, and any dynamic calculations will be run on these values ​​that are not loaded and stored. Such a system and method, in addition to having a zero footprint (no values ​​are loaded and stored), also has the advantages of high availability and zero downtime backup and patching.

[0079] According to an embodiment, when values ​​are obtained from an external source, these values ​​are not stored or cached on the multidimensional database server. Instead, these values ​​are used directly in the cube (eg, for dynamic calculations).

[0080] According to other embodiments, the systems and methods may support caching of values ​​obtained from external sources. For example, this may be done at timed intervals. In this case, as an example, real-time acquired data may be saved to / overwritten with cached data in a multidimensional database environment at configured time intervals. This may be done, for example, when it is desired to have a cache backup or view historical trends. In this case, the multidimensional database cube may still be said to have a zero footprint because if the multidimensional database cube crashes or becomes corrupted, any and all cached data may be cleared and a new multidimensional database cube may be configured to start a new cache without referencing the previous multidimensional database cube.

[0081] Figure 5 The dynamic calculation of external data in a multidimensional database environment according to an embodiment is illustrated.

[0082] According to an embodiment, dynamic calculation 500 is not just Figure 3 In the embodiment of the present invention, instead of relying on stored values ​​and relying only on obtaining values ​​from an external source 520 as shown in FIG, a hybrid approach can be used, in which frequently used values ​​(such as, values ​​C 503 and E 505) are loaded, while other infrequently used values ​​(such as, A 501, B 502 and D 504) are retained in the external source and accessed via an acquisition operation (e.g., via a database interface, such as an SQL database interface) at the external source 520 (or other storage location type). This hybrid approach allows certain values ​​to be loaded, while other values ​​are extracted from the external source on demand (dynamically). For example, such an acquisition operation can occur when a dynamic calculation metric is executed. For example, if a metric G 507 is expected or called to execute, the system and method can obtain the value D 504 from the external source on demand, and use the value E 505 from the local storage device to determine the metric G that can be used in the dynamic calculation 500. However, the value of D will not be stored, but will be discarded after use, thereby reducing the storage footprint used by the dynamic calculation in the multidimensional database server environment.

[0083] According to an embodiment, the values ​​extracted from the external source 520 are determined based on a user query or based on a query from a cube (where dynamically calculated). In this way, only the requested / target data is accessed, while other data that may be located at the external source (e.g., unloaded data) is ignored until a call or request is made that relies on data at that source.

[0084] According to an embodiment, in this manner, where all values ​​within a cube are dynamically calculated, some values ​​are stored, and some values ​​are extracted directly from a data source, then the cube has a reduced footprint compared to if all data were stored. In addition, by actively and automatically selecting values ​​to load and store and values ​​to maintain only at the source, the performance of the multidimensional database can be improved by reducing the size of those values ​​that are not loaded and stored and also maintaining the same performance level for those values ​​that are loaded and stored.

[0085] Figure 6 A dynamic computation continuum is illustrated according to an embodiment.

[0086] According to an embodiment, within the dynamic computing continuum 600, on one side (left side), there is a cube where all data is loaded and stored within a multidimensional database cube. This is desirable when the data is relatively small and is not frequently changed or updated (e.g., a user needs to refresh the data once a day). On the other side of the continuum (far right), there is a cube where all data is leaf data loaded from the outside. These situations are desirable in organizations where there is a large amount of leaf data, or when the source data changes multiple times a day or is constantly refreshed (e.g., when a user needs to see the impact of dynamically changing data (such as monetary data)).

[0087] According to an embodiment, when all data is loaded externally, then the multidimensional database cube does not need to worry about disaster recovery. If the cube crashes and loses all dynamic calculations, the external data source is not affected and the cube can be restarted using the values ​​extracted from the external data source.

[0088] Depending on the embodiment, there are combinations of these two extremes. For example, a user may request that at every configurable time period, the cube access and extract data from an external source and cache that data. However, since the data is cached (and discarded after use), the footprint of the data cube is still minimal because the values ​​are not stored.

[0089] Figure 7 A system for real-time data aggregation in a virtual cube according to an embodiment is illustrated.

[0090] According to an embodiment, a multidimensional database server 707 (e.g., an Essbase server). The multidimensional database server 707 provides many services, such as tracing 703, data and dimension loading 704, security filters 705, and partitioning 706. Such services can be provided for a data source 702, which can be an abstract table representation based on / deployed on a connection 701.

[0091] According to an embodiment, for example, a connection 701 may be defined to an external data source (e.g., a CSV file 710, a database 711 (e.g., cloud or local deployment), a business intelligence or data visualization source 712, another multidimensional database cube 713 (e.g., Essbase), a SQL database 714, HIVE 715, or other database (such as, SQL server, DB2, Teradata) 716). The connection may include, for example, an Essbase connection, a file connection, a JDBC connection, a physical connection, or any other type of connection that can work with and communicate with the selected external data source. On top of this connection, a data source 702 may be defined. The data source may, for example, include a data structure based on the selected external data source. For example, if an Excel worksheet is the selected external data source, then on top of the file connection, the data source may define a data structure that includes many columns, rows, tabs, and parameters of the Excel worksheet. This then defines a data source that the multidimensional database server may act on via a service. Likewise, multiple data sources may be defined for one connection.

[0092] Then, according to an embodiment, the multidimensional database server may utilize, for example, partitioning services for data sources, where, as described above, partitioning services may allow users and administrators to select specific subsets of data and metadata from a data source for analysis using replicated, transparent, or linked partitioning schemes.

[0093] According to an embodiment, the multidimensional database server can also use, for example, data dimensions and loading services to load the selected data into the multidimensional database server (for example, as described above). Figure 3 ).

[0094] According to an embodiment, the multidimensional database server can also utilize, for example, a traceability service to dynamically acquire the selected data in real time to minimize the occupied space of the multidimensional database server (for example, as described above). Figure 4 ).

[0095] According to an embodiment, the multidimensional database server may also utilize multiple services in conjunction with each other. For example, the multidimensional database server may utilize partitioning services, as well as data dimension and loading services and traceability services for one data source. For example, this would allow loading and storing certain data, dynamically acquiring other data, and only for those data dimensions that apply to the configured partitions.

[0096] Depending on the embodiment, the connection can be to any number of external data sources, such as a CSV file 710, a database 711 (e.g., cloud or on-premises), a business intelligence or data visualization source 712, another multidimensional database cube 713 (e.g., Essbase), a SQL database 714, HIVE 715, or other database (such as, SQL Server, DB2, Teradata) 716.

[0097] According to an embodiment, in this way, this allows the multidimensional database server 707 to trace directly to the connected data source, which can be any number of data repositories. Then, the request / target data can be extracted in real time into the multidimensional database server 707, and then the multidimensional database server 707 can use the data in dynamic calculations without storing the real-time data in the multidimensional database server 707.

[0098] Real-time data aggregation in virtual cubes on top of big data

[0099] According to an embodiment, more and more external data is being compiled in the context of "big data" (e.g., petabyte-level information). To perform real-time data aggregation in a virtual cube on top of a big data cluster, a source aggregation agent can be deployed inside the big data cluster. Real-time and partitioned multidimensional database cubes can then be launched for aggregation clients in the big data cluster (e.g., one cube per aggregation agent, one cube for multiple aggregation clients, or multiple cubes for one aggregation client, all depending on the needs of the system).

[0100] Then, according to an embodiment, transparent partitioning can reside on top of a real-time and partitioned multidimensional database cube. This can be extended by distributed queries, source aggregation, and dynamic calculation and real-time access to source data (ie, without loading and storing source data).

[0101] Figure 8 A system for real-time data aggregation in a virtual cube according to an embodiment is illustrated.

[0102] According to an embodiment, a source aggregation agent (not shown) may be deployed in a big data cluster 820. A source aggregation client may, for example, reduce the size of the source data exponentially (e.g., from petabytes to terabytes or gigabytes). Real-time and partitioned multidimensional database cubes 810, 811, 812, 813 for aggregation above aggregated big data may be located at a level above the source data with an aggregation agent, wherein each real-time and partitioned multidimensional cube further reduces the data size that each cube is responsible for dynamically acquiring data from a big data source or one or more source aggregation agents. One or more transparent partitions 800 may be located above the real-time and partitioned multidimensional database cubes 810, 811, 812, 813. In this way, the system and method may achieve expansion through distributed query, source aggregation, dynamic calculation, and real-time access to source data.

[0103] According to an embodiment, a data lake with a source aggregation agent can greatly reduce the size of a large data set. Then, real-time and partitioned multidimensional database cubes 810, 811, 812, and 813 can be located on top of the aggregated data, access the data in real time (as described above), and further reduce the data to a more manageable size. Each of these partitioned multidimensional database cubes can be associated with one or more attributes (such as geographic attributes). For example, in the case where a large data set with an aggregator can contain data from all over the world, each real-time and partitioned multidimensional database cube 810, 811, 812, 813 can, for example, dynamically obtain only data associated with its assigned geographic range (e.g., North America, South America, Africa, Europe, etc.).

[0104] According to an embodiment, once data is cached / computed at a real-time and partitioned multidimensional database cube 810, 811, 812, 813, another partitioned multidimensional database cube (e.g., transparent partitioning) 800 can access the data in the real-time and partitioned multidimensional database cube 810, 811, 812, 813 in real-time, as described above.

[0105] According to an embodiment, in this manner, data from big data sources may be dynamically extracted into a multidimensional database cube 800 where users may manipulate and perform desired calculations in a much more efficient manner.

[0106] According to an embodiment, many benefits are generated from real-time data aggregation in a virtual cube. For example, such a system and method provides real-time access to table data in formulas and aggregations. Likewise, updates to source data will take effect immediately, which is important for modeling "volatile" data-driven programs (e.g., currency) that help users achieve limited and feasible plans.

[0107] As described above, in an embodiment, real-time data aggregation in a virtual cube also provides zero footprint. This in turn eliminates the need for backups and also provides zero downtime patching. High availability is also provided because multiple cubes can be driven by the same tabular data.

[0108] According to an embodiment, real-time data aggregation in a virtual cube provides active-active and scales out to multiple geographic locations and data centers from the same cube. This allows scaling to a large number of users, reducing latency to such users. Reorganization is also faster because currently almost all of the reorganization time is spent on data reshaping. This is avoided because there is no stored data to reshape. Also, due to the above-mentioned computing continuum, users can choose the data aggregation and storage model that suits them, allowing for improved personalization.

[0109] Fig. 9 is a method flow chart of a method for real-time data aggregation in a virtual cube in a multidimensional database.

[0110] At step 910 , the method may provide, at a computer including one or more microprocessors, a multidimensional database server executing on the computer, wherein the multidimensional database server supports at least one multidimensional cube and a data source associated with the multidimensional cube.

[0111] At step 920 , the method may query the data source through the multidimensional cube, the query including an identification of at least one real-time data source accessible to the data source.

[0112] At step 930 , the method may query the data source through the multidimensional cube, the query including an identification of at least one real-time data source accessible to the data source.

[0113] At step 940 , the method may query the data source through the multidimensional cube, the query including an identification of at least one real-time data source accessible to the data source.

[0114] Although various embodiments of the present invention have been described above, it should be understood that they are presented by way of example and not limitation. The embodiments are chosen and described in order to explain the principles of the present invention and its practical application. The embodiments illustrate systems and methods in which the present invention is utilized to improve the performance of the systems and methods by providing new and / or improved features and / or providing benefits such as reduced resource utilization, increased capacity, improved efficiency, and reduced latency.

[0115] In some embodiments, features of the present invention are implemented in whole or in part in a computer including a processor, a storage medium such as a memory, and a network card for communicating with other computers. In some embodiments, features of the present invention are implemented in a distributed computing environment, where one or more computer clusters are connected via a network such as a local area network (LAN), a switched fabric network (e.g., InfiniBand), or a wide area network (WAN). A distributed computing environment can have all computers in a single location, or computer clusters in different remote geographic locations connected via a WAN.

[0116] In some embodiments, features of the present invention are implemented in whole or in part in the cloud as part of or as a service of a cloud computing system based on shared elastic resources delivered to users in a self-service, metered manner using Web technologies. Clouds have five characteristics, as defined by the National Institute of Standards and Technology: on-demand self-service; extensive network access; resource pooling; rapid elasticity; and measured services. Cloud deployment models include: public, private, and hybrid. Cloud service models include software as a service (SaaS), platform as a service (PaaS), database as a service (DBaaS), and infrastructure as a service (IaaS). As used herein, a cloud is a combination of hardware, software, network, and Web technologies that provide shared elastic resources to users in a self-service, metered manner. Unless otherwise specified, clouds as used herein include public, private, and hybrid cloud embodiments, as well as all cloud deployment models, including but not limited to cloud SaaS, cloud DBaaS, cloud PaaS, and cloud IaaS.

[0117] In some embodiments, the features of the present invention are implemented using hardware, software, firmware, or a combination thereof or with the help of them. In some embodiments, the features of the present invention are implemented using a processor configured or programmed to perform one or more functions of the present invention. In some embodiments, the processor is a single-chip or multi-chip processor, a digital signal processor (DSP), a system on a chip (SOC), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a state machine, a discrete gate or transistor logic, a discrete hardware component, or any combination thereof designed to perform the functions described herein. In some implementations, the features of the present invention can be implemented by a circuit system specific to a given function. In other implementations, these features can be implemented in a processor that is configured to perform a specific function using instructions stored, for example, on a computer-readable storage medium.

[0118] In some embodiments, features of the present invention are incorporated into software and / or firmware for controlling the hardware of a processing and / or network system, and for enabling a processor and / or network to interact with other systems utilizing features of the present invention. Such software or firmware may include, but is not limited to, application code, device drivers, operating systems, virtual machines, hypervisors, application programming interfaces, programming languages, and execution environments / containers. Based on the teachings of the present disclosure, a skilled programmer can easily prepare appropriate software coding, which is apparent to a person skilled in the art of software.

[0119] In some embodiments, the present invention includes a computer program product, which is a storage medium or computer-readable medium having instructions stored thereon / in which can be used to program or otherwise configure a system such as a computer to perform any process or function of the present invention. The storage medium or computer-readable medium can include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data. In a particular embodiment, the storage medium or computer-readable medium is a non-transient storage medium or a non-transient computer-readable medium.

[0120] Thus, from one perspective, systems and methods for supporting real-time data aggregation in virtual cubes in a multidimensional database have been described. According to an embodiment, the systems and methods provided herein provide the ability to support real-time data aggregation in a multidimensional database. A multidimensional database can have multiple (distributed) cubes that all reference the same relational database (e.g., source data). Rather than retrieving data from a loaded source, a multidimensional database can query one or more source data locations and aggregate the data in real time.

[0121] The foregoing description is not intended to be exhaustive or to limit the invention to the precise form disclosed. In addition, where a particular series of affairs and steps have been used to describe an embodiment of the invention, it is apparent to those skilled in the art that the scope of the invention is not limited to the described series of affairs and steps. In addition, where a particular combination of hardware and software has been used to describe an embodiment of the invention, it should be appreciated that other combinations of hardware and software are also within the scope of the invention. In addition, although various embodiments describe a particular combination of features of the invention, it should be understood that within the scope of the invention, different combinations of features are apparent to those skilled in the relevant art, such that the features of one embodiment may be incorporated into another embodiment. Moreover, it is apparent to those skilled in the relevant art that various additions, reductions, deletions, changes, and other modifications and changes may be made in form, details, implementation, and application without departing from the spirit and scope of the invention. The broader spirit and scope of the invention are defined by the appended claims and their equivalents.

Claims

1. A system for real-time data aggregation in a virtual cube in a multidimensional database, comprising: A computer containing one or more microprocessors; a multidimensional database server executing on a computer; A multidimensional database stored on a multidimensional database server; a plurality of real-time data cubes of a multidimensional database, wherein each of the plurality of real-time data cubes is associated with a data attribute; transparent partitioning of a multidimensional database, wherein the transparent partitioning resides on top of the plurality of real-time data cubes; as well as a provider service for a multidimensional database server, wherein the provider service provides a data source including a plurality of connections to a database, the database including a real-time data source external to the multidimensional database, wherein the real-time data source includes a big data source, the big data source includes a plurality of source data aggregation clients, and wherein the data source provides on-demand access to the multidimensional database to at least one of the plurality of source data aggregation clients, wherein, in response to a query to the multidimensional database, each of the plurality of real-time data cubes is configured to query the data source, the query including an identification of the real-time data source external to the multidimensional database; Wherein the data source provides on-demand access to each of the plurality of real-time data cubes via the plurality of connections to the database including the real-time data source external to the multidimensional database.

2. The system according to claim 1, in, When executing a query, target data is obtained from the real-time data source through the data source.

3. The system according to claim 2, in, After being acquired by the data source, the target data is used by the multidimensional database for one or more dynamic calculations.

4. The system according to claim 3, in, The one or more dynamic calculations are query based.

5. The system according to claim 4, in, Transparent partitioning resides on top of the plurality of real-time data cubes and the one or more multidimensional database cubes.

6. The system of claim 1, in, Each of the plurality of source data aggregation clients is associated with a separate one of the plurality of real-time data cubes of the multidimensional database.

7. The system of claim 1, in, The multiple source data aggregation clients are deployed inside the big data cluster of the big data source.

8. A method for real-time data aggregation in a virtual cube in a multidimensional database, comprising: Providing a computer including one or more microprocessors; operating a multidimensional database server on said computer; storing the multidimensional database on a multidimensional database server; creating a plurality of real-time data cubes of a multidimensional database on a multidimensional database server, wherein each of the plurality of real-time data cubes is associated with a data attribute; creating a transparent partition of a multidimensional database on a multidimensional database server, wherein the transparent partition resides on the plurality of real-time data cubes; operating a provider service of a multidimensional database server to provide a data source including a plurality of connections to a database, the database including a real-time data source external to the multidimensional database, wherein the real-time data source includes a big data source, the big data source includes a plurality of source data aggregation clients, and wherein the data source provides on-demand access to the multidimensional database to at least one of the plurality of source data aggregation clients; receiving a query against a multidimensional database; querying the data source using the plurality of real-time data cubes, the query including identifying the real-time data source outside the multidimensional database; providing on-demand access to each of the plurality of real-time data cubes via the plurality of connections to the database including the real-time data source external to a multidimensional database; as well as The target data from the real-time data source is used within the multidimensional database without being stored therein.

9. The method according to claim 8, in, When executing a query, target data is obtained from the real-time data source through the data source.

10. The method according to claim 9, in, After being acquired by the data source, the target data is used by the multidimensional database for one or more dynamic calculations.

11. The method according to claim 10, in, The one or more dynamic calculations are query based.

12. The method according to claim 11, in, Transparent partitioning resides on top of the plurality of real-time data cubes and the one or more multidimensional database cubes.

13. The method according to claim 8, in, Each of the plurality of source data aggregation clients is associated with a separate one of the plurality of real-time data cubes of the multidimensional database.

14. The method according to claim 8, in, The multiple source data aggregation clients are deployed inside the big data cluster of the big data source.

15. A non-transitory computer-readable storage medium having instructions for operating real-time data aggregation in a virtual cube in a multidimensional database on a computer comprising one or more microprocessors, which, when read and executed, causes the computer to perform the following steps: operating a multidimensional database server on said computer; storing the multidimensional database on a multidimensional database server; creating a plurality of real-time data cubes of a multidimensional database on a multidimensional database server, wherein each of the plurality of real-time data cubes is associated with a data attribute; creating a transparent partition of a multidimensional database on a multidimensional database server, wherein the transparent partition resides on the plurality of real-time data cubes; operating a provider service of a multidimensional database server to provide a data source including a plurality of connections to a database, the database including a real-time data source external to the multidimensional database, wherein the real-time data source includes a big data source, the big data source includes a plurality of source data aggregation clients, and wherein the data source provides on-demand access to the multidimensional database to at least one of the plurality of source data aggregation clients; receiving a query against a multidimensional database; querying the data source using the plurality of real-time data cubes, the query including identifying the real-time data source outside the multidimensional database; providing on-demand access to each of the plurality of real-time data cubes via the plurality of connections to the database including the real-time data source external to a multidimensional database; as well as Use the target data from the real-time data source within the multidimensional database without storing it therein.

16. The non-transitory computer readable storage medium of claim 15, in, When executing a query, target data is obtained from the real-time data source through the data source.

17. The non-transitory computer readable storage medium of claim 16, in, After being acquired by the data source, the target data is used by the multidimensional database for one or more dynamic calculations.

18. The non-transitory computer readable storage medium of claim 17, in, The one or more dynamic calculations are query based.

19. The non-transitory computer readable storage medium of claim 18, in, Transparent partitioning resides on top of the plurality of real-time data cubes and the one or more multidimensional database cubes.

20. The non-transitory computer readable storage medium of claim 15, in, Each of the plurality of source data aggregation clients is associated with a separate one of the plurality of real-time data cubes of the multidimensional database.

21. A computer program product comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 8 to 14.

Images