JOIN operations on data sets with inconsistent dimensionality

Abstract

A database system comprising an interface and a processor may perform the techniques described in this disclosure. The interface may receive a query to access a first dataset and a second dataset, the query identifying a plurality of dimensions of the first dataset and the second dataset. The first dataset may include a first dimension and the second dataset may include a second dimension but excludes the first dimension. The processor may convert the query into a structured query conforming to a database query language, the structured query defining a join between the first dataset and the second dataset across the second dimension. The processor may also send the structured query to a database and receive an indication in response to the structured query that a virtual table has been created joining the first dataset and the second dataset across the second dimension of the second dataset.

Description

[Technical field]

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 267,634, entitled "JOIN OPERATIONS FOR DATASETS WITH INCONSISTENT DIMENSIONS," filed February 7, 2022, the entire contents of which are incorporated herein by reference. [Background technology]

[0002] Formal database languages ​​such as Structured Query Language (SQL), SQLite, PostgreSQL, Oracle SQL, etc., define the syntax and semantics associated with defining, editing, and otherwise managing datasets stored in a database. As an example, SQL may be a domain-specific language designed for use in programming and managing datasets held in a relational database management system (RDBMS). There are a number of commands supported by the syntax, which may invoke functions to perform various operations on the datasets. One such command, called the JOIN command, may be used to form a clause that combines rows from two or more datasets based on related columns between them.

[0003] According to the syntax of the formal database language, a JOIN command may require "consistent" dimensions in that all of the multiple datasets referenced by the JOIN command must contain a common column. If any one of the datasets referenced in the JOIN command does not contain the base column, the JOIN command fails because the "dimensionality is inconsistent" between the datasets. Such a JOIN command of dimensionally inconsistent datasets may result in inconsistencies that may be rejected by the strict syntax of the formal database language. Although such inconsistencies in database management, such as inconsistent dimensionality between datasets, may need to be avoided, there may be some instances where "inconsistent dimensionality" between inconsistent datasets and corresponding JOIN commands may result in useful aggregate data. Summary of the Invention

[0004] Various aspects of the present technology are directed to enabling a database management system to perform a JOIN operation on multiple datasets having inconsistent dimensionality. A first dataset of the multiple datasets may share one or more columns in common with a second dataset of the multiple datasets, but the JOIN operation may specify certain columns that are not shared, or in other words, not common, between the first and second datasets. To overcome the inconsistent dimensionality between the first and second datasets, the front-end server may convert the JOIN operation into a formal structured database JOIN based on metadata indicating compatibility between columns of the second dataset that are only present in the first dataset. The front-end server may then ignore the compatible dimensions specified by the metadata for the second dataset and form a JOIN command for the common (or in other words, consistent) columns between the first and second datasets. In this regard, the front-end server may form an implicit JOIN operation for the datasets with inconsistent dimensionality.

[0005] The front-end server may define a JOIN operation as one or more JOIN commands that conform to a database query language, such as Structured Query Language (SQL). The JOIN operation may be received via a high-level syntax that inherently promotes readability and does not conform to any structured database query language (in terms of syntax and semantics). The front-end server may perform this translation of the high-level JOIN operation into one or more JOIN commands that conform to a formal database query language on behalf of a user who may not be familiar with database query languages. The front-end server may then transmit these JOIN commands, some of which may identify JOINs to datasets with inconsistent dimensionality to facilitate review of aggregated data across two or more datasets.

[0006] In this regard, rather than returning an error when attempting to perform a JOIN on dimensionally inconsistent datasets, the database management system may form an implicit JOIN command configured to refrain from actively specifying specific compatible columns between the dimensionally inconsistent datasets. Although the results of such a JOIN command on dimensionally inconsistent datasets may return inaccurate aggregated data (e.g., as a result of broadcasting numbers across inconsistent columns that may create irrelevant duplicate data), various aspects of the aggregated data may be useful in understanding trends and other aspects related to the aggregated data. Thus, the results of such a JOIN operation may facilitate further interaction with the aggregated data to reveal trends in such aggregated data, facilitate data analysis, and otherwise deepen understanding.

[0007] Considering that such JOIN operations on dimensionally inconsistent datasets are possible only with manual intervention by back-end data analytics specialists creating custom JOIN transformations that are specific to a given use case, various aspects of the technology may automate JOIN transformations on dimensionally inconsistent datasets. Rather than generating errors when such JOIN operations on dimensionally inconsistent datasets are attempted to be transformed, potentially resulting in a user having to enter JOIN operations to repeatedly attempt to validate JOIN operations on dimensionally inconsistent datasets, various aspects of the technology may improve the operation of the database management system itself in terms of resource consumption (e.g., processor cycles, memory, memory bandwidth, etc., and associated power) as a result of avoiding unnecessary error generation. Through this automation, various aspects of the technology described in this disclosure may facilitate more seamless integration of JOIN operations that do not require extensive manual intervention and code fragmentation, thereby improving the development of the database management system itself.

[0008] In one example, various aspects of the present technology are directed to a database system comprising an interface configured to receive a query for accessing at least a portion of a first dataset and at least a portion of a second dataset, the query identifying two or more dimensions of the first dataset and the second dataset, the first dataset including at least a first dimension of the two or more dimensions and the second dataset including a second dimension of the two or more dimensions and excluding the first dimension of the two or more dimensions; and one or more processors configured to convert the query into one or more structured queries, each conforming to a database query language, the structured queries of the one or more structured queries defining joins between the first dataset and the second dataset across the second dimension of the second dataset, the one or more processors configured to send the structured query to a database storing the first dataset and the second dataset; and in response to the structured query, receive an indication that a virtual table has been created that combines the first dataset and the second dataset across the second dimension of the second dataset.

[0009] In another example, various aspects of the present technology are directed to a method, the method including receiving a query to access at least a portion of a first dataset and at least a portion of a second dataset, the query identifying two or more dimensions of the first dataset and the second dataset, the first dataset including at least a first dimension of the two or more dimensions and the second dataset including a second dimension of the two or more dimensions, excluding the first dimension of the two or more dimensions, converting the query into one or more structured queries, each conforming to a database query language, the structured queries of the one or more structured queries defining joins between the first dataset and the second dataset across the second dimension of the second dataset, sending the structured query to a database storing the first dataset and the second dataset, and receiving an indication in response to the structured query that a virtual table has been created that combines the first dataset and the second dataset across the second dimension of the second dataset.

[0010] In another example, various aspects of the present technology are directed to a non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to receive a query to access at least a portion of a first dataset and at least a portion of a second dataset, the query identifying two or more dimensions of the first dataset and the second dataset, the first dataset including at least a first dimension of the two or more dimensions and the second dataset including a second dimension of the two or more dimensions, and excluding the first dimension of the two or more dimensions; converting the query into one or more structured queries, each conforming to a database query language, the structured queries of the one or more structured queries defining joins between the first dataset and the second dataset across the second dimension of the second dataset; sending the structured query to a database storing the first dataset and the second dataset; and receiving an indication that a virtual table has been created that combines the first dataset and the second dataset across the second dimension of the second dataset in response to the structured query.

[0011] In another example, various aspects of the present technology are directed to an apparatus, the apparatus including means for receiving a query to access at least a portion of a first dataset and at least a portion of a second dataset, the query identifying two or more dimensions of the first dataset and the second dataset, the first dataset including at least a first dimension of the two or more dimensions and the second dataset including a second dimension of the two or more dimensions, excluding the first dimension of the two or more dimensions, means for converting the query into one or more structured queries, each conforming to a database query language, the structured queries of the one or more structured queries defining joins between the first dataset and the second dataset across the second dimension of the second dataset, means for transmitting the structured query to a database storing the first dataset and the second dataset, and means for receiving an indication that a virtual table has been created combining the first dataset and the second dataset across the second dimension of the second dataset in response to the structured query.

[0012] The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 illustrates an example database management system configured to perform various aspects of the implicit JOIN technique described in this disclosure. [Diagram 2] FIG. 2 illustrates an example JOIN operation performed between datasets having inconsistent dimensions, in accordance with various aspects of the techniques described in this disclosure. [Diagram 3] FIG. 2 illustrates an example JOIN operation performed between datasets having inconsistent dimensions, in accordance with various aspects of the techniques described in this disclosure. [Figure 4]FIG. 2 illustrates an example JOIN operation performed between datasets having inconsistent dimensions, in accordance with various aspects of the techniques described in this disclosure. [Diagram 5] 2 is a flowchart illustrating an exemplary operation of the database management system of FIG. 1 in performing various aspects of the implicit JOIN technique described in this disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Data aggregation and data analysis increasingly enable detailed understanding of trends, user actions, and the like, thereby enabling more informed decision making. Data aggregation is performed via database management systems, which may define database schemas for how various data sets (such as tables, including multidimensional tables called hypercubes or online analytical processing (OLAP) cubes) relate to each other within the overall database. Data is increasingly being aggregated and stored in databases in near real-time to uncover trends that may drive such decisions, which may facilitate on-demand data analysis.

[0015] A database management system (sometimes referred to as a "database system") may provide a front-end server that facilitates interaction with clients so that the clients may engage in data analysis on the aggregated data stored in the database. For example, an application developer (sometimes referred to as an "app developer") may review data related to a distributed application ("app") to understand net sales per country, state, or other jurisdiction, revenue per currency, number of active users, etc. The app developer may not be familiar with formal database query languages ​​such as Structured Query Language (SQL), and the front-end server may expose a high-level syntax for performing data analysis that eliminates some of the formality associated with formal database query languages.

[0016] App developers or other clients may interface with a front-end server to define various queries according to a high-level syntax. The front-end server may then translate (or in other words compile) the query into one or more queries conforming to a formal database query language. A formal database language, such as Structured Query Language (SQL), SQLite, PostgreSQL, Oracle SQL, etc., defines the syntax and semantics associated with defining, editing, and otherwise managing datasets stored in a database. As an example, SQL may represent a domain-specific language used in programming and designed to manage datasets held in a relational database management system (RDBMS). There are a number of commands supported by the syntax, which may invoke functions to perform various operations on the datasets. One such command, called a JOIN command, may be used to form a clause that joins rows from two or more datasets based on related columns between them.

[0017] According to the syntax of this formal database language, a JOIN command may require "consistent" dimensionality in that multiple datasets referenced by the JOIN command must all contain a common column. If any one of the datasets referenced in the JOIN command does not contain the base column, the JOIN command will fail because such a JOIN command may result in errors that the strict syntax of the formal database language enforces, sometimes referred to as "inconsistent dimensionality" between the datasets. Although inconsistent dimensionality between datasets may be necessary to avoid such errors in dataset management, there may be some instances where such a JOIN command between inconsistent datasets may result in useful aggregate data.

[0018] According to various aspects of the techniques described in this disclosure, a database management system may perform a JOIN operation on multiple datasets having inconsistent dimensionality. A first dataset of the multiple datasets may share one or more columns in common with a second dataset of the multiple datasets, but the JOIN operation may specify certain columns that are not shared, or in other words, not common, between the first dataset and the second dataset. To overcome the inconsistent dimensionality between the first dataset and the second dataset, the front-end server may convert the JOIN operation into a formal structured database JOIN based on metadata indicating compatibility between columns of the second dataset that are only present in the first dataset. The front-end server may then ignore the compatible dimensions specified by the metadata for the second dataset and form a JOIN command for the common (or in other words, consistent) columns between the first dataset and the second dataset. In this regard, the front-end server may form an implicit JOIN operation for datasets with inconsistent dimensionality.

[0019] The front-end server may define a JOIN operation as one or more JOIN commands that conform to a database query language, such as Structured Query Language (SQL). The JOIN operation may be received via a high-level syntax that inherently promotes readability and does not conform to any structured database query language (in terms of syntax and semantics). The front-end server may perform this translation of the high-level JOIN operation into one or more JOIN commands that conform to a formal database query language on behalf of a user who may not be familiar with database query languages. The front-end server may then transmit these JOIN commands, some of which may identify JOINs to datasets with inconsistent dimensionality to facilitate review of aggregated data across two or more datasets.

[0020] In this regard, rather than returning an error when attempting to perform a JOIN on dimensionally inconsistent datasets, the database management system may form an implicit JOIN command configured to refrain from actively specifying specific compatible columns between the dimensionally inconsistent datasets. Although the results of such a JOIN command on dimensionally inconsistent datasets may return inaccurate aggregated data (e.g., as a result of broadcasting numbers across inconsistent columns that may create irrelevant duplicate data), various aspects of the aggregated data may be useful in understanding trends and other aspects related to the aggregated data. Thus, the results of such a JOIN operation may facilitate further interaction with the aggregated data to reveal trends in such aggregated data, facilitate data analysis, and otherwise deepen understanding.

[0021] Considering that such JOIN operations on dimensionally inconsistent datasets are possible only with manual intervention by back-end data analysis specialists creating custom JOIN transformations that are specific to a given use case, various aspects of the present technology may automate JOIN transformations on dimensionally inconsistent datasets. Rather than generating errors when such JOIN operations on dimensionally inconsistent datasets are attempted to be transformed, potentially resulting in a user having to enter JOIN operations to repeatedly attempt to validate JOIN operations on dimensionally inconsistent datasets, various aspects of the present technology may improve the operation of the database management system itself in terms of resource consumption (e.g., processor cycles, memory, memory bandwidth, etc., and associated power) as a result of avoiding unnecessary error generation. Through this automation, various aspects of the technology described in this disclosure may facilitate a more seamless integration of JOIN operations that do not require extensive manual intervention and code fragmentation, thereby improving the development of the database management system itself.

[0022] 1 illustrates an example database management system 100 configured to perform various aspects of the implicit JOIN technique described in this disclosure. As shown in the example of FIG. 1, database management system 100 (which may also be referred to as “database system 100”) may include a front-end server 102, a database server 104, a distributed storage system 106, and a client computing device 108.

[0023] The front-end server 102 may represent a server configured to host an interface with which the client computing devices 108 can interact to define one or more queries and ultimately view data (or data derived from data) stored as datasets in the distributed storage system 106 under management by the database server 104. The front-end server 102 may include one or more processors 110 and a storage system 112.

[0024] One or more processors 110 may implement functions and / or execute instructions associated with the front-end server 102. Examples of processors 110 include application processors, microcontrollers, central processing units (CPUs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), graphics processing units (GPUs), display controllers, auxiliary processors, one or more sensor hubs, and any other hardware configured to function as a processor, processing unit, or processing device. Modules 114-116 may be operable (or, in other words, may be executed) by processor 110 to perform various actions, operations, or functions of the front-end server 102. That is, modules 114-116 may form executable bytecodes that, when executed, cause processor 110 to perform specific operations (and thereby cause front-end server 102 to become a specific purpose computer performing specific operations) in accordance with various aspects of the techniques described herein. For example, the processor 110 of the front-end server 102 may retrieve and execute instructions stored by the storage system 112 that cause the processor 110 to perform the operations described herein that are caused by the modules 114-116. These instructions, when executed by the processor 110, may cause the front-end server 102 to store information in the storage system 112.

[0025] The storage system 112 in the front-end server 102 may store information for processing during operation of the front-end server 102 (e.g., the front-end server 102 may store data accessed by the modules 114-116 during execution on the front-end server 102). In some examples, the storage system 112 is a temporary storage device, meaning that the primary purpose of the storage system 112 is not long-term storage. The storage system 112 on the front-end server 102 is configured to store information short-term as a volatile memory, such that the stored content may not be retained when power is removed. Examples of volatile memory include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), and other forms of volatile memory known in the art.

[0026] The storage system 112 also includes one or more computer readable storage media in some examples. The storage system 112 may include one or more non-transitory computer readable storage media. The storage system 112 may be configured to store a larger amount of information than can generally be stored by volatile memory. The storage system 112 may also be configured to store information long term as a non-volatile memory space, where information may be retained even after a power on / off cycle. Examples of non-volatile memory include magnetic hard disks, optical disks, flash memory, or forms of electrically programmable memory (EPROM) and electrically erasable and programmable memory (EEPROM). The storage system 112 may store program instructions and / or information (e.g., data) associated with the modules 114-116. The storage system 112 may include memory configured to store data or other information associated with the modules 114-116.

[0027] The database server 104 may represent a server that manages the data sets 107A-107N ("data sets 107") stored in the distributed storage system 106 according to a database schema 105 ("DB schema 105"). The database server 104 specifies the DB schema 105 using a formal language to define relationships between objects (referred to as data sets 107 in this example) stored in the underlying database. The data sets 107 may represent multidimensional tables having multiple rows and columns. In some cases, the multidimensional table may include N dimensions (where N is 3 or more), in which case the multidimensional table may be referred to as an online analytical processing (OLAP) cube ("OLAP cube"), or a "hypercube" (when N is greater than 3).

[0028] The distributed storage system 106 may represent a highly scalable system for dynamic distribution and redistribution of storage across a wide variety of databases and workflows. The distributed storage system 106 may include any number of servers, potentially arranged as a cluster, which may be geographically distributed (e.g., across two or more data centers). The distributed storage system 106 may provide a cluster of storage units (which may be the same, if not substantially the same, as the storage system 112 described above with respect to the front-end server 102), along with various mechanisms for data synchronization and data federation between the cluster nodes.

[0029] Although database management system 100 is described above as including a dedicated device, it may include a single database that hosts data set 107 and presents a front-end interface similar to that described herein with respect to front-end server 102. In this regard, database management system 100 may include one or more servers that implement the functionality of each of front-end server 102, database server 104, and distributed storage system 106. Accordingly, various aspects of the techniques described in this disclosure should not be limited to the example architecture shown in FIG.

[0030] 1, only the front-end server 102 is shown as including a processor 110 and a storage system 112, it should be understood that the database server 104, the distributed storage system 106, and the client computing devices 108 may each include one or more processors similar to the processor 110 and a storage system similar to the storage system 112. Additionally, each of the front-end server 102, the database server 104, the distributed storage system 106, and the client computing devices 108 may include additional hardware not shown in the example of FIG. 1, such as a display device, a network interface card (NIC), interface components (e.g., keyboard, mouse, etc.), as well as software, including an operating system (OS) that presents an application space separate from the OS in which various applications or other modules can run, database management software (e.g., used to define and visualize DB schema 105), etc.

[0031] As described above, the database management system 100 may provide a front-end server 102 to facilitate interaction with clients (e.g., by client computing devices 108) so that the clients may engage in data analysis with respect to the aggregated data stored in the databases in the distributed storage system 106. For example, an application developer (sometimes referred to as an “app developer”) may review data related to a distributed application (an “app”) to understand net sales per country, state, or other jurisdiction, revenue per currency, number of active users, etc. The app developer may not be familiar with formal database query languages ​​such as Structured Query Language (SQL), and the front-end server 102 may expose a high-level syntax for performing data analysis that eliminates some of the formality associated with formal database query languages.

[0032] 1, the processor 110 of the front-end server 102 may be configured to execute a user interface (UI) module 114 that presents an interface accessible by the client computing device 108. The UI module 114 may represent a web-based interface that allows a client, via the client computing device 108, to enter a query 115 conforming to a high-level syntax for performing data analysis. Although the UI module 114 is described as representing a web-based interface, it may present other types of interfaces, including a command line interface (CLI), a graphical user interface ("GUI"), or any other type of interface that facilitates definition of the query 115.

[0033] An app developer or other client may interface with a UI module 114 of the front-end server 102 to define various queries, such as query 115, according to a high-level syntax. In response to receiving query 115, the processor 110 may invoke a query compiler (QC) module 116, which may convert (or in other words compile) the query into one or more structured queries, such as structured query 117 ("structure query 117") that conform to a formal database query language. A formal database language, such as Structured Query Language (SQL), SQLite, PostgreSQL, Oracle SQL, etc., defines syntax and semantics associated with defining, editing, and otherwise managing datasets 107 stored in a database. As an example, SQL may represent a domain-specific language used in programming and designed to manage datasets held in a relational database management system (RDBMS). There are a number of commands supported by the syntax, which may invoke functions to perform various operations on datasets 107. One such command, called a JOIN command, may be used to form a clause that combines rows from two or more data sets based on related columns between them.

[0034] According to the syntax of this formal database language, a JOIN command may require "consistent" dimensionality in that multiple datasets 107 referenced by the JOIN command must all contain a common column. If any one of the datasets 107 referenced in the JOIN command does not contain the base column, the JOIN command will fail because such a JOIN command may result in errors that the strict syntax of the formal database language enforces, sometimes referred to as "inconsistent dimensionality" between the datasets 107. Although inconsistent dimensionality between datasets 107 may be necessary to avoid such errors in dataset management, there may be some instances where such a JOIN command between inconsistent datasets 107 may result in useful aggregate data.

[0035] For example, there are some metrics that are difficult to formulate for one reason or another due to the underlying dimensionality of the datasets 107. Consider a metric on buyers per active user, where "buyers" is specified as dataset 107A, where buyers are specified by tracking period, while "number of active users" is specified as another dataset 107N, which is the same for every tracking period. Thus, the "active users" dataset 107N does not define tracking period as a column (because active users do not change over tracking period), and thus lacks common dimensionality with the "buyers" dataset 107A. In compiling a request to compile a query 115 that defines a request for buyers per active user, the QC module 116 will be faced with a JOIN command between dataset 107A and dataset 107N, which selects dimensions that are inconsistent because dataset 107N lacks a "tracking period" column. Additionally, the number of buyers per active user can be a useful metric for identifying trends related to product management, app development, etc.

[0036] According to various aspects of the techniques described in this disclosure, the database management system 100 may perform a JOIN operation on multiple datasets 107 having inconsistent dimensionality. A first dataset 107A of the multiple datasets 107 may share one or more columns in common with a second dataset 107N of the multiple datasets 107, but the JOIN operation may specify certain columns that are not shared, or in other words, not common, between datasets 107A and 107N (by way of example). To overcome the inconsistent dimensionality between datasets 107A and 107N, the QC module 116 may convert the JOIN operation into a formal structured database JOIN based on metadata 119 indicating compatibility between columns of dataset 107N that are only present in dataset 107A. The QC module 116 may then refrain from specifying (or, in other words, ignore) the compatible dimensions identified by the metadata 119 for the dataset 107N and form a JOIN command for the common (or, in other words, consistent) columns between the datasets 107A and 107N. At this point, the front-end server 102 may form an implicit JOIN operation for the dimensionally inconsistent datasets 107A / 107N.

[0037] The QC module 116 may define a JOIN operation as one or more JOIN commands conforming to a database query language, such as Structured Query Language (SQL). The JOIN operation may be received as a query 115 via a high-level syntax that inherently promotes readability and does not conform to any structured database query language (in terms of syntax and semantics). The QC module 116 may perform this translation of the high-level JOIN operation into one or more JOIN commands conforming to a formal database query language on behalf of a user who may not be familiar with database query languages. The QC module 116 may generate these JOIN commands as structured queries 117. The front-end server 102 may then submit these JOIN commands as structured queries 117, some of which may identify JOINs to datasets with inconsistent dimensions to facilitate review of aggregated data across two or more datasets 107A / 107N.

[0038] In operation, the QC module 116 may present an interface (possibly in combination with the UI module 114) configured to receive a query 115 for accessing at least a portion of the dataset 107A and at least a portion of the dataset 107N. The query 115 may identify (e.g., as a selection) two or more dimensions of the datasets 107A and 107N. In the above example, the dataset 107A includes a first dimension ("tracking period") of the two or more dimensions identified by the query 115, and the dataset 107N includes a second dimension ("active users") of the two or more dimensions identified by the query 115, and excludes the first dimension of the two or more dimensions identified by the query 115.

[0039] The QC module 114 may convert the query 115 into a structured query 117 that conforms to a database query language such as SQL. The structured query 117 may define a JOIN between the datasets 107A and 107B across a second dimension of the two or more dimensions identified by the query 115 (and the second dimension is not the inconsistent first dimension of the two or more dimensions). The metadata 119 may identify the first dimension of the dataset 107A as compatible with the dataset 107N, which may permit the QC module 114 to formulate a JOIN command as the structured query 116 while forgoing specifying the first dimension as part of the structured query 117. The QC module 116 may output or otherwise transmit the structured query 117 to the database server 104, which may access an underlying database that stores the datasets 107A and 107B hosted by the distributed storage system 106.

[0040] The result of structured query 117 may be what is called a "view" in SQL, which may correspond to a so-called virtual table that is specified as an operation (e.g., a JOIN command) and is computed in near real-time (to reduce storage consumption) each time the virtual table is accessed. Views may be resolved and stored separately to improve access speed, but may come at the cost of additional storage consumption. In any case, database server 104 may execute the JOIN command specified by structured query 117 and return an indication that a virtual table has been created that combines datasets 107A and 107N across the second dimension of dataset 107N.

[0041] In this regard, rather than returning an error when attempting to perform a JOIN on dimensionally inconsistent datasets 107, the database management system 100 may form an implicit JOIN command configured to refrain from actively specifying specific compatible columns between the dimensionally inconsistent datasets 107. Although the results of such a JOIN command on dimensionally inconsistent datasets 107 may return inaccurate aggregated data (e.g., as a result of broadcasting numbers across inconsistent columns that may create irrelevant duplicate data), various aspects of the aggregated data may be useful in understanding trends and other aspects related to the aggregated data. Thus, the results of such a JOIN operation may facilitate further interaction with the aggregated data to reveal trends in such aggregated data, facilitate data analysis, and otherwise deepen understanding.

[0042] Considering that such JOIN operations on dimensionally inconsistent datasets 107 are sometimes only possible with manual intervention by back-end data analytics specialists creating custom JOIN transformations that are specific to a given use case, various aspects of the present technology may automate JOIN transformations on dimensionally inconsistent datasets 107. Rather than generating errors when such JOIN operations on dimensionally inconsistent datasets 107 are attempted to be transformed, potentially resulting in a user entering JOIN operations to repeatedly attempt to validate the JOIN operations on dimensionally inconsistent datasets 107, various aspects of the present technology may improve the operation of the database management system 100 itself in terms of resource consumption (e.g., processor cycles, memory, memory bandwidth, etc., and associated power) as a result of avoiding unnecessary error generation.

[0043] Through this automation, various aspects of the techniques described in this disclosure may facilitate more seamless integration of JOIN operations that may not require extensive manual intervention and code fragmentation, thereby improving the development of the database management system 100 itself. That is, to enable JOIN operations on datasets 107 with inconsistent dimensionality, a data analytics specialist would manually implement the JOIN operations and link the JOIN operations to a client, separating the automatically compiled JOIN operations from the manually defined JOIN operations, thereby fragmenting the code base. In this regard, automating the generation of JOIN operations on datasets with inconsistent dimensionality resolves code fragmentation and allows for better integration of query compilation, thereby improving the development of the database management system 100.

[0044] 2-4 are diagrams illustrating example JOIN operations performed between datasets having inconsistent dimensions, according to various aspects of the techniques described in this disclosure. Referring first to the example of FIG. 2, dataset 207A is joined with dataset 207N, both of which may represent the example dataset 107 shown in the example of FIG. 1. A data analysis specialist and / or client may define dataset 207A and dataset 207N as follows: Revenue:[app,currency,revenue] Users: [app, user_count] compatible dimensions: [currency] In this example, the dataset 207A is named "Rvenue" and has three dimensions denoted as "app", "currency", and "revenue", each of which represents a separate column in the dataset 207A. The dataset 207N is named "Users" in this example and has two dimensions denoted as "app" and "user_count", each of which represents a separate column in the dataset 207N. The QC module 116 may keep this information as part of the metadata 119 (the above definitions may also be defined in the DB schema 105). The metadata 119 stored locally on the front-end server 102 and maintained by the QC module 116 separate from the DB schema 105 may further indicate a compatible dimension denoted as "currency" that references the "currency" dimension (or in other words, a column) of the "Rvenue" dataset 207A.

[0045] The UI module 114 can receive a query 115 from the client computing device 108, where the query 115 describes: SELECT app, currency, SUM(revenue) / SUM(user_count) FROM virtual_table GROUP BY 1, 2; In this example, query 115 references (using a "SELECT" command) a common "app" dimension from both dataset 207A and dataset 207N, and an inconsistent "currency" dimension that is included in dataset 207A but excluded from dataset 207N. In response to receiving query 115, UI module 114 may invoke QC module 116.

[0046] QC module 116 can transform query 115 based on metadata 119 to obtain structured query 117, which can define a JOIN between dataset 207A and dataset 207N across the “app” dimension, and exclude the “currency” dimension from the JOIN described in the following structured query 117 if metadata 119 indicates that “currency” is a compatible dimension. SELECT app, currency, SUM(revenue) / SUM(user_count) FROM revenue JOIN users USING(app) GROUP BY 1, 2; In other words, metadata 119 may identify dataset 207N as having the “currency” dimension of dataset 207A as a compatible dimension with dataset 207N. QC module 116 may analyze the compatible dimensions identified by metadata 119 and drop the specification of the compatible dimension “currency” of dataset 207A in the JOIN defined by structured query 117.

[0047] As noted above, generally, a "SELECT" of inconsistent dimensions across two datasets would result in an error. The QC module 116 is instead configured to "ignore," or in other words, refrain from specifying, compatible dimensions in a JOIN operation between datasets 207A / 207N. This results in virtual table 209 having the "app," "currency," and "revenue per user" dimensions as columns in virtual table 209. The ordinals 1 and 2 following the "GROUP BY" command indicate that virtual table 209 groups the aggregate results first by "app" and then by "currency."

[0048] The "user_count" dimension is broadcast across the ignored "currency" dimension in virtual table 209, which effectively attributes a "user_count" of 5 to both "currency" types (e.g., USD and GBP) for "app" of type "UT." Thus, the user_count of "5" is duplicated in rows 2 and 3 of virtual table 209, resulting in a user total of 30 when dataset 207N shows only a total of 25 users. To avoid this type of inconsistency, database server 104 returns an error, but virtual table 209 still provides aggregate data that can identify trends and be useful for decision making.

[0049] To avoid over-extending this broadcasting concept to the point where it may be difficult to follow any inconsistencies introduced, the QC module 116 broadcasts only dimensions on "compatible" dimensions specified or otherwise identified by the metadata 119. If the query 115 specifies a fourth dimension in the dataset 207A that is not indicated (by the metadata 119) to be a compatible dimension in the dataset 207N, the QC module 116 may generate and return an error indicating that it cannot perform the transformation of the query 115. In this way, the QC module 116 may limit the transformation of the query 115 to only those instances where the compatible dimensions are clearly defined in the metadata 119.

[0050] 3, dataset 307A is combined with dataset 307N, both of which may represent the example dataset 107 shown in the example of Figure 1. A data analysis specialist and / or a client may define dataset 307A and dataset 307N as follows: Users:[app,country,user_count] Ecosystem_users:[country, ecosystem_user_count] compatible dimensions: [app] In this example, the dataset 307A is named "Users" and has three dimensions denoted as "app", "country", and "user_count", each of which represents a separate column in the dataset 307A. The dataset 307N is named "Ecosystem_users" in this example and has two dimensions denoted as "country" and "ecosystem_user_count", each of which represents a separate column in the dataset 307N. The QC module 116 may keep this information as part of the metadata 119 (the above definitions may also be defined in the DB schema 105). The metadata 119 stored locally on the front-end server 102 and maintained by the QC module 116 separate from the DB schema 105 may further indicate a compatible dimension denoted as "app" that references the "app" dimension (or in other words, a column) of the "Users" dataset 307A.

[0051] The UI module 114 can receive a query 115 from the client computing device 108, where the query 115 describes: SELECT country, app, SUM(user_count) / SUM(ecosystem_user_count) FROM virtual_table GROUP BY 1, 2; In this example, query 115 references (using a "SELECT" command) a common "country" dimension from both dataset 307A and dataset 307N, and an inconsistent "app" dimension that is included in dataset 307A but excluded from dataset 307N. In response to receiving query 115, UI module 114 may invoke QC module 116.

[0052] QC module 116 can transform query 115 based on metadata 119 to obtain structured query 117, which can define a JOIN between dataset 307A and dataset 307N over the “country” dimension, and exclude the “app” dimension from the JOIN described in the following structured query 117 if metadata 119 indicates that “app” is a compatible dimension. SELECT country, app, SUM(user_count) / SUM(ecosystem_user_count) FROM users JOIN ecosystem_users USING(country) GROUP BY 1, 2; In other words, metadata 119 may identify dataset 307N as having the “app” dimension of dataset 307A as a compatible dimension with dataset 307N. QC module 116 may analyze the compatible dimensions identified by metadata 119 and drop the specification of the compatible dimension “app” of dataset 307A in the JOIN defined by structured query 117.

[0053] As noted above, in general, a "SELECT" of inconsistent dimensions across two datasets would result in an error. The QC module 116 is instead configured to "ignore," or in other words, refrain from specifying, compatible dimensions in a JOIN operation between datasets 307A / 307N. This results in virtual table 309 having the "app," "country," and "percentage of users" dimensions as columns in virtual table 309. The ordinals 1 and 2 following the "GROUP BY" command indicate that virtual table 309 groups the aggregate results first by "country" and then by "app."

[0054] The "user_count" dimension is broadcast across the ignored "app" dimension in virtual table 309, which effectively attributes an "ecosystem_user_count" of 208 to both "app" types (e.g., UT and Drive) for "country" of type "US". Thus, the ecosystem_user_count of "208" is duplicated in rows 2 and 4 of virtual table 309, resulting in an additional user total of 208 when dataset 307N shows only a total of 213 users. To avoid this type of inconsistency, database server 104 returns an error, but virtual table 309 still provides aggregate data that can identify trends and be useful for decision making.

[0055] Again, to avoid over-extending this broadcasting concept to the point where it may be difficult to follow any inconsistencies introduced, the QC module 116 broadcasts only dimensions on "compatible" dimensions specified or otherwise identified by the metadata 119. If the query 115 specifies a fourth dimension in the dataset 307A that is not indicated (by the metadata 119) to be a compatible dimension in the dataset 307N, the QC module 116 may generate and return an error indicating that it cannot perform the transformation of the query 115. In this way, the QC module 116 may limit the transformation of the query 115 to only those instances where a compatible dimension is clearly defined in the metadata 119.

[0056] In the example of Figure 4, dataset 407A is combined with dataset 407N, both of which may represent an example of dataset 107 shown in the example of Figure 1. A data analysis specialist and / or a client may define dataset 407A and dataset 407N as follows: Users:[app, is_testing_user, user_count]compatible dimensions: [is_testing_device] Device_crashes:[app, is_testing_device, crash_count] compatible dimensions: [is_testing_user] In this example, the dataset 407A is named "Users" and has three dimensions denoted as "app", "is_testing_user", and "user_count", each of which represents a separate column in the dataset 407A. The dataset 407N is named "Device_crashes" in this example and has three dimensions denoted as "app", "is_testing_device", and "crash_count", each of which represents a separate column in the dataset 407N. The QC module 116 may keep this information as part of the metadata 119 (the above definition may also be defined in the DB schema 105). The metadata 119 stored locally on the front-end server 102 and maintained by the QC module 116 separate from the DB schema 105 may further indicate a compatible dimension denoted as "is_testing_device", which references the "is_testing_device" dimension (or in other words, a column) of the "Device_crashes" dataset 407N. Metadata 119 also identifies a compatible dimension of the "Users" dataset 407A for dataset 407N as the "is_testing_user" dimension of dataset 407A.

[0057] The UI module 114 can receive a query 115 from the client computing device 108, where the query 115 describes: SELECT app, SUM(crash_count) / SUM(user_count) FROM virtual_table WHERE NOT is_testing_user and NOT is_testing_device GROUP BY 1; In this example, query 115 references the common "app" dimension from both dataset 407A and dataset 407N (using the "SELECT" command), and references the inconsistent "is_testing_user" dimension (for the "is_testing_user" dimension) that is included in dataset 407A but excluded from dataset 407N, and the inconsistent "is_testing_device" (for the "is_testing_device" dimension) that is included in dataset 407N but excluded from dataset 407A (using the "WHERE NOT" and "NOT" commands). In response to receiving query 115, UI module 114 may invoke QC module 116.

[0058] QC module 116 can transform query 115 based on metadata 119 to obtain structured query 117, which can define a JOIN between dataset 407A and dataset 407N across the “app” dimension, and exclude the “is_testing_user” and “is_testing_device” dimensions from the hierarchical JOIN described in structured query 117 below if metadata 119 indicates that “is_testing_user” and “is_testing_device” are compatible dimensions. SELECT app, SUM(crash_count) / SUM(user_count) FROM ( SELECT app, user_count FROM users WHERE NOT is_testing_user )JOIN ( SELECT app, device_count FROM device_crashes WHERE NOT is_testing_device ) USING(app) GROUP BY 1; The foregoing effectively implements an implicit JOIN operation across inconsistent datasets 407A and 407N that facilitates filtering of various rows of data (rows from dataset 407A where is_testing_user is set to TRUE and rows from dataset 407N where is_testing_device is set to TRUE) and provides a virtual table 409 where the common "app" dimension calculates the dimension "non-testing crashes per user" and excludes such rows from the totals "user_count" and "crash_count", respectively.

[0059] 5 is a flow chart illustrating an example operation of the database management system of FIG. 1 in performing various aspects of the implicit join technique described in this disclosure. As described above, the QC module 114 of the front-end server 102 may configure (500) an interface (possibly in combination with the UI module 114) configured to receive a query 115 for accessing at least a portion of the dataset 107A and at least a portion of the dataset 107N. The query 115 may identify (e.g., as a selection) two or more dimensions of the datasets 107A and 107N. In the above example, the dataset 107A includes a first dimension ("tracking period") of the two or more dimensions identified by the query 115, and the dataset 107N includes a second dimension ("active users") of the two or more dimensions identified by the query 115, and excludes the first dimension of the two or more dimensions identified by the query 115.

[0060] QC module 114 may convert query 115 into a structured query 117 that conforms to a database query language, such as SQL (502). Structured query 117 may define a JOIN between datasets 107A and 107B across a second dimension of the two or more dimensions identified by query 115 (and the second dimension is not the inconsistent first dimension of the two or more dimensions). Metadata 119 may identify the first dimension of dataset 107A as compatible with dataset 107N, which may permit QC module 114 to formulate a JOIN command as structured query 117 while forgoing specifying the first dimension as part of the JOIN command that is at least partially represented by structured query 117. The QC module 117 may output or otherwise transmit (504) the structured query 117 to the database server 104, which may access the underlying database that stores the datasets 107A and 107B hosted by the distributed storage system 106.

[0061] The result of the structured query 117 may be what is called a "view" in SQL, which may correspond to a so-called virtual table that is specified as an operation (e.g., a JOIN command) and is calculated in near real-time (to reduce storage consumption) each time the virtual table is accessed. The view may be resolved and stored separately to improve access speed, but at the cost of additional storage consumption. In any case, the database server 104 may execute the JOIN command specified by the structured query 117 and return an indication that a virtual table has been created that combines the datasets 107A and 107N across the second dimension of the datasets 107N. Thus, the QC module 116 may receive an indication that a virtual table has been created that combines the datasets 107A and 107N across the second dimension of the datasets 107N (506).

[0062] In this manner, the QC module 116 functions as a software shim that adapts transformations of the query 115 based on the metadata 119 to form a structured query 117 that includes what may be referred to as an implicit JOIN operation. The JOIN operations are implicit because the QC module 116 infers which dimensions to "ignore" (or in other words, refrain from including in the JOIN operation) based on the compatible dimensions described in the metadata 119. This allows the QC module 116 to prevent inappropriate JOIN operations that would otherwise result in a JOIN between two datasets with incompatible dimensions, and adapt transformations (or in other words compilations) that implicitly form JOIN operations along compatible dimensions that result in data broadcasting (which may introduce inconsistencies), while also resulting in aggregated data that may be useful in spotting trends or otherwise form the basis of decision making.

[0063] In this manner, various aspects of the present technology may enable the following examples. Example 1. A database system comprising: an interface configured to receive a query for accessing at least a portion of a first dataset and at least a portion of a second dataset, the query identifying two or more dimensions of the first dataset and the second dataset, the first dataset including at least a first dimension of the two or more dimensions, the second dataset including a second dimension of the two or more dimensions, and excluding the first dimension of the two or more dimensions; and one or more processors configured to: convert the query into one or more structured queries, each conforming to a database query language, the structured queries of the one or more structured queries defining a join between the first dataset and the second dataset across the second dimension of the second dataset; transmit the structured query to a database storing the first dataset and the second dataset; and receive an indication that a virtual table has been created combining the first dataset and the second dataset across the second dimension of the second dataset in response to the structured query.

[0064] Example 2. The database system of Example 1, wherein the first dataset includes a third dimension that is the same as the second dimension of the second dataset, and the structured query of the one or more structured queries defines the join between the first dataset and the second dataset using the third dimension of the first dataset and the second dimension of the second dataset.

[0065] Example 3. A database system described in any combination of Examples 1 and 2, wherein the first dataset includes a third dimension that is the same as the second dimension of the second dataset, and the structured query of the one or more structured queries defines the join between the first dataset and the second dataset using only the third dimension of the first dataset and the second dimension of the second dataset.

[0066] Example 4. A database system described in any combination of Examples 1 to 3, wherein the one or more processors are configured to store metadata that identifies the second dataset as having the first dimension of the first dataset as a dimension compatible with the second dataset, and the one or more processors, when configured to convert the query, are configured to refrain from specifying the structured query that defines the join as including the first dimension of the first dataset based on the metadata that identifies the second dataset as having the first dimension of the first dataset as a dimension compatible with the second dataset.

[0067] Example 5. The database system of example 4, wherein the metadata is stored separately from a dataset schema that defines the second dataset in the database.

[0068] Example 6. The first dataset includes a third dimension of the two or more dimensions identified by the query and the second dataset excludes the third dimension of the two or more dimensions, and the one or more processors are configured to store metadata identifying the second dataset as having the first dimension of the first dataset as a compatible dimension with the second dataset, but not the third dimension as a compatible dimension with the second dataset; The database system of any combination of Examples 1 to 5, wherein the one or more processors, when configured to convert the query, are configured to: determine, based on the metadata, that the third dimension of the first dataset is not the compatible dimension with the second dataset; and generate an error indicating that the query conversion cannot be performed based on determining that the third dimension of the first dataset is not the compatible dimension with the second dataset.

[0069] Example 7. The database system according to any combination of Examples 1 to 6, wherein the database query language includes a structured query language.

[0070] Example 8. A method comprising: receiving a query for accessing at least a portion of a first dataset and at least a portion of a second dataset, the query identifying two or more dimensions of the first dataset and the second dataset, the first dataset including at least a first dimension of the two or more dimensions, the second dataset including a second dimension of the two or more dimensions, and excluding the first dimension of the two or more dimensions; converting the query into one or more structured queries, each conforming to a database query language, the structured queries of the one or more structured queries defining a join between the first dataset and the second dataset across the second dimension of the second dataset; transmitting the structured query to a database storing the first dataset and the second dataset; and receiving an indication that a virtual table has been created combining the first dataset and the second dataset across the second dimension of the second dataset in response to the structured query.

[0071] Example 9. The method of example 8, wherein the first dataset includes a third dimension that is the same as the second dimension of the second dataset, and the structured query of the one or more structured queries defines the join between the first dataset and the second dataset using the third dimension of the first dataset and the second dimension of the second dataset.

[0072] Example 10. A method according to any combination of claims 8 and 9, wherein the first dataset includes a third dimension that is the same as the second dimension of the second dataset, and the structured query of the one or more structured queries defines the join between the first dataset and the second dataset using only the third dimension of the first dataset and the second dimension of the second dataset.

[0073] Example 11. A method according to any combination of claims 8 to 10, further comprising storing metadata that identifies the second dataset as having the first dimension of the first dataset as a dimension compatible with the second dataset, and converting the query comprises refraining from specifying the structured query that defines the join as including the first dimension of the first dataset based on the metadata that identifies the second dataset as having the first dimension of the first dataset as a dimension compatible with the second dataset.

[0074] Example 12. The method of example 11, wherein the metadata is stored separately from a dataset schema that defines the second dataset in the database.

[0075] Example 13. The method according to any combination of Examples 8 to 12, wherein the first dataset includes a third dimension of the two or more dimensions identified by the query and the second dataset excludes the third dimension of the two or more dimensions, and the method further includes storing metadata identifying the second dataset as having the first dimension of the first dataset as a compatible dimension with the second dataset but not having the third dimension as a compatible dimension with the second dataset, and converting the query includes determining based on the metadata that the third dimension of the first dataset is not a compatible dimension with the second dataset, and generating an error indicating that the conversion of the query cannot be performed based on determining that the third dimension of the first dataset is not a compatible dimension with the second dataset.

[0076] Example 14. The method according to any combination of Examples 8 to 13, wherein the database query language comprises a structured query language.

[0077] Example 15. A non-transitory computer readable storage medium having instructions stored thereon that, when executed, cause one or more processors to receive a query to access at least a portion of a first data set and at least a portion of a second data set, the query identifying two or more dimensions of the first data set and the second data set, the first data set including at least a first dimension of the two or more dimensions, the second data set including a second dimension of the two or more dimensions, and excluding the first dimension of the two or more dimensions, the receiving and the query each being written in a database query language. 11. The non-transitory computer-readable storage medium of claim 10, further comprising: converting a join between the first dataset and the second dataset into one or more structured queries that conform to the one or more structured queries, the structured queries defining a join between the first dataset and the second dataset across the second dimension of the second dataset; transmitting the structured query to a database that stores the first dataset and the second dataset; and receiving an indication in response to the structured query that a virtual table has been created that combines the first dataset and the second dataset across the second dimension of the second dataset.

[0078] Example 16. The non-transitory computer-readable storage medium of Example 15, wherein the first dataset includes a third dimension that is the same as the second dimension of the second dataset, and the structured query of the one or more structured queries defines the join between the first dataset and the second dataset using the third dimension of the first dataset and the second dimension of the second dataset.

[0079] Example 17. A non-transitory computer-readable storage medium according to any combination of Examples 15 and 16, wherein the first dataset includes a third dimension that is the same as the second dimension of the second dataset, and the structured query of the one or more structured queries defines the join between the first dataset and the second dataset using the third dimension of the first dataset and the second dimension of the second dataset.

[0080] Example 18. A non-transitory computer-readable storage medium according to any combination of Examples 15 to 17, wherein the instructions, when executed, cause the one or more processors to store metadata that identifies the second dataset as having the first dimension of the first dataset as a dimension compatible with the second dataset, and the instructions, when executed, cause the one or more processors to convert the query, the query including instructions that, when executed, cause the one or more processors to cease specifying the structured query that defines the join as including the first dimension of the first dataset based on the metadata that identifies the second dataset as having the first dimension of the first dataset as a dimension compatible with the second dataset.

[0081] Example 19. The non-transitory computer-readable storage medium of Example 18, wherein the metadata is stored separately from a dataset schema defining the second dataset in the database.

[0082] Example 20. The non-transitory computer-readable storage medium according to any combination of Examples 15 to 19, wherein the first dataset includes a third dimension of the two or more dimensions identified by the query, and the second dataset excludes the third dimension of the two or more dimensions, and the instructions, when executed, further include instructions that cause the one or more processors to store metadata identifying the second dataset as having the first dimension of the first dataset as a compatible dimension with the second dataset, but not the third dimension as a compatible dimension with the second dataset, and the instructions, when executed, cause the one or more processors to: determine, based on the metadata, that the third dimension of the first dataset is not the compatible dimension with the second dataset; and generate an error indicating that the transformation of the query cannot be performed based on determining that the third dimension of the first dataset is not the compatible dimension with the second dataset.

[0083] Example 21. A non-transitory computer-readable storage medium according to any combination of Examples 15 to 20, wherein the database query language includes a structured query language.

[0084] Embodiments of the subject matter and functional operations described herein may be implemented in any suitable electronic device, such as a personal computer, a mobile phone, a smartphone, a smart watch, a smart television, a mobile audio or video player, a games console, or a combination of one or more of these devices.

[0085] A computing device may include various components such as a memory, a processor, a display, and an input / output unit. The input / output unit may include, for example, a transceiver that may communicate with one or more networks to transmit and receive data. The display may be any suitable display for displaying images, including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), or a light emitting diode (LED) display.

[0086] Various implementations of the systems and techniques described herein may be realized in digital electronic circuitry, integrated circuits, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementation in one or more computer programs executable and / or interpretable by a programmable system including at least one programmable processor, which may be special-purpose or general-purpose, coupled to receive data and instructions from the storage system and to transmit data and instructions to the storage system.

[0087] One or more aspects of the present technology may be implemented as one or more computer program products, e.g., one or more modules of computer program instructions encoded on a computer-readable medium for execution by or to control the operation of a data processing device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter affecting a machine-readable propagated signal, or one or more combinations thereof. The term "data processing device" encompasses all apparatus, devices, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, an apparatus may include code that creates an environment for the execution of the computer program, e.g., code constituting a processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof. A propagated signal is an artificially generated signal, e.g., a mechanically generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to an appropriate receiving device.

[0088] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple associated files (e.g., files that store one or more modules, subprograms, or code portions). A computer program can be deployed to be executed on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0089] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor receives instructions and data from a read-only memory or a random access memory or both.

[0090] The elements of a computer may include a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer also includes one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or is operatively coupled to receive data from or transfer data to them, or both. However, a computer may lack such devices. Computer-readable media suitable for storing computer program instructions and data include all types of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic disks such as internal hard disks or removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0091] Although many specific implementation details are described herein, these should not be construed as limitations on the scope of what may be claimed, but rather as descriptions of features that may be specific to a particular embodiment. Certain features described herein in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features of the invention described in the context of a single embodiment may also be provided in multiple embodiments separately or in any suitable subcombination. Furthermore, even if features may be described above as functioning in a particular combination and originally claimed as such, one or more features from the claimed combination may in some cases be deleted from the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0092] Similarly, although operations are shown in a particular order in the figures, this should not be understood as requiring that such operations be performed in the particular order or sequential order shown, or that all of the operations shown be performed, to achieve desirable results. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the program components and systems described may generally be integrated into a single software product or packaged into multiple software products.

[0093] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims may be performed in a different order and still achieve desirable results. As an example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. 1. A database system comprising: an interface configured to receive a query to access at least a portion of a first dataset and at least a portion of a second dataset, the query identifying two or more dimensions of the first dataset and the second dataset, the first dataset including at least a first dimension of the two or more dimensions, the second dataset including a second dimension of the two or more dimensions, and excluding the first dimension of the two or more dimensions; one or more processors, the one or more processors and converting the query into one or more structured queries, each conforming to a database query language, the structured queries of the one or more structured queries defining joins between the first dataset and the second dataset across the second dimension of the second dataset, the one or more processors being configured to: sending the structured query to a database storing the first data set and the second data set; receiving an indication that a virtual table has been created in response to the structured query that combines the first and second datasets across the second dimension of the second dataset; 4. The method of claim 3, Database system.

2. the first data set includes a third dimension that is the same as the second dimension of the second data set; 2. The database system of claim 1, wherein the structured query of the one or more structured queries defines the join between the first dataset and the second dataset using the third dimension of the first dataset and the second dimension of the second dataset.

3. the first data set includes a third dimension that is the same as the second dimension of the second data set; 3. The database system of claim 1 , wherein the structured query of the one or more structured queries defines the join between the first dataset and the second dataset using only the third dimension of the first dataset and the second dimension of the second dataset.

4. the one or more processors are configured to store metadata that identifies the second dataset as having the first dimension of the first dataset as a compatible dimension with the second dataset; 4. The database system of claim 1, wherein the one or more processors, when configured to convert the query, are configured to refrain from specifying the structured query that defines the join as including the first dimension of the first dataset based on the metadata that identifies the second dataset as having the first dimension of the first dataset as a dimension that is compatible with the second dataset.

5. The database system of claim 4 , wherein the metadata is stored separately from a dataset schema that defines the second dataset in the database.

6. the first data set includes a third dimension of the two or more dimensions identified by the query; the second data set excludes the third dimension of the two or more dimensions; the one or more processors are configured to store metadata identifying the second dataset as having the first dimension of the first dataset as a compatible dimension with the second dataset, but not the third dimension of the first dataset as a compatible dimension with the second dataset; When the one or more processors are configured to transform the query, determining based on the metadata that the third dimension of the first data set is not the compatible dimension with the second data set; generating an error indicating that the transformation of the query cannot be performed based on determining that the third dimension of the first dataset is not the compatible dimension with the second dataset; The database system according to any one of claims 1 to 5, configured to perform the following:

7. The database system according to any one of claims 1 to 6, wherein the database query language comprises a structured query language.

8. receiving a query to access at least a portion of a first dataset and at least a portion of a second dataset, the query identifying two or more dimensions of the first dataset and the second dataset, the first dataset including at least a first dimension of the two or more dimensions, the second dataset including a second dimension of the two or more dimensions, and excluding the first dimension of the two or more dimensions; converting the query into one or more structured queries, each conforming to a database query language, the structured queries of the one or more structured queries defining joins between the first dataset and the second dataset across the second dimension of the second dataset; sending the structured query to a database storing the first data set and the second data set; receiving an indication that a virtual table has been created in response to the structured query that combines the first and second datasets across the second dimension of the second dataset; A method comprising:

9. the first data set includes a third dimension that is the same as the second dimension of the second data set; 9. The method of claim 8, wherein the structured query of the one or more structured queries defines the join between the first dataset and the second dataset using the third dimension of the first dataset and the second dimension of the second dataset.

10. the first data set includes a third dimension that is the same as the second dimension of the second data set; 10. The method of any combination of claims 8 and 9, wherein the structured query of the one or more structured queries defines the join between the first dataset and the second dataset using only the third dimension of the first dataset and the second dimension of the second dataset.

11. storing metadata identifying the second dataset as having the first dimension of the first dataset as a compatible dimension with the second dataset; 11. The method of any combination of claims 8 to 10, wherein transforming the query includes refraining from specifying the structured query that defines the join as including the first dimension of the first dataset based on the metadata that identifies the second dataset as having the first dimension of the first dataset as a dimension that is compatible with the second dataset.

12. The method of claim 11 , wherein the metadata is stored separately from a dataset schema that defines the second dataset in the database.

13. the first data set includes a third dimension of the two or more dimensions identified by the query; the second data set excludes the third dimension of the two or more dimensions; The method further includes storing metadata identifying the second dataset as having the first dimension of the first dataset as a compatible dimension with the second dataset, but not the third dimension of the first dataset as a compatible dimension with the second dataset; Transforming the query includes: determining, based on the metadata, that the third dimension of the first data set is not the compatible dimension with the second data set; generating an error indicating that the transformation of the query cannot be performed based on determining that the third dimension of the first dataset is not the compatible dimension with the second dataset; The method according to any combination of claims 8 to 12, comprising:

14. The method of any combination of claims 8 to 13, wherein the database query language comprises a structured query language.

15. A non-transitory computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors to: receiving a query to access at least a portion of a first dataset and at least a portion of a second dataset, the query identifying two or more dimensions of the first dataset and the second dataset, the first dataset including at least a first dimension of the two or more dimensions, the second dataset including a second dimension of the two or more dimensions, and excluding the first dimension of the two or more dimensions; converting the query into one or more structured queries, each conforming to a database query language, the structured queries of the one or more structured queries defining joins between the first dataset and the second dataset across the second dimension of the second dataset; sending the structured query to a database storing the first data set and the second data set; receiving an indication that a virtual table has been created in response to the structured query that combines the first and second datasets across the second dimension of the second dataset; A non-transitory computer-readable storage medium that causes

16. the first data set includes a third dimension that is the same as the second dimension of the second data set; 16. The non-transitory computer-readable storage medium of claim 15, wherein the structured query of the one or more structured queries defines the join between the first dataset and the second dataset using the third dimension of the first dataset and the second dimension of the second dataset.

17. the first data set includes a third dimension that is the same as the second dimension of the second data set; 17. The non-transitory computer-readable storage medium of any combination of claims 15 and 16, wherein the structured query of the one or more structured queries defines the join between the first dataset and the second dataset using only the third dimension of the first dataset and the second dimension of the second dataset.

18. The instructions, when executed, further cause the one or more processors to store metadata that identifies the second dataset as having the first dimension of the first dataset as a compatible dimension with the second dataset; 18. The non-transitory computer-readable storage medium of any combination of claims 15 to 17, wherein the instructions, when executed, cause the one or more processors to convert the query including instructions that, when executed, cause the one or more processors to stop specifying the structured query defining the join as including the first dimension of the first dataset based on the metadata that identifies the second dataset as having the first dimension of the first dataset as a dimension that is compatible with the second dataset.

19. 20. The non-transitory computer-readable storage medium of claim 18, wherein the metadata is stored separately from a dataset schema that defines the second dataset in the database.

20. the first data set includes a third dimension of the two or more dimensions identified by the query; the second data set excludes the third dimension of the two or more dimensions; the instructions further include instructions that, when executed, cause the one or more processors to store metadata that identifies the second dataset as having the first dimension of the first dataset as a compatible dimension with the second dataset, but not the third dimension of the first dataset as a compatible dimension with the second dataset; The instructions include instructions that, when executed, cause the one or more processors to transform the query, the instructions causing the one or more processors to: determining, based on the metadata, that the third dimension of the first data set is not the compatible dimension with the second data set; and generating an error indicating that the transformation of the query cannot be performed based on determining that the third dimension of the first dataset is not the compatible dimension with the second dataset.

Images