Sharing of Materialized Views in Database Systems

Abstract

Systems, methods, and apparatus for sharing materialized views in a multi-tenant database system. The method includes defining a materialized view based on a source table associated with a first account of the multi-tenant database. The method includes defining cross-account access permissions to the materialized view for a second account so that the second account can read the materialized view without replicating the materialized view. The method includes modifying a source table of the materialized view. The method includes identifying whether the materialized view is out-of-date with respect to the source table by merging the materialized view and the source table.

Description

[0001] Cross-reference to priority application

[0002] This application claims priority to U.S. patent application serial number 16 / 428,395, filed on May 31, 2019, the contents of which are hereby incorporated by reference in their entirety. Technical Field

[0003] The present disclosure relates to databases, and more particularly to data sharing and materialized views in database systems.

[0004] background

[0005] Databases are widely used for data storage and access in computing applications. The goal of database storage is to provide large amounts of information in an organized manner so that it can be accessed, managed, and updated. In a database, data can be organized into rows, columns, and tables. Different database storage systems can be used to store different types of content, such as bibliographic, full-text, digital, and / or image content. Furthermore, in computing, different database systems can be classified based on the organization method of the database. There are many different types of databases, including relational databases, distributed databases, cloud databases, object-oriented databases, and others.

[0006] Databases are used by various entities and companies to store information that may need to be accessed or analyzed. In one example, a retail company may store a list of all sales transactions in a database. The database may include information such as when the transaction occurred, where the transaction occurred, the total cost of the transaction, identifiers and / or descriptions of all items purchased in the transaction, etc. The same retail company may also store, for example, employee information in the same database, which may include employee names, employee contact information, employee work history records, employee compensation rates, etc. Depending on the needs of the retail company, employee information and transaction information may be stored in different tables in the same database. When a retail company wants to know the information stored in a database, it may need to "query" its database. The retail company may want to find data about, for example, the names of all employees working in a specific store, the names of all employees working on a specific date, all transactions conducted for a specific product within a specific time frame, and so on.

[0007] When a retail store wants to query its database to extract certain organized information from the database, a query statement is executed against the database data. The query returns certain data based on one or more query predicates, which indicate what information the query should return. The query extracts specific data from the database and formats the data into a readable form. Queries can be written in a language that the database understands, such as Structured Query Language ("SQL"), whereby the database system can determine what data should be located and how the data should be returned. The query can request any relevant information stored within the database. If the appropriate data can be found to respond to the query, the database has the potential to reveal complex trends and activities. This power can only be exploited by using successfully executed queries.

[0008] In some cases, different organizations, individuals, or companies may want to share database data. For example, an organization may have valuable information stored in a database that can be sold or marketed to a third party. The organization may want to enable the third party to view the data, search the data, and / or run reports on the data. In traditional approaches, data is shared by copying the data to a storage resource accessible to the third party. This enables the third party to read, search, and run reports on the data. However, copying data is time- and resource-intensive, and can consume significant storage resources. Additionally, when the original data is updated by the data owner, those modifications will not propagate to the copied data.

[0009] In view of the foregoing, this paper discloses systems, methods and devices for instant and zero-copy data sharing in a multi-tenant database system. The systems, methods and devices disclosed herein provide means for querying shared data, generating and refreshing materialized views based on shared data, and sharing materialized views. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Non-limiting and non-exhaustive embodiments of the present disclosure are described with reference to the following drawings, wherein, unless otherwise specified, like reference numerals refer to the same or similar parts throughout the various views. The advantages of the present disclosure will be better understood through the following description and drawings, wherein:

[0012] Figure 1 is a schematic block diagram illustrating accounts in a multi-tenant database according to one embodiment;

[0013] Figure 2 is a schematic diagram illustrating a system for providing and accessing database services according to one embodiment;

[0014] Figure 3 is a schematic diagram showing a multi-tenant database with separated storage resources and computing resources according to one embodiment;

[0015] Figure 4 is a schematic block diagram illustrating an object hierarchy according to one embodiment;

[0016] Figure 5 is a schematic diagram illustrating role-based access according to one embodiment;

[0017] Figure 6 is a schematic diagram showing usage authorization between roles according to one embodiment;

[0018] Figure 7 is a schematic diagram illustrating a shared object according to one embodiment;

[0019] Figure 8 is a schematic diagram illustrating cross-account authorization according to one embodiment;

[0020] Fig. 9 is a schematic block diagram illustrating components of a shared component according to one embodiment;

[0021] Fig.10 is a schematic diagram of a system and process flow for generating a materialized view based on shared data according to one embodiment;

[0022] Fig.11 is a schematic diagram of a process flow for generating and refreshing a materialized view according to one embodiment;

[0023] Fig.12 is a schematic diagram of a processing flow for updating a source table of a materialized view and refreshing the materialized view relative to its source table according to one embodiment;

[0024] Fig.13 is a schematic diagram of an instantiated view according to one embodiment;

[0025] Fig.14 is a schematic block diagram of a data processing platform including a computing service manager according to one embodiment;

[0026] Fig.15 is a schematic block diagram of a computing service manager according to one embodiment;

[0027] Fig.16 is a schematic block diagram of an execution platform according to one embodiment;

[0028] Fig.17 is a schematic block diagram of a database processing environment according to one embodiment;

[0029] Fig.18 is a schematic flow chart of a method for generating and refreshing materialized views across accounts in a multi-tenant database system according to one embodiment;

[0030] Fig.19 is a schematic flow chart of a method for sharing materialized views across accounts in a multi-tenant database system according to one embodiment; and

[0031] Fig. 20 is a block diagram depicting an example computing device or system consistent with one or more embodiments disclosed herein.

[0032] Detailed Description

[0033] Disclosed herein are systems, methods, and devices for generating materialized views across accounts in a multi-tenant database system and further for sharing materialized views across accounts. A database system may have multiple accounts or clients, each of which stores a unique data set within the database system. In an example embodiment, the database system may store and manage data for multiple enterprises, and each of the multiple enterprises may have its own account within the database system. In some cases, it may be desirable to allow two or more different accounts to share data. Data may be shared between a provider account and a recipient account that owns and shares the data. If a recipient account can query the data to generate reports or analyze the data based on the data, the data may be more valuable to the recipient account. If a recipient account often runs the same query on the data, the recipient account may wish to generate a materialized view for the query. Materialized views enable recipient accounts to quickly generate query results about the data each time the same query is run without having to read or process all the data.

[0034] In view of the foregoing, the systems, methods, and devices disclosed herein enable data sharing between accounts of a multi-tenant database system. The systems, methods, and devices disclosed herein further enable cross-account generation and refreshing of materialized views based on shared data. The systems, methods, and devices disclosed herein further enable cross-account sharing and refreshing of materialized views, so that limited scope of data can be shared between accounts.

[0035] In an example embodiment of the present disclosure, an account of a multi-tenant database may be associated with a retail store that sells goods provided by a manufacturer. Manufacturers and retail stores may each have their own accounts within the multi-tenant database system. Retail stores may store data about which items supplied by manufacturers are sold and how many items are sold. Retail stores may store other data, such as where items are sold, the selling price of items, whether items are purchased online or in retail stores, demographic information of people who purchase items, and the like. The data stored by retail stores may be of great value to manufacturers. Retail stores and manufacturers may enter into an agreement so that manufacturers can access data about items that have been sold by retail stores. In this example embodiment, the retail store is a provider account because the retail store owns the sales data of the items. The manufacturer is a recipient account because the toy manufacturer will view the data owned by the retail store. The data of the retail store is stored within the multi-tenant database system. The retail store provides the manufacturer with cross-account access rights that allow the manufacturer to read data about the sales of the manufacturer's items. The retail store may restrict the manufacturer from viewing any other data, such as employee data, sales data of other items, and the like. The manufacturer can view and query the data owned by the retail store and provided to the manufacturer. The manufacturer can generate materialized views from the data. Materialized views store query results so the manufacturer can query the data faster. Materialized views can automatically refresh to reflect any updates made to their source tables (i.e., tables owned by the retail store). The manufacturer can make multiple materialized views for many different queries that the manufacturer typically requires. Materialized views can be generated privately by the manufacturer so that the retail store cannot see which materialized views the manufacturer has generated.

[0036] In further embodiments of the present disclosure, and in further the same example scenario as above, a retail store may wish to share summary information with a manufacturer without allowing the manufacturer to view all information stored in the retail store account of a multi-tenant database. In such an embodiment, the retail store may generate a materialized view and share the materialized view only with the manufacturer. In an example scenario, the retail store may generate a materialized view that indicates how many different items produced by the manufacturer are offered for sale by the retail store, how many items produced by the manufacturer have been sold in a specific time period, the average price of items produced by the manufacturer sold by the retail store, and so on. It should be understood that the materialized view can provide any relevant summary information according to the needs of the database client. In an example embodiment, the retail store may share the materialized view only with the manufacturer, so the manufacturer can view the summary information, but not the underlying data, schema, metadata, data organization structure, etc. In this example embodiment, the retail store can automatically refresh the materialized view when the source table of the materialized view has been modified or updated.

[0037] The systems, methods, and devices disclosed herein provide improved means for sharing data, sharing materialized views, generating materialized views based on shared data, and automatically updating materialized views based on shared data. Such systems, methods, and devices as disclosed herein provide significant benefits to database clients that wish to share data and / or read data owned by another party.

[0038] A materialized view is a database object that stores query results. A materialized view is generated based on a source table that supplies query results. A materialized view can be stored locally in a cache resource of an execution node so that it can be quickly accessed when processing a query. A materialized view is usually generated for performance reasons so that query results can be obtained faster and query results can be calculated using fewer processing resources. A materialized view can be cached as a specific table rather than a view so that the materialized view can be updated to reflect any changes made to the source table. The source table can be modified by insert, delete, update and / or merge commands, and these modifications can cause the materialized view to be out of date relative to the source table. When changes have been made to the data in the source table, but these changes have not yet been propagated to the materialized view, the materialized view is "out of date" relative to its source table. When a materialized view is out of date relative to a source table, the accurate query results are no longer determined solely on the materialized view. The embodiments disclosed herein provide an improved device for generating, storing and refreshing a materialized view so that queries can be executed on the materialized view even if the materialized view is out of date relative to its source table.

[0039] Embodiments of the present disclosure enable cross-account data sharing using secure views. A view can be defined as a secure view when it is specifically designated for data privacy or to restrict access to data for all accounts that should not be exposed to the underlying tables. For example, data may be exposed in a secure view when an account only has access to a subset of the data. Secure views allow database accounts to expose restricted sets of data to other accounts or users without exposing the underlying unrestricted data to these other accounts or users. In an embodiment, a provider account can grant a recipient account cross-account access to its data. A provider account can restrict a recipient account to viewing only certain data, and can restrict the recipient account from viewing any underlying organizational patterns or statistics about the data.

[0040] In an embodiment, a secure view provides multiple security guarantees compared to a regular view. In an embodiment, a secure view does not disclose the view definition to non-owners of the view. This affects various operations that access the data dictionary. In an embodiment, a secure view does not disclose information about any underlying data of the view, including the amount of data processed by the view, the tables accessed by the view, and so on. This affects the displayed statistics, which involve the number of bytes and partitions scanned in the query, and those displayed in the query profile for queries involving secure views. In an embodiment, a secure view does not disclose data from the table accessed by the view that is filtered out by the view. In such an embodiment, a client account associated with a non-secure view can access data that will be filtered out by utilizing query optimizations that can cause user expressions to be evaluated before secure expressions (e.g., filtering and merging). In such an embodiment, in order to achieve this requirement, a set of query optimizations that can be applied to queries containing secure views can be limited to ensure that user expressions that may leak data are not evaluated before filtering the view.

[0041] In an embodiment, data in a multi-tenant database system is stored across multiple shared storage devices. Data can be stored in a table, and the data in a single table can be further partitioned or separated into multiple immutable storage devices (referred to herein as micro-partitions). Micro-partitions are immutable storage devices that cannot be updated in place and must be regenerated when the data stored therein is modified. The analog of the micro-partition of a table can be different storage buildings within a storage warehouse area (storage compound). By analogy, a storage warehouse area is similar to a table, and each individual storage building is similar to a micro-partition. Thousands of items are stored in the entire storage warehouse area. Since many items are placed in the storage warehouse area, it is necessary to organize these items in multiple separate storage buildings. These items can be organized across multiple separate storage buildings in any meaningful way. For example, one storage building can store clothing, another storage building can store household items, another storage building can store toys, and so on. Each storage building can be labeled to make it easier to find items. For example, if a person wants to find a stuffed teddy bear, then he will know to go to the storage building where the toy is stored. The storage building where the toys are stored can be further organized into rows of shelves. The toy storage building may be organized so that all stuffed animals are placed on one row of shelves. Thus, someone looking for stuffed bears may know to visit the building where the toys are stored and may know to visit the row where the stuffed animals are stored. Further analogous to database technology, each row of shelves in a storage building storing a warehouse area may be analogous to a database data column within a micro-partition of a table. The labels for each storage building and for each row of shelves are analogous to metadata in a database context.

[0042] When a transaction is executed on a table, all affected micro-partitions in the table are recreated to generate new micro-partitions that reflect the modifications made by the transaction. After the transaction is fully executed, all the original micro-partitions that were recreated can be removed from the database. After each transaction is executed on a table, a new version of the table is generated. If the data in the table undergoes many changes (such as inserts, deletes, updates, and / or merges), the table may go through many versions over a period of time. Each version of the table may include metadata that indicates what transaction generated the table, when the transaction was executed, when the transaction was fully executed, and how the transaction changed one or more rows in the table. The disclosed system, method, and apparatus for low-cost table versioning can be used to provide an efficient means for updating table metadata after one or more changes (transactions) occur on a table.

[0043] Micro-partitions can be thought of as batch processing units, where each micro-partition has a contiguous storage unit. For example, each micro-partition may contain between 50MB and 500MB of uncompressed data (note that the actual size in storage may be smaller because the data may be stored compressed). Groups of rows in a table can be mapped to individual micro-partitions organized in columns. This size and structure allows for extremely fine-grained selection of micro-partitions to be scanned, which can include millions or even hundreds of millions of micro-partitions. This granular selection process may be referred to herein as metadata-based "pruning". Pruning involves using metadata to determine which parts of a table (including which micro-partitions or groups of micro-partitions in a table) are irrelevant to a query, then avoiding those irrelevant micro-partitions when responding to the query and only scanning relevant micro-partitions to respond to the query. Metadata about all rows stored in a micro-partition can be automatically collected, including: the value range of each column in the micro-partition; the number of different values; and / or other attributes for optimized and efficient query processing. In one embodiment, micro-partitioning can be automatically performed on all tables. For example, a table can be transparently partitioned using the sorting that occurs when inserting / loading data.

[0044] A multitenant database or multitenant data warehouse supports multiple different customer accounts at one time. As an example, Figure 1 is a schematic block diagram showing a multi-tenant database or data warehouse that supports many different customer accounts A1, A2, A3, An, etc. Customer accounts can be separated by multiple security controls, including different uniform resource locators (URLs) for connecting to different access credentials, different data storage locations (such as Amazon Web Services S3 buckets), and different account-level encryption keys. Therefore, each customer is only allowed to view, read and / or write the customer's own data. By design, a customer may not be able to see, read or write another customer's data. In some cases, strict separation of customer accounts is the backbone of a multi-tenant data warehouse or database system.

[0045] In some cases, it may be desirable to allow cross-account data sharing and / or cross-account generation and updating of materialized views. However, no current multi-tenant database system allows data to be shared between different customer accounts in an instantaneous, zero-copy, easily controlled manner.

[0046] Based on the foregoing, disclosed herein are systems, methods, and devices that can be implemented in one embodiment for generating, updating, and / or viewing instantiated views based on shared data. The systems, methods, and devices disclosed herein further provide means for sharing instantiated views. Data can be shared so that it can be accessed immediately without copying the data. Instantiated views can be accessed by multiple parties without copying the instantiated views. Some embodiments provide access to data using fine-grained controls to maintain separation of desired data while allowing access to data that a client wishes to share.

[0047] Embodiments disclosed herein provide systems, methods, and devices for sharing a "shared object" or "database object" between a provider account and one or more other accounts in a database system. The provider account shares the shared object or database object with one or more other "recipient" accounts. The provider account can enable one or more recipient accounts to view materialized views and / or generate materialized views based on the provider's data. In one embodiment, a shared object or database object may include database data, such as data stored in a database table owned by the provider account. A shared object or database object may include metadata about the database data, such as the minimum / maximum values ​​of a table or micro-partition of a database, infrastructure or architectural details of the database data, and the like. A shared object may include a list of all other accounts that may receive cross-account access rights to elements of a shared object. The list may indicate, for example, that a second account can use the program logic of a shared object without seeing any underlying code that defines the program logic. The list may further indicate, for example, that a third account can use the database data of one or more tables without seeing any structural information or metadata about the database data. The list may indicate any combination of usage privileges for elements of a shared object, including whether an auxiliary account can view metadata or structural information of database data or program logic. The list can indicate whether the recipient account has permission to generate or update materialized views based on the provider's database data.

[0048] A detailed description of systems and methods consistent with various embodiments of the present disclosure is provided below. Although several embodiments are described, it should be understood that the present disclosure is not limited to any one embodiment, but includes many alternatives, modifications, and equivalents. In addition, although many specific details are set forth in the following description to provide a thorough understanding of the embodiments disclosed herein, some embodiments may be implemented without some or all of these details. In addition, for the sake of clarity, certain technical materials known in the related art are not described in detail to avoid unnecessarily obscuring the present disclosure.

[0049] Now referring to the accompanying drawings, Figure 1is a schematic block diagram showing a multi-tenant database or data warehouse that supports many different customer accounts A1, A2, A3, An, etc. Customer accounts can be separated by multiple security controls, including different uniform resource locators (URLs) for connecting to different access credentials, different data storage locations (such as Amazon Web Services S3 buckets), and different account-level encryption keys. Therefore, each customer is only allowed to view, read and / or write the customer's own data. By design, a customer may not be able to see, read or write another customer's data. In some cases, strict separation of customer accounts is the backbone of a multi-tenant data warehouse or database system.

[0050] Figure 2 2 is a schematic diagram of a system 200 for providing and accessing database data or services. System 200 includes a database system 202, one or more servers 204, and a client computing system 206. Database system 202, one or more servers 204, and / or client computing system 206 may communicate with each other via a network 208, such as the Internet. For example, one or more servers 204 and / or client computing system 206 may access database system 202 via network 208 to query a database and / or receive data from a database. Data from the database may be used by one or more servers 204 or client computing system 206 for any type of computing application. In one embodiment, database system 202 is a multi-tenant database system that hosts data for multiple different accounts.

[0051] The database system 202 includes a sharing component 210 and a storage device 212. The storage device 212 may include a storage medium for storing data. For example, the storage device 212 may include one or more storage devices for storing database tables, schemas, encryption keys, data files, or any other data. The sharing component 210 may include hardware and / or software for implementing cross-account sharing of data or services and / or for associating viewing privileges with data or services. For example, the sharing component 210 may implement cross-account generation and updating of instantiated views based on shared data. The sharing component 210 may define a secure view of database data so that two or more accounts can determine a common data point without revealing the data point itself or any other data point that is not shared between the accounts. For further example, the sharing component 210 may process a query / instruction received from a remote device to access shared data or shared data. The query / instruction may be received from one or more servers 204 or client computing systems 206. In one embodiment, the sharing component 210 is configured to allow data to be shared between accounts without creating duplicate copies of tables, data, etc. outside the shared account. For example, the sharing component may allow computer resources assigned to the shared account to execute any query or instruction provided by the external account.

[0052] In one embodiment, the storage and computing resources for the multi-tenant database 100 are logically and / or physically separated. In one embodiment, storage is a common shared resource between all accounts. Computing resources can be set up as virtual warehouses independently by account. In one embodiment, a virtual warehouse is a group of computing nodes that access data in the storage layer and calculate query results. Separating computing nodes or resources from storage allows each layer to be scaled independently. The separation of storage and computing also allows shared data to be processed independently by different accounts, and computing in one account does not affect computing in other accounts. That is, in at least some embodiments, there is no contention between computing resources when running queries on shared data.

[0053] Figure 3 is a schematic block diagram of a multi-tenant database 300 showing the separation of storage and computing resources. For example, the multi-tenant database 300 may be a data warehouse hosting multiple different accounts (A1, A2, A3 to An). Figure 3 In the example, account A1 has three virtual warehouses running, account A2 has one virtual warehouse running, and account A3 has no virtual warehouses running. In one embodiment, all of these virtual warehouses have access to a storage layer that is separate from the computing nodes of the virtual warehouses. In one embodiment, virtual warehouses can be dynamically provided or removed based on the current workload of the account.

[0054] In one embodiment, the multi-tenant database system 300 uses an object hierarchy in an account. For example, each customer account may contain an object hierarchy. The object hierarchy is typically rooted in the database. For example, a database may contain a schema, which in turn may contain objects such as tables, views, sequences, file formats, and functions. Each of these objects has a specific purpose: tables store relational or semi-structured data; views define logical abstractions for stored data; sequences provide a means for generating increasing numbers; file formats define the way to parse ingested data files; and functions contain user-defined execution procedures. In embodiments disclosed herein, a view may be associated with a secure user-defined function definition so that the underlying data associated with the view is hidden from non-owner accounts that have access to the view.

[0055] Figure 4is a schematic block diagram illustrating an object hierarchy within a customer account. Specifically, an account may include a hierarchy of objects that may be referenced in a database. For example, customer account A1 includes two database objects D1 and D2. Database object D1 includes a schema object S1, which in turn includes a table object T1 and a view object V1. Database object D2 includes a schema object S2, which includes a function object F2, a sequence object Q2, and a table object T2. Customer account A2 includes a database object D3 with a schema object S3 and a table object T3. The object hierarchy may control how objects, data, functions, or other information or services of an account or database system are accessed or referenced.

[0056] In one embodiment, a database system implements role-based access control to manage access to objects in a customer account. In general, role-based access control consists of two basic principles: roles and authorizations. In one embodiment, a role is a special object in a customer account that is assigned to a user. Authorizations between roles and database objects define the privileges that the role has on those objects. For example, when executing the command "show databases", a role with use authorization on a database can "see" the database; a role with select authorization on a table can read from the table but cannot write to it. The role will need to have modify authorization on the table in order to write to it.

[0057] Figure 5 is a schematic block diagram showing role-based access to objects in a customer account. Customer account A1 contains role R1, which has authorizations for all objects in the object hierarchy. Assuming that these authorizations are use authorizations between R1 and D1, D2, S1, S2 and select authorizations between R1 and T1, V1, F2, Q2, T2, a user with activated role R1 can view all objects and read data from all tables, views, and sequences, and can execute function F2 within account A1. Customer account A2 contains role R3, which has authorizations for all objects in the object hierarchy. Assuming that these authorizations are use authorizations between R3 and D3, S3 and select authorizations between R3 and T3, a user with activated role R3 can view all objects and read data from all tables, views, and sequences within account A2.

[0058] Figure 6The usage authorization between roles is shown. With role-based access control, usage rights can also be granted from one role to another. A role with usage authorization for another role "inherits" all access privileges of the other role. For example, role R2 has usage authorization for role R1. A user with activated role R2 (e.g., with corresponding authorization details) can view all objects and read from all objects because role R2 inherits all authorizations of role R1.

[0059] Figure 7 is a schematic block diagram showing a shared object SH1. In an embodiment, a shared object is a column of data across one or more tables, and cross-account access permissions are granted to the shared object so that one or more other accounts can generate, view and / or update materialized views based on the data in the shared object. Customer account A1 contains a shared object SH1. Shared object SH1 has a unique name "SH1" in customer account A1. Shared object SH1 contains role R4, which has authorization to database D2, schema S2, and table T2. The authorization to database D2 and schema S2 can be a use authorization, and the authorization to table T2 can be a select authorization. In this case, table T2 in schema S2 in database D2 will be shared read-only. Shared object SH1 contains a list of references to other customer accounts (including account A2).

[0060] After a shared object is created, the shared object can be imported or referenced by the recipient accounts listed in the shared object. For example, importing a shared object from a provider account may also be importing from other customer accounts. The recipient account can run a command to list all shared objects available for import. The recipient account can list the shared object and subsequently import it only if the shared object is created with a reference that includes the recipient account. In one embodiment, a reference to a shared object in another account is always qualified by the account name. For example, recipient account A2 will reference shared SH1 in provider account A1 with the example qualified name "A1.SH1".

[0061] In one embodiment, processing or importing a shared object may include: creating an alias object in a recipient account; linking the alias object to the topmost shared object in a provider account in an object hierarchy; granting roles in the recipient account usage privileges to the alias object; and granting the recipient account roles usage privileges to roles contained in the shared object.

[0062] In one embodiment, the recipient account that imports the shared object or data creates an alias object. An alias object is similar to a normal object in a customer account. An alias object has its own unique name, which is used to identify the alias object. An alias object can be linked to the topmost object shared in each object hierarchy. If multiple object hierarchies are shared, multiple alias objects can be created in the recipient account. Whenever an alias object is used (e.g., read from an alias object, write to an alias object), the alias object will be replaced by the normal object linked to it in the provider account internally. In this way, an alias object is only a proxy object of a normal object, not a duplicate object. Therefore, when reading from an alias object or writing to an alias object, these operations will affect the original object to which the alias is linked. Like a normal object, when an alias object is created, it will be granted to the user's activation role.

[0063] In addition to the aliased object, a grant is created between the role in the recipient account and the role included in the shared object. This is a role-to-role usage grant across customer accounts. Role-based access control now allows users in the recipient account to access objects in the provider account.

[0064] Figure 8 is a schematic block diagram showing logical authorizations and links between different accounts. A database alias object D5 is created in account A2. Database alias D5 references database D2 via link L1. Role R3 has use authorization G1 on database D5. Role R3 has a second use authorization G2 for role R4 in customer account A1. Authorization G2 is a cross-account authorization between accounts A1 and A2. In one embodiment, role-based access control allows users with activated role R3 in account A2 to access data in account A1. For example, if a user in account A2 wants to read data in table T2, role-based access control will allow this because role R3 has use authorization for role R4, which in turn has select authorization for table T2. For example, a user with activated role R3 can access T2 by running a query or select against "D5.S2.T2".

[0065] The use of object aliases and cross-account authorization from roles in the recipient account to roles in the provider account allows users in the recipient account to access information in the provider account. In this way, the database system can share data between different customer accounts in an instantaneous, zero-copy, and easy-to-control manner. Sharing can be instantaneous because alias objects and cross-account authorization can be created in milliseconds. Sharing can be zero-copy because no data must be copied in the process. For example, all queries or selections can be made directly to the shared objects in the provider account without creating a copy in the recipient account. Sharing is also easy to control because it utilizes easy-to-use technology for role-based access control. In addition, in an embodiment with separate storage and computing, when queries are executed on shared data, there is no contention between computing resources. Therefore, different virtual warehouses in different customer accounts can process shared data separately. For example, a first virtual warehouse for a first account can use the data shared by the provider account to process database queries or statements, while a second virtual warehouse for a second account or provider account can use the shared data of the provider account to process database queries or statements.

[0066] Fig. 9 902-914 are provided by way of example only and may not all be included in all embodiments. For example, each of the components 902-914 may be included in a separate device or system or may be implemented as part of a separate device or system.

[0067] The cross-account permission component 902 is configured to create and manage permissions or authorizations between accounts. The cross-account permission component 902 can generate a shared object in a provider account. For example, a user of a provider account can provide input indicating that one or more resources should be shared with another account. In one embodiment, a user can select an option to create a new shared object so that resources can be shared with an external account. In response to user input, the cross-account permission component 902 can create a shared object in a provider account. The shared object can include a role that can be granted access rights to resources to share with an external account. An external account can include a customer account or other account separate from a provider account. For example, an external account can be another account hosted on a multi-tenant database system.

[0068] When created, a shared object may be granted permissions to one or more resources within a provider account. Resources may include a database, schema, table, sequence, or function of a provider account. For example, a shared object may contain a role (i.e., a shared role) that is granted permissions to read, select, query, or modify a data storage object such as a database. Permissions may be granted to a shared object or a shared role in a shared object in a manner similar to how permissions are granted to other roles using role-based access control. A user is able to access an account and grant permissions to a shared role so that the shared role can access resources that should be shared with external accounts. In one embodiment, a shared object may include a list of objects and access levels to which a shared role has permissions.

[0069] Shared objects can also be made available or linked to specific external accounts. For example, a shared object can store a list of accounts in a provider account that have permissions to a shared role or shared object. A user with a provider account can add or remove accounts from the account list. For example, a user can modify the list to control which accounts can access objects shared via a shared object. External accounts listed or identified in a shared object can be given access to resources and granted shared role access rights to the shared object. In one embodiment, a specific account can perform a search to identify shared objects or provider accounts that have been shared with the specific account. A recipient or user of a specific account can view a list of available shared objects.

[0070] Alias ​​component 904 is configured to generate aliases for data or data objects shared by separate accounts. For example, an alias object can be created in a recipient account that corresponds to a shared resource shared by a provider account. In one embodiment, an alias object is created in response to a recipient account accepting a shared resource or attempting to access a shared resource for the first time. An alias object can serve as an alias for a data object of the highest object hierarchy shared by a provider account (e.g., see Figure 8 , where D5 is an alias of D2). Alias ​​component 904 can also generate links between alias objects and shared objects (see, for example, Figure 8 , where L1 is the link between D5 and D2). The link may be created and / or stored in the form of an identifier or name of the original or "real" object. For example, Figure 8 The link L1 in may include an identifier for D2 stored in the alias object D5, the identifier comprising a unique system wide name, such as "A1.D2".

[0071] The alias component 904 can also grant access rights to alias objects to roles in the recipient account (the account with which the provider account shares data or resources) (see, for example, Figure 8In addition, the alias component 904 can also grant the role in the recipient account to the shared role in the shared object of the provider account (for example, see Figure 8 G2). With the created alias object, the link between the alias object and the object in the provider account, and the authorization of the role in the recipient account, the recipient account can freely run queries, declare or "view" the shared data or resources in the provider account.

[0072] Request component 906 is configured to receive a request from an account to access a shared resource in a different account. The request may include a database query, a select statement, etc. to access the resource. In one embodiment, the request includes a request for an alias object of the requesting account. Request component 906 may identify a resource linked to the alias object, such as a database or table in a provider account. Request component 906 may identify a linked object based on an identifier of the alias object.

[0073] The request component 906 can be further configured to receive a request from an account to count common data points between two accounts. The request component 906 can be associated with a first account and can receive a request from a second account to generate a secure connection between the two accounts and determine how many and which data points are shared between the two accounts. A data point can be a single subject or column identifier or multiple subjects.

[0074] The access component 908 is configured to determine whether an account has access to a shared resource of a different account. For example, if a first account requests access to a resource of a different second account, the access component 908 can determine whether the second account has been granted access to the first account. The access component 908 can determine whether the requesting account has access by determining whether the sharing object identifies the requesting account. For example, the access component 908 can check whether the requesting account exists in a list of accounts stored by the sharing object. The access component 908 can also check whether the sharing object that identifies the requesting account has access rights (e.g., authorization) to the recipient data resource in the provider account.

[0075] In one embodiment, access component 908 can check for the existence of authorization from a shared role in a provider account to a requesting role in a requesting account. Access component 908 can check for the existence of a link between an alias object targeted by a database request or statement, or for authorization between a requesting role and an alias object. For example, access component 908 can check Figure 8 The presence or absence of one or more of L1, G1, and G2 shown. In addition, access component 908 can check authorization between roles in the shared object and objects (such as tables or databases) of the provider account. For example, access component 908 can check Figure 8Whether there is authorization between the role R4 in and the database D2. If the access component 908 determines that the requesting account has access to the shared resource, the sharing component 210 or the processing component 910 can satisfy the request. If the access component 908 determines that the requesting account does not have permission to the requested data or object, the request will be rejected.

[0076] The processing component 910 is configured to process database requests, queries or statements. The processing component 910 can process and provide a response to a request from one account to access or use data or services in another account. In one embodiment, the processing component 910 provides a response to the request by processing the request using raw data in a provider account different from the requesting account. For example, the request can be directed to a database or table stored in or for a first account, and the processing component 910 can use the database or table of the first account to process the request and return a response to the second account of the request.

[0077] In one embodiment, processing component 910 performs processing of shared data without creating a duplicate table or other data source in the requesting account. Typically, data must first be ingested into the account that wishes to process or perform an operation on the data. Processing component 910 can save processing time, latency, and / or storage resources by allowing a recipient account to access shared resources in a provider account without creating a copy of the data resource in the recipient account.

[0078] The processing component 910 can use different processing resources for different accounts to perform processing of the same data. For example, a first virtual warehouse for a first account can use data shared by a provider account to process a database query or statement, while a second virtual warehouse for a second account or provider account can use the shared data of the provider account to process a database query or statement. Using separate processing resources to process the same data may prevent contention for processing resources between accounts. Processing resources may include dynamically provided processing resources. In one embodiment, processing of shared data is performed using the virtual warehouse of the requesting account, even if the data may be in storage of a different account.

[0079] The security view component 912 is configured to define a security view of a shared object, a data field of a shared object, a data field of a database object, etc. In an embodiment, the security view component 912 defines a security view using the SECURE keyword in the view field, and the SECURE attribute can be set or unset on the view using the ALTER VIEW command. In various embodiments, the security view component 912 can implement such a command only under the manual guidance of a client account, or can be configured to automatically implement such a command. The security view component 912 can change the parser to support the security keyword before the view name and the new change view rule. In an embodiment, the change view rule can be more general to include other view-level attributes. In terms of metadata support, competition can be effectively stored as a table, and the change may involve changing the table data persistence object, which includes a security flag indicating whether the view is a secure view (this can be implemented outside the view text including the security flag). Secure user-defined function definitions (i.e., table data persistence objects) can be hidden from users who are not view owners. In such an embodiment, a command to display a view will return results to the owner of the view as usual, but will not return the secure user-defined function definition to a non-owner second account that has access to the view.

[0080] The safe view component 912 can change the transformation of the parse tree, such as view merging and predicate information. The specification implementation can include annotating the query block so that the query block is designated as being from a safe view. In such an implementation, the query block cannot be combined with external query blocks (e.g., view merging) or with expressions (e.g., via filter pushdown).

[0081] The security view component 912 can rewrite the query plan tree during optimization (e.g., during filter pull-up and / or filter push-down). The security view component 912 can be configured to ensure that no expressions originating from the security view cannot be pushed down below the view boundaries. The security view component 912 can be configured to achieve this by implementing a new type of projection that behaves the same as a standard projection, but because it is not a standard projection, it cannot match any rewrite rule preconditions. Therefore, the associated rewrites are not applied. The security view component 912 can be configured to identify which type of projection (e.g., a standard projection or a secure projection) is to be generated after a query block has been designated as coming from a secure user-defined function definition or not coming from a secure user-defined function definition.

[0082] The security view component 912 is configured to optimize the performance of the security view in the zero-copy data sharing system. In various embodiments known in the art, it is known that the security view causes a performance loss, which may effectively weaken the ability of the optimizer to apply certain transformations. Such an embodiment can be improved by considering certain transformations to be safe, where the security indicates that the operation being transformed will not have any side effects on the system. Such side effects may be caused by user-defined functions (UDFs) that perform operations that cannot easily identify unsafe operations or operations that may not reveal information about the data value that caused the failure (e.g., division by zero or some similar operations). The security view component 912 can annotate expressions with security attributes of expressions and then enable transformations that allow expressions to be pushed into the security view boundary if the expression is considered safe. If it is known that the expression will not produce errors and the expression does not contain user-defined functions (UDFs), the expression can be considered safe. The security view component 912 can determine whether an expression produces an error by utilizing an expression attribute framework, wherein the expression attribute stores an indication of whether the expression may produce an error.

[0083] Instantiated view component 914 is configured to generate and / or update instantiated views of shared data. Instantiated view component 914 is further configured to share instantiated views that can have security view definitions. Instantiated view component 914 can be integrated in the execution resources assigned to the provider account and / or the recipient account. In an embodiment, the provider account authorizes the recipient account to view the data of the provider account. The provider account can give the recipient account unlimited access to view the data of the provider account, or the provider account can grant the recipient account authorization to view its data using the security view definition. The security view definition can ensure that only part of the data is visible to the recipient account and / or ensure that the basic mode of the relevant data is not visible to the recipient account. In an embodiment, the provider account provides authorization to the recipient account to generate a specific instantiated view generated according to the provider's data. The provider account can grant this authorization and still prohibit the recipient account from viewing any actual data.

[0084] The instantiated view component 914 generates and refreshes the instantiated view. In an embodiment, the provider account generates the instantiated view based on its own data. The instantiated view component 914 can generate the instantiated view for the provider account. The instantiated view component 914 can further grant cross-account access rights to the recipient account so that the recipient account can view the instantiated view. In an embodiment, the recipient account requests the instantiated view, and the instantiated view component 914 generates the instantiated view. In such an embodiment, the instantiated view component can be integrated into the execution platform of the recipient account.

[0085] The instantiated view component 914 refreshes the instantiated view relative to its source table. In an embodiment, the instantiated view is generated and stored by a recipient account, while the source table of the instantiated view is stored and managed by a provider account. When the data in the source table is updated, the instantiated view component 914 is configured to refresh the instantiated view to propagate the updates to the instantiated view. If the source table has been updated and those updates have not yet been propagated to the instantiated view, the instantiated view is out of date relative to its source table. According to the systems, methods, and devices disclosed herein, queries can still be executed using an out-of-date instantiated view by merging the instantiated view with its source table to identify any differences between the instantiated view and the source table. The instantiated view component 914 is configured to merge the instantiated view with its source table to identify whether any updates have been made to the source table since the last refresh of the instantiated view.

[0086] The materialized view component 914 is configured to share a materialized view. In an embodiment, the materialized view component 914 shares a materialized view with another account in a multi-tenant database. The materialized view component 914 can cause the materialized view to automatically refresh relative to its source table so that the shared version of the materialized view is up to date relative to the data in the source table. The materialized view can be shared with another account so that the other account has visibility only to the summary information contained in the materialized view, and not to the source tables of the materialized view or any underlying schema, data, metadata, etc.

[0087] Fig.10 1000 is a schematic diagram of a system 1000 for generating a materialized view based on shared data. The materialized view is generated based on stored data associated with a provider 1002 account. The provider 1002 account includes a provider execution platform 1004 configured to perform tasks on the provider 1002 data. The provider 1002 data is accessed by a recipient 1006 account via a recipient execution platform 1008 with the aid of a shared object 1010. It should be understood that the terms "provider" and "recipient" are merely illustrative and may be alternatively referred to as a first account and a second account, a provider and a consumer, and the like. Either the provider 1002 or the recipient 1006 may generate a materialized view 1012 based on data in the shared object 1010 data. Analysis based on the materialized view 1014 may be performed by either the provider 1002 or the recipient 1006.

[0088] In an embodiment, provider 1002 and recipient 1006 are different accounts associated with the same cloud-based database administrator. In an embodiment, provider 1002 and recipient 1006 are associated with different cloud-based and / or traditional database systems. Provider 1002 includes provider execution platform 1004, which has one or more execution nodes capable of performing processing tasks on database data of provider 1002, wherein the database data is stored in a data storage associated with provider 1002. Recipient 1006 includes recipient execution platform 1008, which has one or more execution nodes capable of performing processing tasks on database data of recipient 1006, wherein the database data is stored in a data storage associated with recipient 1006. The data storage may include a cloud-based scalable storage device so that the database data is distributed across multiple shared storage devices accessible by an execution platform such as 1004 or 1008. Materialized view 1012 is generated by either provider 1002 or recipient 1006. When the provider 1002 generates a materialized view 1012, the provider execution platform 1004 can directly access the provider 1002 data. When the recipient 1006 generates a materialized view 1012, the recipient execution platform 1008 can generate the materialized view 1012 by reading the data in the shared object 1010. The data in the shared object 1010 is owned by the provider 1002 and is accessible to the recipient 1006. The data in the shared object 1010 can have a secure view definition.

[0089] In an embodiment, a shared object 1010 is defined by a provider 1002 and is available to a recipient 1006. A shared object 1010 is a transient and zero-copy means for a recipient 1006 to access data or other objects owned by a provider 1002. A shared object 1010 may be data, a table, one or more micro-partitions of a table, a materialized view, a function, a user-defined function, a schema, and the like. A shared object 1010 may be defined by a provider execution platform 1004 and accessible to a recipient execution platform 1008. A shared object 1010 may provide read-only (but not write) access to provider 1002 data, such that a recipient execution platform 1008 may read provider 1002 data, but may not update or add to provider 1002 data. A shared object 1010 may include a secure view definition such that a recipient execution platform 1008 may not view underlying data in base tables owned by a provider 1002.

[0090] The materialized view 1012 is generated by either the provider 1002 and / or the recipient 1006. The provider 1002 may generate the materialized view directly from its data stored on disk or in cache storage. The provider 1002 may make the materialized view 1012 available to the recipient 1006 directly and / or through the shared object 1010. The materialized view 1012 may have a secure view definition so that the underlying data cannot be seen by other accounts (including the recipient 1006), but the results of the materialized view can be seen. The recipient 1006 may generate the materialized view based on the data in the shared object 1010. In an embodiment, the provider 1002 is notified when another account such as the recipient 1006 generates the materialized view 1012 based on the provider 1002 data. In an embodiment, if the authorized recipient 1006 account generates the materialized view 1012 based on the data within the shared object 1010, the provider 1002 is not notified.

[0091] Fig.11 1 shows a schematic block diagram of a process flow 1100 for incremental update of a materialized view. The process flow 1100 may be performed by any suitable computing device (including, for example, a computing service manager 1402 (see Fig.14 ) and / or shared component 210). Processing flow 1100 includes creating a materialized view at 1102, wherein the materialized view is based on a source table. Processing flow 1100 includes updating the source table at 1104, which may include inserting a new micro-partition into the source table at 1106 and / or removing a deleted micro-partition from the source table at 1108. Processing flow 1100 includes querying the materialized view and the source table at 1110. The query at 1110 includes detecting whether any updates have occurred on the source table that are not reflected in the materialized view. For example, a new micro-partition may be added to the source table that has not yet been added to the materialized view. The deleted micro-partition may be removed from the source table, but it remains in the materialized view. Processing flow 1100 includes applying the update to the materialized view at 1112. Applying updates 1112 can include refreshing the materialized view by inserting new micro-partitions into the materialized view at 1114 , and / or compacting the materialized view by removing deleted micro-partitions at 1116 .

[0092] Fig.12 1402 (see FIG. 1403 ) is a schematic block diagram of an example process flow 1200 for incremental updates of a materialized view. The process flow 1200 may be performed by any suitable computing device (including, for example, a computing service manager 1402 (see FIG. 1404 )). Fig.14) and / or shared component 210). Processing flow 1200 includes generating a materialized view based on a source table at 1202. Processing flow 1200 includes updating the source table at 1204, which may include inserting a micro-partition at 1206 and / or removing a micro-partition at 1208. The processing flow includes querying the materialized view and the source table to detect any updates to the source table that are not reflected in the materialized view at 1210. Processing flow 1200 includes applying the updates to the materialized view at 1212.

[0093] In an embodiment, the source table is owned by a provider account, and the materialized view is generated by a recipient account at 1202. The source table may be stored across one or more of a plurality of shared storage devices shared between a plurality of accounts in a multi-tenant database system. The provider account and the recipient account may each store and manage database data within the database system. The provider account may store and manage database data within the database system, and the recipient account may only connect to the database system to view and / or query data owned and / or managed by other accounts without storing any data of its own. A separate execution resource may be associated with each of the provider account and the recipient account. In an embodiment, the source table is associated with the provider account. The provider account provides the recipient account with cross-account access rights, so that the recipient account may view data in the source table, may view partial data in the source table, may generate materialized views based on the source table, may query the source table, and / or may run user-defined functions on the source table. The provider account may generate a "shared object" including any of the aforementioned access rights.

[0094] A shared object can have a secure view definition so that certain portions of the source table are hidden from the recipient account. A recipient account can have unrestricted access to a source table so that the recipient account can view the underlying data, schemas, organizational structures, etc. A recipient account may have limited access to a source table so that the recipient account can only view query results or materialized views but not any underlying data, schemas, organizational structures, etc. The provider account can determine the level of access granted to the recipient account, and the specific parameters of the access can be tailored based on the type of data stored in the source table and the provider account's needs to keep certain aspects of the data hidden from view.

[0095] In an embodiment, a materialized view is generated at 1202 by an execution resource allocated to a recipient account and / or requested by a user associated with the recipient account. After the recipient account receives the shared object from the provider account, the recipient account can access to generate a materialized view based on the source table. The materialized view generated by the recipient account can be stored in a disk storage or cache storage allocated to the recipient account. The materialized view generated by the recipient account can be stored in a disk storage or cache storage that has been allocated to the provider account and is accessible to the recipient account.

[0096] In an embodiment, a materialized view is generated by execution resources allocated to a provider account and / or requested by a user associated with the provider account at 1202. The materialized view can be generated and managed by the provider account and accessible to one or more recipient accounts. In an embodiment, the materialized view itself is a shared object that can be accessed by one or more recipient accounts.

[0097] Regardless of whether the provider account or the recipient account generates or requests a materialized view, the source table is updated at 1203 by the provider account. The source table may be owned and managed by the provider account, and one or more recipient accounts may have read-only access to the source table. A user associated with the provider account may enter a data manipulation language (DML) command to update the source table. Such a DML command may cause new data to be inserted into the source table, may cause data to be deleted from the source table, may cause data to be merged, and / or may cause data to be updated or changed in the source table. In an embodiment, one or more micro-partitions of the source table are regenerated for any changes that occur to the source table. For example, an insert command may simply result in the generation of one or more new micro-partitions without changing any existing micro-partitions. A delete command may remove rows from the source table, and the delete command may be executed by regenerating one or more micro-partitions so that the deleted rows are removed from one or more micro-partitions. An update command may change data entries in the source table, and the update command may be executed by regenerating one or more micro-partitions so that the modified rows are deleted and regenerated using the updated information. When the provider account updates the source table at 1204, a notification can be provided to the recipient account that the materialized view is now out-of-date with respect to the source table. The materialized view can be automatically refreshed to reflect any updates made to the source table.

[0098] exist Fig.12In the example shown, a materialized view is generated at 1202 by scanning a source table. As shown, the source table includes a data set {1 2 3 4 5 6}. Each reference number {1 2 3 4 5 6} may represent a micro-partition within the source table. It should be understood that the source table may have any number of micro-partitions, and may have hundreds or thousands of micro-partitions. The corresponding materialized view includes data sets [1 (1 2 3)] and [2 (4 5 6)], which may indicate micro-partitions in a database, where micro-partitions are immutable storage objects in a database. The materialized view may be divided into a first data set [1 (1 2 3)] and a second data set [2 (4 5 6)] because the materialized view includes too much data to be stored in a single data set. In an embodiment, the size of the data set is determined based on the cache storage capacity in the execution node of the vehicle instantiated view. Each data set of the materialized view (including [1 (1 2 3)] and [2 (4 5 6)]) itself may be an immutable storage device referred to as a micro-partition in this article. In an embodiment, the source table dataset {1 2 3 4 5 6} is stored across one or more storage devices assigned to a provider account. In various embodiments, the instantiated view datasets [1 (1 2 3)] and [2 (4 5 6)] can be stored across one or more storage devices assigned to a provider account or a recipient account. When the instantiated view dataset is stored in a storage device assigned to a provider account, the instantiated view dataset can be accessible to one or more recipient accounts without replicating the instantiated view dataset. Vice versa, when the instantiated view dataset is stored in a storage device assigned to a recipient account, the instantiated view dataset can be accessible to the provider account and / or one or more additional recipient accounts without replicating the instantiated view dataset.

[0099] exist Fig.12 In the example shown, at 1204, the source table is updated by adding (+7) and removing (-2) (see Δ1). The update to the source table at 1204 can be performed by the execution resource assigned to the provider account. In an embodiment, only the provider account has the ability to write data to the source table. At 1206, two micro-partitions are inserted into the source table by adding (+8) and adding (+9) (see Δ2). At 1208, two micro-partitions are removed from the source table by removing (-1) and (-3) (see Δ3). As shown by Δ (increment), the overall update to the source table includes {+7+8+9-1-2-3}, which includes each of the individual updates made on the source table (see Δ1, Δ2, and Δ3). The overall update to the source table will add micro-partitions numbered 7, 8, and 9, and delete micro-partitions numbered 1, 2, and 3.

[0100] exist Fig.12In the example shown, the materialized view and source table are queried at 1210. The query can be requested by either the provider account or the recipient account. The recipient account can issue a query on the source table only if the recipient account has access permissions that enable the recipient account to query the data in the source table. The processing time of the query can be accelerated by using a materialized view. However, since the source table is updated at 1204, the materialized view is no longer fresh relative to the source table. The execution of the query includes merging the materialized view and the source table to identify any modifications made to the source table that are not reflected in the materialized view. After the materialized view and the source table are merged, the materialized view is scanned, and the source table is scanned. The source table is scanned, and micro-partitions numbered {7 8 9} are detected in the source table, but these micro-partitions are not detected in the materialized view. The materialized view is scanned, and the materialized view data set [1 (1 2 3)] is detected in the materialized view but not in the source table. This materialized view dataset includes information about micro-partitions (1 2 3) in the source table, which no longer exist in the source table because they were removed by the update command at 1204 and the delete command at 1208.

[0101] exist Fig.12 In the example shown, updates are applied to the materialized view at 1212. Updates to the materialized view may be performed by execution resources assigned to the account that requested and / or generated the materialized view. If the materialized view is stored in storage resources assigned to a provider account, the materialized view may be updated by execution resources assigned to the provider account. If the materialized view is stored in storage resources assigned to a recipient account, the materialized view may be refreshed by execution resources assigned to the recipient account. The materialized view is scanned, and the system detects materialized view data sets [1(1 2 3)] and [2(4 5 6)], and the system detects that data set [1(1 2 3)] exists in the materialized view but does not exist in the source table. The system determines that the materialized view data set [1(1 2 3)] should be removed from the materialized view. The system deletes data set [1(1 2 3)] so that the materialized view data set [2(4 5 6)] is retained. The system scans the source table and finds that micro-partitions {7 8 9} exist in the source table, but information about these micro-partitions does not exist in the materialized view. The system updates the materialized view to include information about micro-partition {7 8 9}. The materialized view is now refreshed with respect to the source table and includes information about micro-partition {4 5 6 7 8 9}. The materialized view now includes two data sets, namely [2(4 5 6)] and [3(7 89)].

[0102] Fig.1313 shows an example micro-partition of a source table 1302 and an example materialized view 1304 generated based on the source table. The source table 1302 is linearly transformed to generate multiple micro-partitions (see partition No. 7, partition No. 12, and partition No. 35). Fig.13 In one embodiment shown, multiple micro-partitions can generate a single micro-partition of the materialized view 1304. A micro-partition is an immutable storage object in a database system. In an embodiment, the micro-partitions are represented as an explicit list of micro-partitions, and in some embodiments, this may be particularly expensive. In an alternative embodiment, the micro-partitions are represented as a series of micro-partitions. In a preferred embodiment, the micro-partitions are represented as a DML version that indicates the last refresh and the last compression of the source table 1302.

[0103] Example source table 1302 is labeled “source table number 243” to illustrate that any number of source tables may be used to generate materialized view 1304, that materialized view 1304 may index each of multiple source tables (see the “Table” column in materialized view 1304), and / or that any number of many materialized views may be generated for many possible source tables. Fig.13 As shown in the example embodiment in , the source table 1302 includes three micro-partitions. The three micro-partitions of the source table 1302 include partition No. 7, partition No. 12, and partition No. 35. Fig.13 Micro-partitions are shown indexed in materialized view 1304 under the "Partition" column. Fig.13 Further illustrated is that materialized view 1304 includes a single micro-partition based on the three micro-partitions of source table 1302. Additional micro-partitions may be added to and / or removed from materialized view 1304 as incremental updates are made to materialized view 1304.

[0104] like Fig.13 As shown in the example implementation in FIG. 1 , the materialized view 1304 for the source table includes four columns. The partition column indicates which micro-partition of the source table is applicable to the row of the materialized view 1304. Fig.13 As shown in the example in , the partition column has rows for partition 7, partition 12, and partition 35. In some embodiments, these partitions may refer to micro-partitions that constitute an immutable storage device that cannot be updated in place. Materialized view 1304 includes a table column that indicates an identifier of the source table. Fig.13 In the example shown, the identifier of the source table is the number 243. Materialized view 1304 includes a column "column 2" that corresponds to "column 2" in the source table. Fig.13In the example shown, column 2 of the source table may have an "M" entry or an "F" entry to indicate whether the person identified in that row is male or female. It should be understood that the data in the exemplary column 2 may include any suitable information. Common data entries that may be of interest include, for example, name, address, identification information, price information, statistics, demographic information, descriptor information, etc. It should be understood that there is no limitation on what data is stored in the column, and Fig.13 The male / female identifiers shown are for example purposes only. Materialized view 1304 further includes a total (SUM) column. The total column indicates how many entries in each partition have a "male" or "female" identifier in column 2. In the example embodiment, partition 7 includes 50,017 male identifiers and 37,565 female identifiers; partition 12 includes 43,090 male identifiers and 27,001 female identifiers; and partition 35 includes 34,234 male identifiers and 65,743 female identifiers. It should be understood that the total column in the example materialized view 1304 is for illustration purposes only. Materialized view 1304 does not necessarily provide the totals of the data of the source table, but can provide any desired measure. The calculations shown in materialized view 1304 will be determined on a case-by-case basis based on the needs of the party requesting the materialized view.

[0105] like Fig.13 As shown, the metadata between source table 1302 and materialized view 1304 is consistent. The metadata of materialized view 1304 is updated to reflect any updates to the metadata of source table 1302.

[0106] Fig.14 1400 is a block diagram depicting an example embodiment of a data processing platform 1400. Fig.14 As shown, the computing service manager 1402 communicates with the queue 1404, the client account 1408, the metadata 1406, and the execution platform 1416. In an embodiment, the computing service manager 1402 does not receive any direct communications from the client account 1408, but only receives job-related communications from the queue 1404. In particular embodiments, the computing service manager 1402 can support any number of client accounts 208, such as end users providing data storage and retrieval requests, system administrators managing the systems and methods described herein, and other components / devices that interact with the computing service manager 1402. As used herein, the computing service manager 1402 may also be referred to as a "global service system" that performs various functions discussed herein.

[0107] The computing service manager 1402 communicates with the queue 1404. The queue 1404 can provide jobs to the computing service manager 1402 in response to a triggering event. One or more jobs can be stored in the queue 1404 in a received order and / or priority order, and each of these one or more jobs can be transmitted to the computing service manager 1402 for scheduling and execution. The queue 1404 can determine the job to be executed based on a triggering event (such as the ingestion of data, the deletion of one or more rows in a table, the update of one or more rows in a table, the materialized view is out of date relative to its source table, the expression reaches a predefined clustering threshold indicating that the table should be re-clustered, etc.). In an embodiment, the queue 1404 includes an entry for refreshing the materialized view. The queue 1404 can include an entry for refreshing the materialized view generated from a local source table (i.e., local to the same account operating the computing service manager 1402) and / or an entry for refreshing the materialized view generated from a shared source table managed by a different account.

[0108] The computing service manager 1402 is also coupled to metadata 1406, which is associated with the complete data stored throughout the data processing platform 1400. In some embodiments, the metadata 1406 includes a summary of the data stored in the remote data storage system and the data available from the local cache. In addition, the metadata 1406 may include information about how the data is organized in the remote data storage system and the local cache. The metadata 1406 allows systems and services to determine whether a data segment needs to be accessed without having to load or access the actual data from the storage device.

[0109] In an embodiment, the computing service manager 1402 and / or the queue 1404 may determine that a job should be executed based on the metadata 1406. In such an embodiment, the computing service manager 1402 and / or the queue 1404 may scan the metadata 1406 and determine that a job should be executed to improve data organization or database performance. For example, the computing service manager 1402 and / or the queue 1404 may determine that a new version of a source table for a materialized view has been generated, and the materialized view has not yet been refreshed to reflect the new version of the source table. The metadata 1406 may include a transactional change tracking stream that indicates when a new version of the source table was generated and when the materialized view was last refreshed. Based on the metadata 1406 transaction stream, the computing service manager 1402 and / or the queue 1404 may determine that a job should be executed. In an embodiment, the computing service manager 1402 determines that a job should be executed based on a triggering event, and stores the job in the queue 1404 until the computing service manager 1402 is ready to schedule and manage the execution of the job.

[0110] The computing service manager 1402 may receive rules or parameters from the client account 1408, and such rules or parameters may guide the computing service manager 1402 to schedule and manage internal jobs. The client account 1408 may indicate that the internal job should only be executed at a specific time or should only utilize a set maximum amount of processing resources. The client account 1408 may further indicate one or more triggering events that should prompt the computing service manager 1402 to determine that the job should be executed. The client account 1408 may provide parameters about how many times a task can be re-executed and / or when the task should be re-executed.

[0111] The computing service manager 1402 is further coupled to an execution platform 1416, which provides a plurality of computing resources for performing various data storage and data retrieval tasks, as discussed in more detail below. The execution platform 1416 is coupled to a plurality of data storage devices 1412a, 1412b, and 1412n that are part of the storage platform 1410. Although Fig.14 1412a, 1412b, and 1412n are shown, but the execution platform 1416 is capable of communicating with any number of data storage devices. In some embodiments, the data storage devices 1412a, 1412b, and 1412n are cloud-based storage devices located in one or more geographic locations. For example, the data storage devices 1412a, 1412b, and 1412n can be part of a public cloud infrastructure or a private cloud infrastructure. The data storage devices 1412a, 1412b, and 1412n can be hard disk drives (HDDs), solid-state drives (SSDs), storage clusters, Amazon S3 TM Storage system or any other data storage technology. In addition, the storage platform 1410 may include a distributed file system (such as Hadoop Distributed File System (HDFS)), an object storage system, etc.

[0112] In a particular embodiment, the communication links between the computing service manager 1402, the queue 1404, the metadata 1406, the client account 1408 and the execution platform 1416 are implemented via one or more data communication networks. Similarly, the communication links between the execution platform 1416 and the data storage devices 1412a-1412n in the storage platform 1410 are implemented via one or more data communication networks. These data communication networks can utilize any communication protocol and any type of communication medium. In some embodiments, the data communication network is a combination of two or more data communication networks (or sub-networks) coupled to each other. In an alternative embodiment, these communication links are implemented using any type of communication medium and any communication protocol.

[0113] like Fig.14 As shown, data storage devices 1412a, 1412b, and 1412n are separated from the computing resources associated with execution platform 1416. The architecture supports dynamic changes to data processing platform 1400 based on changing data storage / retrieval needs and changing needs of users and systems accessing data processing platform 1400. Support for dynamic changes allows data processing platform 1400 to scale rapidly in response to changing needs for systems and components within data processing platform 1400. The separation of computing resources from data storage devices supports the storage of large amounts of data without requiring a corresponding large amount of computing resources. Similarly, this separation of resources supports a significant increase in computing resources used at a particular time without a corresponding increase in available data storage resources.

[0114] The computing service manager 1402, queue 1404, metadata 1406, client account 1408, execution platform 1416 and storage platform 1410 are in Fig.14 1404, metadata 1406, client accounts 1408, execution platform 1416, and storage platform 1410. However, each of computing service manager 1402, queue 1404, metadata 1406, client accounts 1408, execution platform 1416, and storage platform 1410 can be implemented as a distributed system (e.g., multiple systems / platforms distributed at multiple geographic locations). In addition, each of computing service manager 1402, metadata 1406, execution platform 1416, and storage platform 1410 can be expanded or reduced (independently of each other) based on changes to requests received from queue 1404 and / or client accounts 208 and changes in the demand for data processing platform 1400. Therefore, in the described embodiment, data processing platform 1400 is dynamic and supports regular changes to meet current data processing needs.

[0115] During typical operation, the data processing platform 1400 processes a plurality of jobs received from the queue 1404 or determined by the computing service manager 1402. These jobs are scheduled and managed by the computing service manager 1402 to determine when and how to execute the job. For example, the computing service manager 1402 may divide the job into a plurality of discrete tasks, and may determine what data is needed to execute each of the plurality of discrete tasks. The computing service manager 1402 may assign each of the plurality of discrete tasks to one or more nodes of the execution platform 1416 to process the task. The computing service manager 1402 may determine what data is needed to process the task, and further determine which nodes within the execution platform 1416 are best suited to process the task. Some nodes may have cached the data required to process the task, and are therefore good candidates for processing the task. The metadata 1406 assists the computing service manager 1402 in determining which nodes in the execution platform 1416 have cached at least a portion of the data required to process the task. One or more nodes in the execution platform 1416 use the data cached by the node and the data retrieved from the storage platform 1410 when necessary to process the task. It is desirable to retrieve as much data as possible from the cache within the execution platform 1416 because the retrieval speed is generally much faster than retrieving data from the storage platform 1410.

[0116] like Fig.14 As shown, the data processing platform 1400 separates the execution platform 1416 from the storage platform 1410. In this arrangement, the processing resources and cache resources in the execution platform 1416 operate independently of the data storage resources 1412a-1412n in the storage platform 1410. Therefore, the computing resources and cache resources are not limited to specific data storage resources 1412a-1412n. On the contrary, all computing resources and all cache resources can retrieve data from and store data in any data storage resource in the storage platform 1410. In addition, the data processing platform 1400 supports adding new computing resources and cache resources to the execution platform 1416 without making any changes to the storage platform 1410. Similarly, the data processing platform 1400 supports adding data storage resources to the storage platform 1410 without making any changes to the nodes in the execution platform 1416.

[0117] Fig.15 1402 is a block diagram depicting an embodiment of a computing service manager 1402. Fig.15As shown, the computing service manager 1402 includes an access manager 1502 and a key manager 1504 coupled to a data storage device 1506. The access manager 1502 handles authentication and authorization tasks for the system described herein. The key manager 1504 manages the storage and authentication of keys used during authentication and authorization tasks. For example, the access manager 1502 and the key manager 1504 manage keys for accessing data stored in remote storage devices (e.g., data storage devices in the storage platform 1410). As used herein, remote storage devices may also be referred to as "permanent storage devices" or "shared storage devices." The request processing service 1508 manages received data storage requests and data retrieval requests (e.g., jobs to be executed on database data). For example, the request processing service 1508 may determine the data required to process a received data storage request or data retrieval request. The necessary data may be stored in a cache within the execution platform 1416 (as discussed in more detail below), or may be stored in a data storage device in the storage platform 1410. The management console service 1510 supports administrators and other system administrators access to various systems and processes. Additionally, the management console service 1510 may receive requests to execute jobs and monitor the workload on the system.

[0118] The computing service manager 1402 also includes a job compiler 1512, a job optimizer 1514, and a job executor 1510. The job compiler 1512 parses the job into multiple discrete tasks and generates execution code for each of the multiple discrete tasks. The job optimizer 1514 determines the best method to execute the multiple discrete tasks based on the data that needs to be processed. The job optimizer 1514 also handles various data pruning operations and other data optimization techniques to improve the speed and efficiency of executing the job. The job executor 1516 executes the execution code of the job received from the queue 1404 or determined by the computing service manager 1402.

[0119] The job scheduler and coordinator 1518 sends the received jobs to the appropriate service or system for compilation, optimization and dispatch to the execution platform 1416. For example, the jobs can be prioritized and processed in this priority order. In an embodiment, the job scheduler and coordinator 1518 determines the priority of internal jobs scheduled by the computing service manager 1402 and other "external" jobs (such as user queries that can be scheduled by other systems in the database but can utilize the same processing resources in the execution platform 1416). In some embodiments, the job scheduler and coordinator 1518 identifies or assigns specific nodes in the execution platform 1416 to handle specific tasks. The virtual warehouse manager 1520 manages the operation of multiple virtual warehouses implemented in the execution platform 1416. As described below, each virtual warehouse includes multiple execution nodes, each of which includes a cache and a processor.

[0120] In addition, the computing service manager 1402 includes a configuration and metadata manager 1522 that manages information related to data stored in remote data storage devices and local caches (i.e., caches in the execution platform 1416). As discussed in more detail below, the configuration and metadata manager 1522 uses metadata to determine which data files need to be accessed to retrieve data for processing a particular task or job. The monitor and workload analyzer 1524 oversees the processes executed by the computing service manager 1402 and manages the distribution of tasks (e.g., workloads) across virtual warehouses and execution nodes in the execution platform 1416. The monitor and workload analyzer 1524 also reallocates tasks as needed based on the changing workload of the entire data processing platform 1400 and further reallocates tasks based on user (i.e., "external") query workloads that can also be processed by the execution platform 1416. The configuration and metadata manager 1522 and the monitor and workload analyzer 1524 are coupled to the data storage device 1526. Fig.15 Data storage devices 1506 and 1526 in represent any data storage devices within data processing platform 1400. For example, data storage devices 1506 and 1526 may represent caches in execution platform 1416, storage devices in storage platform 1410, or any other storage devices.

[0121] The computing service manager 1402 also includes a sharing component 210 as disclosed herein. The sharing component 210 is configured to provide cross-account access permissions, and may further be configured to generate and update cross-account materialized views.

[0122] Fig.16 is a block diagram depicting an embodiment of the execution platform 1416. Fig.16As shown, execution platform 1416 includes multiple virtual warehouses, including virtual warehouse 1, virtual warehouse 2 and virtual warehouse n. Each virtual warehouse includes multiple execution nodes, and each execution node includes a data cache and a processor. Virtual warehouses can perform multiple tasks in parallel using multiple execution nodes. As discussed herein, execution platform 1416 can add new virtual warehouses and discard existing virtual warehouses in real time based on the current processing needs of the system and users. This flexibility allows execution platform 1416 to quickly deploy a large amount of computing resources when needed, without having to continue to pay for those computing resources when they are no longer needed. All virtual warehouses can access data in any data storage device (e.g., any storage device in storage platform 1410).

[0123] although Fig.16 Each virtual warehouse shown in includes three execution nodes, but a particular virtual warehouse may include any number of execution nodes. In addition, the number of execution nodes in a virtual warehouse is dynamic, so that new execution nodes are created when there is additional demand, and existing execution nodes are deleted when they are no longer needed.

[0124] Each virtual warehouse has access to Fig.14 Thus, a virtual warehouse does not have to be assigned to a specific data storage device 1412a-1412n, but can access data from any data storage device 1412a-1412n within the storage platform 1410. Similarly, Fig.16 Each execution node shown in can access data from any data storage device 1412a-1412n. In some embodiments, a specific virtual warehouse or a specific execution node can be temporarily assigned to a specific data storage device, but the virtual warehouse or execution node can access data from any other data storage device later.

[0125] exist Fig.16In the example of , virtual warehouse 1 includes three execution nodes 1602a, 1602b and 1602n. Execution node 1602a includes cache 1604a and processor 1606a. Execution node 1602b includes cache 1604b and processor 1606b. Execution node 1602n includes cache 1604n and processor 1606n. Each execution node 1602a, 1602b and 1602n is associated with processing one or more data storage and / or data retrieval tasks. For example, a virtual warehouse can handle data storage and data retrieval tasks associated with internal services (such as clustering services, instantiated view refresh services, file compression services, storage program services, or file upgrade services). In other embodiments, a specific virtual warehouse can handle data storage and data retrieval tasks associated with a specific data storage system or a specific category of data.

[0126] Similar to the virtual warehouse 1 discussed above, virtual warehouse 2 includes three execution nodes 1612a, 1612b and 1612n. Execution node 1612a includes a cache 1614a and a processor 1616a. Execution node 1612n includes a cache 1614n and a processor 1616n. Execution node 1612n includes a cache 1614n and a processor 1616n. In addition, virtual warehouse 3 includes three execution nodes 1622a, 1622b and 1622n. Execution node 1622a includes a cache 1624a and a processor 1626a. Execution node 1622b includes a cache 1624b and a processor 1626b. Execution node 1622n includes a cache 1624n and a processor 1626n.

[0127] In some embodiments, with respect to data being cached by the execution node, Fig.16 The execution nodes shown are stateless. For example, these execution nodes do not store or otherwise maintain state information about the execution nodes or data cached by a particular execution node. Therefore, in the event of an execution node failure, the failed node can be transparently replaced with another node. Since there is no state information associated with the failed execution node, a new (replacement) execution node can easily replace the failed node without having to worry about recreating specific state.

[0128] although Fig.16 The execution nodes shown each include a data cache and a processor, but alternative embodiments may include execution nodes that include any number of processors and any number of caches. In addition, the size of the cache may vary between different execution nodes. Fig.16The cache shown in stores data retrieved from one or more data storage devices in storage platform 1410 in a local execution node. Therefore, the cache reduces or eliminates the bottleneck problem that occurs in a platform that continuously retrieves data from a remote storage system. Instead of repeatedly accessing data from a remote storage device, the systems and methods described herein access data from a cache in an execution node, which is significantly faster and avoids the bottleneck problem discussed above. In some embodiments, the cache is implemented using a high-speed memory device that provides fast access to cached data. Each cache can store data from any storage device in storage platform 1410.

[0129] In addition, cache resources and computing resources can vary between different execution nodes. For example, an execution node may include a large amount of computing resources and a minimum of cache resources, so that the execution node can be used for tasks that require a large amount of computing resources. Another execution node may include a large amount of cache resources and a minimum of computing resources, so that the execution node can be used for tasks that require a large amount of data to be cached. Yet another execution node may include cache resources that provide faster input-output operations, which is useful for tasks that require rapid scanning of large amounts of data. In some embodiments, based on the expected tasks that the execution node will perform, the cache resources and computing resources associated with a specific execution node are determined when the execution node is created.

[0130] In addition, the cache resources and computing resources associated with a particular execution node may change over time based on the changing tasks performed by the execution node. For example, if the tasks performed by the execution node become more processor intensive, more processing resources may be allocated to the execution node. Similarly, if the tasks performed by the execution node require a larger cache capacity, more cache resources may be allocated to the execution node.

[0131] Although virtual warehouses 1, 2, and n are associated with the same execution platform 1416, multiple computing systems at multiple geographic locations may be used to implement the virtual warehouses. For example, virtual warehouse 1 may be implemented by a computing system at a first geographic location, while virtual warehouses 2 and n are implemented by another computing system at a second geographic location. In some embodiments, these different computing systems are cloud-based computing systems maintained by one or more different entities.

[0132] In addition, each virtual warehouse Fig.161 is shown as having multiple execution nodes. Multiple execution nodes associated with each virtual warehouse can be implemented using multiple computing systems located at multiple geographic locations. For example, an instance of virtual warehouse 1 implements execution nodes 1602a and 1602b on a computing platform at one geographic location, and implements execution node 1602n at a different computing platform at another geographic location. The selection of a particular computing system to implement an execution node can depend on various factors, such as the resource level required for a particular execution node (e.g., processing resource requirements and cache requirements), the resources available at a particular computing system, the communication capabilities of a network within or between geographic locations, and which computing systems have implemented other execution nodes in the virtual warehouse.

[0133] The execution platform 1416 is also fault tolerant. For example, if a virtual warehouse fails, the virtual warehouse will be quickly replaced by a different virtual warehouse located in a different geographical location.

[0134] A particular execution platform 1416 may include any number of virtual warehouses. In addition, the number of virtual warehouses in a particular execution platform is dynamic, so that new virtual warehouses are created when additional processing and / or cache resources are needed. Similarly, existing virtual warehouses may be deleted when the resources associated with the virtual warehouses are no longer needed.

[0135] In some embodiments, virtual warehouses can operate on the same data in the storage platform 1410, but each virtual warehouse has its own execution node with independent processing and cache resources. This configuration allows requests on different virtual warehouses to be processed independently without interfering with each other. This independent processing, combined with the ability to dynamically add and remove virtual warehouses, supports adding new processing capabilities for new users without affecting the performance observed by existing users.

[0136] In an embodiment, different execution platforms 1416 are assigned to different accounts in the multi-tenant database 100. This can ensure that data stored in caches in different execution platforms 1416 is accessible only to associated accounts. The size of each different execution platform 1416 can be customized to accommodate the processing needs of each account in the multi-tenant database 100. In an embodiment, the provider account has its own execution platform 1416, and the recipient account has its own execution platform 1416. In an embodiment, the recipient account receives a shared object from the provider account, which enables the recipient account to generate a materialized view based on the data owned by the provider account. The execution platform 1416 of the recipient account can generate a materialized view. When the source table of the materialized view (i.e., the data owned by the provider account) is updated, the execution platform 1416 of the provider account will perform the update. If the recipient account generates a materialized view, the execution platform 1416 of the recipient account can be responsible for refreshing the materialized view relative to its source table.

[0137] Fig.17 1 is a block diagram depicting an example operating environment 1700 in which a queue 1404 communicates with multiple virtual warehouses under a virtual warehouse manager 1702. In environment 1700, the queue 1404 can access multiple database shared storage devices 1706a, 1706b, 1706c, 1706d, 1706e, and 1706n through multiple virtual warehouses 1704a, 1704b, and 1704n. Fig.17 1404 can access virtual warehouses 1704a, 1704b, and 1704n through the computing service manager 1402 (see Figure 1 ). In certain embodiments, the databases 1706a-1706n are contained in the storage platform 1410 and are accessible by any virtual warehouse implemented in the execution platform 1416. In some embodiments, the queue 1404 can access one of the virtual warehouses 1704a-1704n using a data communication network such as the Internet. In some implementations, a client account can specify that a queue 1404 (configured to store internal jobs to be completed) should interact with a specific virtual warehouse 1704a-1704n at a specific time.

[0138] In an embodiment (as shown), each virtual warehouse 1704a-1704n can communicate with all databases 1706a-1706n. In some embodiments, each virtual warehouse 1704a-1704n is configured to communicate with a subset of all databases 1706a-1706n. In such an arrangement, an individual client account associated with a data set can send all data retrieval and data storage requests through a single virtual warehouse and / or send it to a certain subset of databases 1706a-1706n. In addition, in the case where a certain virtual warehouse 1704a-1704n is configured to communicate with a specific subset of databases 1706a-1706n, the configuration is dynamic. For example, a virtual warehouse 1704a can be configured to communicate with a first subset of databases 1706a-1706n, and can be reconfigured to communicate with a second subset of databases 1706a-1706n later.

[0139] In an embodiment, the queue 1404 sends data retrieval, data storage and data processing requests to the virtual warehouse manager 1702, which routes the requests to the appropriate virtual warehouses 1704a-1704n. In some implementations, the virtual warehouse manager 1702 provides dynamic allocation of jobs to the virtual warehouses 1704a-1704n.

[0140] In some embodiments, the fault-tolerant system creates a new virtual warehouse in response to a failure of a virtual warehouse. The new virtual warehouse can be in the same virtual warehouse group, or can also be created in a different virtual warehouse group in a different geographical location.

[0141] The systems and methods described herein allow data to be stored and accessed as a service separate from computing (or processing) resources. Even if no computing resources have been allocated from the execution platform 1416, data can be used in the virtual warehouse without reloading the data from the remote data source. Therefore, the data is available independently of the allocation of computing resources associated with the data. The described system and method are useful for any type of data. In a specific embodiment, the data is stored in a structured optimized format. The separation of data storage / access services from computing services also simplifies data sharing between different users and groups. As described herein, each virtual warehouse can access any data to which it has access permissions, even if other virtual warehouses are accessing the same data at the same time. The architecture supports running queries without storing any actual data in the local cache. The systems and methods described herein are capable of transparent dynamic data movement, which moves data from remote storage devices to local caches as needed in a transparent manner to system users. In addition, since any virtual warehouse can access any data due to the separation of data storage services from computing services, the architecture supports data sharing without prior data movement.

[0142] Fig.18 1 is a schematic block diagram of a method 1800 for generating a materialized view across accounts in a multi-tenant database system. The method 1800 may be performed by any suitable computing device, such as the computing service manager 1402, the sharing component 210, and / or the materialized view component 914. The method 1800 may be implemented by processing resources allocated to the first account and / or the second account.

[0143] The method 1800 begins, and a computing resource defines a shared object in a first account at 1802. The shared object includes data associated with the first account. The method 1800 includes granting a cross-account access right to the shared object to a second account at 1804, so that the second account has access to the shared object without duplicating the shared object. The method 1800 includes generating a materialized view based on the shared object at 1806. The method 1800 includes updating the data associated with the first account at 1808. The method 1800 includes identifying whether the materialized view is out-of-date with respect to the shared object by merging the materialized view and the shared object at 1810.

[0144] Fig.19 1 is a schematic block diagram of a method 1900 for sharing a materialized view across accounts in a multi-tenant database system. The method 1900 may be performed by any suitable computing device, such as a computing service manager 1402, a sharing component 210, and / or a materialized view component 914. The method 1900 may be implemented by processing resources allocated to a first account and / or a second account.

[0145] Method 1900 begins, and at 1902, a computing resource defines a materialized view based on a source table associated with a first account of a multi-tenant database. Method 1900 continues, and at 1904, the computing resource defines cross-account access permissions to the materialized view for a second account, such that the second account can read the materialized view without replicating the materialized view. Method 1900 continues, and at 1906, the computing resource modifies a source table for the materialized view. Method 1900 continues, and at 1908, the computing resource identifies whether the materialized view is out-of-date with respect to the source table by merging the materialized view and the source table.

[0146] Fig. 20 is a block diagram depicting an example computing device 2000. In some embodiments, computing device 2000 is used to implement one or more systems and components discussed herein. In addition, computing device 2000 can interact with any of the systems and components described herein. Thus, computing device 2000 can be used to perform various programs and tasks, such as those discussed herein. Computing device 2000 can be used as a server, client, or any other computing entity. Computing device 2000 can be any of a variety of computing devices, such as a desktop computer, a notebook computer, a server computer, a handheld computer, a tablet computer, etc.

[0147] The computing device 2000 includes one or more processors 2002, one or more memory devices 2004, one or more interfaces 2006, one or more mass storage devices 2008, and one or more input / output (I / O) devices 2010, all coupled to a bus 2012. The one or more processors 2002 include one or more processors or controllers that execute instructions stored in the one or more memory devices 2004 and / or the one or more mass storage devices 2008. The one or more processors 2002 may also include various types of computer-readable media, such as cache memory.

[0148] One or more memory devices 2004 include various computer-readable media, such as volatile memory (e.g., random access memory (RAM)) and / or nonvolatile memory (e.g., read-only memory (ROM)). One or more memory devices 2004 may also include a rewritable ROM, such as a flash memory.

[0149] The one or more mass storage devices 2008 include various computer-readable media, such as tapes, disks, optical disks, solid-state memory (e.g., flash memory), etc. Various drives may also be included in the one or more mass storage devices 2008 to enable reading and / or writing of various computer-readable media. The one or more mass storage devices 2008 include removable media and / or non-removable media.

[0150] The one or more I / O devices 2010 include various devices that allow data and / or other information to be input to or retrieved from the computing device 2000. Example I / O devices 2010 include a cursor control device, a keyboard, a keypad, a microphone, a monitor or other display device, speakers, a printer, a network interface card, a modem, a lens, a CCD or other image capture device, and the like.

[0151] The one or more interfaces 2006 include various interfaces that allow the computing device 2000 to interact with other systems, devices, or computing environments. The one or more example interfaces 2006 include any number of different network interfaces, such as interfaces to a local area network (LAN), a wide area network (WAN), a wireless network, and the Internet.

[0152] The bus 2012 allows the one or more processors 2002, the one or more memory devices 2004, the one or more interfaces 2006, the one or more mass storage devices 2008, and the one or more I / O devices 2010 to communicate with each other and with other devices or components coupled to the bus 2012. The bus 2012 represents one or more of several types of bus structures, such as a system bus, a PCI bus, an IEEE 1394 bus, a USB bus, and the like.

[0153] For purposes of illustration, programs and other executable program components are shown herein as discrete blocks, although it should be understood that such programs and components may reside in different storage components of the computing device 2000 at different times and be executed by one or more processors 2002. Alternatively, the systems and processes described herein may be implemented in hardware or a combination of hardware, software, and / or firmware. For example, one or more application specific integrated circuits (ASICs) may be programmed to perform one or more of the systems and processes described herein. As used herein, the term "module" or "component" is intended to convey an implementation device for performing a process such as by hardware or a combination of hardware, software, and / or firmware, so as to perform all or part of the operations disclosed herein.

[0154] Example

[0155] The following examples relate to other embodiments.

[0156] Example 1 is a system for cross-account data sharing in a multi-tenant database. The system includes a device for defining a shared object in a first account, the shared object including data associated with the first account. The system includes a device for granting a second account cross-account access rights to the shared object, so that the second account has access to the shared object without copying the shared object. The system includes a device for generating a materialized view based on the shared object. The system includes a device for updating data associated with the first account. The system includes a device for identifying whether the materialized view is out of date relative to the shared object by merging the materialized view and the shared object.

[0157] Example 2 is a system according to Example 1, wherein the device for identifying whether a materialized view is out-of-date relative to a shared object includes: a device for merging the materialized view and the shared object; a device for identifying whether data in the shared object has been modified since the last refresh of the materialized view, wherein the data in the shared object can be modified by one or more of updating, deleting, or inserting; and a device for refreshing the materialized view relative to the shared object in response to identifying modifications to the shared object since the last refresh of the materialized view.

[0158] Example 3 is a system according to any one of Examples 1-2, further including a device for querying shared objects, the device for querying comprising: a device for merging materialized views and shared objects; and a device for executing queries based on information in the materialized view and any modifications made to the shared objects since the last refresh of the materialized view.

[0159] Example 4 is a system according to any one of Examples 1-3, further including: a device for storing data associated with a first account in one or more shared storage devices among a plurality of shared storage devices; a device for defining an execution platform associated with the first account, the first account having read access and write access to the data associated with the first account; and a device for defining an execution platform associated with a second account, the second account having read access to a shared object.

[0160] Example 5 is a system according to any one of Examples 1-4, wherein: a device for generating an instantiated view based on a shared object is incorporated into an execution platform associated with a second account; a device for updating data associated with a first account is incorporated into an execution platform associated with the first account; and a device for identifying whether an instantiated view is outdated relative to a shared object is incorporated into an execution platform associated with the second account.

[0161] Example 6 is a system according to any one of Examples 1-5, further comprising a device for defining a security view definition for a materialized view, the device for defining the security view definition comprising one or more of the following devices: a device for granting a second account read access and write access to the materialized view; a device for granting a first account read access to the materialized view; or a device for hiding the materialized view from the first account so that the first account cannot view whether the materialized view has been generated.

[0162] Example 7 is a system according to any one of Examples 1-6, further comprising a device for defining viewing privileges for cross-account access rights to a shared object, so that the basic details of the shared object include a security view definition, wherein the basic details of the shared object include one or more of the following: a data field in the shared object; a column of data in the shared object; a structural element of a base table of the shared object; or a large amount of data in the shared object.

[0163] Example 8 is a system according to any of Examples 1-7, wherein the means for defining view privileges for cross-account access rights to a shared object includes means for hiding the view privileges from the second account and means for making the view privileges visible to the first account.

[0164] Example 9 is a system according to any of Examples 1-8, wherein the means for defining a shared object includes one or more of: a means for defining an object name unique to a first account; a means for defining an object role; or a means for generating a reference list comprising a list of one or more accounts eligible to receive cross-account access to the shared object.

[0165] Example 10 is a system according to any one of Examples 1-9, further including: a device for receiving a request from a second account to generate an instantiated view based on specific data associated with the first account; a device for identifying whether the specific data is included in the shared object; a device for granting the second account authorization to generate an instantiated view based on the specific data; and a device for providing a notification to the first account indicating that the second account has received authorization to generate an instantiated view based on the specific data.

[0166] Example 11 is a method for cross-account data sharing in a multi-tenant database. The method includes defining a shared object in a first account, the shared object including data associated with the first account. The method includes granting cross-account access rights to the shared object to a second account, so that the second account has access to the shared object without copying the shared object. The method includes generating a materialized view based on the shared object. The method includes updating the data associated with the first account. The method includes identifying whether the materialized view is out of date relative to the shared object by merging the materialized view and the shared object.

[0167] Example 12 is a method according to Example 11, wherein identifying whether the materialized view is out-of-date relative to the shared object includes: merging the materialized view and the shared object; identifying whether data in the shared object has been modified since the last refresh of the materialized view, wherein the data in the shared object can be modified by one or more of updating, deleting, or inserting; and refreshing the materialized view relative to the shared object in response to identifying the modification to the shared object since the last refresh of the materialized view.

[0168] Example 13 is a method according to any of Examples 11-12, further comprising querying the shared object by: merging the materialized view and the shared object; and executing the query based on information in the materialized view and any modifications made to the shared object since the last refresh of the materialized view.

[0169] Example 14 is a method according to any one of Examples 11-13, further comprising: storing data associated with a first account in one or more shared storage devices among a plurality of shared storage devices; defining an execution platform associated with the first account, the first account having read access and write access to the data associated with the first account; and defining an execution platform associated with a second account, the second account having read access to a shared object.

[0170] Example 15 is a method according to any one of Examples 11-14, wherein: generating an instantiated view based on a shared object is processed by an execution platform associated with the second account; updating data associated with the first account is processed by an execution platform associated with the first account; and identifying whether the instantiated view is out of date relative to the shared object is processed by an execution platform associated with the second account.

[0171] Example 16 is a processor that can be configured to execute instructions stored in a non-transitory computer-readable storage medium, the instructions including: defining a shared object in a first account, the shared object including data associated with the first account; granting cross-account access rights to the shared object to a second account so that the second account has access to the shared object without copying the shared object; generating a materialized view based on the shared object; updating the data associated with the first account; and identifying whether the materialized view is out of date relative to the shared object by merging the materialized view and the shared object.

[0172] Example 17 is a processor according to Example 16, wherein identifying whether the materialized view is out-of-date relative to the shared object includes: merging the materialized view and the shared object; identifying whether data in the shared object has been modified since the last refresh of the materialized view, wherein the data in the shared object can be modified by one or more of updating, deleting, or inserting; and refreshing the materialized view relative to the shared object in response to identifying the modification to the shared object since the last refresh of the materialized view.

[0173] Example 18 is a processor according to any of Examples 16-17, wherein the instructions also include querying the shared object by: merging the materialized view and the shared object; and executing the query based on information in the materialized view and any modifications made to the shared object since the last refresh of the materialized view.

[0174] Example 19 is a processor according to any one of Examples 16-18, wherein the instructions also include defining a security view definition for the materialized view in one or more of the following ways: granting the second account read access and write access to the materialized view; granting the first account read access to the materialized view; or hiding the materialized view from the first account so that the first account cannot view whether the materialized view is generated.

[0175] Example 20 is a processor according to any of Examples 16-19, wherein the instructions also include defining view privileges for cross-account access permissions to a shared object, so that the underlying details of the shared object include a security view definition, wherein the underlying details of the shared object include one or more of the following: a data field in the shared object; a column of data in the shared object; a structural element of a base table of the shared object; or a large amount of data in the shared object.

[0176] Example 21 is a system for sharing a materialized view across accounts in a multi-tenant database. The system includes means for defining a materialized view based on a source table, the source table being associated with a first account of the multi-tenant database. The system includes means for defining cross-account access rights to the materialized view for a second account, such that the second account has the right to read the materialized view. The system includes means for modifying a source table of the materialized view. The system includes means for identifying whether the materialized view is out-of-date relative to the source table by merging the materialized view and the source table.

[0177] Example 22 is a system according to Example 21, wherein the device for defining cross-account access permissions to a materialized view also includes a device for defining cross-account access permissions such that a second account does not have permission to read a source table of the materialized view or does not have permission to write to a source table of the materialized view.

[0178] Example 23 is a system according to any one of Examples 21-22, wherein the device for identifying whether the materialized view is out-of-date relative to the source table includes: a device for merging the materialized view and the source table; and a device for identifying whether data in the source table has been modified since the last refresh of the materialized view, wherein the system also includes a device for refreshing the materialized view relative to the source table in response to identifying modifications to the source table since the last refresh of the materialized view.

[0179] Example 24 is a system according to any one of Examples 21-23, further including: a device for storing a source table associated with a first account in one or more storage devices among a plurality of storage devices shared across a multi-tenant database; a device for defining an execution platform associated with the first account, the first account having read access and write access to the source table of the materialized view; and a device for defining an execution platform associated with a second account having read access to the materialized view.

[0180] Example 25 is a system according to any one of Examples 21-24, wherein: the execution platform associated with the first account includes a device for defining a materialized view; and the execution platform associated with the first account includes a device for modifying a source table of the materialized view.

[0181] Example 26 is a system according to any one of Examples 21-25, further comprising a device for defining view privileges for cross-account access rights to a materialized view, such that the underlying details of a source table of the materialized view include a security view definition, wherein the underlying details of the source table include one or more of the following: a data field in the source table; a column of data in the source table; a structural element of the source table; a large amount of data in the source table; metadata of the source table; or a transaction log of modifications made to the source table.

[0182] Example 27 is a system according to any of Examples 21-26, wherein the means for defining view privileges for cross-account access rights to a materialized view includes means for hiding the view privileges from a second account and means for making the view privileges visible to a first account.

[0183] Example 28 is a system according to any of Examples 21-27, further comprising means for defining a reference list comprising a list of one or more accounts eligible to receive cross-account access to the materialized view.

[0184] Example 29 is a system according to any one of Examples 21-28, further including: a device for receiving a request from a second account to generate a materialized view based on specific data stored in a source table; a device for providing the request to the first account for approval or rejection; and a device for providing a notification to the second account indicating whether the request is approved or rejected by the first account.

[0185] Example 30 is a system according to any one of Examples 21-29, further comprising: means for providing a notification to the second account indicating that the materialized view is out-of-date with respect to the source table in response to identifying that the materialized view is out-of-date with respect to the source table.

[0186] Example 31 is a method for sharing a materialized view across accounts in a multi-tenant database. The method includes defining a materialized view based on a source table, the source table being associated with a first account of the multi-tenant database. The method includes defining cross-account access permissions to the materialized view for a second account, such that the second account has permission to read the materialized view. The method includes modifying a source table of the materialized view. The method includes identifying whether the materialized view is out of date relative to the source table by merging the materialized view and the source table.

[0187] Example 32 is a method according to Example 31, wherein defining cross-account access permissions to the materialized view also includes defining cross-account access permissions such that the second account does not have permission to read the source table of the materialized view or does not have permission to write to the source table of the materialized view.

[0188] Example 33 is a method according to any one of Examples 31-32, wherein identifying whether a materialized view is out-of-date relative to a source table includes: merging the materialized view and the source table; and identifying whether data in the source table has been modified since the last refresh of the materialized view; wherein the method also includes refreshing the materialized view relative to the source table in response to identifying modifications to the source table since the last refresh of the materialized view.

[0189] Example 34 is a method according to any one of Examples 31-33, further comprising defining view privileges for cross-account access permissions to a materialized view, such that the underlying details of a source table of the materialized view include a security view definition, wherein the underlying details of the source table include one or more of the following: a data field in the source table; a column of data in the source table; a structural element of the source table; a large amount of data in the source table; metadata of the source table; or a transaction log of modifications made to the source table.

[0190] Example 35 is a method according to any of Examples 31-34, further comprising defining a reference list comprising a list of one or more accounts eligible to receive cross-account access to the materialized view.

[0191] Example 36 is a processor configurable to execute instructions stored in a non-transitory computer-readable storage medium. The instructions include defining a materialized view based on a source table, the source table being associated with a first account of a multi-tenant database. The instructions include defining cross-account access rights to the materialized view for a second account, such that the second account has permission to read the materialized view. The instructions include modifying a source table of the materialized view. The instructions include identifying whether the materialized view is out-of-date relative to the source table by merging the materialized view and the source table.

[0192] Example 37 is a processor according to Example 36, wherein defining cross-account access permissions to the materialized view also includes defining cross-account access permissions such that the second account does not have permission to read the source table of the materialized view or does not have permission to write to the source table of the materialized view.

[0193] Example 38 is a processor according to any one of Examples 36-37, wherein identifying whether a materialized view is out of date relative to a source table includes: merging the materialized view and the source table; and identifying whether data in the source table has been modified since the last refresh of the materialized view, wherein the processor also includes refreshing the materialized view relative to the source table in response to identifying modifications to the source table since the last refresh of the materialized view.

[0194] Example 39 is a processor according to any one of Examples 36-38, wherein the instruction also includes defining view privileges for cross-account access permissions to a materialized view, so that the underlying details of a source table of the materialized view include a security view definition, wherein the underlying details of the source table include one or more of the following: a data field in the source table; a column of data in the source table; a structural element of the source table; a large amount of data in the source table; metadata of the source table; or a transaction log of modifications made to the source table.

[0195] Example 40 is a processor according to any of Examples 36-39, wherein the instructions further comprise defining a reference list comprising a list of one or more accounts eligible to receive cross-account access to the materialized view.

[0196] Example 41 is a device comprising a device for performing a method according to any of Examples 1-40 or implementing a device or system of any of the above.

[0197] Example 42 is a machine-readable storage device comprising machine-readable instructions, which, when executed, are used to implement the method according to any one of Examples 1-40 or the apparatus for implementing any of the above.

[0198] Various technologies or some aspects or parts thereof can be in the form of program code (i.e., instructions) embodied in tangible media (such as floppy disks, CD-ROMs, hard disk drives, non-transitory computer-readable storage media, or any other machine-readable storage media), wherein when the program code is loaded into and executed by a machine such as a computer, the machine becomes a device for implementing various technologies. In the case of executing program code on a programmable computer, a computing device may include a processor, a storage medium (including volatile and non-volatile memory and / or storage element) readable by the processor, at least one input device, and at least one output device. Volatile and non-volatile memory and / or storage element may be RAM, EPROM, flash memory, optical disk drive, magnetic hard disk drive, or another medium for storing electronic data. One or more programs that can implement or utilize various technologies described herein may use application programming interfaces (APIs), reusable controls, etc. These programs may be implemented in high-level programs or object-oriented programming languages ​​to communicate with a computer system. However, if necessary, one or more programs may be implemented in assembly language or machine language. In any case, the language may be a compiled or interpreted language and combined with a hardware implementation.

[0199] It should be understood that many of the functional units described in this specification can be implemented as one or more components, which is a term used to more specifically emphasize its implementation independence. For example, a component can be implemented as a hardware circuit including a customized very large scale integration (VLSI) circuit or gate array, such as an off-the-shelf semiconductor of a logic chip, transistor or other discrete components. A component can also be implemented in a programmable hardware device (such as a field programmable gate array, programmable array logic, a programmable logic device, etc.).

[0200] Components may also be implemented in software so as to be executed by various types of processors. The identified components of executable code may, for example, include one or more physical or logical blocks of computer instructions, which may, for example, be organized as objects, programs, or functions. However, the executable files of the identified components need not be physically located together, but may include different instructions stored in different locations which, when logically connected together, comprise the component and achieve the stated purpose for the component.

[0201] In practice, the components of executable code may be a single instruction or many instructions, and may even be distributed over several different code segments, in different programs, and on several storage devices. Similarly, operational data may be identified and described herein within the components, and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations including different storage devices, and may exist at least in part only as electronic signals on a system or network. Components may be passive or active, including agents operable to perform desired functions.

[0202] References throughout this specification to "an example" mean that a particular feature, structure, or characteristic described in conjunction with the example is included in at least one embodiment of the present disclosure. Therefore, the phrase "in an example" appearing in various places throughout this specification does not necessarily refer to the same embodiment.

[0203] As used herein, for convenience, multiple items, structural elements, constituent elements, and / or materials may be presented in a common list. However, these lists should be interpreted as if each member in the list were individually identified as an independent and unique member. Therefore, based solely on its representation in a common group without an indication to the contrary, any individual member in such a list should not be interpreted as being in fact equivalent to any other member in the same list. In addition, various embodiments and examples of the present disclosure and alternatives to its various components may be referenced herein. It should be understood that these embodiments, examples, and alternatives should not be interpreted as de facto equivalents of each other, but should be regarded as separate and autonomous representations of the present disclosure.

[0204] Although the foregoing has been described in detail for the sake of clarity, it will be apparent that certain changes and modifications may be made without departing from its principles. It should be noted that there are many alternative ways to implement the processes and devices described herein. Therefore, the present embodiment should be considered illustrative rather than restrictive.

[0205] Those skilled in the art will appreciate that many changes may be made to the details of the above embodiments without departing from the basic principles of the present disclosure. Therefore, the scope of the present disclosure should be determined solely by the appended claims.

Claims

1. A system for sharing materialized views across accounts in a multi-tenant database, the system comprising: Means for defining a materialized view according to a source table, the source table being associated with a first account of the multi-tenant database, wherein the source table comprises a plurality of micro-partitions, the plurality of micro-partitions being immutable storage objects; means for defining cross-account access rights to the materialized view for a second account so that the second account can read the materialized view; means for modifying, by the first account, the source table of the materialized view, wherein the modification comprises deleting at least one micro-partition and inserting at least one new micro-partition based on execution of a transaction; means for identifying, by the second account, whether the materialized view is out-of-date with respect to the source table by merging the materialized view and the source table; and The device for updating the materialized view comprises: after the materialized view and the source table are merged, scanning the source table to detect the insertion of the at least one new micro-partition, scanning the materialized view to detect the deletion of the at least one micro-partition, and updating the materialized view based on the detection of the insertion of the at least one new micro-partition and the detection of the deletion of the at least one micro-partition.

2. The system according to claim 1, wherein: The means for defining cross-account access permissions to the materialized view also includes means for defining the cross-account access permissions so that the second account has no permission to read the source table of the materialized view or has no permission to write to the source table of the materialized view.

3. The system according to claim 1, wherein: The means for identifying whether the materialized view is outdated relative to the source table comprises: means for identifying whether data in the source table has been modified since the last refresh of the materialized view; The system further includes means for refreshing the materialized view relative to the source table in response to identifying a modification to the source table since the last refresh of the materialized view.

4. The system according to claim 1, further comprising: means for storing a source table associated with the first account on one or more storage devices of a plurality of storage devices shared across the multi-tenant database; means for defining an execution platform associated with the first account, the first account having read access and write access to the source table of the materialized view; and Means for defining an execution platform associated with the second account, the second account having read access to the materialized view.

5. The system of claim 4, wherein: an execution platform associated with said first account including means for defining said materialized view; and An execution platform associated with the first account includes means for modifying the source table of the materialized view.

6. The system according to claim 1, further comprising means for defining a view privilege for cross-account access to the materialized view so that the base details of the source table of the materialized view include a security view definition, wherein the base details of the source table include one or more of the following: The data fields in the source table; A column of data in the source table; structural elements of the source table; The large amount of data in the source table; metadata of the source table; or A transaction log of modifications made to the source table.

7. The system according to claim 6, wherein: The means for defining view privileges for cross-account access to the materialized view includes means for hiding the view privileges from the second account and means for making the view privileges visible to the first account.

8. The system of claim 1, further comprising means for defining a reference list comprising a list of one or more accounts eligible to receive cross-account access to the materialized view.

9. The system of claim 1, further comprising: means for receiving a request from the second account to generate the materialized view based on specific data stored in the source table; means for providing the request to the first account for approval or denial; as well as Means for providing a notification to the second account indicating whether the request was approved or denied by the first account.

10. The system of claim 1, further comprising means for indicating, by the second account in response to identifying that the materialized view is out-of-date with respect to the source table, that the materialized view is out-of-date with respect to the source table.

11. A method for sharing a materialized view across accounts in a multi-tenant database, the method comprising: A materialized view is defined according to a source table, the source table being associated with a first account of the multi-tenant database, wherein: The source table includes a plurality of micro-partitions, and the plurality of micro-partitions are immutable storage objects; defining a cross-account access permission to the materialized view for the second account, so that the second account can read the materialized view; Modifying the source table of the materialized view by the first account, wherein the modification includes deleting at least one micro-partition and inserting at least one new micro-partition based on the execution of a transaction; identifying, by the second account, whether the materialized view is out-of-date with respect to the source table by merging the materialized view and the source table; and The materialized view is updated, wherein, after the materialized view and the source table are merged, the source table is scanned to detect the insertion of the at least one new micro-partition, the materialized view is scanned to detect the deletion of the at least one micro-partition, and the materialized view is updated based on the detection of the insertion of the at least one new micro-partition and the detection of the deletion of the at least one micro-partition.

12. The method according to claim 11, wherein: Defining cross-account access permissions to the materialized view also includes defining the cross-account access permissions so that the second account does not have permission to read the source table of the materialized view or does not have permission to write to the source table of the materialized view.

13. The method according to claim 11, wherein: Identifying whether the materialized view is out-of-date with respect to the source table includes: identifying whether data in the source table has been modified since the last refresh of the materialized view; The method further includes refreshing the materialized view relative to the source table in response to identifying a modification to the source table since a last refresh of the materialized view.

14. The method of claim 11, further comprising defining a view privilege for cross-account access to the materialized view such that the underlying details of the source table of the materialized view include a security view definition, wherein the underlying details of the source table include one or more of the following: The data fields in the source table; A column of data in the source table; structural elements of the source table; The large amount of data in the source table; metadata of the source table; or A transaction log of modifications made to the source table. 15 . The method of claim 11 , further comprising defining a reference list comprising a list of one or more accounts eligible to receive cross-account access to the materialized view.

16. A processor configurable to execute instructions stored in a computer-readable storage medium, the instructions comprising: A materialized view is defined based on a source table, the source table being associated with a first account of a multi-tenant database, wherein: The source table includes a plurality of micro-partitions, and the plurality of micro-partitions are immutable storage objects; defining a cross-account access permission to the materialized view for the second account, so that the second account can read the materialized view; Modifying the source table of the materialized view by the first account, wherein the modification includes deleting at least one micro-partition and inserting at least one new micro-partition based on the execution of a transaction; identifying, by the second account, whether the materialized view is out-of-date with respect to the source table by merging the materialized view and the source table; and The materialized view is updated, wherein, after the materialized view and the source table are merged, the source table is scanned to detect the insertion of the at least one new micro-partition, the materialized view is scanned to detect the deletion of the at least one micro-partition, and the materialized view is updated based on the detection of the insertion of the at least one new micro-partition and the detection of the deletion of the at least one micro-partition.

17. The processor of claim 16, wherein: Defining cross-account access permissions to the materialized view also includes defining the cross-account access permissions so that the second account does not have permission to read the source table of the materialized view or does not have permission to write to the source table of the materialized view.

18. The processor of claim 16, wherein: Identifying whether the materialized view is out-of-date with respect to the source table includes: identifying whether data in the source table has been modified since the last refresh of the materialized view; The instructions further include refreshing the materialized view relative to the source table in response to identifying a modification to the source table since a last refresh of the materialized view.

19. The processor of claim 16, wherein the instructions further comprise a view privilege defining cross-account access rights to the materialized view such that the underlying details of the source table of the materialized view comprise a secure view definition, wherein the underlying details of the source table comprise one or more of: The data fields in the source table; A column of data in the source table; structural elements of the source table; The large amount of data in the source table; metadata of the source table; or A transaction log of modifications made to the source table.

20. The processor of claim 16, wherein the instructions further comprise defining a reference list comprising a list of one or more accounts eligible to receive cross-account access to the materialized view.

Images