Method and system for implementing sub-table replication and computer readable storage medium

Abstract

Methods and systems for implementing sub-table replication and computer-readable storage media are provided. A method for implementing sub-table replication includes a processor detecting an update to an equity table, the equity table including a plurality of equity table rows associated with a plurality of client identifiers, a plurality of consumer account identifiers, and a plurality of provider account identifiers, the update to the equity table being associated with a first equity table row of the equity table rows, the update including a first client identifier of the client identifiers, a first consumer account identifier of the consumer account identifiers, or a first provider account identifier of the provider account identifiers, performing a filter on a data table based on the update to the equity table, the data table including a plurality of data table rows associated with the client identifiers and including an equity column and data content, the equity column including the provider account identifiers, detecting an update to the equity column, and performing incremental replication of the data table by causing version-based replication to be performed.

Description

[0001] This application is a divisional application of the application for patent with the application number 202110467165.X, the title of which is “Method and system for implementing sub-table replication and computer readable storage medium”, the filing date of which is April 28, 2021.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims priority to U.S. Provisional Patent Application Serial No. 63 / 032,163, filed May 29, 2020, the contents of which are incorporated by reference in their entirety. TECHNICAL FIELD

[0004] The present disclosure relates to a system, method, and apparatus for a database, and more particularly to a sub-database replication. BACKGROUND

[0005] Databases are widely used for data storage and access in computing applications. A database can include tables that include or reference data with rows and columns, which can be read, modified, or deleted using queries.

[0006] In some cases, it can be beneficial to replicate database data in multiple locations or on multiple storage devices. Replicating data can prevent system failures that can render data inaccessible over a cloud network or can result in data loss or permanent unreadability. However, data replication across networks that include different regions can incur latency and cost. SUMMARY

[0007] According to an aspect of the present disclosure, there is provided a method for implementing sub-table replication, comprising: detecting, by a processor, a first update to an equity table, the equity table comprising a plurality of equity table rows, the equity table rows being associated with a plurality of client identifiers, a plurality of consumer account identifiers, and a plurality of provider account identifiers, the first update being associated with a first equity table row of the equity table rows, the first update comprising: a first client identifier of the client identifiers, a first consumer account identifier of the consumer account identifiers; or a first provider account identifier of the provider account identifiers; performing a filtering on a data table based on the first update, the data table comprising a plurality of data table rows, the plurality of data table rows being associated with the client identifiers and comprising data content; detecting a second update to the equity table; and performing an incremental replication of the data table by causing a full inventory replication for a plurality of provider accounts associated with the provider account identifiers in the equity table at a next flush.

[0008] According to another aspect of the present application, there is also provided a system for implementing sub-table replication, comprising: a processor; and a memory having instructions stored therein that, when executed by the processor, cause the system to perform operations comprising: detecting a first update to an equity table, the equity table comprising a plurality of equity table rows, the equity table rows being associated with a plurality of client identifiers, a plurality of consumer account identifiers, and a plurality of provider account identifiers, the first update being associated with a first equity table row of the equity table rows, the first update comprising: a first client identifier of the client identifiers, a first consumer account identifier of the consumer account identifiers; or a first provider account identifier of the provider account identifiers; performing a filter on a data table based on the first update, the data table comprising a plurality of data table rows, the plurality of data table rows being associated with the client identifiers and comprising data content; detecting a second update to the equity table; and performing an incremental replication of the data table by causing a full inventory replication to be performed at a next flush for a plurality of provider accounts associated with the provider account identifiers in the equity table.

[0009] According to yet another aspect of the present application, there is provided a non-transitory computer-readable storage medium comprising instructions that, when processed by a computer, configure the computer to perform operations comprising: detecting a first update to an equity table, the equity table comprising a plurality of equity table rows, the equity table rows being associated with a plurality of client identifiers, a plurality of consumer account identifiers, and a plurality of provider account identifiers, the first update being associated with a first equity table row of the equity table rows, the first update comprising: a first client identifier of the client identifiers, a first consumer account identifier of the consumer account identifiers; or a first provider account identifier of the provider account identifiers; performing a filter on a data table based on the first update, the data table comprising a plurality of data table rows, the plurality of data table rows being associated with the client identifiers and comprising data content; detecting a second update to the equity table; and performing an incremental replication of the data table by causing a full inventory replication to be performed at a next flush for a plurality of provider accounts associated with the provider account identifiers in the equity table. BRIEF DESCRIPTION OF DRAWINGS

[0010] In the drawings, which are not necessarily drawn to scale, like numerals can describe similar components in different views. To easily identify the discussion of any particular element or act, the most significant digit or digits in any

[0011] Figure 1 It is a graphical representation of some examples of networked environments in which the content of this disclosure can be deployed.

[0012] Figure 2 The diagram illustrates a graphical representation of a replication controller using schema filtering and table filtering to implement sub-database replication, based on some examples.

[0013] Figure 3 It is a graphical representation of how replication controllers, based on some examples, enable data sharing from a database to clients within the same region.

[0014] Figure 4 A graphical representation of a table replication controller implementing table replication is shown, based on some examples.

[0015] Figure 5 A graphical representation of a replication controller implementing subtable replication is shown, based on some examples.

[0016] Figure 6 The diagram illustrates a graphical representation of a replication controller using the rights of a runtime application to implement subtable replication, based on some examples.

[0017] Figure 7 The diagram illustrates a graphical representation of a replication controller using the rights of a runtime application to implement subtable replication, based on some examples.

[0018] Figure 8 The process 800 for implementing sub-database replication according to one embodiment is shown.

[0019] Figure 9 An example from one embodiment is shown. Figure 8 Details of operation 804.

[0020] Figure 10 The diagram illustrates a graphical representation of a replication controller using equity columns within a table to implement subtable replication, based on some examples.

[0021] Figure 11 The diagram illustrates a graphical representation of a replication controller using equity columns within a table to implement subtable replication, based on some examples.

[0022] Figure 12 The diagram illustrates a graphical representation of a replication controller using equity columns within a table to implement subtable replication, based on some examples.

[0023] Figure 13 The process 1300 for implementing sub-database replication according to one embodiment is shown.

[0024] Figure 14 An example from one embodiment is shown. Figure 13 Details of operation 1308.

[0025] Figure 15 is a diagrammatic representation of a machine in the form of a computer system, within which a set of instructions can be executed to cause the machine to perform any one or more of the methodologies discussed herein, according to some example embodiments.

[0026] Figure 16 is a block diagram illustrating a software architecture, within which the present disclosure can be implemented, according to an example embodiment. DETAILED DESCRIPTION

[0027] Databases can store vast amounts of data in an organized manner across a networked environment for providers and clients. For example, a provider can store data for multiple clients in the same database. When the clients are in different geographic regions, it can be necessary to replicate the database to provide the clients with access to the data. Replicating an entire database across a networked environment can be expensive and can cause significant latency due to the size of the database. Therefore, it is desirable to replicate relevant portions of the database to different regions.

[0028] Among other things, the embodiments described in this disclosure improve the functionality of the database service system 100 by implementing the replication of sub-databases to different target regions in a network. Using sub-database replication, the database service system 100 reduces replication costs and replication latency by filtering non-critical objects.

[0029] Figure 1 is a diagrammatic representation of a networked environment in which the present disclosure can be deployed, according to some examples. Figure 1 The database service system 100 in is a processing platform that provides database services. In one embodiment, the database service system 100 can implement database replication or sub-database replication. Sub-database replication includes, for example, schema replication, table replication, sub-table replication, and the like.

[0030] The database service system 100 includes a database service manager 108 that is accessible by a plurality of users via a network 110 (e.g., the Internet). The users can access the database service manager 108 using a client device 102, a client device 106, and a client device 104, respectively. The database service manager 108 can support any number of users that desire to access data or services of the database service system 100. The users of the client device 102, the client device 104, the client device 106 can include, for example, end users that provide data storage as well as retrieval queries and requests, system administrators that manage the systems and methods described herein, software applications that interact with the database, and other components / devices that interact with the database service manager 108.

[0031] The database service manager 108 can provide various services and functionality that support the operation of systems and components within the database service system 100. The database service manager 108 accesses stored metadata associated with data stored throughout the database service system 100. In some implementations, the metadata includes a summary of data stored in remote data storage systems (e.g., database 112, database 116, database 114, etc.) as well as data available from local caches. Additionally, the metadata can include information about how data is organized in the remote data storage systems and local caches.

[0032] The database service manager 108 also communicates with multiple data storage devices including database 112, database 116, and database 114 to perform various data storage and data retrieval operations. Although three databases 112, 114, and 116 are shown in Figure 1

[0033] Although the database service manager 108 and the databases 112, 114, and 116 are shown as single components in Figure 1 Although the database service manager 108 and the databases 112, 114, and 116 are shown as single components in

[0034] Figure 1 As shown, according to some implementations, the database service manager 108 includes a replication controller 118 that implements database replication or sub-database replication in the database service system 100.

[0035] ​​Database replication involves copying an entire primary database (e.g., database 112) to a secondary database (e.g., database 116). In database replication, the database is the atomic unit of replication, such that a database can be either completely replicated or not replicated at all. Major drawbacks of database replication include high cost and latency associated with replicating large databases in their entirety. Furthermore, users are not provided with the opportunity to minimize replication cost and latency by selecting or excluding objects in a database from replication. For example, some users want to exclude objects from their databases from replication. In one implementation, replication controller 118 can restructure the database to be replicated such that the desired unit of replication matches the user's personal database boundaries.

[0036] Schema filtering and table filtering

[0037] Also to provide flexibility to users, replication controller 118 can implement subdatabase replication. Figure 2 A graphical representation 200 is shown of replication controller 118 implementing subdatabase replication using schema filtering and table filtering, according to some examples.

[0038] Within each database, there are many layers including schemas and tables. Schemas are logical containers in a database, and tables are another container within a schema with rows and columns. Schemas and tables can be represented as objects to allow schema-level filtering and table-level filtering.

[0039] In this implementation, replication controller 118 receives from each user an inclusion list or an exclusion list or any combination thereof. The inclusion list can include: a list of objects in a database to be replicated; and a target database associated with each object in the list. The objects can be on a schema, a table, or any combination thereof. The exclusion list can include: a list of objects in a database to be excluded from replication into a target database.

[0040] As Figure 2 shown, the user as a provider has an account PI, an account P2, and an account P3 in regions A, B, and C, respectively. Replication controller 118 receives from the user's client device 102 an inclusion list that identifies schema S2 and table Tl for replication into a target database in account P2 in region B, and that identifies schema S4 and tables T3 and T4 for replication into a target database in account P4 in region C. In this example, replication controller 118 can also receive from the user's client device 102 an exclusion list that identifies table T2 included in schema S2 to exclude from replication into the target database in account P2 in region B.

[0041] Alternatively, the replication controller 118 can receive an exclusion list from the user's client device 102 that identifies schema S1, schema S3, schema S4, and tables T2, T3, and T4 to exclude from the target database in account P2 in region B, and identifies schema S1, schema S2, schema S3, and tables T1, T2 from the target database in account P3 in region C.

[0042] The replication controller 118 then replicates the objects from the primary database in account PI in region A to the secondary databases in accounts P2 and P3 based on the inclusion list or the exclusion list, respectively.

[0043] In one implementation, the replication controller 118 generates a replication policy based on the inclusion list or the exclusion list. For example, the replication controller 118 can apply a database replication policy to limit replication of schema and tables by name using the exclusion list. Alternatively, the replication controller 118 can apply a database replication policy to enable replication of schema and tables by name using the inclusion list. In one implementation, the replication policy can be a new first stream database object that consumers can use to qualify schema and tables to be replicated to a target account. The primary database can also have multiple policies and multiple secondary databases.

[0044] Data sharing

[0045] Figure 3 is a graphical representation 300 of the replication controller 118 implementing data sharing from a database to clients within the same region according to some examples.

[0046] Some users in the database service system 100 are providers (e.g., provider users) that maintain and share data with their clients, who are also users of the database service system 100. As shown in Figure 3 The data can be stored in a single table (e.g., a data table) that includes a column for consumer identification (ID) (e.g., client_id). The table can also include columns for name, date, time. Data for all of the provider user's clients (e.g., client_id 1, client_id 2, client_id 3) can be stored in the table.

[0047] Referring to Figure 3 The provider user is also provided with an entitlement table to manage sharing of data with their clients. The entitlement table includes columns for consumer ID (e.g., client_id) and consumer account name (e.g., consumer account) associated with the database service system 100. According to Figure 3As shown in the equity table, consumer ID 1 and consumer ID 3 (e.g., client_id 1 and client_id 3) also have accounts within the database service system 100 as consumer account CI, consumer account C3, respectively.

[0048] Since the consumer account CI and the consumer account C3 are in the same region as the provider account, the replication controller 118 does not need to replicate the data. The replication controller 118 uses the secure view to join the data table with the equity table on the consumer IDs to share the relevant data from the data table with each of the consumer accounts. The replication controller 118 can receive queries from the consumer accounts CI and C3 for their data from the data table. In response, as shown in Figure 3 the replication controller 118 can select and share the rows associated with the consumer ID 1 (e.g., client_id 1) that has the consumer account CI. Similarly, the replication controller 118 can select and share the rows associated with the consumer ID 3 (e.g., client_id 3) that has the consumer account C3. Thus, the replication controller 118 is able to generate personalized sharing where a subset of the table rows can be shared with each consumer account. In one example, the replication controller 118 identifies the current consumer account that is querying, and selects and shares the rows associated with the identified consumer account.

[0049] Table-level replication

[0050] When the consumer accounts are in a different region from the data table (e.g., the master table), the replication controller 118 can replicate the entire table to the provider account in the different region and implement Figure 3 personalized sharing in each of the different regions. Figure 4 A graphical representation 400 of the replication controller 118 implementing table replication is shown in accordance with some examples.

[0051] In Figure 4 the provider user has a provider account PI in the central region where the data table (e.g., the master data table) is stored. The provider user stores data for clients in the data table that have consumer IDs (e.g., client_ids) 1, 2, 3, 4. As shown in the equity table, client_id 1 has a consumer account CI, client_id 2 has two consumer accounts C2-W, consumer account C2-E, client_id 3 has a consumer account C3, and client_id 4 has a consumer account C4. Figure 4 The equity table in also includes columns that associate the consumer IDs (client_ids) with the provider accounts (e.g., provider account PI, provider account P2, provider account P3) across the different regions.

[0052] Since at least one consumer account (e.g., CI, C2-E, C2-W, C3, C4) is in the western region as well as the eastern region, the replication controller 118 replicates the entire data table to the provider account P2 in the western region and the provider account P3 in the eastern region. Once the data table is in the provider account P2, the replication controller 118 can share (e.g., personalize share) a subset of the table rows in the secondary table in the provider account P2 with each consumer account C2-W and the consumer account C3 in the western region. Similarly, once the data table is in the provider account P3, the replication controller 118 can share a subset of the table rows in the secondary table in the provider account P3 with each consumer account C2-E and the consumer account CI in the eastern region.

[0053] Since the consumer account C4 is in the same central region as the provider user and the (primary) data table, the replication controller 118 can respond to the query request from the consumer account C4 by sharing the data rows associated with client_id 4 to the consumer account C4.

[0054] Figure 4 A drawback of the table replication in is that since the replication controller 118 replicates the entire table between regions, rows in the primary data table that can not be needed in a region are being replicated to the region. This can result in higher cost and higher latency.

[0055] Figure 5 A graphical representation 500 showing the replication controller 118 implementing sub-table replication is shown in accordance with some examples. In this implementation, the replication controller 118 replicates a subset of the table rows that need to be shared in a given region to the provider account in the region. For example, the replication controller 118 selects a subset of the table rows (client_id 2, client_id 3) that need to be shared to consumers (e.g., C3 and C2-W) and replicates the subset to the provider account P2 in the western region. Similarly, the replication controller 118 selects a subset of the table rows (client_id 1, client_id 2) that need to be shared to consumers (e.g., CI and C2-E) and replicates the subset to the provider account P3 in the eastern region. Thus, in contrast to the table replication in, Figure 4 Figure 5 In contrast to the table replication in,

[0056] ​Selecting rows from a data table to replicate is challenging because data in a master data table can be materialized into contiguous storage units called micro-partitions. A table can be a collection of micro-partitions. Each micro-partition is a file containing between 50MB and 500MB of uncompressed data. The size of a micro-partition can be equal to or less than 16 megabytes (MB). Groups of rows in a table can be mapped into a single micro-partition organized by column.

[0057] In addition, using a data manipulation language (DML), changes can be made to data in a data table. In some implementations, changes can be made by any DML statement. Examples of manipulating data can include, but are not limited to, selecting, updating, changing, merging data, and inserting data into a table. When new data is inserted, the database service manager 108 creates a new micro-partition. When data is updated, the database service manager 108 marks the micro-partition storing the data for deletion and creates a new micro-partition for the updated data.

[0058] In one implementation, assuming many rows in a table are packed into the micro-partitions to perform row-level filtering, the replication controller 118 opens each micro-partition to see if rows in it are to be replicated for different regions. This row-level filtering achieves precision but makes replication slow. The provider user materializes rows into separate objects

[0059] In another implementation, rows can be materialized into separate objects (e.g., separate tables) so that new micro-partitions are created for rows that need to be replicated. In one implementation, the provider user materializes rows from a data table to be replicated into separate tables. Specifically, the provider user can create a new table and specify rows from a (master) data table to be replicated in it. Rows in the new table are materialized into new micro-partitions, and the provider user requests the replication controller 118 to replicate the new table. Since only relevant rows for a region are replicated, the replication cost and replication latency are low. However, there is a high storage cost because a duplication of the data to be replicated is needed; there is a high extract, transform, load (ETL) cost and high ETL latency because a transformation needs to be done for each row of data to be stored in the new table and then replicated, and the effort required by the provider consumer is high.

[0060] The replication controller materializes rows into materialized views

[0061] In another implementation, to alleviate the effort required by the provider user, the replication controller 118 can materialize rows from a data table to be replicated into materialized views.

[0062] A materialized view is a database object that includes the final or intermediate results of a database query. A materialized view can include a locally cached copy of database data, a subset of rows or columns of a table, a joined result, a result of an aggregate function, etc. A materialized view can be defined by a client or a system administrator and can include any appropriate information. Materialized views are often generated to help perform specific general-purpose queries.

[0063] A materialized view as disclosed in this application is an illustrative specification of a persistent query result that is automatically maintained and transparently utilized. In one example, a materialized view includes a local copy of data located remotely, or can include a subset of rows and / or columns of a source data table (which can be referred to as a "partition" or "micro-partition") or a joined result or can alternatively include a summary using an aggregate function. A materialized view is generated by materialization, where the cached query result is analogous to remembering the value of a function in a functional language. A materialized view improves the performance of expensive queries by materializing and reusing commonly used intermediate query results in a workload. A materialized view is used to improve query performance at the cost of maintenance cost and increased storage requirements.

[0064] In one example, a materialized view can include a summary of rows from a data table that are relevant to a provider user or a summary of rows that are associated with a query from a provider user. When a replication controller 118 detects an update with respect to a data table, the replication controller 118 can update the materialized view accordingly. For example, when an update with respect to a data table includes inserting a new micro-partition into a source table, the replication controller 118 refreshes the materialized view by inserting the new micro-partition into the materialized view. Further, when an update with respect to a data table includes deleting a micro-partition, the replication controller 118 can compact the materialized view by removing the deleted micro-partition from the materialized view. The data table can thus be replicated as-is. For example, the replication controller 118 can establish and implement policies based on a replication target account. Since only relevant rows for a region are replicated, the replication cost and replication latency are low, but the storage cost is still high and there is an increased cost associated with the materialized view.

[0065] Replication controller replicates a subset of rows (sub-table replication)

[0066] Instead of materializing rows as separate new micro-partitions, in another implementation, the replication controller 118 can replicate a subset of rows based on a replication target account without the overhead of Figure 5 replicating rows as shown.

[0067] Row-level filtering based on provider user-specified columns

[0068] In one embodiment, to replicate a subset of rows, the replication controller 118 receives from the provider consumer an identification of the column in the data table to be filtered and the filtering behavior to be applied. For example, in Figure 5 the provider consumer P1 can indicate that the consumer ID column is to be filtered for the client_id 1 rows and that the client_id 1 rows in provider account P3 in the East region are to be replicated for consumer C1.

[0069] As discussed above, the replication controller 118 can perform row-level filtering by opening each micro partition to see if rows therein are to be replicated for different regions. This row-level filtering achieves precision, but makes replication performance slow.

[0070] File-level filtering based on provider user specified columns

[0071] The metadata for each file (micro partition) can indicate the minimum and maximum values in each column of the file. Thus, for each micro partition, the metadata indicates the minimum consumer ID (client_id) and the maximum consumer ID (client_id). For example, a micro partition with a minimum client_id of 1 and a maximum client_id of 3 indicates the possibility of a row associated with client_id 2. Without opening the micro partition, the replication controller 118 can replicate the micro partition for client_id 1, client_id 2, and client_id 3. By avoiding opening the micro partition, replication performance is faster, but precision is also lower. Specifically, the replication controller 118 implementing file-level filtering can replicate unnecessary rows to some regions.

[0072] To improve precision, the replication controller 118 can cluster the table based on the column that defines the allocation policy to generate micro partitions that are better ordered in that column. However, there is an increased cost associated with clustering.

[0073] Storing provider user specified routing policies

[0074] As discussed above, in addition to indicating rows in a data table to be replicated, a provider user can also indicate a routing policy. For example, in Figure 5 the provider consumer P1 can indicate that the consumer ID column is to be filtered for the client_id 1 rows and that the client_id 1 rows in provider account P3 in the East region are to be replicated for consumer C1. As Figure 5As shown, the entitlements table can be a separate table where the provider user can specify the routing policy. In another implementation, the entitlements table can also be stored as metadata to provide a faster access path to the routing policy. For example, the entitlements table can be a new first class object for personalized sharing that is used to qualify rows for which consumers CI, C2, C3 have rights.

[0075] To implement sub-table replication, the replication controller 118 can use the entitlements applied at runtime, or the replication controller 118 can use the entitlements column within the (primary) data table that are maintained by the provider user or automated by the replication controller 118 or a combination thereof.

[0076] Sub-table replication: Entitlements applied at runtime

[0077] Figure 6 A graphical representation 600 is shown of the replication controller 118 implementing sub-table replication using entitlements applied at runtime, according to some examples.

[0078] In Figure 6 In the middle region, the provider user has a provider account PI with a data table 604 in the middle region that has data rows for its clients 1, 2, 3, 4 (e.g., client id 1, client id 2, client id 3, client id 4). The provider user indicates in the entitlements table 602 that client id 1 is associated with consumer account CI in the eastern region and provider account P3. Accordingly, this change in the entitlements table 602 indicates that the provider user wants to replicate the consumer account CI from the middle region to the eastern region.

[0079] In one example, the provider user’s update to the entitlements table 602 is a change to the entitlements table 602 that is detected by the replication controller 118, and causes the replication controller 118 to initiate replication associated with the change, where the provider user’s update to the entitlements table 602 is to indicate that client id 1 is associated with consumer account CI in provider account P3.

[0080] In another example, the replication controller 118 can receive a request for consumer account CI from provider account P3. In this example, the provider user can specify provider account P3 for consumer account CI.

[0081] Using this request, the replication controller 118 can use the minimum client_id and maximum client_id in the micro-partition metadata to filter the micro-partitions in the (file-level) data table to locate the micro-partitions to replicate to the provider account P3. The replication controller 118 replicates the located micro-partitions to the provider account P3.

[0082] When the provider user updates the entitlement table, the replication controller 118 needs to perform additional sub-table replication. Figure 7 A graphical representation 700 of the replication controller 118 implementing sub-table replication using entitlements applied at runtime is shown, according to some examples.

[0083] In Figure 7 In the example, the provider user indicates that the client_id 2 in the entitlement table 602 is associated with the consumer account C2-E in the East region and the provider account P3. This indicates that a new consumer wants to replicate data to the East region.

[0084] The replication controller 118 can perform incremental replication, which uses two mechanisms: version-based and full manifest.

[0085] In version-based replication, when the data table produces a change, the replication controller 118 can detect the change to the data table and perform replication based on the new version of the data table. For example, performing version-based replication can include identifying rows in the data table that are affected due to the change and replicating these rows in the data table to the target account (e.g., the provider account).

[0086] In full manifest replication, when the entitlement table produces a change, the replication controller 118 can select all rows in the data table that are associated with the provider account identifier in the entitlement table that is affected due to the change.

[0087] Version-based replication is faster than full manifest. In one implementation, when the entitlement table 602 changes, the replication controller 118 causes full manifest to be performed for each target account (e.g., the provider account) at the next flush. While the amount of work required by the provider user is low, the replication cost and replication latency associated with sub-table replication using entitlements applied at runtime is high.

[0088] Process to implement sub-table replication: entitlements applied at runtime

[0089] Although the flow diagrams can illustrate a process as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations can be re-arranged. A process is terminated when its operations are completed. A process can correspond to a method, a procedure, an algorithm, etc. The operations of the methods can be performed all or in part by one or more aspects of the suitable system such as the systems described herein or any portion thereof such as a processor included in any of the systems. For example, the processes can be performed by the replication controller 118 or a processor included in the replication controller 118, or a processor in the database service manager 108, or a combination thereof.

[0090] Figure 8 A method 800 to implement sub-database replication is shown in accordance with one embodiment. At operation 802, a processor in the replication controller 118 detects a first update to the entitlement table 602. As shown in Figure 6 and Figure 7 The entitlement table 602 can include entitlement table rows associated with a client identifier, a consumer account identifier, and a provider account identifier. The entitlement table 602 can include entitlement table columns that store the consumer account identifier and the provider account identifier. As shown in Figure 6 and Figure 7 The provider account identifier identifies a provider account in a plurality of geographic regions. The first update can be associated with a first one of the entitlement table rows. The first update can include, for example, a first one of the client identifiers, a first one of the consumer account identifiers, or a first one of the provider account identifiers.

[0091] At operation 804, the processor performs a filtering of the data table 604 based on the first update. As shown in Figure 6 and Figure 7 The data table 604 can include data table rows that are associated with a client identifier and include data content (e.g., an event, a date, a payload, etc.). In one embodiment, the processor receives a replication request from the first provider account that includes the first consumer account identifier.

[0092] At operation 806, the processor detects a second update to the entitlement table 602. Examples of the first update and the second update to the entitlement table 602 include adding a new entitlement table row to the entitlement table, deleting one of the entitlement table rows, or changing information included in one of the entitlement table rows. In one example, the changing information included in one of the entitlement table rows includes changing the client identifier, the consumer account identifier, or the provider account identifier.

[0093] At operation 808, the processor performs the incremental replication of the data table 604 by causing a full copy to be performed at the next refresh for the provider account associated with the provider account identifier in the entitlements table 602.

[0094] In one example, the processor performs the filtering of the data table 604 in operation 804 by performing file-level filtering. Figure 9 Details of operation 804 from Figure 8 are shown in accordance with one embodiment.

[0095] At operation 902, the processor identifies a micro partition of the plurality of micro partitions in the data table 604 that has metadata associated with the first client identifier. Each of the micro partitions in the data table 604 can include one or more of the plurality of data table rows.

[0096] In one example, the micro partition in the data table 604 includes metadata that includes a minimum client identifier and a maximum client identifier. In this example, the metadata is associated with the first client identifier when the first client identifier is within a range established by the minimum client identifier and the maximum client identifier.

[0097] At operation 904, the processor replicates the identified micro partition to a first provider account associated with the first provider account identifier.

[0098] Sub-table replication: Entitlement column within table

[0099] To enable version-based replication to be performed, at least one change to the master data table 604 is required. For example, a table DML can be triggered to cause the replication controller 118 to perform version-based replication. Figure 10 A graphical representation 1000 of sub-table replication implemented by the replication controller 118 using an entitlement column 1002 within a table is shown in accordance with some examples.

[0100] In Figure 10 , the (master) data table includes an entitlement column 1002 (e.g., a target account column) that indicates a provider account associated with a row in the table. In one embodiment, each time the entitlements table 602 changes, the provider user updates the column in the data table 1004. The update to the entitlement column 1002 in the data table causes the replication controller 118 to perform version-based replication. For example, the replication controller 118 can update the data table 1004 version and create a new micro partition to reflect the change in the entitlements table 602. In this embodiment, the replication controller 118 performing version-based replication includes using the entitlement column 1002 to replicate rows in the data table 1004 to a target account (e.g., a provider account).

[0101] Figure 11 A graphical representation 1100 of sub-table replication using in-table entitlement columns by replication controller 118 is shown, according to some examples.

[0102] In Figure 11 , the provider user updates entitlement table 602 to indicate that client_id 2 is associated with consumer account C2-E in the eastern region and provider account P3. In this implementation, the provider user also updates entitlement column 1002 in data table 1004 to reflect that the row with client_id 2 is also associated with provider account P3.

[0103] As Figure 10 shown, the update to entitlement column 1002 in data table 1004 causes replication controller 118 to perform version-based replication. In this implementation, the replication controller 118 performing version-based replication includes replicating the row in data table 1004 to a target account (e.g., a provider account) using entitlement column 1002. As Figure 10 and Figure 11 shown, sub-table replication using in-table entitlement columns 1002 reduces replication cost and replication latency, but there is a higher ETL cost and ETL latency, and the effort required by the provider user is much higher.

[0104] Sub-table replication: virtual entitlement column within table

[0105] To mitigate the effort required by the provider user, in one implementation, replication controller 118 can maintain a virtual entitlement column within the data table. Figure 12 A graphical representation 1200 of sub-table replication using a virtual entitlement column within a table by replication controller 118 is shown, according to some examples.

[0106] In Figure 12 , the (primary) data table includes a virtual entitlement column 1202 (e.g., a target account column) that indicates the provider account associated with the row in data table 1004. The virtual entitlement column 1202 is maintained by replication controller 118.

[0107] As Figure 12As shown, the provider user updates the benefits table 602 to indicate that client_id 1 is associated with consumer account C1 and provider account P3 in the Eastern Region, and client_id 2 is associated with consumer accounts C2-E and provider account P3 in the Eastern Region. When the provider user updates the benefits table 602, the replication controller 118 populates the virtual benefits column 1202 in data table 1004. In this embodiment, the replication controller 118 updates the virtual benefits column 1202 for the rows of client_id 1 and client_id 2, indicating provider account P3. In response to this update to the virtual benefits column 1202 in data table 1004, the replication controller 118 can perform version-based replication. In this embodiment, the replication controller 118 performing version-based replication includes copying the rows in data table 1004 to the target account (e.g., the provider account) using the virtual benefits column 1202.

[0108] Although ETL costs and ETL latency remain high, using methods such as Figure 12 The virtual equity column 1202 in the data table 1004 shown reduces replication costs and latency. Furthermore, because the replication controller 118 maintains the virtual equity column 1202, the workload required by the provider user is reduced.

[0109] The process of replicating a sub-table involves: the equity column within the table or a virtual equity column within the table.

[0110] Figure 13 The process 1300 for implementing sub-database replication according to one embodiment is illustrated. At operation 1302, the processor of the replication controller 118 detects an update to the equity table 602. (As...) Figure 10 , Figure 11 and Figure 12 As shown, benefits table 602 may include benefits table rows associated with client identifiers, consumer account identifiers, and provider account identifiers. Benefit table 602 may include benefits table columns storing consumer account identifiers and provider account identifiers. Figure 10 , Figure 11 and Figure 12 As shown, the provider account identifier identifies provider accounts in multiple geographic regions. An update to the benefits table 602 can be associated with a first benefits table row in the benefits table rows. An update to the benefits table 602 can include, for example, a first client identifier in the client identifier; a first consumer account identifier in the consumer account identifier; or a first provider account identifier in the provider account identifier.

[0111] Examples of updates to the equity table 602 in operation 1302 include adding a new equity table row to the equity table, deleting one of the equity table rows, or changing information included in one of the equity table rows. In one example, changing information included in one of the equity table rows includes changing a client identifier, a consumer account identifier, or a provider account identifier.

[0112] At operation 1304, the processor performs filtering of the data table 1004 based on the update to the equity table 602. As shown in Figure 10 、 Figure 11 and Figure 12 , the data table 1004 can include a data table row associated with a client identifier, and the data table 1004 includes the equity column 1002 (or the virtual equity column 1202) and data content (e.g., events, dates, payloads, etc.). As shown in Figure 10 、 Figure 11 and Figure 12 , the equity column 1002 (or the virtual equity column 1202) can include a provider account identifier. In one implementation, the processor receives a replication request from a first provider account that includes a first consumer account identifier. In one example, the processor can perform file-level filtering in operation 1304 as detailed in Figure 13 .

[0113] At operation 1306, the processor detects an update to the equity column 1002 (or the virtual equity column 1202). For example, the equity column 1002 in Figure 10 and Figure 11 is populated or updated by a provider user via a client device. The processor detecting an update to the virtual equity column 1202 can also include determining that the update to the equity table 602 includes a change in one of the provider account identifiers in the equity table 602, and the processor detecting an update to the virtual equity column 1202 can also include populating the virtual equity column 1202 based on the one of the provider account identifiers in the equity table 602.

[0114] At operation 1308, the processor performs incremental replication of the data table 1004 by performing version-based replication. Figure 14 Details of the process of version-based replication in operation 1308 from Figure 13 are shown according to one implementation. At operation 1402, the processor updates a version identifier associated with the data table 1004. At operation 1404, the processor adds a plurality of micro-partitions in the data table 1004 that reflect the update to the equity column 1002 (or the virtual equity column 1202). At operation 1406, the processor replicates data table rows to a plurality of provider accounts associated with the provider account identifiers in the equity table 602 using the equity column 1002 (or the virtual equity column 1202).

[0115] Machine architecture

[0116] Figure 15 is a graphical representation of a machine 1500 in which the instructions 1510 (e.g., software, a program, an application, an applet, an app, or other executable code) can be executed by the machine 1500 to cause the machine 1500 to perform any one or more of the methodologies discussed herein. For example, the instructions 1510 can cause the machine 1500 to perform any one or more of the methods described herein. The instructions 1510 transform the general, non-programmed machine 1500 into a particular machine 1500 programmed to carry out the described and illustrated functions in the manner described. The machine 1500 can operate as a standalone device or can be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1500 can operate in the capacity of a server machine or a client machine in server-client network environments, or as a peer machine in peer-to-peer (or distributed) network environments. The machine 1500 can comprise, but not be limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular telephone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions 1510, sequentially or otherwise, that specify actions to be taken by machine 1500. Further, while only a single machine 1500 is illustrated, the term “machine” shall also be taken to include a collection of machines 1500 that individually or jointly execute the instructions 1510 to perform any one or more of the methodologies discussed herein. For example, the machine 1500 can include any of the client device 102 or a plurality of server devices forming part of the database service manager 108. In some examples, the machine 1500 can further include both client and server systems, wherein some operations of a particular method or algorithm are performed on the server-side and wherein some operations of the particular method or algorithm are performed on the client-side.

[0117] The machine 1500 can include processors 1504, memory 1506, and input / output (I / O) components 638, which can be configured to communicate with each other via a bus 1538. In an example, the processors 1504 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a radio-frequency integrated circuit (RFIC), another processor, or combinations of any of these) can include, for example, at least one processor 1508 that executes instructions 1510. The term “processor” is intended to encompass multi-core processors that can include two or more independent processors (sometimes referred to as “cores”) that can execute instructions contemporaneously. Although FIG. 15 shows the processor 1504 that executes instructions 1510, in another example, the processor 1508 includes multiple processors that each execute instructions 1510. Although FIG. 15 shows the processor 1504 as a single component, in another example, the processor 1508 includes two or more components, which can or can not be Figure 15 Although FIG. 15 shows multiple processors 1504, the machine 1500 can include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0118] The memory 1506 includes a main memory 1512, a static memory 1514, and a storage unit 1516, each of which can be accessed via the bus 1538 by the processor 1504. The main memory 1512, the static memory 1514, and the storage unit 1516 store the instructions 1510 that implement any one or more of the methodologies or functions described herein. The instructions 1510 can also reside, completely or

[0119] The I / O components 1502 can include various components that receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so forth. The specific I / O components 1502 that are included in the machine 1500 will depend on the type and Figure 15Many other components not shown in FIG. 15 can also be included. In various examples, the I / O components 1502 can include user output components 1524 and user input components 1526. The user output components 1524 can include visual components (e.g., a display such as a plasma display panel (PDP), a light-emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., speakers), haptic components (e.g., a vibratory motor, resistance mechanisms), other signal generators, and so forth. The user input components 1526 can include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input components), pointing components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or another pointing instrument), tactile input components (e.g., a physical button, a touch screen that provides location and / or force of touches or touch gestures, or other tactile input components), audio input components (e.g., a microphone), and the like.

[0120] In another example, the I / O components 1502 can include biometric components 1528, motion components 1530, environmental components 1532, or position components 1534, among a myriad of other components. For example, the biometric components 1528 include components to detect expressions (e.g., hand expressions, facial expressions, vocal expressions, body gestures, or eye tracking), measure biosignals (e.g., blood pressure, heart rate, body temperature, perspiration, or brain waves), identify a person (e.g., voice identification, retinal identification, facial identification, fingerprint identification, or electroencephalogram-based identification), and the like.

[0121] The environmental components 1532 include, for example, one or more cameras (with still and / or video capabilities), illumination sensors (e.g., photometers), temperature sensors (e.g., one or more thermometers that detect ambient temperature), humidity sensors, pressure sensors (e.g., barometers), acoustic sensors (e.g., one or more microphones that detect background noise), proximity sensors (e.g., infrared sensors that detect nearby objects), gas sensors (e.g., gas detection sensors to detect concentrations of hazardous gases for safety or to measure pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to a physical environment.

[0122] The position components 1534 include location sensor components 1534, altitude sensor components 1534, orientation sensor components 1534, and the like.

[0123] Communication can be implemented using a wide variety of technologies. The I / O components 1502 further include communication components 1536 operable to couple the machine 1500 to a network 1520 or devices 1522 via respective coupling or connections. For example, the communication components 1536 can include a network interface component or another suitable device to interface with the network 1520. In further examples, the communication components 1536 can include wired components (e.g., low power, Wi- components, and other communication components to provide communication via other modalities. The devices 1522 can be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a USB).

[0124] Moreover, the communication components 1536 can detect identifiers or include components operable to detect identifiers. For example, the communication components 1536 can include radio frequency identification (RFID) tag reader components, NFC smart tag detection components, optical reader components (e.g., an optical sensor to detect one-dimensional bar codes such as Universal Product Code (UPC) bar code, multi-dimensional bar codes such as Quick Response (QR) code, Aztec code, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D bar code, and other optical codes), or acoustic detection components (e.g., microphones to identify acoustic signals indicating a specific location). Additionally, a variety of information can be derived via the communication components 1536, such as location via Internet Protocol (IP) geolocation, location via Wi-Fi® signal triangulation, location via detecting an NFC beacon signal that can indicate a specific location, and so forth.

[0125] The various memories (e.g., the main memory 1512, the static memory 1514, and the memory of the processor 1504) and the storage unit 1516 can store one or more sets of instructions and data structures (e.g., software) embodying or used by any one or more of the methodologies or functions described herein. These instructions (e.g., instructions 1510), when executed by the processor 1504, cause various operations to implement the disclosed examples.

[0126] The instructions 1510 can be transmitted or received using a transmission medium via the network 1520 (for example, a communication network including the Internet) and using any one of multiple, well-known transfer protocols (for example, hypertext transfer protocol (HTTP)). Similarly, the instructions 1510 can be transmitted or received using a transmission medium via the coupling (for example, a peer-to-peer coupling) to the devices 1522.

[0127] Software Architecture

[0128] Figure 16 is a block diagram 1600 illustrating a software architecture 1604, which can be installed on any one or more of the devices described herein. The software architecture 1604 is supported by hardware of a machine 1602, such as the machine 1602 includes processors 1620, memory 1626, and I / O components 1638. In this example, the software architecture 1604 can be conceptualized as a stack of layers, where each layer provides particular functionality. The software architecture 1604 includes layers such as an operating system 1612, libraries 1610, frameworks 1608, and applications 1606. Operationally, the applications 1606 invoke API calls 1650 through the software stack and receive messages 1652 in response to the API calls 1650.

[0129] The operating system 1612 manages hardware resources and provides common services. The operating system 1612 includes, for example, a kernel 1614, services 1616, and drivers 1622. The kernel 1614 acts as an abstraction layer between the hardware and the other software layers. For example, the kernel 1614 provides memory management, processor management (for example, scheduling), component management, networking, and security settings, among other functionality. The services 1616 can provide other common services that the or Low power drivers, flash memory drivers, serial communication drivers (for example, USB drivers), WI- Firmware drivers, audio drivers, power management drivers, and so forth.

[0130] The libraries 1610 provide a common low-level infrastructure used by the applications 1606. The libraries 1610 can include system libraries 1618 (e.g., C standard library) providing functionality such as memory allocation functions, string manipulation functions, mathematic functions, and the like. The libraries 1610 can further include API libraries 1624, such as media libraries (e.g., libraries to support presentation and manipulation of various media formats such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), graphics libraries (e.g., an OpenGL framework used to render two and three dimensional graphics on a display), database libraries (e.g., SQLite to provide various relational database functions), web libraries (e.g., WebKit to provide web browsing functionality), and the like. The libraries 1610 can also include a wide variety of other libraries 1628 to provide many other APIs to the applications 1606.

[0131] The frameworks 1608 provide a common high-level infrastructure used by the applications 1606. For example, the frameworks 1608 provide various graphical user interface (GUI) functions, high-level resource management, and high-level location services. The frameworks 1608 can provide a broad spectrum of other APIs that can be used by the applications 1606, some of which can be specific to a particular operating system or platform.

[0132] In an example, the applications 1606 include a home application 1636, a contacts application 1630, a browser application 1632, a book reader application 1634, a location application 1642, a media application 1644, a messaging application 1646, a game application 1648, and a broad assortment of other applications such as a third party application 1640. The applications 1606 are programs that execute functions defined in the programs. Various programming languages can be employed to create one or more of the applications 1606, structured in a variety of manners, such as object-oriented programming languages (e.g., Objective-C, Java, or C++) or procedural programming languages (e.g., C or assembly language). In a specific example, the third party application 1640 (e.g., an application developed by an entity other than the vendor of the particular platform) can be an Android TM or IOS TM application developed using the ANDROID TM or IOS TM software development kit (SDK). mobile software running on a mobile operating system of the iPhone® phone or other mobile operating system. In this example, the third party application 1640 can invoke an API call 1650 provided by the operating system 1612 to facilitate the functionality described herein.

[0133] In addition, the present technology can be configured as follows:

[0134] (1) A method of implementing child table replication, comprising:

[0135] detecting, by a processor, a first update to an equity table, the equity table comprising a plurality of equity table rows,

[0136] the equity table rows being associated with a plurality of client identifiers, a plurality of consumer account identifiers, and a plurality of provider account identifiers,

[0137] the first update being associated with a first equity table row of the equity table rows, the first update comprising: a first client identifier of the client identifiers, a first consumer account identifier of the consumer account identifiers; or a first provider account identifier of the provider account identifiers;

[0138] performing filtering on a data table based on the first update, the data table comprising a plurality of data table rows, the plurality of data table rows being associated with the client identifiers and comprising data content;

[0139] detecting a second update to the equity table; and

[0140] performing incremental replication of the data table by causing full inventory replication at a next flush for a plurality of provider accounts associated with the provider account identifiers in the equity table.

[0141] (2). The method of (1), wherein performing filtering further comprises:

[0142] performing file-level filtering by:

[0143] identifying a micro partition of a plurality of micro partitions in the data table, the micro partition having metadata associated with the first client identifier, each of the micro partitions in the data table comprising one or more of the plurality of data table rows; and

[0144] copying the identified micro partition to a first provider account associated with the first provider account identifier.

[0145] (3). The method of (2), wherein:

[0146] Each of the plurality of micro-partitions in the data table includes metadata, the metadata including a minimum client identifier and a maximum client identifier.

[0147] (4). The method of (3), wherein,

[0148] The metadata is associated with the first client identifier when the first client identifier is within a range established by the minimum client identifier and the maximum client identifier.

[0149] (5). The method of (1), wherein the plurality of provider account identifiers identify provider accounts in a plurality of geographic regions.

[0150] (6). The method of (1), wherein the balance sheet further includes a plurality of balance sheet columns storing the consumer account identifiers and the provider account identifiers.

[0151] (7). The method of (1), further comprising:

[0152] receiving a replication request from the first provider account, the replication request including a first consumer account identifier.

[0153] (8). The method of (1), wherein the first update and the second update to the balance sheet include adding a new balance sheet row to the balance sheet, deleting one of the balance sheet rows, or changing information included in one of the balance sheet rows.

[0154] (9). The method of (8), wherein changing information included in one of the balance sheet rows includes changing the client identifier, the consumer account identifier, or the provider account identifier.

[0155] (10). A system that implements sub-table replication, comprising:

[0156] a processor; and

[0157] a memory having instructions stored therein that, when executed by the processor, cause the system to perform operations comprising:

[0158] detecting a first update to a balance sheet,

[0159] the balance sheet including a plurality of balance sheet rows, the balance sheet rows being associated with a plurality of client identifiers, a plurality of consumer account identifiers, and a plurality of provider account identifiers,

[0160] The first update is associated with a first balance sheet row of the balance sheet rows, the first update comprising: a first client identifier of the client identifier, a first consumer account identifier of the consumer account identifier; or a first provider account identifier of the provider account identifier;

[0161] performing filtering on a data table based on the first update, the data table comprising a plurality of data table rows, the plurality of data table rows being associated with the client identifier and comprising data content;

[0162] detecting a second update to the balance sheet; and

[0163] performing incremental replication of the data table by causing full inventory replication at a next refresh for a plurality of provider accounts associated with the provider account identifier in the balance sheet.

[0164] (11). The system of (10), wherein performing filtering further comprises performing file-level filtering by:

[0165] identifying a micro partition of a plurality of micro partitions in the data table, the micro partition having metadata associated with the first client identifier, each micro partition of the micro partitions in the data table comprising one or more of the plurality of data table rows; and

[0166] copying the identified micro partition to a first provider account associated with the first provider account identifier.

[0167] (12). The system of (11), wherein,

[0168] each micro partition of the plurality of micro partitions in the data table comprises metadata, the metadata comprising a minimum client identifier and a maximum client identifier.

[0169] (13). The system of (12), wherein,

[0170] the metadata is associated with the first client identifier when the first client identifier is within a range established by the minimum client identifier and the maximum client identifier.

[0171] (14). The system of (10), wherein the plurality of provider account identifiers identify provider accounts in a plurality of geographic regions.

[0172] (15). The system of (10), wherein the balance sheet further comprises a plurality of balance sheet columns storing the consumer account identifier and the provider account identifier.

[0173] (16). The system of (10), further comprising:

[0174] receiving a replication request from the first provider account, the replication request including a first consumer account identifier.

[0175] (17). The system of (10), wherein the first update and the second update to the entitlement table comprise: adding a new entitlement table row to the entitlement table; deleting one of the entitlement table rows; or changing information included in one of the entitlement table rows.

[0176] (18). The system of (17), wherein changing information included in one of the entitlement table rows comprises: changing the client identifier, the consumer account identifier, or the provider account identifier.

[0177] (19). A non-transitory computer-readable storage medium comprising instructions that, when processed by a computer, configure the computer to perform operations comprising:

[0178] detecting a first update to an entitlement table,

[0179] the entitlement table comprising a plurality of entitlement table rows, the entitlement table rows being associated with a plurality of client identifiers, a plurality of consumer account identifiers, and a plurality of provider account identifiers,

[0180] the first update being associated with a first entitlement table row of the entitlement table rows, the first update comprising: a first client identifier of the client identifiers, a first consumer account identifier of the consumer account identifiers; or a first provider account identifier of the provider account identifiers;

[0181] performing filtering on a data table based on the first update, the data table comprising a plurality of data table rows, the plurality of data table rows being associated with the client identifiers and including data content;

[0182] detecting a second update to the entitlement table; and

[0183] performing incremental replication of the data table by causing full inventory replication at a next flush for a plurality of provider accounts associated with the provider account identifiers in the entitlement table.

[0184] (20). The non-transitory computer-readable storage medium of (19), wherein performing filtering further comprises: performing file-level filtering.

[0185] (21). The non-transitory computer-readable storage medium of (20), wherein performing the file-level filtering further comprises:

[0186] identifying a micro partition of a plurality of micro partitions in the data table, the micro partition having metadata associated with the first client identifier.

[0187] (22). The non-transitory computer-readable storage medium of (21), wherein performing the file-level filtering further comprises:

[0188] copying the identified micro partition to a first provider account associated with the first provider account identifier.

[0189] (23). The non-transitory computer-readable storage medium of (22), wherein each of the micro partitions in the data table comprises one or more of the plurality of data table rows.

[0190] (24). The non-transitory computer-readable storage medium of (23), wherein:

[0191] each of the plurality of micro partitions in the data table comprises metadata, the metadata comprising a minimum client identifier and a maximum client identifier.

[0192] (25). The non-transitory computer-readable storage medium of (24), wherein:

[0193] the metadata is associated with the first client identifier when the first client identifier is within a range established by the minimum client identifier and the maximum client identifier.

[0194] (26). The non-transitory computer-readable storage medium of (19), wherein the plurality of provider account identifiers identify provider accounts in a plurality of geographic regions.

[0195] (27). The non-transitory computer-readable storage medium of (19), wherein the credit table further comprises a plurality of credit table columns storing the consumer account identifier and the provider account identifier.

[0196] (28). The non-transitory computer-readable storage medium of (19), wherein the operations further comprise:

[0197] receiving a copy request from the first provider account, the copy request comprising a first consumer account identifier.

[0198] (29). The non-transitory computer-readable storage medium of (19), wherein the first update and the second update to the equity table comprise: adding a new equity table row to the equity table; deleting one of the equity table rows; or changing information included in one of the equity table rows.

[0199] (30). The non-transitory computer-readable storage medium of (29), wherein changing information included in one of the equity table rows comprises: changing the client identifier, the consumer account identifier, or the provider account identifier.

[0200] Glossary

[0201] “Carrier signal” means any intangible medium that is capable of storing, encoding, or carrying the instructions for execution by a machine and includes digital or analog communications signals or other intangible media to facilitate such communication. Instructions can be transmitted or received over a network via a network interface device using a transmission medium.

[0202] “Client device” means any machine that interfaces to a communications network to obtain resources from one or more server systems or other client devices. A client device can be, but is not limited to, a mobile phone, desktop computer, laptop computer, portable digital assistants (PDAs), smart phones, tablets, ultrabooks, netbooks, a plurality of laptop computers, multi-processor systems, microprocessor-based or programmable consumer electronics, game consoles, set-top boxes, or any other communication device that a user can use to access a network.

[0203] “Communication network” means one or more portions of a network that can be an ad hoc network, intranet, extranet, virtual private network (VPN), local area network (LAN), wireless LAN (WLAN), wide area network (WAN), wireless WAN (WWAN), metropolitan area network (MAN), the Internet, a portion of the Internet, a portion of the Public Switched Telephone Network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, a Wi-Fi® network, another type of network, or a combination of two or more such networks. The network, another type of network, or a combination of two or more such networks. For example, a network or a portion of a network can include a wireless network or a cellular network, and the coupling can be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or other types of cellular or wireless coupling. In this example, the coupling can implement any of a variety of types of data transfer techniques for which a wireless connection can be appropriate, such as Radio Frequency identification (RFID), Near Field Communication (NFC), Infrared (IR) data transfer, Bluetooth® connection, WiFi® connection, or other types of data transfer techniques. A wireless connection can implement any of a variety of types of wireless connection techniques, such as single carrier radio transmission, Evolution-Data Optimized (EVDO) technology, General Packet Radio Service (GPRS) technology, Enhanced Data Rates for GSM Evolution (EDGE) technology, third Generation Partnership Project (3GPP) including 3G, fourth generation wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High Speed Packet Access (HSPA), Worldwide Interoperability for Microwave Access (WiMAX), Long Term Evolution (LTE) standard, others defined by various standards setting organizations, other long-range protocols, or other data transfer techniques.

[0204] A“component” refers to a device, physical entity or logic having boundaries defined by function or subroutine calls, branches points, APIs, or other technologies that provide for the partitioning or modularization of a particular processing or control function. Components can be combined via their interfaces to create a machine process. A component can be a packaged functional hardware unit as well as a part of a specific function, program, or task that is typically used in conjunction with other components to complete a larger process. Components can constitute either software components (e.g., code embodied on a machine-readable medium) or hardware components. A“hardware component” is a tangible unit capable of performing certain operations and can be configured or arranged in a certain physical manner. In various example embodiments, one or more computer systems (e.g., a standalone computer system, a client computer system, or a server computer system) or one or more hardware components of a computer system (e.g., a processor or a group of processors) can be configured by software (e.g., an application or application portion) as a hardware component that operates to perform certain operations as described herein. A hardware component can also be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware component can include dedicated circuitry or logic that is permanently configured to perform certain operations. A hardware component can be a special-purpose processor, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). A hardware component can also include programmable logic or circuitry that is temporarily configured by software to perform certain operations. For example, a hardware component can include software executed by a general-purpose processor or other programmable processor. Once configured by such software, hardware components become specific machines (or specific components of a machine) uniquely tailored to perform the configured functions and are no longer general-purpose processors. It will be appreciated that theWhere multiple hardware components are present in a simultaneous manner, communication can be achieved through signal transmission (e.g., over appropriate circuits and buses) between or among two or more of the hardware components. In embodiments where multiple hardware components are configured or instantiated at different times, communication between such hardware components can for example be achieved by storing information in memory structures to which the multiple hardware components have access, and retrieving that information as appropriate. For example, one hardware component can perform an operation and store output of that operation in a memory device to which it is communicatively coupled. A further hardware component can then access the memory device at a subsequent time to retrieve and process the stored output. The hardware components can also initiate communications with input or output devices, and can operate on resources (e.g., collection of information). The various operations of example methods described herein can be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute processor-implemented components that operate to perform one or more operations or functions described herein. As used herein, “processor- implemented component” refers to a hardware component implemented using one or more processors. Similarly, the methods described herein can be at least partially processor-implemented, with a particular processor or processors being an example of hardware. For example, at least some of the operations of a method can be performed by one or more processors or processor-implemented components. Moreover, a processor or processors can additionally operate to support performance of the relevant operations in a “cloud computing” environment or as a “software as a service” (SaaS). For example, at least some of the operations can be performed by a group of computers (as examples of machines including processors), with these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., an API). The performance of certain of the operations can be distributed among the processors, not only residing within a single machine, but deployed across a number of machines. In some example embodiments, the processors or processor-implemented components can be located in a single geographic location (e.g., within a home environment, an office environment, or a server farm). In other example embodiments, the processors or processor-implemented components can be distributed across a number of geographic locations.

[0205] “Computer-readable storage medium” refers to both machine-storage media and transmission media. Thus, the term includes both storage devices / media and carrier waves / modulated data signals. The terms “machine-readable medium,” “computer-readable medium,” and “device-readable medium” mean the same thing and can be used interchangeably in this disclosure.

[0206] “machine-storage medium” refers to a single or multiple storage devices and media (e.g., a centralized or distributed database, and associated caches and servers) that store executable instructions, routines, and data. Thus, the term should be taken to include, but not be limited to, solid-state memories, and optical and magnetic media, including memory internal or external to processors. Specific examples of machine-storage media, computer-storage media, and equipment-storage media include non-volatile memory, including by way of example semiconductor memory devices, e.g., erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGA, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms “machine-storage medium,” “equipment-storage medium,” “computer-storage medium,” and “device-storage medium” mean the same thing and can be used interchangeably in this disclosure. The terms “machine-storage medium,” “computer-storage medium,” and “device-storage medium” explicitly do not encompass transitory or propagating signals.

[0207] “non-transitory computer-readable storage medium” refers to a tangible medium that is capable of storing, encoding, or carrying the instructions for execution by a machine.

[0208] “signal medium” refers to any intangible medium that is capable of storing, encoding, or carrying the instructions for execution by a machine and that causes a machine to perform any of the disclosed methods and can include a digitally or an analogically modulated, spread-spectrum signal communication. The term “signal medium” shall be taken to include any form of a modulated data signal, carrier wave, and so on. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. The terms “transmission medium” and “signal medium” mean the same thing and can be used interchangeably in this disclosure.

Claims

1. A method of implementing child table replication, comprising: detecting, by a processor, an update to an equity table, the equity table comprising a plurality of equity table rows, the plurality of equity table rows being associated with a plurality of client identifiers, a plurality of consumer account identifiers, and a plurality of provider account identifiers, the update to the equity table being associated with a first equity table row of the plurality of equity table rows, the update to the equity table comprising a first client identifier of the plurality of client identifiers, a first consumer account identifier of the plurality of consumer account identifiers, or a first provider account identifier of the plurality of provider account identifiers; performing filtering on a data table based on the update to the equity table, the data table comprising a plurality of data table rows, the plurality of data table rows being associated with the plurality of client identifiers and comprising an equity column and data content, the equity column comprising the plurality of provider account identifiers; detecting an update to the equity column; and performing incremental replication of the data table by causing version-based replication to be performed. performing incremental replication further comprises:

2. The method of claim 1, wherein, updating a version identifier associated with the data table, adding a plurality of micro-partitions in the data table that reflect the update to the equity column, and replicating the plurality of data table rows to a plurality of provider accounts associated with the plurality of provider account identifiers in the equity table using the equity column. the equity column is populated or updated by a provider user via a client device.

3. The method of claim 1, wherein, the equity column comprises a virtual equity column.

4. The method of claim 1, wherein, detecting the update to the equity column further comprises:

5. The method of claim 4, wherein, determining that the update to the equity table comprises a change to one of the plurality of provider account identifiers in the equity table; and populating the virtual equity column based on the change to the one of the plurality of provider account identifiers in the equity table. performing filtering further comprises:

6. The method of claim 2, wherein, performing file-level filtering by: identifying, from the plurality of micro-partitions in the data table, a micro-partition having metadata associated with the first client identifier, each of the plurality of micro-partitions in the data table comprising one or more of the plurality of data table rows, and replicating the identified micro-partition to a first provider account associated with the first provider account identifier. each of the plurality of micro-partitions in the data table comprises metadata, the metadata comprising a minimum client identifier and a maximum client identifier.

7. The method of claim 6, wherein, the metadata is associated with the first client identifier when the first client identifier is within a range established by the minimum client identifier and the maximum client identifier.

8. The method of claim 7, wherein, the plurality of provider account identifiers identify provider accounts in a plurality of geographic regions.

9. The method of claim 2, wherein, the equity table further comprises a plurality of equity table columns storing the plurality of consumer account identifiers and the plurality of provider account identifiers.

10. The method of claim 2, wherein, 11. The method of claim 1, further comprising: receiving a replication request from the first provider account, the replication request comprising a first consumer account identifier. ​ 12. The method of claim 1, wherein, The update to the equity table includes adding a new equity table row to the equity table, deleting one of the equity table rows, or altering information included in one of the equity table rows.

13. The method of claim 12, wherein, Altering information included in one of the equity table rows includes altering the client identifier, the consumer account identifier, or the provider account identifier.

14. A system to implement sub-table replication, comprising: a processor; and a memory having instructions stored therein that, when executed by the processor, cause the system to perform operations comprising: detecting an update to an equity table, the equity table including a plurality of equity table rows, the equity table rows being associated with a plurality of client identifiers, a plurality of consumer account identifiers, and a plurality of provider account identifiers, the update to the equity table being associated with a first equity table row of the equity table rows, the update to the equity table including a first client identifier of the client identifiers, a first consumer account identifier of the consumer account identifiers, or a first provider account identifier of the provider account identifiers; performing a filter on a data table based on the update to the equity table, the data table including a plurality of data table rows, the plurality of data table rows being associated with the client identifiers and including an equity column and data content, the equity column including the provider account identifiers; detecting an update to the equity column; and performing an incremental replication of the data table by causing a version-based replication to be performed.

15. The system of claim 14, wherein, Performing an incremental replication further includes: updating a version identifier associated with the data table, adding a plurality of micro-partitions in the data table that reflect the update to the equity column, and replicating the data table rows to a plurality of provider accounts associated with the provider account identifiers in the equity table using the equity column.

16. The system of claim 14, wherein, The equity column is populated or updated by a provider user via a client device.

17. The system of claim 14, wherein, The equity column includes a virtual equity column, and wherein detecting the update to the equity column further includes: determining that the update to the equity table includes a change to one of the provider account identifiers in the equity table; and populating the virtual equity column based on the change to the one of the provider account identifiers in the equity table.

18. The system of claim 15, wherein, Performing a filter further includes: performing a file-level filter by: identifying micro-partitions having metadata associated with the first client identifier from the plurality of micro-partitions in the data table, each of the plurality of micro-partitions in the data table including one or more of the plurality of data table rows, and replicating the identified micro-partitions to a first provider account associated with the first provider account identifier.

19. The system of claim 18, wherein: each of the plurality of micro-partitions in the data table includes metadata including a minimum client identifier and a maximum client identifier.

20. The system of claim 19, wherein: The metadata is associated with the first client identifier when the first client identifier is within a range established by the minimum client identifier and the maximum client identifier.

21. The system of claim 14, wherein, The update to the equity table includes adding a new equity table row to the equity table, deleting one of the equity table rows, or altering information included in one of the equity table rows.

22. A non-transitory computer-readable storage medium comprising instructions that, when processed by a computer, configure the computer to perform operations comprising: detecting an update to an equity table, the equity table comprising a plurality of equity table rows, the equity table rows being associated with a plurality of client identifiers, a plurality of consumer account identifiers, and a plurality of provider account identifiers, the update to the equity table being associated with a first equity table row of the equity table rows, the update to the equity table including a first client identifier of the client identifiers, a first consumer account identifier of the consumer account identifiers, or a first provider account identifier of the provider account identifiers; performing filtering of a data table based on the update to the equity table, the data table comprising a plurality of data table rows, the plurality of data table rows being associated with the client identifiers and including an equity column and data content, the equity column including the provider account identifiers; detecting an update to the equity column; and performing incremental replication of the data table by causing version-based replication to be performed.

23. The non-transitory computer-readable storage medium of claim 22, wherein, Performing incremental replication further comprises: updating a version identifier associated with the data table, adding a plurality of micro-partitions in the data table that reflect the update to the equity column, and replicating the data table rows to a plurality of provider accounts associated with the provider account identifiers in the equity table using the equity column.

24. The non-transitory computer-readable storage medium of claim 22, wherein, The equity column is populated or updated by a provider user via a client device.

25. The non-transitory computer-readable storage medium of claim 22, wherein, The equity column comprises a virtual equity column.

26. The non-transitory computer-readable storage medium of claim 25, wherein, Detecting the update to the equity column further comprises: determining that the update to the equity table includes a change to one of the provider account identifiers in the equity table; and populating the virtual equity column based on the change to the one of the provider account identifiers in the equity table.

27. The non-transitory computer-readable storage medium of claim 23, wherein, Performing filtering further comprises: performing file-level filtering by: identifying a micro-partition from the plurality of micro-partitions in the data table that has metadata associated with the first client identifier, each of the plurality of micro-partitions in the data table comprising one or more of the plurality of data table rows, and replicating the identified micro-partition to a first provider account associated with the first provider account identifier.

28. The non-transitory computer-readable storage medium of claim 27, wherein: each of the plurality of micro-partitions in the data table comprises metadata, the metadata including a minimum client identifier and a maximum client identifier.

29. The non-transitory computer-readable storage medium of claim 28, wherein, the metadata is associated with the first client identifier when the first client identifier is within a range established by the minimum client identifier and the maximum client identifier.

30. The non-transitory computer-readable storage medium of claim 22, wherein, the update to the equity table includes adding a new equity table row to the equity table, deleting one of the equity table rows, or altering information included in one of the equity table rows.

Images