Change-aware snapshot replication

Abstract

The disclosed techniques and solutions may provide improved snapshot replication. Typically, an initial copy obtained using snapshot replication is periodically updated. However, if the data in the source data object does not change relative to the data in the replica data object, the update process may unnecessarily consume computing resources. The disclosed techniques check before obtaining a new snapshot to determine whether a snapshot copy is outdated. The check may be performed upon manual request or upon occurrence of a trigger, such as receiving a query to access a replica data object or according to a schedule. The information of the current and previous versions of the remote data object may be compared to determine whether the copy is outdated, such as a digest value of the content of the remote data object or a timestamp associated with the remote data object.

Description

Technical Field

[0001] The present disclosure generally relates to data replication. Certain embodiments relate to determining whether a source data object has changed relative to a target data object before copying data from the source data object to the target data object. Background Art

[0002] It is increasingly common for enterprises to store data in a variety of systems, including in one or more local systems and in one or more cloud-based systems. The systems may be of different types, such as storing data in different formats (e.g., a relational database versus a database storing JSON documents) or using different database management systems to store data (e.g., using software and / or hardware provided by different vendors). Even where data is stored in the same format and using software from the same vendor, there may be differences in what data is stored at a particular location and the schema used to store the data.

[0003] To help address these issues, various techniques have been used, including data replication and database federation. In a federated database environment, a request for a database operation (such as a query) can specify a source at a local database system or at a "remote" database accessed using data federation. In some cases, both local and remote data sources can be specified in the same query, such as a query with data retrieved from a local database table and data retrieved from a remote federated database system.

[0004] Data replication provides a strategic advantage to the business by enabling the target system to access and leverage data that originally resided in the source system, supplementing the information already stored locally within the target system itself. In this architecture, the source system essentially serves as a more authoritative or primary source of truth regarding the replicated data. This means that while the target system maintains its intrinsic data, it can also leverage the additional information replicated from the source system to enhance its data processing or analytical capabilities. However, given the status of the source system as a higher source of truth, any discrepancies or conflicts involving the replicated data are typically resolved in favor of the source system.

[0005] For various reasons, such as query performance, in some scenarios, data replication may be a better solution than data federation. However, the data of the source system is generally not "static" because data can be added to the source system, removed from the source system, or changed at the source system. Therefore, instead of copying the data from the source system to the target system once, a process is implemented to update the data at the target system based on the changes made at the source system.

[0006] One way to perform these updates is to implement real-time replication, where changes at the source system are transferred to the target system as they are made at the source system, typically using a log-based approach. While this can help ensure that the data in the target system is the same as the data in the source system, the real-time replication process can be complex and computationally expensive at both the source and target systems. A particular disadvantage of log-based approaches is that both the source and target systems typically need to be tightly integrated. For example, the target system is typically designed based on the implementation of the source system so that appropriate logs can be acquired and parsed. Typically, this results in real-time replication being difficult to implement between systems from different vendors.

[0007] Therefore, another type of replication that can be used is snapshot replication, in which all or selected data sources (such as specific tables identified for replication) have their contents periodically sent to a target system so that the target system can have a more recent version of the source system's data. However, because snapshot replication is not real-time replication, over time, the target system's data may become out of date compared to the source system's data. In addition, performing snapshot updates may be computationally expensive because, typically, during a snapshot update, all content identified for replication at the source system is sent to the target system. Therefore, there is room for improvement. Summary of the invention

[0008] This summary is provided to introduce some concepts in a simplified form, which are further described in the detailed description below. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0009] The disclosed techniques and solutions may provide improved snapshot replication. Typically, an initial replica obtained using snapshot replication is updated periodically. However, if the data in the source data object has not changed relative to the data in the replica data object, the update process may unnecessarily consume computing resources. The disclosed techniques perform a check before obtaining a new snapshot to determine whether the snapshot copy is out of date. The check may be performed upon manual request or upon a trigger, such as receiving a query to access the replica data object or according to a schedule. The current version and previous version information of the remote data object may be compared to determine whether the replica is out of date, such as a summary value of the content of the remote data object or a timestamp associated with the remote data object.

[0010] In one aspect, the present disclosure provides a process for performing a snapshot copy update after determining that a current snapshot copy is out of date compared to a remote data object. A first instance of a first data object type is created at a local data source. A first data set of the first instance of the first data object type is received from a remote data source. The first data set is data extracted from a remote data object of the remote data source. The data is stored in the first instance of the first object type. It is determined that the first instance of the first object type is out of date compared to the remote data object. A second data set of the first instance of the first data object type is received. The second data set is updated data for the remote data object. At least a portion of the first data set is replaced with data of the second data set in the first instance of the first object type.

[0011] The present disclosure also includes a computing system and a tangible non-transitory computer-readable storage medium configured to perform the above-described method or including instructions for performing the above-described method. As described herein, various other features and advantages may be incorporated into the technology as desired. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a diagram depicting an example database system that can be used to implement aspects of the disclosed technology.

[0013] Figure 2 Is a diagram that depicts a computing environment in which a database system can access data in a remote computing system using data federation (including through virtual tables) or using local tables with data replicated from the remote computing system.

[0014] Figure 3 It is a diagram showing a conventional data replication technique.

[0015] Figure 4 is a diagram and a flow chart of a snapshot replication process according to the present disclosure.

[0016] Figure 5 is a flow chart of the disclosed process of performing a query including a hint regarding a preferred behavior if a snapshot copy is determined to be out of date.

[0017] Figure 6 is a diagram showing remote and replica tables and data elements that can be used to determine whether the replica table is up to date with respect to the remote table.

[0018] Figure 7 is a flow chart of a process for performing a snapshot copy update after determining that a current snapshot copy is out of date compared to a remote data object.

[0019] Figure 8 is a diagram of an example computing system in which some described embodiments may be implemented.

[0020] Fig. 9 is an example cloud computing environment that can be used in conjunction with the techniques described herein. DETAILED DESCRIPTION

[0021] Example 1 - Overview

[0022] It is increasingly common for enterprises to store data in a variety of systems, including in one or more local systems and in one or more cloud-based systems. The systems may be of different types, such as storing data in different formats (e.g., a relational database versus a database storing JSON documents) or using different database management systems to store data (e.g., using software and / or hardware provided by different vendors). Even where data is stored in the same format and using software from the same vendor, there may be differences in what data is stored at a particular location and the schema used to store the data.

[0023] To help address these issues, various techniques have been used, including data replication and database federation. In a federated database environment, a request for a database operation (such as a query) can specify a source at a local database system or at a "remote" database accessed using data federation. In some cases, both local and remote data sources can be specified in the same query, such as a query with data retrieved from a local database table and data retrieved from a remote federated database system.

[0024] Data replication provides a strategic advantage to the business by enabling the target system to access and leverage data that originally resided in the source system, supplementing the information already stored locally within the target system itself. In this architecture, the source system essentially serves as a more authoritative or primary source of truth regarding the replicated data. This means that while the target system maintains its intrinsic data, it can also leverage the additional information replicated from the source system to enhance its data processing or analytical capabilities. However, given the status of the source system as a higher source of truth, any discrepancies or conflicts involving the replicated data are typically resolved in favor of the source system.

[0025] For various reasons, such as query performance, in some scenarios, data replication may be a better solution than data federation. However, the data of the source system is generally not "static" because data can be added to the source system, removed from the source system, or changed at the source system. Therefore, instead of copying the data from the source system to the target system once, a process is implemented to update the data at the target system based on the changes made at the source system.

[0026] One way to perform these updates is to implement real-time replication, where changes at the source system are communicated to the target system as they are made at the source system, typically using a log-based approach. While this can help ensure that the data in the target system is the same as the data in the source system, the real-time replication process can be complex and computationally expensive at both the source and target systems. A particular disadvantage of log-based approaches is that both the source and target systems typically need to be tightly integrated. For example, the target system is typically designed based on the implementation of the source system so that appropriate logs can be obtained and parsed. Typically, this results in real-time replication being difficult to implement between systems from different vendors.

[0027] Therefore, another type of replication that can be used is snapshot replication, in which all or selected data sources (such as specific tables identified for replication) have their contents periodically sent to a target system so that the target system can have a more recent version of the source system's data. However, because snapshot replication is not real-time replication, over time, the target system's data may become out of date compared to the source system's data. In addition, performing snapshot updates may be computationally expensive because, typically, during a snapshot update, all content identified for replication at the source system is sent to the target system. Therefore, there is room for improvement.

[0028] As a specific example of how a local database can access data at a remote data source, the SDA (Smart Data Access) data virtualization technology of SAP SE of Walldorf, Germany allows data in a "virtual table" of a local database system to access data that is replicated from a remote system to a local system, or to access data directly from a remote system through data federation. In some embodiments, a given virtual table can switch between accessing data from a local copy or accessing data directly from a remote system. Although the disclosed technology can be used in SAP SDA, the disclosed technology can be used in other scenarios involving snapshot (or "batch") replication from a remote system to a local system.

[0029] As discussed, snapshot replication can have lower overhead than real-time replication, but the process of performing snapshot updates can still be expensive and data can become stale, depending on how often the data changes and how often or if updated snapshots are obtained. Typically, snapshot updates are requested manually or on a schedule. The problem with manually requesting a snapshot update is that it requires affirmative action on the part of the user or the computing process. Thus, for example, a user may need to consider whether data from a remote data source may be out-of-date compared to when the last snapshot was taken before executing a query. However, the user may forget to perform this action, resulting in query results that may have out-of-date data. Alternatively, a user may request an update even if the data at the source system has not changed compared to the current snapshot at the local system, and incur the resulting computational overhead.

[0030] In other cases, the snapshot may be refreshed periodically, for example according to a schedule. However, this may suffer from the same problems as manually triggered snapshot updates. The update may be performed before the data has been changed, or before a "significant" number of changes have occurred, thereby incurring unnecessary computational costs, or queries may be executed against stale data before the scheduled update occurs.

[0031] The present disclosure provides techniques for improved snapshot replication. Specifically, the disclosed techniques allow a current snapshot of a replica data of a local system to be compared with the current state of subject data at a remote system. Rather than directly comparing data from the remote system and the replica system, summary data of the data at the remote system can be compared with summary data of the data in the replica of the local system. If the comparison indicates that a change has occurred, the local system can obtain an updated replica from the remote system. Thus, a snapshot update is performed only when the data at the remote system has changed.

[0032] The present disclosure provides specific examples of the disclosed technology implemented in a relational database system, such as for use in a process involving a local replica database table and a remote database table. However, the disclosed technology can be used with other types of data objects, such as storing data in formats such as JSON, XML, CSV, PARQUEST, BSON (binary JSON), Apache Kafka, Apache CASSANDRA, or SAP BW objects.

[0033] In some cases, the remote object and the replica object in which the data is stored are of the same type, while in other cases, the remote object and the replica object may be of different types. As explained later in this disclosure, a feature or characteristic of a first version of a remote data object (such as reflected in the replica) may be compared to a feature or characteristic of a second version of the remote data object (such as reflected in a current version of the object at the remote system), where if the value of the feature or characteristic is different, a change may be indicated and an update process may be triggered. If the value of the feature or characteristic of the version of the remote object used to generate or populate the replica object is available, then the feature or characteristic does not need to be calculated from the replica object.

[0034] Various types of summary data may be used to determine whether data maintained at a remote system has changed compared to the data in the snapshot copy at the local system. In one embodiment, a hash value of the snapshot copy may be calculated and compared to a hash value of the current version of the data at the remote system. In another embodiment, current table statistics for a table may be compared to table statistics associated with the snapshot copy. Other metadata about a particular data object may be used for comparison purposes, such as the date the table was last modified, or information about the size of the table.

[0035] The comparison can be initiated and performed according to various criteria. In some cases, the comparison can be performed when a query requesting data from a replica is received at the local system, or at a similar activity, such as when a query plan compilation or recompilation is requested. In another example, the comparison can be performed according to a schedule.

[0036] Depending on the implementation, the triggering of the comparison process and the update process may be performed at the remote system or the local system. In the case of checking whether a replica is up-to-date based on a query request, the comparison process may be initiated by the local system. However, the local system may send a data value to the remote system, where the remote system then determines whether the replica is up-to-date, or the local system may request a value from the remote system (or the remote system may send it a value), and the local system may then determine whether the replica is up-to-date. In the case of other types of triggers, such as determining whether a replica is up-to-date based on a schedule, the check may be initiated at the remote system or the local system.

[0037] The present disclosure generally continues to discuss determining whether a new snapshot copy should be obtained based on information about an entire data object (such as a relational database table). However, at least some aspects of the disclosed technology can be applied at a more granular level. For example, rather than determining whether the contents of an entire table are different from a snapshot copy, it can be determined whether one or more specific columns of the snapshot copy have different characteristics than the corresponding one or more columns of the current version of the remote data source for the copy. Similarly, metadata for a column (such as a modified date or size) can be used to determine whether a snapshot copy should be updated, rather than (or in addition to) looking at the data of the entire table.

[0038] Thus, the disclosed techniques may provide a number of advantages. By checking whether a snapshot copy may differ from the source data on the remote system, unnecessary data transfer and processing may be avoided compared to scenarios where a new snapshot is obtained based on a schedule or trigger event. The comparison used to determine changes and obtain a new snapshot may be automated, which may save a user from having to manually determine whether an updated snapshot should be obtained and have the update process performed. In addition, the disclosed techniques provide flexibility in processing queries, such as allowing those submitting queries to indicate what action should be taken if a snapshot copy is determined to be potentially out of date.

[0039] Example 2 describes an example database system that can be used to implement the disclosed technology. Example 3 provides an example of a virtual table, where the virtual table includes logical pointers that can be updated to point to different locations, including locations in a federated data source or a local data source (including a local table or a table maintained in a cache). However, it should be understood that the disclosed technology does not require the use of such a virtual table. That is, the disclosed technology can be applied to any implementation of a replica table of data using snapshot replication. Examples 4-8 more specifically describe the disclosed technology for performing change-aware snapshot replication.

[0040] Example 2 - Sample database schema

[0041] Figure 1 An example database environment 100 is shown. Database environment 100 may include client 104. Although a single client 104 is shown, client 104 may represent multiple clients. One or more clients 104 may be OLAP clients, OLTP clients, or a combination thereof.

[0042] The client 104 communicates with the database server 106. Through various subcomponents, the database server 106 can process requests for database operations, such as requests to store, read, or manipulate data (i.e., CRUD operations). The session manager component 108 can be responsible for managing connections between the client 104 and the database server 106, such as clients that communicate with the database server using a database programming interface (such as Java Database Connectivity (JDBC), Open Database Connectivity (ODBC), or Database Shared Library (DBSL)). Typically, the session manager 108 can manage connections with multiple clients 104 at the same time. The session manager 108 can perform functions such as creating new sessions for client requests, assigning client requests to existing sessions, and authenticating access to the database server 106. For each session, the session manager 108 can maintain a context that stores a set of parameters related to the session, such as settings related to committing database transactions or transaction isolation levels (such as statement-level isolation or transaction-level isolation).

[0043] For other types of clients 104, such as web-based clients (such as clients using the HTTP protocol or similar transport protocol), the client can interface with an application manager component 110. Although illustrated as a component of the database server 106, in other implementations, the application manager 110 can be external to the database server 106, but in communication with the database server 106. The application manager 110 can initiate new database sessions with the database server 106 in a manner similar to the session manager 108, and perform other functions.

[0044] The application manager 110 may determine the type of application requesting the database operation and mediate execution of the request at the database server 106, such as by invoking or executing a procedure call, generating a query language statement, or converting data between formats usable by the client 104 and the database server 106. In a particular example, the application manager 110 receives a request for a database operation from the client 104, but does not store information related to the request, such as state information.

[0045] Once a connection is established between the client 104 and the database server 106, including when established through the application manager 110, the client request is typically executed using a query language, such as structured query language (SQL). In executing the request, the session manager 108 and the application manager 110 may communicate with the query interface 112. The query interface 112 may be responsible for creating a connection with the appropriate execution component of the database server 106. The query interface 112 may also be responsible for determining whether the request is associated with a previously cached statement or stored procedure, and calling a stored procedure or associating a previously cached statement with the request.

[0046] At least some types of requests for database operations (such as statements that write data or manipulate data in a query language) can be associated with a transaction context. In at least some embodiments, each new session can be assigned to a transaction. Transactions can be managed by a transaction manager component 114. The transaction manager component 114 can be responsible for operations such as coordinating transactions, managing transaction isolation, tracking running and closed transactions, and managing the submission or rollback of transactions. When performing these operations, the transaction manager 114 can communicate with other components of the database server 106.

[0047] The query interface 112 may communicate with a query language processor 116 (such as a structured query language processor). For example, the query interface 112 may forward query language statements or other database operation requests from the client 104 to the query language processor 116. The query language processor 116 may include a query language executor 120, such as an SQL executor, which may include a thread pool 124. Some requests for database operations or components thereof may be executed directly by the query language processor 116. Other requests or components thereof may be forwarded by the query language processor 116 to another component of the database server 106. For example, transaction control statements (such as commit or rollback operations) may be forwarded by the query language processor 116 to the transaction manager 114. In at least some cases, the query language processor 116 is responsible for executing operations (e.g., select, update, delete) that retrieve or manipulate data. Other types of operations (such as queries) may be sent by the query language processor 116 to other components of the database server 106. The query interface 112 and the session manager 108 may maintain and manage context information associated with requests for database operations. In particular embodiments, query interface 112 may maintain and manage context information for requests received via application manager 110 .

[0048] When the session manager 108 or the application manager 110 establishes a connection between the client 104 and the database server 106, a client request such as a query can be assigned to a thread of the thread pool 124, such as using the query interface 112. In at least one embodiment, a thread is associated with a context for performing processing activities. The thread can be managed by the operating system of the database server 106, or by another component of the database server, or in combination with another component of the database server. Typically, at any point, the thread pool 124 contains multiple threads. In at least some cases, the number of threads in the thread pool 124 can be dynamically adjusted, such as in response to the activity level at the database server 106. In certain aspects, each thread of the thread pool 124 can be assigned to multiple different sessions.

[0049] When a query is received, the session manager 108 or application manager 110 may determine whether an execution plan for the query already exists, such as in the plan cache 136. If a query execution plan exists, the cached execution plan may be retrieved and forwarded to the query language executor 120, such as using the query interface 112. For example, the query may be sent to an execution thread of a thread pool 124 determined by the session manager 108 or application manager 110. In a particular example, the query plan is implemented as an abstract data type.

[0050] If the query is not associated with an existing execution plan, the query may be parsed using a query language parser 128. The query language parser 128 may, for example, check the query language statements of the query to ensure that they have the correct syntax and confirm that the statements are otherwise valid. For example, the query language parser 128 may check whether the tables and records specified in the query language statements are defined in the database server 106.

[0051] The query may also be optimized using a query language optimizer 132. The query language optimizer 132 may manipulate elements of a query language statement to allow the query to be processed more efficiently. For example, the query language optimizer 132 may perform operations such as unnesting a query or determining an optimized execution order for various operations in the query (such as operations within a statement). After optimization, an execution plan may be generated or compiled for the query. In at least some cases, the execution plan may be cached, such as in a plan cache 136, and the execution plan may be retrieved (such as by a session manager 108 or an application manager 110) if the query is received again.

[0052] For purposes of the present disclosure, one task that may be performed by the query language optimizer 132 is to determine where a request for a database operation or portion thereof should be executed. For example, a complex query may be submitted that reads data from multiple data sources. At least one of the data sources may be a virtual table, and the request may be executed on a replica table on the database server 106, or the request may be sent to a remote system via data federation, and the results returned to the database server.

[0053] Once a query execution plan is generated or received, the query language executor 120 can oversee the execution of the query execution plan. For example, the query language executor 120 can call the appropriate subcomponents of the database server 106.

[0054] When executing a query, the query language executor 120 can call a query processor 140, which can include one or more query processing engines. The query processing engine can include, for example, an OLAP engine 142, a connection engine 144, an attribute engine 146, or a calculation engine 148. The OLAP engine 142 can, for example, apply rules to create an optimized execution plan for an OLAP query. The connection engine 144 can be used to implement relational operators, typically for non-OLAP queries, such as joins and aggregation operations. In a particular embodiment, the attribute engine 146 can implement column data structures and access operations. For example, the attribute engine 146 can implement merge functions and query processing functions, such as scanning columns.

[0055] In some cases, such as if a query involves complex or internally parallelized operations or sub-operations, the query executor 120 may send the operations or sub-operations of the query to a job executor component 154, which may include a thread pool 156. The execution plan for the query may include multiple plan operators. In a particular embodiment, each job execution thread of the job execution thread pool 156 may be assigned to a separate plan operator. The job executor component 154 may be used to execute at least a portion of the operators of the query in parallel. In some cases, the plan operator may be further divided and parallelized, such as having the operations access different parts of the same table simultaneously. Using the job executor component 154 may increase the load on one or more processing units of the database server 106, but may improve the execution time of the query.

[0056] The query processing engine of the query processor 140 can access data stored in the database server 106. The data can be stored in a row store 162 in a row-by-row format, or in a column store 164 in a column-by-column format. In at least some cases, the data can be transformed between the row-by-row format and the column-by-column format. Certain operations performed by the query processor 140 can access or manipulate data in the row store 162, the column store 164, or at least for certain types of operations (such as joins, merges, and subqueries), access or manipulate data in both the row store 162 and the column store 164. In at least some aspects, the row store 162 and the column store 164 can be maintained in main memory.

[0057] Persistence layer 168 may communicate with row store 162 and column store 164. Persistence layer 168 may be responsible for actions such as committing write transactions, storing redo log entries, rolling back transactions, and periodically writing data to storage to provide persistent data 172.

[0058] When executing a request for a database operation (such as a query or transaction), the database server 106 may need to access information stored at another location (such as another database server). The database server 106 may include a communication manager 180 component for managing such communications. The communication manager 180 may also mediate communications between the database server 106 and the client 104 or application manager 110 when the application manager is located outside the database server.

[0059] In some cases, database server 106 may be part of a distributed database system that includes multiple database servers. At least a portion of the database servers may include some or all of the components of database server 106. In some cases, the database servers of the database system may store multiple copies of data. For example, a table may be replicated at more than one database server. Additionally or alternatively, information in the database system may be distributed among multiple servers. For example, a first database server may save a copy of a first table, and a second database server may save a copy of a second table. In other embodiments, information may be divided among the database servers. For example, a first database server may save a first portion of a first table, and a second database server may save a second portion of the first table.

[0060] In executing requests for database operations, database server 106 may need to access other database servers or other information sources within the database system or at external systems (such as external systems where remote data objects are located). Communications manager 180 can be used to mediate such communications. For example, communications manager 180 can receive and route requests for information from components of database server 106 (or from another database server), and receive and route replies.

[0061] The database server 106 may include components for coordinating data processing operations involving remote data sources. Specifically, the database server 106 includes a data federation component 190 that at least partially processes requests to access data maintained at a remote system. In performing its functions, the data federation component 190 may include one or more adapters 192, wherein the adapter may include logic, settings, or connection information that can be used to communicate with the remote system, such as for obtaining information to help generate a virtual parameterized data object or use the virtual parameterized data object to perform a request for data (such as issuing a request to the remote system for data accessed using the corresponding parameterized data object of the remote system). Examples of adapters include "connectors" as implemented in technology available from SAP SE of Walldorf, Germany. In addition, the disclosed technology can use the underlying technology of data federation technology, such as SAP SE's Smart Data Access (SDA) and Smart Data Integration (SDI).

[0062] Example 3 - Example virtual table including an updateable logical pointer

[0063] Figure 2 A computing environment 200 is shown in which the disclosed embodiments may be implemented. Figure 2 The basic computing environment 200 includes a number of features that may be common to different embodiments of the disclosed technology, including one or more applications 208 that may access a central computing system 210, which may be a cloud computing system. The central computing system 210 is shown as a monolithic / single system, but it should be understood that, particularly in a cloud environment, the central computing system may include multiple computing systems that function together as a single system. For example, the central computing system 210 may be implemented as multiple "nodes," including an anchor node and zero or more non-anchor nodes. The central computing system 210 may also be a more typical "distributed" database system, including a master node and one or more worker nodes.

[0064] Central computing system 210 may function in this way by providing access to data stored in one or more remote database systems 212. Remote database systems 212, in turn, may be accessed by one or more applications 214. In some cases, application 214 may also be application 208. That is, some applications may only (directly) access data in central computing system 210, some applications may only access data in remote database systems 212, and other applications may access data in both the central computing system and the remote database systems.

[0065] The central computing system 210 may include a query processor 220. The query processor 220 may include multiple components, including a query optimizer 222 and a query executor 224. The query optimizer 222 may be responsible for determining a query execution plan 226 for a query to be executed using the central computing system 210. The query plan 226 generated by the query optimizer 222 may include a logical plan indicating, for example, the order of operations (e.g., joins, projections) to be performed in the query and a physical plan for implementing such operations. Once developed by the query optimizer 222, the query plan 226 may be executed by the query executor 224. The query plan 226 may be stored in a query plan cache 228 as a cached query plan 230. When a query is resubmitted for execution, the query processor 220 may determine whether there is a cached query plan 230 for the query. If so, the cached query plan 230 may be executed by the query executor 224. If not, the query optimizer 222 generates the query plan 226. In some cases, cached query plan 230 may be invalidated, such as if changes are made to the database schema or at least a component of the database schema (eg, a table or view) used by the query.

[0066] The data dictionary 234 (which may also be referred to as a system catalog) may maintain one or more database schemas for the central computing system 210. In some cases, the central computing system 210 may implement a multi-tenant environment, and different tenants may have different database schemas. In at least some cases, at least some database schema elements may be shared by multiple database schemas.

[0067] The data dictionary 234 may include definitions (or schemas) for different types of database objects, such as a schema for a table or view. Although the following discussion refers to tables for ease of explanation, it should be understood that the discussion can be applied to other types of database objects, particularly database objects associated with retrievable data, such as materialized views. A table schema may include information such as the name of the table, the number of attributes (or columns or fields) in the table, the names of the attributes, the data types of the attributes, the order in which the attributes should be displayed, primary key values, foreign keys, associations with other database objects, partition information, or replication information.

[0068] The table schemas maintained by the data dictionary 234 may include a local table schema 236, which may represent tables that are primarily maintained on the central computing system 210. The data dictionary 234 may include a replica table schema 238, which may represent tables for which at least a portion of the table data is stored in the central computing system 210 (or is primarily managed by the database management system of the central computing system, even if stored outside the central computing system, such as in a data lake or another cloud service). Tables having data associated with the replica table schema 238 will typically have their data updated periodically from a source table (such as a remote table 244 of a data repository 242 of a remote database system 212).

[0069] Replication may be accomplished using one or both of the replication service 246 of the remote database system 212 or the replication service 248 of the central computing system 210. In a specific example, the replication service may be a Smart Data Integration (SDI) service, SAP Landscape Transformation Replication Server, SAP Data Services, SAP Replication Server, SAP Event Stream Processor, or SAP HANA Direct Extractor Connect, all of which are from SAP SE of Walldorf, Germany. As discussed, replication may use real-time replication technology, or may use snapshot / batch replication.

[0070] In some cases, data in remote database system 212 can be accessed by central computing system 210 without duplicating data from the remote database system, such as using federation techniques. Data dictionary 234 can store virtual table schema 252 for virtual tables that map to remote tables, such as remote table 244 of remote database system 212. Data in remote table 244 can be accessed using federation service 256, such as using the Smart Data Access protocol of SAP SE of Walldorf, Germany. Federation service 256 can be responsible for converting query operations into a format that can be processed by the appropriate remote database system 212, sending query operations to the remote database system, receiving query results, and providing query results to query executor 224.

[0071] The data dictionary 234 may include an updateable virtual table schema 260 having an updateable logical pointer 262. The updated virtual table schema 260 may optionally be associated with state information 264. The table pointer 262 may be a logical pointer for identifying which table should be accessed for data of the corresponding virtual table schema 260. For example, depending on the state of the table pointer 262, the table pointer may point to a remote table 244 of a remote database system 212 or a replica table 266 (which may be generated from the remote table 244) in a data repository 268 located in the central computing system 210. The data repository 268 may also store data for a local table 270, which may be defined by the local table schema 236.

[0072] Table pointer 262 may change between remote table 244 and replica table 266. In some cases, a user may manually change the table pointed to by table pointer 262. In other cases, table pointer 262 may change automatically, such as in response to detecting a defined condition.

[0073] The state information 264 may include an indicator identifying the virtual table schema 260 as being associated with the remote table 244 or the replica table 266. The state information 264 may also include information about the replication state of the replica table 266. For example, once a request is made to change the table pointer 262 to point to the replica table 266, it may take time before the replica table is ready for use. The state information 264 may include whether the replication process has started, has completed, or the progress status of generating the replica table 266.

[0074] Changes to the updateable virtual table schema 260 and management of the replica tables 266 associated with the virtual table schema may be managed by a virtual table service 272. Although illustrated as a separate component of the central computing system 210, the virtual table service 272 may be incorporated into other components of the central computing system 210, such as the query processor 220 or the data dictionary 234.

[0075] When executing a query, the query is processed by query processor 220, including executing the query using query executor 224 to obtain data from one or both of data repository 242 of remote database system 212 or data repository 268 of central computing system 210. Query results may be returned to application 208. Query results may also be cached, such as in cache 278 of central computing system 210. The cached results may be represented as cache view 280 (e.g., materialized query results).

[0076] The application 214 can access data in the remote database system 212, such as through the session manager 286. The application 214 can modify the remote table 244. The changes made by the application 214 are reflected in the remote table when the table pointer 262 of the updateable virtual table schema 260 references the remote table 244. When the table pointer 262 references the replica table 266, the changes made by the application 214 can be reflected in the replica table using the replication service 246 or the replication service 248.

[0077] Example 4 - Example Copy Technique

[0078] Figure 3 A prior art technique for replicating data from a remote system 304 to a local system 308 is shown. The remote system 304 includes a remote table 312. The data from the remote table 312 is replicated to a replica table 316 of the local system 308.

[0079] Data from the remote table 312 can be replicated to the replica table 316 in a variety of ways. The real-time replication process 320 sends incremental data changes made to the remote table 312 to the replica table 316 as they occur, such as using techniques that may include change data capture or trigger-based replication. As mentioned in Example 1, while real-time replication can be advantageous in helping to ensure that the replica table 316 is up-to-date with respect to the remote table 312, it can be complex to implement and require significant computing resources.

[0080] Figure 3 Also shown is a snapshot replication process 330. In the snapshot replication process 330, snapshots 334 of the remote table 312 are periodically obtained. Typically, the snapshot 334 includes all the data of the remote table 312 at the point in time when the snapshot was taken. The snapshot data 334 can be sent to the local system 308. The current replica table 316 can be replaced with the snapshot data 334.

[0081] As shown, snapshots are typically obtained periodically, so process 330 can be referred to as periodic replication. Since replication is periodic, a new snapshot 334 is obtained after a delay period 340. The delay period 340 can be a delay between schedules, events, or manual update requests. As mentioned in Example 1, snapshot replication can be simpler to implement and can be less resource intensive than real-time replication, but the presence of the delay period 340 can cause queries at the local system 308 to retrieve data that is outdated compared to the current version of the remote table 312.

[0082] Example 5 - Example Snapshot Replication with Update Check

[0083] Figure 4 FIG. 2 shows a snapshot replication process according to the present disclosure. Figure 3 similar, Figure 4 The system 404 includes a remote table 408 and a local system 412 includes a local replica table 416. The local replica table 416 is a snapshot of the remote table 408 at a specific point in time. A snapshot replication process 420 is used to replicate data from the remote table 408 to the replica table 416.

[0084] Figure 4Details of the snapshot copy process 420 are shown. The snapshot copy process 420 can begin at 424, where a determination is made as to whether a particular event has occurred that causes the process to proceed further. Examples of events can be: a request such as from a user or application process to determine whether the local table 416 should be updated with a more recent snapshot of the remote table 408, a scheduled event such as a specific instance triggered by a specific date / time or the passage of time according to a schedule such as determining whether the snapshot should be updated every hour, a query involving the local table 416, or an indication that the data of the remote table 408 has changed.

[0085] If it is determined at 424 that the event has not occurred, process 420 can return to 424 until it is determined that the event has occurred. If it is determined at 424 that the event has occurred, process 420 can proceed to 428. At 428, it is determined whether the data in replica table 416 is still consistent with the data in remote table 408. For the resources required to copy data, send data, and compare data, it would be expensive to perform a comparison of the actual data in both remote table 408 and replica table 416, making it more efficient to simply perform a snapshot update. Therefore, the comparison at 428 typically considers information about remote table 408 and replica table 416, but does not compare all the data of the two tables. Details about what types of data can be compared at 428 are discussed in Example 7.

[0086] If it is determined at 428 that the replica table 416 is consistent with the remote table 408 based on at least the comparison criteria, the process 420 can return to 424. If it is determined at 428 that the replica table 416 is inconsistent with the remote table 408, the contents of the table can be synchronized at 432. Synchronizing the contents at 432 can include replacing the data in the replica table 416 with data from a new snapshot of the contents of the remote table 408.

[0087] Optionally, metadata of remote table 408, metadata of replica table 416, or metadata of both tables may be updated at 440. Updating metadata at 440 may include updating values ​​used for comparison at 428. For example, a new hash value, last modified date, table size, or set of statistics for replica table 416 (corresponding to metadata of remote table 408 at the time of the snapshot) may be stored.

[0088] Example 6 - Example query processing with update check and optional custom processing

[0089] As explained in Example 1, in some cases, a query request that includes access to a local copy table may trigger a check to determine whether the snapshot copy is up to date according to certain criteria, such as corresponding to Figure 4Event 424 of process 420. Typically, if no changes are detected, the query can be performed using the current snapshot copy. However, if the snapshot copy is determined to be outdated, a different process can be implemented.

[0090] If the snapshot copy is determined to be out of date, the query may be executed using the current snapshot copy, even though some data accessed by the query may be out of date. As another option, the query may be rejected if the snapshot copy is determined to be out of date, or the query may be queued pending completion of the snapshot update. A given behavior may be "hard-coded" into the system, or a default behavior may be specified, but may be selectively overridden.

[0091] In a particular example, a particular query may directly specify the desired behavior. That is, a query may include "hints," such as keywords that indicate to the query executor or query optimizer what behavior is expected. A query hint may indicate that the query should be executed without delay on the current version of the snapshot copy, even though the data may be out of date compared to the remote table. A query hint may alternatively indicate that query processing should be delayed until an updated snapshot is obtained. In another example, a query hint may be used to indicate that a query should be executed using a remote table rather than a snapshot copy, such as using data federation techniques. In yet another example, a query hint may specify that a failure message should be returned if the snapshot is not up to date.

[0092] Figure 5 An example process 520 is shown in which the query executor 514 executes the query 504 based on default behavior or using non-default behavior if a query hint is provided. Although described as being performed by the query executor, at least some operations of the process 520 may be performed by a query optimizer.

[0093] A query is received at 524. A determination is made at 528 that the query accesses a replica table. At 532, a check is performed to determine if the replica table is up-to-date with respect to the remote table. If so, the query is executed at 536 using the current replica table, and process 520 may end at 540.

[0094] If it is determined at 532 that the replica table is not up to date, a check is performed at 544 to determine whether the query 504 includes a hint 506 indicating that the query should be executed on the current version of the replica table, rather than the default behavior of waiting until an updated snapshot is obtained. If it is determined at 544 that the query 504 includes the hint 506, the query is processed using the existing replica at 548, and the process 520 can end at 540.

[0095] If it is determined at 544 that query 504 does not include hint 506, then at 552 it is determined whether the query includes hint 508 indicating that if the local copy table is out of date, the query should be executed on the remote table. If hint 508 exists, then at 556 the query is executed using the remote table, and process 520 can end at 540.

[0096] If it is determined at 552 that query 504 does not include hint 508, the query can be queued at 560 until the local copy table is updated. Once the local copy table is updated, query 504 can be executed at 564 and process 520 can end at 540.

[0097] Note that process 520 is merely one example of how the disclosed technology may be used. In other embodiments, only a check for hint 506 or hint 508 is implemented, and the database system may only support hint 506 or hint 508. Similarly, in process 520, the default behavior is to delay query execution until an updated snapshot of the replica table is obtained. However, a process may have a default behavior of executing the query using the current out-of-date replica, and may use hints to determine whether an alternative execution should be delayed pending updates or whether a remote table should be used, using a join query execution. If the local replica table is out-of-date, another process may have a default behavior of executing the query using a join using the remote table, and may use hints to determine whether an alternative execution may be performed on the local system after receiving the updated snapshot, or whether the query should be executed on the local system using the current but out-of-date replica table.

[0098] Example 7 - Example element for determining whether two data objects are different or triggering an update analysis process

[0099] Figure 6 Various table characteristics are shown that can be used to determine whether a replica table is up-to-date compared to a remote table from which data is copied to the replica table. Figure 6 Included is a remote system 604 having a remote table 606 and a local system 608 having a replica table 610 corresponding to a snapshot of the remote table.

[0100] Remote table 606 is shown as having content 620. Content 620 may include one or more semantic attributes 624 and one or more synchronization attributes 626. Semantic attributes 624 provide data "about" a particular entity represented as an entry in remote table 606. For example, an entry in remote table 606 may correspond to an employee, a customer, or a sales transaction. In the case of an employee, semantic attributes 624 may include attributes such as name, employee identification number, address, job title, or salary rate.

[0101] Synchronization attributes 626 are attributes that are specifically used or can be used to determine whether the data in the remote table 606 is different from the data in the replica table 610. Synchronization attributes 626 can be, but need not be, semantic attributes 624. A specific example of a synchronization attribute 626 is a timestamp, such as a timestamp indicating when a particular row of a table was last added, deleted, or modified. The content 640 of the replica table 610 can include semantic attributes 644 and synchronization attributes 646 corresponding to the semantic attributes 624 and 626, but if the replica table is not up-to-date with respect to the remote table, the values ​​of the attributes may differ between the tables 606, 610. The replica table 610 can include the last modified data as the synchronization attribute 644, where the value corresponds to the last modification date of the remote table 606 at the most recent snapshot. Therefore, determining whether the replica table 610 is out of date can be performed by querying the remote table 606 to determine whether the remote table contains a record with a value of the timestamp synchronization attribute 626, where the value of the timestamp synchronization attribute 626 is later than the most recent value reflected in the timestamp synchronization attribute 646 of the replica table 610.

[0102] Instead of including the "last modified" date as a synchronization attribute 626, the overall last modified date 632a of the remote table 606 can be stored in metadata 630 associated with the remote table and having metadata elements 632 (shown as 632a-632g). The replica table 608 includes metadata 650 having metadata elements 652 (shown as metadata elements 652a-652g) and can include the last modified date 652a associated with the remote table 606 during the most recent snapshot. When a request is made to check whether the remote table 606 has changed relative to the replica table 610, the metadata 630 can be queried to determine whether the last modified date 632a is the same as the last modified date 652a of the metadata 650. If the values ​​are not the same, an updated replica can be obtained.

[0103] As previously described, in some scenarios, the determination of whether a new snapshot should be obtained for the replica table 610 may be performed on a more granular basis than the entire remote table 606. For example, it may be that changes to some semantic attributes 624 of the remote table 606 are of greater concern than changes to other semantic attributes 624. In this case, when the semantic attributes 624 of particular concern have changed compared to the corresponding semantic attributes 644 of the replica table 610, it may be necessary to update the snapshot. Therefore, the metadata 630 may include a last modified date 632b for a particular semantic attribute 624 (such as a particular column of the remote table 606), and the metadata 650 of the replica table 610 may include a last modified date 652b for the corresponding semantic attribute 646. The determination of whether the replica table 610 is current may then be performed by determining whether the last modified date 632b of a given semantic attribute 624 is the same as the last modified date 652b of the corresponding semantic attribute 644.

[0104] Table statistics 632c for remote table 606 reflected in metadata 630 may also be used to determine when the remote table may have different content than that in replica table 610. While metadata 630 stores current statistics, metadata 650 for replica table 610 includes statistics 652c that were copied from metadata 630 when the last snapshot update occurred. While any suitable type of table statistics 632c may be used, examples of table statistics include:

[0105] Table size: The size of a table in memory, which may include memory for table data as well as memory for structures such as indexes or system administration.

[0106] Record count: The number of records (rows) in the table. The average record length can also be tracked.

[0107] Column statistics: Information about the data stored in each column (attribute) of a table, such as the number of distinct values, maximum and minimum values, most common value, null count, and average data size.

[0108] Partition information: Information about how the table is partitioned, which can include the number of partitions and the number of rows or columns in each partition, or the overall size (memory or disk) of each partition.

[0109] Index statistics: Information such as the index identifier, index size, and the number of leaf nodes in the index.

[0110] Histogram: Information about the distribution statistics of column values.

[0111] Size on disk / clustering information: Information about the size on disk of a particular table and about which rows with similar column values ​​are stored close to each other on disk.

[0112] Note that some of the table statistics 632c may have dependencies on the remote system 604. For example, information about memory usage, especially for index or system management, may be specific to the implementation of the remote system 604, and the local system 608 may not have the same memory usage, even if the data in the replica table 610 is the same as the data of the remote table 606. However, this does not prevent the statistics 632c from being used to determine whether the replica table 610 may be out of date, because the comparison is made between two states of the remote table 606 at the remote system 604, rather than between the actual statistics of the replica table 610 and the statistics of the remote table 606. In other words, such metadata 650 corresponds to the metadata 630 at the time of the snapshot.

[0113] The contents of remote table 606 can be used to compare table states in other ways. For example, the contents of remote table 606 (i.e., the actual values ​​stored in the remote table according to the definition of the remote table) can be provided as input to a hash algorithm to produce a hash value 632d. The hash value 632d can be provided to the local system 608 and stored in metadata 650 as a hash value 652d. When determining whether the replica table 610 has the same contents as the remote table 606, the current hash value 632d of the remote table can be compared with the hash value 652d of the most recent snapshot copy for the replica table.

[0114] The current hash value 632 of the remote table 606 can be determined (calculated) at different times, depending on the specific implementation decision. For example, the hash value 632d can be determined when the local system 608 requests it, such as as part of a process of checking whether an updated snapshot copy should be obtained. In other cases, the hash value 632d can be updated periodically at the remote system 604, including when the remote table 606 changes, although this may involve significant overhead for tables with frequently changing data. On the other hand, periodic updating of the hash value 632d may be sufficient, although if the remote table has actually changed, but the updated hash value has not yet been calculated, it may create a risk of a "false negative" indication that the replica table 610 is not different from the remote table 606.

[0115] Although examples using hash algorithms and hash functions are discussed, similar types of techniques may be used, including techniques that are less computationally expensive, such as using checksums. As with hash values, in at least some cases, the checksum may be calculated using the entire contents of the remote table 606. Hash values ​​and checksums are examples of a more general class of digests or digest values, where other types of digests may also be used in the disclosed techniques. Furthermore, although hash values ​​and checksums may be calculated on table contents represented in a table, they may alternatively be calculated in another manner, such as on a binary representation of the table contents.

[0116] One or both of metadata 630 or metadata 650 may include a synchronization schedule 632e, 652, which may be used by synchronization logic 670 of remote system 604 or synchronization logic 672 of local system 608, respectively. More typically, metadata 650 includes a synchronization schedule 652e, and a check is performed according to the schedule to determine whether an updated snapshot copy should be obtained. In another embodiment, remote system 604 may periodically send updated snapshot copies to local system 608, such as according to schedule 632e of metadata 630. The schedule may be expressed in various ways, such as specifying that a check to obtain a new snapshot copy at a specific date / time should be performed according to a periodic schedule (such as every hour or every day), or performing the check within a specified window, such as using one or both of a "not later than" date or a "not earlier than" date. Synchronization (or, more specifically, checking to determine whether synchronization should be performed) may also be performed as needed, including based on the last synchronization date 632f, 652f of metadata 630, 650. For example, the command may be "if LAST_SYNC_DATE > 1 day then sync".

[0117] Metadata 630 or metadata 650 may also include triggers 632g, 652g. In this case, "trigger" does not refer to a trigger-based database process, such as a trigger-based update or replication. Instead, "trigger" refers to some other event that makes it possible to determine whether the replica table 610 is consistent with the remote table 606. Synchronous scheduling or performing checks when receiving queries that include operations on the replica table 610 can be considered as examples of triggers 632g, 652g, although they have been enumerated and described separately in this discussion. Other examples of triggers 632g, 653g can be a threshold number of changes made to the remote table 606 or a threshold number of queries that access the replica table 610 are received. Compared to static scheduling, such triggers 632g, 652g can introduce dynamic aspects to determine whether a replica update should be performed, because if there are more updates to the remote table 606 (indicating that the replica table 610 may be more significantly out of date) or if there is greater activity on the replica table (indicating that the replica table may be more interesting, and therefore prioritize update checks compared to "cold" data), then update checks can occur frequently.

[0118] Although the specification (including this example 7) generally describes sending a copy of the entire contents of the remote table 606 when the contents are determined to be different from the replica table 610, in other cases, less than all data may be sent, such as data that has been added since the last replica snapshot was obtained. As a specific example, a table may be designated as append-only. For these types of data, only the rows appended after the date of the last replica snapshot need to be sent to the local system 608.

[0119] Example 8 - Example Operation

[0120] Figure 7 700 is a flow chart of a process 700 for performing a snapshot copy update after determining that a current snapshot copy is out of date compared to a remote data object. At 705, a first instance of a first data object type is created at a local data source. At 710, a first data set of the first instance of the first data object type is received from a remote data source. The first data set is data extracted from a remote data object of the remote data source. At 715, the data is stored in the first instance of the first object type. At 720, it is determined that the first instance of the first object type is out of date compared to the remote data object. At 725, a second data set of the first instance of the first data object type is received. The second data set is updated data for the remote data object. At 730, at least a portion of the first data set is replaced with data of the second data set in the first instance of the first object type.

[0121] Example 9 - Calculation System

[0122] Figure 8 A generalized example of a suitable computing system 800 is depicted in which the described innovations may be implemented. Computing system 800 is not intended to suggest any limitation as to the scope of use or functionality of the present disclosure, as the innovations may be implemented in various general-purpose or special-purpose computing systems.

[0123] refer to Figure 8 , the computing system 800 includes one or more processing units 810, 815 and memories 820, 825. Figure 8 , the basic configuration 830 is included within the dashed line. Processing units 810, 815 execute computer executable instructions, such as for implementing the database environment and associated methods described in Examples 1-8. The processing unit can be a general-purpose central processing unit (CPU), a processor in an application-specific integrated circuit (ASIC), or any other type of processor. In a multi-processing system, multiple processing units execute computer executable instructions to increase processing power. For example, Figure 8 A central processing unit 810 and a graphics processing unit or co-processing unit 815 are shown. Tangible memory 820, 825 may be volatile memory (e.g., registers, cache, RAM), non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.), or some combination of both accessible by the processing unit(s) 810, 815. The memory 820, 825 stores software 880 implementing one or more innovations described herein in the form of computer-executable instructions suitable for execution by the processing unit(s) 810, 815.

[0124] The computing system 800 may have additional features. For example, the computing system 800 includes storage 840, one or more input devices 850, one or more output devices 860, and one or more communication connections 870. An interconnection mechanism (not shown) such as a bus, controller, or network interconnects the components of the computing system 800. Typically, operating system software (not shown) provides an operating environment for other software executed in the computing system 800 and coordinates the activities of the components of the computing system 800.

[0125] Tangible storage 840 may be removable or non-removable and include disks, tapes or cassettes, CD-ROMs, DVDs, or any other medium that can be used to store information in a non-transitory manner and that can be accessed within computing system 800. Storage 840 stores instructions for software 880 implementing one or more innovations described herein.

[0126] The input device(s) 850 may be a touch input device (such as a keyboard, mouse, pen, or trackball), a voice input device, a scanning device, or another device that provides input to the computing system 800. The output device(s) 860 may be a display, a printer, a speaker, a CD burner, or another device that provides output from the computing system 800.

[0127] Communication connection(s) 870 enable communication with another computing entity (such as another database server) via a communication medium. The communication medium transmits information such as computer executable instructions, audio or video input or output, or other data in a modulated data signal. A modulated data signal is a signal that has one or more of its characteristics set or changed in a manner that encodes the information in the signal. By way of example and not limitation, the communication medium may use electrical, optical, RF or other carriers.

[0128] The innovations may be described in the general context of computer executable instructions, such as those included in program modules that are executed in a computing system on a target real or virtual processor. Typically, program modules or components include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules may be combined or split between program modules as desired. Computer executable instructions for program modules may be executed within a local or distributed computing system.

[0129] The terms "system" and "device" are used interchangeably herein. Unless the context clearly indicates otherwise, neither term implies any limitation on the type of computing system or computing device. In general, a computing system or computing device may be local or distributed, and may include any combination of special-purpose hardware and / or general-purpose hardware with software that implements the functionality described herein.

[0130] For purposes of presentation, the detailed description uses terms such as "determine" and "use" to describe computer operations in a computing system. These terms are high-level abstractions for operations performed by a computer and should not be confused with actions performed by a human. The actual computer operations corresponding to these terms vary depending on the implementation.

[0131] Example 10 - Cloud computing environment

[0132] Fig. 9 An example cloud computing environment 900 is depicted in which the described techniques may be implemented. Cloud computing environment 900 includes cloud computing services 910. Cloud computing services 910 may include various types of cloud computing resources, such as computer servers, data repositories, network resources, etc. Cloud computing services 910 may be centrally located (e.g., provided by a data center of an enterprise or organization) or distributed (e.g., provided by various computing resources located in different locations, such as different data centers and / or located in different cities or countries).

[0133] Cloud computing services 910 are utilized by various types of computing devices (e.g., client computing devices) such as computing devices 920, 922, and 924. For example, computing devices (e.g., 920, 922, and 924) may be computers (e.g., desktop or laptop computers), mobile devices (e.g., tablet computers or smart phones), or other types of computing devices. For example, computing devices (e.g., 920, 922, and 924) may utilize cloud computing services 910 to perform computational operations (e.g., data processing, data storage, etc.).

[0134] Example 11 - Implementation

[0135] Although the operations of some of the disclosed methods are described in a particular sequential order for ease of presentation, it should be understood that this description encompasses rearrangement unless the specific language set forth herein requires a particular ordering. For example, the operations described in sequence may be rearranged or performed simultaneously in some cases. In addition, for simplicity, the accompanying drawings may not show the various ways in which the disclosed methods may be used in conjunction with other methods.

[0136] Any disclosed method may be implemented as computer executable instructions or a computer program product stored on one or more computer readable storage media (such as tangible non-transitory computer readable storage media) and executed on a computing device (e.g., any available computing device, including a smartphone or other mobile device that includes computing hardware). A tangible computer readable storage medium is any available tangible medium that can be accessed within a computing environment (e.g., one or more optical media disks such as DVDs or CDs, volatile memory components (such as DRAM or SRAM), or non-volatile memory components (such as flash memory or a hard drive)). As an example and with reference to Figure 8 , computer-readable storage media include memories 820 and 825 and storage device 840. The term computer-readable storage medium does not include signals and carrier waves. In addition, the term computer-readable storage medium does not include communication connections (eg, 870).

[0137] Any computer executable instructions for implementing the disclosed technology and any data created and used during the embodiments disclosed in the implementation method can be stored on one or more computer readable storage media. The computer executable instructions can be, for example, a dedicated software application or a part of a software application accessed or downloaded via a web browser or other software application (such as a remote computing application). Such software can be executed, for example, on a single local computer (e.g., any suitable commercially available computer) or in a network environment using one or more network computers (e.g., via the Internet, a wide area network, a local area network, a client-server network (such as a cloud computing network) or other such network).

[0138] For the sake of clarity, only certain selected aspects of software-based implementations are described. Other details known in the art are omitted. For example, it should be understood that the disclosed technology is not limited to any particular computer language or program. For example, the disclosed technology can be implemented by software written in C++, Java, Perl, JavaScript, Python, Ruby, ABAP, structured query language, Adobe Flash, or any other suitable programming language, or in some examples, by a markup language such as html or XML or a combination of suitable programming languages ​​and markup languages. Similarly, the disclosed technology is not limited to any particular computer or hardware type. Certain details of suitable computers and hardware are known and do not need to be elaborated in this disclosure.

[0139] In addition, any software-based embodiments (including, for example, computer-executable instructions for causing a computer to perform any disclosed method) can be uploaded, downloaded, or remotely accessed via suitable communication means. Such suitable communication means include, for example, the Internet, the World Wide Web, an intranet, a software application, cables (including fiber optic cables), magnetic communications, electromagnetic communications (including RF, microwave, and infrared communications), electronic communications, or other such communication means.

[0140] The disclosed methods, apparatus, and systems should not be construed as limiting in any way. Rather, the present disclosure is directed to all novel and non-obvious features and aspects of the various disclosed embodiments, individually and in various combinations and sub-combinations with each other. The disclosed methods, apparatus, and systems are not limited to any particular aspect or feature or combination thereof, nor do the disclosed embodiments require the presence of any one or more specific advantages or problems to be solved.

[0141] The technology from any example can be combined with the technology described in any one or more other examples. In view of the many possible embodiments to which the principles of the disclosed technology can be applied, it should be recognized that the illustrated embodiments are examples of the disclosed technology and should not be considered as limiting the scope of the disclosed technology. On the contrary, the scope of the disclosed technology includes what is covered by the scope and spirit of the appended claims.

Claims

1. A computing system comprising: at least one memory; one or more hardware processor units coupled to the at least one memory; as well as One or more computer-readable storage media storing computer-executable instructions that, when executed, cause the computing system to perform operations comprising: creating a first instance of a first data object type at a local data source; receiving a first data set of said first instance of said first data object type from a remote data source, said first data set being extracted from a remote data object of said remote data source; storing the first data set in the first instance of the first data object type; determining that the first instance of the first data object type is out-of-date compared to the remote data object; receiving a second data set of the first instance of the first data object type, the second data set being update data for the remote data object; and At least a portion of the first data set is replaced with data of the second data set in the first instance of the first data object type.

2. The computing system of claim 1, wherein the operations further comprise: Receive inquiries; determining that the query accesses the first instance of the first data object type; as well as The determination is initiated based at least in part on determining that the query accesses the first instance of the first data object type.

3. The computing system of claim 2, wherein the operations further comprise: Execution of the query is delayed until the replacement has been completed.

4. The computing system of claim 3, wherein: The delay is performed in response to a command in the query.

5. The computing system of claim 2, the operations further comprising: The query is performed before the replacing.

6. The computing system of claim 5, wherein: Executing the query prior to the replacing is performed in response to a command in the query.

7. The computing system of claim 2, the operations further comprising: At least a portion of the query accessing the first instance of the first data object type is performed on the remote data object.

8. The computing system of claim 7, wherein: At least a portion of executing the query to access the first instance of the first data object type on the remote data object is performed in response to a command in the query.

9. The computing system of claim 1, wherein: The determining that the first instance of the first data object type is out of date includes comparing a digest value generated from content of the first data set to a digest value generated from content of the remote data object and maintained in current metadata of the remote data object.

10. The computing system of claim 1, wherein: The determining that the first instance of the first data object type source is out of date includes comparing statistical information maintained for the first instance of the first data object type with statistical information maintained in current metadata of the remote data object.

11. The computing system of claim 1, wherein: The determining that the first instance of the first data object type is out of date includes comparing a last modified date maintained for the first instance of the first data object type with a last modified date maintained in current metadata for the remote data object.

12. The computing system of claim 1, wherein: The determining that the first instance of the first data object type is out of date includes comparing a size maintained for the first instance of the first data object type to a size maintained in current metadata for the remote data object.

13. The computing system of claim 1, wherein: The first data object type includes an identifier that indicates whether a query accessing an instance of the first data object type is to be executed on the local data source or on the remote data object using the instance of the first data object type.

14. The computing system of claim 1, wherein: The determination is initiated based on the occurrence of a trigger.

15. The computing system of claim 1, wherein: The determination is initiated based on a synchronization schedule.

16. The computing system of claim 1, wherein: The remote data object and the first data object type share a set of common semantic properties.

17. The computing system of claim 1, wherein: The remote data object and the first data object type are relational database tables.

18. The computing system of claim 1, wherein: The determining is performed at the remote data source.

19. A method implemented in a computing system, the computing system comprising at least one hardware processor and at least one memory coupled to the at least one hardware processor, the method comprising: creating a first instance of a first data object type at a local data source; receiving a first data set of said first instance of said first data object type from a remote data source, said first data set being extracted from a remote data object of said remote data source; storing the first data set in the first instance of the first data object type; determining that the first instance of the first data object type is out-of-date compared to the remote data object; receiving a second data set of the first instance of the first data object type, the second data set being update data of the remote data object; as well as At least a portion of the first data set is replaced with data of the second data set in the first instance of the first data object type.

20. One or more non-transitory computer-readable storage media comprising: computer executable instructions that, when executed by a computing system including at least one hardware processor and at least one memory coupled to the at least one hardware processor, cause the computing system to create a first instance of a first data object type at a local data source; computer-executable instructions that, when executed by the computing system, cause the computing system to receive, from a remote data source, a first data set of the first instance of the first data object type, the first data set being extracted from a remote data object of the remote data source; computer-executable instructions that, when executed by the computing system, cause the computing system to store the first data set in the first instance of the first data object type; computer-executable instructions that, when executed by the computing system, cause the computing system to determine that the first instance of the first data object type is out-of-date compared to the remote data object; computer-executable instructions that, when executed by the computing system, cause the computing system to receive a second data set for the first instance of the first data object type, the second data set being update data for the remote data object; as well as Computer executable instructions that, when executed by the computing system, cause the computing system to replace at least a portion of the first data set with data of the second data set in the first instance of the first data object type.

Images