Data artifact instance to facilitate flexible data access

Abstract

This document describes techniques and solutions for providing flexible access to data during the execution of a data access request. Multiple instances of a data artifact are created, where different instances of the data artifact provide access to different data sources containing data associated with the data access request. When a data access request is executed, a specific data artifact instance can be used during the execution of the data access request. In some cases, switching logic can be used to determine which data artifact instance to use when executing a data access request. Techniques for facilitating the creation of data artifact instances corresponding to modeling artifacts are also described.

Description

Technical Field

[0001] This disclosure generally relates to data access requests and data artifacts that can be used to process data access requests. Specific embodiments provide multiple instances of data artifacts, wherein different instances can be used to access data associated with different data sources. Background Technology

[0002] Enterprises are increasingly storing data across a variety of systems, including one or more on-premises systems and one or more cloud systems. These systems can be of different types—such as storing data in different formats (e.g., a relational database versus a database storing Java documents)—or using different database management systems (e.g., using software and / or hardware from different vendors). Even if data is stored in the same format using software from the same vendor, the location of the data and the schema used to store it can differ.

[0003] To help address these issues, federated databases were created. In contrast to distributed databases, which are single database management systems involving multiple computer nodes (e.g., a coordinator node and one or more worker nodes), federated databases include the ability to make data from multiple different database management systems (or other data sources or data stores) available through a common platform or interface.

[0004] Even for data stored in a master (non-federal) data store, the data can be stored in different ways. For example, data can be copied from another data source (including data sources that are or can be used as federated data sources) and stored in the master data store. For other data, the master data store can serve as the primary repository for the data, where the data can be optionally made available to other data stores via replication or federation.

[0005] Typically, when creating a data model or data query, specific data components (such as fields in a database table) are associated with a specific data source, and data model artifacts and data queries can be created in a way that suits one or more related data sources. However, things can get complicated when it's unclear how data access / data sources should be structured, such as when importing a data model. Furthermore, the way typical data models reference associated data is static. This can limit flexibility, such as when different data access options would benefit in different situations. Therefore, there is room for improvement. Summary of the Invention

[0006] This overview is provided to introduce some concepts in a simplified form, which will be further described in detail below. This overview is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0007] This paper describes techniques and solutions for providing flexible access to data during the execution of a data access request. Multiple instances of a data artifact are created, where different instances of the data artifact provide access to different data sources containing data associated with the data access request. When a data access request is executed, a specific data artifact instance can be used during the execution of the data access request. In some cases, switching logic can be used to determine which data artifact instance to use when executing a data access request. Furthermore, techniques for facilitating the creation of data artifact instances corresponding to modeling artifacts are described.

[0008] In one aspect, a method for processing data access requests is provided, wherein selection is made among two or more data artifact instances to be used to execute the data access request. A first data access request is received. The first data access request specifies at least a first data artifact in a hierarchy, which includes multiple data artifacts distributed across multiple layers within that hierarchy. At least a portion of the multiple data artifacts has multiple distinct instances. The distinct instances of the data artifacts specify data associated with the data artifacts at different storage locations.

[0009] The first layer of the hierarchy includes at least a first data artifact among a plurality of data artifacts, which includes at least a first instance and at least a second instance. It is determined that at least a first instance of the at least first data artifact will be used to process the first data access request.

[0010] Processing a first data access request. Processing includes retrieving first data from at least a first instance of at least a first data artifact, or at a location specified by at least a first instance of at least a first data artifact. Returning at least a portion of the first data, or second data generated or derived at least partially from at least a portion of the first data, in response to the first data access request.

[0011] On the other hand, a method is provided for instantiating multiple data artifact instances corresponding to modeling artifacts. The method receives a modeling pattern comprising multiple modeling artifacts distributed across multiple layers of a hierarchy. It determines multiple instances of a first data artifact to be created, corresponding to a first modeling artifact among the multiple modeling artifacts. The first instance of the first data artifact is instantiated. The first instance of the first data artifact is instantiated at a first layer of a hierarchy comprising the data artifact pattern corresponding to at least a portion of the multiple modeling artifacts. The first instance of the first data artifact includes first data, or indicates a source of the first data corresponding to the first data artifact.

[0012] A second instance of the first data artifact is instantiated at the first level of the hierarchy. The second instance of the first data artifact includes second data or a source indicating second data corresponding to the first data artifact. The second data includes all or part of the first data, or is different from the first data in whole or in part.

[0013] In another aspect, a method for creating multiple instances of a data artifact is provided. An instruction is received to create a first instance of a first data artifact in a hierarchical schema, the hierarchical schema including multiple data artifacts distributed across multiple layers of a hierarchy in the hierarchical schema. The first instance of the first data artifact is created at a first layer of the multiple layers of the hierarchy. The first instance of the first data artifact includes or points to first data.

[0014] The system receives an instruction to create a second instance of the first data artifact. The second instance of the first data artifact is created at the first level of the hierarchy. The second instance of the first data artifact includes or points to second data. The second data includes all or part of the first data, or is different from the first data in whole or in part.

[0015] Receives an instruction to create at least a first instance of a second data artifact in a hierarchical pattern at the second level of the hierarchy. The first instance of the first data artifact points to at least a first instance of the second data artifact, and the second level of the hierarchy is lower than the first level of the hierarchy. Creates at least a first instance of the second data artifact.

[0016] This disclosure also includes computing systems and tangible, non-transitory computer-readable storage media configured to perform or include instructions for performing the methods described above. As described herein, various other features and advantages may be incorporated into the technology as desired. Attached Figure Description

[0017] Figure 1 It is a diagram showing the computational architecture of a hierarchical modeling pattern and the data artifacts corresponding to the artifacts in the modeling pattern.

[0018] Figure 2 This is a diagram illustrating how multiple instances of a data artifact corresponding to a modeling artifact are used with switching logic when executing a data access request.

[0019] Figure 3 This is a diagram showing a data artifact pattern that includes multiple instances of data artifacts corresponding to artifacts in the modeling pattern.

[0020] Figure 4 Sample code is presented, demonstrating how to annotate artifacts with information that facilitates the creation of multiple data artifact instances and the use of such data artifact instances to process data access queries.

[0021] Figure 5 Sample code for higher-level modeling artifacts built upon the basic modeling artifacts is presented.

[0022] Figure 6 The code for the basic layer modeling artifact is presented. Figure 5 Higher-level modeling artifacts are built upon the basic-level modeling artifacts.

[0023] Figure 7 Sample code for the switching logic is presented, which can be used in conjunction with... Figure 5 Switch between data artifact instances corresponding to higher-level modeling artifacts.

[0024] Figure 8 Presented Figure 7 Example code for the switching logic that references a data artifact instance.

[0025] Figure 9 Presented for use with Figure 6 Example code for the switching logic of at least one data artifact corresponding to the basic layer modeling artifact.

[0026] Figure 10 and Figure 11 Presented Figure 9 Example code for the data artifact referenced in the switching logic.

[0027] Figure 12 This is a flowchart of an example operation for processing a data access request that can use one of two instances of data artifacts.

[0028] Figure 13 This is a flowchart illustrating an example operation for creating a data artifact instance based on a model artifact.

[0029] Figure 14 This is a flowchart of example operations for creating a data artifact hierarchy, where higher-level data artifact instances reference lower-level data artifacts or data artifact instances.

[0030] Figure 15 This is a diagram illustrating a computing architecture with a virtual table schema, which has logical pointers that can target remote tables or replica tables.

[0031] Figure 16A and Figure 16B Sample column definitions are provided that can be used with database artifacts that can be used to manage virtual table schemas or replica tables used with virtual table schemas.

[0032] Figure 17 Sample query language statements are provided for performing actions related to the virtual table schema.

[0033] Figure 18 This is a diagram illustrating an example computing system in which some of the described embodiments can be implemented.

[0034] Figure 19 This is an example cloud computing environment that can be used in conjunction with the technologies described in this article. Detailed Implementation

[0035] Example 1—Overview

[0036] Enterprises are increasingly storing data across a variety of systems, including one or more on-premises systems and one or more cloud systems. These systems can be of different types—such as storing data in different formats (e.g., a relational database versus a database storing Java documents)—or using different database management systems (e.g., using software and / or hardware from different vendors). Even if data is stored in the same format and using software from the same vendor, the location of the data and the schema used to store it can differ.

[0037] To help address these issues, federated databases were created. In contrast to distributed databases, which are single database management systems involving multiple computer nodes (e.g., a coordinator node and one or more worker nodes), federated databases include the ability to make data from multiple different database management systems (or other data sources or data stores) available through a common platform or interface.

[0038] Even for data stored in a master (non-federal) data store, the data can be stored in different ways. For example, data can be copied from another data source (including data sources that are or can be used as federated data sources) and stored in the master data store. For other data, the master data store can serve as the primary repository for the data, where the data can be optionally made available to other data stores via replication or federation.

[0039] Typically, when creating a data model or data query, specific data components (such as fields in a database table) are associated with a specific data source, and data model artifacts and data queries can be created in a way that suits one or more related data sources. However, things can get complicated when it's unclear how data access / data sources should be structured, such as when importing a data model. Furthermore, the way typical data models reference associated data is static. This can limit flexibility, such as when different data access options would benefit in different situations. Therefore, there is room for improvement.

[0040] As mentioned above, in some cases, when a schema is defined, data artifacts, such as tables or views, can be specified as remote (federated), replicated to the primary data source, or native to the primary data source. In some cases of importing a data model, connections between data artifacts may not exist or may be unavailable; therefore, individual data artifacts can be configured to access their remote counterparts.

[0041] Maintaining relationships between data artifacts during the import process allows for the construction of patterns that achieve various objectives, such as data transfer volume, data access speed, data latency (e.g., whether using at least some stale data is acceptable—such as a local copy of data being outdated relative to a primary copy on a remote system), and data access or authorization considerations (e.g., whether a specific user can access a specific data artifact). Furthermore, allowing different data access paths based on different use cases is also beneficial. In one scenario, using stale data to achieve faster response times may be acceptable, while in another scenario, using stale data may be unacceptable, and processing latency may be tolerable.

[0042] This disclosure provides techniques that can be used to allow the construction of schemas in which different artifacts within the schema can access data using different methods, and in which access methods can be created according to various rules. This disclosure also provides techniques for schemas to allow access to the same underlying data artifact in different ways, such as allowing access to the data artifact via data federation if recent data is needed, or allowing access to a local copy (e.g., replicated data) if recent data is not needed.

[0043] This disclosure provides techniques for allowing data associated with data artifacts in a hierarchy to be accessed in multiple ways, wherein a data artifact can be represented as two instances. In a particular example, access can be switched from a “view” instance accessing a data artifact to an instance of a data artifact that provides direct access to the data associated with that data artifact (or, at least the access does not traverse lower levels of the hierarchy), which accesses data from a data artifact at a lower level of the hierarchy, wherein the lower-level data artifact itself may have multiple access options. That is, there are at least two instances of a data artifact, one referring to data at a lower level of the hierarchy, and the other referring to data directly associated with that data artifact, such as data available via replication (local) or federation (accessing a remote data source, such as a remote table or view, which may be a view being computed in progress or a materialized view).

[0044] In some cases, data artifacts that provide direct access to data can themselves offer flexible data access, such as having logical pointers with values ​​indicating a specific data source, and wherein the values ​​of the logical pointers can be dynamically updated to indicate different data sources. As an example, a logical pointer can be updated to point to data sources that are federated, locally available via replication, locally available, cache-associated, of different storage types (e.g., data dynamically rescheduled between physical storage or permanent storage such as disks), or to different types of remote storage devices, such as data in a data lake or cloud storage.

[0045] In certain implementations, a switching view can be used to switch between instances of data artifacts. A switching view can have conditional statements where conditions define which of a plurality of available data artifacts are used for a given data request (e.g., a query). For example, a query such as SQL can reference a switching view. Elements in the query (e.g., an operation specified in the query, a data source specified in the query, parameters specified in the query such as values ​​used to filter data, or combinations thereof) or semantic flags in the query (e.g., "use data artifact X", "use union", "use local", "time sensitive", "use recent data") can be used to determine which of the plurality of instances of the data artifact should be used to process the query.

[0046] Lower-level data artifacts in a hierarchy can themselves have multiple access methods and can have mechanisms for determining which access method should be used for a particular request. In some cases, the access method used for a higher-level data artifact can propagate down to lower-level artifacts. Alternatively, parameters used to determine which access method should be used can propagate down to determine how the lower-level data artifact will be accessed, where the selected access technology is independent of the technology chosen for the higher-level data artifact (an instance of the data artifact). Since switching criteria can include conditional statements, these statements can include the default access method if no information in the request indicates switching from the default access technology. For different data artifacts in a hierarchy where switching is an option, the default access technology can be selected independently.

[0047] In another implementation, multiple instances of a data artifact may exist for one or more data artifacts in a hierarchy, but access cannot be determined based on switching views or a similar structure for a specific request. In some cases, the request itself may specify the appropriate data artifact instance for the desired specific access method (e.g., a data artifact instance may have an identifier that distinguishes it, such as a user being able to select data artifact instance A or A', where data artifact instances A or A' are instances of the same data artifact where data is accessed using different technologies). Alternatively, the request processing logic may include other types of switching logic. For example, the logic may specify that a lower-level artifact should be used unless a specific access method is available for a given data artifact.

[0048] In yet another example, multiple queries (or more generally, requests) can be defined, each providing a different combination of access technologies for the data artifacts within the query. Therefore, the user or process only needs to select the appropriate query to execute, without needing to construct the query or have detailed knowledge of the relevant schema. The selection can be indirect, such as by providing standard user interface elements that are associated with flags indicating which type of query should be selected. Specifically, the user interface screen can provide the user with options to define the query (e.g., selecting specific fields of interest and selection criteria) and indicate, for example, whether data recency or query speed is more important, allowing the software logic to construct or select the appropriate query.

[0049] The disclosed technology also facilitates the implementation of flexible access techniques. For example, given a specific pattern as input, the software process can create appropriate data artifact instances for the artifacts of the input pattern. For instance, artifacts in the pattern (models or design-time data artifacts) can be used to instantiate runtime data artifacts that reference lower-level artifacts, runtime data artifacts that provide access to data via copying or union, and optionally switching logic or data artifacts (e.g., switching views). Default switching logic can also be provided.

[0050] Users or processes may choose to enable the exposed flexible access technologies, such as by including appropriate annotations in the model data artifact that trigger code to create runtime data artifact instances as described above.

[0051] When the source data is not located at the primary data source (where the implementation schema / data artifact hierarchy is located), data artifacts can be created to access the data from the remote data source, such as via federation or by replicating the data (thus, essentially creating a local copy of the data, even if the data source is not the primary data source).

[0052] Example 2—An example architecture for implementing a data artifact pattern with multiple data artifact instances.

[0053] Figure 1 A computing environment 100 capable of implementing the disclosed technology is shown. The computing environment 100 includes a client system 104 communicating with a data storage 108. In a particular example, the data storage 108 is a relational database system. The client system 104 may be a system used by an end user, or it may represent a computing platform, framework, or middleware accessed by another end-user system (not shown).

[0054] Client system 104 includes modeling pattern 112. In some cases, modeling pattern 112 may be a virtual data model. In other cases, modeling pattern 112 may be a design time structure for creating corresponding runtime artifacts, which may include creating artifacts in a virtual data model or data storage 108. When using a virtual data model, artifacts in the virtual data model may reference corresponding artifacts in data storage 108.

[0055] As shown in the figure, modeling pattern 112 may include multiple hierarchically arranged artifacts 116 (e.g., modeling artifacts). Artifacts 116 may be built upon each other, such as allowing higher-level artifacts to select data from lower-level artifacts, and optionally supplementing or modifying aspects of the lower-level artifacts. Although Figure 1 The diagram shows a relatively simple hierarchy of article 116, but modeling pattern 112 can contain multiple hierarchies, and these hierarchies can be more complex than... Figure 1 The diagram shown is significantly more complex. Furthermore, modeling pattern 112 can have artifacts 116 arranged in a non-hierarchical manner.

[0056] Data storage 108 may include data 118, such as data stored in various data artifacts 120 (as shown in 120a-120d), such as tables or views in a relational database system (used as data storage). In other cases, data artifacts do not directly store data but provide access to it. For example, a data artifact may be objects in a virtual data model that are mapped to data in a data source, such as tables or views in a relational database system (or other sources of structured, semi-structured, or unstructured data). In other cases, a data artifact may be a physical table or view, or a similar structure in other sources of structured, semi-structured, or unstructured data.

[0057] Data artifact 120 can be of different types, where the type can indicate how the data artifact accesses or retrieves data, or how the data artifact is used. Data can be stored in a local data artifact 120a. Local data artifact 120a can be an artifact where data is stored in data storage 108, and the data storage is the primary source of data for local data artifact 120a. That is, data from local data artifact 120a can be copied to other data storage, but data storage 108 determines the final instance of data stored in the local data artifact.

[0058] Data artifact 120b represents a copied (or duplicated) data artifact. The copied data artifact 120b contains data stored in data storage 108, but another instance of the related data artifact (e.g., an instance stored on another data storage) determines the final instance of the data in the copied data artifact and sends or causes the related data to be sent to data storage 108 for storage in the copied data artifact (which is typically the latest source of its data).

[0059] Federation data artifact 120c is similar to replica data artifact 120b, but the data is not permanently stored in the data store. Nevertheless, the replicated data is subject to deletion or modification of the data stored in data store 108 (e.g., by copying data from the source data artifact to data store 108, or by deleting the replica data artifact). That is, federated data can be obtained from data artifact 124 in data store 128 of remote system 130, used for a specific purpose (e.g., query), and deleted after the query has been executed or the data is no longer needed or actively used. In some cases, data obtained through federation can be stored in a cache, but this data can be removed from the cache according to a cache management policy without needing to be deleted accordingly at data artifact 124.

[0060] Although shown as part of data 118, in some cases, the combined data artifact 120c exists in the data dictionary 132 (or information schema or similar structure) of the data storage 108, but the data of the combined data artifact is not stored as part of data 118.

[0061] Data storage 108 is also shown to include switching artifact 120d. At least in some embodiments, switching artifact 120d does not store data. Instead, switching artifact 120d is used to switch between artifacts 120 in data storage 108, regardless of whether the artifact is part of data 118 or defined as part of data dictionary 132. Consistently, like federated data artifact 120c, in some cases, switching artifacts are defined in data dictionary 132 (i.e., switching artifact definition 140), but switching artifact 120d is not included in data 118.

[0062] Data dictionary 132 is shown to include federated data artifact definition 136. For example, data obtained through federation can be provided to a requesting process (e.g., a query) without needing to be stored in data 118, although the data may be associatedly stored in data storage 108, typically such as in memory included in the cache as described above or in a permanent storage device associated with the data storage.

[0063] Data dictionary 132 may include definitions 144 of other data artifacts 120, including data artifacts that typically exist as part of data 118. For example, definitions may be used for local artifacts 120a, such as database tables or database views.

[0064] Another type of data artifact definition in data dictionary 132 (and optional data artifacts, not shown in data 118) is virtual artifact definition 148. Virtual artifact definition 148 may include definitions of the structure of data artifacts (whether local or remote) and may indicate the location of data, such as using logical pointers. The value of a logical pointer can be dynamically updated to point to different data artifacts, including those in different locations, that maintain data that should be used, such as when a data artifact is used in a query. For example, a logical pointer may be updated to point to a replicated data artifact 120b or a federated data artifact 120c (or directly to a data artifact 124 on remote system 130). Logical pointers may point to other data artifacts, data artifact definitions, or other locations where data associated with a data artifact definition can be stored, including data stored in a data lake, cache, or storage where data can be dynamically moved between storage types (e.g., memory buffers or disk-based storage devices).

[0065] Example 3—Example data artifact pattern with switching logic

[0066] Figure 2This illustrates a scenario where dynamic access to data artifacts corresponding to modeling artifacts in data model 200 can be achieved by providing dynamic, variable access to the data. Data model 200 is illustrated using two modeling artifacts: modeling object 210 represents a cube (e.g., an OLAP cube), and the cube reference represents a modeling artifact 214 representing a lower-level data source such as a table or view. That is, data model 200 is hierarchically organized, where higher-level artifacts, such as cube artifact 210, are built upon lower-level objects such as modeling artifact 214. In this particular example, cube modeling artifact 210 can select dimensions or facts associated with modeling artifact 214.

[0067] Data model 200 uses only two modeling artifacts 210 and 214 to represent the relevant technical concepts. In practice, the hierarchy is typically deeper (as shown by dashed boxes 218 and 220) and wider than in data model 200. For example, in practice, a cube modeling object can be built on multiple views, which are at least partially derived from lower-level views or tables. The techniques described for the simple data model 200 can be implemented in much more complex data models in a similar manner.

[0068] Data model 200 can be transformed into a schema 230 of runtime data artifacts (where at least some data artifacts are instantiated as two or more distinct instances, which are collectively referred to as "data artifacts" for ease of presentation, since each instance corresponds to a common artifact, only the data sources differ between instances). Runtime data artifacts can represent data artifacts in a virtual or physical data model associated with one or more data sources. For example, a data artifact can represent a virtual data artifact mapped to (i) a physical data artifact, (ii) a physical data artifact, (iii) or a combination of (i) and (ii). Examples of physical data artifacts include database tables and database views. Examples of virtual data artifacts include virtual tables, virtual views, or definitions of artifacts or data that modify how virtual tables or views store or process data. Views can include views that provide a cube definition for OLAP, such as views on a star or snowflake schema of a database, or views on top of another view defined relative to a star or snowflake schema.

[0069] Mode 230 includes a first set of data artifacts 236 and 238, which are runtime artifacts corresponding to cube modeling artifact 210 and modeling artifact 214, respectively. Data artifacts 236 and 238 are associated with a first type of data source. For example, data artifacts 236 and 238 may refer to remote, but not necessarily federated, data sources. In a particular example, data artifacts 236 and 238 are defined in the SAP HANA cloud (available from SAP SE in Walldorf, Germany), but the underlying data is stored in an on-premise SAP HANA installation. Therefore, queries using one or both of data artifacts 236 and 238 are sent to the on-premise database, and the response data (which may be the entire contents of a data artifact such as a table, but typically limited using various filtering conditions) is returned to the cloud-based database. In another example, data artifacts 236 and 238 may be local data artifacts that maintain a master copy of the relevant data, or they may be replicated data artifacts. In yet another example, at least one higher-level artifact may reference a lower-level artifact.

[0070] Pattern 230 also includes a second set of data artifacts 244 and 246, which are runtime objects corresponding to cube modeling objects 210 and 214, respectively (and can also be considered as alternative instances of data artifacts 236 and 238). Data artifacts 244 and 246 are associated with a second type of data source, which is different from the first type of data source.

[0071] Different data artifacts 236, 238, 244, and 246 have been described, along with mechanisms for selecting between instances of these data artifacts. Figure 2 In the example, switching views 250 and 252 are used to switch between data artifacts 236 and 244, and between data artifacts 238 and 246, respectively. The operation of switching views 250 and 252 can be determined using switching criteria 256 and 258. In one example, switching criteria 256 and 258 can be implemented using or in a manner similar to a "switch" statement in a programming language such as C++. More generally, similar functionality can be achieved in other ways, such as by using one or more conditional statements (e.g., IF, IF / ELSE).

[0072] One or more conditions can be defined, whereby, if a condition is met, a data artifact instance of a given layer in pattern 230 will be used. If another condition is met (or not met), another data artifact instance of the given layer in the pattern will be used. Switching criteria 256, 258 may include default conditions for selecting a specific data artifact instance at a specific layer in pattern 230. Although pattern 230 shows two data artifact instances for a given switching view (or more generally, a switch or switching criterion, whether or not it is present within the view), a switching view may have more than two data artifact instances as possible targets.

[0073] Switching criteria can be specified in any suitable manner. An example of a switching criterion is an operation, data source, or value specified in a data request (e.g., a database query, such as in SQL). For example, a join involving a specific database table or a specific column of a database table can be used to select one of several possible data sources. Alternatively, specifying a join operation on all possible data artifact instances may be sufficient to select one of several possible data artifact instances. Data requests can be given names or identifiers, and these names or identifiers can be used to select data artifact instances. Similarly, identifiers of users, computing systems, applications, or application processes can be used to select between data artifact instances. Data requests can contain information that can be used as switching criteria or be provided along with such information. For example, a query can be sent with indicators or flags that indicate whether it is time-sensitive or whether data accuracy (i.e., obtaining the most recent data) is preferred.

[0074] Generally, the switching at switching view 252 can be handled in a similar manner to switching view 250. However, more complex issues may arise. For example, if the data request specifies data from two layers of the hierarchy, questions may arise about whether the same type of switching as data artifact instances 244 and 246 should be applied at the layers of data artifact instances 236 and 238, or whether switching view 252 should independently determine which data artifact 244 or 246 to use. Similarly, if switching view 250 determines to use data artifact 236, but needs to retrieve data from a lower-level artifact instance, switching view 252 may need to determine whether to use data artifact 244 or data artifact 246.

[0075] In one implementation, the data access request itself may be provided to switch view 252 and may be determined as described in switch view 250. In another implementation, relevant parameters are passed to switch view 252 for determination, but the entire data request is not sent. In yet another implementation, switch view 250 may allow switch view 252 to be completely bypassed, or bypassed by providing input that guides switch view 252 to select data artifact 244 or data artifact 246.

[0076] As described above, switching criteria can be implemented to allow users or processes to prioritize certain objectives, such as data access speed, data accuracy (recentness), processing or access considerations (e.g., if a particular data source is unavailable or encounters technical problems). Since user or process needs can change, users or processes can change how they access data within mode 230, or the switching logic can automatically enable such changes.

[0077] Note that switching views 250 has the effect of retrieving data for the corresponding cube modeling artifact 210 outside the hierarchy of mode 230. That is, when using data artifact 236, data from the lower-level data artifact 238 can be used. When using data artifact 244, the data is not obtained from the lower-level artifacts 238 and 246, but directly from the artifact instance (which can be a data artifact or a definition corresponding to the data artifact specified in data artifact 244).

[0078] Beyond simply having the ability to select different data artifacts for a given modeling artifact, it can be beneficial to select specific data artifacts that include specific types of data access based on their position within the hierarchy of data artifacts. For example, it has been described that specific data artifacts can represent access to replicated data. In some cases, it may be advantageous to implement replication at lower levels, including as low as possible within the hierarchy or stack of data artifacts. One reason for this is that data often becomes more denormalized at higher levels in the schema, which can create redundancy in data replication (e.g., the same data is copied multiple times). Furthermore, usage restrictions (e.g., authorization for specific users or processes) are often implemented at higher levels of the schema, and access to data at such levels can be difficult without specific authorization.

[0079] Higher-level APIs are often parameterized, which can make it difficult to adapt to data replication. Examples of parameters include those that identify the user, language, or date. As mentioned above, user parameters can be associated with access permissions. Some views are configured to return results only in the specific language (or multiple languages) indicated in the parameters provided as part of the data request. Date parameters can be used to indicate the specific date the data is requested (such as "today"), rather than a specific time interval.

[0080] Data replication processes typically cannot accept such parameters; if these parameters are not provided, the data may be unselectable for / from the view. Accordingly, these views can be configured to obtain data via federation (where the parameters are supported). Views at lower levels of the hierarchy may not include these parameters, thus data replication becomes an option, and in some cases, the preferred option (including, as mentioned above, because at such lower levels there are no defined data redundancy / denormalization issues / parameters, making data acquisition via federation potentially infeasible).

[0081] While using replicated data sources at lower levels of the hierarchy may be beneficial, using data federation at higher levels can be advantageous. For example, since access permissions / authorizations are typically specified at higher levels of the hierarchy, this information can be helpful in obtaining data from federated data sources (because authorizations for lower-level data artifacts may not be defined / specified).

[0082] Example 4—An example data artifact pattern with multiple data artifact instances, including instances referencing lower-level data artifacts.

[0083] Figure 3 This refers to alternative techniques for implementing publicly disclosed technologies. Such techniques can be helpful when view switching or similar logic is unavailable or unable to achieve the desired functionality. For example, if view switching exists, but passing data requests or parameters via schemas is not feasible, this technique can be used. Figure 3 The technology shown.

[0084] Figure 3 and Figure 2 Similarly, it includes data model 300 and schema 330, such as schemas for runtime data artifacts. Data model 300 is slightly more complex than data model 200 because it includes query modeling artifact 308 in addition to cube modeling artifact 312 and table or view artifact 316 (or other types of data artifacts, constructs, or objects).

[0085] Pattern 330 is shown as containing alternative data artifacts 320 and 322 corresponding to query modeling artifact 308, and alternative data artifacts 326 and 328 corresponding to cube modeling artifact 312. In this case, only a single data artifact 332 corresponding to table or view modeling artifact 316 is provided. This is because while data artifacts 320 and 326 can obtain data from lower-level data artifacts in the hierarchy of pattern 330, data artifact 332 is at the lowest level of the pattern, and its data is not available from lower-level artifacts in pattern 330. Therefore, the data for data artifact 332 must be obtained from a data source outside the hierarchy, such as using data federation. If the data source implementing pattern 330 has available data (whether local, replicated, cached, etc.), multiple data artifacts can be provided for each data artifact, regardless of the hierarchy, such as through federation. Figure 2 As described.

[0086] Otherwise, data artifacts 320 and 326 can be handled according to... Figure 2 The descriptions of data artifacts 236 and 238 are implemented, and data artifacts 322, 328 (and 332) can be implemented according to the descriptions of data artifacts 244 and 246.

[0087] In the absence of explicit switching logic, other mechanisms can be used to select data artifact instances at different layers of the hierarchy (in the case of defining multiple data artifacts, at least in some implementations, not all layers of the hierarchy are required to have multiple data artifact instances). In some cases, users can manually select specific data artifact instances for a query, where information can be provided that allows users to understand the nature of data access for a given data artifact (e.g., the query can be labeled "view stack" or "union"). Similarly, such queries can be defined for users, and users can simply select the appropriate query (where information about the query can be provided, at least about the advantages that can be achieved by selecting one query instead of another). In other cases, some or all of the criteria described for switching views 250, 252 can be used to define (or modify) queries. For example, data artifacts can be named using standard naming conventions. When a query is received, switching rules / criteria can be used to select the relevant data artifact name to be used in query execution.

[0088] If desired, the configuration of mode 330 can be implemented as described for mode 230. That is, it may be advantageous to use replication at lower levels of the hierarchy and union at higher levels of the hierarchy.

[0089] Example 5—Example data artifact including additional data source options

[0090] In some cases, data artifact instances or data artifacts can also provide flexible data access. Examples 10-12 provide details of a technique for implementing such data artifacts, including data artifacts within a database schema. Further examples of such data artifacts are virtual tables and fabric virtual tables implemented using techniques available from SAP SE in Walldorf, Germany.

[0091] In the example, data artifacts (or instances, referred to as data artifacts below for ease of presentation) include logical pointers. Logical pointers can be updated to point to different data sources / data locations. For example, a logical pointer can be updated to point to a joined table or a replicated table. Other data sources that can be referenced using logical pointers include local tables, cached tables, tables in dynamic storage devices (e.g., storage devices where data is migrated between different storage types, such as using algorithms that attempt to place more important data in storage devices with faster access times), or data in a data lake within a cloud-based storage device.

[0092] In some cases, such as when data is available in multiple locations simultaneously, the value of a logical pointer can be updated during runtime to point to the appropriate storage location. Ideally, the logical pointer can be restored to its original or default value once the data access request has been satisfied. In other cases, it may not be desirable to allow individual data access requests to modify the logical pointer value that indicates the data source of the data artifact.

[0093] In other cases, the value of the logical pointer to a data artifact can vary based on factors such as the performance of the data source, the performance of the data access query, or other considerations, but is independent of any particular data access request. For example, if query speed is more important than the potential issue of data in the replica table becoming outdated compared to the replicated data source, then the data source administrator can determine that a replica table should be used instead of a join table.

[0094] Example 6—An example that facilitates the creation of data artifact instances

[0095] As discussed in Example 8 with respect to a specific example artifact, the disclosed techniques can be used to facilitate the creation of data artifact instances, relationships between data artifacts, and the execution of data access requests using data artifact instances.

[0096] For example, logic can be implemented to automatically create data artifact instances for modeling artifacts. Such actions can be explicitly requested. For instance, a command or user procedure can invoke a function to create a data artifact instance for an artifact in a specific artifact schema or for a selected artifact. Alternatively, a schema or artifacts within a schema can have annotations indicating that a data instance should be created. Instructions for creating data artifact instances can include information that can help determine the nature of the data artifact instance, configure logic for selecting between data artifact instances, and establish relationships between data artifacts and data artifact instances at different levels of the data artifact hierarchy. Schemas can be analyzed to determine which artifacts have been selected for data artifact instance creation, and which appropriate data artifacts (and optional switching logic) have been created.

[0097] Example 7—Example Product Code

[0098] Figure 4 Example code 400 (pseudo-CSN) for defining artifacts such as modeling artifacts 404 is shown, including annotations on the artifacts to indicate that they should be created using flexible deployment options.

[0099] In definition 408 of artifact 404 "I_Product", comment 410 indicates that a data artifact should be created for the artifact to allow flexible access to the data associated with the artifact via token 412 (indicating variable deployment) and value 414 (true, indicating that variable deployment is enabled). Code 400 also specifies that two data artifacts 416 and 420 should be created to provide variable access.

[0100] Code 400 provides a general format for indicating comments on artifacts to assist or initiate the creation of data artifacts. In practice, Code 400 may include additional details for artifact 404 and data artifacts 416, 420. For artifact 404, Code 400 may include defining information such as the selection of data from one or more data sources, the definition of additional data elements, modifications to data elements from referenced data sources, or information indicating how the data associated with data artifacts 416, 420 will be processed or how such data will be used. Similar details may be included for data artifacts 416, 420.

[0101] In the case of data artifact 416, code 400 can specify another artifact, such as a modeling artifact or another data artifact, referenced by the data artifact. That is, as already described, an artifact can refer to an artifact at a lower level in the hierarchy. Therefore, data artifact 416 can specify a lower-level artifact to be used when obtaining data associated with artifact 416.

[0102] Data artifact 420 may represent a data artifact that can reference a replicated data source or a federated data source, including the ability to be updatable between one of these specific data sources. The code of data artifact 420 may optionally include the initial allocation of the data source.

[0103] In other cases, logic can be used to automatically define, at least partially, the data artifact corresponding to artifact 404, instead of specifying data artifacts 416 and 420 in the code. Token (or command or flag) 412 can provide an indication of what action should be taken.

[0104] Code 400 may provide one or both deployment hints 430 and runtime hints 434. Deployment hint 430 helps determine, for example, where or what type of data source should be created (e.g., replicated, federated, local). Runtime hint 434 may include information that helps route data access requests between data artifacts 416 and 420, as described above. Runtime hint 434 may also instruct on data artifacts associated with data artifact 416 or 420, including defining relationships to lower-level data artifacts / data artifact instances for the purposes of data artifact 416, or enabling switching or routing information to be sent to the relevant data artifact. Runtime hint 434 may also be used to create links between artifact 404 and data artifacts 416 and 420.

[0105] Example 8—Example code illustrating the disclosed technology

[0106] Figure 5 Code 500 (in pseudo-CSN or similar format) is shown and can be included in an artifact such as a modeling artifact. Code 500 includes a plurality of data elements 504 specifying an artifact, wherein at least a portion of the data elements 504 can be associated with data in one or more data artifacts created using the disclosed techniques (or data available in one or more data artifacts created using the disclosed techniques). A given data element 504 can be associated with a name / identifier 508 and a type 510.

[0107] Code 500 also includes data selection of artifacts in the form of query 520. Query 520 identifies a specific data source 524 (such as a table or view), in this case having the name "I_SalesDocument" and a specific data category 528 (e.g., a field / column of a table or view, which can be defined as a data element in data source 524).

[0108] Figure 6Code 600 is shown for the "I_SalesDocument" artifact 524 referred to by code 500. In this case, code 600 only defines data element 604, rather than selecting data from another data source. Therefore, artifact 524 can be considered a "basic" or "lowest-level" artifact because it does not depend on any other artifact.

[0109] Figure 7 Example code 700 (in pseudo-CSN, or a CSN-like format) is shown. Example code 700 defines switching logic for determining the data artifact to be used to process a specific data access request, which can be used for at least one layer of a hierarchy of data artifacts having multiple data artifacts that can be used to process data access requests. Specifically, code 700 can represent... Figure 2 Examples of switching views 250 and 252.

[0110] Statement 710 creates a data artifact, in this case, that is a view and can be referenced by a data access request (or that determines whether the data artifact should be used in processing a given data access request). Code 700 also defines several conditional statements 720. For each conditional statement 720, an input value (or flag) is examined to determine whether the input value specifies that a remote data source 724 should be used (or, more specifically, a particular data element, such as a column, from the remote data source). If yes, data 724 is selected from the remote data source. If not, data is selected from a lower-level data artifact 728 in the data artifact hierarchy.

[0111] Figure 8 Code 800 for creating data artifact 728 (e.g., in a pseudo-CSN or similar format) and code 850 for creating remote data source 724 are shown. Note that code 800 includes a data selection operation 810 that selects data from data artifact 524 referenced in code 500. Therefore, depending on the result of the switching logic in code 700, data is obtained either from a lower-level data artifact or from a remote data source.

[0112] Figure 9 Code 900 is shown, which defines the switching logic for the “I_SalesDocument” artifact 524 defined in Code 600. Code 900 is typically constructed as described for Code 700. Since artifact 524 is the underlying artifact, Code 900 is not required in some cases. However, including switching logic can be advantageous to help avoid confusion when writing or executing data access requests.

[0113] Although code 900 is shown as referencing a remote data source 910 and a data artifact in the data artifact hierarchy 920, in some cases, all conditional statements can indicate the use of the remote data source if only the remote data source exists. Alternatively, a data artifact in the data artifact hierarchy can reference the remote data source to correctly obtain the data, such as... Figure 10 The code 1000 is shown in the example. Similar definitions can be used in other cases where a single data source is available, such as only a local data source or only a replicated data source.

[0114] Figure 11 Definition 1100 of remote data source 724 is shown.

[0115] Example 9—Example of flexible access use cases

[0116] Exposed technologies that provide variable access to data, including automatically determining access types at runtime or through manual or semi-automatic methods, can be used in a variety of scenarios. As mentioned earlier, some scenarios involve determining whether reducing data latency or reducing processing workload or processing time is more important. However, these considerations may be relevant to specific use cases, or variable access may have other uses.

[0117] One use case is for testing or demonstrating data access software. For example, data replication can be resource-intensive. When building and testing software solutions, acquiring data via federation can be beneficial because it can be a more lightweight process. Once the system is ready for deployment, some of the data artifacts can be configured to access the data using data replication instead of data federation.

[0118] The ability to switch between different access modalities can also be used to determine how the system should be configured, at least initially. For example, the performance of data federation can be evaluated against the performance of data replication to help determine the default access technology and under what conditions a different access technology should be used (e.g., defining switching logic). This information can also be used dynamically at runtime to determine which access method provides the best cost / benefit outcome, where switching can occur if different criteria indicate that it is beneficial (e.g., threshold performance parameters can be set to determine when a switch should occur).

[0119] Variable access can also facilitate a top-down modeling approach. This approach can be adopted by end users (e.g., developers working for companies customizing software solutions from another vendor or creating models from scratch). Top-down modeling can be useful for many reasons, including that it may be easier to begin defining artifacts based on how they will be used in a software application before determining the best model for storing data referenced by higher-level artifacts. Top-down modeling can also help users with the most knowledge / relevance to define artifacts. In the example above, the application developer, data scientist, or end user may best understand what data is needed and the best format for accessing that data (including considerations based on application performance), but the database administrator may be most familiar with the performance of the database storing the underlying data.

[0120] Top-down modeling not only facilitates users switching from working with tables to working with views (or more generally, from higher-level artifacts to lower-level artifacts, including lower-level artifacts that store data used by higher-level artifacts), but also facilitates users switching between different access methods. As already described, artifacts at higher levels may benefit from accessing data through federation rather than replication, while replication may be more advantageous at lower levels of the modeling / data artifact hierarchy.

[0121] Variable access can also be used to preview data. For example, if a replication process is being defined or evaluated, a preview of the data to be replicated can be obtained via union. That is, some or all of the data in the model being created can eventually be obtained via replication, but during the modeling or replication process, the setup data can be obtained via union.

[0122] Similarly, variable data access using publicly available technologies can facilitate peer-to-peer modeling, such as specific data stores (like databases) used by specific departments within a company. Typically, such models are initially created using data federation, at least in part because users may lack permission to initiate data replication, for example, due to associated processing loads. However, at some point, it may be desirable to switch access to data replication, such as if the peer-to-peer model proves particularly useful or is adopted more widely.

[0123] By adding or configuring appropriate data artifacts (e.g., defining a replication process for instances of data artifacts that acquire data via replication), the peer-to-peer model can be adapted to replication scenarios. These scenarios are also beneficial because acquiring data via replication or federation can be simpler and less resource-intensive than implementing an ETL (extract, transform, load) process to obtain the data source (e.g., treating it as local data). Furthermore, both data federation and replication help reduce data obsolescence, as ETL processes are typically performed less frequently than replication processes.

[0124] Variable access can be used to provide data rollback techniques. For example, if data access is "normal" via data federation and the federation process is unavailable, data can be obtained via data replication, even if the replicated data may be somewhat outdated compared to the data available via federation. Similarly, if the replication process fails, or the replica data source is unavailable, data can be obtained via federation, even if this may result in longer processing times.

[0125] Publicly available technologies can help reduce modeling effort by allowing greater reuse of models and runtime artifacts. That is, for one user, data replication might be better; for another, data federation might be better. The same artifacts can be used, where different users use different types of artifact instances or switching logic, instead of each user having to develop separate artifacts.

[0126] Publicly available techniques can also be used to check the consistency of data available using different access technologies. Furthermore, this can help determine whether data latency associated with replication is tolerable, or whether data federation should instead be used to obtain the data. Consistency checks can also be used to help determine whether access permissions associated with data federation produce data that matches the data retrieved using data replication.

[0127] Example 10—Example Operations for Implementing Publicly Disclosed Technologies

[0128] Figure 12 This is a flowchart of a method 1200 for processing data access requests, wherein two or more instances of data artifacts are selected to perform the data access request.

[0129] At 1210, a first data access request is received. The first data access request specifies at least a first data artifact in a hierarchy, which includes multiple data artifacts distributed across multiple layers within that hierarchy. At least a portion of the multiple data artifacts has multiple distinct instances. The distinct instances of the data artifacts specify data associated with the data artifacts at different storage locations.

[0130] The first layer of the hierarchy includes at least the first data artifact among a plurality of data artifacts containing at least a first instance and at least a second instance. At 1220, it is determined that at least the first instance of the first data artifact will be used to process the first data access request.

[0131] In step 1230, a first data access request is processed. The processing in step 1230 includes retrieving first data from at least a first instance of the data artifact, or at a location specified by at least a first instance of the first data artifact. In step 1240, in response to the data access request, at least a portion of the first data, or second data generated or derived at least partially from at least a portion of the first data, is returned.

[0132] Figure 13 This is a flowchart of a method 1300 for instantiating multiple data artifact instances corresponding to a modeling artifact.

[0133] At 1310, a modeling pattern comprising multiple modeling artifacts distributed across multiple layers of a hierarchy is received. At 1320, multiple instances of a first data artifact corresponding to a first modeling artifact among the multiple modeling artifacts are determined. At 1330, a first instance of the first data artifact is instantiated. The first instance of the first data artifact is instantiated at a first layer of a hierarchy comprising a data artifact pattern corresponding to at least a portion of the multiple modeling artifacts. The first instance of the first data artifact includes first data, or indicates a source of the first data corresponding to the first data artifact. At 1340, a second instance of the first data artifact is instantiated at a first layer of the hierarchy. The second instance of the first data artifact includes second data, or indicates a source of the second data corresponding to the first data artifact. The second data includes all or part of the first data, or is entirely or partially different from the first data.

[0134] Figure 14 This is a flowchart of method 1400 for creating multiple instances of data artifacts.

[0135] At 1410, an instruction is received to create a first instance of a first data artifact in a hierarchical pattern, the hierarchical pattern comprising multiple data artifacts (which may be represented as multiple data artifact instances) distributed across multiple layers of a hierarchy in the hierarchical pattern. At 1420, the first instance of the first data artifact is created at the first layer of the multiple layers of the hierarchy. The first instance of the first data artifact includes or points to first data.

[0136] At 1430, an instruction to create a second instance of the first data artifact is received. At 1440, the second instance of the first data artifact is created at the first level of the hierarchy. The second instance of the first data artifact includes or points to second data. The second data includes all or part of the first data, or is wholly or partially different from the first data.

[0137] At 1450, an instruction is received to create at least a first instance of a second data artifact in the hierarchical pattern at the second level of the hierarchy. The first instance of the first data artifact points to at least a first instance of the second data artifact, and the second level of the hierarchy is lower than the first level of the hierarchy. At 1460, at least a first instance of the second data artifact is created.

[0138] Example 11—Example schema of a virtual table with a remote or replicated target table

[0139] Figure 15 An architecture 1500 that can implement the disclosed embodiments is shown. Figure 15 The basic architecture 1500 includes several features that may be common to different embodiments of the disclosed technology, including one or more applications 1508 that can access a central computing system 1510, which may be a cloud computing system. The central computing system 1510 can operate by providing access to data stored in one or more remote database systems 1512. In turn, the remote database system 1512 can be accessed by one or more applications 1514. In some cases, application 1514 may also be application 1508. That is, some applications can only (directly) access data in the central computing system 1510, some applications can only access data in the remote database system 1512, while other applications can access data in both the central computing system and the remote database systems.

[0140] Other examples described herein may include components other than applications 1508, 1514, remote database system 1512, and central computing system 1510. Furthermore, the architectural components 1508, 1510, 1512, 1514, and especially the central computing system 1510, may be configured differently in different examples, as will be described further.

[0141] Central computing system 1510 may include query processor 1520. Query processor 1520 may include multiple components, including query optimizer 1522 and query executor 1524. Query optimizer 1522 may be responsible for determining query execution plan 1526 for queries to be executed using central computing system 1510. Query plan 1526 generated by query optimizer 1522 may include both a logical plan indicating, for example, the order of operations to be performed in the query (e.g., join, projection) and a physical plan for implementing such operations. Once developed by query optimizer 1522, query plan 1526 may be executed by query executor 1524. Query plan 1526 may be stored as a cached query plan 1530 in query plan cache 1528. When a query is resubmitted for execution, query processor 1520 may determine whether a cached query plan 1530 exists for that query. If so, cached query plan 1530 may be executed by query executor 1524. If not, the query optimizer 1522 generates a query plan 1526. In some cases, the cached query plan 1530 may become invalid, such as if changes are made to the database schema, or at least to components of the database schema used by the query (e.g., tables or views).

[0142] Data dictionary 1534 can maintain one or more database schemas for central computing system 1510. In some cases, central computing system 1510 can implement a multi-tenant environment, and different tenants can have different database schemas. In at least some cases, multiple database schemas can share at least some database schema elements.

[0143] Data dictionary 1534 can include definitions (or schemas) of different types of database objects, such as schemas for tables or views. While the following discussion refers to tables for ease of explanation, it should be understood that this discussion can be applied to other types of database objects, particularly those associated with retrieveable data, such as materialized views. A table schema can include information such as the table name, the number of attributes (or columns or fields) in the table, the names of the attributes, the data types of the attributes, the order in which the attributes should be displayed, primary key values, foreign keys, relationships with other database objects, partitioning information, or replication information.

[0144] The table schema maintained by data dictionary 1534 may include local table schema 1536, which may represent tables primarily maintained on central computing system 1510. Data dictionary 1534 may include replica table schema 1538, which may represent tables whose table data is at least partially stored on central computing system 1510 (or primarily managed by the database management system of the central computing system, even if stored outside the central computing system, such as in a data lake or another cloud service). Tables with data associated with replica table schema 1538 will typically periodically update their data from source tables, such as remote table 1544 of data store 1542 on remote database system 1512.

[0145] Replication can be accomplished using one or both of the replication services 1546 of the remote database system 1512 or 1548 of the central computing system 1510. In a specific example, the replication service could be any of the following: Intelligent Data Integration (SDI) services in an SAP SE system in Walldorf, Germany; SAP Landscape Transformation Replication Server; SAP Data Services; SAP Replication Server; SAP Event Stream Processor; or SAP HANA Direct Extractor Connection.

[0146] In some cases, the central computing system 1510 can access data in a remote database system 1512 without copying data from the remote database system, such as using federations. The data dictionary 1534 can store remote table schemas 1552 for remote tables, such as remote table 1544 of the remote database system 1512. Data in remote table 1544 can be accessed using a federation service 1556, such as using SAP SE's Intelligent Data Access Protocol from Waldorf, Germany. The federation service 1556 can be responsible for translating query operations into a format that can be processed by the appropriate remote database system 1512, sending the query operations to the remote database system, receiving the query results, and providing the query results to the query executor 1524.

[0147] Data dictionary 1534 may include virtual table schema 1560. Virtual table schema 1560 may be associated with table pointer 1562 and optionally with status information 1564. Table pointer 1562 may be a logical pointer used to identify which table should be accessed for data corresponding to virtual table schema 1560. For example, depending on the status of table pointer 1562, the table pointer may point to a remote table 1544 in remote database system 1512 or to a replica table 1566 (which may be generated from remote table 1544) located in data store 1568 in central computing system 1510. Data store 1568 may also store data in local table 1570, which may be defined by local table schema 1536.

[0148] As will be described further, table pointer 1562 can be changed between remote table 1544 and replica table 1566. In some cases, the user can manually change the table pointed to by table pointer 1562. In other cases, table pointer 1562 can be changed automatically, such as in response to the detection of a defined condition.

[0149] Status information 1564 may include an indicator that identifies virtual table mode 1560 as associated with remote table 1544 or replica table 1566. Status information 1564 may also include information about the replication status of replica table 1566. For example, once a request is made to change table pointer 1562 to point to replica table 1566, time may be spent before the replica table is ready for use. Status information 1564 may include the progress status of whether the replication process has started, completed, or generated replica table 1566.

[0150] Changes to the virtual table schema 1560 and the replica table 1566 associated with the virtual table schema can be managed by the virtual table service 1572. Although the virtual table service 1572 is shown as a separate component of the central computing system 1510, it can be incorporated into other components of the central computing system 1510, such as the query processor 1520 or the data dictionary 1534.

[0151] When a query is executed, it is processed by query processor 1520, including executing the query using query executor 1524 to retrieve data from one or both of the data store 1542 of remote database system 1512 or the data store 1568 of central computing system 1510. The query results can be returned to application 1508. Query results can also be cached, such as in cache 1578 of central computing system 1510. The cached results can be represented as a cached view 1580 (e.g., materialized query results).

[0152] Application 1514 can access data in remote database system 1512, such as through session manager 1586. Application 1514 can modify remote table 1544. When table pointer 1562 of virtual table schema 1560 references remote table 1544, changes made by application 1514 are reflected in the remote table. When table pointer 1562 references replica table 1566, changes made by application 1514 can be reflected in the replica table using replication service 1546 or replication service 1548.

[0153] Example 12—Example database artifact supporting virtual tables

[0154] Figure 16A and Figure 16B An example database artifact that can be used to implement the techniques disclosed herein is shown.

[0155] Database systems can track query performance. Queries that meet specific criteria, such as having a long enough execution time to meet a threshold or having resource usage that meets the threshold, can be designated as expensive queries. Expensive queries can be tracked, such as by creating records for that query in the corresponding table.

[0156] Figure 16A Table 1600 lists the properties of column 1602, which can be added to tables tracking expensive queries (e.g., M_EXPENSIVE_STATEMENTS). Column 1602 TABLE_TYPES can list whether the query includes a table formatted for row storage, a table formatted for column storage, or a table associated with a virtual table schema (and therefore can have a target table of variable types, such as a remote table, a replica table, or a cache table). In some cases, the query may include multiple table types, in which case column 1602 may include multiple values. In other cases, if the query includes multiple table types, multiple entries of the query can be included in the expensive query's table, where each entry includes a single value from column 1602.

[0157] In other cases, tables similar to Table 1600 are included, but without specifically identifying the table type being queried. Instead, values ​​such as Boolean values ​​can indicate whether the query includes any virtual tables. In another implementation, tables similar to Table 1600 are included; however, for queries that include virtual tables, information specifying the target type of the virtual table in the query is included (e.g., remote table target, replica table target, cached table target).

[0158] Figure 16A Table 1606 lists the properties of column 1608, which can be added to a table listing virtual table properties (e.g., columns in a table describing the virtual table schema). Column 1608 HAS_REPLICA can be used to provide indicators, such as Boolean values, indicating whether the virtual table targets a replica table. For example, this value can be false if the virtual table targets a remote table. This value can be updated to true if the virtual table is updated to target a replica table.

[0159] Figure 16A Table 1612 lists the properties of various columns that can be included in the table VIRTUAL_TABLE_REPLICAS, which contains information about replica tables targeted by one or more virtual table schemas. Column 1614 provides the schema name for the schema that includes the virtual table. For example, the schema name can identify... Figure 7One of the schemas 714. Column 1616 lists the names of the virtual table schemas in the schemas provided in column 1614. For example, the schema name could identify virtual table schema 718. Similarly, columns 1618 and 1620 can provide the names of the database schema and the replica table schema for the replica table referenced by the virtual table schema identified in column 1616. Note that in some cases, the replica table can be associated with a schema other than a specific user database schema (e.g., the schema identified in column 1614), such as a schema associated with a central computing system and used internally to manage virtual tables according to this disclosure.

[0160] Column 1622 can provide the type of the replica table. In some cases, this type can refer to the type of replication used for the replica table, such as whether asynchronous or synchronous replication is used to copy data from the source table (e.g., a remote table) to the replica table. In other cases, the type can refer to how queries are processed while the replica table is being prepared. Specifying asynchronous may result in the remote table continuing to be used for queries while the replica table is being prepared. In other words, DDL statements that change the table state may be effective before the query behavior actually changes. Specifying synchronous may result in queries being rejected or suspended until the replication is complete—in this case, DDL changes and query behavior changes are essentially concurrent. In a further implementation, the synchronous specification in column 1622 can indicate that a joined table (or the table position currently reflected in the value of the table pointer, such as the table position in the virtual table cache) should be used for queries until the latest replica table is available, and the asynchronous specification indicates that the replica table should be used for queries immediately, even if it is not up-to-date compared to its source table (e.g., a remote or joined table).

[0161] Column 1624 indicates whether the virtual table is currently using a replica table. That is, if column 1624 is FALSE, the remote table can be specified as the target of the virtual table schema. If column 1624 is true, replication of the remote table is enabled, and the virtual table schema targets the replica table.

[0162] Column 1626 can be used to indicate whether a replicated table can be shared by multiple virtual tables (e.g., different database schemas with multiple users), such as a value of true if the table is shareable, and false otherwise. Column 1628 can be used to indicate how the data in the replicated table will be loaded, such as when the replicated table is stored in a system with dynamic storage. The value of column 1628 can indicate whether the table is held entirely in permanent storage or in a buffer cache, and whether it can be loaded by pages or by specific columns. If the replicated table is also a shared table, various protocols can be used to determine the value of column 1628. For example, if multiple users request a replicated table but use different values ​​for column 1628, the protocol can choose to use the last requested value for column 1628. That is, if the replicated table already exists, a request to create a replicated table with a value for column 1628 that is different from the current value will result in a change to the value in column 1628 of the existing replicated table, rather than creating a new replicated table.

[0163] Alternatively, certain values ​​for column 1628 can be prioritized. For example, if any virtual table schema requests that the replica table be column-loadable, then this value will override any request that the replica table be page-loadable. However, if the schema associated with a request for a column-loadable table chooses to discard the replica table (e.g., switch to a remote table), but another schema is using that replica table, then discarding the replica table may cause the replica table to change from column-loadable to page-loadable, at least unless another explicit request is received from another database schema that the replica table should be column-loadable.

[0164] refer to Figure 16B Table 1660 can be used to track information about the replica table. Table 1660 can be used to provide additional information for the replica table, such as for monitoring the status of the replica table. Table 1660 may include column 1662 containing host information and column 1664 containing system port information where the replica table can be accessed (and may be used, for example, to indicate whether the replica table is located in memory, in a buffer cache, in a persistent storage device, in a data lake, etc.).

[0165] Table 1660 includes columns 1666, 1668, 1670, and 1672 for the schema name of the replica table, the table name of the replica table, the replica type, and the schema name of the virtual table. These can typically be, at least as per usual, for... Figure 16A This can be achieved as described in columns 1618, 1620, 1614, and 1616 of Table 1612. Column 1670 can specify the copy type and can be achieved, at least generally, as described in column 1622 of Table 1612.

[0166] A copy table can be associated with a subset, such as a partition. Column 1676 can be used to indicate a specific table partition associated with the records in table 1660, where a value of "0" can be used for an unpartitioned table.

[0167] Column 1680 can be used to indicate the status of the replicated table, such as whether replication is disabled, enabled, or whether a replica is currently being built. In some cases, whether replication is disabled or enabled may be related to whether any virtual table schema targets the replicated table. However, in some cases, the table can be replicated even if it is not currently the target of a virtual table schema. If a replica is being built, the progress of the replication process can be recorded in column 1678 (e.g., whether the replication process is 25% complete, 50% complete, 75% complete, 99% complete, etc.).

[0168] Replication can be enabled or disabled, depending on whether the virtual table schema targets a replicated table. The times associated with enabling or disabling replication can be recorded in columns 1682 and 1684, respectively.

[0169] In some implementations, the replica table can be shared. When the replica table is shareable, column 1686 can be used to indicate multiple virtual table schemas targeting a given replica table.

[0170] Example 13—Example Query Language Operations Supporting Virtual Tables

[0171] Figure 17 Example query language statements, such as DDL statements, that can be used to implement the various features described in this disclosure, including those described in Examples 10 and 11. Statement 1708 is an example statement for creating a virtual table that targets a replica table created from a remote table. Statement 1708 includes arguments for the name of the virtual table, the location of the remote table (which may include the remote table name), any properties optionally specified for the remote table / location, an indicator of whether the replica table will be shared, information optionally for partitioning the replica table, any optional load cell parameters for the replica table (e.g., whether the replica table will be page-loadable or column-loadable), and the location of the replica table (e.g., in a location associated with an in-memory table, in a data lake, or in a location managed by a dynamic storage manager).

[0172] Statement 1712 is an example command for creating a replica table for a remote table whose virtual table schema is the target. As described in statement 1708, statement 1712 includes arguments for the virtual table name, whether the table will be shared, optional information about partitioning or load cell information, and the location of the replica table.

[0173] Statement 1716 is an example command for discarding a table of copies. Statement 1716 takes the name of the virtual table schema and an identifier indicating whether all copies should be discarded or only copies at one or more specified locations as arguments.

[0174] If the virtual table schema is associated with both a remote table and a replica table, then statements 1720 and 1722 can be used to switch between targeting the replica table and targeting the remote table, respectively. Statements 1720 and 1722 take the name of the virtual table schema and an identifier as arguments regarding whether all or one or more specified replicas should be targeted or not.

[0175] Statement 1726 can be used to change the location of a replica table, such as between the primary database storage device (e.g., in-memory storage) and permanent storage, dynamic storage, or external storage (e.g., data lake storage). Statement 1726 takes the name of the virtual table schema, the current location of the replica table, and the location where the replica table should be moved as arguments.

[0176] Statement 1730 can be used to change the properties of the copy table, such as partition information or load cell type. The arguments of statement 1730 include the name of the virtual table schema, the location of the copy table, and an indicator of the property to be changed, such as partition information or load cell information.

[0177] Query language statements can be used to create or modify tables to select other aspects of exposed technologies, either using them or not. For example, DDL statements for creating a table can include enabling caching (e.g., in...). Figure 15 The options for caching virtual tables (in case of 1504) are available. DDL statements can be used to change the caching behavior of existing tables, such as enabling or disabling caching. Similarly, caching behavior can be changed, such as specifying whether data should be expected to be placed in the cache or fetched and cached when data is requested. Caching options, including cache size or priority for specific tables (for example, used to determine which tables should be removed from the cache when it is full), can also be specified by DDL statements or similar commands.

[0178] Example 14—Computing Systems

[0179] Figure 18 A general example of a suitable computing system 1800 that can realize the said innovation is described. The computing system 1800 is not intended to impose any limitation on the scope or functionality of this disclosure, as the innovation can be implemented in a variety of general-purpose or special-purpose computing systems.

[0180] refer to Figure 18The computing system 1800 includes one or more processing units 1810, 1815 and memories 1820, 1825. Figure 18 In this diagram, the basic configuration 1830 is included within the dashed lines. Processing units 1810 and 1815 execute computer-executable instructions, such as those used to implement the features described in Examples 1-13. The processing units can be general-purpose central processing units (CPUs), processors in application-specific integrated circuits (ASICs), or any other type of processor. In a multiprocessor system, multiple processing units execute computer-executable instructions to increase processing power. For example, Figure 18 A central processing unit 1810 and a graphics processing unit or coprocessor 1815 are shown. Physical memories 1820 and 1825 may be volatile memory (e.g., registers, cache, RAM), non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.), or some combination of both, accessible by the processing units 1810 and 1815. Memories 1820 and 1825 store software 1880 implementing one or more innovations described herein in the form of computer-executable instructions suitable for execution by the processing units 1810 and 1815.

[0181] The computing system 1800 may have additional features. For example, the computing system 1800 includes a storage device 1840, one or more input devices 1850, one or more output devices 1860, and one or more communication connections 1870, including input devices, output devices, and communication connections for user interaction. Interconnection mechanisms (not shown), such as buses, controllers, or networks, interconnect the components of the computing system 1800. Typically, operating system software (not shown) provides an operating environment for other software executing in the computing system 1800 and coordinates the activities of the components of the computing system 1800.

[0182] The physical storage device 1840 may be removable or non-removable, including a magnetic disk, magnetic tape or cassette tape, CD-ROM, DVD, or any other medium that can be used to store information in a non-transitory manner, and may be accessed within the computing system 1800. The storage device 1840 stores instructions for software 1880 that implements one or more of the innovations described herein.

[0183] Input device 1850 may be a touch input device, such as a keyboard, mouse, pen or trackball, voice input device, scanning device, or other device that provides input to computing system 1800. Output device 1860 may be a monitor, printer, speaker, CD burner, or another device that provides output from computing system 1800.

[0184] The communication connection 1870 allows communication with another computing entity via a communication medium. The communication medium conveys information, such as computer-executable instructions, audio or video input or output, or other data, in the form of modulated data signals. A modulated data signal is a signal in which one or more characteristics are set or altered in such a way that information is encoded within it. By way of example and not limitation, the communication medium can be electrical, optical, RF, or other carriers.

[0185] These innovations can be described in the general context of computer-executable instructions, such as those included in program modules, which execute on a computing system targeting a real or virtual processor. Typically, program modules or components include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of a program module can be combined or split among program modules as desired. The computer-executable instructions of a program module can execute within a local or distributed computing system.

[0186] The terms “system” and “device” are used interchangeably herein. Neither term implies any limitation on the type of computing system or computing device unless the context clearly indicates otherwise. In general, a computing system or computing device can be local or distributed and can include any combination of dedicated hardware and / or general-purpose hardware with software that implements the functions described herein.

[0187] In the various examples described herein, modules (e.g., components or engines) can be "coded" to perform certain operations or provide certain functions, instructing computer-executable instructions that can execute the module to perform such operations, cause such operations to be performed, or otherwise provide such functions. While the functionality described with respect to a software component, module, or engine can be executed as a discrete software unit (e.g., a program, function, class method), it does not need to be implemented as a discrete unit. That is, the functionality can be incorporated into a larger or more general program, such as one or more lines of code within a larger or more general program.

[0188] For ease of description, the terms "determine" and "use" are used in detail to describe computer operations in a computing system. These terms are high-level abstractions of the operations performed by the computer and should not be confused with human actions. The actual computer operations corresponding to these terms vary depending on the implementation method.

[0189] Example 15—Cloud Computing Environment

[0190] Figure 19An example cloud computing environment 1900 that can implement the technology is described. Cloud computing environment 1900 includes cloud computing service 1910. Cloud computing service 1910 may include various types of cloud computing resources, such as computer servers, data repositories, network resources, etc. Cloud computing service 1910 may be location-centralized (e.g., provided by a data center of an enterprise or organization) or distributed (e.g., provided by various computing resources located in different locations, such as different data centers and / or located in different cities or countries).

[0191] The cloud computing service 1910 is utilized by various types of computing devices (e.g., client computing devices), such as computing devices 1920, 1922, and 1924. For example, computing devices (e.g., 1920, 1922, and 1924) can be computers (e.g., desktop computers or laptop computers), mobile devices (e.g., tablet computers or smartphones), or other types of computing devices. For example, computing devices (e.g., 1920, 1922, and 1924) can utilize the cloud computing service 1910 to perform computing operations (e.g., data processing, data storage, etc.).

[0192] Example 16—Implementation Method

[0193] While some of the disclosed methods are described in a specific, sequential order for ease of presentation, it should be understood that this description includes rearrangement unless the specific language used herein requires a particular order. For example, operations described sequentially may be rearranged or performed concurrently in some cases. Furthermore, for simplicity, the accompanying figures may not show the various ways in which the disclosed methods can be combined with other methods.

[0194] Any disclosed method can be implemented as computer-executable instructions or a computer program product stored on one or more computer-readable storage media and executed on a computing device (e.g., any available computing device, including smartphones or other mobile devices containing computing hardware). A tangible computer-readable storage medium is any available tangible medium that can be accessed within a computing environment (e.g., one or more optical media discs, such as DVDs or CDs, volatile memory components (such as DRAM or SRAM), or non-volatile memory components (such as flash memory or hard disks)). As an example and reference. Figure 18 Computer-readable storage media include memories 1820 and 1825 and storage device 1840. The term "computer-readable storage medium" excludes signals and carrier waves. Furthermore, the term "computer-readable storage medium" excludes communication connections (e.g., 1870).

[0195] Any computer-executable instructions used to implement the disclosed technology, and any data created and used during the implementation of the disclosed embodiments, may be stored on one or more computer-readable storage media. The computer-executable instructions may be, for example, part of a dedicated software application or a software application accessed or downloaded via a web browser or other software application (such as a remote computing application). Such software may execute on, for example, a single local computer (e.g., any suitable commercially available computer) or in a network environment using one or more networked computers (e.g., via the Internet, a wide area network, a local area network, a client-server network (such as a cloud computing network or other such strictly networked networks)).

[0196] For clarity, only selected aspects based on software implementations are described. It should be understood that the disclosed techniques are not limited to any particular computer language or program. For example, the disclosed techniques can be implemented by software written in C++, Java, Perl, JavaScript, Python, Ruby, ABAP, SQL, Adobe Flash, or any other suitable programming language, or in some examples, by markup languages ​​such as HTML or XML, or a combination of suitable programming languages ​​and markup languages. Similarly, the disclosed techniques are not limited to any particular computer or hardware type.

[0197] Furthermore, any software-based implementation (including, for example, computer-executable instructions for causing a computer to perform any disclosed method) can be uploaded, downloaded, or remotely accessed via suitable communication means. Such suitable communication means include, for example, the Internet, the World Wide Web, intranets, software applications, cables (including fiber optic cables), magnetic communications, electromagnetic communications (including RF, microwave, and infrared communications), electronic communications, or other such communication means.

[0198] The disclosed methods, apparatuses, and systems should not be construed as limiting in any way. Rather, this disclosure refers to all novel and non-obvious features and aspects of the various disclosed embodiments, individually and in various combinations and sub-combinations of each other. The disclosed methods, apparatuses, and systems are not limited to any particular aspect or feature or combination thereof, nor are the disclosed embodiments required to have any one or more particular advantages or problems to be solved.

[0199] The techniques from any example can be combined with the techniques described in any one or more other examples. Given that the principles of the disclosed techniques can be applied to many possible embodiments, it should be recognized that the illustrated embodiments are examples of the disclosed techniques and should not be considered as limiting the scope of the disclosed techniques. Rather, the scope of the disclosed techniques includes what is covered by the scope and spirit of the appended claims.

Claims

1. A computing system, comprising: At least one hardware processor; At least one memory, coupled to at least one hardware processor; and One or more computer-readable storage media storing computer-executable instructions that, when executed, cause a computing system to perform operations, including: Receive an instruction to create a first instance of a first data artifact in a hierarchical pattern, the hierarchical pattern including multiple data artifacts distributed across multiple layers of a hierarchy of the hierarchical pattern. Create a first instance of a first data artifact at the first of the multiple layers of the hierarchy, wherein the first instance of the first data artifact includes or points to the first data; Receive instructions to create a second instance of the first data artifact; A second instance of the first data artifact is created at the first level of the hierarchy. The second instance of the first data artifact includes or points to the second data, wherein the second data includes all or part of the first data, or is wholly or partially different from the first data. The instruction is received to create at least a first instance of a second data artifact in a hierarchical pattern at a second level of the hierarchy, wherein the first instance of the first data artifact points to at least a first instance of the second data artifact, and the second level of the hierarchy is lower than the first level of the hierarchy; and Create at least the first instance of the second data artifact.

2. The computing system according to claim 1, further comprising: Receive a first data access request, the first data access request specifying a first data artifact; A first instance of the first data artifact will be used to process the first data access request; Processing a first data access request, the processing includes retrieving first data from at least a first instance of a first data artifact, or retrieving first data at a location specified by the first instance of the first data artifact; as well as In response to a first data access request, return at least a portion of the first data, or return second data that is at least partially generated or derived from at least a portion of the first data.

3. The computing system according to claim 2, wherein, Determining that a first instance of a first data artifact will be used to process a first data access request includes: analyzing at least one data access operation specified in the first data access request, wherein the at least one data access operation is a type that at least partially determines that a first instance of the first data artifact is used to process the first data access request and a second instance of the first data artifact is not used to process the first data access request.

4. The computing system according to claim 2, wherein, Determining that a first instance of a first data artifact will be used to process a first data access request includes: determining that an identifier of the source of the first data access request indicates that a first instance of the first data artifact will be used to process the first data access request.

5. The computing system according to claim 2, wherein, Determining that a first instance of a first data artifact will be used to process a first data access request includes: determining that the first data access request includes an indicator of a first data access request type or is associated with an indicator of a first data access request type, the indicator of the first data access request type being used at least in part to determine that a first instance of the first data artifact will be used to process the first data access request.

6. The computing system according to claim 2, further comprising: Receive a second data access request, which specifies the first data artifact; For the first layer of the hierarchy, a second instance of the first data artifact is determined to be used to process the second data access request; Processing a second data access request, the processing includes retrieving third data from at least a second instance of a first data artifact, or retrieving third data at a location specified by a second instance of the first data artifact, wherein the third data includes all or part of the first data, or is wholly or partially different from the first data; and In response to a second data access request, return at least a portion of the third data, or return fourth data that is at least partially generated or derived from at least a portion of the third data.

7. The computing system according to claim 2, wherein, The first data access request specifies a second data artifact, the second data artifact including a first instance and a second instance, and the operation further includes: The first instance of the second data artifact will be used to process the first data access request.

8. The computing system according to claim 7, wherein, Determining that a first instance of the second data artifact will be used to process the first data access request includes: In response to determining that a first instance of a first data artifact will be used to process a first data access request, a command is sent instructing or indicating that a first instance of a second data artifact will be used to process the first data access request; or The first data access request, or parameters or operations included in or associated with the first data access request, are provided to switching logic for selecting between a first instance of the second data artifact and a second instance of the second data artifact, wherein the output of the switching logic is at least partially independent of the determination that the first instance of the first data artifact will be used to process the first data access request.

9. One or more computer-readable storage media, comprising: Computer-executable instructions, when executed by a computing system including at least one hardware processor and at least one memory coupled to the at least one hardware processor, cause the computing system to receive instructions to create a first instance of a first data artifact in a hierarchical pattern, the hierarchical pattern including a plurality of data artifacts distributed in a plurality of layers of a hierarchy of the hierarchical pattern. Computer-executable instructions, when executed by a computing system, cause the computing system to create a first instance of a first data artifact at the first of a plurality of layers of a hierarchy, the first instance of the first data artifact including or pointing to first data; Computer-executable instructions, when executed by a computing system, cause the computing system to receive instructions to create a second instance of a first data artifact; Computer-executable instructions, when executed by a computing system, cause the computing system to create a second instance of a first data artifact at a first layer of the hierarchy, the second instance of the first data artifact including or pointing to second data, wherein the second data includes all or part of the first data, or is wholly or partially different from the first data; Computer-executable instructions, when executed by a computing system, cause the computing system to receive instructions to create at least a first instance of a second data artifact in a hierarchical schema at a second level of the hierarchy, wherein the first instance of the first data artifact points to at least a first instance of the second data artifact, and the second level of the hierarchy is lower than the first level of the hierarchy; and Computer-executable instructions, when executed by a computing system, cause the computing system to create at least a first instance of a second data artifact.

10. The one or more computer-readable storage media according to claim 9, further comprising: Computer-executable instructions, when executed by a computing system, cause the computing system to receive a modeling pattern, the modeling pattern comprising multiple modeling artifacts distributed across multiple layers of the modeling pattern hierarchy; Computer-executable instructions, when executed by a computing system, cause the computing system to determine that multiple instances of a first data artifact will be created, the first data artifact corresponding to a first modeling artifact among multiple modeling artifacts, wherein, in response to determining that multiple instances of the first data artifact are to be created, a first instance of creating the first data artifact and a second instance of creating the first data artifact are executed.

11. The one or more computer-readable storage media of claim 10, further comprising: Computer-executable instructions, when executed by a computing system, enable the computing system to determine that the first layer is the highest layer of the hierarchical model; and The computer-executable instructions, when executed by the computing system, cause the computing system to, in response to determining that the first layer is the highest layer of the hierarchical pattern, set a second instance of the first data artifact to retrieve the first data via data association.

12. The one or more computer-readable storage media according to claim 11, further comprising: Computer-executable instructions, when executed by a computing system, cause the computing system to set a first instance of a first data artifact to retrieve data from a second data artifact, or to set data that is available using the second data artifact.

13. The one or more computer-readable storage media of claim 10, further comprising: Computer-executable instructions, when executed by a computing system, enable the computing system to determine that the second layer is the lowest layer of the hierarchical model; and The computer-executable instructions, when executed by the computing system, cause the computing system to, in response to determining that the second layer is the lowest layer of the hierarchical pattern, set the second data artifact to retrieve data via data copying.

14. The one or more computer-readable storage media of claim 10, further comprising: Computer-executable instructions, when executed by a computing system, cause the computing system to set a first instance of a first data artifact to retrieve data from a second data artifact, or to set data that is available using the second data artifact.

15. One or more computer-readable storage media according to claim 10, wherein, The first data artifact includes logical pointers that can be set to point to different data sources.

16. A method implemented in a computing system, the computing system including at least one hardware processor and at least one memory coupled to the at least one hardware processor, the method comprising: Receive an instruction to create a first instance of a first data artifact in a hierarchical pattern, the hierarchical pattern including multiple data artifacts distributed across multiple layers of a hierarchy of the hierarchical pattern. Create a first instance of a first data artifact at the first of the multiple layers of the hierarchy, wherein the first instance of the first data artifact includes or points to the first data; Receive instructions to create a second instance of the first data artifact; A second instance of the first data artifact is created at the first level of the hierarchy. The second instance of the first data artifact includes or points to the second data, wherein the second data includes all or part of the first data, or is wholly or partially different from the first data. The instruction is received to create at least a first instance of a second data artifact in a hierarchical pattern at a second level of the hierarchy, wherein the first instance of the first data artifact points to at least a first instance of the second data artifact, and the second level of the hierarchy is lower than the first level of the hierarchy; and Create at least the first instance of the second data artifact.

17. The method according to claim 16, wherein, The first instance of the first data artifact includes a view that references at least the first instance of the second data artifact.

18. The method according to claim 17, wherein, The second data artifact includes at least a first instance of a third data artifact at a reference level, the third data artifact being at least a first instance of a third data artifact at a lower level than the second data artifact.

19. The method of claim 16, further comprising: Receive a first data access request, the first data access request specifying that a first instance of a first data artifact will be used to process the first data access request; and Receive a second data access request, which specifies that a second instance of the first data artifact will be used to process the second data access request.

20. The method of claim 16, further comprising: Receive the first data access request; The analysis includes operations or information that are within or associated with the first data access request; The first data access request is executed using a first instance of the first data artifact, based at least in part on the analysis of operations or information included in or associated with the first data access request. Receive the second data access request; The analysis includes operations or information that are within or associated with the second data access request; and The second data access request is executed using a second instance of the first data artifact, based at least in part on the analysis of operations or information included in or associated with the second data access request.

Images