Projection constraints in query processing systems

Abstract

When the database system receives a query, the constraint system implements projection constraints on data values stored in specified columns of the shared data set. Projection constraints identify that data in a column may be limited to be projected (e.g., rendered, read, output) into output for a received query while allowing specified operations to be performed on the data and providing corresponding output. For example, the projection constraint may indicate a context of a query that triggers the constraint, such as triggering the constraint based on a user submitting the query. Implementing projection constraints on queries received at a database system allows the data to be anonymously shared and used by entities to perform various operations without the need for tagging the data.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of priority to U.S. patent application serial number 17 / 934,814, filed on September 23, 2022, which claims the benefit of priority to U.S. Provisional Application No. 63 / 366,281, filed on June 13, 2022, which are incorporated herein by reference in their entireties. Technical Field

[0002] The present disclosure generally relates to efficiently managing access to shared data in a database. background

[0003] Databases are used for data storage and access in computing applications. The goal of database storage is to provide large amounts of information in an organized manner so that it can be accessed, managed, and updated. In a database, data can be organized into rows, columns, and tables. A database platform can have different databases managed by different users. Users may seek to share their database data with each other; however, it is difficult to share database data in a secure and scalable manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The various drawings depict only example embodiments of the present disclosure and should not be considered to limit the scope of the present disclosure.

[0005] Figure 1 An example computing environment is shown in which a network-based database system may implement a projection constraint according to some example embodiments.

[0006] Figure 2 is a block diagram illustrating components of a computing service manager according to some example embodiments.

[0007] Figure 3 is a block diagram illustrating components of an execution platform according to some example embodiments.

[0008] Figure 4 Examples of database restrictions on access to database data are shown, according to some example embodiments.

[0009] Figure 5 Example table data is shown according to some example embodiments.

[0010] Figure 6A-6C An example database schema is shown for implementing query templates for multiple entities sharing data in a data cleansing environment, according to some example embodiments.

[0011] Fig. 7A and Figure 7BAn example database architecture for implementing a defined access clean room is shown, according to some example embodiments.

[0012] Figure 8 is a block diagram illustrating components of a restraint system according to some example embodiments.

[0013] Figure 9A-9D Various data sharing scenarios are shown in which projection constraints may be implemented according to some example embodiments.

[0014] Fig.10 A flowchart of a method for implementing projection constraint-based access to a data set on a distributed database is shown, according to some example embodiments.

[0015] Fig.11 A diagrammatic representation of a machine in the form of a computer system within which a set of instructions may be executed, for causing the machine to perform any one or more of the methodologies discussed herein, is shown according to some embodiments of the present disclosure. Detailed Description

[0016] The following description includes systems, methods, techniques, instruction sequences and computing machine program products that embody the illustrative embodiments of the present disclosure. In the following description, for the purpose of explanation, many specific details are set forth to provide an understanding of the various embodiments of the subject matter of the present invention. However, it is apparent to those skilled in the art that the embodiments of the subject matter of the present invention can be implemented without these specific details. Typically, well-known instruction instances, protocols, structures and techniques are not necessarily shown in detail.

[0017] Databases are used by various entities (e.g., businesses, people, organizations, etc.) to store data. For example, a retailer may store data describing purchases (e.g., product, date, price) and data describing the purchaser (e.g., name, address, email address). Similarly, an advertiser may store data describing the performance of its advertising campaigns, such as the advertisements served to users, the dates on which the advertisements were served, information about the user (e.g., name, address, email address), etc.

[0018] In some cases, entities may wish to share their data with each other. For example, retailers and advertisers may wish to share their data to determine the effectiveness of an advertising campaign, such as by determining whether a user who was provided with an advertisement for a product ultimately purchased the product. In these types of cases, the entities may wish to maintain the confidentiality of some or all of the data they have collected and stored in their respective databases. For example, retailers and / or advertisers may wish to maintain the confidentiality of personally identifiable information (PII) such as user names, addresses, email addresses, credit card numbers, etc.

[0019] Current solutions address this problem through the use of tokenization, where data values ​​stored in corresponding databases are anonymized into tokens in a consistent and deterministic manner before being shared. For example, each entity can use a hashing algorithm to generate a hash value (e.g., a token) that represents each data value stored in the database. Entities share these tokens with each other, which can then be used to compare data and find matching values ​​while maintaining the confidentiality of the underlying data.

[0020] However, tokenization has several disadvantages. One disadvantage is that tokenization does not allow for fuzzy or approximate matches. In some cases, data may not be entered into various databases in a consistent format or correctly. Since tokenization utilizes a deterministic algorithm to generate tokens, only exact matches can be identified and these variations in format and data entries (e.g., fuzzy matches) are not taken into account. Another disadvantage of tokenization is the complexity and computing resources required to generate the tokens. For example, a hashing algorithm is used to generate hash values ​​for a large number of data values ​​stored in a corresponding database. Another disadvantage of tokenization is limited functionality because tokens can only be used to find exact matches between two data sets. Other functions, such as determining the number of data entries that do not match but include a specified string or character set, cannot be performed.

[0021] To mitigate these issues, a database system may utilize a constraint system that enforces projection constraints on data values ​​stored in a specified column of a shared data set when the database system receives a query. A projection constraint identifies that data in a column may be restricted from being projected (e.g., rendered, read, output) into an output for a received query while allowing specified operations to be performed on the data and providing corresponding output. For example, a projection constraint may indicate a context of a query that triggers the constraint, such as based on a user submitting the query.

[0022] Entities that share data can define projection constraints that will be attached to various columns of a shared data set. For example, an entity can define one or more columns to which a projection constraint should be attached, as well as conditions for triggering the constraint. When a database system receives a query directed to a shared data set, the constraint system accesses the data required to process the query from the shared database and determines whether to attach a projection constraint to any column of the shared data set from which data is accessed. If a projection constraint is attached to one of the columns, the constraint system determines whether the projection constraint should be implemented based on the context of the query and generates output accordingly. For example, if a projection constraint should be implemented, the constraint system can generate output that does not include data values ​​stored in the column, but can provide output determined based on the constrained data, such as the number of matches, the number of fuzzy matches, the number of matches including a specified string, unconstrained data associated with the constrained data, and the like.

[0023] Enforcing projection constraints on queries received at a database system allows data to be anonymously shared and used by entities to perform various operations without the need to tokenize the data. Thus, the difficulty and computational resources required for tokenization are eliminated, and additional functions and operations (e.g., fuzzy matching) can be performed while maintaining the confidentiality of the specified data values.

[0024] Figure 1 An example computing environment 100 is shown in which a network-based database system 102 can implement projection constraints according to some example embodiments. In order to avoid obscuring the subject matter of the present invention with unnecessary detail, Figure 1 Various functional components that are not closely related to conveying an understanding of the subject matter of the present invention are omitted. However, those skilled in the art will readily recognize that various additional functional components can be included as part of the computing environment 100 to facilitate additional functions not specifically described herein. In other embodiments, the computing environment can include another type of network-based database system or cloud data platform.

[0025] As shown, computing environment 100 includes a cloud storage platform 104 (e.g., AWS ® , Microsoft Azure BlobStorage ® , or Google Cloud Storage). The network-based database system 102 is a network-based system for reporting and analyzing integrated data from one or more different sources including one or more storage locations within a cloud storage platform 104. The cloud storage platform 104 includes multiple computing machines and provides computer system resources, such as data storage and computing power, to the network-based database system 102 on demand.

[0026] The network-based database system 102 includes a computing service manager 108, an execution platform 110, and one or more metadata databases 112. The network-based database system 102 hosts and provides data reporting and analysis services to multiple client accounts.

[0027] The computing service manager 108 coordinates and manages the operation of the network-based database system 102. The computing service manager 108 also performs query optimization and compilation, and manages clusters of computing services (also referred to as "virtual warehouses") that provide computing resources. The computing service manager 108 can support any number of client accounts, such as end users providing data storage and retrieval requests, system administrators managing the systems and methods described herein, and other components / devices that interact with the computing service manager 108.

[0028] The computing service manager 108 also communicates with a client device 114. The client device 114 corresponds to a user of one of a plurality of client accounts supported by the network-based database system 102. The user may utilize the client device 114 to submit data storage, retrieval, and analysis requests to the computing service manager 108.

[0029] The computing service manager 108 is also coupled to one or more metadata databases 112, which store metadata related to various functions and aspects associated with the network-based database system 102 and its users. For example, the metadata database 112 may include a summary of data stored in a remote data storage system and data available from a local cache. In addition, the metadata database 112 may include information about how to divide and organize data in a remote data storage system (e.g., a cloud storage platform 104) and a local cache. As discussed herein, a "micro-partition" is a bulk storage unit, and each micro-partition has a continuous storage unit. For example, each micro-partition may contain between 50 MB and 500 MB of uncompressed data (note that the actual size in storage may be smaller because the data may be stored compressed). Groups of rows in a table may be mapped to separate micro-partitions organized in columns. This size and structure allows for extremely fine-grained selection of micro-partitions to be scanned, which may include millions or even hundreds of millions of micro-partitions. This granular selection process for micro-partitions to be scanned is referred to herein as "pruning." Pruning involves using metadata to determine which parts of a table (including which micro-partitions or groups of micro-partitions in a table) are irrelevant to a query, thereby avoiding those irrelevant micro-partitions when responding to the query and only scanning relevant micro-partitions to respond to the query. Metadata about all rows stored in a micro-partition can be automatically collected, including: the value range of each column in the micro-partition; the number of different values; and / or other attributes for optimized and efficient query processing. In one embodiment, micro-partitioning can be automatically performed on all tables. For example, a table can be transparently partitioned using sorting that occurs when inserting / loading data. However, it should be understood that this disclosure of micro-partitioning is merely exemplary and should be considered non-limiting. It should be understood that micro-partitioning can include other database storage devices without departing from the scope of the present disclosure. The information stored by the metadata database 112 (e.g., a key-value data storage) allows systems and services to determine whether a piece of data (e.g., a given partition) needs to be accessed without loading or accessing the actual data from the storage device.

[0030] The computing service manager 108 is also coupled to an execution platform 110, which provides a plurality of computing resources for performing various data storage and data retrieval tasks. The execution platform 110 is coupled to a cloud storage platform 104. The cloud storage platform 104 includes a plurality of data storage devices 120-1 to 120-N. In some embodiments, the data storage devices 120-1 to 120-N are cloud-based storage devices located in one or more geographic locations. For example, the data storage devices 120-1 to 120-N may be part of a public cloud infrastructure or a private cloud infrastructure. The data storage devices 120-1 to 120-N may be hard disk drives (HDDs), solid state drives (SSDs), storage clusters, Amazon S3 TM Storage system or any other data storage technology. In addition, the cloud storage platform 104 may include a distributed file system (such as Hadoop Distributed File System (HDFS)), an object storage system, and the like.

[0031] The execution platform 110 includes a plurality of computing nodes. A set of processes on the computing nodes executes a query plan compiled by the computing service manager 108. The set of processes may include: a first process that executes the query plan; a second process that uses a least recently used (LRU) strategy to monitor and delete cache files and implement an out of memory (OOM) error mitigation process; a third process that extracts health information from process logs and states to send back to the computing service manager 108; a fourth process that establishes communication with the computing service manager 108 after system boot; and a fifth process that handles all communications with the computing cluster for a given job provided by the computing service manager 108 and transmits information back to the computing service manager 108 and other computing nodes of the execution platform 110.

[0032] In some embodiments, the communication links between the elements of computing environment 100 are implemented via one or more data communication networks. These data communication networks can utilize any communication protocol and any type of communication medium. In some embodiments, the data communication network is a combination of two or more data communication networks (or subnetworks) coupled to each other. In alternative embodiments, these communication links are implemented using any type of communication medium and any communication protocol.

[0033] The computing service manager 108, the metadata database 112, the execution platform 110 and the cloud storage platform 104 are Figure 1104 as separate discrete components. However, each of the computing service manager 108, the metadata database 112, the execution platform 110, and the cloud storage platform 104 can be implemented as a distributed system (e.g., distributed across multiple systems / platforms at multiple geographic locations). In addition, each of the computing service manager 108, the metadata database 112, the execution platform 110, and the cloud storage platform 104 can scale up or down (independently of each other) based on changes in received requests and changing needs of the network-based database system 102. Therefore, in the described embodiment, the network-based database system 102 is dynamic and supports regular changes to meet current data processing needs.

[0034] During typical operation, the network-based database system 102 processes multiple jobs determined by the computing service manager 108. These jobs are scheduled and managed by the computing service manager 108 to determine when and how to execute the job. For example, the computing service manager 108 can divide the job into multiple discrete tasks, and can determine what data is needed to execute each of the multiple discrete tasks. The computing service manager 108 can assign each of the multiple discrete tasks to one or more nodes of the execution platform 110 to process the task. The computing service manager 108 can determine what data is needed to process the task, and further determine which nodes within the execution platform 110 are most suitable for processing the task. Some nodes may have cached the data required to process the task and are therefore good candidates for processing the task. The metadata stored in the metadata database 112 assists the computing service manager 108 in determining which nodes in the execution platform 110 have cached at least a portion of the data required to process the task. One or more nodes in the execution platform 110 use the data cached by the node and the data retrieved from the cloud storage platform 104 when necessary to process the task. It is desirable to retrieve as much data as possible from the cache within the execution platform 110 because the retrieval speed is typically much faster than retrieving data from the cloud storage platform 104 .

[0035] like Figure 1 As shown, the computing environment 100 separates the execution platform 110 from the cloud storage platform 104. In this arrangement, the processing resources and cache resources in the execution platform 110 operate independently of the data storage devices 120-1 to 120-N in the cloud storage platform 104. Therefore, the computing resources and cache resources are not limited to specific data storage devices 120-1 to 120-N. Instead, all computing resources and all cache resources can retrieve data from any data storage resource in the cloud storage platform 104, and store data therein.

[0036] Figure 2is a block diagram illustrating components of the computing service manager 108 according to some embodiments of the present disclosure. Figure 2 As shown, the computing service manager 108 includes a credential management system 204 and an access manager 202 coupled to a data storage device 206, which is an example of a metadata database 112. The access manager 202 handles authentication and authorization tasks for the system described herein. The credential management system 204 facilitates access to external resources, such as data resources in a remote storage device, using remotely stored credentials. As used herein, a remote storage device may also be referred to as a "persistent storage device" or a "shared storage device." For example, the credential management system 204 may create and maintain remote credential storage definitions and credential objects (e.g., in the data storage device 206). The remote credential storage definition identifies the remote credential storage and includes access information for accessing security credentials from the remote credential storage. The credential object uses non-sensitive information (e.g., a text string) to identify one or more security credentials that will be retrieved from the remote credential storage for accessing external resources. When a request to call an external resource is received at runtime, credential management system 204 and access manager 202 use information stored in data storage device 206 (e.g., access metadata database, credential objects, and credential store definitions) to retrieve security credentials for accessing the external resource from a remote credential store.

[0037] The request processing service 208 manages received data storage requests and data retrieval requests (e.g., jobs to be performed on database data). For example, the request processing service 208 can determine data for processing received queries (e.g., data storage requests or data retrieval requests). The data may be stored in a cache within the execution platform 110, or in a data storage device in the cloud storage platform 104.

[0038] Management console service 210 supports access to various systems and processes by administrators and other system managers. In addition, management console service 210 can receive requests for executing jobs and monitoring workloads on the system.

[0039] The computing service manager 108 also includes a job compiler 212, a job optimizer 214, and a job executor 216. The job compiler 212 parses the job into multiple discrete tasks and generates execution code for each of the multiple discrete tasks. The job optimizer 214 determines the best method to execute the multiple discrete tasks based on the data that needs to be processed. The job optimizer 214 also handles various data pruning operations and other data optimization techniques to improve the speed and efficiency of executing the job. The job executor 216 executes the execution code of the job received from the queue or determined by the computing service manager 108.

[0040] The job scheduler and coordinator 218 sends the received jobs to the appropriate service or system for compilation, optimization and dispatch to Figure 1 For example, the jobs can be prioritized and then processed in order of priority. In an embodiment, the job scheduler and coordinator 218 determines the jobs that are Figure 1 The job scheduler and coordinator 218 prioritizes internal jobs scheduled by the computing service manager 108 over other "external" jobs (such as user queries that may be scheduled by other systems in the database but may utilize the same processing resources in the execution platform 110). In some embodiments, the job scheduler and coordinator 218 identifies or assigns specific nodes in the execution platform 110 to process specific tasks. The virtual warehouse manager 220 manages the operation of multiple virtual warehouses implemented in the execution platform 110. For example, the virtual warehouse manager 220 can generate a query plan for executing a received query. As discussed in further detail below, the data clean room system 230 allows dynamic restricted data access to shared data sets. As discussed in further detail below, the constraint system 240 provides projection constraints on data values ​​stored in specified columns of shared data sets.

[0041] As shown, the computing service manager 108 includes a configuration and metadata manager 222 that manages information related to data stored in remote data storage devices and local buffers (e.g., buffers in the execution platform 110). The configuration and metadata manager 222 uses metadata to determine which data files need to be accessed to retrieve data for processing a particular task or job. The monitor and workload analyzer 224 supervises the processes executed by the computing service manager 108 and manages the allocation of tasks (e.g., workloads) between execution nodes and virtual warehouses in the execution platform 110. The monitor and workload analyzer 224 also reallocates tasks as needed based on the changing workload throughout the network-based database system 102, and can also reallocate tasks based on user (e.g., "external") query workloads that can also be processed by the execution platform 110. The configuration and metadata manager 222 and the monitor and workload analyzer 224 are coupled to a data storage device 226. The data storage device 226 represents any data storage device within the network-based database system 102. For example, data storage device 226 may represent a buffer in execution platform 110 , a storage device in cloud storage platform 104 , or any other storage device.

[0042] As described in the embodiments herein, the computing service manager 108 verifies all communications from an execution platform (e.g., execution platform 110) to verify that the content and context of the communication are consistent with the tasks known to be assigned to the execution platform. For example, an instance of an execution platform executing query A should not be allowed to request access to a data source D (e.g., data storage device 226) that is not related to query A. Similarly, a given execution node (e.g., Figure 3 Execution node 302-1 may need to communicate with another execution node (e.g., Figure 3 2) of the execution node, but should be prohibited from communicating with a third execution node (e.g., execution node 312-1), and any such illegal communication may be recorded (e.g., in a log or other location). In addition, the information stored by a given execution node is limited to data related to the current query, and any other data is not available, which is achieved by destruction or encryption (in the case where the key is not available).

[0043] Figure 3 is a diagram showing some embodiments of the present disclosure Figure 1 A block diagram of the components of the execution platform 110 is shown. Figure 3 As shown, the execution platform 110 includes multiple virtual warehouses, including virtual warehouse 1, virtual warehouse 2 and virtual warehouse N. Each virtual warehouse includes multiple execution nodes, and each execution node includes a data cache and a processor. The virtual warehouse can execute multiple tasks in parallel by using multiple execution nodes. As discussed herein, the execution platform 110 can add new virtual warehouses and discard existing virtual warehouses in real time based on the current processing needs of the system and users. This flexibility allows the execution platform 110 to quickly deploy large amounts of computing resources when needed without having to continue paying for those computing resources when they are no longer needed. All virtual warehouses can access data in any data storage device (e.g., any storage device in the cloud storage platform 104).

[0044] Although Figure 3 Each virtual warehouse shown in includes three execution nodes, but a particular virtual warehouse may include any number of execution nodes. In addition, the number of execution nodes in a virtual warehouse is dynamic, so that new execution nodes are created when there is additional demand, and existing execution nodes are deleted when they are no longer useful.

[0045] Each virtual warehouse can access Figure 1 Thus, the virtual warehouse does not have to be assigned to a specific data storage device 120-1 to 120-N, but can access data in any of the data storage devices 120-1 to 120-N within the cloud storage platform 104. Similarly, Figure 3 Each execution node shown in can access data from any one of the data storage devices 120-1 to 120-N. In some embodiments, a specific virtual warehouse or a specific execution node can be temporarily assigned to a specific data storage device, but the virtual warehouse or execution node can later access data in any other data storage device.

[0046] exist Figure 3 In the example of , virtual warehouse 1 includes three execution nodes 302-1, 302-2 and 302-N. Execution node 302-1 includes cache 304-1 and processor 306-1. Execution node 302-2 includes cache 304-2 and processor 306-2. Execution node 302-N includes cache 304-N and processor 306-N. Each execution node 302-1, 302-2 and 302-N is associated with processing one or more data storage and / or data retrieval tasks. For example, a virtual warehouse can handle data storage and data retrieval tasks associated with internal services (such as clustering services, instantiated view refresh services, file compression services, stored procedure services, or file upgrade services). In other embodiments, a specific virtual warehouse can handle data storage and data retrieval tasks associated with a specific data storage system or a specific category of data.

[0047] Similar to the virtual warehouse 1 discussed above, virtual warehouse 2 includes three execution nodes 312-1, 312-2 and 312-N. Execution node 312-1 includes a cache 314-1 and a processor 316-1. Execution node 312-2 includes a cache 314-2 and a processor 316-2. Execution node 312-N includes a cache 314-N and a processor 316-N. In addition, virtual warehouse 3 includes three execution nodes 322-1, 322-2 and 322-N. Execution node 322-1 includes a cache 324-1 and a processor 326-1. Execution node 322-2 includes a cache 324-2 and a processor 326-2. Execution node 322-N includes a cache 324-N and a processor 326-N.

[0048] In some embodiments, Figure 3 The illustrated execution nodes are stateless with respect to data cached by the execution nodes. For example, these execution nodes do not store or otherwise maintain state information about the execution nodes or data cached by a particular execution node. Thus, in the event of an execution node failure, the failed node can be transparently replaced with another node. Since there is no state information associated with the failed execution node, a new (replacement) execution node can easily replace the failed node without having to worry about recreating specific state.

[0049] Although Figure 3 The execution nodes shown each include a data cache and a processor, but alternative embodiments may include execution nodes that include any number of processors and any number of caches. In addition, the size of the cache may vary between different execution nodes. Figure 3 The cache shown in Figure 2 stores data from the local execution node. Figure 1 The data retrieved from one or more data storage devices in the cloud storage platform 104 is stored in the cache. Therefore, the cache reduces or eliminates the bottleneck problem that occurs in platforms that continuously retrieve data from remote storage systems. Instead of repeatedly accessing data from remote storage devices, the systems and methods described herein access data from caches in execution nodes, which is significantly faster and avoids the bottleneck problem discussed above. In some embodiments, the cache is implemented using a high-speed memory device that provides fast access to cached data. Each cache can store data from any storage device in the cloud storage platform 104.

[0050] In addition, cache resources and computing resources can vary between different execution nodes. For example, an execution node may include a large amount of computing resources and a minimum of cache resources, so that the execution node can be used for tasks that require a large amount of computing resources. Another execution node may include a large amount of cache resources and a minimum of computing resources, so that the execution node can be used for tasks that require a large amount of data to be cached. Yet another execution node may include cache resources that provide faster input-output operations, which is useful for tasks that require rapid scanning of large amounts of data. In some embodiments, based on the expected tasks that the execution node will perform, the cache resources and computing resources associated with a specific execution node are determined when the execution node is created.

[0051] In addition, the cache resources and computing resources associated with a particular execution node may change over time based on the changing tasks performed by the execution node. For example, if the tasks performed by the execution node become more processor intensive, more processing resources may be allocated to the execution node. Similarly, if the tasks performed by the execution node require a larger cache capacity, more cache resources may be allocated to the execution node.

[0052] Although virtual warehouses 1, 2, and N are associated with the same execution platform 110, multiple computing systems at multiple geographic locations may be used to implement the virtual warehouses. For example, virtual warehouse 1 may be implemented by a computing system at a first geographic location, while virtual warehouses 2 and N are implemented by another computing system at a second geographic location. In some embodiments, these different computing systems are cloud-based computing systems maintained by one or more different entities.

[0053] In addition, each virtual warehouse Figure 3 302-1 and 302-2 are shown as having multiple execution nodes. Multiple execution nodes associated with each virtual warehouse can be implemented using multiple computing systems located at multiple geographic locations. For example, an instance of virtual warehouse 1 implements execution nodes 302-1 and 302-2 on a computing platform at one geographic location, while implementing execution node 302-N at a different computing platform at another geographic location. The selection of a particular computing system to implement an execution node can depend on various factors, such as the resource level required for a particular execution node (e.g., processing resource requirements and cache requirements), the resources available at a particular computing system, the communication capabilities of a network within or between geographic locations, and which computing systems have implemented other execution nodes in the virtual warehouse.

[0054] The execution platform 110 is also fault tolerant. For example, if a virtual warehouse fails, the virtual warehouse will be quickly replaced by a different virtual warehouse located in a different geographical location.

[0055] A particular execution platform 110 may include any number of virtual warehouses. In addition, the number of virtual warehouses in a particular execution platform is dynamic, so that new virtual warehouses are created when additional processing and / or cache resources are needed. Similarly, existing virtual warehouses may be deleted when the resources associated with the virtual warehouses are no longer useful.

[0056] In some embodiments, virtual warehouses can operate on the same data in the cloud storage platform 104, but each virtual warehouse has its own execution node with independent processing and cache resources. This configuration allows requests on different virtual warehouses to be processed independently without interfering with each other. This independent processing, combined with the ability to dynamically add and remove virtual warehouses, supports adding new processing capabilities for new users without affecting performance.

[0057] Figure 4 A dynamically constrained data clean room system 230 is shown in accordance with some example embodiments. Figure 4 In the example below, the first database account 405 and the second database account 450 share data in the data clean room system 230, and either account can issue queries against the data. In the following example, the first database account 405 provides data to the second database account 450 (e.g., using the approved statement table 410, the row access policy engine (RAP) 415, the source data 420, and the shared source data 425), and it should be understood that the second database account 450 can similarly share data with the first database account 405 (e.g., using the approved statement table 455, the row access policy engine (RAP) 460, the source data 465, and the shared source data 470).

[0058] exist Figure 4In the example of , the data clean room system 230 implements a row access policy scheme (e.g., row access policy engine 415, row access policy engine 460) for a shared data set (e.g., source data 420, source data 465) of a first database account and a second database account. In some example embodiments, the row access policy engine 460 is implemented as a database object of the network-based database system 102, which restricts the use and / or sharing of the source data of the database account in the clean room. In some example embodiments, the database object in the network-based database system 102 is a data structure for storing and / or referencing data. In some example embodiments, the network-based database system 102 implements one or more of the following objects: a database table, a view, an index, a stored procedure of the database system, a user-defined function of the database system, or a sequence. In some example embodiments, when the network-based database system 102 creates a database object type, the object is locked, and because the network-based database system 102 restricts the object type using the source code of the database system, a new object type cannot be created. In some example embodiments, a database object instance is an instance of a database object type created by the database system 102 when an object is created (e.g., such as a new table, an index on the table, a view on the same table, or a new stored procedure object). The row access policy engine 460 provides row-level security to the data of the network-based database system 102 by using row access policies to determine which rows are returned in a query result. Examples of row access policies include: allowing a specific role to view the rows of a table (e.g., the user role of the end user who issued the query), or including a mapping table defined according to the policy to determine access to rows in a given query result. In some example embodiments, a row access policy is a schema-level object of the network-based database system 102 that determines whether a given row in a table or view can be viewed based on different types of database statements (including SELECT statements) or rows selected through UPDATE, DELETE, and MERGE statements.

[0059] In some example embodiments, a row access policy includes conditions and functions that transform data at query runtime when those conditions are met. Policy data is implemented to limit exposure of sensitive data. Policy data can further restrict object owners (e.g., roles with OWNERSHIP permissions on objects such as tables or views) that typically have full access to the underlying data. In some example embodiments, a single row access policy engine is set for different tables and views to be implemented simultaneously. In some example embodiments, a row access policy can be added to a table or view when the object is created or after the object is created.

[0060] In some example embodiments, a row access policy includes an expression that can specify a database object (e.g., a table or view) and uses conditional expression functions and context functions to determine which rows should be visible in a given context. The following is an example of a row access policy implemented at query runtime: (A) For the data specified in the query, the network-based database system 102 determines whether a row access policy is set on the database object. If the policy is added to the database object, all rows are protected by the policy. (B) The distributed database system then creates a dynamic security view of the database object (e.g., a secure database view). (C) The policy expression is evaluated. For example, the policy expression can specify a "current statement" expression that only proceeds if the "current statement" is in an approved statement table or the current role of the user issuing the query is a previously specified and allowed role. (D) Based on the evaluation of the policy, the restriction engine generates a query output, such as source data to be shared from a first database account to a second database account, where the query output only contains rows that evaluate to true (TRUE) based on the policy definition.

[0061] Continue to refer Figure 4 , the content of the approved statement table is agreed upon by the first database account 405 and the second database account 450 or generated by the first database account 405 and the second database account 450 in other ways. For example, the user who manages the first database account 405 and the second database account 450 agrees on a query language acceptable to both parties, and includes the query language in the approved statement table, and the agreed language is stored in the approved statement table 410 on the first database account 405, and is also stored in the approved statement table 455 in the second database account 450. As an illustrative example, Figure 5 As shown, source data 420 for a first database account 405 may include a first email dataset 500 for users of the first database account, and source data 465 for a second database account 450 may include a second email dataset 550 for users of the second database account. The two database accounts may seek to determine how many matches there are in the email addresses of users in their respective datasets, where the results returned are numbers (e.g., each has end users, and the two database accounts are interested in the number of users they share, but do not want to share data on the actual users). To do this, the two database accounts store a "SELECT COUNT" in the approved query request table. In this way, count queries that select and join source data can continue, but request and potentially return a "count" of all user data. The query cannot continue because it is not in the approved statement table (e.g., approved statement table 410 and approved statement table 455) of the corresponding data set account.

[0062] In addition, although Figure 4 Only two database accounts are shown in FIG, but the data clean room system 230 enables two or more database accounts to share data through the clean room architecture. In past approaches, the data clean room data was obfuscated (e.g., tokenized) and then shared in the data clean room, and the complexity of matching the obfuscated data may result in limiting the data clean room data to only two parties at a time. In contrast, in Figure 4 In the method of, according to some example embodiments, the third database account (not in Figure 4 ) can use the data clean room system 230 in the computing service manager 108 to provide a third-party shared data set 477, and can issue a database statement that connects data from the three data sets, such as SELECTCOUNT, which is for the connection data from the source data 420, the shared source data 470 from the second database account 450, and the third-party shared data set 477 from a third database account (for example, as opposed to a requester database account that shares data with the first provider database account, and the requester database account also uses the encryption function sequence provided by the first provider account and the second provider account to associate the data with another second provider database account).

[0063] Figure 6A-6C An example data cleanroom architecture for sharing data between multiple parties is shown in accordance with some example embodiments. In the example shown, party_1 database account 600 is Fig. 6A In the database account 605 of Fang_2, Figure 6B and the party_3 database account 610 is in Figure 6C , in which data is transferred (e.g., copied, shared) between different accounts, as indicated by the broken arrows labeled with reference to other figures; e.g., in Figure 6B , a "Party 2 Outbound Share" is shared from Party_2 database account 605 to Party_1 database account 600, where the share is marked as "Party 2 Share" and is Fig. 6A and Figure 6B The following data flows are connected by broken arrows. Figure 6A-6C For example, in operation 650, the party_1 database account 600 creates its APPROVED_STATEMENTS in its own database instance (e.g., in Fig. 6A ); Similarly, at operation 650, the party_2 database account 605 creates its APPROVED_STATEMENTS in its own database instance (e.g., Figure 6B), and further, the party_3 database account 610 creates its APPROVED_STATEMENTS in its own database instance (e.g., in Figure 6C ).

[0064] At operation 650, each party creates an APPROVED_STATEMENTS table that will store the query request SQL statements that have been verified and approved. In some example embodiments, one party creates the approved statement table, which is then stored by the other parties. In some example embodiments, each of the parties creates its own approved statement table, and a given query against the shared data must satisfy each in the approved statement table, otherwise the query cannot proceed (e.g., " " must be on each corresponding party's list of approved statements in order for the statement containing " ” queries operate on data shared between parties in the cleanroom).

[0065] Each party creates a row access strategy that will be applied to the source table shared with each other party for cleanroom request processing at operation 655. The row access strategy will check the current_statement() function against the value stored in the APPROVED_STATEMENTS table.

[0066] At operation 660, each party will generate their AVAILABLE_VALUES table, which acts as a data dictionary for other parties to understand which columns and values ​​they can use in query requests. In some example embodiments, the available values ​​include schemas, allowed columns, and metadata specifying prohibited row or cell values. In some example embodiments, the available values ​​data is not the actual data itself (e.g., source data), but rather specifies what data (e.g., which columns of source data) other parties (e.g., consumer accounts) can access for their corresponding shared data jobs (e.g., overlap analysis).

[0067] Return to reference Figure 6C At operation 665, each party agrees on one or more query templates that can be used for the query request. For example, if the media publisher and advertiser work together in a clean room, they can approve an "audience overlap" query template. The query template will store the connection information and other static logic while using placeholders for variables (select fields, filters, etc.).

[0068] As an additional example, one party is a provider account that specifies which statements are stored in the available statements table (e.g., thereby specifying how any consumer account that wants to access the provider data will access the provider's data). In addition, in some example embodiments, the provider account also provides one or more query templates for use by any party (e.g., a consumer account) seeking to access the provider's data based on the query templates. For example, a query template may include a blank or placeholder " ”, which can be replaced by a specific field via a consumer request (e.g., the specific field can be a column from the consumer data or a column from the provider data). Any changes to the query template (e.g., adding an asterisk " to select all records) will be rejected by data restrictions on the provider data (for example, the row access policy (RAP) acts as a firewall for the provider data).

[0069] Next, in operation 670 ( Fig. 6A ), one of the parties (e.g., in this example, party_1 database account 600) will generate a cleanroom query request by calling the GENERATE_QUERY_REQUEST stored procedure. This procedure inserts the new request into the QUERY_REQUESTS table. This table is shared with every other party along with the source data table with row access strategy enabled, the party's AVAILABLE_VALUES table, and the REQUEST_STATUS table.

[0070] At operation 675, each party has a stream object created for the other party's QUERY_REQUESTS table, thereby capturing any inserts to that table. If the stream object has data, the task object will run according to the set schedule and execute the VALIDATE_QUERY stored procedure.

[0071] At operation 680, the VALIDATE_QUERY process is configured to: (1) Ensure that the query request selects and filters columns with valid attributes by comparing them to the AVAILABLE_VALUES table. (2) Ensure that the query template accepts the submitted variables. (3) Ensure that thresholds or other query restrictions are applied. (4) Generate a create table as select (CTAS) statement and store it in the APPROVED_STATEMENTS table if validation succeeds. (5) Update the REQUEST_STATUS table with success or failure. If successful, the create table as select (CTAS) statement is also added to the record.

[0072] The GENERATE_QUERY_REQUEST process will also call the VALIDATE_QUERY process against the requesting party's account at operation 685. This is to ensure that each additional party generated query matches the requesting party as an additional layer of verification.

[0073] The REQUEST_STATUS table shared by each party is updated with the status from the VALIDATE_QUERY process at operation 690. The GENERATE_QUERY_REQUEST process will wait and poll each REQUEST_STATUS table until a status is returned.

[0074] Once each party has returned a status, the GENERATE_QUERY_REQUEST procedure will compare all CTAS statements to ensure they match (if the status is approved) at operation 699. If they all match, the procedure will execute the statements and generate a result table.

[0075] Fig. 7A and Figure 7B An example data architecture for implementing a defined access clean room using a native application according to some example embodiments is shown. In some example embodiments, the native application is configured so that the provider can create local state objects (e.g., tables) and local computation objects (e.g., stored procedures, external functions, tasks), and also share objects representing application logic in the consumer account. In some example embodiments, the native application is installed in the consumer account as a sharable database instance. For example, the provider can generate a native application including stored procedures and external functions that analyze and enrich the data in a given consumer account. The consumer can install the provider's native application as a database in the consumer's account and call the stored procedures that provide application functionality in the installed native application. In some example embodiments, the native application is configured to write only to the database in the consumer account. In addition, in some example embodiments, the provider's native application can be packaged with one or more other objects (such as tables, views, and stored procedures of the provider account), which are then generated in the consumer account when installed via the installer script. In some example embodiments, the native application installer script is configured to: (1) create a local object in a consumer account, and (2) control visibility of the object in the native application with different consumer accounts that may install the provider's native application.

[0076] Fig. 7A The provider database account 702 is shown, and Figure 7B A consumer database account 751 is shown, where Fig. 7A and Figure 7BConnections between are shown using circled capital letters (for example, A, B, C, and D). Fig. 7A At operation 705, the provider database account 702 generates a defined access cleanroom 704 (DCR). At operation 710, the provider database account 702 shares the installer cleanroom stored procedure 706 as a native database application with the consumer database account 751. Fig. 7A In operation 715, the provider database account 702 shares the source data 708 as a source data database view 711 in the clean room 712, and then the source data 708 is accessed by the consumer database account 751 as (in Figure 7B ) source data 714. Although source data 714 is accessible as a share by consumer database account 751, source data 714 may be empty (e.g., not yet populated) and controlled by a data firewall 716 (such as the row access policy of provider database account 702 as described above). Figure 7B In operation 720 , the customer database account 751 creates the clean room customer database 718 .

[0077] At operation 725, the consumer database account 751 creates a database storage 721 to store the source data 714 shared from the provider database account 702. At operation 730, the consumer database account 751 shares a request table 722 with the provider database account 702 as a consumer-defined cleanroom sharing request table 723 (in Fig. 7A In operation 735, the provider database account 702 creates a consumer storage database 724 to store the request table 723 received from the consumer database account 751 as a consumer share. In addition, the provider database account 702 creates a management object 737, which includes a stream object for tracking changes on the request table 723, and a task object in the management object 737 to execute the processing request storage process 743 when a new request is entered into the request table 723 (for example, requests from consumers and users, which are entered into the request table 722 and automatically shared as entries in the request table 723). Figure 7B In operation 745, the consumer database account 751 creates a database storage 721 to store the provider's shared source data 714 (in Figure 7B ), which starts the stored procedure installer script that generates the runtime instance of the native application 757. Figure 7BIn operation 750, according to some example embodiments, a data clean room native application 757 is executed and created using the native application installer process 706, creating a clean room schema and all objects within the clean room as specified in the native application installer process 706. In addition, the native application installer process 706 grants permissions to tables and request data storage processes. In addition, the native application installer process 706 creates an application internal schema 759 for request processing.

[0078] At operation 755 , consumer database account 751 generates a clean room request by calling request stored procedure 789 and passing in a query template name (e.g., the name of a template from query templates 756 , a template repository), selecting grouping by columns, filters, a privacy budget to be achieved, and any other parameters required or otherwise passed in for the selected query template.

[0079] At operation 760 , the consumer database account 751 implements a request stored process 789 that is configured to (1) generate a query based on a query template and parameters passed in, (2) sign the query request using an encryption key created by the data clean room native application 757 to authenticate to the provider database account 702 that the data clean room native application 757 issued the request, (3) apply a differential privacy noise parameter to the query results based on an epsilon value (also known as a privacy budget) passed in with the query, and (4) automatically share the query with the provider as an entry in request table 723 when the query is entered into request table 722 .

[0080] exist Fig. 7AAt operation 765, the provider database account 702 implements such a flow for capturing the inserted entry into the request table 723, and then triggering the management object task 737 to execute the process request stored procedure 743. At operation 770, the process request stored procedure 743 performs a query to verify the request. In some example embodiments, the verification performed by the process request stored procedure 743 includes (1) determining whether the encrypted request key matches the provider key, (2) confirming that the request originates from a corresponding pre-authorized consumer account (e.g., the consumer database account 751), (3) confirming that the query uses a valid template from the template 746 (e.g., from a plurality of valid and pre-configured templates authorized by the provider), (4) confirming whether the instant ID of the data clean room native application 757 matches the expected instance ID, and (5) confirming that the provider database account 702 is the expected or pre-configured account. At operation 775, if the request is valid, the provider database account 702 updates the status to "approved" in the request log 776, which configures the data firewall 716 (e.g., row access policy) to provide access to one or more rows from the source data 708; wherein the rows provided by the RAP are then shared with the consumer database account 751 as source data 714. Figure 7B In the example, once the data is shared into the source data 714 , the consumer database account 751 can execute queries within the data clean room native application 757 on the consumer database account 751 (eg, via an execution node of the consumer database account 751 ).

[0081] Figure 8 2 is a block diagram illustrating components of constraint system 240 according to some example embodiments. As previously explained, databases are used by various entities (e.g., businesses, people, organizations, etc.) to store data. For example, a retailer may store data describing a purchase (e.g., product, date, price) and data describing the purchaser (e.g., name, address, email address). Similarly, an advertiser may store data describing the performance of its advertising campaigns, such as the advertisements served to the user, the date the advertisements were served, information about the user (e.g., name, address, email address), etc.

[0082] In some cases, entities may wish to share their data with each other. For example, retailers and advertisers may wish to share their data to determine the effectiveness of an advertising campaign, such as by determining whether a user who was provided with an advertisement for a product ultimately purchased the product. In these types of cases, the entities may wish to maintain the confidentiality of some or all of the data they collect and store in their respective databases. For example, retailers and / or advertisers may wish to maintain the confidentiality of personally identifiable information (PII) such as user names, addresses, email addresses, credit card numbers, etc. As another example, an entity sharing data may wish to maintain the confidentiality of its proprietary data such as its customer lists.

[0083] Current solutions address this problem through the use of tokenization, where data values ​​stored in corresponding databases are anonymized into tokens in a consistent and deterministic manner before being shared. For example, each entity may use a hashing algorithm to generate a hash value (e.g., a token) that represents each data value stored in the database. Entities share these tokens with each other, which can then be used to compare data and find matching values ​​while maintaining the confidentiality of the underlying data.

[0084] However, tokenization has several disadvantages. One disadvantage is that tokenization does not allow for fuzzy or approximate matches. In some cases, data may not be entered into various databases in a consistent format or correctly. Because tokenization utilizes a deterministic algorithm to generate tokens, only exact matches can be identified and these variations in format and data entries (e.g., fuzzy matches) are not taken into account. Another disadvantage of tokenization is the complexity and computing resources required to generate the tokens. For example, a hashing algorithm is used to generate hash values ​​for a large number of data values ​​stored in a corresponding database. Another disadvantage of tokenization is limited functionality because tokens can only be used to find exact matches between two data sets. Other functions, such as determining the number of data entries that do not match but include a specified string or character set, cannot be performed.

[0085] To mitigate these issues, when the network-based database system 102 receives a query, the constraint system 240 enforces a projection constraint on the data values ​​stored in a specified column of the shared data set. The projection constraint identifies that the data in the column may be restricted from being projected (e.g., rendered, read, output) into the output for the received query, while allowing the specified operations to be performed on the data and providing the corresponding output. For example, the projection constraint may indicate the context of the query that triggers the constraint, such as based on the user / role, account, and / or share associated with the query.

[0086] Entities that share data can define projection constraints that will be attached to various columns of a shared data set. For example, an entity can define one or more columns to which a projection constraint should be attached, as well as conditions for triggering the constraint. When the network-based database system 102 receives a query directed to a shared data set, the constraint system 240 accesses the data required to process the query from the shared database, and determines whether to attach a projection constraint to any column of the shared data set from which data is accessed. If a projection constraint is attached to one of the columns, the constraint system 240 determines whether the projection constraint should be implemented based on the context of the query, and generates output accordingly. For example, if a projection constraint should be implemented, the constraint system 240 can generate an output that does not include data values ​​stored in the column, but can provide output determined based on the constrained data, such as the number of matches, the number of fuzzy matches, the number of matches including a specified string, unconstrained data associated with the constrained data, etc.

[0087] Enforcing projection constraints on queries received at the network-based database system 102 allows data to be anonymously shared and used by entities to perform various operations without the need to tokenize the data. Thus, the difficulty and computational resources required for tokenization are eliminated, and additional functions and operations (e.g., fuzzy matching) can be performed while maintaining the confidentiality of the specified data values.

[0088] When processing queries against a shared data set, constraint system 240 may be implemented within network-based database system 102. For example, in some embodiments, constraint system 240 may be implemented within a clean room provided by data clean room system 230.

[0089] As shown, constraint system 240 includes a projection constraint generation component 802 , a query reception component 804 , a data access component 806 , a column identification component 808 , a projection constraint determination component 810 , a query context determination component 812 , an implementation determination component 814 , and a projection constraint implementation component 816 .

[0090] The projection constraint generation component 802 enables entities to establish projection constraints (e.g., projection constraint policies) on shared data sets. For example, the projection constraint generation component 802 can provide a user interface or other communication means that allows entities to define projection constraints associated with their data, which is maintained and managed by the network-based database system 102. To define the projection constraints, the projection constraint generation component 802 enables users to provide data that defines the shared data sets and columns to which the projection constraints should be attached. For example, a user can submit data that defines a specific column and / or a set of columns in a shared data set to which a projection constraint should be attached.

[0091] In addition, the projection constraint generation component 802 enables the user to define conditions for triggering the projection constraint. This can include defining a specific context and / or multiple contexts that trigger the implementation of the projection constraint. For example, the projection constraint generation component 802 can enable the user to define roles for users, accounts, and / or shares that will trigger the projection constraint and / or be able to project the constrained data columns.

[0092] After receiving the data defining the projection constraints, the projection constraint generation component 802 generates a file attached to the identified column. In some embodiments, the file may include a Boolean function based on the conditions provided for the projection constraints. For example, if the projection constraint should be implemented with respect to the query, the Boolean function may provide an output of true, and if the projection constraint should not be implemented with respect to the query, the Boolean function may provide an output of false. Attaching the file to the column will establish the projection constraints on the data column for subsequent queries.

[0093] The query receiving component 804 receives a query for a shared data set. The query may include data defining the data to be accessed and one or more operations to be performed on the data. The operation may include any type of operation used in connection with the data maintained by the network-based database system 102, such as a join operation, a read operation, etc. The query receiving component 804 may provide data associated with the query to other components of the constraint system 240, such as the data access component 806 and the query context determination component 812.

[0094] The data access component 806 accesses the data set based on the query received by the query receiving component 804. For example, the data access component can access data from columns and / or sub-columns of the shared data set that are identified by the query and / or required to generate output based on the received query. The data access component 806 can provide the accessed data to other components of the constraint system 240, such as the projection constraint enforcement component 816.

[0095] The column identification component 808 determines, in response to the query, the columns associated with the data accessed by the data access component 806. This includes the columns and / or sub-columns from which the data is accessed. The column identification component 808 can provide data identifying the columns to other components of the constraint system 240, such as the projection constraint determination component 810.

[0096] The projection constraint determination component 810 determines whether a projection constraint (e.g., a projection constraint policy) is attached to any of the columns identified by the column identification component 808. For example, the projection constraint determination component 810 determines whether a file defining a projection constraint is attached to any of the columns and / or sub-columns identified by the column identification component 810. The projection constraint determination component 810 can provide data indicating whether a projection constraint is attached to any of the columns and / or sub-columns identified by the column identification component 810 and / or provide the data indicating whether a projection constraint is attached to any of the columns and / or the file defining the projection constraint to other components of the constraint system 240, such as the implementation determination component 814.

[0097] The query context determination component 812 determines a context associated with the received query. For example, the query context determination component 812 can use data associated with the received query to determine the context, such as by determining a role of the user submitting the query, an account of the network-based database system 102 associated with the submitted query, a data share associated with the query, etc. The query context determination component 812 can provide data defining the determined query context to other components of the constraint system 240, such as the implementation determination component 814.

[0098] The implementation determination component 814 determines whether the projection constraint should be implemented with respect to the received query. For example, the implementation determination component 814 uses the data received from the projection constraint determination component 810 and / or the file defining the projection constraint and the context of the query received from the query context determination component 812 to determine whether the projection constraint should be implemented, the received data indicating whether the projection constraint is attached to any column.

[0099] If the query constraint is not attached to any column, the implementation determination component 814 determines that the projection constraint should not be implemented relative to the query. Alternatively, if the projection constraint is attached to one of the columns, the implementation determination component 814 uses the context of the query to determine whether the projection constraint should be implemented. For example, the implementation determination component 814 can use the context of the query to determine whether the conditions defined in the file attached to the column meet the triggering projection constraint. In some embodiments, the implementation determination component 814 can use the context of the query as the input to the Boolean function defined by the projection constraint to determine whether the projection constraint is triggered. For example, if the Boolean function returns a true value, the implementation determination component 814 determines that the projection constraint should be implemented. Alternatively, if the Boolean function returns a false value, the implementation determination component 814 determines that the projection constraint should not be implemented.

[0100] The enforcement determination component 814 can provide data indicating whether the projected constraint should be enforced to other components of the constraint system 240 , such as the projected constraint enforcement component 816 .

[0101] The projection constraint enforcement component 816 enforces projection constraints with respect to a query. For example, the projection constraint enforcement component 816 may prohibit the output of a query from including data values ​​in any constrained column of a shared data set. This may include completely rejecting a query based on operations included in the query, such as rejecting the query if the query request simply outputs the value of a constrained column. However, compared to current solutions (e.g., tokenization), the projection constraint enforcement component 816 may allow many other operations to be performed while maintaining the confidentiality of the data values ​​in the restricted columns, thereby enabling additional functionality. For example, the projection constraint enforcement component 816 allows an operation to provide an output indicating the number of data values ​​within a column that match (including fuzzy matches) a specified key value or multiple specified key values ​​in another column. As an example, two tables may be joined on a column subject to a projection constraint using case-insensitive or approximate matching. Tokenization solutions are generally not suitable for these purposes.

[0102] The projection constraint enforcement component 816 may also allow users to filter and perform other operations on data values ​​stored in the projection-constrained columns. For example, if an email address column is projection-constrained, an analyst end user is prevented from enumerating all email addresses, but may be allowed to perform operations on the predicate " ” is true.

[0103] The projection constraint enforcement component 816 can provide the output of the query to the requesting user's client device.

[0104] As explained previously, as referenced above Figure 4-7B As discussed, in some example embodiments, constraint system 240 is integrated into a database clean room. A database clean room enables two or more end users of a network-based database system 102 to share and collaborate on their sensitive data without directly revealing the data to other participants. As an example, according to some example embodiments, constraint system 240 may implement a projection constraint in a clean room to perform a database end user intersection operation (e.g., Company A and Company B want to know which database end users they have in common without disclosing their complete database end user lists to each other).

[0105] As an example, according to some example embodiments, constraint system 240 may implement projection constraints in a clean room to perform enrichment operations. For example, a company may implement constraint system 240 to provide enrichment analysis. For example, (1) given a credit card number, provide demographic information about the card owner, or (2) IP geolocation: given an IP address, provide the geolocation most likely associated with that IP. The enrichment provider wants to enable consumers to look up key attributes of values ​​of interest (e.g., IP or credit card number data) (possibly using fuzzy matching) without allowing consumers to download all available key information.

[0106] In some example embodiments, when a query is submitted by a user and compiled, projection constraints are enforced by the constraint system 240. The SQL compiler of the constraint system 240 analyzes each individual column accessed based on the query to determine the lineage of the column, for example, where the data comes from. In some example embodiments, the constraint-based approach of the constraint system 240 is integrated in a SQL-based system as discussed herein, however, it should be understood that the constraint-based approach of the constraint system 240 can be integrated with any different query language or query system other than SQL in a similar manner. In this way, the user submits a query, and the constraint system 240 determines the meaning of the query, considers any applicable projection constraints, and ensures that the query complies with the applicable constraints. In some example embodiments, if the data comes from a column subject to a projection constraint, the constraint system 240 checks whether the constraint should be enforced based on the context (such as the role of the user executing the query). If the constraint is intended to be enforced, the constraint system 240 prevents the column or any value derived directly from the column from being included in the query output. In some example embodiments, the use of constrained columns is allowed based on specific preconfigured conditions (eg, in a WHERE clause for a filter condition, a GROUP BY clause for an aggregate query, etc.) and other contexts.

[0107] In some example embodiments, constraint system 240 implements constraints using a local policy framework (e.g., dynamic data masking (column masking) and row access policies) of network-based database system 102. In some example embodiments, constraint system 240 attaches a given projection constraint policy to one or more specific columns, similar to the masking policies of network-based database system 102. In these example embodiments, the projection constraint policy body is evaluated to determine whether and how to restrict access to the column when a given query is received from a consumer end user.

[0108] Figure 9A-9D Various data sharing scenarios are shown in which projection constraints may be implemented according to some example embodiments. Fig.9AA data sharing scenario 910 is shown in which a provider 902 shares its data with one or more consumers 904. In this type of scenario, shared data 906 (e.g., a shared data set) is associated with and managed by a single entity (e.g., provider 902) and is shared with one or more other entities (e.g., consumer 904). Therefore, shared data 906 is not a combination of data provided by multiple entities. In this type of scenario, the constraint system 240 can be implemented to implement projection constraints on queries submitted by one or more consumers 904. The provider 902 can implement projection constraints by specifying which data columns the consumer 904 cannot project to protect any sensitive data. For example, the provider 902 can establish a projection constraint to prohibit each of the consumers 904 from projecting data in a protected column, or set a projection constraint to change whether data can be projected based on the context of the query (such as which consumer 904 submitted the query).

[0109] Fig. 9B A data sharing scenario 920 is shown in which a provider 902 shares data with a consumer 904, and the shared data 906 is combined with the data of the consumer 904. In this type of scenario, the shared data 906 is associated with and managed by a single entity (e.g., the provider 902), and is shared with one or more other entities (e.g., the consumers 904), which combine the shared data 906 with their own data. In this type of scenario, the constraint system 240 can be implemented to enforce projection constraints on queries submitted by one or more consumers 904. The provider 902 can implement projection constraints by specifying which data columns the consumer 904 cannot project to protect any sensitive data, while allowing the consumer 904 to perform operations on the shared data 906 based on the consumer's data. For example, the consumer 904 can perform an operation to determine and output the number of matches between the consumer's data and the data in the constrained columns of the shared data 906, but the consumer may be prohibited from projecting the data values ​​of the constrained columns. As in Fig.9A In the example shown, provider 902 may establish projection constraints to prohibit each of consumers 904 from projecting data in a protected column, or set projection constraints to change whether data can be projected based on the context of the query, such as which consumer 904 submitted the query.

[0110] Fig. 9CA data sharing scenario 930 is shown in which data shared by multiple providers 902 is combined and shared with consumers 904. In this type of scenario, shared data 906 is a combination of data associated with and managed by multiple entities (e.g., providers 902), and shared data 906 is shared with one or more other entities (e.g., consumers 904). In this type of scenario, the constraint system 240 can be implemented to enforce projection constraints on queries submitted by one or more consumers 904. Each of the providers 902 can implement projection constraints by specifying which data columns the consumer 904 cannot project to protect any sensitive data shared by the corresponding provider 902. In this type of embodiment, queries submitted by consumers 904 will be evaluated based on the query constraints provided by each provider 902.

[0111] Shared data 906 can be accessed by consumers 904 without being combined with the consumer's data (e.g. Fig.9A ), or consumer 904 may combine shared data 906 with the consumer's own data (as shown in Fig. 9B Each provider 902 can establish a projection constraint to prohibit each of the consumers 904 from projecting data in a protected column, or set a projection constraint to change whether data can be projected based on the context of the query (such as which consumer 904 submitted the query).

[0112] Fig.9D A data sharing scenario 940 is shown in which a provider 902 shares data with an internal user 908. In this type of scenario, the shared data 906 is data associated with and managed by a single entity (e.g., the provider 902), and the shared data 906 is shared with one or more other users (e.g., the internal user 908) associated with the entity. In this type of scenario, the constraint system 240 can be implemented to enforce projection constraints on queries submitted by one or more internal users. The provider 902 can implement projection constraints by specifying which columns of data cannot be projected by the internal user 908 to protect any sensitive data shared by the provider 902. For example, the provider 902 can establish a projection constraint to prohibit each internal user 908 from projecting data in a protected column, or set a projection constraint to change whether data can be projected based on the context of the query (such as the role of the internal user 908 who submitted the query).

[0113] Fig.10 A flow chart of a method 1000 for implementing projection constraint-based access to a data set on a distributed database is shown, according to some example embodiments.

[0114] At operation 1002, the query receiving component 804 receives a query directed to a shared data set. The query may include data defining the data to be accessed and one or more operations to be performed on the data. The operations may include any type of operation used in connection with the data maintained by the network-based database system 102, such as a join operation, a read operation, etc. The query receiving component 804 may provide the data associated with the query to other components of the constraint system 240, such as the data access component 806 and the query context determination component 812.

[0115] At operation 1004, the data access component 806 accesses a data set from the shared data set to perform the operation of the query. For example, the data access component can access data from columns and / or sub-columns of the shared data set, the columns and / or sub-columns being identified by the query and / or required to generate output based on the received query. The data access component 806 can provide the accessed data to other components of the constraint system 240, such as the projection constraint enforcement component 816.

[0116] At operation 1006, the column identification component 808 determines the columns of the shared data set from which the data set is accessed. This includes the columns and / or sub-columns from which the data is accessed. The column identification component 808 can provide data identifying the columns to other components of the constraint system 240, such as the projection constraint determination component 810.

[0117] At operation 1008, the projection constraint determination component 810 determines whether a projection constraint policy is attached to any of the identified columns. For example, the projection constraint determination component 810 determines whether a file defining a projection constraint is attached to any of the columns and / or sub-columns identified by the column identification component 810. The projection constraint determination component 810 can provide data indicating whether a projection constraint is attached to any column and / or a file defining a projection constraint to other components of the constraint system 240, such as the implementation determination component 814.

[0118] At operation 1010, the query context determination component 812 determines the context of the query. For example, the query context determination component 812 can use data associated with the received query to determine the context, such as by determining a role of a user submitting the query, an account of the network-based database system 102 associated with the submitted query, a data share associated with the query, etc. The query context determination component 812 can provide data defining the determined query context to other components of the constraint system 240, such as the implementation determination component 814.

[0119] At operation 1012, the implementation determination component 814 determines whether the projection constraint should be implemented with respect to the received query. For example, the implementation determination component 814 uses data received from the projection constraint determination component 810 and / or a file defining the projection constraint and the context of the query received from the query context determination component 812 to determine whether the projection constraint should be implemented, the received data indicating whether the projection constraint is attached to any column.

[0120] If the query constraint is not attached to any column, the implementation determination component 814 determines that the projection constraint should not be implemented relative to the query. Alternatively, if the projection constraint is attached to one of the columns, the implementation determination component 814 uses the context of the query to determine whether the projection constraint should be implemented. For example, the implementation determination component 814 can use the context of the query to determine whether the conditions defined in the file attached to the column meet the triggering projection constraint. In some embodiments, the implementation determination component 814 can use the context of the query as the input to the Boolean function defined by the projection constraint to determine whether the projection constraint is triggered. For example, if the Boolean function returns a true value, the implementation determination component 814 determines that the projection constraint should be implemented. Alternatively, if the Boolean function returns a false value, the implementation determination component 814 determines that the projection constraint should not be implemented.

[0121] The enforcement determination component 814 can provide data indicating whether the projected constraint should be enforced to other components of the constraint system 240 , such as the projected constraint enforcement component 816 .

[0122] At operation 1014, the projection constraint enforcement component 816 enforces the projection constraints with respect to the query. For example, the projection constraint enforcement component 816 may prohibit the output of the query from including data values ​​from any constrained columns of the shared data set. This may include completely rejecting the query based on the operations included in the query, such as rejecting the query if the query request simply outputs the values ​​of the constrained columns. However, compared to current solutions (e.g., tokenization), the projection constraint enforcement component 816 may allow many other operations to be performed while maintaining the confidentiality of the data values ​​in the restricted columns, thereby achieving additional functionality. For example, the projection constraint enforcement component 816 allows an operation to provide an output indicating the number of data values ​​within a column that match (including fuzzy matches) a specified key value or multiple specified key values ​​in another column. As an example, two tables may be joined on a column subject to projection constraints using case-insensitive or approximate matching. Tokenization solutions are generally not suitable for these purposes.

[0123] The projection constraint enforcement component 816 may also allow users to filter and perform other operations on data values ​​stored in the projection-constrained columns. For example, if an email address column is projection-constrained, an analyst end user is prevented from enumerating all email addresses, but may be allowed to perform operations on the predicate " ” is true.

[0124] The projection constraint enforcement component 816 can provide the output of the query to the requesting user's client device.

[0125] Implementations of the described subject matter may include one or more of the features, alone or in combination, as set out below by way of example.

[0126] Example 1 is a method, comprising: receiving a first query pointing to a shared data set, the first query identifying a first operation; accessing the first data set from the shared data set to perform the first operation, the first data set including data accessed from a first column of the shared data set; determining, by at least one hardware processor, that a projection constraint policy is attached to the first column, the projection constraint policy restricting output of data values ​​stored in the first column; determining, based on the context of the first query, that the projection constraint policy should be implemented with respect to the first query; and generating an output for the first query based on the first data set and the first operation, the output for the first query not including the data values ​​stored in the first column based on determining that the projection constraint policy should be implemented with respect to the first query.

[0127] In Example 2, the subject matter of Example 1 includes, wherein the context of the first query is based on at least one of: a role of a user submitting the first query, an account associated with the first query, and a data share associated with the first query.

[0128] In Example 3, the subject matter of any of Examples 1-2 also includes: receiving a second query pointing to a shared data set, the second query identifying a second operation; accessing a second data set from the shared data set to perform the second operation, the second data set including data accessed from a first column of the shared data set; determining that a projection constraint policy is attached to the first column; based on the context of the second query, determining that the projection constraint policy should not be implemented with respect to the second query; and generating an output for the second query based on the second data set and the second operation, based on determining that the projection constraint policy should not be implemented with respect to the second query, the output for the second query including the data value stored in the first column.

[0129] In Example 4, the subject matter of any of Examples 1-3 includes, wherein the shared data set includes a first data set associated with a first entity and a second data set associated with a second entity, the first entity being associated with a first account of a database management system that receives a first query that has been received from a computing system associated with the second entity.

[0130] In Example 5, the subject matter of any of Examples 1-4 includes, wherein the second entity is associated with a second account of the database management system, the first account being different than the second account.

[0131] In Example 6, the subject matter of any of Examples 1-5 further includes receiving data defining a first projection constraint strategy from a computing system associated with the first entity.

[0132] In Example 7, the subject matter of any of Examples 1-6 further includes receiving data defining a second projection constraint policy from a computing system associated with the second entity.

[0133] In Example 8, the subject matter of any of Examples 1-6 also includes: receiving a second query pointing to a shared data set, the second query identifying a second operation; accessing a second data set from the shared data set to perform a second operation, the second data set including data accessed from a second column of the shared data set, the second column being different from the first column; determining that a projection constraint policy is attached to the second column, the projection constraint policy associated with the second column restricting output of data values ​​stored in the second column; determining, based on the context of the second query, whether the projection constraint policy associated with the second column should be implemented relative to the second query; and generating an output for the second query based on the second data set and the second operation, and the determination of whether the projection constraint policy associated with the second column should be implemented relative to the second query.

[0134] In Example 9, the subject matter of any of Examples 1-8 includes, wherein the first operation comprises a concatenation operation.

[0135] In Example 10, the subject matter of any of Examples 1-9 includes, wherein the first data output indicates a number of matching values ​​in the first column and the second column of the shared data set.

[0136] Example 11 is a system comprising: one or more computer processors; and one or more computer-readable media storing instructions, which, when executed by the one or more computer processors, cause the system to perform operations including: receiving a first query directed to a shared data set, the first query identifying a first operation; accessing a first data set from the shared data set to perform the first operation, the first data set including data accessed from a first column of the shared data set; determining that a projection constraint policy is attached to the first column, the projection constraint policy restricting output of data values ​​stored in the first column; based on the context of the first query, determining that the projection constraint policy should be implemented relative to the first query; and generating an output for the first query based on the first data set and the first operation, the output for the first query not including the data values ​​stored in the first column based on determining that the projection constraint policy should be implemented relative to the first query.

[0137] In Example 12, the subject matter of Example 11 includes, wherein the context of the first query is based on at least one of: a role of a user submitting the first query, an account associated with the first query, and a data share associated with the first query.

[0138] In Example 13, the subject matter of any of Examples 11-12 includes that the operation also includes: receiving a second query pointing to the shared data set, the second query identifying a second operation; accessing a second data set from the shared data set to perform the second operation, the second data set including data accessed from a first column of the shared data set; determining that a projection constraint policy is attached to the first column; based on the context of the second query, determining that the projection constraint policy should not be implemented with respect to the second query; and generating an output for the second query based on the second data set and the second operation, based on determining that the projection constraint policy should not be implemented with respect to the second query, the output for the second query including the data value stored in the first column.

[0139] In Example 14, the subject matter of any of Examples 11-13 includes, wherein the shared data set includes a first data set associated with a first entity and a second data set associated with a second entity, the first entity being associated with a first account of a database management system that receives a first query that has been received from a computing system associated with the second entity.

[0140] In Example 15, the subject matter of any of Examples 11-14 includes, wherein the second entity is associated with a second account of the database management system, the first account being different than the second account.

[0141] In Example 16, the subject matter of any of Examples 11-15 includes that the operations further include: receiving data defining a first projection constraint strategy from a computing system associated with the first entity.

[0142] In Example 17, the subject matter of any of Examples 11-16 includes that the operations further include: receiving data defining a second projection constraint strategy from a computing system associated with the second entity.

[0143] In Example 18, the subject matter of any one of Examples 11-18 includes that the operation also includes: receiving a second query pointing to a shared data set, the second query identifying a second operation; accessing a second data set from the shared data set to perform the second operation, the second data set including data accessed from a second column of the shared data set, the second column being different from the first column; determining that a projection constraint policy is attached to the second column, the projection constraint policy associated with the second column restricting output of data values ​​stored in the second column; determining, based on the context of the second query, whether the projection constraint policy associated with the second column should be implemented relative to the second query; and generating an output for the second query based on the second data set and the second operation, and the determination of whether the projection constraint policy associated with the second column should be implemented relative to the second query.

[0144] In Example 19, the subject matter of any of Examples 11-18 includes, wherein the first operation comprises a join operation.

[0145] In Example 20, the subject matter of any of Examples 11-19 includes, wherein the first data output indicates a number of matching values ​​in the first column and the second column of the shared data set.

[0146] Example 21 is a non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors of one or more computing devices, cause the one or more computing devices to perform operations, the operations comprising: receiving a first query directed to a shared data set, the first query identifying a first operation; accessing a first data set from the shared data set to perform the first operation, the first data set including data accessed from a first column of the shared data set; determining that a projection constraint policy is attached to the first column, the projection constraint policy restricting output of data values ​​stored in the first column; determining, based on the context of the first query, that the projection constraint policy should be implemented relative to the first query; and generating an output for the first query based on the first data set and the first operation, the output for the first query not including the data values ​​stored in the first column based on determining that the projection constraint policy should be implemented relative to the first query.

[0147] In Example 22, the subject matter of Example 21 includes, wherein the context of the first query is based on at least one of: a role of a user submitting the first query, an account associated with the first query, and a data share associated with the first query.

[0148] In Example 23, the subject matter of any of Examples 21-22 includes that the operation also includes: receiving a second query pointing to the shared data set, the second query identifying a second operation; accessing a second data set from the shared data set to perform the second operation, the second data set including data accessed from a first column of the shared data set; determining that a projection constraint policy is attached to the first column; based on the context of the second query, determining that the projection constraint policy should not be implemented with respect to the second query; and generating an output for the second query based on the second data set and the second operation, based on determining that the projection constraint policy should not be implemented with respect to the second query, the output for the second query including the data value stored in the first column.

[0149] In Example 24, the subject matter of any of Examples 21-23 includes, wherein the shared data set includes a first data set associated with a first entity and a second data set associated with a second entity, the first entity being associated with a first account of a database management system that receives a first query that has been received from a computing system associated with the second entity.

[0150] In Example 25, the subject matter of any of Examples 21-24 includes, wherein the second entity is associated with a second account of the database management system, the first account being different than the second account.

[0151] In Example 26, the subject matter of any of Examples 21-25 includes that the operations further include: receiving data defining a first projection constraint strategy from a computing system associated with the first entity.

[0152] In Example 27, the subject matter of any of Examples 21-26 includes that the operations further include: receiving data defining a second projection constraint strategy from a computing system associated with the second entity.

[0153] In Example 28, the subject matter of any one of Examples 21-27 includes that the operation also includes: receiving a second query pointing to a shared data set, the second query identifying a second operation; accessing a second data set from the shared data set to perform the second operation, the second data set including data accessed from a second column of the shared data set, the second column being different from the first column; determining that a projection constraint policy is attached to the second column, the projection constraint policy associated with the second column restricting output of data values ​​stored in the second column; determining, based on the context of the second query, whether the projection constraint policy associated with the second column should be implemented relative to the second query; and generating an output for the second query based on the second data set and the second operation, and the determination of whether the projection constraint policy associated with the second column should be implemented relative to the second query.

[0154] In Example 29, the subject matter of any of Examples 21-28 includes, wherein the first operation comprises a join operation.

[0155] In Example 30, the subject matter of any of Examples 21-29 includes, wherein the first data output indicates a number of matching values ​​in the first column and the second column of the shared data set.

[0156] Fig.11 A diagrammatic representation of a machine 1100 in the form of a computer system is shown, in accordance with an example embodiment, within which a set of instructions may be executed to cause the machine 1100 to perform any one or more of the methods discussed herein. Specifically, Fig.11 A diagrammatic representation of a machine 1100 is shown in the form of an example computer system in which instructions 1116 (e.g., software, programs, applications, applet, app, or other executable code) for causing the machine 1100 to perform any one or more of the methods discussed herein may be executed. For example, the instructions 1116 may cause the machine 1100 to be executed by one or more processors described herein. Fig.10 As another example, the instructions 1116 may cause the machine 1100 to implement a portion of the data flow described herein. In this manner, the instructions 1116 convert a general non-programmed machine into a specific machine 1100 (e.g., Figure 1 Client device 114, Figure 1The computing service manager 108, Figure 1 110 ), the particular machine 1100 being specifically configured to perform any of the functions described and illustrated in the manner described herein.

[0157] In alternative embodiments, the machine 1100 operates as a standalone device or can be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1100 can operate as a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1100 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a smart phone, a mobile device, a network router, a network switch, a network bridge, or any machine capable of executing instructions 1116 sequentially or otherwise, the instructions 1116 specifying actions to be taken by the machine 1100. In addition, although only a single machine 1100 is shown, the term "machine" should also be understood to include a collection of machines 1100 that execute instructions 1116 individually or jointly to perform any one or more of the methods discussed herein.

[0158] The machine 1100 includes a processor 1110, a memory 1130, and an input / output (I / O) component 1150, which are configured to communicate with each other, for example, via a bus 1102. In an example embodiment, the processor 1110 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 1112 and a processor 1114 that may execute instructions 1116. The term "processor" is intended to include a multi-core processor 1110, which may include two or more independent processors (sometimes referred to as "cores") that may execute instructions 1116 simultaneously. Although Fig.11 Multiple processors 1110 are shown, but the machine 1100 may include a single processor with a single core, a single processor with multiple cores (eg, a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0159] The memory 1130 may include a main memory 1132, a static memory 1134, and a storage unit 1131, all of which may be accessed by the processor 1110, such as via the bus 1102. The main memory 1132, the static memory 1134, and the storage unit 1131 include a machine storage medium 1138 that may store instructions 1116 embodying any one or more of the methodologies or functions described herein. The instructions 1116 may also reside, in whole or in part, within the main memory 1132, within the static memory 1134, within the storage unit 1131, within at least one of the processors 1110 (e.g., within a cache memory of the processor), or any suitable combination thereof during execution by the machine 1100.

[0160] I / O components 1150 include components for receiving input, providing output, generating output, transmitting information, exchanging information, capturing measurements, etc. The specific I / O components 1150 included in a particular machine 1100 will depend on the type of machine. For example, a portable machine such as a mobile phone will likely include a touch input device or other such input mechanism, while a headless server machine will be less likely to include such a touch input device. It should be understood that I / O components 1150 may include Fig.11 Many other components not shown in the figure. The I / O components 1150 are grouped according to function only to simplify the following discussion, and the grouping is by no means limiting. In various example embodiments, the I / O components 1150 may include output components 1152 and input components 1154. The output components 1152 may include visual components (e.g., displays such as plasma display panels (PDPs), light emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tubes (CRTs)), acoustic components (e.g., speakers), other signal generators, etc. The input components 1154 may include alphanumeric input components (e.g., keyboards, touch screens configured to receive alphanumeric inputs, optical keyboards, or other alphanumeric input components), pointing-based input components (e.g., mice, touch pads, trackballs, joysticks, motion sensors, or another pointing instrument), tactile input components (e.g., physical buttons, touch screens or other tactile input components that provide the location and / or force of touch or touch gestures), audio input components (e.g., microphones), etc.

[0161] A variety of technologies may be used to implement communication. The I / O component 1150 may include a communication component 1164 that is operable to couple the machine 1100 to a network 1181 via a coupler 1183, or to couple the machine 1100 to a device 1180 via a coupling device 1182. For example, the communication component 1164 may include a network interface component or another suitable device that interfaces with the network 1181. In another example, the communication component 1164 may include a wired communication component, a wireless communication component, a cellular communication component, and other communication components that provide communication via other modalities. The device 1180 may be another machine or any of a variety of peripheral devices (e.g., a peripheral device coupled via a universal serial bus (USB)). For example, as described above, the machine 1100 may correspond to any one of the client device 114, the computing service manager 108, and the execution platform 110, and the machine 1100 may include any of these systems and devices.

[0162] Various memories (e.g., 1130, 1132, 1134 and / or the memory and / or storage unit 1131 of the processor 1110) may store one or more sets of instructions 1116 and data structures (e.g., software) that embody or are utilized by any one or more of the methods or functions described herein. When executed by the processor 1110, these instructions 1116 cause various operations to implement the disclosed embodiments.

[0163] As used herein, the terms "machine storage medium", "device storage medium" and "computer storage medium" mean the same and may be used interchangeably in this disclosure. These terms refer to a single or multiple storage devices and / or media (e.g., centralized or distributed databases and / or associated caches and servers) that store executable instructions and / or data. Accordingly, these terms should be considered to include, but are not limited to, solid-state memory and optical and magnetic media, including memory internal or external to the processor. Specific examples of machine storage media, computer storage media and / or device storage media include non-volatile memory, including, for example: semiconductor memory devices (e.g., erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), field programmable gate array (FPGA), and flash memory devices); magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM optical disks. The terms "machine storage medium", "computer storage medium" and "device storage medium" specifically exclude carrier waves, modulated data signals and other such media (at least some of which are included in the term "signal media" discussed below).

[0164] In various example embodiments, one or more portions of network 1181 may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a portion of the Internet, a portion of a public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, a Wi-Fi® network, another type of network, or a combination of two or more such networks. For example, network 1181 or a portion of network 1181 may include a wireless or cellular network, and coupling device 1182 may be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or another type of cellular or wireless coupling. In this example, the coupling device 1182 can implement any of a variety of types of data transmission technologies, such as single-carrier radio transmission technology (1xRTT), Evolution-Data Optimized (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rates for GSM Evolution (EDGE) technology, including 3G, fourth-generation wireless (4G) networks, universal mobile telecommunications system (UMTS), high-speed packet access (HSPA), worldwide interoperability for microwave access (WiMAX), the third generation partnership project (3GPP) of the long-term evolution (LTE) standards, other technologies defined by various standard setting organizations, other long-range protocols, or other data transmission technologies.

[0165] The instructions 1116 may be transmitted or received over the network 1181 using a transmission medium via a network interface device (e.g., a network interface component included in the communication component 1164) and using any of a number of well-known transmission protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, the instructions 1116 may be transmitted or received to the device 1180 using a transmission medium via a coupling device 1182 (e.g., a peer coupling device). The terms "transmission medium" and "signal medium" have the same meaning and may be used interchangeably in this disclosure. The terms "transmission medium" and "signal medium" should be understood to include any intangible medium capable of storing, encoding, or carrying instructions 1116 for execution by the machine 1100, and include digital or analog communication signals or other intangible media that facilitate the communication of such software. Therefore, the terms "transmission medium" and "signal medium" should be understood to include any form of modulated data signals, carrier waves, etc. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.

[0166] The terms "machine-readable medium," "computer-readable medium," and "device-readable medium" mean the same and may be used interchangeably in this disclosure. These terms are defined to include both machine storage media and transmission media. Thus, these terms include both storage devices / medium and carrier / modulated data signals.

[0167] The various operations of the example methods described herein may be performed at least in part by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform related operations. Similarly, the method described herein may be implemented by a processor at least in part. For example, at least some operations of the method described herein may be performed by one or more processors. The execution of certain operations may be distributed between one or more processors, which not only reside in a single machine, but also are deployed across multiple machines. In some example embodiments, one or more processors may be located in a single location (e.g., in a home environment, an office environment, or a server farm), and in other embodiments, the processors may be distributed in multiple locations.

[0168] Although embodiments of the present disclosure have been described with reference to specific example embodiments, it is apparent that various modifications and changes may be made to these embodiments without departing from the broader scope of the subject matter of the present invention. Therefore, the specification and the drawings are considered to be illustrative rather than restrictive. The drawings forming a part of the present application illustrate specific embodiments in which the subject matter may be implemented by way of illustration and not limitation. The illustrated embodiments are described in sufficient detail to enable those skilled in the art to implement the teachings disclosed herein. Other embodiments and embodiments derived therefrom may be used so that structural and logical replacements and changes may be made without departing from the scope of the present disclosure. Therefore, this detailed description should not be understood in a limiting sense, and the scope of the various embodiments is limited only by the appended claims together with the equivalents to the full scope enjoyed by these claims.

[0169] Such embodiments of the subject matter of the present invention may be referred to herein individually and / or collectively by the term "invention", which is merely for convenience and is not intended to voluntarily limit the scope of the present application to any single invention or inventive concept (if more than one invention or inventive concept is actually disclosed). Therefore, although specific embodiments have been illustrated and described herein, it should be understood that the specific embodiments shown can be replaced with any arrangement calculated to achieve the same purpose. The present disclosure is intended to cover any and all modifications or variations of the various embodiments. After reading the above description, those skilled in the art will understand the combination of the above embodiments and other embodiments not specifically described herein.

[0170] In this document, the terms "a" or "an," as is common in patent documents, are used to include one or more than one, independent of any other instances or uses of "at least one" or "one or more." In this document, the term "or" is used to refer to a non-exclusive or, so that "A or B" includes "A but not B," "B but not A," and "A and B," unless otherwise stated. In the appended claims, the terms "including" and "in which" are used as the plain-English equivalents of the respective terms "comprising" and "wherein." Furthermore, in the appended claims, the terms "including" and "comprising" are open-ended; that is, systems, devices, articles, or processes that include elements in addition to those listed after such terms in a claim are still considered to fall within the scope of that claim.

Claims

1. A method comprising: receiving a first query directed to a shared data set, the first query identifying a first operation; accessing a first data set from the shared data set to perform the first operation, the first data set comprising data accessed from a first column of the shared data set; determining, by at least one hardware processor, that a projection constraint policy is attached to the first column, the projection constraint policy restricting output of data values ​​stored in the first column; Based on the context of the first query, determining that the projection constraint strategy should be implemented with respect to the first query; as well as Based on the first data set and the first operation, an output for the first query is generated, the output for the first query not including the data value stored in the first column based on determining that the projection constraint strategy should be enforced with respect to the first query.

2. The method according to claim 1, wherein: The context of the first query is based on at least one of: a role of a user submitting the first query, an account associated with the first query, and a data share associated with the first query.

3. The method according to claim 1, further comprising: receiving a second query directed to the shared data set, the second query identifying a second operation; accessing a second data set from the shared data set to perform the second operation, the second data set comprising data accessed from a first column of the shared data set; Determine that the projection constraint strategy is attached to the first column; Based on the context of the second query, determining that the projection constraint strategy should not be implemented with respect to the second query; as well as Based on the second data set and the second operation, an output for the second query is generated, the output for the second query comprising the data value stored in the first column based on determining that the projection constraint strategy should not be enforced with respect to the second query.

4. The method according to claim 1, wherein: The shared data set includes a first data set associated with a first entity and a second data set associated with a second entity, the first entity being associated with a first account of a database management system that receives the first query, the query having been received from a computing system associated with the second entity.

5. The method according to claim 4, wherein: The second entity is associated with a second account of the database management system, the first account being different than the second account.

6. The method according to claim 4, further comprising: Data defining the first projection constraint strategy is received from a computing system associated with the first entity.

7. The method according to claim 6, further comprising: Data defining a second projection constraint strategy is received from a computing system associated with the second entity.

8. The method according to claim 1, further comprising: receiving a second query directed to the shared data set, the second query identifying a second operation; accessing a second data set from the shared data set to perform the second operation, the second data set comprising data accessed from a second column of the shared data set, the second column being different from the first column; determining that a projection constraint policy is attached to the second column, the projection constraint policy associated with the second column restricting output of data values ​​stored in the second column; determining, based on the context of the second query, whether the projection constraint strategy associated with the second column should be implemented with respect to the second query; as well as An output for the second query is generated based on the second data set and the second operation, and a determination of whether the projection constraint strategy associated with the second column should be enforced with respect to the second query.

9. The method according to claim 1, wherein: The first operation includes a connection operation.

10. The method according to claim 1, wherein: The first data output indicates a number of matching values ​​in the first column and the second column of the shared data set.

11. A system comprising: one or more computer processors; and One or more computer-readable media storing instructions that, when executed by the one or more computer processors, cause the system to perform operations including: receiving a first query directed to a shared data set, the first query identifying a first operation; accessing a first data set from the shared data set to perform the first operation, the first data set comprising data accessed from a first column of the shared data set; determining that a projection constraint policy is attached to the first column, the projection constraint policy restricting output of data values ​​stored in the first column; Based on the context of the first query, determining that the projection constraint strategy should be implemented with respect to the first query; as well as Based on the first data set and the first operation, an output for the first query is generated, the output for the first query not including the data value stored in the first column based on determining that the projection constraint strategy should be enforced with respect to the first query.

12. The system according to claim 11, wherein: The context of the first query is based on at least one of: a role of a user submitting the first query, an account associated with the first query, and a data share associated with the first query.

13. The system according to claim 11, wherein: The operations also include: receiving a second query directed to the shared data set, the second query identifying a second operation; accessing a second data set from the shared data set to perform the second operation, the second data set comprising data accessed from a first column of the shared data set; Determine that the projection constraint strategy is attached to the first column; Based on the context of the second query, determining that the projection constraint strategy should not be implemented with respect to the second query; and Based on the second data set and the second operation, an output for the second query is generated, the output for the second query comprising the data value stored in the first column based on determining that the projection constraint strategy should not be enforced with respect to the second query.

14. The system according to claim 11, wherein: The shared data set includes a first data set associated with a first entity and a second data set associated with a second entity, the first entity being associated with a first account of a database management system that receives the first query, the query having been received from a computing system associated with the second entity.

15. The system of claim 14, wherein: The second entity is associated with a second account of the database management system, the first account being different than the second account.

16. The system of claim 14, the operations further comprising: Data defining the first projection constraint strategy is received from a computing system associated with the first entity.

17. The system of claim 16, the operations further comprising: Data defining a second projection constraint strategy is received from a computing system associated with the second entity.

18. The system of claim 11, the operations further comprising: receiving a second query directed to the shared data set, the second query identifying a second operation; accessing a second data set from the shared data set to perform the second operation, the second data set comprising data accessed from a second column of the shared data set, the second column being different from the first column; determining that a projection constraint policy is attached to the second column, the projection constraint policy associated with the second column restricting output of data values ​​stored in the second column; determining, based on the context of the second query, whether the projection constraint strategy associated with the second column should be implemented with respect to the second query; as well as An output for the second query is generated based on the second data set and the second operation, and a determination of whether the projection constraint strategy associated with the second column should be enforced with respect to the second query.

19. The system of claim 11, wherein: The first operation includes a connection operation.

20. The system of claim 11, wherein: The first data output indicates a number of matching values ​​in the first column and the second column of the shared data set.

21. A computer-readable medium storing instructions that, when executed by one or more computer processors of one or more computing devices, cause the one or more computing devices to perform operations comprising: receiving a first query directed to a shared data set, the first query identifying a first operation; accessing a first data set from the shared data set to perform the first operation, the first data set comprising data accessed from a first column of the shared data set; determining that a projection constraint policy is attached to the first column, the projection constraint policy restricting output of data values ​​stored in the first column; Based on the context of the first query, determining that the projection constraint strategy should be implemented with respect to the first query; as well as Based on the first data set and the first operation, an output for the first query is generated, the output for the first query not including the data value stored in the first column based on determining that the projection constraint strategy should be enforced with respect to the first query.

22. The computer-readable medium of claim 21, wherein: The context of the first query is based on at least one of: a role of a user submitting the first query, an account associated with the first query, and a data share associated with the first query.

23. The computer readable medium of claim 21, the operations further comprising: receiving a second query directed to the shared data set, the second query identifying a second operation; accessing a second data set from the shared data set to perform the second operation, the second data set comprising data accessed from a first column of the shared data set; Determine that the projection constraint strategy is attached to the first column; Based on the context of the second query, determining that the projection constraint strategy should not be implemented with respect to the second query; as well as Based on the second data set and the second operation, an output for the second query is generated, the output for the second query comprising the data value stored in the first column based on determining that the projection constraint strategy should not be enforced with respect to the second query.

24. The computer-readable medium of claim 21, wherein: The shared data set includes a first data set associated with a first entity and a second data set associated with a second entity, the first entity being associated with a first account of a database management system that receives the first query, the query having been received from a computing system associated with the second entity.

25. The computer readable medium of claim 24, wherein: The second entity is associated with a second account of the database management system, the first account being different than the second account.

26. The computer readable medium of claim 24, the operations further comprising: Data defining the first projection constraint strategy is received from a computing system associated with the first entity.

27. The computer readable medium of claim 26, the operations further comprising: Data defining a second projection constraint strategy is received from a computing system associated with the second entity.

28. The computer readable medium of claim 21, the operations further comprising: receiving a second query directed to the shared data set, the second query identifying a second operation; accessing a second data set from the shared data set to perform the second operation, the second data set comprising data accessed from a second column of the shared data set, the second column being different from the first column; determining that a projection constraint policy is attached to the second column, the projection constraint policy associated with the second column restricting output of data values ​​stored in the second column; determining, based on the context of the second query, whether the projection constraint strategy associated with the second column should be implemented with respect to the second query; as well as An output for the second query is generated based on the second data set and the second operation, and a determination of whether the projection constraint strategy associated with the second column should be enforced with respect to the second query.

29. The computer-readable medium of claim 21, wherein: The first operation includes a connection operation.

30. The computer-readable medium of claim 21, wherein: The first data output indicates a number of matching values ​​in the first column and the second column of the shared data set.

Images