Projection constraints in query processing systems

Abstract

When a query is received by the database system, the constraint system enforces projection constraints on the data values ​​stored in a specified column of the shared dataset. Projection constraints identify data in a column that may be restricted from being projected (e.g., rendered, read, or output) into the output for the received query, while allowing specified operations to be performed on the data and provide corresponding output. For example, a projection constraint may indicate the context of the query that triggers the constraint, such as triggering the constraint based on the user submitting the query. Enforcing projection constraints on queries received at the database system allows data to be anonymously shared and used by entities to perform various operations without the need to tokenize the data.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority to U.S. patent application serial number 17 / 934,814, filed on September 23, 2022, which claims the benefit of priority to U.S. Provisional Application No. 63 / 366,281, filed on June 13, 2022, which are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure generally relates to efficiently managing access to shared data in a database. background

[0004] Databases are used for data storage and access in computing applications. The goal of database storage is to provide large amounts of information in an organized manner so that it can be accessed, managed, and updated. In a database, data can be organized into rows, columns, and tables. Database platforms can have different databases managed by different users. Users may seek to share their database data with each other; however, sharing database data in a secure and scalable manner is difficult. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The various drawings depict only example embodiments of the present disclosure and should not be considered as limiting the scope of the present disclosure.

[0006] Figure 1 An example computing environment is shown in which a network-based database system may implement projection constraints according to some example embodiments.

[0007] Figure 2 is a block diagram illustrating components of a computing service manager according to some example embodiments.

[0008] Figure 3 is a block diagram illustrating components of an execution platform according to some example embodiments.

[0009] Figure 4 Shown are examples of database restrictions on access to database data, according to some example embodiments.

[0010] Figure 5 Example table data is shown according to some example embodiments.

[0011] Figures 6A-6C An example database schema is shown for implementing query templates for multiple entities sharing data in a data cleansing environment, according to some example embodiments.

[0012] Figure 7A and Figure 7B An example database architecture for implementing a defined access clean room is shown, according to some example embodiments.

[0013] Figure 8 is a block diagram illustrating components of a restraint system according to some example embodiments.

[0014] Figures 9A-9D Various data sharing scenarios are shown in which projection constraints may be implemented according to some example embodiments.

[0015] Figure 10 A flowchart of a method for implementing projection constraint-based access to a data set on a distributed database is shown, according to some example embodiments.

[0016] Figure 11 A diagrammatic representation of a machine in the form of a computer system within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, is shown according to some embodiments of the present disclosure. Detailed description

[0017] The following description includes systems, methods, techniques, instruction sequences, and computing machine program products that embody illustrative embodiments of the present disclosure. In the following description, for the purpose of explanation, many specific details are set forth to provide an understanding of the various embodiments of the subject matter of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the subject matter of the present invention can be implemented without these specific details. Typically, well-known instruction instances, protocols, structures, and techniques are not necessarily shown in detail.

[0018] Databases are used by various entities (e.g., businesses, people, organizations, etc.) to store data. For example, a retailer might store data describing purchases (e.g., product, date, price) and data describing the purchaser (e.g., name, address, email address). Similarly, advertisers might store data describing the performance of their advertising campaigns, such as the ads served to users, the dates the ads were served, and information about the users (e.g., name, address, email address).

[0019] In some cases, entities may wish to share their data with one another. For example, a retailer and an advertiser may wish to share their data to determine the effectiveness of an advertising campaign, such as by determining whether users who were presented with an advertisement for a product ultimately purchased that product. In these types of situations, the entities may wish to maintain the confidentiality of some or all of the data they have collected and stored in their respective databases. For example, the retailer and / or advertiser may wish to maintain the confidentiality of personally identifiable information (PII) such as user names, addresses, email addresses, credit card numbers, etc.

[0020] Current solutions address this issue through the use of tokenization, where data values ​​stored in corresponding databases are anonymized into tokens in a consistent and deterministic manner before being shared. For example, each entity can use a hashing algorithm to generate a hash value (e.g., a token) that represents each data value stored in the database. Entities share these tokens with each other, which can then be used to compare data and find matching values ​​while maintaining the confidentiality of the underlying data.

[0021] However, tokenization has several disadvantages. One disadvantage is that tokenization does not allow for fuzzy or approximate matches. In some cases, data may not be entered into various databases in a consistent format or correctly. Because tokenization utilizes a deterministic algorithm to generate tokens, it can only identify exact matches and does not account for these variations in format and data entries (e.g., fuzzy matches). Another disadvantage of tokenization is the complexity and computational resources required to generate the tokens. For example, a hashing algorithm is used to generate hash values ​​for a large number of data values ​​stored in corresponding databases. Another disadvantage of tokenization is its limited functionality because tokens can only be used to find exact matches between two data sets. Other functions, such as determining the number of data entries that do not match but include a specified string or character set, cannot be performed.

[0022] To mitigate these issues, database systems can utilize a constraint system that enforces projection constraints on data values ​​stored in specified columns of a shared dataset when the database system receives a query. Projection constraints identify that data in a column may be restricted from being projected (e.g., rendered, read, or output) into the output for the received query, while allowing specified operations to be performed on the data and providing corresponding output. For example, a projection constraint can be triggered based on the context of the query, such as a user submitting the query indicating that the constraint has been triggered.

[0023] Entities that share data can define projection constraints that will be attached to various columns of the shared dataset. For example, an entity can define one or more columns to which a projection constraint should be attached, as well as the conditions for triggering the constraint. When the database system receives a query directed to a shared dataset, the constraint system accesses the data required to process the query from the shared database and determines whether to attach a projection constraint to any column of the shared dataset from which data is accessed. If a projection constraint is attached to one of the columns, the constraint system determines whether the projection constraint should be enforced based on the context of the query and generates output accordingly. For example, if a projection constraint should be enforced, the constraint system may generate output that does not include the data values ​​stored in the column, but may provide output determined based on the constrained data, such as the number of matches, the number of fuzzy matches, the number of matches that include a specified string, unconstrained data associated with the constrained data, and the like.

[0024] Enforcing projection constraints on queries received at a database system allows data to be anonymously shared and used by entities to perform various operations without the need to tokenize the data. Thus, the difficulty and computational resources required for tokenization are eliminated, and additional functions and operations (e.g., fuzzy matching) can be performed while maintaining the confidentiality of the specified data values.

[0025] Figure 1 1 shows an example computing environment 100 in which a network-based database system 102 can implement projection constraints according to some example embodiments. To avoid obscuring the present subject matter with unnecessary detail, Figure 1 Various functional components that are not closely related to conveying an understanding of the subject matter of the present invention have been omitted. However, those skilled in the art will readily recognize that various additional functional components can be included as part of computing environment 100 to facilitate additional functionality not specifically described herein. In other embodiments, the computing environment may include another type of network-based database system or cloud data platform.

[0026] As shown, the computing environment 100 includes a cloud storage platform 104 (e.g., AWS ® Microsoft Azure BlobStorage ® , or Google Cloud Storage). Network-based database system 102 is a network-based system for reporting and analyzing integrated data from one or more different sources, including one or more storage locations within cloud storage platform 104. Cloud storage platform 104 includes multiple computing machines and provides computer system resources, such as data storage and computing power, to network-based database system 102 on demand.

[0027] The network-based database system 102 includes a computing service manager 108, an execution platform 110, and one or more metadata databases 112. The network-based database system 102 hosts and provides data reporting and analysis services to multiple client accounts.

[0028] The computing service manager 108 coordinates and manages the operations of the network-based database system 102. The computing service manager 108 also performs query optimization and compilation, and manages a cluster of computing services (also known as a "virtual warehouse") that provide computing resources. The computing service manager 108 can support any number of client accounts, such as end users providing data storage and retrieval requests, system administrators who manage the systems and methods described herein, and other components / devices that interact with the computing service manager 108.

[0029] The computing service manager 108 also communicates with a client device 114. The client device 114 corresponds to a user of one of multiple client accounts supported by the network-based database system 102. The user can utilize the client device 114 to submit data storage, retrieval, and analysis requests to the computing service manager 108.

[0030] The computing service manager 108 is also coupled to one or more metadata databases 112, which store metadata related to various functions and aspects associated with the network-based database system 102 and its users. For example, the metadata database 112 may include a summary of data stored in remote data storage systems and data available from local caches. Furthermore, the metadata database 112 may include information about how data is partitioned and organized in remote data storage systems (e.g., cloud storage platform 104) and local caches. As discussed herein, a "micro-partition" is a bulk unit of storage, with each micro-partition comprising contiguous storage units. For example, each micro-partition may contain between 50 MB and 500 MB of uncompressed data (note that the actual size in storage may be smaller, as data may be stored compressed). Groups of rows in a table may be mapped to separate micro-partitions organized in columns. This size and structure allows for extremely fine-grained selection of micro-partitions to scan, which can include millions or even hundreds of millions of micro-partitions. This granular selection process for micro-partitions to scan is referred to herein as "pruning." Pruning involves using metadata to determine which parts of a table (including which micro-partitions or groups of micro-partitions within the table) are irrelevant to a query, thereby avoiding those irrelevant micro-partitions when responding to the query and scanning only relevant micro-partitions to respond to the query. Metadata about all rows stored in a micro-partition can be automatically collected, including: the value range for each column in the micro-partition; the number of distinct values; and / or other attributes used for optimized and efficient query processing. In one embodiment, micro-partitioning can be automatically performed on all tables. For example, a table can be transparently partitioned using sorting that occurs when inserting / loading data. However, it should be understood that this disclosure of micro-partitioning is merely exemplary and should be considered non-limiting. It should be understood that micro-partitioning can include other database storage devices without departing from the scope of this disclosure. The information stored by the metadata database 112 (e.g., a key-value data store) allows systems and services to determine whether a piece of data (e.g., a given partition) needs to be accessed without loading or accessing the actual data from the storage device.

[0031] The computing service manager 108 is also coupled to an execution platform 110, which provides a plurality of computing resources for performing various data storage and data retrieval tasks. The execution platform 110 is coupled to a cloud storage platform 104. The cloud storage platform 104 includes a plurality of data storage devices 120-1 to 120-N. In some embodiments, the data storage devices 120-1 to 120-N are cloud-based storage devices located in one or more geographic locations. For example, the data storage devices 120-1 to 120-N can be part of a public cloud infrastructure or a private cloud infrastructure. The data storage devices 120-1 to 120-N can be hard disk drives (HDDs), solid state drives (SSDs), storage clusters, Amazon S3 TM Storage system or any other data storage technology. In addition, the cloud storage platform 104 may include a distributed file system (such as Hadoop Distributed File System (HDFS)), an object storage system, etc.

[0032] The execution platform 110 includes multiple compute nodes. A collection of processes on the compute nodes executes a query plan compiled by the compute service manager 108. The collection of processes may include: a first process that executes the query plan; a second process that uses a least recently used (LRU) strategy to monitor and delete cache files and implement out-of-memory (OOM) error mitigation; a third process that extracts health information from process logs and status and sends it back to the compute service manager 108; a fourth process that establishes communication with the compute service manager 108 after system boot; and a fifth process that handles all communications with the compute cluster for a given job provided by the compute service manager 108 and transmits information back to the compute service manager 108 and other compute nodes of the execution platform 110.

[0033] In some embodiments, the communication links between the elements of computing environment 100 are implemented via one or more data communication networks. These data communication networks can utilize any communication protocol and any type of communication medium. In some embodiments, the data communication network is a combination of two or more data communication networks (or subnetworks) coupled to each other. In alternative embodiments, these communication links are implemented using any type of communication medium and any communication protocol.

[0034] The computing service manager 108, metadata database 112, execution platform 110 and cloud storage platform 104 are Figure 1104 are shown as separate, discrete components. However, each of the computing service manager 108, metadata database 112, execution platform 110, and cloud storage platform 104 can be implemented as a distributed system (e.g., distributed across multiple systems / platforms in multiple geographic locations). In addition, each of the computing service manager 108, metadata database 112, execution platform 110, and cloud storage platform 104 can scale up or down (independently of each other) based on changes in received requests and the changing needs of the network-based database system 102. Thus, in the described embodiment, the network-based database system 102 is dynamic and supports regular changes to meet current data processing needs.

[0035] During typical operation, network-based database system 102 processes multiple jobs determined by computing service manager 108. These jobs are scheduled and managed by computing service manager 108 to determine when and how the jobs should be executed. For example, computing service manager 108 may divide the job into multiple discrete tasks and determine what data is required to execute each of the multiple discrete tasks. Computing service manager 108 may assign each of the multiple discrete tasks to one or more nodes of execution platform 110 to process the task. Computing service manager 108 may determine what data is required to process the task and further determine which nodes within execution platform 110 are best suited to handle the task. Some nodes may already have cached the data required to process the task and therefore be good candidates for processing the task. Metadata stored in metadata database 112 assists computing service manager 108 in determining which nodes within execution platform 110 have cached at least a portion of the data required to process the task. One or more nodes within execution platform 110 use the data cached by the nodes and, if necessary, data retrieved from cloud storage platform 104 to process the task. It is desirable to retrieve as much data as possible from the cache within the execution platform 110 because the retrieval speed is typically much faster than retrieving data from the cloud storage platform 104 .

[0036] like Figure 1 As shown, the computing environment 100 separates the execution platform 110 from the cloud storage platform 104. In this arrangement, the processing resources and cache resources in the execution platform 110 operate independently of the data storage devices 120-1 through 120-N in the cloud storage platform 104. Therefore, the computing resources and cache resources are not limited to specific data storage devices 120-1 through 120-N. Instead, all computing resources and all cache resources can retrieve data from and store data in any data storage resource in the cloud storage platform 104.

[0037] Figure 2is a block diagram illustrating components of the computing service manager 108 according to some embodiments of the present disclosure. Figure 2 As shown, the computing service manager 108 includes a credential management system 204 and an access manager 202 coupled to a data storage device 206, which is an example of a metadata database 112. The access manager 202 handles authentication and authorization tasks for the system described herein. The credential management system 204 facilitates access to external resources, such as data resources in a remote storage device, using remotely stored credentials. As used herein, a remote storage device may also be referred to as a "persistent storage device" or "shared storage device." For example, the credential management system 204 may create and maintain (e.g., in the data storage device 206) a remote credential store definition and a credential object. The remote credential store definition identifies the remote credential store and includes access information for accessing security credentials from the remote credential store. The credential object uses non-sensitive information (e.g., a text string) to identify one or more security credentials that are retrieved from the remote credential store to access external resources. When a request to call an external resource is received at runtime, the credential management system 204 and the access manager 202 use the information stored in the data storage device 206 (e.g., the access metadata database, credential objects, and credential store definitions) to retrieve security credentials for accessing the external resource from the remote credential store.

[0038] The request processing service 208 manages received data storage requests and data retrieval requests (e.g., jobs to be executed on database data). For example, the request processing service 208 may determine the data to be used to process a received query (e.g., a data storage request or a data retrieval request). The data may be stored in a cache within the execution platform 110 or in a data storage device within the cloud storage platform 104.

[0039] The management console service 210 supports access by administrators and other system managers to various systems and processes. Additionally, the management console service 210 may receive requests to execute jobs and monitor workloads on the system.

[0040] The computing service manager 108 also includes a job compiler 212, a job optimizer 214, and a job executor 216. The job compiler 212 parses a job into multiple discrete tasks and generates execution code for each of the multiple discrete tasks. The job optimizer 214 determines the optimal method for executing the multiple discrete tasks based on the data to be processed. The job optimizer 214 also handles various data pruning operations and other data optimization techniques to improve the speed and efficiency of job execution. The job executor 216 executes the execution code of a job received from a queue or determined by the computing service manager 108.

[0041] The job scheduler and coordinator 218 sends the received jobs to the appropriate service or system for compilation, optimization and dispatch to Figure 1 For example, the jobs can be prioritized and then processed in order of priority. In an embodiment, the job scheduler and coordinator 218 determines the jobs that are Figure 1 The job scheduler and coordinator 218 prioritizes internal jobs scheduled by the compute service manager 108 over other "external" jobs (such as user queries that may be scheduled by other systems in the database but may utilize the same processing resources in the execution platform 110). In some embodiments, the job scheduler and coordinator 218 identifies or assigns specific nodes in the execution platform 110 to process specific tasks. The virtual warehouse manager 220 manages the operation of multiple virtual warehouses implemented in the execution platform 110. For example, the virtual warehouse manager 220 can generate a query plan for executing a received query. As discussed in further detail below, the data clean room system 230 allows for dynamically restricted data access to shared data sets. As discussed in further detail below, the constraint system 240 provides projection constraints on data values ​​stored in specified columns of the shared data sets.

[0042] As shown, the compute service manager 108 includes a configuration and metadata manager 222, which manages information related to data stored in remote data storage devices and local buffers (e.g., buffers within the execution platform 110). The configuration and metadata manager 222 uses metadata to determine which data files need to be accessed to retrieve data for processing a particular task or job. A monitor and workload analyzer 224 oversees the processes executed by the compute service manager 108 and manages the distribution of tasks (e.g., workloads) across execution nodes and virtual repositories within the execution platform 110. The monitor and workload analyzer 224 also reallocates tasks as needed based on the changing workload across the network-based database system 102 and can also reallocate tasks based on user (e.g., "external") query workloads that can also be processed by the execution platform 110. The configuration and metadata manager 222 and the monitor and workload analyzer 224 are coupled to a data storage device 226. Data storage device 226 represents any data storage device within the network-based database system 102. For example, data storage device 226 may represent a buffer in execution platform 110 , a storage device in cloud storage platform 104 , or any other storage device.

[0043] As described in the embodiments herein, the computing service manager 108 verifies all communications from an execution platform (e.g., execution platform 110) to verify that the content and context of the communication are consistent with the tasks known to be assigned to the execution platform. For example, an instance of an execution platform executing query A should not be allowed to request access to a data source D (e.g., data storage device 226) that is not related to query A. Similarly, a given execution node (e.g., Figure 3 Execution node 302-1) may need to communicate with another execution node (e.g., Figure 3 2) is prohibited from communicating with a third execution node (e.g., execution node 312-1), and any such illegal communications may be recorded (e.g., in a log or other location). Furthermore, the information stored by a given execution node is limited to data relevant to the current query, with any other data unavailable, which is achieved by destruction or encryption (if the key is unavailable).

[0044] Figure 3 is a diagram showing some embodiments of the present disclosure Figure 1 A block diagram of the components of the execution platform 110 is shown. Figure 3 As shown, the execution platform 110 includes multiple virtual warehouses, including virtual warehouse 1, virtual warehouse 2, and virtual warehouse N. Each virtual warehouse includes multiple execution nodes, and each execution node includes a data cache and a processor. The virtual warehouse can execute multiple tasks in parallel by using multiple execution nodes. As discussed herein, the execution platform 110 can add new virtual warehouses and discard existing virtual warehouses in real time based on the current processing needs of the system and users. This flexibility allows the execution platform 110 to quickly deploy large amounts of computing resources when needed without having to continue paying for those computing resources when they are no longer needed. All virtual warehouses can access data in any data storage device (e.g., any storage device in the cloud storage platform 104).

[0045] Although Figure 3 Each virtual warehouse shown in includes three execution nodes, but a particular virtual warehouse may include any number of execution nodes. Furthermore, the number of execution nodes in a virtual warehouse is dynamic, such that new execution nodes are created when there is additional demand, and existing execution nodes are deleted when they are no longer useful.

[0046] Each virtual warehouse can access Figure 1 Thus, a virtual warehouse does not have to be assigned to a specific data storage device 120-1 to 120-N, but can access data in any of the data storage devices 120-1 to 120-N within the cloud storage platform 104. Similarly, Figure 3 Each execution node shown in FIG can access data from any one of the data storage devices 120 - 1 to 120 -N. In some embodiments, a specific virtual repository or a specific execution node can be temporarily assigned to a specific data storage device, but the virtual repository or execution node can later access data in any other data storage device.

[0047] exist Figure 3 In the example shown in FIG, virtual warehouse 1 includes three execution nodes 302-1, 302-2, and 302-N. Execution node 302-1 includes a cache 304-1 and a processor 306-1. Execution node 302-2 includes a cache 304-2 and a processor 306-2. Execution node 302-N includes a cache 304-N and a processor 306-N. Each execution node 302-1, 302-2, and 302-N is associated with processing one or more data storage and / or data retrieval tasks. For example, a virtual warehouse may process data storage and data retrieval tasks associated with internal services (such as a clustering service, an instantiated view refresh service, a file compression service, a stored procedure service, or a file upgrade service). In other embodiments, a specific virtual warehouse may process data storage and data retrieval tasks associated with a specific data storage system or a specific category of data.

[0048] Similar to virtual warehouse 1 discussed above, virtual warehouse 2 includes three execution nodes 312-1, 312-2, and 312-N. Execution node 312-1 includes a cache 314-1 and a processor 316-1. Execution node 312-2 includes a cache 314-2 and a processor 316-2. Execution node 312-N includes a cache 314-N and a processor 316-N. Additionally, virtual warehouse 3 includes three execution nodes 322-1, 322-2, and 322-N. Execution node 322-1 includes a cache 324-1 and a processor 326-1. Execution node 322-2 includes a cache 324-2 and a processor 326-2. Execution node 322-N includes a cache 324-N and a processor 326-N.

[0049] In some embodiments, Figure 3 The illustrated execution nodes are stateless with respect to data cached by the execution nodes. For example, these execution nodes do not store or otherwise maintain state information about the execution nodes or the data cached by a particular execution node. Therefore, in the event of an execution node failure, the failed node can be transparently replaced with another node. Because there is no state information associated with the failed execution node, the new (replacement) execution node can easily replace the failed node without having to worry about recreating specific state.

[0050] Although Figure 3 The execution nodes shown each include a data cache and a processor, but alternative embodiments may include execution nodes that include any number of processors and any number of caches. Additionally, the size of the caches may vary between different execution nodes. Figure 3 The cache shown in the figure stores data from the local execution node. Figure 1 Caches data retrieved from one or more data storage devices in the cloud storage platform 104. Thus, caching reduces or eliminates bottlenecks that arise in platforms that continually retrieve data from remote storage systems. Rather than repeatedly accessing data from remote storage devices, the systems and methods described herein access data from caches in execution nodes, which is significantly faster and avoids the bottlenecks discussed above. In some embodiments, caches are implemented using high-speed memory devices that provide fast access to cached data. Each cache can store data from any storage device in the cloud storage platform 104.

[0051] In addition, cache resources and computing resources can vary between different execution nodes. For example, one execution node may contain a large amount of computing resources and minimal cache resources, thereby making the execution node useful for tasks that require a large amount of computing resources. Another execution node may contain a large amount of cache resources and minimal computing resources, thereby making the execution node useful for tasks that require caching large amounts of data. Yet another execution node may contain cache resources that provide faster input-output operations, which is useful for tasks that require rapid scanning of large amounts of data. In some embodiments, the cache resources and computing resources associated with a particular execution node are determined when the execution node is created based on the expected tasks that the execution node will perform.

[0052] Additionally, the cache resources and compute resources associated with a particular execution node can change over time based on the changing tasks being performed by the execution node. For example, if the tasks being performed by an execution node become more processor-intensive, more processing resources can be allocated to that execution node. Similarly, if the tasks being performed by an execution node require greater cache capacity, more cache resources can be allocated to that execution node.

[0053] Although virtual warehouses 1, 2, and N are associated with the same execution platform 110, multiple computing systems in multiple geographic locations may be used to implement the virtual warehouses. For example, virtual warehouse 1 may be implemented by a computing system in a first geographic location, while virtual warehouses 2 and N may be implemented by another computing system in a second geographic location. In some embodiments, these different computing systems are cloud-based computing systems maintained by one or more different entities.

[0054] In addition, each virtual warehouse Figure 3 1 and 302-2 are shown as having multiple execution nodes. Multiple computing systems located in multiple geographic locations can be used to implement the multiple execution nodes associated with each virtual warehouse. For example, an instance of virtual warehouse 1 implements execution nodes 302-1 and 302-2 on one computing platform at one geographic location, while implementing execution node 302-N on a different computing platform at another geographic location. The selection of a particular computing system to implement an execution node can depend on various factors, such as the resource level required for the particular execution node (e.g., processing resource requirements and cache requirements), the resources available at the particular computing system, the communication capabilities of the network within or between geographic locations, and which computing systems have already implemented other execution nodes in the virtual warehouse.

[0055] The execution platform 110 is also fault-tolerant. For example, if a virtual warehouse fails, the virtual warehouse will be quickly replaced by a different virtual warehouse located in a different geographical location.

[0056] A particular execution platform 110 may include any number of virtual warehouses. Furthermore, the number of virtual warehouses in a particular execution platform may be dynamic, allowing new virtual warehouses to be created as additional processing and / or caching resources are needed. Similarly, existing virtual warehouses may be deleted when the resources associated with them are no longer useful.

[0057] In some embodiments, virtual repositories can operate on the same data within the cloud storage platform 104, but each virtual repositories has its own execution node with independent processing and cache resources. This configuration allows requests to different virtual repositories to be processed independently without interfering with each other. This independent processing, combined with the ability to dynamically add and remove virtual repositories, supports adding new processing capacity for new users without impacting performance.

[0058] Figure 4 A dynamically constrained data clean room system 230 is shown in accordance with some example embodiments. Figure 4 In the example below, a first database account 405 and a second database account 450 share data in the data cleanroom system 230, and either account can issue queries against the data. In the following example, the first database account 405 provides data to the second database account 450 (e.g., using the approved statement table 410, the row access policy engine (RAP) 415, the source data 420, and the shared source data 425), and it should be understood that the second database account 450 can similarly share data with the first database account 405 (e.g., using the approved statement table 455, the row access policy engine (RAP) 460, the source data 465, and the shared source data 470).

[0059] exist Figure 4In the example, data cleanroom system 230 implements a row access policy scheme (e.g., row access policy engine 415, row access policy engine 460) for a shared data set (e.g., source data 420, source data 465) between a first database account and a second database account. In some example embodiments, row access policy engine 460 is implemented as a database object in network-based database system 102 that restricts the use and / or sharing of the database account's source data within the cleanroom. In some example embodiments, a database object in network-based database system 102 is a data structure used to store and / or reference data. In some example embodiments, network-based database system 102 implements one or more of the following objects: a database table, a view, an index, a database system stored procedure, a database system user-defined function, or a sequence. In some example embodiments, when network-based database system 102 creates a database object type, the object is locked, and because network-based database system 102 restricts object types using the database system's source code, new object types cannot be created. In some example embodiments, a database object instance is an instance of a database object type created by the database system 102 when an object is created (e.g., such as a new table, an index on the table, a view on the same table, or a new stored procedure object). The row access policy engine 460 provides row-level security for data in the network-based database system 102 by using row access policies to determine which rows are returned in query results. Examples of row access policies include allowing a specific role to view rows of a table (e.g., the user role of the end user issuing the query) or including mapping tables defined according to the policy to determine access to rows in a given query result. In some example embodiments, a row access policy is a schema-level object of the network-based database system 102 that determines whether a given row in a table or view can be viewed based on different types of database statements, including SELECT statements, or rows selected through UPDATE, DELETE, and MERGE statements.

[0060] In some example embodiments, a row access policy includes conditions and functions that transform data at query runtime when those conditions are met. Policy data is implemented to limit exposure to sensitive data. Policy data can further restrict object owners (e.g., roles with OWNERSHIP permissions on objects such as tables or views), who typically have full access to the underlying data. In some example embodiments, a single row access policy engine is set up for different tables and views that are implemented simultaneously. In some example embodiments, row access policies can be added to tables or views when the objects are created or afterward.

[0061] In some example embodiments, a row access policy includes an expression that can specify a database object (e.g., a table or view) and uses conditional expression functions and context functions to determine which rows should be visible in a given context. The following is an example of a row access policy implemented during query execution: (A) For the data specified in the query, the network-based database system 102 determines whether a row access policy has been set for the database object. If a policy has been added to the database object, all rows are protected by the policy. (B) The distributed database system then creates a dynamic security view (e.g., a secure database view) of the database object. (C) The policy expression is evaluated. For example, the policy expression may specify a "current statement" expression that only proceeds if the "current statement" is in an approved statement table or if the current role of the user issuing the query is a previously specified and permitted role. (D) Based on the policy evaluation, the restriction engine generates a query output, such as source data to be shared from a first database account to a second database account, where the query output only includes rows that evaluate to true based on the policy definition.

[0062] Continue to refer Figure 4 The contents of the approved statement table are agreed upon by the first database account 405 and the second database account 450 or generated in other ways by the first database account 405 and the second database account 450. For example, the users who manage the first database account 405 and the second database account 450 agree upon a query language acceptable to both parties and include the query language in the approved statement table, and the agreed upon language is stored in the approved statement table 410 on the first database account 405 and also in the approved statement table 455 in the second database account 450. As an illustrative example, Figure 5 As shown, source data 420 for a first database account 405 may include a first email dataset 500 for users of the first database account, and source data 465 for a second database account 450 may include a second email dataset 550 for users of the second database account. Both database accounts may seek to determine how many matches there are in the email addresses of users in their respective datasets, where the results returned are numbers (e.g., each has end users, and both database accounts are interested in the number of users they share, but do not want to share data on the actual users). To do this, both database accounts store a "SELECT COUNT" in the approved query request table. In this way, count queries that select and join source data can continue, but request and potentially return "counts" of all user data. The query cannot continue because it is not in the approved statement table (e.g., approved statement table 410 and approved statement table 455) of the corresponding dataset account.

[0063] In addition, although Figure 4 Only two database accounts are shown in FIG, but the data clean room system 230 enables two or more database accounts to share data through the clean room architecture. In past approaches, the data clean room data was obfuscated (e.g., tokenized) and then shared in the data clean room, and the complexity of matching the obfuscated data may result in limiting the data clean room data to only two parties at a time. In contrast, in Figure 4 In the method, according to some example embodiments, the third database account (not in Figure 4 ) can use the data clean room system 230 in the computing service manager 108 to provide a third-party shared data set 477, and can issue a database statement that connects data from the three data sets, such as SELECT COUNT, which is for the connected data from the source data 420, the shared source data 470 from the second database account 450, and the third-party shared data set 477 from a third database account (for example, as opposed to a requester database account that shares data with the first provider database account, and the requester database account also uses the cryptographic function sequence provided by the first provider account and the second provider account to relate the data to another second provider database account).

[0064] Figures 6A-6C An example data cleanroom architecture for sharing data between multiple parties is shown in accordance with some example embodiments. In the example shown, party_1 database account 600 is Figure 6A In the case of Fang_2 database account 605, Figure 6B In, and the square_3 database account 610 is in Figure 6C In which data is transferred (e.g., copied, shared) between different accounts, as indicated by the broken arrows with reference to the labels of other figures; e.g., in Figure 6B In the example, a "Party 2 Outbound Share" is shared from Party_2 database account 605 to Party_1 database account 600, where the share is marked as "Party 2 Share" and is Figure 6A and Figure 6B The following data flows are connected by broken arrows. Figures 6A-6C For example, in operation 650, the party_1 database account 600 creates its APPROVED_STATEMENTS in its own database instance (e.g., in Figure 6A ); Similarly, at operation 650, the party_2 database account 605 creates its APPROVED_STATEMENTS in its own database instance (e.g., Figure 6B), and further, the party_3 database account 610 creates its APPROVED_STATEMENTS in its own database instance (e.g., in Figure 6C shown in ).

[0065] At operation 650, each party creates an APPROVED_STATEMENTS table that will store the query request SQL statements that have been verified and approved. In some example embodiments, one party creates the approved statement table, which is then stored by the other parties. In some example embodiments, each of the parties creates its own approved statement table, and a given query against the shared data must satisfy each of the approved statement tables, otherwise the query cannot proceed (e.g., " " must be on each corresponding party's list of approved statements in order for the statement containing " " queries operate on data shared between parties in the cleanroom).

[0066] Each party creates a row access policy that will be applied to the source table shared with each other party for cleanroom request processing at operation 655. The row access policy will check the current_statement() function against the value stored in the APPROVED_STATEMENTS table.

[0067] At operation 660, each party generates its AVAILABLE_VALUES table, which acts as a data dictionary for other parties to understand which columns and values ​​they can use in query requests. In some example embodiments, the available values ​​include a schema, allowed columns, and metadata specifying prohibited row or cell values. In some example embodiments, the available values ​​data is not the actual data itself (e.g., source data), but rather specifies what data (e.g., which columns of the source data) other parties (e.g., consumer accounts) can access for their respective shared data jobs (e.g., overlap analysis).

[0068] Return Reference Figure 6C At operation 665, each party agrees on one or more query templates that can be used for query requests. For example, if a media publisher and advertiser are working together in a cleanroom, they might approve an "audience overlap" query template. The query template will store connection information and other static logic, while using placeholders for variables (select fields, filters, etc.).

[0069] As an additional example, one party is a provider account that specifies which statements are stored in the available statements table (e.g., thereby specifying how any consumer account that wants to access the provider's data will access the provider's data). Furthermore, in some example embodiments, the provider account also provides one or more query templates for use by any party (e.g., a consumer account) that seeks to access the provider's data based on the query templates. For example, a query template may include a blank or placeholder " ”, which can be replaced by a specific field via a consumer request (e.g., the specific field can be a column from the consumer data or a column from the provider data). Any changes to the query template (e.g., adding an asterisk “ to select all records) will be rejected by data restrictions on the provider data (for example, row access policies (RAPs) act as a firewall for the provider data).

[0070] Next, in operation 670 ( Figure 6A ), one of the parties (for example, Party_1 database account 600 in this example) generates a cleanroom query request by calling the GENERATE_QUERY_REQUEST stored procedure. This procedure inserts the new request into the QUERY_REQUESTS table. This table is shared with every other party, along with the source data table with row access policies enabled, the party's AVAILABLE_VALUES table, and the REQUEST_STATUS table.

[0071] At operation 675, each party has a stream object created for the other party's QUERY_REQUESTS table, thereby capturing any inserts into that table. If the stream object has data, the task object will run according to the set schedule and execute the VALIDATE_QUERY stored procedure.

[0072] At operation 680, the VALIDATE_QUERY process is configured to: (1) Ensure that the query request selects and filters columns with valid attributes by comparing them to the AVAILABLE_VALUES table. (2) Ensure that the query template accepts the submitted variables. (3) Ensure that thresholds or other query restrictions are applied. (4) Generate a create table as select (CTAS) statement and store it in the APPROVED_STATEMENTS table if validation is successful. (5) Update the REQUEST_STATUS table with success or failure. If successful, the create table as select (CTAS) statement is also added to the record.

[0073] The GENERATE_QUERY_REQUEST process will also call the VALIDATE_QUERY process against the requesting party's account at operation 685. This is to ensure that each additional party generated query matches the requesting party as an additional layer of verification.

[0074] The REQUEST_STATUS table shared by each party is updated with the status from the VALIDATE_QUERY process at operation 690. The GENERATE_QUERY_REQUEST process will wait and poll each REQUEST_STATUS table until a status is returned.

[0075] Once each party has returned a status, the GENERATE_QUERY_REQUEST procedure will compare all CTAS statements to ensure they match (if the status is approved) in operation 699. If they all match, the procedure will execute the statements and generate a result table.

[0076] Figure 7A and Figure 7B An example data architecture for implementing a defined access cleanroom using a native application is shown, according to some example embodiments. In some example embodiments, the native application is configured so that the provider can create local state objects (e.g., tables) and local computation objects (e.g., stored procedures, external functions, tasks), and also share objects representing application logic in the consumer account. In some example embodiments, the native application is installed in the consumer account as a sharable database instance. For example, the provider can generate a native application that includes stored procedures and external functions that analyze and enrich data in a given consumer account. The consumer can install the provider's native application as a database in the consumer's account and call stored procedures in the installed native application that provide application functionality. In some example embodiments, the native application is configured to write only to the database in the consumer account. Furthermore, in some example embodiments, the provider's native application can be packaged with one or more other objects (such as tables, views, and stored procedures for the provider account), which are then generated in the consumer account during installation via an installer script. In some example embodiments, the native application installer script is configured to: (1) create a local object in a consumer account, and (2) control visibility of the object in the native application with different consumer accounts that may install the provider's native application.

[0077] Figure 7A The provider database account 702 is shown, and Figure 7B A customer database account 751 is shown, where Figure 7A and Figure 7BConnections between are shown using circled capital letters (for example, A, B, C, and D). Figure 7A In operation 705, the provider database account 702 generates a defined access cleanroom 704 (DCR). In operation 710, the provider database account 702 shares the installer cleanroom stored procedure 706 as a native database application with the consumer database account 751. Figure 7A In operation 715, the provider database account 702 shares the source data 708 as a source data database view 711 in the clean room 712, and the source data 708 is then accessed by the consumer database account 751 as (in Figure 7B ) source data 714. Although source data 714 is accessible as a share by consumer database account 751, source data 714 may be empty (e.g., not yet populated) and controlled by a data firewall 716 (such as the row access policy of provider database account 702 as described above). Figure 7B In operation 720 , the customer database account 751 creates the cleanroom customer database 718 .

[0078] In operation 725, the customer database account 751 creates a database storage 721 to store the source data 714 shared from the provider database account 702. In operation 730, the customer database account 751 shares a request table 722 with the provider database account 702 as a customer-defined cleanroom sharing request table 723 (in Figure 7A In operation 735, the provider database account 702 creates a consumer storage database 724 to store the request table 723 received from the consumer database account 751 as a consumer share. In addition, the provider database account 702 creates a management object 737, which includes a flow object for tracking changes on the request table 723, and a task object in the management object 737 to execute the process request stored procedure 743 when a new request is entered into the request table 723 (for example, a request from a consumer and a user, which is entered into the request table 722 and automatically shared as an entry in the request table 723). Figure 7B In operation 745, the consumer database account 751 creates a database store 721 to store the provider's shared source data 714 (in Figure 7B ), which starts the stored procedure installer script that generates the runtime instance of the native application 757. Figure 7BIn some example embodiments, at operation 750, a data cleanroom native application 757 is executed and created using the native application installer process 706, creating a cleanroom schema and all objects within the cleanroom as specified in the native application installer process 706. Furthermore, the native application installer process 706 grants permissions to tables and request data storage processes. Furthermore, the native application installer process 706 creates an application internal schema 759 for request processing.

[0079] At operation 755 , consumer database account 751 generates a cleanroom request by calling request stored procedure 789 , passing in a query template name (e.g., the name of a template from query templates 756 , a template repository), selecting grouping by columns, filters, a privacy budget to be achieved, and any other parameters required for the selected query template or otherwise passed in.

[0080] At operation 760 , the consumer database account 751 implements a request stored process 789 that is configured to (1) generate a query based on a query template and passed-in parameters, (2) sign the query request using an encryption key created by the data clean room native application 757 to authenticate to the provider database account 702 that the data clean room native application 757 issued the request, (3) apply a differential privacy noise parameter to the query results based on an epsilon value (also known as a privacy budget) passed in with the query, and (4) automatically share the query with the provider as an entry in the request table 723 when the query is entered into the request table 722 .

[0081] exist Figure 7AAt operation 765, provider database account 702 implements a flow for capturing an insert entry into request table 723 and subsequently triggering management object task 737 to execute process request stored procedure 743. At operation 770, process request stored procedure 743 performs a query to validate the request. In some example embodiments, the validation performed by process request stored procedure 743 includes (1) determining whether the encrypted request key matches the provider key, (2) confirming that the request originates from a corresponding pre-authorized consumer account (e.g., consumer database account 751), (3) confirming that the query uses a valid template from template 746 (e.g., a plurality of valid and pre-configured templates authorized by the provider), (4) confirming that the instant ID of data cleanroom native application 757 matches the expected instance ID, and (5) confirming that provider database account 702 is the expected or pre-configured account. At operation 775, if the request is valid, the provider database account 702 updates the status to "approved" in the request log 776, which configures the data firewall 716 (e.g., row access policy) to provide access to one or more rows from the source data 708; wherein the rows provided by the RAP are then shared as source data 714 to the consumer database account 751. Figure 7B In the example, once the data is shared to the source data 714 , the consumer database account 751 can execute queries within the data cleanroom native application 757 on the consumer database account 751 (eg, via an execution node of the consumer database account 751 ).

[0082] Figure 8 is a block diagram illustrating components of constraint system 240 according to some example embodiments. As previously explained, databases are used by various entities (e.g., businesses, people, organizations, etc.) to store data. For example, a retailer may store data describing purchases (e.g., product, date, price) and data describing the purchaser (e.g., name, address, email address). Similarly, advertisers may store data describing the performance of their advertising campaigns, such as the advertisements served to users, the dates on which the advertisements were served, and information about the users (e.g., name, address, email address).

[0083] In some cases, entities may wish to share their data with one another. For example, a retailer and an advertiser may wish to share their data to determine the effectiveness of an advertising campaign, for example, by determining whether users who were presented with a product advertisement ultimately purchased the product. In these types of situations, the entities may wish to maintain the confidentiality of some or all of the data they collect and store in their respective databases. For example, the retailer and / or advertiser may wish to maintain the confidentiality of personally identifiable information (PII) such as user names, addresses, email addresses, credit card numbers, etc. As another example, the entities sharing data may wish to maintain the confidentiality of their proprietary data, such as their customer lists.

[0084] Current solutions address this issue through the use of tokenization, where data values ​​stored in corresponding databases are anonymized into tokens in a consistent and deterministic manner before being shared. For example, each entity can use a hashing algorithm to generate a hash value (e.g., a token) that represents each data value stored in the database. Entities share these tokens with each other, which can then be used to compare data and find matching values ​​while maintaining the confidentiality of the underlying data.

[0085] However, tokenization has several disadvantages. One disadvantage is that tokenization does not allow for fuzzy or approximate matches. In some cases, data may not be entered into various databases in a consistent format or correctly. Because tokenization utilizes a deterministic algorithm to generate tokens, it can only identify exact matches and does not account for these variations in format and data entries (e.g., fuzzy matches). Another disadvantage of tokenization is the complexity and computational resources required to generate the tokens. For example, a hashing algorithm is used to generate hash values ​​for a large number of data values ​​stored in corresponding databases. Another disadvantage of tokenization is limited functionality because tokens can only be used to find exact matches between two data sets. Other functions, such as determining the number of data entries that do not match but include a specified string or character set, cannot be performed.

[0086] To mitigate these issues, when the network-based database system 102 receives a query, the constraint system 240 enforces projection constraints on the data values ​​stored in a specified column of a shared dataset. Projection constraints identify that the data in the column may be restricted from being projected (e.g., rendered, read, or output) into the output for the received query, while allowing the specified operations to be performed on the data and provide corresponding output. For example, a projection constraint can indicate the context of the query that triggers the constraint, such as based on the user / role, account, and / or shares associated with the query.

[0087] Entities that share data can define projection constraints that will be attached to various columns of a shared dataset. For example, an entity can define one or more columns to which a projection constraint should be attached, as well as the conditions for triggering the constraint. When the network-based database system 102 receives a query directed to a shared dataset, the constraint system 240 accesses the data required to process the query from the shared database and determines whether to attach a projection constraint to any column of the shared dataset from which data is accessed. If a projection constraint is attached to one of the columns, the constraint system 240 determines whether the projection constraint should be implemented based on the context of the query and generates output accordingly. For example, if a projection constraint should be implemented, the constraint system 240 can generate output that does not include data values ​​stored in the column, but can provide output determined based on the constrained data, such as the number of matches, the number of fuzzy matches, the number of matches that include a specified string, unconstrained data associated with the constrained data, etc.

[0088] Enforcing projection constraints on queries received at the network-based database system 102 allows data to be anonymously shared and used by entities to perform various operations without the need to tokenize the data. Thus, the difficulty and computational resources required for tokenization are eliminated, and additional functions and operations (e.g., fuzzy matching) can be performed while maintaining the confidentiality of the specified data values.

[0089] When processing queries against a shared data set, constraint system 240 may be implemented within network-based database system 102. For example, in some embodiments, constraint system 240 may be implemented within a clean room provided by data clean room system 230.

[0090] As shown, constraint system 240 includes a projection constraint generation component 802 , a query reception component 804 , a data access component 806 , a column identification component 808 , a projection constraint determination component 810 , a query context determination component 812 , an enforcement determination component 814 , and a projection constraint enforcement component 816 .

[0091] The projection constraint generation component 802 enables entities to establish projection constraints (e.g., projection constraint policies) on shared datasets. For example, the projection constraint generation component 802 may provide a user interface or other communication means that allows entities to define projection constraints associated with their data, which is maintained and managed by the network-based database system 102. To define projection constraints, the projection constraint generation component 802 enables users to provide data defining the shared datasets and columns to which the projection constraints should be attached. For example, a user may submit data defining a specific column and / or set of columns in a shared dataset to which a projection constraint should be attached.

[0092] In addition, the projection constraint generation component 802 enables the user to define conditions for triggering projection constraints. This can include defining a specific context and / or multiple contexts that trigger the implementation of the projection constraint. For example, the projection constraint generation component 802 can enable the user to define user, account, and / or share roles that will trigger the projection constraint and / or be able to project the constrained data columns.

[0093] After receiving data defining projection constraints, projection constraint generation component 802 generates a file that is attached to the identified columns. In some embodiments, the file may include a Boolean function based on the conditions provided for the projection constraints. For example, if the projection constraint should be enforced with respect to the query, the Boolean function may provide a true output, and if the projection constraint should not be enforced with respect to the query, the Boolean function may provide a false output. Attaching the file to the column establishes the projection constraints for the data column for subsequent queries.

[0094] The query receiving component 804 receives a query for a shared data set. The query can include data defining the data to be accessed and one or more operations to be performed on the data. The operations can include any type of operation used in connection with the data maintained by the network-based database system 102, such as a join operation, a read operation, etc. The query receiving component 804 can provide the data associated with the query to other components of the constraint system 240, such as the data access component 806 and the query context determination component 812.

[0095] The data access component 806 accesses a data set based on a query received by the query receiving component 804. For example, the data access component can access data from columns and / or sub-columns of a shared data set that are identified by the query and / or required to generate output based on the received query. The data access component 806 can provide the accessed data to other components of the constraint system 240, such as the projection constraint enforcement component 816.

[0096] The column identification component 808, in response to a query, determines the columns associated with the data accessed by the data access component 806. This includes the columns and / or sub-columns from which the data is accessed. The column identification component 808 can provide the data identifying the columns to other components of the constraint system 240, such as the projection constraint determination component 810.

[0097] The projection constraint determination component 810 determines whether a projection constraint (e.g., a projection constraint policy) is attached to any of the columns identified by the column identification component 808. For example, the projection constraint determination component 810 determines whether a file defining a projection constraint is attached to any of the columns and / or sub-columns identified by the column identification component 810. The projection constraint determination component 810 can provide data indicating whether a projection constraint is attached to any of the columns and / or sub-columns identified by the column identification component 810 and / or provide the data indicating whether a projection constraint is attached to any of the columns and / or the file defining the projection constraint to other components of the constraint system 240, such as the implementation determination component 814.

[0098] The query context determination component 812 determines a context associated with the received query. For example, the query context determination component 812 can use data associated with the received query to determine the context, such as by determining the role of the user submitting the query, the account of the network-based database system 102 associated with the submitted query, the data share associated with the query, etc. The query context determination component 812 can provide the data defining the determined query context to other components of the constraint system 240, such as the implementation determination component 814.

[0099] The enforcement determination component 814 determines whether a projection constraint should be enforced with respect to a received query. For example, the enforcement determination component 814 uses data received from the projection constraint determination component 810 and / or a file defining the projection constraint, and the context of the query received from the query context determination component 812 to determine whether the projection constraint should be enforced, the received data indicating whether the projection constraint is attached to any columns.

[0100] If the query constraint is not attached to any column, the implementation determination component 814 determines that the projection constraint should not be implemented with respect to the query. Alternatively, if the projection constraint is attached to one of the columns, the implementation determination component 814 uses the context of the query to determine whether the projection constraint should be implemented. For example, the implementation determination component 814 can use the context of the query to determine whether the conditions defined in the file attached to the column are satisfied to trigger the projection constraint. In some embodiments, the implementation determination component 814 can use the context of the query as an input to a Boolean function defined by the projection constraint to determine whether the projection constraint is triggered. For example, if the Boolean function returns a true value, the implementation determination component 814 determines that the projection constraint should be implemented. Alternatively, if the Boolean function returns a false value, the implementation determination component 814 determines that the projection constraint should not be implemented.

[0101] The enforcement determination component 814 can provide data indicating whether the projected constraint should be enforced to other components of the constraint system 240 , such as the projected constraint enforcement component 816 .

[0102] The projection constraint enforcement component 816 enforces projection constraints with respect to a query. For example, the projection constraint enforcement component 816 can prohibit the output of a query from including data values ​​in any constrained columns of a shared dataset. This can include rejecting a query entirely based on operations included in the query, such as rejecting the query if the query requests a simple output of the value of a constrained column. However, compared to current solutions (e.g., tokenization), the projection constraint enforcement component 816 can allow many other operations to be performed while maintaining the confidentiality of the data values ​​in the constrained columns, thereby enabling additional functionality. For example, the projection constraint enforcement component 816 allows an operation to provide an output indicating the number of data values ​​within a column that match (including fuzzy matches) a specified key value or multiple specified key values ​​in another column. As an example, two tables can be joined on a projection-constrained column using case-insensitive or approximate matching. Tokenization solutions are generally not suitable for these purposes.

[0103] The projection constraint enforcement component 816 may also allow users to filter and perform other operations on data values ​​stored in the projection-constrained columns. For example, if an email address column is projection-constrained, the analyst end user is prevented from enumerating all email addresses, but may be allowed to perform operations on the predicate " ” is true.

[0104] The projection constraint enforcement component 816 can provide the output of the query to the requesting user's client device.

[0105] As previously explained, as referenced above Figure 4-7B As discussed, in some example embodiments, constraint system 240 is integrated into a database cleanroom. A database cleanroom enables two or more end users of a network-based database system 102 to share and collaborate on their sensitive data without directly revealing the data to the other participants. For example, according to some example embodiments, constraint system 240 can implement projection constraints in the cleanroom to perform a database end-user intersection operation (e.g., Company A and Company B want to know which database end users they have in common without revealing their complete database end-user lists to each other).

[0106] As an example, according to some example embodiments, constraint system 240 can implement projection constraints in a cleanroom to perform enrichment operations. For example, a company can implement constraint system 240 to provide enrichment analysis. For example, (1) given a credit card number, provide demographic information about the card owner, or (2) IP geolocation: given an IP address, provide the geolocation most likely associated with that IP. The enrichment provider wants to enable consumers to find key attributes of the value of interest (e.g., IP or credit card number data) (possibly using fuzzy matching) without allowing consumers to download all available key information.

[0107] In some example embodiments, projection constraints are enforced by constraint system 240 when a query is submitted by a user and compiled. The SQL compiler of constraint system 240 analyzes each individual column accessed in the query to determine the column's lineage, e.g., where the data came from. In some example embodiments, the constraint-based approach of constraint system 240 is integrated into a SQL-based system as discussed herein; however, it should be understood that the constraint-based approach of constraint system 240 can be integrated in a similar manner with any query language or query system other than SQL. In this manner, a user submits a query, and constraint system 240 determines the meaning of the query, considers any applicable projection constraints, and ensures that the query complies with the applicable constraints. In some example embodiments, if data comes from a column subject to a projection constraint, constraint system 240 checks whether the constraint should be enforced based on the context (such as the role of the user executing the query). If the constraint is intended to be enforced, constraint system 240 prevents the column or any values ​​directly derived from the column from being included in the query output. In some example embodiments, the use of constrained columns is allowed based on certain preconfigured conditions (eg, in a WHERE clause for a filter condition, a GROUP BY clause for an aggregate query, etc.) and other contexts.

[0108] In some example embodiments, constraint system 240 implements constraints using the local policy framework of network-based database system 102 (e.g., dynamic data masking (column masking) and row access policies). In some example embodiments, constraint system 240 attaches a given projection constraint policy to one or more specific columns, similar to the masking policies of network-based database system 102. In these example embodiments, the projection constraint policy body is evaluated to determine whether and how to restrict access to the column when a given query is received from a consumer end-user.

[0109] Figures 9A-9D Various data sharing scenarios are shown in which projection constraints may be implemented according to some example embodiments. Figure 9AA data sharing scenario 910 is shown in which a provider 902 shares its data with one or more consumers 904. In this type of scenario, shared data 906 (e.g., a shared dataset) is associated with and managed by a single entity (e.g., provider 902) and is shared with one or more other entities (e.g., consumers 904). Therefore, shared data 906 is not a combination of data provided by multiple entities. In this type of scenario, constraint system 240 can be implemented to enforce projection constraints on queries submitted by one or more consumers 904. Provider 902 can implement projection constraints by specifying which data columns consumers 904 cannot project, thereby protecting any sensitive data. For example, provider 902 can establish projection constraints to prohibit each of consumers 904 from projecting data in protected columns, or set projection constraints to change whether data can be projected based on the context of the query (such as which consumer 904 submitted the query).

[0110] Figure 9B A data sharing scenario 920 is shown in which a provider 902 shares data with a consumer 904, and the shared data 906 is combined with the data of the consumer 904. In this type of scenario, the shared data 906 is associated with and managed by a single entity (e.g., the provider 902), and is shared with one or more other entities (e.g., consumers 904), which combine the shared data 906 with their own data. In this type of scenario, the constraint system 240 can be implemented to enforce projection constraints on queries submitted by one or more consumers 904. The provider 902 can implement projection constraints by specifying which data columns the consumer 904 cannot project to protect any sensitive data, while allowing the consumer 904 to perform operations on the shared data 906 based on the consumer's data. For example, the consumer 904 can perform an operation to determine and output the number of matches between the consumer's data and the data in the constrained columns of the shared data 906, but the consumer may be prohibited from projecting the data values ​​of the constrained columns. As shown in FIG. Figure 9A In the example shown, provider 902 can establish projection constraints to prohibit each of consumers 904 from projecting data in protected columns, or set projection constraints to change whether data can be projected based on the context of the query (such as which consumer 904 submitted the query).

[0111] Figure 9CA data sharing scenario 930 is illustrated, in which data shared by multiple providers 902 is combined and shared with a consumer 904. In this type of scenario, shared data 906 is a combination of data associated with and managed by multiple entities (e.g., providers 902), and shared data 906 is shared with one or more other entities (e.g., consumers 904). In this type of scenario, constraint system 240 can be implemented to enforce projection constraints on queries submitted by one or more consumers 904. Each provider 902 can implement projection constraints by specifying which data columns a consumer 904 cannot project, thereby protecting any sensitive data shared by the respective provider 902. In this type of embodiment, queries submitted by a consumer 904 will be evaluated based on the query constraints provided by each provider 902.

[0112] Shared data 906 can be accessed by consumers 904 without being combined with the consumer's data (e.g. Figure 9A ), or the consumer 904 can combine the shared data 906 with the consumer's own data (as shown in Figure 9B Each provider 902 can establish projection constraints to prohibit each of the consumers 904 from projecting data in protected columns, or set projection constraints to change whether data can be projected based on the context of the query (such as which consumer 904 submitted the query).

[0113] Figure 9D A data sharing scenario 940 is shown in which a provider 902 shares data with an internal user 908. In this type of scenario, shared data 906 is data associated with and managed by a single entity (e.g., provider 902), and is shared with one or more other users associated with that entity (e.g., internal users 908). In this type of scenario, constraint system 240 can be implemented to enforce projection constraints on queries submitted by one or more internal users. Provider 902 can implement projection constraints by specifying which columns of data cannot be projected by internal users 908, thereby protecting any sensitive data shared by provider 902. For example, provider 902 can establish projection constraints to prohibit each internal user 908 from projecting data in protected columns, or set projection constraints to change whether data can be projected based on the context of the query (such as the role of the internal user 908 who submitted the query).

[0114] Figure 10 Shown is a flowchart of a method 1000 for implementing projection constraint-based access to a data set on a distributed database, according to some example embodiments.

[0115] At operation 1002, the query receiving component 804 receives a query directed to a shared data set. The query may include data defining the data to be accessed and one or more operations to be performed on the data. The operations may include any type of operation used in connection with the data maintained by the network-based database system 102, such as a join operation, a read operation, etc. The query receiving component 804 may provide the data associated with the query to other components of the constraint system 240, such as the data access component 806 and the query context determination component 812.

[0116] At operation 1004, the data access component 806 accesses a dataset from the shared dataset to perform the query operation. For example, the data access component can access data from columns and / or sub-columns of the shared dataset that are identified by the query and / or required to generate output based on the received query. The data access component 806 can provide the accessed data to other components of the constraint system 240, such as the projection constraint enforcement component 816.

[0117] At operation 1006, the column identification component 808 determines the columns of the shared dataset from which the dataset is accessed. This includes the columns and / or sub-columns from which the data is accessed. The column identification component 808 can provide the data identifying the columns to other components of the constraint system 240, such as the projection constraint determination component 810.

[0118] At operation 1008, the projection constraint determination component 810 determines whether a projection constraint policy is attached to any of the identified columns. For example, the projection constraint determination component 810 determines whether a file defining a projection constraint is attached to any of the columns and / or sub-columns identified by the column identification component 810. The projection constraint determination component 810 can provide data indicating whether a projection constraint is attached to any column and / or a file defining a projection constraint to other components of the constraint system 240, such as the implementation determination component 814.

[0119] At operation 1010, the query context determination component 812 determines the context of the query. For example, the query context determination component 812 can use data associated with the received query to determine the context, such as by determining the role of the user submitting the query, the account of the network-based database system 102 associated with the submitted query, the data share associated with the query, etc. The query context determination component 812 can provide the data defining the determined query context to other components of the constraint system 240, such as the implementation determination component 814.

[0120] At operation 1012, the enforcement determination component 814 determines whether the projection constraint should be enforced with respect to the received query. For example, the enforcement determination component 814 uses data received from the projection constraint determination component 810 and / or a file defining the projection constraint, and the context of the query received from the query context determination component 812 to determine whether the projection constraint should be enforced, the received data indicating whether the projection constraint is attached to any columns.

[0121] If the query constraint is not attached to any column, the implementation determination component 814 determines that the projection constraint should not be implemented with respect to the query. Alternatively, if the projection constraint is attached to one of the columns, the implementation determination component 814 uses the context of the query to determine whether the projection constraint should be implemented. For example, the implementation determination component 814 can use the context of the query to determine whether the conditions defined in the file attached to the column are satisfied to trigger the projection constraint. In some embodiments, the implementation determination component 814 can use the context of the query as an input to a Boolean function defined by the projection constraint to determine whether the projection constraint is triggered. For example, if the Boolean function returns a true value, the implementation determination component 814 determines that the projection constraint should be implemented. Alternatively, if the Boolean function returns a false value, the implementation determination component 814 determines that the projection constraint should not be implemented.

[0122] The enforcement determination component 814 can provide data indicating whether the projected constraint should be enforced to other components of the constraint system 240 , such as the projected constraint enforcement component 816 .

[0123] At operation 1014, the projection constraint enforcement component 816 enforces the projection constraints with respect to the query. For example, the projection constraint enforcement component 816 can prohibit the output of the query from including data values ​​from any constrained columns of the shared dataset. This can include rejecting the query entirely based on the operations included in the query, such as rejecting the query if the query simply requests the output of the value of the constrained column. However, compared to current solutions (e.g., tokenization), the projection constraint enforcement component 816 can allow many other operations to be performed while maintaining the confidentiality of the data values ​​in the constrained columns, thereby enabling additional functionality. For example, the projection constraint enforcement component 816 allows operations to provide an output indicating the number of data values ​​within a column that match (including fuzzy matches) a specified key value or multiple specified key values ​​in another column. As an example, two tables can be joined on a projection-constrained column using case-insensitive or approximate matching. Tokenization solutions are generally not suitable for these purposes.

[0124] The projection constraint enforcement component 816 may also allow users to filter and perform other operations on data values ​​stored in the projection-constrained columns. For example, if an email address column is projection-constrained, the analyst end user is prevented from enumerating all email addresses, but may be allowed to perform operations on the predicate " ” is true.

[0125] The projection constraint enforcement component 816 can provide the output of the query to the requesting user's client device.

[0126] Implementations of the described subject matter may include one or more of the features, alone or in combination, as illustrated below by way of example.

[0127] Example 1 is a method comprising: receiving a first query directed to a shared data set, the first query identifying a first operation; accessing a first data set from the shared data set to perform the first operation, the first data set including data accessed from a first column of the shared data set; determining, by at least one hardware processor, that a projection constraint policy is attached to the first column, the projection constraint policy restricting output of data values ​​stored in the first column; determining, based on the context of the first query, that the projection constraint policy should be implemented with respect to the first query; and generating an output for the first query based on the first data set and the first operation, the output for the first query not including the data values ​​stored in the first column based on determining that the projection constraint policy should be implemented with respect to the first query.

[0128] In Example 2, the subject matter of Example 1 includes, wherein the context of the first query is based on at least one of: a role of a user submitting the first query, an account associated with the first query, and a data share associated with the first query.

[0129] In Example 3, the subject matter of any of Examples 1-2 also includes: receiving a second query pointing to the shared dataset, the second query identifying a second operation; accessing a second dataset from the shared dataset to perform the second operation, the second dataset including data accessed from a first column of the shared dataset; determining that a projection constraint policy is attached to the first column; based on the context of the second query, determining that the projection constraint policy should not be implemented with respect to the second query; and generating an output for the second query based on the second dataset and the second operation, the output for the second query including the data value stored in the first column based on determining that the projection constraint policy should not be implemented with respect to the second query.

[0130] In Example 4, the subject matter of any of Examples 1-3 includes, wherein the shared data set includes a first data set associated with a first entity and a second data set associated with a second entity, the first entity being associated with a first account of a database management system that receives a first query, the query having been received from a computing system associated with the second entity.

[0131] In Example 5, the subject matter of any of Examples 1-4 includes, wherein the second entity is associated with a second account of the database management system, the first account being different than the second account.

[0132] In Example 6, the subject matter of any of Examples 1-5 further includes receiving data defining a first projection constraint policy from a computing system associated with the first entity.

[0133] In Example 7, the subject matter of any of Examples 1-6 further includes receiving data defining a second projection constraint policy from a computing system associated with the second entity.

[0134] In Example 8, the subject matter of any of Examples 1-6 also includes: receiving a second query pointing to a shared dataset, the second query identifying a second operation; accessing a second dataset from the shared dataset to perform the second operation, the second dataset including data accessed from a second column of the shared dataset, the second column being different from the first column; determining that a projection constraint policy is attached to the second column, the projection constraint policy associated with the second column restricting output of data values ​​stored in the second column; determining, based on the context of the second query, whether the projection constraint policy associated with the second column should be implemented relative to the second query; and generating an output for the second query based on the second dataset and the second operation, and the determination of whether the projection constraint policy associated with the second column should be implemented relative to the second query.

[0135] In Example 9, the subject matter of any one of Examples 1-8 includes, wherein the first operation comprises a join operation.

[0136] In Example 10, the subject matter of any of Examples 1-9 includes, wherein the first data output indicates a number of matching values ​​in the first column and the second column of the shared dataset.

[0137] Example 11 is a system comprising: one or more computer processors; and one or more computer-readable media storing instructions that, when executed by the one or more computer processors, cause the system to perform operations comprising: receiving a first query directed to a shared data set, the first query identifying a first operation; accessing a first data set from the shared data set to perform the first operation, the first data set comprising data accessed from a first column of the shared data set; determining that a projection constraint policy is attached to the first column, the projection constraint policy restricting output of data values ​​stored in the first column; determining, based on the context of the first query, that the projection constraint policy should be implemented with respect to the first query; and generating an output for the first query based on the first data set and the first operation, the output for the first query excluding the data values ​​stored in the first column based on determining that the projection constraint policy should be implemented with respect to the first query.

[0138] In Example 12, the subject matter of Example 11 includes, wherein the context of the first query is based on at least one of: a role of a user submitting the first query, an account associated with the first query, and a data share associated with the first query.

[0139] In Example 13, the subject matter of any of Examples 11-12 includes that the operation further includes: receiving a second query directed to the shared dataset, the second query identifying a second operation; accessing a second dataset from the shared dataset to perform the second operation, the second dataset including data accessed from a first column of the shared dataset; determining that a projection constraint policy is attached to the first column; based on the context of the second query, determining that the projection constraint policy should not be implemented with respect to the second query; and generating an output for the second query based on the second dataset and the second operation, the output for the second query including the data value stored in the first column based on determining that the projection constraint policy should not be implemented with respect to the second query.

[0140] In Example 14, the subject matter of any of Examples 11-13 includes, wherein the shared data set includes a first data set associated with a first entity and a second data set associated with a second entity, the first entity being associated with a first account of a database management system that receives a first query, the query having been received from a computing system associated with the second entity.

[0141] In Example 15, the subject matter of any of Examples 11-14 includes, wherein the second entity is associated with a second account of the database management system, the first account being different than the second account.

[0142] In Example 16, the subject matter of any of Examples 11-15 includes that the operations further comprise: receiving data defining a first projection constraint policy from a computing system associated with the first entity.

[0143] In Example 17, the subject matter of any of Examples 11-16 includes that the operations further comprise receiving data defining a second projection constraint policy from a computing system associated with the second entity.

[0144] In Example 18, the subject matter of any of Examples 11-18 includes that the operation further includes: receiving a second query directed to the shared dataset, the second query identifying a second operation; accessing a second dataset from the shared dataset to perform the second operation, the second dataset including data accessed from a second column of the shared dataset, the second column being different from the first column; determining that a projection constraint policy is attached to the second column, the projection constraint policy associated with the second column restricting output of data values ​​stored in the second column; determining, based on the context of the second query, whether the projection constraint policy associated with the second column should be implemented relative to the second query; and generating an output for the second query based on the second dataset and the second operation, and the determination of whether the projection constraint policy associated with the second column should be implemented relative to the second query.

[0145] In Example 19, the subject matter of any of Examples 11-18 includes, wherein the first operation comprises a join operation.

[0146] In Example 20, the subject matter of any of Examples 11-19 includes, wherein the first data output indicates a number of matching values ​​in the first column and the second column of the shared data set.

[0147] Example 21 is a non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors of one or more computing devices, cause the one or more computing devices to perform operations, the operations comprising: receiving a first query directed to a shared data set, the first query identifying a first operation; accessing a first data set from the shared data set to perform the first operation, the first data set including data accessed from a first column of the shared data set; determining that a projection constraint policy is attached to the first column, the projection constraint policy restricting output of data values ​​stored in the first column; determining, based on the context of the first query, that the projection constraint policy should be implemented with respect to the first query; and generating an output for the first query based on the first data set and the first operation, the output for the first query not including the data values ​​stored in the first column based on determining that the projection constraint policy should be implemented with respect to the first query.

[0148] In Example 22, the subject matter of Example 21 includes, wherein the context of the first query is based on at least one of: a role of a user submitting the first query, an account associated with the first query, and data sharing associated with the first query.

[0149] In Example 23, the subject matter of any of Examples 21-22 includes that the operation further includes: receiving a second query directed to the shared dataset, the second query identifying a second operation; accessing a second dataset from the shared dataset to perform the second operation, the second dataset including data accessed from a first column of the shared dataset; determining that a projection constraint policy is attached to the first column; based on the context of the second query, determining that the projection constraint policy should not be implemented with respect to the second query; and generating an output for the second query based on the second dataset and the second operation, the output for the second query including the data value stored in the first column based on determining that the projection constraint policy should not be implemented with respect to the second query.

[0150] In Example 24, the subject matter of any of Examples 21-23 includes, wherein the shared dataset includes a first dataset associated with a first entity and a second dataset associated with a second entity, the first entity being associated with a first account of a database management system that receives a first query, the query having been received from a computing system associated with the second entity.

[0151] In Example 25, the subject matter of any of Examples 21-24 includes, wherein the second entity is associated with a second account of the database management system, the first account being different than the second account.

[0152] In Example 26, the subject matter of any of Examples 21-25 includes that the operations further comprise: receiving data defining a first projection constraint policy from a computing system associated with the first entity.

[0153] In Example 27, the subject matter of any of Examples 21-26 includes that the operations further comprise receiving data defining a second projection constraint policy from a computing system associated with the second entity.

[0154] In Example 28, the subject matter of any of Examples 21-27 includes that the operation further includes: receiving a second query directed to the shared dataset, the second query identifying a second operation; accessing a second dataset from the shared dataset to perform the second operation, the second dataset including data accessed from a second column of the shared dataset, the second column being different from the first column; determining that a projection constraint policy is attached to the second column, the projection constraint policy associated with the second column restricting output of data values ​​stored in the second column; determining, based on the context of the second query, whether the projection constraint policy associated with the second column should be implemented with respect to the second query; and generating an output for the second query based on the second dataset and the second operation, and the determination of whether the projection constraint policy associated with the second column should be implemented with respect to the second query.

[0155] In Example 29, the subject matter of any one of Examples 21-28 includes, wherein the first operation comprises a join operation.

[0156] In Example 30, the subject matter of any of Examples 21-29 includes, wherein the first data output indicates a number of matching values ​​in the first column and the second column of the shared dataset.

[0157] Figure 11 A diagrammatic representation of a machine 1100 in the form of a computer system within which a set of instructions may be executed for causing the machine 1100 to perform any one or more of the methodologies discussed herein is shown, according to an example embodiment. Figure 11 A diagrammatic representation of a machine 1100 is shown in the form of an example computer system within which instructions 1116 (e.g., software, programs, applications, applet, apps, or other executable code) for causing the machine 1100 to perform any one or more of the methodologies discussed herein may be executed. For example, the instructions 1116 may cause the machine 1100 to execute one or more processors described herein. Figure 10 As another example, the instructions 1116 may cause the machine 1100 to implement portions of the data flows described herein. In this manner, the instructions 1116 convert a general-purpose, non-programmed machine into a specific machine 1100 (e.g., Figure 1 Client device 114, Figure 1Computing service manager 108, Figure 1 Execution platform 110), this particular machine 1100 is specifically configured to perform any of the functions described and illustrated in the manner described herein.

[0158] In alternative embodiments, the machine 1100 operates as a standalone device or can be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1100 can operate as a server or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1100 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a smartphone, a mobile device, a network router, a network switch, a network bridge, or any other machine capable of executing instructions 1116, sequentially or otherwise, to specify actions to be taken by the machine 1100. Further, while a single machine 1100 is illustrated, the term "machine" shall also be taken to include any collection of machines 1100 that individually or jointly execute instructions 1116 to perform any one or more of the methodologies discussed herein.

[0159] The machine 1100 includes a processor 1110, a memory 1130, and input / output (I / O) components 1150 that are configured to communicate with each other, for example, via a bus 1102. In an example embodiment, the processor 1110 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 1112 and a processor 1114 that may execute instructions 1116. The term "processor" is intended to include a multi-core processor 1110 that may include two or more independent processors (sometimes referred to as "cores") that may execute instructions 1116 concurrently. Although Figure 11 Multiple processors 1110 are shown, but the machine 1100 may include a single processor with a single core, a single processor with multiple cores (eg, a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0160] The memory 1130 may include a main memory 1132, a static memory 1134, and a storage unit 1131, all of which may be accessed by the processor 1110, such as via the bus 1102. The main memory 1132, the static memory 1134, and the storage unit 1131 include a machine storage medium 1138 that may store instructions 1116 embodying any one or more of the methodologies or functions described herein. The instructions 1116, during execution by the machine 1100, may also reside, in whole or in part, within the main memory 1132, the static memory 1134, the storage unit 1131, within at least one of the processors 1110 (e.g., within a cache memory of the processor), or any suitable combination thereof.

[0161] The I / O components 1150 include components for receiving input, providing output, generating output, transmitting information, exchanging information, capturing measurements, and the like. The specific I / O components 1150 included in a particular machine 1100 will depend on the type of machine. For example, a portable machine such as a mobile phone will likely include a touch input device or other such input mechanism, while a headless server machine will be less likely to include such a touch input device. It will be appreciated that the I / O components 1150 may include Figure 11 Many other components are not shown in the figure. The I / O components 1150 are grouped by function solely to simplify the following discussion and are by no means limiting. In various exemplary embodiments, the I / O components 1150 may include output components 1152 and input components 1154. Output components 1152 may include visual components (e.g., displays such as plasma display panels (PDPs), light-emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tubes (CRTs)), acoustic components (e.g., speakers), other signal generators, and the like. Input components 1154 may include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, an optical keyboard, or other alphanumeric input components), pointing-based input components (e.g., a mouse, trackpad, trackball, joystick, motion sensor, or another pointing instrument), tactile input components (e.g., physical buttons, a touch screen or other tactile input components that provide location and / or force of a touch or touch gesture), audio input components (e.g., a microphone), and the like.

[0162] Communication can be achieved using a variety of technologies. I / O components 1150 may include a communication component 1164 operable to couple machine 1100 to network 1181 via coupler 1183 or to device 1180 via coupling device 1182. For example, communication component 1164 may include a network interface component or another suitable device that interfaces with network 1181. In other examples, communication component 1164 may include a wired communication component, a wireless communication component, a cellular communication component, or other communication components that provide communication via other modalities. Device 1180 may be another machine or any of a variety of peripheral devices (e.g., a peripheral device coupled via a universal serial bus (USB)). For example, as described above, machine 1100 may correspond to any of client device 114, computing service manager 108, and execution platform 110, and machine 1100 may include any of these systems and devices.

[0163] Various memories (e.g., 1130, 1132, 1134 and / or the memory and / or storage unit 1131 of the processor 1110) may store one or more sets of instructions 1116 and data structures (e.g., software) that embody or are utilized by any one or more of the methods or functions described herein. When executed by the processor 1110, these instructions 1116 cause various operations to implement the disclosed embodiments.

[0164] As used herein, the terms "machine storage medium," "device storage medium," and "computer storage medium" have the same meaning and may be used interchangeably in this disclosure. These terms refer to one or more storage devices and / or media (e.g., centralized or distributed databases and / or associated caches and servers) that store executable instructions and / or data. Accordingly, these terms should be considered to include, but are not limited to, solid-state memory and optical and magnetic media, including memory internal or external to a processor. Specific examples of machine storage medium, computer storage medium, and / or device storage medium include non-volatile memory, including, for example, semiconductor memory devices (e.g., erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), field programmable gate arrays (FPGAs), and flash memory devices); magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms "machine storage medium," "computer storage medium," and "device storage medium" specifically exclude carrier waves, modulated data signals, and other such media (at least some of which are included in the term "signal media," discussed below).

[0165] In various example embodiments, one or more portions of network 1181 may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a portion of the Internet, a portion of a public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, a Wi-Fi® network, another type of network, or a combination of two or more such networks. For example, network 1181 or a portion of network 1181 may include a wireless or cellular network, and coupling 1182 may be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or another type of cellular or wireless coupling. In this example, the coupling device 1182 can implement any of a variety of types of data transmission technologies, such as single-carrier radio transmission technology (1xRTT), Evolution-Data Optimized (EVDO) technology, General Packet Radio Service (GPRS) technology, Enhanced Data rates for GSM Evolution (EDGE) technology, the Third Generation Partnership Project (3GPP) standards including 3G, fourth-generation wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High Speed ​​Packet Access (HSPA), Worldwide Interoperability for Microwave Access (WiMAX), Long Term Evolution (LTE), other technologies defined by various standards-setting organizations, other long-range protocols, or other data transmission technologies.

[0166] Instructions 1116 can be transmitted or received over network 1181 using a transmission medium via a network interface device (e.g., a network interface component included in communication component 1164) and utilizing any of a number of well-known transmission protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, instructions 1116 can be transmitted or received to device 1180 using a transmission medium via coupling device 1182 (e.g., a peer-to-peer coupling device). The terms "transmission medium" and "signal medium" are synonymous and are used interchangeably in this disclosure. The terms "transmission medium" and "signal medium" should be understood to include any intangible medium capable of storing, encoding, or carrying instructions 1116 for execution by machine 1100, including digital or analog communication signals or other intangible media that facilitate the communication of such software. Accordingly, the terms "transmission medium" and "signal medium" should be understood to include any form of modulated data signal, carrier wave, or the like. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.

[0167] The terms "machine-readable medium," "computer-readable medium," and "device-readable medium" are synonymous and may be used interchangeably in this disclosure. These terms are defined to include both machine storage media and transmission media. Thus, these terms include both storage devices / medium and carrier waves / modulated data signals.

[0168] The various operations of the example methods described herein may be performed, at least in part, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Similarly, the methods described herein may be processor-implemented, at least in part. For example, at least some of the operations of the methods described herein may be performed by one or more processors. The execution of certain operations may be distributed between one or more processors that not only reside in a single machine but are also deployed across multiple machines. In some example embodiments, one or more processors may be located in a single location (e.g., in a home environment, an office environment, or a server farm), while in other embodiments, the processors may be distributed across multiple locations.

[0169] Although embodiments of the present disclosure have been described with reference to specific example embodiments, it will be apparent that various modifications and changes may be made to these embodiments without departing from the broader scope of the subject matter of the present invention. Accordingly, the description and drawings are to be regarded as illustrative and not restrictive. The drawings forming a part of this application show, by way of illustration and not limitation, specific embodiments in which the subject matter may be implemented. The illustrated embodiments are described in sufficient detail to enable those skilled in the art to implement the teachings disclosed herein. Other embodiments and embodiments derived therefrom may be used so that structural and logical substitutions and changes may be made without departing from the scope of the present disclosure. Therefore, this detailed description should not be understood in a limiting sense, and the scope of the various embodiments is limited solely by the appended claims, together with the full range of equivalents to which such claims are entitled.

[0170] Such embodiments of the subject matter of the present invention may be referred to herein individually and / or collectively by the term "invention," which is merely for convenience and is not intended to voluntarily limit the scope of this application to any single invention or inventive concept (if more than one invention or inventive concept is actually disclosed). Thus, although specific embodiments have been illustrated and described herein, it should be understood that any arrangement calculated to achieve the same purpose may be substituted for the specific embodiments shown. This disclosure is intended to cover any and all modifications or variations of the various embodiments. Combinations of the above embodiments, as well as other embodiments not specifically described herein, will become apparent to those skilled in the art upon reading the above description.

[0171] In this document, the terms "a" or "an," as is common in patent documents, are used to include one or more than one, independent of any other instance or usage of "at least one" or "one or more." In this document, the term "or" is used to refer to a non-exclusive or, so that "A or B" includes "A but not B," "B but not A," and "A and B," unless otherwise stated. In the appended claims, the terms "including" and "in which" are used as the plain-English equivalents of the respective terms "comprising" and "wherein." Furthermore, in the appended claims, the terms "including" and "comprising" are open-ended; that is, systems, apparatus, articles, or processes that include elements in addition to those listed after such terms in a claim are still considered to fall within the scope of that claim.

Claims

1. A method comprising: receiving a first query from a consumer account directed to a shared data set of a provider account, the first query identifying a first operation and including available values ​​specifying data accessible by the consumer account via the query request; accessing a first data set from the shared data set to perform the first operation, the first data set comprising data accessed from a first column of the shared data set; determining, by at least one hardware processor, that the provider account attaches a projection constraint policy to the first column, the projection constraint policy restricting output of a data value stored in the first column to the consumer account; determining, based on the context of the first query, that the projection constraint strategy should be implemented with respect to the first query; implementing the projection constraint strategy with respect to the first query, the implementing comprising enabling the consumer account to perform the first operation on the data value stored in the first column while maintaining confidentiality of the data value; as well as Based on the first data set and the first operation, an output for the first query is generated, the output for the first query maintaining confidentiality of the data value by not including the data value stored in the first column based on implementing the projection constraint strategy with respect to the first query.

2. The method according to claim 1, wherein The context of the first query is based on at least one of: a role of a user submitting the first query, an account associated with the first query, and a data share associated with the first query.

3. The method according to claim 1, further comprising: receiving a second query directed to the shared data set, the second query identifying a second operation; accessing a second data set from the shared data set to perform the second operation, the second data set comprising data accessed from a first column of the shared data set; Determining that the projection constraint strategy is attached to the first column; determining, based on the context of the second query, that the projection constraint strategy should not be implemented with respect to the second query; as well as An output for the second query is generated based on the second data set and the second operation, the output for the second query comprising the data value stored in the first column based on determining that the projection constraint strategy should not be enforced with respect to the second query.

4. The method according to claim 1, wherein The shared data set includes a first data set associated with a first entity and a second data set associated with a second entity, the first entity being associated with the provider account of the database management system that receives the first query, the first query having been received from a computing system associated with the second entity.

5. The method according to claim 4, wherein The second entity is associated with the consumer account of the database management system, and the first entity is different from the second entity.

6. The method according to claim 4, further comprising: Data defining a first projection constraint policy is received from the computing system associated with the first entity.

7. The method according to claim 6, further comprising: Data defining a second projection constraint policy is received from the computing system associated with the second entity.

8. The method according to claim 1, further comprising: generating a defined access cleanroom in the provider account, the provider account being associated with a provider entity; installing an application instance implementing the defined cleanroom access in the consumer account, the consumer account being associated with a consumer entity; as well as Source provider data is shared by the provider account using the defined access cleanroom, the sharing enabling the source provider data to be accessed by the consumer account via the application instance.

9. The method according to claim 1, wherein The first operation includes a connection operation.

10. The method according to claim 1, wherein The output of the first query indicates a number of matching values ​​in the first column and the second column of the shared data set.

11. A system comprising: one or more hardware processors of the machine; and at least one memory storing instructions that, when executed by the one or more hardware processors, cause the system to perform operations comprising: receiving a first query from a consumer account directed to a shared data set associated with a provider account, the first query identifying a first operation and including available values ​​specifying data accessible by the consumer account via the query request; accessing a first data set from the shared data set to perform the first operation, the first data set comprising data accessed from a first column of the shared data set; determining, by at least one hardware processor, that the provider account attaches a projection constraint policy to the first column by the provider account, the projection constraint policy restricting output of a data value stored in the first column to the consumer account; determining, based on the context of the first query, that the projection constraint strategy should be implemented with respect to the first query; implementing the projection constraint strategy with respect to the first query, the implementing comprising enabling the consumer account to perform the first operation on the data value stored in the first column while maintaining confidentiality of the data value; as well as Based on the first data set and the first operation, an output for the first query is generated, the output for the first query maintaining confidentiality of the data value by not including the data value stored in the first column based on implementing the projection constraint strategy with respect to the first query.

12. The system according to claim 11, wherein The context of the first query is based on at least one of: a role of a user submitting the first query, an account associated with the first query, and a data share associated with the first query.

13. The system of claim 11, the operations further comprising: receiving a second query directed to the shared data set, the second query identifying a second operation; accessing a second data set from the shared data set to perform the second operation, the second data set comprising data accessed from a first column of the shared data set; Determining that the projection constraint strategy is attached to the first column; determining, based on the context of the second query, that the projection constraint strategy should not be implemented with respect to the second query; as well as Based on the second data set and the second operation, an output for the second query is generated, the output for the second query including the data value stored in the first column based on determining that the projection constraint strategy should not be enforced with respect to the second query.

14. The system according to claim 11, wherein: The shared data set includes a first data set associated with a first entity and a second data set associated with a second entity, the first entity being associated with the provider account of the database management system that receives the first query, the first query having been received from a computing system associated with the second entity.

15. The system according to claim 14, wherein: The second entity is associated with the consumer account of the database management system, and the first entity is different from the second entity.

16. The system of claim 14, the operations further comprising: Data defining a first projection constraint policy is received from the computing system associated with the first entity.

17. The system of claim 16, wherein the operations further comprise: Data defining a second projection constraint policy is received from the computing system associated with the second entity.

18. The system of claim 11, the operations further comprising: generating a defined access cleanroom in the provider account, the provider account being associated with a provider entity; installing an application instance implementing the defined cleanroom access in the consumer account, the consumer account being associated with a consumer entity; as well as Source provider data is shared by the provider account using the defined access cleanroom, the sharing enabling the source provider data to be accessed by the consumer account via the application instance.

19. The system according to claim 11, wherein: The first operation includes a connection operation.

20. The system of claim 11, wherein: The output of the first query indicates a number of matching values ​​in the first column and the second column of the shared data set.

21. A machine storage medium storing instructions that, when executed by one or more computer processors of one or more computing devices, cause the one or more computing devices to perform operations comprising: receiving a first query from a consumer account directed to a shared data set associated with a provider account, the first query identifying a first operation and including available values ​​specifying data accessible by the consumer account via the query request; accessing a first data set from the shared data set to perform the first operation, the first data set comprising data accessed from a first column of the shared data set; determining, by at least one hardware processor, that the provider account attaches a projection constraint policy to the first column by the provider account, the projection constraint policy restricting output of a data value stored in the first column to the consumer account; determining, based on the context of the first query, that the projection constraint strategy should be implemented with respect to the first query; implementing the projection constraint strategy with respect to the first query, the implementing comprising enabling the consumer account to perform the first operation on the data value stored in the first column while maintaining confidentiality of the data value; as well as Based on the first data set and the first operation, an output for the first query is generated, the output for the first query maintaining confidentiality of the data value by not including the data value stored in the first column based on implementing the projection constraint strategy with respect to the first query.

22. The machine storage medium of claim 21, wherein: The context of the first query is based on at least one of: a role of a user submitting the first query, an account associated with the first query, and a data share associated with the first query.

23. The machine storage medium of claim 21, the operations further comprising: receiving a second query directed to the shared data set, the second query identifying a second operation; accessing a second data set from the shared data set to perform the second operation, the second data set comprising data accessed from a first column of the shared data set; Determining that the projection constraint strategy is attached to the first column; determining, based on the context of the second query, that the projection constraint strategy should not be implemented with respect to the second query; as well as Based on the second data set and the second operation, an output for the second query is generated, the output for the second query including the data value stored in the first column based on determining that the projection constraint strategy should not be enforced with respect to the second query.

24. The machine storage medium of claim 21, wherein: The shared data set includes a first data set associated with a first entity and a second data set associated with a second entity, the first entity being associated with the provider account of the database management system that receives the first query, the first query having been received from a computing system associated with the second entity.

25. The machine storage medium of claim 24, wherein: The second entity is associated with the consumer account of the database management system, and the first entity is different from the second entity.

26. The machine storage medium of claim 24, the operations further comprising: Data defining a first projection constraint policy is received from the computing system associated with the first entity.

27. The machine storage medium of claim 26, the operations further comprising: Data defining a second projection constraint policy is received from the computing system associated with the second entity.

28. The machine storage medium of claim 21, the operations further comprising: generating a defined access cleanroom in the provider account, the provider account being associated with a provider entity; installing an application instance implementing the defined cleanroom access in the consumer account, the consumer account being associated with a consumer entity; as well as Source provider data is shared by the provider account using the defined access cleanroom, the sharing enabling the source provider data to be accessed by the consumer account via the application instance.

29. The machine storage medium of claim 21, wherein: The first operation includes a connection operation.

30. The machine storage medium of claim 21, wherein: The output of the first query indicates a number of matching values ​​in the first column and the second column of the shared data set.

Images