Defending Against Adversarial Queries in Data Governance Systems

Abstract

An apparatus and associated method for defending against adversarial queries. A policy enforcement hypergraph is constructed to represent a set of security policies. The hypergraph is then repeatedly traversed to determine if a user's behavior has changed over time. The user's behavior is measured by referencing vertices or edges of the hypergraph. If the user's behavior is determined to have changed over time, enforcement actions based on the security policies are taken.

Description

[Technical Field]

[0001] The present disclosure relates to machine learning, and more particularly to robust inference and adversarial machine learning. [Background technology]

[0002] The development of the EDVAC system in 1948 is often considered the beginning of the computer age. Since then, computer systems have evolved into highly complex devices. Today's computer systems typically include a combination of sophisticated hardware and software components, application programs, operating systems, processors, buses, memory, and input / output devices. Advances in semiconductor processing and computer architectures have continuously increased performance, leading to more advanced computer software to take advantage of this increased performance, resulting in computer systems that are far more powerful than those of just a few years ago.

[0003] An application of these functions is the database. A database is generally a computer-based system for storing and retrieving information, and there are many different types. The most common database is the relational database, which generally refers to a tabular database that stores and defines data so that it can be reorganized and accessed in many different ways. A relational database management system (RDBMS) is a computer database management system (DBMS) that uses relational techniques for storing and retrieving data.

[0004] Regardless of the particular architecture of the database, in a DBMS, a requesting entity (e.g., an application or an operating system) requests access to a specified database by issuing a database access request. In a relational database, such requests may include, for example, simple catalog search requests, transactions that operate on reading, modifying, or adding specified records in the database, or a combination thereof. These requests may be made using a high-level query language such as Structured Query Language (SQL). The term "query" generally refers to a series of commands for retrieving data from a stored database. Queries can take the form of SQL commands that allow a program to select, insert, update, locate data, etc. Summary of the Invention

[0005] According to an embodiment of the present disclosure, a method for defending against adversarial queries is provided. The method includes constructing a policy enforcement hypergraph representing a set of security policies. The method further includes repeatedly traversing the hypergraph to determine whether a user's behavior has changed over time. The user's behavior is measured by referencing vertices or edges of the hypergraph. The method further determines that the user's behavior has changed over time based on the traversal of the hypergraph, and in response, takes enforcement action based on the security policies.

[0006] Advantageously, this approach provides adaptive defenses against queries that may unintentionally disclose secure information and reacts to user behavior.

[0007] According to embodiments of the present disclosure, a hypergraph represents assets, users, and security policies as vertices and relationships, and contexts and events as edges. Advantageously, many of the tools developed for analyzing hypergraphs can be applied to the problem of defending against adversarial queries.

[0008] According to an embodiment of the present disclosure, the hypergraph includes vertices and edges, and the method further includes training a machine learning model to determine changes in the vertices and edges over time. Advantageously, this approach allows the model to adapt over time and in response to changing conditions.

[0009] According to embodiments of the present disclosure, the method uses sequence models of time-series data to evaluate the behavior of vertices and edges over time, which advantageously allows the tools available in sequence models to be applied to the problem of defending against adversarial queries.

[0010] According to an embodiment of the present disclosure, the method includes receiving a series of queries to a system, analyzing the series of queries to identify changes in the vertices and edges of each hypergraph over time, generating a suspicion score in response to the analyzing, comparing the suspicion score to a predetermined robustness criterion, and taking enforcement action in response to the comparing. Advantageously, the use of the suspicion score allows for the use of a quantitative metric for the analysis.

[0011] According to an embodiment of the present disclosure, the method further comprises receiving adjustments to the robustness criteria, which advantageously allows the system to adapt based on the received information.

[0012] According to embodiments of the present disclosure, enforcement actions include revoking access rights, which advantageously allows the system to take actions that help protect the system from attacks.

[0013] According to embodiments of the present disclosure, the construction of the policy enforcement hypergraph may utilize one of a graph convolutional neural network and a hypergraph random walk model. Advantageously, the use of a graph convolutional neural network allows the system to learn in response to different forms of threats that the system may encounter.

[0014] According to embodiments of the present disclosure, the method may further include performing one-shot learning of embedding vertices in a graph to estimate similarities between sets of vertices in the hypergraph. Advantageously, this may help models to be developed and / or evolved more quickly.

[0015] According to embodiments of the present disclosure, the method may further include iteratively traversing the hypergraph with a long short-term memory (LSTM) model, which advantageously allows available LSTM tools to be applied to the hypergraph.

[0016] According to embodiments of the present disclosure, the hypergraph includes edges that are used to define which policies apply to users and assets in which contexts, which advantageously allows known hypergraph techniques to be applied to determining which users and assets are involved in a particular asset.

[0017] Embodiments of the present disclosure also include an apparatus including a memory and a processor configured to perform the above-described method steps. Similarly, a computer-readable medium may include instructions that direct the processor to perform the above-described method steps.

[0018] The above summary is not intended to describe each illustrated embodiment or every embodiment of the present disclosure.

[0019] The drawings included in this application are incorporated into and constitute a part of this specification. They illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the disclosure. The drawings are merely illustrative of particular embodiments and are not intended to limit the disclosure. [Brief explanation of the drawings]

[0020] [Figure 1] FIG. 1 is a pictorial block diagram illustrating one embodiment of a data processing system (DPS) consistent with some embodiments. [Figure 2] 1 is a pictorial diagram illustrating a cloud computing environment, consistent with some embodiments. [Figure 3] 1 is a pictorial diagram depicting abstraction model layers, consistent with some embodiments. [Figure 4A] FIG. 1 is a block diagram illustrating an example ML model, consistent with some embodiments. [Figure 4B] 1 is a flowchart illustrating an embodiment of a method for training an ML model, consistent with some embodiments. [Figure 5] FIG. 1 is a block diagram presenting an example policy enforcement graph, consistent with some embodiments. [Figure 6] 1 is a pictorial diagram presenting an example policy enforcement hypergraph, consistent with some embodiments. [Figure 7] 1 is a flowchart illustrating one method of creating an enforcement graph, consistent with some embodiments. [Figure 8] FIG. 1 is a pictorial diagram illustrating an exemplary application of an enforcement hypergraph in a data ecosystem including two data assets and two users, consistent with some embodiments. [Figure 9] 1 is a flowchart illustrating a method for identifying potential privacy violations that may result from inference vulnerabilities in a data ecosystem, consistent with some embodiments. [Figure 10]1 is a flowchart illustrating one method for identifying anomalous or suspicious user behavior in a data ecosystem, consistent with some embodiments. [Figure 11A] 11A-B (collectively FIG. 11) are portions of a flowchart illustrating one method of using an enforcement graph or enforcement hypergraph to identify and be robust against queries that may lead to infringement, consistent with some embodiments. [Figure 11B] 11A-B (collectively FIG. 11) are portions of a flowchart illustrating one method of using an enforcement graph or enforcement hypergraph to identify and be robust against queries that may lead to infringement, consistent with some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0021] While the invention is amenable to various modifications and alternative forms, specific features thereof have been shown by way of example in the drawings and will be described in detail. It is to be understood, however, that it is not intended to limit the invention to the particular embodiments described. On the contrary, it is intended to cover all modifications, equivalents, and alternatives falling within the scope of the invention.

[0022] Aspects of the present disclosure relate to machine learning, and more particularly to robust inference and adversarial machine learning. While the present disclosure is not necessarily limited to such applications, various aspects of the present disclosure can be understood through a discussion of various examples using this context.

[0023] Databases often contain sensitive or other confidential information that requires some level of security to protect it from access, such as medical records. As such, most DBMSs include features to protect the security and privacy of that material. One possible approach to protecting this material is to restrict access to selected users. To this end, DBMSs can implement user profiles that specify levels of privileges. Whether a particular user (or program) can access certain data depends on that user's privilege level, which is usually specified in the respective user profile.

[0024] However, despite being inflexible and static, the aforementioned approaches can be exploited. For example, in a medical database, to ensure the anonymity of patients with sensitive health information in the database, the only results a certain class of users may be allowed to see are the patient's clinic number. Despite these rules, a hostile user (e.g., someone intentionally attempting to circumvent these protections) may be able to determine the patient's identity with a high degree of certainty by issuing a series of carefully crafted queries, possibly in combination with other information known to the user. Such a process is referred to herein as query union analysis. Below is an example series of queries designed to identify a specific individual according to their clinic number (which, in this example, is an identifier that uniquely identifies the individual) and the number of unique patient records each query returns: [Table 1]

[0025] Each of the above queries, considered independently, would return a reasonable number of results to protect individual privacy. However, collectively, the number of results that satisfy all conditions might be significantly smaller, perhaps as few as one. That is, through a series of seemingly safe queries, an adversarial user might be able to learn a particular patient's clinic number. Having determined one individual's clinic number, the adversarial user could then perform other queries, for example, any queries that return the clinic number along with other information, to obtain sensitive information about that particular patient.

[0026] In the future, an adversarial user may be able to use deep learning to automatically generate a series of queries that probe for vulnerabilities in a DBMS that allow access to one or more data assets in a way that the DBMS should not allow. One example scenario would be where an adversary specifies a target (e.g., a set of entries, access rights, etc.) and slightly modifies the parameters of the query to exploit a gap in a rule-based or score-based policy.

[0027] However, rule-based or score-based policies that are strict enough to prevent such and other forms of attacks may prevent legitimate users (e.g., medical researchers) from accessing the wide range of data needed for their projects, potentially limiting the usefulness of the database for legitimate purposes. Thus, one feature and advantage of some embodiments of the present disclosure is their ability to provide a balance between data accessibility and security.

[0028] Thus, one beneficial feature of some embodiments is an enforcement graph and / or hypergraph system that is robust against these attacks. Also, some embodiments may include using generative adversarial models and other deep learning methods to provide richer reasoning related to object similarity, performance and user behavior-based scores, etc. Furthermore, some embodiments may enable near-optimal distribution of data assets in an ecosystem with respect to latency, risk, access frequency, and other metrics.

[0029] <Data Processing System> FIG. 1 illustrates one embodiment of a data processing system (DPS) 100a, 100b (generically referred to herein as DPS 100) consistent with some embodiments. FIG. 1 only depicts representative major components of DPS 100, and these individual components may have greater complexity than depicted in FIG. 1 . In some embodiments, DPS 100 may be implemented as a personal computer, a server computer, a portable computer such as a laptop or notebook computer, a personal digital assistant (PDA), a tablet computer, or a smartphone, a processor integrated into a larger device such as an automobile, an airplane, a teleconferencing system, or a home appliance, a smart device, or any other suitable type of electronic device. Furthermore, components other than or in addition to those illustrated in FIG. 1 may be present, and the number, type, and configuration of such components may vary.

[0030] The data processing system 100 of FIG. 1 may include multiple processing units 110a-110d (generally, processors 110 or CPUs 110), which may be connected by a system bus 122 to a main memory 112, a mass storage interface 114, a terminal / display interface 116, a network interface 118, and an input / output ("I / O") interface 120. The mass storage interface 114 in this embodiment may connect the system bus 122 to one or more mass storage devices, such as a direct access storage device 140, a USB drive 141, or a readable / writable optical disk drive 142, or a combination thereof. The network interface 118 may enable the DPS 100a to communicate with another DPS 100b over a network 106. The main memory 112 may include an operating system 124, multiple application programs 126, and program data 128.

[0031] The embodiment of DPS 100 of FIG. 1 may be a general-purpose computing device. In these embodiments, processor 110 may be any device capable of executing program instructions stored in main memory 112, and may itself be constructed from one or more microprocessors and / or integrated circuits. In some embodiments, DPS 100 may include multiple processors and / or processing cores, as is typical of larger, more powerful computer systems, while in other embodiments, computing system 100 may include only a single processor system, a single processor designed to mimic a multiprocessor system, or both. Furthermore, processor 110 may be implemented using multiple heterogeneous data processing systems 100 in which main processor 110 exists with secondary processors on a single chip. As another illustrative example, processor 110 may be a symmetric multiprocessor system including multiple processors 110 of the same type.

[0032] When DPS 100 boots, the associated processor 110 may initially execute program instructions that comprise an operating system 124. The operating system 124 may manage the physical and logical resources of DPS 100. These resources may include main memory 112, mass storage interface 114, terminal / display interface 116, network interface 118, and system bus 122. Like processor 110, some DPS 100 embodiments may utilize multiple system interfaces 114, 116, 118, 120, and bus 122, which, in turn, may each include its own individual, fully programmed microprocessor.

[0033] The instructions for the operating system 124 and / or application programs 126 (generally, “program code,” “computer-usable program code,” or “computer-readable program code”) may initially be located on a mass storage device in communication with the processor 110 via the system bus 122. The program code in different embodiments may be implemented on different physical or tangible computer-readable media, such as the memory 112 or a mass storage device. In the illustrative example of FIG. 1 , the instructions may be stored in a functional form of persistent storage on the direct access storage device 140. These instructions may then be loaded into the main memory 112 for execution by the processor 110. However, the program code may also be located in a functional form on a computer-readable medium that is selectively removable in some embodiments, such as the direct access storage device 140 or the read / writeable optical disk drive 142. It may be loaded into or transferred to the DPS 100 for execution by the processor 110.

[0034] 1, the system bus 122 may be any device that facilitates communication between the processor 110, the main memory 112, and the interfaces 114, 116, 118, 120. Additionally, although the system bus 122 in this embodiment is a relatively simple single bus structure that provides a direct communication path between the system bus 122, other bus structures are consistent with this disclosure including, but not limited to, point-to-point links, multiple hierarchical buses in a hierarchical, star, or web configuration, parallel paths, redundant paths, etc.

[0035] Main memory 112 and mass storage device 140 may work in concert to store operating system 124, application programs 126, and program data 128. In some embodiments, main memory 112 may be a random-access semiconductor memory device (“RAM”) capable of storing data and program instructions. While FIG. 1 conceptually depicts main memory 112 as a single monolithic entity, main memory 112 in some embodiments may be a more complex arrangement, such as a hierarchy of caches and other memory devices. For example, main memory 112 may exist in multiple levels of caches, and these caches may be further divided by function, such that one cache holds instructions and another cache holds non-instruction data used by processor 110. Main memory 112 may be further distributed and associated with different processors 110 or sets of processors 110, as known in any of a variety of so-called non-uniform memory access (NUMA) computer architectures. Additionally, some embodiments may utilize a virtual addressing mechanism that allows DPS 100 to behave as if it has access to a large single storage entity rather than access to multiple smaller storage entities (such as main memory 112 and mass storage device 140).

[0036] 1 as being contained within main memory 112 of DPS 100a, some or all of them may, in some embodiments, be physically located on a different computer system (e.g., DPS 100b) and accessed remotely, for example, via network 106. Furthermore, operating system 124, application programs 126, and program data 128 are not necessarily all contained entirely within the same physical DPS 100a at the same time, but may even reside in physical or virtual memory of another DPS 100b.

[0037] In some embodiments, the system interfaces 114, 116, 118, and 120 can support communication with a variety of storage and I / O devices. The mass storage interface 114 may support the attachment of one or more mass storage devices 140, which can include rotating magnetic disk drive storage devices, solid-state storage devices (SSDs) that use integrated circuit assemblies as memory to persistently store data, and typically use flash memory or a combination of the two. Additionally, the mass storage devices 140 can include other devices and assemblies including arrays of disk drives (commonly referred to as RAID arrays) and / or archival storage media configured to appear as a single mass storage device to a host, such as hard disk drives, tape (e.g., mini-DV), writable compact discs (e.g., CD-R and CD-RW), digital versatile discs (e.g., DVD, DVD-R, DVD+R, DVD+RW, DVD-RAM), holographic storage systems, Blue Laser Discs, IBM Millipede devices, and the like. I / O interface 120 may support the attachment of one or more I / O devices, such as a keyboard 181, a mouse 182, a modem 183, or a printer (not shown).

[0038] Terminal / display interface 116 may be used to connect one or more displays 180 directly to data processing system 100. These displays 180 may be non-intelligent (i.e., dumb) terminals, such as LED monitors, or may themselves be fully programmable workstations that allow IT administrators and users to communicate with DPS 100. Note, however, that while display interface 116 may be provided to support communication with one or more displays 180, computer system 100 does not necessarily require a display 180, as all necessary interaction with users and other processes may occur over network 106.

[0039] Network 106 may be any suitable network or combination of networks and may support any suitable protocol for communicating data, code, or both to and from multiple DPSs 100. Accordingly, network interface 118 may be any device that facilitates such communication, regardless of whether the network connection is made using current analog or digital technology or both, or via some future network mechanism. Suitable networks 106 include, but are not limited to, networks implemented using one or more of the “InfiniBand” or IEEE (Institute of Electrical and Electronics Engineers) 802.3x “Ethernet” specifications, cellular transmission networks, wireless networks implementing any of the IEEE 802.11x, IEEE 802.16, General Packet Radio Service (“GPRS”), FRS (Family Radio Service), or Bluetooth specifications, ultra-wideband (“UWB”) technology such as that described in FCC 02-48, or the like. Those skilled in the art will appreciate that many different networks and transport protocols may be used to implement network 106. The Transmission Control Protocol / Internet Protocol ("TCP / IP") suite includes suitable network and communication protocols.

[0040] <Cloud Computing> 2 illustrates one embodiment of a cloud environment suitable for an edge-based, scalable, and dynamic transfer learning mechanism. While this disclosure includes detailed descriptions of cloud computing, it is understood that implementation of the teachings described herein is not limited to a cloud computing environment. Rather, embodiments of the present invention may be practiced in conjunction with any other type of computing environment now known or later developed.

[0041] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with the service provider. This cloud model may include at least five characteristics, at least three service models, and at least four implementation models.

[0042] The characteristics are as follows: On-Demand Self-Service: Cloud consumers can unilaterally provision computing capacity, such as server time or network storage, automatically as needed, without the need for human interaction with the service provider. Broad network access: Computing power is available over the network and can be accessed through standard mechanisms, facilitating use by heterogeneous thin or thick client platforms (e.g., cell phones, laptops, PDAs). Resource Pooling: Computing resources from a provider are pooled and offered to multiple consumers using a multi-tenant model. Various physical and virtual resources are dynamically allocated and reallocated based on demand. Consumers generally have no control or knowledge of the exact location of the resources they are provided with, resulting in a sense of location independence. However, consumers may be able to determine location at a higher level of abstraction (e.g., country, state, data center). Rapid Elasticity: Computing capacity can be provisioned quickly and elastically, sometimes automatically, to instantly scale out and quickly release to instantly scale in. To the consumer, the computing capacity available for provisioning often appears unlimited and can be purchased at any time and in any quantity. Metered Services: Cloud systems leverage measurement capabilities at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, active user count) to automatically control and optimize resource usage. Resource usage can be monitored, controlled, and reported to provide transparency to both providers and consumers of utilized services.

[0043] The service model is as follows: Software as a Service (SaaS): The functionality offered to the consumer is the availability of a provider's applications running on a cloud infrastructure. The applications can be accessed from a variety of client devices through a thin client interface such as a web browser (e.g., webmail). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functionality, except for limited user-specific application configuration settings. Platform as a Service (PaaS): The functionality offered to consumers is the deployment onto a cloud infrastructure of applications they create or acquire using programming languages ​​and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but does have control over the deployed applications and, in some cases, the configuration of the hosting environment. Infrastructure as a Service (IaaS): The functionality offered to consumers is the provisioning of processors, storage, networking, and other basic computing resources onto which they can deploy and run any software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but they do have control over the operating systems, storage, and deployed applications, and in some cases partial control over some network components (e.g., host firewalls).

[0044] The deployment model is as follows: Private Cloud: This cloud infrastructure is dedicated to a specific organization and can be managed by that organization or a third party, and can exist on-premise or off-premise. Community Cloud: This cloud infrastructure is shared by multiple organizations to support a specific community with common concerns (e.g., mission, security requirements, policy, and compliance). This cloud infrastructure can be managed by those organizations or a third party and can exist on-premises or off-premises. Public Cloud: This cloud infrastructure is available to the general public or large industry organizations and is owned by an organization that sells cloud services. Hybrid cloud: This cloud infrastructure combines two or more cloud models (private, community, or public), each of which retains its inherent identities but is bound by standards or specific technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0045] A cloud computing environment is a service-oriented environment that emphasizes statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0046] Referring to FIG. 2, an exemplary cloud computing environment 50 is shown. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10, with which local computing devices used by cloud consumers (e.g., a personal digital assistant (PDA) or mobile phone 54A, a desktop computer 54B, a laptop computer 54C, or an automobile computer system 54N, or combinations thereof) can communicate. The nodes 10 can communicate with each other. The nodes 10 can be physically or virtually grouped (not shown) in one or more networks, such as, for example, a private, community, public, or hybrid cloud, or combinations thereof, as described above. This enables the cloud computing environment 50 to provide infrastructure, platform, or software as a service, or combinations thereof, for which cloud consumers are not required to maintain resources on their local computing devices. It should be understood that the types of computing devices 54A-N shown in FIG. 2 are merely exemplary, and that the computing nodes 10 and the cloud computing environment 50 can communicate with any type of electronic device via any type of network or network-addressable connection (e.g., using a web browser), or both.

[0047] Referring to Figure 3, a set of functional abstraction model layers provided by cloud computing environment 50 (Figure 2) is shown. It should be understood in advance that the components, layers, and functions shown in Figure 3 are merely exemplary, and embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0048] Hardware and software layer 60 includes hardware and software components. Examples of hardware components include mainframe 61, reduced instruction set computer (RISC) architecture-based server 62, server 63, blade server 64, storage device 65, and network and network components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0049] The virtualization layer 70 provides an abstraction layer from which the following virtual entities can be provided, for example: virtual servers 71, virtual storage 72, virtual networks including virtual private networks 73, virtual applications and operating systems 74, and virtual clients 75.

[0050] By way of example, the management layer 80 may provide the following functions: Resource provisioning 81 enables dynamic procurement of computing and other resources utilized to execute tasks within the cloud computing environment; Metering and pricing 82 enables cost tracking as resources are utilized within the cloud computing environment and billing or invoicing for the consumption of these resources; By way of example, these resources may include application software licenses; Security enables identification and verification of cloud consumers and tasks, as well as protection for data and other resources; User portal 83 provides consumers and system administrators with access to the cloud computing environment; Service level management 84 enables allocation and management of cloud computing resources so that requested service levels are met; Service level agreement (SLA) planning and fulfillment 85 enables advance arrangement and procurement of anticipated future cloud computing resources required in accordance with SLAs.

[0051] The workload layer 90 provides examples of functionality available to a cloud computing environment. Examples of workloads and functionality that can be provided from this layer include mapping and navigation 91, software development and lifecycle management 92, virtual classroom instruction delivery 93, data analytics processing 94, transaction processing 95, and application software 96.

[0052] <Machine Learning> The machine learning module 96, in some embodiments, may implement a machine learning model ("ML model"). An ML model may be any software system that recognizes patterns. In some embodiments, the ML model comprises multiple artificial neurons interconnected through connection points called synapses or gates. Each synapse encodes the strength of the connection between the output of one neuron and the input of another neuron. The output of each neuron is determined by the sum of the inputs received from other neurons connected to it, and therefore by the outputs of these "upstream" connected neurons and the strength of the connection, as determined by the synaptic weights.

[0053] ML models can be trained to solve specific problems (e.g., speech recognition, query generation, database security, etc.) by adjusting synaptic weights so that specific classes of inputs produce desired outputs. This weight adjustment procedure in these embodiments is known as "training" or "learning." Ideally, these adjustments lead to a pattern of synaptic weights during the learning process that converges toward an optimal solution for a given problem based on some cost function. In some embodiments, artificial neurons may be organized into layers.

[0054] FIG. 4A illustrates an exemplary ML model 400 for deep learning that may be implemented as at least part of application software 96, consistent with some embodiments. ML model 400 includes multiple layers 4051-405n. Each layer includes weights 4051w-405nw and biases 4051b-405nb (only some of which are labeled for clarity). Layer 4051, which receives external data, is the input layer. Layer 405n, which generates the final result, is the output layer. Some embodiments include multiple hidden layers 4052-405n-1 between the input and output layers, with typically hundreds of such hidden layers. Some of the hidden layers 4052-405n-1 may have different sizes, organizations, and purposes than the other hidden layers 4052-405n-1. For example, some of the hidden layers of the ML model may be convolutional layers, while other hidden layers may be fully connected layers, deconvolutional layers, or recurrent layers.

[0055] Referring now to FIG. 4B, one embodiment of an ML model training method 450 is depicted, described with reference to speech recognition as an example and consistent with some embodiments. At operation 452, the system can receive and then load training data. In this example, the input dataset can include a series of example queries sent to a database system. At operation 454, the training data is prepared to reduce sources of bias, typically including deduplication, normalization, and order randomization. At operation 456, a model is selected for training, and initial synaptic weights are initialized (e.g., randomized). Depending on the underlying task, suitable models include, but are not limited to, feedforward techniques (e.g., convolutional neural networks), regulatory feedback-based systems, radial basis function (RBF) techniques, and recurrent neural network-based techniques (e.g., long short-term memory (LSTM)). At operation 458, the selected model is used to predict an output using input data elements, and the prediction is compared to the corresponding target data. Then, in operation 460, the gradient (e.g., the difference between the predicted value and the target value) may be used to update the synaptic weights in some embodiments. This process may be repeated, updating the weights at each iteration, until the training data is exhausted or the model reaches an acceptable level of accuracy and / or precision. In operation 462, the resulting model may optionally be compared to previously unevaluated data to verify and test its performance.

[0056] Aspects of some embodiments of the present disclosure can utilize reinforcement learning and / or evolutionary strategy training techniques instead of the gradient-based systems described above. These techniques may be desirable for use in the generative adversarial models (GANs) described below because the discriminator and generator portions do not need to be trained with sample data. Rather, the ML models in a GAN can learn through trial and error. For example, a series of successful trials may reinforce the process because one of the two ML models in the GAN successfully solves a problem.

[0057] <Enforcement Graph> A graph generally refers to a data structure containing nodes (or vertices) and edges (or lines or arcs) connecting any two nodes in the graph. An enforcement graph generally refers to a graph used to enforce data policies. In some embodiments, an enforcement graph may be used to provide graph traversal-based policy enforcement, similarity estimation between entities, policies, or assets, and vertex correlation.

[0058] 5, an example policy enforcement graph 500 consistent with some embodiments is presented. The example policy enforcement graph 500 may be a graph used to enforce a data policy. Vertex types in the policy enforcement graph 500 may include users / requesters (e.g., User A), assets (e.g., datasets, databases, hardware), such as dataset D and dataset E, policies (e.g., Policy B and Policy C, which have policy attributes 521-523 and policy attributes 524-526, respectively, which may describe what types of systems and / or users are authorized to utilize resources such as dataset D and / or dataset E, governance entities, abstract data types, policies regarding where data can be stored, policies regarding where certain types of data are allowed to be stored, etc.).

[0059] The edge type between vertices may depend on the source vertex (e.g., User A) and the target vertex (e.g., Dataset D). An example edge (depicted as an arrow in FIG. 5) in policy enforcement graph 500 includes a request for an asset, as illustrated by an edge / arrow from User A and Dataset D or Dataset E, or both. That is, the edge may indicate an action in some embodiments.

[0060] Other edges in policy enforcement graph 500 may represent the evaluation of an asset (e.g., dataset D) with respect to policies that describe which users the policies apply to, whether those users are allowed to use the asset, etc. For example, some edges may describe how an asset should be used for a particular entity (system or user or both). Other edges may describe information required in a response (using dynamic logic) to user A, as depicted as response 505.

[0061] For example, edge 501 includes the following information for user A, the resource requester: requested resource: dataset D, policy attributes: policy attribute 521 and policy attribute 523. Edge 502 may include the following information for user A, the resource requester: requested resource: dataset D, policy attributes: policy attribute 522. Edge 503 may include the following information for user A, the resource requester: requested resource: dataset D, policy attributes: policy attribute 525 and policy attribute 526. Edge 504 may include the following information for user A, the resource requester: requested resource: dataset E, policy attributes: policy attribute 524, policy attribute 525, and policy attribute 526.

[0062] In this manner, this example edge 501-504 may describe policy information regarding dataset D and / or dataset E associated with user A. In this manner, edges such as edges 501-504 may describe which users are authorized to access and / or transform particular assets (resources), which assets are requested by particular users, which policies apply to particular users and / or assets, and which rules (i.e., portions of policies) apply to particular assets.

[0063] Affinity vector 506 can describe how well a particular resource conforms to a particular policy attribute. For example, affinity vector 506 may indicate that dataset D conforms 31% (0.31) to policy attribute 522 and dataset E conforms 40% (0.40) to policy attribute 522. Thus, dataset E may be better suited (based on policy B) to respond to a response to user A's data in this example.

[0064] While policy enforcement graph 500 can provide the information necessary to determine whether a particular asset is available to user A based on the associated policy, it can be difficult to utilize that information when determining whether to grant user A access to dataset D or dataset E. That is, because the information shown in FIG. 5 is primarily in a tabular format, it can be difficult to understand the interaction of different datasets with different policies and their attributes, and whether a particular user is allowed access to these different datasets based on the different policies and their attributes.

[0065] Thus, some embodiments may extend the policy enforcement graph 500 into a policy enforcement hypergraph. A data policy enforcement hypergraph generally references combinatorial objects that are more general and expressive than simple graphs by using hyperedges that may express the application of policies to specific resources and for specific requesters. Hypergraphs can provide a hierarchical representation of policies as hypervertices (induced subhypergraphs) that enable efficient knowledge representation. Such hypervertices can be easily modified, and vertices within hyperedges can be collapsed / transformed before querying according to the policy enforcement graph protocol.

[0066] A hyperedge consistent with some embodiments may define a set of vertices that may include X users and Y data assets, indicating that X users have access to Y assets. Some embodiments may enable inference on hyperedges to reason about which users can access which assets and what policies apply. Suitable inference methods may include, but are not limited to, hypergraph Laplacian analysis, random walks, and hypergraph convolution.

[0067] Referring now to FIG. 6 , an exemplary policy enforcement hypergraph 600 is presented, consistent with some embodiments. As shown in FIG. 6 , policy enforcement hypergraph 600 may include User A, Policy B, Policy C, Dataset D, Dataset E, and Similarity Vector 506 found in policy enforcement graph 500 shown in FIG. 5 . Additionally, policy enforcement hypergraph 600 may include Rule 631, Rule 632, Rule 633, Rule 634, Rule 635, and Rule 636, which may be components of Policy Attributes 521, Policy Attribute 522, Policy Attribute 523, Policy Attribute 524, Policy Attribute 525, and Policy Attribute 526, respectively, shown in FIG. 5 . Thus, for example, a policy may describe various characteristics of a particular asset / resource, or a rule may be one such characteristic. Additionally, policy enforcement hypergraph 600 may include Hyperedges I-III, which may provide new capabilities for enforcing policies regarding access to particular resources, such as datasets.

[0068] One beneficial feature of some policy enforcement hypergraph 600 embodiments includes the power of hyperedge I, hyperedge II, and hyperedge III when traversing policy enforcement hypergraph 600 to identify policies for a particular resource (e.g., dataset D) when evaluating the resource as they apply to a particular user / system (e.g., user A). For example, when user A attempts to access a particular resource, rather than walking / traversing through the various vertices and edges shown in policy enforcement graph 500, which may require heavy processing resources to interpret the vertices and edges for the policies they represent, by walking / traversing through the hyperedges (e.g., hyperedge I, hyperedge II, or hyperedge III, or a combination thereof) of policy enforcement hypergraph 600, a data system may be able to quickly and efficiently evaluate the policies associated with user A attempting to access those resources.

[0069] More specifically, in the example of Figure 6, traversing the hyperedges of policy enforcement hypergraph 600 may enable the system to quickly apply a particular policy rule to a particular resource for a particular user. In Figure 6, for purposes of illustration, assume that hyperedge I is defined for user A, policy rule 631, policy rule 633, policy rule 635, policy rule 636, and dataset D. Further, assume that hyperedge II is defined for user A, policy rule 632, policy C, and dataset E. Assume that hyperedge III is defined for dataset D, dataset E, policy rule 632, and affinity vector 606 ([0.31, 0.4]).

[0070] In this way, if a system (e.g., a Deep Neural Network - DNN) is trained to identify the policy to use for User A, it may be able to quickly recognize that Hyperedge I and Hyperedge II are related. More specifically, in this example, if User A is requesting data from Dataset D, Hyperedge I may be the relevant hyperedge because it is the only hyperedge related to User A and Dataset D.

[0071] Thus, one beneficial feature of some policy enforcement hypergraph embodiments is that they may enable a system to perform graph traversal-based, path-based policy enforcement using similarity inference between entities (e.g., users), policies, and assets (e.g., resources such as the datasets shown in FIG. 6 ) by traversing specific hyperedges in the policy enforcement hypergraph 600. Another beneficial feature of some policy enforcement hypergraph embodiments is that they may enable policy application analysis (suggesting changes in scope, adding / removing policies). Thus, in some embodiments, a policy enforcement graph (e.g., a data policy enforcement graph) is represented as a data policy enforcement hypergraph, which is a combinatorial object that is more general and expressive than a simple graph by using a hypergraph to represent the application of policies to specific resources and to specific requesters. This hypergraph provides a hierarchical representation of policies as hypervertices (induced subhypergraphs), enabling efficient knowledge representation. Such hypervertices are easily modified, and vertices within hyperedges can be collapsed / transformed before querying according to the policy enforcement graph protocol.

[0072] In one embodiment, traversal through policy enforcement hypergraph 600 and decisions developed therefrom are performed in a rule-based manner, i.e., rules define which path through policy enforcement hypergraph 600 to take depending on which resource is being requested, the profile of the requesting entity, etc. These rules are then used to guide a user and / or train a neural network to determine whether the requestor should be granted access to the requested resource, as described herein.

[0073] FIG. 7 is a flowchart illustrating one method 700 of creating an enforcement graph, consistent with some embodiments. At operation 705, vertices may be added as assets, users, rules, policies, or other abstract data types. At operation 710, edges and hyperedges may be added as correlations, contexts, or events to these vertices to build an enforcement graph or an enforcement hypergraph, or both. Then, at operation 715, embeddings for the edges / hyperedges and vertices may be trained using techniques discussed in more detail with respect to FIGS. 5-6.

[0074] Next, at operation 720, a sequence ML model, such as a long short-term memory (LSTM), transformer model, etc., may be trained using supervised or semi-supervised learning on the labeled set of embeddings and embedding sequences. An LSTM model may include multiple artificial cells interconnected through connection points called gates. Each cell in some embodiments may include an input gate, an output gate, and a forget gate. Cells may be interconnected such that the output gate of one cell becomes the input gate of another cell.

[0075] Cells in some LSTM embodiments can store their values ​​over arbitrary time intervals, and gates can be subentities that regulate the flow of information into and out of cells. Each gate in an LSTM model may further encode the strength of the connection between the output of one cell and the input of another cell. The output of each cell is determined by the aggregate of inputs received from other cells connected to it through the gate, and thus may be determined by the outputs of these "upstream" connected cells, with the strength of the connection determined by the numerical weights on the gates. Thus, embodiments using LSTM models are considered desirable because they can store values ​​over arbitrary time intervals. This, in turn, may enable events in input time series data to be classified for specific entities. However, other types of ML models are also consistent with this disclosure.

[0076] 7, the output of the selected sequence model may be evaluated at the new vertex or edge / hyperedge for evaluation with respect to that vertex or edge / hyperedge at operation 725. Optionally, an existing access control system may be used to assist in determining the output. In some embodiments, the output / result of the existing access control system may be weighted by a predetermined factor in any decision made by the disclosed hypergraph ML decision model.

[0077] As described in more detail below, in some embodiments, existing access control systems can create / specify access control rules based on inferences generated by an adversarial ML model (e.g., denying access to a user if the suspicion inferred by the adversarial ML model exceeds a predetermined threshold).

[0078] At operation 730, appropriate enforcement action (e.g., revoking certain access rights) may be taken in response to the output of the selected ML model.

[0079] <Inference vulnerability> An inference vulnerability generally refers to a vulnerability where an adversary uses one or more users to access one or more independent data assets and exploits inferences unavailable to a strict subset of the data assets to compromise privacy. Inference vulnerabilities can result in a complete compromise (where the adversary can infer the exact secret information) or a partial compromise (where the adversary can reduce the secret value to a few possible values). One feature and advantage of some embodiments is the ability to identify inference vulnerabilities by assessing similarities and relationships within and between groups of users and groups of data assets. In particular, users may exhibit fraudulent or suspicious access behavior over time, potentially indicating account compromise or fraud.

[0080] 8 depicts an example application 800 of an enforcement hypergraph in a data ecosystem including two data assets and two users, consistent with some embodiments. The data assets in this example are Data Asset A and Data Asset B, which correspond to databases for Hospital A and Hospital B, respectively. Both data assets in this example have been anonymized by decade of birth and zip code, per best practices.

[0081] In this example, the users are User A and User B. However, both are accessed / controlled by an adversarial user. In this illustration, User A requests and receives 12 entries displayed from Data Asset A, and User B requests and receives 12 entries displayed from Data Asset B. An adversary can now take all of this information and use it to infer protected information, for example, if the adversarial user knows that Patient P is under 30, lives in zip code 130**, and has visited both of these hospitals, they can infer that Person C has AIDS.

[0082] While rule-based systems can provide detection of some attacks, suspicious behavior can also be time-based access behavior that is similar to other behaviors over time and can lead to a data breach. For example, users that are part of an inference attack may have similar behaviors and therefore may be correlated by a similarity metric. As described in more detail below, this correlation allows some embodiments to determine that for the two users and data assets in this example, application 800 may form a hyperedge that can be labeled as inference-vulnerable.

[0083] 9 is a flowchart illustrating a method 900 for identifying potential privacy violations that may result from inference vulnerabilities in a data ecosystem comprising an enforcement graph or an enforcement hypergraph, or both, in a manner that leverages topological movement and behavior in the hypergraph over time, consistent with some embodiments. At operation 905, some embodiments may identify, for each user, the data assets to which the user has access. This may be defined using a rule-based system, or computed edges or hypergraphs, or both. For computed edges, some embodiments may use edge inference from a node embedding model, where an "event" may include several groups of users accessing several groups of data assets.

[0084] Next, for each event, an edge and / or hyperedge encompassing the event may be created at operation 910. The new edge and / or hyperedge may be labeled as “inference-vulnerable” or “inference-not vulnerable” at operation 915. A deep learning model (one such model is a “deep hyperedge”) may then be trained to perform supervised training on these labeled edges and / or hyperedges at operation 920. Next, at operation 925, potential inference-based privacy violations may be evaluated by deploying and subsequently running / using the trained ML model on future events or hyperedges in the enforcement hypergraph. In some embodiments, the ML model may output a score associated with the probability of an inference-based privacy violation given the input event.

[0085] One beneficial feature of some embodiments is the ability to generate inferences that exploit topological behavior over time that is present in the early stages of hypergraph learning. In this manner, method 900 may be used to identify and analyze topological behavior and may be used in combination with other existing access control decision systems, rule-based or otherwise.

[0086] <User behavior analysis> 10 is a flowchart illustrating one method 1000 for identifying anomalous or suspicious user behavior in a data ecosystem comprising an enforcement graph and / or hypergraph policy engine, consistent with some embodiments. The method 1000 depicted in FIG. 10 may be desirable because it can leverage topological movement and behavior in the graph and / or hypergraph over time.

[0087] At operation 1005, data assets that each user can access are identified, which in some embodiments may be defined by rule-based edges, computational edges, or hyperedges. For computational edges, edge inference from a node embedding model may be used, and an "event" may be some group of users accessing some group of data assets. Then, at operation 1010, embeddings may be created for each of the users at each time step within the desired range.

[0088] A sequence of embeddings for a user (e.g., n embeddings for n time steps) may be labeled as “suspicious” or “non-suspicious” in operation 1015, which may be repeated for each user in operation 1020. The trained deep learning sequence model may then be used in operation 1025 to perform supervised training on these labeled sequences. Potentially suspicious behavior may be evaluated in operation 1030 by running the trained model on new sequences of embeddings (e.g., for a future set of new users and / or time steps). The resulting suspicion score may be output in operation 1035, which may be related to the probability that the behavior is suspicious or fraudulent.

[0089] At operation 1040, the suspicions may be compared to a predetermined robustness criterion. The predetermined factor may be selected by an administrator using a slider or other graphical user interface element. In response to the comparison, at operation 1050, one or more of the series of queries may be selectively blocked or converted to prevent a data breach. Additionally or alternatively, the series of queries may be sent to an administrator.

[0090] This method 1000 may be desirable because it can identify and analyze topological behavior and can be used in combination with other existing access control decision systems, rule-based or otherwise.

[0091] <Generative Adversarial Network (GAN)> Some embodiments include graph-based generative adversarial models and / or hypergraph GANs, which can query a data ecosystem having an enforcement graph and / or hypergraph. In some embodiments, a user node under the control of an adversarial user, optionally with access to a predicted set of metadata, may be simulated by the generator portion of the GAN. In other embodiments, the generator portion of the GAN may itself issue a series of queries that attempt to attack / exploit the discriminator portion of the GAN. The discriminator portion of the GAN may, in turn, propose actions (e.g., policies) to defend against these types of attacks.

[0092] The generator may then be given a goal. It can then create and issue queries to multiple users and / or a series of users to achieve that goal by making small adjustments to parameters (e.g., the number, IDs, and types of users under its control requesting different assets, the time the assets are requested, other metadata of the requests made by the users, etc.). These queries may be repeated (e.g., producing the same null result) until the query or series of queries is approved by the system, potentially resulting in a data breach.

[0093] The classifier can be completed alongside the generator in adversarial robustness training, where the classifier ML model modifies its policies in response to a successful attack by the generator to prevent that particular set of queries. In some embodiments, this may involve using successful events as adversarial examples as training examples in a robustness "decision maker" model. In this way, a new class of policies (i.e., adversarial inference policies) can be generated that may return confidence / risk scores for potentially affected users upon identifying adversarial behavior. Depending on the value of the score from the adversarial inference policy, the data ecosystem may selectively "reject" outputs in response to some or all of the set of input requests, automatically notify administrators of the potential attack, or automatically perform transformations upon receipt of queries to protect the underlying data, or both.

[0094] More specifically, Figures 11A-B (collectively Figure 11) are portions of a flowchart illustrating one method 1100 of using an enforcement graph or enforcement hypergraph to identify and be robust against queries that may lead to compromise and / or introduce inference vulnerabilities in a data ecosystem, consistent with some embodiments. The queries in Figure 11 may be generated by a GAN generator to "appear" harmless to the data ecosystem, but when combined with external data and / or metadata, may nevertheless introduce compromise. In some embodiments, the GAN classifier may function as a query classifier to protect the data ecosystem.

[0095] In method 1100, data assets accessible to each user may be identified at operation 1105, for example, using a rule-based system, computed edges, or computed hyperedges. For computed edges or hyperedges, edge inference from a node embedding model may be used. At operation 1110, the GAN classifier may begin by initializing an enforcement graph or a hypergraph, or both, as an enforcement service for the data ecosystem.

[0096] Next, at operation 1115, the GAN generator may be initialized and given a target, such as accessing "n" data assets using any number of predicted users and queries. The generator may then be configured to generate a series of queries seeking the target, adjusting query parameters, to generate an adversarial query or sequence of adversarial queries at operation 1120. Operation 1120 may be repeated until the target is obtained at operation 1125. In response, the causal query may be marked as a "successful adversarial query" at operation 1130.

[0097] The classifier may then be instructed to implement a policy to defend against successful adversarial queries at operation 1135. This may include training a deep learning-based "decision maker" model for this example, if applicable. This method 1100 may be continually iterated to become increasingly robust against adversarial queries.

[0098] Next, action 11 40 In operation 1115, the generator may be given a new goal. The flow may then return to operation 1115. 40 may be repeated for a predetermined number of cycles until a predetermined robustness criterion is met (eg, robust against predicted attacks for a predetermined time), and so on.

[0099] After the classifier training is complete, the classifier may be deployed to a policy engine of the production system in operation 1145. At operation 1150, the policy engine may then begin receiving and analyzing a series of queries using the trained classifier and generating a suspicion score based on the analysis. At operation 1155, the suspicion score may be compared to a predetermined criterion. The predetermined criterion may be selected by an administrator using a slider or other graphical user interface element. In response to the comparison, at operation 1160, one or more of the series of queries may be selectively blocked or converted to prevent a data breach. Additionally or alternatively, the series of queries may be sent to an administrator.

[0100] <Computer program product> The present invention may be a system, method, or computer program product, or combination thereof, integrated at any possible level of technical detail. The computer program product may include a computer-readable storage medium having stored thereon computer-readable program instructions for causing a processor to carry out aspects of the present invention.

[0101] A computer-readable storage medium may be a tangible device capable of retaining and storing instructions for use by an instruction execution device. The computer-readable storage medium may be, by way of example only, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or a suitable combination thereof. More specific examples of computer-readable storage media include portable computer diskettes, hard disks, RAM, ROM, EPROM (or flash memory), SRAM, CD-ROMs, DVDs, memory sticks, floppy disks, mechanically encoded devices having instructions recorded thereon, such as punch cards or ridge-in-groove structures, and suitable combinations thereof. As used herein, a computer-readable storage medium should not be construed as a transitory signal per se, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or an electrical signal transmitted over a wire.

[0102] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof). The network may be comprised of copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface of each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage on a computer-readable storage medium within the respective computing / processing device.

[0103] Computer-readable program instructions for carrying out operations of the present invention may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or object-oriented programming languages ​​such as Smalltalk, C++, etc., and procedural programming languages ​​such as the "C" programming language and similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, as a standalone software package, or partially on the user's computer. Alternatively, the computer may be executed partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) can execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the computer-readable program instructions in order to carry out aspects of the present invention.

[0104] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0105] These computer-readable program instructions can be provided to a processor of a computer or other programmable data processing apparatus to create a machine, such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions can also be stored in a computer-readable storage medium connectable to a computer, programmable data processing apparatus, or other device, or combination thereof, that functions in a particular way, such that the computer-readable storage medium on which the instructions are stored constitutes one of several products including instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0106] Computer-readable program instructions, such as instructions to perform the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams on a computer, other programmable apparatus, or other device, can also be loaded into a computer, other programmable data processing apparatus, or other device to perform a series of operational steps on the computer, other programmable apparatus, or other device to produce a computer-implemented process.

[0107] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of executable implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, which constitute one or more executable instructions for implementing the specified logical function(s). In some alternative embodiments, the functions shown in the blocks may occur out of the order shown in the figures. For example, two blocks shown in succession may actually be accomplished as a single step, executed concurrently, substantially concurrently, partially, or fully in a time-overlapping manner, or the blocks may be executed in the reverse order depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, can be implemented by a special-purpose hardware-based system that performs the specified functions or operations or executes a combination of special-purpose hardware and computer instructions.

[0108] <General> The description of various embodiments of the present disclosure is presented for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. It will be apparent to those skilled in the art that many modifications and variations are possible without departing from the scope of the described embodiments. The terms used herein have been selected to best explain the principles of the embodiments, practical applications or technical improvements to technology found in the market, or to enable those skilled in the art to understand the embodiments described herein.

[0109] The presently described embodiments are therefore to be considered in all respects as illustrative and not restrictive, and reference should be made to the appended claims to determine the scope of the invention.

Claims

1. A method for defending against adversarial queries, comprising: constructing a policy enforcement hypergraph that represents a set of security policies; repeatedly traversing the hypergraph to determine if a user's behavior is changing over time, the user's behavior being measured by referencing vertices or edges of the hypergraph; determining that the user's behavior has changed over time based on the traversal of the hypergraph, and in response, taking enforcement action based on a security policy; A method comprising:

2. The hypergraph is Assets, users, and security policies are the vertices and relationships between them. Context and Events as Edges The method of claim 1 ,

3. the hypergraph includes vertices and edges; The method further includes training a machine learning model to determine changes in the vertices and edges over time. The method of claim 2.

4. The method of claim 3 , further comprising using a sequence model of time-series data to evaluate the behavior of the vertices and edges over time.

5. receiving a series of queries to the system; analyzing the set of queries to identify changes in the vertices and edges of each hypergraph over time; generating a suspicion score in response to said analyzing; comparing said suspicion score with a predetermined robustness criterion; taking said enforcement action in response to said comparing; The method of claim 4 further comprising:

6. The method of claim 5 , further comprising receiving an adjustment to the robustness criterion.

7. The method of claim 1 , wherein the enforcement action comprises revoking access rights.

8. The method of claim 1 , wherein the construction of the policy enforcement hypergraph utilizes a graph convolutional neural network.

9. The method of claim 1 , wherein the construction of the policy enforcement hypergraph utilizes a hypergraph random walk model.

10. The method of claim 1 , further comprising performing one-shot learning of embedding vertices in a graph to estimate similarities between a set of vertices in the hypergraph.

11. The method of claim 1 , wherein the hypergraph is iteratively traversed by a long short-term memory (LSTM) model.

12. the hypergraph includes edges; The edges are used to define which policies apply to users and assets in which contexts. The method of claim 1.

13. Memory and A processor configured to perform operations according to the method of any of claims 1 to 12; A hostile query defense device, including:

14. 1. A computer program for a hostile query defense device, the computer program comprising: comprising program instructions for performing operations according to the method of any one of claims 1 to 12, Computer program.

15. 1. A method for defending a system against hostile queries, comprising: providing an enforcement graph; using the enforcement graph to enforce data policies of the system; using a generative adversarial model (GAN) for queries against the enforcement graph to detect potential adversarial query-based attacks against the enforcement graph; providing a policy for protecting the enforcement graph from the potential adversarial attacks; A method comprising:

16. vertices in the enforcement graph are assigned to represent first elements selected from the group consisting of users, assets, policies, and data types; an edge in the enforcement graph is assigned to represent a second element selected from the group consisting of a user request, a user asset, and a user policy for the first element; 16. The method of claim 15.

17. analyzing the enforcement graph to determine system resources to which each user has access; using the GAN to simulate a hostile user attempting to access a set of system resources, the hostile user attempting to exploit an inference vulnerability; 16. The method of claim 15, further comprising:

18. determining that the adversarial query is successful, and in response, training a policy engine using the enforcement graph with the successful adversarial query.

16. The method of claim 15, further comprising:

19. The method of claim 18 , further comprising repeating the determining and training operations until the enforcement graph reaches a predetermined robustness score.

20. 20. The method of claim 19, further comprising deploying the policy engine in response to the enforcement graph satisfying the predetermined robustness score.

21. receiving a series of queries to the system; evaluating the set of queries using the policy engine and, in response, generating a suspicion score; comparing said suspicion score with a predetermined robustness criterion; selectively blocking one or more queries in the set of queries in response to said comparing; 21. The method of claim 20, further comprising:

22. The method of claim 21 , further comprising adjusting the robustness criteria in the policy engine.

23. The method of claim 15 , wherein the enforcement graph is an enforcement hypergraph and the GAN is a hypergraph GAN.

24. Memory and A processor configured to perform the operations according to the method of any of claims 15 to 23; The hostile query defense device includes:

25. 1. A computer program for a hostile query defense device, the computer program comprising: comprising program instructions for performing operations according to the method of any of claims 15 to 23, Computer program.

Images