Enforcing access controls for generative machine learning models
The SGML model with access control mechanisms and domain-restricted adapters addresses the challenge of unauthorized data access in generative models, ensuring secure and flexible access control by measuring enforcement effectiveness.
Patent Information
- Application Number
- PCT/US2026/028022
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2026-03-16
- Filing Date
- 2026-05-13
- Publication Date
- 2026-11-19
AI Technical Summary
Existing access control frameworks for generative machine learning models are unreliable in ensuring that only authorized users can access specific data silos, leading to potential leaks of sensitive information, and existing techniques lack flexibility and effectiveness in enforcing access controls.
Implementing a secure generative machine learning (SGML) model with access control mechanisms that restrict data access based on security domains, using adapters to ensure responses are relevant to authorized domains, and employing metrics like access advantage to measure enforcement effectiveness.
The SGML model effectively enforces access controls, ensuring that responses are relevant to authorized domains, thereby preventing unauthorized access and maintaining data security while allowing flexible access to different users based on their credentials.
Smart Images

Figure US2026028022_19112026_PF_FP_ABST
Abstract
Description
ENFORCING ACCESS CONTROLS FOR GENERATIVE MACHINE LEARNING MODELSBACKGROUND
[0001] Machine learning models provide important decision making features for various applications across a wide variety of fields. Given their ubquity, greater importance has been placed on understanding the implications of machine learning model design and training data set choices on machine learning model performance. Systems and techniques that can provide greater adoption of machine learning models can be, therefore, highly desirable. In enterprise settings, orgamizational data can be seggregated, siloed and carefully protected by elaborate access control frameworks. These access control frameworks may not be reliable if a machine learning model (such as an LLM) fine-tuned on datasets pertaining to different domains, serves requests for downstream tasks from individuals who may not have the access privileges for some or all of the inferences. Furthermore, it is important to measure the effectiveness of access control framework in protecting organizational data. Systems and techniques that can provide access control such that the inference responses are secure and measurable by using one or more methods to measure the effectiveness of the access control is, highly desirable.SUMMARY
[0002] Systems, methods, and apparatus to implement enforcing access controls to a generative machine leanring model and methods to measure the effectiveness of enforcing access controls are described herein are described herein. An inference request directed to generative machine learning model may be received. A response to the inference request may be generated by a technique that includes identifying one or more security domains associated with the inference request, identifying one or more adapters for the generative machine learning model that are useable to generate a response that is restricted to the one or more security domains, and using the one or more adapters for the generative machine learning model to generate the response to the inference request. The response may then be returned for the inference request.
[0003] A response to the inference request may be generated by a technique that includes identifying a one or more security domains associated with the inference request, identifying one or more adapters for the generative machine learning model that enforce a content restriction in the response to the one or more security domains, and using the one or more adapters for the generative machine learning model to generate the responses to the inference request. The responses may be to different submissions of a same inference request associated with differentusers. A security enforcement metric for the generative machine learning model may be generated for one or more of the plurality of security domains based, at least in part, on a difference in relevance scores of the respective responses to different submissions of the same inference request associated with the different users.BRIEF DESCRIPTION OF THE DRAWINGS
[0004] FIG. 1 illustrates an example method of enforcing access control, according to some embodiments.
[0005] FIG. 2 illustrates an example method of enforcing access control when one security domain is trained for an adapter, according to some embodiments.
[0006] FIG. 3 illustrates an example method of enforcing access control for a series of security domain trained per adapter that are further merged to have one or more security domains per adapter, according to some embodiments.
[0007] FIG. 4 illustrates an example method of enforcing access control for one or more security domains trained for an adapter, according to some embodiments.
[0008] FIG. 5 illustrates a sample access control table that is used to enforce access control and maps users to one or more security domains, according to various embodiments.
[0009] FIG. 6 illustrates a flow diagram of a sample access control mechanism that is used to build and deploy a secure generative machine learning model, according to various embodiments.
[0010] FIG. 7 illustrates a flowchart that describes a series of steps that implements the secure generative machine learning model, according to various embodiments.
[0011] FIG. 8 illustrates a high-level diagram that provides a framework for the use of SGML model for an access control and audit service, according to some embodiments.
[0012] FIG. 9 illustrates a sample user interface of a measurement and validation service that sets up various parameters to generate an inference request, and receive a response, according to various embodiments.
[0013] FIG. 10A illustrates a sample user interface of a measurement and validation service that sets up various parameters to evaluate and measure domain distinguishability, according to various embodiments.
[0014] FIG. 10B illustrates a sample plot generated to evaluate Domain Distinguishability Index, according to various embodiments.
[0015] FIG. 11A illustrates a sample user interface screen for the measurement and verification service 809 for evaluating utility gap index, according to various embodiments.
[0016] FIG. 1 IB illustrates a sample graph showing the effect of an SGML model trained with adapters for varying number of security domains, according to various embodiments.
[0017] FIG. 12A illustrates a sample user-interface screen that can be used to configure various metric-based parameters of a measurement and validation service 809 for utility comparison, according to various embodiments.
[0018] FIG. 12B illustrates the effect of utility scores during training and testing time of various machine learning models to determine loss gap, according to various embodiments.
[0019] FIG. 13 A illustrates a sample user-interface that can be used by the measurement and verification service to save parameters to train and test various datasets and models, according to various embodiments.
[0020] FIG. 13B illustrates a sample user-interface that can be used to evaluate loss gap for different tuned ML models based on training and testing results, according to various embodiments.
[0021] FIG. 14 illustrates various components that comprise an access control and audit service, according to various embodiments.
[0022] FIG. 15 illustrates a flowchart for generating a security enforcement metric for a generative machine learning model for one or more security domains according to various embodiments.
[0023] FIG. 16 illustrates a sequence diagram that can describe the flow of events between a client and the access control and audit service via a programming interface, according to various embodiments.
[0024] FIG. 17 illustrates a flow-chart that describes the sequence of events processed by an access control and audit service, according to various embodiments. FIG. 18 illustrates a computing system configured to implement the methods and techniques described herein, according to various embodiments.
[0025] While the disclosure is described herein by way of example for several embodiments and illustrative drawings, those skilled in the art will recognize that the disclosure is not limited to embodiments or drawings described. It should be understood that the drawings and detailed description hereto are not intended to limit the disclosure to the particular form disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents and alternatives falling within the spirit and scope as defined by the appended claims. Any headings used herein are for organizational purposes only and are not meant to limit the scope of the description or the claims. As used herein, the word “may” is used in a permissive sense (e.g., meaning having the potential to) rather than the mandatory sense (e.g. meaning must). Similarly, the words “include”, “including”, and “includes” mean including, but not limited to.
[0026] Various units, circuits, or other components may be described as "configured to" perform a task or tasks. In such contexts, "configured to" is a broad recitation of structure generally meaning "having circuitry that" performs the task or tasks during operation. As such, the unit / circuit / component can be configured to perform the task even when the unit / circuit / component is not currently on. In general, the circuitry that forms the structure corresponding to "configured to" may include hardware circuits. Similarly, various units / circuits / components may be described as performing a task or tasks, for convenience in the description. Such descriptions should be interpreted as including the phrase "configured to." Reciting a unit / circuit / component that is configured to perform one or more tasks is expressly intended not to invoke 35 U. S. C. § 112(f) interpretation for that unit / circuit / component.
[0027] This specification includes references to "one embodiment" or "an embodiment." The appearances of the phrases "in one embodiment" or "in an embodiment" do not necessarily refer to the same embodiment, although embodiments that include any combination of the features are generally contemplated, unless expressly disclaimed herein. Particular features, structures, or characteristics may be combined in any suitable manner consistent with this disclosure.DETAILED DESCRIPTION OF EMBODIMENTS
[0028] Large Language Models (LLMs) and / or other generative machine learning models, of which an LLM is one example, offer unprecedented natural language processing capabilities. Accordingly, these generative machine learning models are being adopted in a vast range of applications across the entire computing industry. For example, LLMs are routinely used by individuals, both personally and as a part of an organization, for everyday tasks that include information retrieval, content creation, comprehension, summarization, planning, code generation and debugging, education, classification and recommendation, to name a few. In fact, the day may not be too far off in the future when generative machine learning models become the primary interface to a large swath of computing and information extraction tasks.
[0029] In various embodiments, in enterprise settings, generative machine learning models are used to perform a wide variety of computing tasks using organization-wide data. Using generative machine learning models that have a wide purview over organizational data can bring massive troves of information and utility, including the ability to combine learnings from disparate information silos of the organization to the fingertips of individuals in the organization. However, making all the learnings from organizational data available to any individual who can query the generative machine learning models becomes a critical security challenge. Organizations have access control structures and hierarchies that tightly control information flow to and fromindividuals within them. Information access via generative machine learning models, if not carefully controlled, is likely to break down those access control structures and hierarchies.
[0030] As an example, consider government agencies using LLMs for a multitude of tasks. The data in government agencies can be segregated among multiple “clearance levels” and users can get access to just the data they have access privileges for. Any other agency data can be inaccessible to the users. An LLM trained on this agency -wide data can leak privileged information to unauthorized users, thus breaking the agency’s access control framework that works on the raw data. Another example can be role-based access control: Consider a health clinic setting, where individuals performing different “roles” (doctors, nurses, technicians, administrative staff, patients, etc.) interact with LLMs to perform many tasks. The roles of the users determine what part of the clinic-wide data they should have access to. An LLM trained on the clinic-wide data can be easily tricked into leaking information to unauthorized users.
[0031] Proposals to build system prompts that can instruct a generative machine learning model to control what information can be generated in the output can help curb some leakage of sensitive information to unauthorized users. However, they do not achieve absolute security, and clever jailbreaking prompts can be used to overrule these system prompts. Other techniques implement tagging generative machine learning models inference requests with encrypted access credentials to authenticate users and block unauthorized inference requests. However, such techniques lack the flexibility needed to enable access to disparate learnings from the generative machine learning models for different users based on their access credentials. In various embodiments, access control enforcement for agentic systems and Retrieval Augmented Generation (RAG) systems may be implemented.
[0032] In various embodiments, challenges to access control enforcement for generative machine learning (GML) models may result from a multitude of data silos, when GML models are trained or tuned on data. Access control enforcement may guarantee that users who do not have access to specific data silos cannot retrieve information from those silos by sending carefully crafted inference requests to the GML trained on data from those silos. To satisfy this guarantee, access control enforcement can be a zero-sum game, so probabilistic solutions are not useable because of the possibility of unauthorized access.
[0033] In various embodiments, the notion of an access control mechanism in GMLs may be evaluated in terms of the relevance of responses generated by an GML to inference requests sent by users. In various embodiments, the notion of security domains may be used as a way to describe these techniques. For example, response relevance can be used to prove correctness of access control mechanisms. In various embodiments, metrics called access advantage can be captured toempirically quantify the effectiveness of an access control mechanism on GMLs. As discussed in detail below, the ability to provide access control enforcement, as well as measure its effectiveness, may improve the performance of various systems, services, applications, or devices that implement or otherwise make use of GMLs. As one of ordinary skill in the art may appreciate, the ability to provide and measure effectiveness of an access control mechanism may allow for the use of GMLs in many different scenarios where access control mechanisms are regulatory and / or technical requirement, thus improving the performance of systems, services, application or devices to use of state of the art GMLs.
[0034] For example, in various embodiments, the following performance improvements may be achieved, such as a secure generative machine learning (SGML) model - a generative machine learning model that provides access control security for the data ingested through training / tuning, and even through contexts / prompts. The notions of access control and formal metrics may be quantified so that it can be shown that SGML model mechanisms provide access control that satisfy access control guarantees.
[0035] In at least one embodiment, a security domain may be a collection of data records that require identical credentials for access (e.g. files with the same group in their access control lists). In at least one embodiment, an SGML model can be configured so that a previously trained machine learning model (e.g., a foundational LLM) can be fine-tuned over data from different security domains with an added constraint to satisfy access enforcement guarantees- responses to inference-time queries may be generated from learnings on data coming from just the security domains the user has access to. This added constraint can be enforced via access control mechanisms that govern how the SGML model uses data from different security domains. The added constraint can be further measured and validated using one or more metrics.
[0036] FIG. 1 illustrates a high-level diagram that provides a framework for the use of an SGML model for an access control system 100. The access control system 109 can comprise different security domains. This can be represented mathematically as n different security domains § = U”=1Sj. FIG. 1 shows user A 101 that can be authorized for security domain A and user B 102 that is authorized for security domain B. User A 101 can also be set as an active domain. An active security domain represents the security domain selected for inference while issuing an inference request such as a query or a prompt. In some embodiments, there can be more than be more than one security domain trained per adapter that can represent active domains. The access control mechanism 110 authenticates the user to verify if a user (such as user A or user B) can have access to any of the security domains that can be part of an access control table (such as FIG. 5) for that particular user. For example, FIG. 5 shows that User A is allowed to access security domain Sa.Since User A has access to only one security domain, it can also be selected as the active domain. However, User C has access to security domain Sz and Sx; here either of the security domains (Sz, Sx) can be selected as an active domain. If the user credentials and the active security domain are authenticated based on the access control table, then the inference request can be activated so that the user can be allowed to access the security domain referenced in the inference request.
[0037] After the user credentials and inference request are authenticated, the access control mechanism 110 then activates the inference request based on the active domain that is selected by enabling the appropriate one or more adapters (discussed more in FIG. 2, FIG. 3, FIG.4) so that appropriate model parameters are activated. For example, in FIG. 5 for User A, security domain Sa can be the active domain and parameter Wa can be used to generate inference. As shown in FIG. 1, user A 101 is authorized for security domain A which can be configured as an active domain 103. An inference request 105 sent from user A’s credentials to the ACM 103 results in looking up an access control table (such as the one illustrated in FIG. 5) and further activating the inference request to security domain A portion of the SGML (112, 115) along with the foundational model 116 to generate an inference 117 for user A, that is relayed as a response (119,107) back to user A. This kind of response may be referred to as a relevant response, as the response is relevant to the active domain.
[0038] User B 102 can be authorized for security domain B, but it is a non-active domain, meaning security domain B is not chosen as an active domain for the inference request. If User B sends an inference request for an active domain, the access control mechanism 110 can still authenticate the userB as it can be a valid user; astheuser is registered for a valid security domain B 104. However, since the active domain in FIG.l can be security domain A, the inference request sent from User B will not activate any of the parameters associated with security domain A 113 and hence will SGML model 114 may generate a response that does not contain any information pertaining to security domain A (118,120,108). Such a response can be said to be a non-relevant response 108. In some embodiments, user B can be an attacker who may try to get inference responses for domains they may not be permitted to access. FIG. 1 illustrates a simple example of an access control system 109 that can provide a secure process for generating inferences. In this example, user B may not be able to receive inferences for security domain A because the access control system may be designed so that only user A is authorized to receive inferences pertaining to security domain A based on inference requests made by user A. However, User B can still receive responses pertaining to foundational model 116 related inferences. In some embodiments, foundational models can be pretrained baseline models that can be readily available such as Mistral, Llama models.
[0039] In some embodiments, relevant and non-relevant responses can also be described mathematically. Training data set consisting of data from domains D =(here Ds.can be a data set sampled from data distribution Ds.of domain Sj). The model fDcan be the SGML model tuned using data set D. Parameters W can be the set of / D’s parameters. Model tuning can change values of a subset of W. A security domain s,- (such as security domain A 103 in FIG. 1) can affect a subset of parameters 1 / I4,£ W if data from Ds. can be used to change parameters 1 / 14; (such as 115 in FIG. 1) during model fine-tuning. An access control mechanism M 110 can be defined that dictates the mapping of security domain stto parameters 1 / 1;. FIG. 5 illustrates an example of such a mapping. A SGML model that can be fine-tuned using data set D can be said to be permissioned, denoted as / p*, if it can use the access control mechanism M to map its parameters W to a plurality of security domains from S, where each security domain staffects parameters 1 / 14,— W • Operationally, during fine-tuning, M can specify which set of model parameters 1 / 14; are tuned for a given security domain s,-. Similarly, during inference, M can specify which set of model parameters should be used to answer a inference request based on the user’s access credentials. SGML model can reside in an enclosing system that authenticates users who send inference requests toThe system determines the user it’s access credentials credu and calls authenticate(credu) that takes user credentials credu and maps them to a subset of security domains Suthat u can access. Sucan be maintained by the enclosing system and is never exposed to user u. This process ensures u cannot arbitrarily change Su. Each of user it’s subsequent query Qu is annotated with Suby the enclosing system. M 110 determines the model parameters WSu(such as 115) used to generate a response rSu(such as 117,119, 107) to qu(such as 105), where WSu= U,-65,(IV,-.
[0040] Mathematically, a relevant request can be described in terms of the mathematical terms defined above. Given a SGML modelan inference request qufrom user u, and Suthe security domains user u has access to, r = fo flu) is the response byto query qu. Response r is said to be relevant to Su(e.g., r =uses parameters WSuto generate r. Similarly, for an inference request qufrom user u, and Svthe security domains user u has access to, r =is the response byto query qu, response r is said to be irrelevant to Sv(e.g., rdoes not use parameters Wsto generate r. In some embodiments, the definition of relevant response that can be expressed mathematically does not exclude additional security domains’ affected parameters, as well as pretrained model parameters, from being used in generating the response.We say that an access control mechanism M is correctly enforced on SGML model if every response r generated for every user query quis relevant to Su.
[0041] The above definition of relevant response can provide more value if a proposed access control mechanism M can be algorithmically verified, that may require an empirically quantifiable metric to determine if the implementation (and the algorithm by extension) of M can be verified. This can be important for auditing purposes. In some embodiments, a metric called response relevance score, relv(q), Su), which quantifies the information gained on data in the security domain q belongs to by observing responses to queries generated using model parameters WSuaffected by security domains of Su. The term relvf / ^7(q), Su), is expected to be higher when q E DSU, compared to when q 0 DSu.
[0042] In some embodiments, the domain of response relevance score may be restricted to the real number interval [0, 1], where 1 is the best expected score for relevance. The response relevance score itself can be represented by another empirical metric such as prediction accuracy, or logits for the expected response. However, given that machine learning models in general are generalization engines, in practice, response relevance scores may be configured to be less than 1. This configuration can be effectively addressed by measuring response relevance scores for security domains that the user has access to and comparing it to response relevance scores for security domains that the user does not have access to. This comparative term can be referred to as an access advantage metric.
[0043] Access Advantage can be expressed mathematically as: Given SGML modeltrained over data set D consisting of data from domains § = U =i st, with access control mechanism M, a subset of security domains SU§, ^achieves a-access advantage w.r.t. Suif:where relv() is the response relevance score on the corresponding security domain subset (Suor Sv), 0 is a “difference” operator specific to the access control assessment method (e.g. subtraction), and a is an advantage threshold that lies in the range [0,1],
[0044] The access advantage metric can rely heavily on the assumption thatperforms significantly better on security domains user u has access to compared to security domains u does not have access to. In other words,may have explicit segregation between the different security domains, as dictated by M. The existing approaches to model fine-tuning fail to achieve this goal as the model is traditionally trained on all the security domains without any built-in domain segregation mechanism, e.g., M = 0, which can be similar to not having an access control table illustrated in FIG. 5.
[0045] In various embodiments, access advantage can be a metric for auditors to determine if an access control mechanism can truly achieve the segregation of security domains as intended. Hence it can be in the auditor’s best interest to have a dataset combination that can be closer to the ideal combination, wherein SuCl Sv= 0. Access advantage can diminish when SuCl Sv0.
[0046] In at least some embodiments, retrieval augmented generation (RAG) based LLM deployments explicitly can have effectiveness of access control mechanisms formally or empirically measured. In various embodiments, relevant response and access advantage extends to RAG systems as well, closing that gap in formalism and empirical evaluation of access control protocols.
[0047] For evaluating the access control mechanisms, a classic adversarial game can be considered between the system S enclosing the modeland the auditor A. A can choose security domain access by emulating an end user, send arbitrary inference requests to the SGML model via S and observe the responses. A can replay the game multiple times as a different user to conclude if the access control is correctly implemented.
[0048] Audit Game. The formal game between auditor A and system S is as follows: Stepl: Auditor c / Z chooses security domain set Suand emulates user u. c / Z sends user credentials creduand query q E T)Suto system 5. Step 2: 5 verifies the user credential creduand sends back the model response(q) to c / Z. Step 3: c / Z chooses security domain set Svsuch that SvA Su= (p and emulates user v. c / Z sends user credentials credvand the same query q E T)Suto 5. Step4: 5 verifies the user credential credvand sends back the model responseto c / Z. Step 5: c / Z concludes the access control mechanism is correctly implemented if the access advantage relv( / D(q), Su) © relv( / ^(q), Sv) > a.
[0049] The auditor A can have superuser privileges to choose arbitrary domain access unlike an ordinary user. This can be by design to allow the auditor to evaluate the correctness of the claimed access control while still following the protocol of sending inference requests to the model as a benign user.
[0050] In at least some embodiments, Parameter Efficient Fine-Tuning (PEFT) based methods can be used to obtain model parameter segregation for security domains. In some embodiments, LoRA PEFT adapter may be used, but the algorithms mentioned above can be applied to other types of adapters. In some embodiments, PEFT is a method of fine-tuning LLMs. While fine tuning with PEFT, the foundational model of the LLM can be kept the same, and a few trainable adapter components can be inserted into the LLM. The points of placing these adapter components can be carefully chosen, and the net additional weights that are to be trained can be very small comparedto sum total of all weights used for the large language model. Comparable results can be obtained by adding a few weights pertaining to the adapters and this can avoid retraining the entire large language model.
[0051] The algorithms described herein may steer security domain data to train select adapters. Each security domain can have a unique identifier (domain Id). Access control mechanism can build a map between security domains and adapters such as the access control table discussed above and illustrated in FIG. 5. The access control table can be used to steer all examples from a security domain to the corresponding adapter(s) for training. The access control table can also be used to steer queries to the correct adapters at test time and training time.
[0052] In some embodiments, an algorithm can be designed so that one adapter can be mapped to a Security Domain, and that users have access to at most one security domain. FIG. 2 illustrates the base algorithm (referred to as trained single mapping algorithm) that performs a simple 1-1 mapping between security domains and adapters. It may be assumed that the number of security domains is known beforehand and use that knowledge to instantiate corresponding number of adapters. During training, each minibatch can be sampled from one security domain, and the domain’s Id can be used to select the adapter to train. At test time, a user’s inference request can be annotated with the domain Id the user has access to. The domain Id can be used to activate the adapter for the corresponding security domain.
[0053] In some embodiments, an algorithm can be developed so that one or more adapters that each represent one security domain can be merged so that a single adapter that can be used for one or more security domains (referred to as merged mapping multi-domain algorithm hereafter). Users have access to data from multiple security domains in many application settings, and therefore the one-to-one mapping (such as FIG. 2) may not be practical in many applications. Therefore, merged mapping multi domain (MMMD) algorithm can be developed so that one or more adapters can be merged into a single adapter for one or more security domains. An access control table can have an updated mapping after implementing MMMD algorithm which can be used by the ACM 103 for inference requests. Without using the MMMD algorithm, if the access control table is updated for users to access multiple security domains, queries coming from such users enables all corresponding adapters, whose activations are averaged at test time that tend to disruptively interfere with each other resulting in catastrophic degradation in model performance. Several merging algorithms may be implemented including TIES and DARE, and Singular Value Decomposition (SVD). In at least some embodiments, the combination of security domains that users may have access to may be known beforehand. Thus, after training adapters for individual security domains (as in singular mapping algorithm), the application can be designed to mergeadapters for the combinations of security domains known beforehand. Correspondingly, the access control table is updated with the domain IDs and the merged adapters. These new mappings can be used at test time to activate the correct merged adapters for users having access to multiple security domains. In some embodiments, MMMD algorithm can be more robust to cross-adapter interference than activating a plurality of single mapping algorithms, especially when more than two adapters may be merged.
[0054] In some embodiments, an algorithm can be developed so that one or more security domains can be trained into a single adapter (can be referred to as single adapter multiple domain algorithm). Thus, each combination of security domains that users have access to can be mapped to a unique adapter. In various embodiments, a security domain can occur in numerous combinations of security domains (e.g. in FIG. 3 one adapter can be configured for security domains A and B, and another adapter can be configured for security domains A, B and C).
[0055] In various embodiments, the careful mapping of security domains (or groups of security domains) to the correct adapters and steering of training examples from security domains to the corresponding adapters can ensure precise parameter segregation for security domains. In some embodiments, users cannot tamper with their access credentials at test time further aids the SGML model enclosing system to determine the correct set of security domains corresponding to a query. The inference request steering that happens through the combination of ACM and SGML model that contains an aggregation of adapters trained for different combinations of one or more security domains, guarantees that all responses generated by the SGML model are relevant to the user’s security domains.
[0056] In some embodiments, FIG. 2 illustrates a single mapping algorithm method that maps a security domain to an adapter in a one-on-one mapping. As discussed earlier, single mapping algorithm maps one adapter to a security domain. During the tuning period 203, each adapter weights (205,206) are tuned for their respective security domains (201,202), so that after tuning 212, there is a trained single mapping (TSM) model 213 that receives requests for a particular security domain (207 or 208), and inference is based on a combination of the respective security domain (207 or 208) and the foundation model 209 as shown in FIG.2. In some embodiments, the foundation model 209 can be a baseline large language model. For example, after fine tuning adapter A for security domain A (henceforth security domain can be interchangeably used with domain), an inference request pertaining to the TSM model 213 could activate security domain A portion 207 of the TSM model 213 along with the foundation model 209 to generate an inference 210. Similarly, an inference request pertaining to the TSM model 213 could activate security domain B portion 208 of the TSM model 213 along with the foundation model 209 to generate aninference 211. FIG. 2 also illustrates an after-tuning 232 phase, integrated with an access control system 230 wherein multiple users (user A 231, user B 232) are authenticated to an access control mechanism 229. The access control mechanism comprises an access control table that maps the users with the security domains. The access control table can also have other details pertaining to the TSM model such as adapters used, weights trained which can translate inference requests (IR) to activate a specific domain component of the TSM model. FIG. 5 illustrates a sample access control table. For example, user A 231 can send an inference request B 214 to the access control mechanism 229 which activates an inference request 221 to domain B (222,226) and foundational model 228 of the TSM model 232, wherein the TSM model 232 provides a response A (217,219) back to the user B 232. Similarly, user B 232 can send an inference request B 215 to the access control mechanism 229 which activates an inference request to domain B (223,227) and foundational model 228 of the TSM model 232, wherein the TSM model 232 provides a response B (220,218) back to the user B 232.
[0057] In some embodiments, FIG. 3 illustrates a merged mapping multiple domain (MMMD) algorithm that initially maps a security domain to an adapter that is later merged into a single adapter based on the credentials of one or more user’s access to one or more security domains. As shown in FIG. 3 dataset A 301 can be fine tuned using adapter A 304 and similarly dataset B 302 can be fine tuned using adapter B 305. Once the fine-tuning is satisfied for each of the adapters (A and B), a merge operation is performed to merge the adapters into a single adapter 307 that represents the model parameters for both dataset A 301 and dataset B 302. Prior to merging the adapters A and B, a verification step can be used to ensure that the adapters that are to be merged into a single adapter are based on an access control table that maps a user’s access to one or more security domains. FIG. 5 illustrates a sample access control table. The single adapter 307 can be further tested and after the tuning 308 is complete, can result in a merged mapping multiple domain (MMMD) model 311 as shown in FIG. 3. The MMMD model 311 can be integrated into an access control system 321 that comprises an access control mechanism 315 that maps various users that have access to the one or more security domains. A typical user flow is also illustrated in FIG. 3 wherein user A 313 can initiate an inference requests 312 pertaining to dataset A 301 or dataset B 302 or an inference request that pertains to both dataset A and dataset B. An access control mechanism 315 uses an access control table (such as the one illustrated in FIG. 5) to activate the inference request (IR) for the security domains A, B 312 into an ACM activated IR 316 for security domains A, B. The ACM activated inference requests will enable MMMD model to provide an inference response (320, 317,314) based on the single adapter tuned for security domains A and B 318 as well as the baseline foundational model 319.
[0058] In some embodiments, FIG. 4 illustrates a single adapter multiple domain (SAMD) algorithm that fine-tunes multiple security domains into a single adapter. As shown in FIG. 4 different security domain sets A, B (401,402) or security domains A, B, C (401, 402, 403) are tuned 404 into a single adapter (A, B) 406 or adapter (A, B, C) 407. As shown in FIG. 4 after finetuning 408 is complete, inferences pertaining to security domains A and B are combined with the function model 412 to generate a response 413. Similarly, inferences pertaining to security domains A, B and C are combined with the functional model 412 to generate a response 414. In some embodiments, the combination of adapters that are to be trained can be based on the access control table (as shown in FIG. 5). Based on the access control table the process of tuning 404 the adapters can be decided upfront. The SAMD model can be integrated into an access control system 416 that comprises an access control mechanism 417 that maps users to one or more security domains based on an access control table. A typical user flow is also illustrated in FIG. 4 wherein user A 410 can send inference requests (IR) pertaining to security domains A and B 412 to an ACM 417 that activates the inference request (IR) 418 to domain A, B 419 portion of the SAMD model. As illustrated in FIG. 4 the inference response (424, 426,413) that can be sent back to the user 410 can be a combination of the domain A, B 421 and the functional model 423. In some embodiments, the functional model 423 can be a baseline generative machine model that is readily available such as Llama, Mistral models. Similarly, as illustrated in FIG. 4 an inference request (IR) 414 from user B 411 for domains A, B, C can be sent to an ACM 417 that activates the IR 418 for the pertinent security domains 420 to generate inference (425,427,415) based on domains A, B, C 400, 422 as well as the functional block 423 that can be sent to user B 411.
[0059] In some embodiments, FIG. 5 illustrates an access control table that maps users to one or more security domains. An access control table 500 is illustrated to have a mapping of users to different security domains along with the model parameters. In some embodiments, the access control table 500 can be generated based on an access control policy. In some embodiments, an access control policy can be a higher level document that illustrates various mappings in an organization that can be used to generate granular tables or sub-policies based on the needs of various departments of an organization. Another table such as a group table 550 is illustrated to have a grouping user into various groups that can further be mapped to access control table 500. Another user table 500 can map users to a unique identifier such as a domain ID as illustrated in FIG. 5. Different tables can be used by different departments in an organization. For example, an infrastructure team can generate users with an ID, associate the users with various groups and generate an unique domain ID for each of the users based on the combination of security domains they are allowed to access, and a development team can have more granular table for fine tuningvarious adapters that maps the security domains to different model parameters as shown in table 500. Depending on the complexity of the organization and the requirements, in some embodiments, a relational database can be established mapping various relationships to different tables such as an access control table 500, group table 550 and user table 570. The access control table can also have other mappings such as the type of adapter that is used. As discussed earlier, many PEFT based adapters such as LoRA adapters can be used to fine-tune machine learning models. The access control table can also have mapping associated with the type of algorithm that can be used to fine-tune the models, such as the algorithms discussed in FIG. 2, 3 and 4.
[0060] In some embodiments, FIG. 6 illustrates a diagram that entails various components that can be used to build and deploy a secure generative machine learning (SGML) model. The process can start with a create request 601 that can comprise a list of items needed to build and deploy a SGML model. The list of items can include a baseline generative machine learning model, one or more security domain data sets, an access control policy, the type of adapters that can be used to fine-tune the ML model. The access control policy can be derived from various tables pertaining to access control as illustrated in FIG. 5. The create request can be sent through a programming interface 602. Based on the access control policy, the map generator 603 can generate a map of users who may have access to various security domains. This map can further assist in deciding the development team to map various security domains that are to be fine-tuned with adapters. Different algorithms can be used based on the security domain mapping. If it is a single user to a single domain mapping, the adapters can be fine-tuned for just a specific domain. If multiple domains are used by a user, then adapters can be fine-tuned for multiple domains for the user. The adaptor generator 604 uses the map generated by the map generator 603 to generate a plurality of adapters based on the user, security domain mappings.
[0061] Different algorithms, such as the ones discussed in FIG. 2,3 and 4 can be used to generate different adapter combinations based on the security domains that need to be trained. Once the model parameters are fine-tuned for various adapters, the last step involves building the final model 605 that can include using a training data set to train the model and further verifying the training dataset with a test dataset. Different kinds of datasets can be used based on the application specific use-case. In some embodiments datasets such as WMDP, GPQA, SimpleQA, RCV1 can be used to train and test the models. In some embodiments, different kinds of baseline machine learning models can be used to add the adapters. Different kind of metrics can be used to assess the working of the models. In some embodiments, the loss gap which is the difference between the inference data during testing and training can be measured, across different datasets as well as across different baseline models, and different SGML models.
[0062] In some embodiments, a table can be generated to assess how the loss gap compares across different datasets, different trained models (which is a combination of baseline models and the adapters) and also across different types of algorithms used to train the models. For example, the SGML model discussed herein can be compared with a fully trained baseline machine learning model, wherein all the parameters are retrained. The results can further be verified by an auditor with metrics discussed above. If the results of the SGML model trained with a few adapters (based on the security domains and user mappings) are comparable to a fully retrained baseline model, then the SGML model can be ready to be deployed 607 in a provider network as an endpoint. In some embodiments, a provider network can be a cloud computing service provider that provides various tools and resources needed to deploy the SGML model endpoint. In some embodiments, the deployed SGML endpoint can include compute instances specialized for the SGML model, load balancers, container management services, databases so that the application can scale based on the request / response workloads. After the SGML endpoint is deployed, the application can be ready to serve inference requests at scale, and the SGML endpoint 607 can receive requests at scale and respond with inference responses at scale.
[0063] In some embodiments, FIG. 7 illustrates a flowchart that describes the steps used to build and deploy a secure generative machine learning (SGML) model, similar to the techniques discussed above with regard to FIG. 6. The process can start with 702, receiving a request to create a secure generative machine learning model based at least in part on a foundational generative machine learning model, one or more security domain datasets, an access control policy, wherein the access control policy maps the combination of one or more security domain datasets to the one or more users. Based on the request, 704 use the generative machine learning model and the one or more security domain datasets to train one or more adaptors pertaining to the one or more security domain datasets to generate a trained secure generative machine learning model. After training and testing the SGML model, 706 deploy the trained SGML model so that one or more users can query the trained SGML model to generate an inference that can be sent to the user as a response, based on the access control policy, wherein the trained SGML model is deployed at a cloud provider network.
[0064] FIG. 8 illustrates a high-level diagram that provides a framework for the use of SGML model for an access control and audit service (ACA) 800. The ACA service 800 can comprise different security domains. This can be represented mathematically as n different security domains § = U”=1Sj. FIG. 8 shows user A 801 that is authorized for security domain A and user B 802 that is authorized for security domain B. User A 801 can also be set as an active domain. An active security domain represents one or more security domains selected while issuing an inferencerequest such as a query or a prompt. The access control mechanism (ACM) 803 authenticates the user to verify if a user (such as user A or user B) has access to one or more security domains that is part of an access control table (such as FIG. 5) for that particular user. For example, FIG. 5 shows that user A 801 is allowed to access security domain Sa. User A 801 has access to only one security domain that can also be selected as the active domain, whereas User C in FIG. 5 has access to security domain Szand Sx, and so either of the security domains (Sz, Sx) can be selected as an active domain. The combination of security domains that can be accessed by the user as well as the one or more security domains that can be chosen to be the active domain can be determined by looking up the access control table. The access control table can be considered to be the gateway that activates the inference requests to be routed to appropriate components of the SGML model. Inference requests activated by the ACM 803 can henceforth be called ACM activated inference requests. After the user credentials and inference request are authenticated, the ACM 803 then activates the inference request based on the active domain that is selected by enabling the appropriate one or more adapters so that the appropriate model parameters are activated. The concept of adapters are discussed in detail later. In some embodiments, adaptors can be a method of fine-tuning a generative machine learning model such as an SGML model such that the number of parameters trained can be a small fraction of the total number of parameters used for fully training the SGML model. For example, in FIG. 5, for user A, security domain Sacan be the active domain and parameter Wacan be used to generate inference. During the training phase, an adapter can be trained for parameter Wafor security domain Sa (such as 805 in FIG.8) and integrated with the foundational model (such as 806 in FIG. 8). In some embodiments, foundational models can be baseline LLMs such as Llama, Mistral. For a very simple model, such as the one illustrated in FIG. 8 the combination of an adapter trained for security domain A parameters 805 and foundational model 806 can be collectively called as an SGML model 804. The SGML model 804 can generate an inference 818 for inference request Q 810 pertaining to security domain A from user A 801 which is authenticated by ACM 803 using an access control table as shown in FIG. 5. The ACM 803 activates 815 security domain A parameters 805 to generate inference 818. The inference 818 for user A 801 can be relayed as response 829 / 811 back to the user A 801. This kind of response (such as response 811) is called a relevant response, as the response is relevant to the active domain 825. User B 802 can be authorized for security domain B, but it is a nonactive domain, meaning security domain B is not chosen as an active domain for the inference request. If user B 802 sends the same inference request Q 812 as user A for an active domain, but the difference being that it is a different user, the access control mechanism can still authenticate the user B 802 as it can be a valid user, as well as the user is registered for a valid security domainB; however, since the active domain 825 in FIG.8 is security domain A, the inference request sent from User B 816 may not activate any of the parameters associated with security domain A, and hence will generate a response that does not contain any information pertaining to security domain A. In the example illustrated in FIG. 8, the inference response 819 would be similar to a foundational model 806 response (such as a Llama or Mistral model) because ACM 803 does not activate any adapters for the inference request Q 812. Such a response is said to be a non-relevant response 813 as it does not generate the response desired by user B 802. FIG. 8 illustrates a simple example of an access control system 800 that provides a secure process for generating inferences. In this example, user B 802 will not be able to receive inferences for security domain A because the access control system is designed so that only user A 801 is authorized to receive inferences pertaining to security domain A. However, user B 802 can still receive responses pertaining to pretrained foundational model 806 related inferences. In some embodiments, user B 802 can simulate a user who wants to obtain information pertaining to domains (such as security domain A) that they may not be authorized to obtain, and they can be part of an organization with appropriate credentials (such as access to security domain B). The inferences (818, 819) can be stored and retrieved 821 from a datastore 1407 by various users. As discussed above, metrics can be used to quantify the effectiveness of the access control mechanism. A service, such as a measurement and validation service 809 can be used to quantify the effectiveness of the access control. The metrics used to quantify the effectiveness of access control can be discussed herein with more details later. A user 808 can be provided with complete access and privileges (similar to a root account in many organizations) so that different combinations of requests 823 can be performed using the measurement and validation service 809 to measure and validate the metrics described herein. In some embodiments, this user 808 can be an auditor.
[0065] In some embodiments, relevant and non-relevant responses can also be elaborated mathematically. Training data set consisting of data from domains D =(here Ds.is a data set sampled from data distribution Ds.of domain Sj). The model fDcan be the SGML model tuned using data set D. Parameters W can be the set of / D’s parameters. Model tuning can change values of a subset of W. A security domain s;- (such as security domain A 825 in FIG. 8) can affect a subset of parameters HsU W if data from Ds. can be used to change parameters Ws(such as security domain A parameters 805 in FIG. 8) during model fine-tuning. An access control mechanism M can be defined that dictates the mapping of security domain stto parameters WsSGML model that can be fine-tuned using data set D can be said to be permissioned, denoted as f, if it can use the access control mechanism M to map its parameters W to a plurality of securitydomains from S, where each security domain s;- affects parameters Ws. £ W. FIG. 5 shows an access control table that maps various security domains to their corresponding parameters. Operationally, during fine-tuning, M can specify which set of model parametersare tuned for a given security domain s;-. Similarly, during inference, M can specify which set of model parameters should be used to answer an inference request based on the user’s access credentials. For example, in FIG. 8, inference request Q 810 can be processed at ACM 803 into an ACM activated inference request 814 which specifies security domain A parameters 805 can be used for user A 815 and the foundational model 806 for user B 816. SGML model can reside in an enclosing system that authenticates users who send inference requests to. The system (such as ACM 803 in FIG. 8) determines the user it’s access credentials credu and calls authenticate(credu) that takes user credentials credu and maps them to a subset of security domains Suthat u can access. Sucan be maintained by the enclosing system and is never exposed to user u. This process ensures u cannot arbitrarily change Su. Each of user it’s subsequent query qutocan be annotated with Suby the enclosing system. M determines the model parameters Ws(such as 805 in FIG. 8) used to generate a response rSu(such as 829, 811 for user A 802 in FIG. 8) to qu(such as 810 in FIG. 8), where WSu= UjesuVFj. Mathematically, a relevant request can be described in terms of the mathematical terms defined above. Given a SGML modelan inference requestufrom user u, and Suthe security domains user u has access to, r = fo flu) is the response by to query qu. Response r (such as 811 in FIG. 8) is said to be relevant to Su(e.g., r = rs) if fo uses parameters WSuto generate r. Similarly, for an inference requestufrom user u (such as user B 802 in FIG. 8), and Sv(such as 826 in FIG. 8) the security domains user u has access to, r = fo flu) is the response byto query qu, response r (such as 813 in FIG. 8) is said to be irrelevant to Sv(e.g., r = rSv)does not use parameters WSu(such as 816 in FIG. 8) to generate r. In some embodiments, the definition of relevant response that can be expressed mathematically does not exclude additional security domains’ affected parameters, as well as pretrained model parameters, from being used in generating the response. An access control mechanism M can be said to be correctly enforced on SGML modelif every response r generated for every user query quis relevant to Su.
[0066] The above definition of relevant response can provide more value if a proposed access control mechanism M can be algorithmically verified, that may require an empirically quantifiable metric to determine if the implementation (and the algorithm by extension) of M can be verified. This can be important for auditing purposes. In some embodiments, a metric called response relevance score, relv( / p* (q), Su), which quantifies the information gained on data in the securitydomain q belongs to by observing responses to queries generated using model parameters WSuaffected by security domains of Su. The term relv (q), Su), is expected to be higher when q E DSU, compared to when q 0 DSu.
[0067] In some embodiments, the domain of response relevance score may be restricted to the real number interval [0, 1], where 1 is the best expected score for relevance. The response relevance score itself can be represented by another empirical metric such as prediction accuracy, or logits for the expected response. However, given that ML models in general are generalization engines, in practice, response relevance score can be less than 1. This restriction can be effectively addressed by measuring response relevance score for security domains that the user has access to and comparing it to response relevance score for security domains that the user does not have access to. This comparative term can be called as access advantage.
[0068] Access Advantage can be expressed mathematically as: Given SGML modeltrained over data set D consisting of data from domains § = U”=1st, with access control mechanism M, a subset of security domains Sv<^ §, / ^achieves a-access advantage w.r.t. Suif:EqeT> Su, Sv£; SuriSv=<p [relvq^J. S,,) 0 relv(TDM(9), S )] > awhere relv() is the response relevance score on the corresponding security domain subset (Suor Sv), © is a “difference” operator specific to the access control assessment method (e.g. subtraction), and a is an advantage threshold that lies in the range [0,1],
[0069] The access advantage metric relies heavily on the assumption thatperforms significantly better on security domains user u has access to compared to security domains u does not have access to. In other words,should have explicit segregation between the different security domains, as dictated by M. The existing approaches to model fine-tuning fail to achieve this goal as the model is traditionally trained on all the security domains without any built-in domain segregation mechanism, e.g., M = 0, which can be similar to not having an access control table.
[0070] In various embodiments, access advantage is a metric for auditors to determine if an access control mechanism is truly achieving the segregation of security domains as intended. Hence it is in the auditor’ s best interest to ensure that SuCl Sv= 0. Access advantage can diminish significantly when SuCl Sv0, leading to incorrect conclusions about the efficacy of the access control mechanism. For example, in some embodiments, in FIG. 8 auditor 828 can determine that user profile A 801 and user profile B 802 are such that SuCl Sv= 0. In such a case, access advantage can be closer to 1. However, if auditor 828 can determine that user profile A 801 and user profile B 802 are such that SuCl Sv0. In such a case, access advantage may not be closer to 1, and thescore can depend on how related the two security domains are. Similar security domains can lead to a score closer to 0 and different security domains can lead to a score closer to 1.
[0071] In at least some embodiments, retrieval augmented generation (RAG) based LLM deployments explicitly can have effectiveness of access control mechanisms formally or empirically measured. In various embodiments, relevant response and access advantage extends to RAG systems as well, closing that gap in formalism and empirical evaluation of access control protocols.
[0072] For evaluating the access control mechanisms, a classic adversarial game can be considered between the system S enclosing the modeland the auditor A. A can choose security domain access by emulating an end user (such as user 808 in FIG. 8), send arbitrary queries to the model via S (such as ACA service 800 in FIG. 8) and observe the responses. A can replay the game multiple times as a different user (such as with user A profile 801 or user B 802 profile in FIG. 8) to conclude if the access control can be correctly implemented.
[0073] Audit Game. The formal game between auditor A and system S can be as follows:1. Auditor c / Z chooses security domain set Suand emulates user u. c / Z sends user credentials creduand query q E T)Suto system 5.2. 5 verifies the user credential creduand sends back the model responseto A.3. c / Z chooses security domain set Svsuch that SvA Su= (p and emulates user v. c / Z sends user credentials credvand the same query q E T)Suto 5.4. 5 verifies the user credential credvand sends back the model responseto cA.5. < A concludes the access control mechanism is correctly implemented if the access advantage relv( / ^ (q), Su) © relv( ^f(q), Sv) > a.
[0074] In some embodiments, the auditor A (such as 808 in FIG. 8) can have superuser privileges to choose arbitrary domain access unlike an ordinary user. This is by design to allow the auditor to evaluate the correctness of the claimed access control while still following the protocol of generating inference requests to the SGML model as a benign user.
[0075] In at least some embodiments, Parameter Efficient Fine-Tuning (PEFT) based methods can be used to obtain model parameter segregation for security domains. In some embodiments, LoRA PEFT adapter may be used, but the algorithms mentioned above can be applied to other types of adapters. In some embodiments, PEFT is a method of fine-tuning large language models. While fine tuning with PEFT, the baseline model of the large language model (such as Llama, Mistral)are kept the same, and a few trainable adapter components are inserted into the large language model. The points of placing these adapter components are carefully chosen, and the net additional weights that are to be trained are very small compared to sum total of all weights used for the large language model. Comparable results can be obtained by adding a few weights pertaining to the adapters and this avoids retraining the entire large language model.
[0076] The algorithms described herein steer security domain data to train select adapters. Each security domain has a unique identifier (domain Id). Access control mechanism builds a map between security domains and adapters trained for various parameters for the respective one or more security domains such as the access control table discussed above and illustrated in FIG. 5. The access control table can be used to steer all inference requests from a security domain to the corresponding adapter(s) for training. The access control table can also be used to steer queries to the correct adapters at test time and training time.
[0077] In some embodiments, two novel instantiations of the access advantage metric can be used to quantify access control efficacy independently of any particular system design (including SGML model). The metrics can be shown to fit into the framework for empirically assessing access control mechanisms in SGML model through audit methods (sometimes called “games” herein). The metrics can be formalized into black-box procedures that can capture: 1. distinguishability of security domain-specific responses using a domain distinguishability index (DDI), and 2. utility disparity induced by access restrictions for a utility gap index (UGI). The auditing methods can measure how effectively an auditor can conclude if the access control mechanism may correctly be implemented, by verifying if the correct domain adapter can be activated for an inference request. This effectiveness can be directly correlated with the access advantage score for the target security domain(s). Higher access advantage score denotes stronger access control enforcement. A perfectly separated system provides the auditor with an access advantage score of 1.0. The environment, that includes threat settings or access privileges can be kept constant for the audit methods.
[0078] In some embodiments, Domain Distinguishability Index (DDI) game assesses whether the auditor can effectively conclude if the correct security domains were used based on the generated responses. The primary motivation of this game can be to measure the distinguishability of different security domains' distributions. In Stepl of the game, Auditor c / Z chooses security domain set Suand emulates user u. c / Z sends user credentials creduand query q E T)Suto system 5. S verifies the user credential creduand sends back the model response(q) to c / Z, similar to the flow described for user A 801 in FIG. 8. In Step 2 of the game, c / Z chooses security domain set Svsuch that SvA Su= (p and emulates user v. Jl sends user credentials credvand the same queryq G T)Suto 5. 5 verifies the user credential credvand sends back the model response to c / Z, similar to the flow described for user B 802 in FIG. 8. In step 3 of the game, c / Z sends the models responses and domain information to membership inference oracle 0 (such as measurement and validation service 809 in FIG. 8) to obtain domain distinguishability score where m(-) is a membership inference metric in the [ 0,1 ] range.In some embodiments, an AUC-ROC graph or a true positive rate (TPR) at 1% false positive rate (FPR), abbreviated as TPR @ 1%FPR, can be used to measure membership inference. c / Z concludes the access control mechanism is correctly implemented if the domain distinguishability score where a can be a metric specific threshold.
[0079] The Domain Distinguishability Index (DDI) metric measures average adversarial performance in the domain distinguishability game across all non-overlapping domain sets. In some embodiments, membership inference attacks (MIAs) leverage a sampled member data set (from the target model's training set) and a sampled non-member data set to assess privacy leakage. The more accurately an adversary separates and classifies samples as members or non-members, the higher the privacy risk. The MIA framework can be adopted for access control assessment to distinguish security domains. Members (q G DSu) can be distinguished from non-members (q G T)Sv) for the target domain Su. In some embodiments, the target domain can also mean the active domain and can be interchangeably used. MIA framework is discussed more, later on, while discussing FIG. 10A and FIG. 10B. In some embodiments, for any security domain set Su, the auditor can access samples from Su’s training data (member set) and samples from Sv’s training data (non-member set), where SvA Su= <p. The auditor may then evaluate how successfully the DDI metric can distinguish the member set from the non-member set when the SGML model is activated for Su. The evaluation described above can occur for all security domains, providing an aggregate access advantage score. In some embodiments, some of the metrics used to quantify auditor’s success includes Attack ROC curves, Attack TPR at low FPR as instantiations of access advantage, where higher scores indicate enforcement as Suspecific responses can be more distinguishable. In some embodiments, the Receiver Operating Characteristics (ROC) curve illustrates the trade-off between the True Positive Rate (TPR) and the False Positive Rate (FPR) for the attacks. The FPR measures the proportion of non-member samples that are incorrectly classified as members, while the TPR represents the proportion of member samples that are correctly identified as members. Attack ROC curves represent the area under the ROC curve (AUC - ROC) as an aggregate metric to assess the overall success of the attacks. AUC-ROC can be a threshold-independent metric, and it can show the probability that a positive instance (member)has a higher score than a negative instance (non-member). In some embodiments, Attack TPR at low FPR metric can be crucial for determining the effectiveness of an attack at confidently identifying members of the training dataset without falsely classifying non-members as members. The FPR thresholds are usually kept low (such as 1% up to 5%). For example, the TPR at an FPR of 1% is calculated by setting the detection threshold so that only 1% of non-member samples can be predicted as members.
[0080] In some embodiments, Domain Distinguishability Index (DDI)) can be expressed as follows. For a domain universe § consisting of n security domains, let p denote the PermLLM trained on data D from all security domains with access control mechanism. For each ordered pair of domain sets ( St£ §,5 c § )with no overlap (i.e., StA Sj = (p ), let O^Si,s^ =
[0081] During testing and measurement, standard deviation across all domain setpairs are measured to capture variability. DDI can be viewed as an access advantage metric, where the response relevance score relv for Ston query q, relv( ^r(q), Sj), is a binary value on whether the membership inference oracle O 's output is above a membership threshold. The difference operator © is the MIA method specific composition of response relevance for all the samples in the member and non-member sets. A higher DDI indicates more robust separation between security domains. In some embodiments, MIAs such as Zlib, Loss, Mink, Mink++ can be used to test SGML model.
[0082] In some embodiments, a Utility Gap Index (UGI) can be evaluated and measured. The second game UGI can evaluate how distinctly the responses from two different security domains impact the utility perceived by users. The rationale behind this game can be to confirm that enforced access controls result in meaningful variations in response quality. In Step 1, Auditor c / Z chooses security domain set Suand emulates user u. c / Z sends user credentials creduand query q E T>Suto system 5. 5 verifies the user credential creduand sends back the model response to C / Z. In step 2, c / Z chooses security domain set Svsuch that SvA Su= (p and emulates user v. c / Z sends user credentials credvand the same query q E T)Suto 5. 5 verifies the user credential credvand sends back the model response p (cp) to c / . In step 3, given a utility function ( / (•) that outputs values in [0,1] range, c / Z concludes the access control mechanism is correctlyimplemented if the utility gap score> a, where a can be a UGI metric threshold.
[0083] The utility function [ / (•) can be any metric that is pertaining to the application. In some embodiments, metrics to evaluate the utility of the model generations can include Bleurt Score (can be used interchangeably with the abbreviation bleurt), Bert F-l Score (can be used interchangeably with the abbreviation bert) and verbatim accuracy (can be used interchangeably with the abbreviation acc). These metrics measure how similar the generated text can be to the ground truth. Bleurt and bert measure the semantic similarity, bleu measures the fraction of common n-grams, and acc gives a binary decision of whether the generated text verbatim matches the ground truth. All the metrics lie in a [0,1] range, where values close to 1 indicate high model utility. The game described above can aggregate the mean utility gaps across all domain set pairs to obtain the Utility Gap Index (UGI) metric. Utility Gap Index (UGI) can be empirically expressed as follows. Let ( / (•) be a chosen utility metric and for a domain set pair (5)with no overlap (i.e., Stn S;= ), UtilityGap (s^)([7) =| ( / ( / ^(q) I 5U) - ( / ( / ^(q) I 5V) I. The Utility Gap Index aggregates utility gaps across all ordered domain set pairs: Ay= ESICSISJCS UtilityGapwhere E is the expectation over all domain sets.
[0084] UGI can also be an instantiation of the access advantage metric in which the relevance score for security domain set 5) on query q can be the utility value itself, relv( ^r(q), Sj) =I Si)>and the operator Q computes the absolute difference of those relevance scores across the sampled queries. A larger UGI indicates that enforced access controls yield more pronounced-and thus more easily perceivable-differences in response quality between security domains. As with DDI, the standard deviation across pairs to characterize variability can be reported.
[0085] Theoretically, UGI, represented as AU(-.) may reach 1.0, but empirically tests can generate much smaller yet still substantial access advantage gaps while evaluating users with access to a single security domain. These gaps can be significantly impacted by domain distributions and the strictness of the scoring metric. For example, one dataset can exhibit higher Ableuor high Aaccvalues because it can have the highest number of distinct domains ( 10 in total). In some embodiments, the results can also depend on the type of metric used. For example, Ableuand Aacccan have the largest values as these metrics look for verbatim pattern matches, thus requiring the model to memorize the nuances in the target domain. On the other hand, other UGI metrics such as Ableurtand Abertlook for approximate similarities, and hence are impacted by the similarities across the domains. This suggests that the verbatim matching metrics, AbZeuand Aacc, are bettermodel utility metrics for measuring access advantage compared to the similarity based metrics bieurt and Abert• For large data sets like RCV1, all the metrics can achieve similar values as the model begins to generalize more. Even when values are not close to 1, they can still provide credence to the fact that the domains are different, and the access control protocol discussed above works as expected due to the utility gaps. In some embodiments, the access advantage threshold a is dependent on the type of utility metric: verbatim matching metrics Ableuand Aacchave higher threshold than similarity based metrics Ableurtand Abert. For Aaccmetric, a > 0.2 is sufficient to infer that access control is happening correctly.
[0086] FIG. 9 illustrates a sample user interface of a measurement and validation service that sets up various parameters to generate an inference request, and receive a response, according to various embodiments. The measurement and validation (MV) service 809 can be used by an auditor to select a user 901 from a plurality of users and authenticate 908 with ACM service 803. If the authentication is successful, the security domains can be populated based on the mappings in the access control table as shown in FIG. 5. An active domain 903 can be chosen so that the MV service can determine the security domain(s) the requests 906 pertains to. A SGML model 904 can be chosen from a plurality of SGML models that are trained based on the mapping in the access control table (such as FIG. 5). A dataset 905 can be chosen from a plurality of dataset. In some embodiments, datasets such as WMDP GPQA, SimpleQA or RCV1 can be selected. Each of the datasets can have a different characteristic. For example, RCV1 is a benchmark dataset on text categorization. After all the parameters that are deemed necessary to perform inference requests are chosen, inference requests 906 can be sent 909, and responses 907 can be saved 910 in a datastore 1407. Receiving responses 907 over different datasets can help get a better understanding of the behavior of the various SGML models. The auditor can mix and match various combinations of users, active domains, SGML models, and datasets to evaluate the various responses.
[0087] FIG. 10A illustrates a sample user interface of a measurement and validation service that sets up various metric-based parameters to evaluate and measure domain distinguishability, according to various embodiments. The measurement and validation (MV) service 809 can have a separate screen(s) to evaluate Domain Distinguishability Index (DDI). This screen can be used in conjunction with screen 1 (FIG. 9) to evaluate DDI. For example, Screen 1 (FIG. 9) can be used for running different combinations of user inference requests for various combinations of security domains, dataset, SGML models. The data saved 910 can be then used in conjunction with Screen 2 (FIG. 10A). In some embodiments, the screen can contain a choice to choose the appropriate index to evaluate 1001 such as DDI as discussed above. In some embodiments, any metric suitable for the application can be chosen using a screen shown in FIG. 10 A. Similarly, the type of metric1002 can be selected that is pertaining to the index 1001 selected. In some embodiments, as discussed above AUC-ROC, TPR@ 1% FPR, TPR @5% FPR can be selected. These metrics can help quantify the success of the membership inference attacks. The type of dataset 1003 can be chosen based on the application. In some embodiments WMDP, SimpleQA, RCV1, GPQA can be used to test the application with different types of datasets. To evaluate DDI, different kinds of MIA methods can be experimented with to evaluate the SGML’s model ability to distinguish one security domain from another. In some embodiments, MIA methods 1004 can be Loss method, Zlib method, Ref method, Min-K method, Min-K++ method. Each of these methods can help calculate membership scores that helps evaluate DDI. SGML model 1005 from a plurality of SGML models can be selected. Generate plot 1006 can generate data based on the metrics chosen.
[0088] FIG. 10B illustrates a sample plot generated to evaluate Domain Distinguishability Index, according to various embodiments. Generate Plot 1006 can generate a plot shown in this figure, where the TPR and FPR are analyzed and saved by the MV service 809 based on various inference requests / responses for different metric-based parameters that include dataset, MIA methods, SGML models. In some embodiments, selecting a particular dataset, MIA method, SGML model can generate an AUC-ROC curve as shown in FIG. 10B. Similarly, curves can be generated for other DDI based metrics such as TPR@ 1% FPR, TPR @5% FPR.
[0089] FIG. 11A illustrates a sample user interface screen for the measurement and verification service 809 for evaluating utility gap index, according to various embodiments. The measurement and verification service 809 can provide a visual interface (such as this one ) to select one or more metric-based parameters for evaluating UGI such as selecting an SGML model 1102 from a plurality of SGML models. The number of security domains 1103 represents number of security domains trained per adapter of the SGML model that can be found in the access control table as shown in FIG. 5. Dataset 1104 represents the type of dataset used for generating the inference response. In some embodiments WMDP, SimpleQA, RCV1, GPQA can be used to test the application with different types of datasets. Foundational ML model 1105 represent the foundational SGML model that can be used to train the SGML model. In some embodiments, Llama, Mistral are foundational LLM models. Metrics pertaining to the utility gap 1106 can be selected to generate plot 1107 that is shown in FIG. 1 IB. Similar to FIG. 10A, FIG. 11 A can be used in conjunction with FIG. 9. The user interface in FIG. 9 can be used to generate inference responses based on various combinations of metric-based parameters chosen such as active domain, SGML model, datasets. The inference responses can be used in conjunction with the UGI metric-based parameters chosen in FIG. 11A to generate plots 1107 like the one shown in FIG.1 IB. In some embodiments, utility gap metrics 1106 such as Ableu, Aacc, Ableurtcan be used. Aoperator represents the difference between the utility scores across two different security domains. Metrics such as bleurt, bleu, acc, bert were discussed above; these metrics can measure how similar the generated text can be to the ground truth.
[0090] FIG. 1 IB illustrates a sample graph showing the effect of an SGML model trained with adapters for varying number of security domains, according to various embodiments. In some embodiments, different plots can be generated based on whether the metrics are used for SGML models trained with one security domain per adapter, or SGML models trained with multiple security domains per adapter. The utility gap metric can be used to evaluate the effectiveness of training SGML models with different combinations of security domains per adapter. SGML models trained with one security domain per adapter can be evaluated to show domain distinguishability as well as how well the domains are utilized with the utility metrics. As the number of security domains per adapter increases, the SGML model inferences can get generalized and this can be measured using the UGI metrics, as shown in FIG. 11B. SGML models can be trained in different ways. In some embodiments, an SGML model can have one security domain trained per adapter. In other SGML models, there can be many single security domain per adapters merged into multiple security domain per adapter. In some other SGML models, multiple security domains can be trained per adapter. The graph measures the effectiveness of utilizing various security domains (UGI index) with one or more active security domains. In some embodiments, UGI gap metric 1121 can be measured using utility gap metrics 1106 such as Ableu, Aacc, Ableurt. The graph shows that as the number of active domains 1122 used in generating inference response increases, the single domain per adapter 1126 SGML model is not able to utilize the various security domains to generate an appropriate inference response. The results improve for 1125 (adapters trained for single security domain merged into a single multi-domain per adapter SGML model) but still drops off as the number of the active domains increase. However, SGML model trained for multiple security domains per adapter 1123 shows the best result with a higher UGI score that is above the threshold 1124.
[0091] FIG. 12A illustrates a sample user-interface screen that can be used to configure various metric-based parameters of a measurement and validation service 809 for utility comparison, according to various embodiments. FIG. 12A is similar to FIG. 11 A, as both the screens utilize the UGI metric. The difference between FIG. 11 A and FIG. 12A can be that in FIG. 12 A, the actual utility score 1205 for a security domain may be measured instead of a utility gap (which is the difference between the utility scores between two security domains) in FIG. 11 A. The utility score is useful in evaluating the effectiveness of the SGML model 1202, especially during training and testing time periods. Different SGML models can be trained and tested and the results between thetraining and testing can help determine the loss gap which can be expressed as the difference between the training utility score and the testing utility score. This can be performed across different datasets 1203 and different foundational ML models 1204. In some embodiments, utility score can be measured using metric methods such as bleurt, acc, bert, and / or bleu. These metrics measure how similar the generated text is to the ground truth. Metric bluert and bert measure the semantic similarity, bleu measures the fraction of common n-grams and acc can provide a binary decision of whether the generated text verbatim matches the ground truth. All the metrics can lie in [0,1] range, where values close to 1 indicate high model utility. Similar to FIG. 10A and FIG.11 A, FIG. 12A can be used in conjunction with FIG. 9, wherein FIG. 9 can be used to generate inference responses for various combinations of users, active domains, SGML models, dataset, and combined with the metric-based parameters chosen in FIG. 12 A, plots 1206 can be generated as shown in FIG. 12B.
[0092] FIG. 12B illustrates the effect of utility scores during training and testing time of various machine learning models to determine loss gap, according to various embodiments. FIG. 12B can be generated based on the metric-based parameters chosen in FIG. 12A and inference responses saved using FIG. 9. The graph can provide utility score comparison results 1223 for various tuned models such as a fully fine tuned ML model 1224 and SGML model 1225 based on training dataset 1226 and testing dataset 1227. Different combinations can be attempted based on different datasets 1222 and different utility metrics 1221. In some embodiments, utility metrics 1221 can be UGI metric methods discussed before such as Bleurt score (bleurt), Scarebleu score (bleu), verbatim accuracy (acc), Bert Fl score (bert). Similar utility scores can indicate that the performance of various tuned ML models are similar for the type of metrics chosen. For example, similar scores for a fully fine tuned model 1224 and SGML model 1225 can demonstrate that for the metrics chosen, the models perform similarly. Different datasets chosen can have different impact on test results. For example, full fine-tuning a model can either completely overfit and set high utility score for some datasets or completely underfit and set low utility score for some other datasets. This can be tested using foundational models to see how different foundational models can impact the results. In some embodiments, dataset can have many security domains, thus each of the SGML model -based domain adapter can only see a fraction of data compared to other adapter type models such as LoRA’ s approach to adapters. Factors such as those listed above may need to be considered while interpreting the data results.
[0093] FIG. 13 A illustrates a sample user-interface that can be used by the measurement and verification service to save parameters to train and test various datasets and models, according to various embodiments. The measurement and verification service (screen 8) 809 can provide a setof metrics-based parameters that can be saved for training and testing various SGML models 1303 based on different foundational models 1301 for different datasets 1302. In some embodiments WMDP, SimpleQA, RCV1, GPQA can be used to test the application with different types of datasets. Model specific tuning data such as learning rate 1304, epochs 1305 can also used as the training-based parameters that may be needed to perform training and tuning operations 1307. In some embodiments, datasets 1302 that can be chosen such that the pretrained foundational models 1301 may not have been exposed. High initial training loss on a dataset can indicate that the dataset may not have been used in pretraining or may have been forgotten by the models. However, the training loss can be gradually reduced during fine-tuning of the SGML models. Different combinations of the parameters (1301-1305) can be tried to get a sense of the performance of the SGML models during testing. The generalization loss gap described in FIG. 13B can be based on training-based parameters saved in FIG. 13 A. In some embodiments, generalization loss gap can be the gap between model’s loss on training and test data sets.
[0094] FIG. 13B illustrates a sample user interface that can be used to evaluate generalization loss gap for different tuned ML models based on training and testing results, according to various embodiments. The graph shows how much of a loss gap occurs between the training dataset 1326 and the test dataset 1327 for various models. This can help evaluate how different variations of adapter based ML models (such as SGML models discussed herein 1324, LoRA adapter-based fine tuning ML models) can perform compared to a fully fine-tuned foundational ML model 1324. If the loss gap 1328 can be within an acceptable range, such as a predefined threshold, then the graph can help prove that SGML models are very effective and the adapters may have saved considerable time by training only a fraction of the parameters to train the SGML model compared to a fully fine-tuned ML model which can have a very large number of hyperparameters and can take a very long time to retrain. The tests can be performed on different foundational models 1321 as well as different datasets 1322 to generate quantifiable metrics that can show whether the SGML models can work similar to fully fine tuned ML model or not.
[0095] FIG. 14 illustrates various components that comprise an access control and audit service, according to various embodiments. The access control and audit (ACA) service 800 can receive a request 1409 from a user, and can handle all the processing necessary for generating a response 1410. The ACA service can comprise many components as shown in FIG. 14. The access control mechanism 803 (ACM) can authenticate a request 1409 and lookup an access control table (such as the one shown in FIG. 5) to route the request 1409 to the appropriate one or more adapters of the SGML model to generate an inference response. The inference response can be stored in a datastore 1407 and sent to the client through the interface 1401. The ACM service can also beresponsible for the generation of the access control table based at least in part on user credentials and adapters that are to be trained for one or more security domains for each of the users. The measurement and validation service 809 discussed in detail in FIG. 9, FIG. 10A, FIG. 10B, FIG.11 A, FIG. 11B, FIG. 12A, FIG. 12B, FIG. 13A and FIG. 13B can provide a user with different responses pertaining to measurement of the inferences as well as validation of the metrics used for determining the effectiveness of SGML models. The interface manager 1402 can act as a gateway between the external interface 1401 and all the internal components shown in FIG. 14. The UI interface and graphics library 1403 can provide all the UI and graphics-based libraries needed to interface with a user to provide appropriate responses, such as the ones shown in FIG. 9 - FIG.13B. The datastore 1407 can store various kinds of data that can include various foundational ML models 1404, various inferences, reports, summaries, various datasets 1405, various versions of SGML models 1406 based on different foundational models 1404, various MIA 1408 and metrics libraries 1407. Each of the libraries (1403 - 1406) can be their own service that are responsible for managing and deploying different types and versions of their respective components. For example, foundational ML library 1404 can be a service that has various types of foundational ML models and many versions of each of the foundational ML models and can provide the software / infrastructure required to interact with various other components and services shown in FIG. 14 to deploy the different types and versions of the foundational ML models. Similarly, dataset library 1405, SGML model library 1406, metrics library 1407, MIA library 1408 can be their own respective service that can have various types of respective models and versions that comprise software / infrastructure required to interact with various other components and services (such as ones shown in FIG. 14) to deploy the different types and versions of the respective libraries. The interface 1401 can be a client / server architecture where the user interface is local so that the client and server can be located locally such as a desktop or an industrial local server, or it can be a programmatic interface where the client and server are not located locally where the client can be any computer interfacing through the cloud to an ACA service deployed as an endpoint in a cloud infrastructure environment. Various other client / server architectures are possible where the ACA can reside on another industrial server that is outside of the client software and the client software can reside on any other desktop / laptop or mobile device.
[0096] FIG. 15 illustrates a flowchart for generating a security enforcement metric for a generative machine learning model for one or more security domains according to various embodiments. As indicated at 1501, a generative machine learning model can generate respective responses to different submissions of a same inference request associated with different users, wherein individual ones of the different users can have access to one or more different security domains ofa plurality of security domains supported by the generative machine learning model. Furthermore, as indicated at 1502, a security enforcement metric can be generated for the generative machine learning model for one or more of the plurality of security domains based, at least in part on a difference in relevance scores of the respective responses to the different submissions of the same inference request determined associated with the different users.
[0097] FIG. 16 illustrates a sequence diagram that can describe the flow of events between a client and the access control and audit service via a programming interface, according to various embodiments. A client 1601 can push one or more user profiles 1603 containing all the information necessary to authenticate and make inference requests for the one or more users. The access control and audit (AC A) service 800 receives the requests through a programmatic interface 1602. The AC A service saves the user profiles 904 and acknowledges back to the client. The client 1601 requests to select one of the users from the plurality of users based on the user profile 1605. The ACA service authenticates the user and acknowledges the authentication to the client 1606. The client can push more parameters pertaining to inference and metric-based processing that includes an active domain, metrics-based parameters 1607 to the ACA service. The ACA service 800 can acknowledge the saved parameters 1608 and can have the configurations required for processing inference requests as well as requests pertaining to metric-scores. In some embodiments, the metric-scores can be UGI metrics such as utility scores, utility gap, DDI membership scores and loss gap numbers. The client can send multiple requests iteratively based on a series of combinations of user profiles, active domains, metric-based parameters that can quantify the metric-scores 1609, 1611. The ACA service can save the responses in a datastore 1407 and send acknowledgements back to the client 1610, 1612. The client can also send requests to generate summarized reports, tables based on the inference and metric-based responses 1613. The ACA service can save the reports in the datastore 1407 and send the responses to the client 1614.
[0098]
[0099] FIG. 17 illustrates a flow-chart that describes the sequence of events processed by an access control and audit service, according to various embodiments. The access control and audit (ACA) service can be initialized 1700 to process requests from a client such as client 1601. Based on the request from a client such as client 1601, the ACA service can select a user that has access to one or more security domains from a plurality of user profiles 1701. The ACA can attempt to authenticate 1702 the user credentials and look up the access control table as shown in FIG. 5 and acknowledge success or failure 1711 to the client. The ACA service can select one or more active domains and metric-based parameters 1703 based on client’s request. After the user, active domains, metric-based parameters are configured the ACA service can accept multiple inferencerequests directed to the SGML model 1704 and perform metric-based processing to quantify one or more metrics 1705. The processing of inference requests and performing metric-specific processing can occur iteratively over many inference requests 1706, active domains 1707 and users 1708 as illustrated in FIG. 17. The AC A service can also perform more aggregated processing of the inferences and metrics-based data 1709 to generate reports, tables based on client requests. The AC A service can be running till it can be stopped 1710. FIG. 18 illustrates a computing system configured to implement the methods and techniques described herein, according to various embodiments. The computer system 1800 may be any of various types of devices, including, but not limited to, a personal computer system, desktop computer, laptop or notebook computer, mainframe computer system, handheld computer, workstation, network computer, a consumer device, application server, storage device, a peripheral device such as a switch, modem, router, etc., or in general any type of computing device.
[0100] The mechanisms for implementing subject level privacy attack analysis for federated learning, as described herein, may be provided as a computer program product, or software, that may include a non-transitory, computer-readable storage medium having stored thereon instructions, which may be used to program a computer system (or other electronic devices) to perform a process according to various embodiments. A non-transitory, computer-readable storage medium may include any mechanism for storing information in a form (e.g., software, processing application) readable by a machine (e.g., a computer). The machine-readable storage medium may include, but is not limited to, magnetic storage medium (e.g., floppy diskette); optical storage medium (e.g., CD-ROM); magneto-optical storage medium; read only memory (ROM); random access memory (RAM); erasable programmable memory (e.g., EPROM and EEPROM); flash memory; electrical, or other types of medium suitable for storing program instructions. In addition, program instructions may be communicated using optical, acoustical or other form of propagated signal (e.g., carrier waves, infrared signals, digital signals, etc.)
[0101] In various embodiments, computer system 1800 may include one or more processors 1870; each may include multiple cores, any of which may be single or multi -threaded. Each of the processors 1870 may include a hierarchy of caches, in various embodiments. The computer system 1800 may also include one or more persistent storage devices 1860 (e.g. optical storage, magnetic storage, hard drive, tape drive, solid state memory, etc.) and one or more system memories 810 (e.g., one or more of cache, SRAM, DRAM, RDRAM, EDO RAM, DDR 10 RAM, SDRAM, Rambus RAM, EEPROM, etc.). Various embodiments may include fewer or additional components not illustrated (e.g., video cards, audio cards, additional network interfaces, peripheraldevices, a network interface such as an ATM interface, an Ethernet interface, a Frame Relay interface, etc.)
[0102] The one or more processors 1870, the storage device(s) 1850, and the system memory 1810 may be coupled to the system interconnect 1840. One or more of the system memories 1810 may contain program instructions 1820. Program instructions 1820 may be executable to implement various features described above, including enforcing and / or auditing access controls to generative machine learning models and other techniques 1822 as discussed above, in some embodiments as described herein. Program instructions 1820 may be encoded in platform native binary, any interpreted language such as Java™ byte-code, or in any other language such as C / C++, Java™, etc. or in any combination thereof.
[0103] In one embodiment, Interconnect 1890 may be configured to coordinate I / O traffic between processors 1870, storage devices 1860, and any peripheral devices in the device, including network interfaces 1850 or other peripheral interfaces, such as input / output devices 1880. In some embodiments, Interconnect 1890 may perform any necessary protocol, timing or other data transformations to convert data signals from one component (e.g., system memory 1810) into a format suitable for use by another component (e.g., processor 1870). In some embodiments, Interconnect 1890 may include support for devices attached through various types of peripheral buses, such as a variant of the Peripheral Component Interconnect (PCI) bus standard or the Universal Serial Bus (USB) standard, for example. In some embodiments, the function of Interconnect 1890 may be split into two or more separate components, such as a north bridge and a south bridge, for example. In addition, in some embodiments some or all of the functionality of Interconnect 890, such as an interface to system memory 1810, may be incorporated directly into processor 870.
[0104] Network interface 1850 may be configured to allow data to be exchanged between computer system 1800 and other devices attached to a network, such as other computer systems, or between nodes of computer system 1800. In various embodiments, network interface 1850 may support communication via wired or wireless general data networks, such as any suitable type of Ethernet network, for example; via telecommunications / telephony networks such as analog voice networks or digital fiber communications networks; via storage area networks such as Fibre Channel SANs, or via any other suitable type of network and / or protocol.
[0105] Input / output devices 1880 may, in some embodiments, include one or more display terminals, keyboards, keypads, touchpads, scanning devices, voice or optical recognition devices, or any other devices suitable for entering or retrieving data by one or more computer system 1800. Multiple input / output devices 1880 may be present in computer system 1800 or may be distributedon various nodes of computer system 1800. In some embodiments, similar input / output devices may be separate from computer system 1800 and may interact with one or more nodes of computer system 1800 through a wired or wireless connection, such as over network interface 1850.
[0106] Those skilled in the art will appreciate that computer system 1800 is merely illustrative and is not intended to limit the scope of the methods for providing enhanced accountability and trust in distributed ledgers as described herein. In particular, the computer system and devices may include any combination of hardware or software that may perform the indicated functions, including computers, network devices, internet appliances, PDAs, wireless phones, pagers, etc. Computer system 1800 may also be connected to other devices that are not illustrated or instead may operate as a stand-alone system. In addition, the functionality provided by the illustrated components may in some embodiments be combined in fewer components or distributed in additional components. Similarly, in some embodiments, the functionality of some of the illustrated components may not be provided and / or other additional functionality may be available.
[0107] Those skilled in the art will also appreciate that, while various items are illustrated as being stored in memory or on storage while being used, these items or portions of them may be transferred between memory and other storage devices for purposes of memory management and data integrity. Alternatively, in other embodiments some or all of the software components may execute in memory on another device and communicate with the illustrated computer system via inter-computer communication. Some or all of the system components or data structures may also be stored (e.g., as instructions or structured data) on a computer-accessible medium or a portable article to be read by an appropriate drive, various examples of which are described above. In some embodiments, instructions stored on a computer-accessible medium separate from computer system 1800 may be transmitted to computer system 1800 via transmission media or signals such as electrical, electromagnetic, or digital signals, conveyed via a communication medium such as a network and / or a wireless link. Various embodiments may further include receiving, sending or storing instructions and / or data implemented in accordance with the foregoing description upon a computer-accessible medium. Accordingly, the present invention may be practiced with other computer system configurations.
[0108] Embodiments of the present disclosure can be described in view of the following clauses:Clause 1. A system, comprising:at least one processor;a memory, comprising program instructions that when executed by the at least one processor cause the at least one processor to:receive an inference request directed to a generative machine learning model; generate a result for the inference request, wherein to generate the result, the program instructions cause the at least one processor to:identify a one or more security domains associated with the inference request;identify one or more adapters for the generative machine learning model that enforce content restrictions in the result to the one or more security domains; anduse the one or more adapters for the generative machine learning model to generate the result; andprovide the result of the inference request.Clause 2. The system of clause 1, wherein the one or more adapters have a one-on-one mapping with the one or more security domains for the generative machine learning model.Clause 3. The system of clause 1, wherein the one or more adapters have a one-on-one mapping with the one or more security domains that are merged into a merged adapter that has one-to-many mapping with the one or more security domains for the generative machine learning model.Clause 4. The system of clause 1, wherein the one or more adapters are trained to have a one-to-many mapping with the one or more security domains for the generative machine learning model.Clause 5. The system of clause 3, wherein the merging of the one or more adapters that have a one-to-one mapping with the one or more security domains into the merged adapter is performed using Singular Value Decomposition (SVD).Clause 6. The system of any one of clauses 1-5, wherein the one or more adapters are mapped to the one or more security domains based at least in part on an access control table that comprises a mapping of one or more users to one or more security domains.Clause 7. The system of clause 6, wherein the inference request received from a user, activates the one or more adapters based at least in part on the access control table that comprises a mapping of the user to one or more security domains.Clause 8. A computer-implemented method, comprising:receiving an inference request directed to a generative machine learning model; generating a result for the inference request, wherein to generate the result:identifying a one or more security domains associated with the inference request; identifying one or more adapters for the generative machine learning model that enforce content restrictions in the result to the one or more security domains; andusing the one or more adapters for the generative machine learning model to generate the result; andproviding the result of the inference request.Clause 9. The method of clause 8, wherein the one or more adapters have a one-on-one mapping with the one or more security domains for the generative machine learning model.Clause 10. The method of clause 8, wherein the one or more adapters have a one-on-one mapping with the one or more security domains that are merged into a merged adapter that has one-to-many mapping with the one or more security domains for the generative machine learning model.Clause 11. The method of clause 8, wherein the one or more adapters are trained to have a one-to-many mapping with the one or more security domains for the generative machine learning model.Clause 12. The method of clause 10, wherein the merging of the one or more adapters that have a one-to-one mapping with the one or more security domains into the merged adapter is performed using Singular Value Decomposition (SVD).Clause 13. The method of any one of clauses 8-12, wherein the one or more adapters are mapped to the one or more security domains based at least in part on an access control table that comprises a mapping of one or more users to one or more security domains.Clause 14. The method of clause 13, wherein the inference request received from a user, activates the one or more adapters based at least in part on the access control table that comprises a mapping of the user to one or more security domains.Clause 15. One or more non-transitory, computer-readable storage media, storing program instructions that when executed on or across one or more computing devices, cause the one or more computing devices to implement:receiving an inference request directed to a generative machine learning model; generating a result for the inference request, wherein to generate the result:identifying a one or more security domains associated with the inference request; identifying one or more adapters for the generative machine learning model that enforce content restrictions in the result to the one or more security domains; andusing the one or more adapters for the generative machine learning model to generate the result; andproviding the result of the inference request.Clause 16. The one or more non-transitory computer-readable storage media of clause 15, wherein the one or more adapters have a one-on-one mapping with the one or more security domains for the generative machine learning model.Clause 17. The one or more non-transitory computer-readable storage media of clause 15, wherein the one or more adapters have a one-on-one mapping with the one or more security domains that are merged into a merged adapter that has one-to-many mapping with the one or more security domains for the generative machine learning model.Clause 18. The one or more non-transitory computer-readable storage media of clause 15, wherein the one or more adapters are trained to have a one-to-many mapping with the one or more security domains for the generative machine learning model.Clause 19. The one or more non-transitory computer-readable storage media of any one of clauses 15-18, wherein the one or more adapters are mapped to the one or more security domains based at least in part on an access control table that comprises a mapping of one or more users to one or more security domains.Clause 20. The one or more non-transitory computer-readable storage media of clause 19, wherein the inference request received from a user, activates the one or more adapters based at least in part on the access control table that comprises a mapping of the user to one or more security domains.Clause 21. A system, comprising:at least one processor;a memory, comprising program instructions that when executed by the at least one processor cause the at least one processor to:cause a generative machine learning model to generate respective responses to different submissions of a same inference request associated with different users, wherein individual ones of the different users have access to one or more different security domains of a plurality of security domains supported by the generative machine learning model;generate a security enforcement metric for the generative machine learning model for one or more of the plurality of security domains based, at least in part, on a difference in relevance scores of the respective responses to the different submissions of the same inference request determined associated with the different users.Clause 22. The system of clause 21, wherein the difference in relevance scores of the respective responses is used to generate a utility gap index that measures how distinctly the respective responses impact the utility perceived by the different users.Clause 23. The system of clause 21, wherein the difference in relevance scores of the respective responses is used to generate a domain distinguishability index that measures if the correct security domains is used based at least in part on the respective responses.Clause 24. The system of clause 21, wherein the difference in relevance scores of the respective responses is used to generate a metric that is compared with a threshold to indicate success or failure of enforcement with respect to the plurality of security domains.Clause 25. The system of clause 21, wherein the difference in relevance score of responses generated by the generative machine learning model is expected to be higher when a response is for a user that has access to a security domain that is an active domain, compared to another response for another user that has access to a security domain that is not an active domain.Clause 26. The system of any one of clauses 21-25, wherein the difference in relevance scores of the respective responses for one or more users generated by the generative machine learning model that uses one or more adapters mapped to one or more security domains is based at least in part on an access control table that comprises a mapping of the one or more users to the one or more security domains.Clause 27. The system of clause 26, wherein the difference in score of responses generated by the generative machine learning model is measured for inference requests performed by an auditor who has superuser privileges to select the one or more different security domains from the access control table.Clause 28. A computer-implemented method, comprising:causing a generative machine learning model to generate respective responses to different submissions of a same inference request associated with different users, wherein individual ones of the different users have access to one or more different security domains of a plurality of security domains supported by the generative machine learning model;generating a security enforcement metric for the generative machine learning model for one or more of the plurality of security domains based, at least in part, on a difference in relevance scores of the respective responses to the different submissions of the same inference request determined associated with the different users.Clause 29. The method of clause 28, wherein the difference in relevance scores of the respective responses is used to generate a utility gap index that measures how distinctly the respective responses impact the utility perceived by the different users.Clause 30. The method of clause 28, wherein the difference in relevance scores of the respective responses is used to generate a domain distinguishability index that measures if the correct security domains is used based at least in part on the respective responses.Clause 31. The method of clause 28, wherein the difference in relevance scores of the respective responses is used to generate a metric that is compared with a threshold to indicate success or failure of enforcement with respect to the plurality of security domains.Clause 32. The method of clause 28, wherein the difference in relevance score of responses generated by the generative machine learning model is expected to be higher when a response is for a user that has access to a security domain that is an active domain, compared to another response for another user that has access to a security domain that is not an active domain.Clause 33. The method of any one of clauses 28-32, wherein the difference in relevance scores of the respective responses for one or more users generated by the generative machine learning model that uses one or more adapters mapped to one or more security domains is based at least in part on an access control table that comprises a mapping of the one or more users to the one or more security domains.Clause 34. The method of clause 33, wherein the difference in score of responses generated by the generative machine learning model is measured for inference requests performed by an auditor who has superuser privileges to select the one or more different security domains from the access control table.Clause 35. One or more non-transitory, computer-readable storage media, storing program instructions that when executed on or across one or more computing devices, cause the one or more computing devices to implement:causing a generative machine learning model to generate respective responses to different submissions of a same inference request associated with different users, wherein individual ones of the different users have access to one or more different security domains of a plurality of security domains supported by the generative machine learning model;generating a security enforcement metric for the generative machine learning model for one or more of the plurality of security domains based, at least in part, on a difference in relevance scores of the respective responses to the differentsubmissions of the same inference request determined associated with the different users.Clause 36. The one or more non-transitory computer-readable storage media of clause 35, wherein the difference in relevance scores of the respective responses is used to generate a utility gap index that measures how distinctly the respective responses impact the utility perceived by the different users.Clause 37. The one or more non-transitory computer-readable storage media of clause 35, wherein the difference in relevance scores of the respective responses is used to generate a domain distinguishability index that measures if the correct security domains is used based at least in part on the respective responses.Clause 38. The one or more non-transitory computer-readable storage media of clause 35, wherein the difference in relevance scores of the respective responses is used to generate a metric that is compared with a threshold to indicate success or failure of enforcement with respect to the plurality of security domains.Clause 39. The one or more non-transitory computer-readable storage media of clause 35, wherein the difference in relevance score of responses generated by the generative machine learning model is expected to be higher when a response is for a user that has access to a security domain that is an active domain, compared to another response for another user that has access to a security domain that is not an active domain.Clause 40. The one or more non-transitory computer-readable storage media of any one of clauses 35-39, wherein the difference in score of responses generated by the generative machine learning model is measured for inference requests performed by an auditor who has superuser privileges to select the one or more different security domains from the access control table.
[0109] Although the embodiments above have been described in considerable detail, numerous variations and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the following claims be interpreted to embrace all such variations and modifications.
Claims
CLAIMSWHAT IS CLAIMED:
1. A system, comprising:at least one processor;a memory, comprising program instructions that when executed by the at least one processor cause the at least one processor to:receive an inference request directed to a generative machine learning model; generate a result for the inference request, wherein to generate the result, the program instructions cause the at least one processor to:identify a one or more security domains associated with the inference request;identify one or more adapters for the generative machine learning model that enforce content restrictions in the result to the one or more security domains; anduse the one or more adapters for the generative machine learning model to generate the result; andprovide the result of the inference request.
2. The system of claim 1, wherein the one or more adapters have a one-on-one mapping with the one or more security domains for the generative machine learning model.
3. The system of claim 1, wherein the one or more adapters have a one-on-one mapping with the one or more security domains that are merged into a merged adapter that has one-to-many mapping with the one or more security domains for the generative machine learning model.
4. The system of claim 1, wherein the one or more adapters are trained to have a one-to-many mapping with the one or more security domains for the generative machine learning model.
5. The system of claim 3, wherein the merging of the one or more adapters that have a one-to-one mapping with the one or more security domains into the merged adapter is performed using Singular Value Decomposition (SVD).
6. The system of claim 1, wherein the one or more adapters are mapped to the one or more security domains based at least in part on an access control table that comprises a mapping of one or more users to one or more security domains.
7. The system of claim 6, wherein the inference request received from a user, activates the one or more adapters based at least in part on the access control table that comprises a mapping of the user to one or more security domains.
8. A computer-implemented method, comprising:receiving an inference request directed to a generative machine learning model; generating a result for the inference request, wherein to generate the result:identifying a one or more security domains associated with the inference request; identifying one or more adapters for the generative machine learning model that enforce content restrictions in the result to the one or more security domains; andusing the one or more adapters for the generative machine learning model to generate the result; andproviding the result of the inference request.
9. The method of claim 8, wherein the one or more adapters have a one-on-one mapping with the one or more security domains for the generative machine learning model.
10. The method of claim 8, wherein the one or more adapters have a one-on-one mapping with the one or more security domains that are merged into a merged adapter that has one-to-many mapping with the one or more security domains for the generative machine learning model.
11. The method of claim 8, wherein the one or more adapters are trained to have a one-to-many mapping with the one or more security domains for the generative machine learning model.
12. The method of claim 10, wherein the merging of the one or more adapters that have a one-to-one mapping with the one or more security domains into the merged adapter is performed using Singular Value Decomposition (SVD).
13. The method of claim 8, wherein the one or more adapters are mapped to the one or more security domains based at least in part on an access control table that comprises a mapping of one or more users to one or more security domains.
14. The method of claim 13, wherein the inference request received from a user, activates the one or more adapters based at least in part on the access control table that comprises a mapping of the user to one or more security domains.
15. One or more non-transitory, computer-readable storage media, storing program instructions that when executed on or across one or more computing devices, cause the one or more computing devices to implement:receiving an inference request directed to a generative machine learning model; generating a result for the inference request, wherein to generate the result:identifying a one or more security domains associated with the inference request; identifying one or more adapters for the generative machine learning model that enforce content restrictions in the result to the one or more security domains; andusing the one or more adapters for the generative machine learning model to generate the result; andproviding the result of the inference request.
16. The one or more non-transitory computer-readable storage media of claim 15, wherein the one or more adapters have a one-on-one mapping with the one or more security domains for the generative machine learning model.
17. The one or more non-transitory computer-readable storage media of claim 15, wherein the one or more adapters have a one-on-one mapping with the one or more security domains that are merged into a merged adapter that has one-to-many mapping with the one or more security domains for the generative machine learning model.
18. The one or more non-transitory computer-readable storage media of claim 15, wherein the one or more adapters are trained to have a one-to-many mapping with the one or more security domains for the generative machine learning model.
19. The one or more non-transitory computer-readable storage media of claim 15, wherein the one or more adapters are mapped to the one or more security domains based at leastin part on an access control table that comprises a mapping of one or more users to one or more security domains.
20. The one or more non-transitory computer-readable storage media of claim 19, wherein the inference request received from a user, activates the one or more adapters based at least in part on the access control table that comprises a mapping of the user to one or more security domains.
