Method for use of a large language model by a user, and associated training method, training and use method and training and use system

Abstract

The present invention relates to a method for use of a large language model by a user, wherein the large language model is trained on training data with different confidentiality levels and is in the form of a plurality of confidential models, each confidential model being associated with a confidentiality level; the use method comprising the following steps: - acquiring (310) a use request from the user; - evaluating (320) access rights of the user, the access rights determining a confidentiality level of the training data accessible by the user; - retrieving (330) a confidential model corresponding to the confidentiality level determined by the access rights of the user; and - implementing (340) interactions of the user with the retrieved confidential model.

Description

[0001] DESCRIPTION

[0002] TITLE: Method for a user to use a large language model, and associated learning method, learning and use method, and learning and use system

[0003] The present invention relates to a method of using a large language model by a user.

[0004] The present invention also relates to a learning method, a learning and use method, and a learning and use system, associated with such a use method.

[0005] The field of invention lies at the intersection of the fields of artificial intelligence and cybersecurity, and in particular in the field of applications of large language models that use sensitive data.

[0006] As is well known, a large language model (LLM) is a linguistic model based on deep neural networks and possessing a large number of parameters (generally on the order of a billion or more). Training these models takes place in several phases, notably pre-training and fine-tuning.

[0007] Pre-training is the initial phase of training a large language model. During this phase, the model is trained on a large amount of unlabeled, usually public, textual data (e.g., books, articles, websites, etc.) with the aim of capturing the underlying patterns, structures, and semantic knowledge present in the text corpus.

[0008] Fine-tuning allows the large pre-trained language model to be adapted to specific tasks using task-specific labeled data.

[0009] Other training and adaptation phases can be performed when using the large language model. For example, in-context learning relies on multiple input-output examples to train the model to generate the desired response. There are different types of in-context learning, such as zero-shot learning, single-shot learning, and few-shot learning, which refer to the number of examples provided. Developing a large language model requires the availability of a large volume of data during the model training phase.This data may include proprietary and / or sensitive data (e.g., intellectual property) or private data (e.g., personal data of customers or users), while the model may be vulnerable to data leak attacks, such as model inversion attacks, which aim to extract information revealing details about a model's underlying data; membership inference attacks, which aim to determine the presence of an example in a model's training data; and reconstruction attacks, where an attacker seeks to reconstruct samples of training data using the model's gradients. Furthermore, it has been observed that a large language model can memorize training data, and this data can reappear in the model's responses with either correct or incorrect content (i.e., hallucinations).In this context, a user with access to a large language model trained and / or adapted on confidential data to which they do not have the right to access, can now access this data directly or indirectly (by inference).

[0010] The confidential data used by a large language model may have the same level of confidentiality or multiple levels of confidentiality and / or different types of permissions by group, role, and / or attribute, etc., requiring fine-grained access control. Furthermore, access rights to confidential data may also adapt to changing contexts, such as time, location, or device used (e.g., access rights to confidential data for an employee who has changed positions or left the company could be modified or revoked).

[0011] The confidential data used by a large language model may be subject to varying levels of confidentiality or authorization policies. In this document, the term "confidentiality level" will be used generically to also encompass the concept of "authorization policy" for greater clarity. These policies may include multiple levels of confidentiality as well as access mechanisms based on specific groups, roles, or attributes, thus requiring fine-grained access control compatible with models such as Role-Based Access Control (RBAC), Mandatory Access Control (MAC), Discretionary Access Control (DAC), or Attribute-Based Access Control (ABAC), among others.

[0012] In the current state of the art, various fine-tuning techniques exist that allow a pre-trained model to be adapted using specific data. However, these fine-tuning techniques aim to reduce the cost of fine-tuning in terms of computation and memory footprint, and do not provide confidentiality protection for the data used.

[0013] The confidentiality of the data used is traditionally protected using encryption schemes, e.g., partially or totally homomorphic encryption and secure multi-party computation. Homomorphic encryption allows users to perform calculations on encrypted data without decrypting it beforehand, while secure multi-party computation allows several participants to correctly execute a calculation together without revealing their respective inputs. These encryption approaches are generally used during the model's inference phase to protect user inputs and are not used during the training or adaptation phases, since the data being manipulated is encrypted and therefore unreadable. Furthermore, these approaches have a significant computational cost.

[0014] Anonymization approaches such as k-anonymization and differential privacy make it difficult to re-identify individual subjects within the data. k-anonymization involves removing or generalizing information from the data so that each subject cannot be distinguished from at least k - 1 other subjects. Differential privacy aims to reveal statistical information about data while protecting the privacy of individual subjects, ensuring that the presence or absence of a single subject does not affect the results of a query.In particular, an approach using differential privacy, called Private Aggregation of Teacher Ensembles (PATE), involves transferring the knowledge of a set of teacher models to a student model. Intuitive privacy is ensured by the teachers learning about disjoint sensitive data, while strong privacy is guaranteed by the noisy aggregation of teacher responses. The student model poses questions to the teacher models, so the student model only learns about sensitive data indirectly. Furthermore, differential privacy is inserted between the student and teacher models as an additional layer of privacy protection.Thus, the student model does not learn from the responses of the teachers' models, but from an aggregate of the teachers' models, with a (ideally small) element of randomness to better protect privacy. These aforementioned anonymization approaches aim to protect individuals' data confidentially, but do not allow for control over data access. At the model level, there are machine unlearning approaches. The goal of unlearning is to disrupt the model's understanding of sensitive data to the point that it can no longer retain meaningful information about that data, for example, by re-labeling sensitive data with randomly chosen incorrect labels, and then retraining the model with the modified data.To achieve exact untraining, the SISA (Sharding, Isolation, Slicing, and Aggregation) approach considers a model that is an aggregate of several submodels trained on disjoint partitions of the data. By dividing, isolating, slicing, and aggregating the training data, SISA allows for targeted data removal without a complete retraining of the model. On the other hand, approximate untraining focuses on reducing the influence of the data points targeted by untraining through an influence function (a tool of explainable AI), efficiently updating the model parameters to an acceptable level. While it does not completely remove influence, approximate untraining significantly reduces computational, storage, and time costs. Amnestic untraining involves selectively undoing the training steps that involved the targeted data.During training, a list of examples appearing in the different data sets is maintained, along with the parameter updates for each set. Sensitive data is removed by reverting parameter updates only for the sets containing that sensitive data. Unlearning techniques aim to guarantee individuals' right to be forgotten by allowing the deletion of their data stored in the model, but they do not strengthen access control to their data.

[0015] Data privacy protection can also be achieved with approaches that maintain data locality, such as federated learning, which allows multiple clients to collaboratively train a model while keeping their data local, or split learning, which is used to split the model into several parts distributed, for example, between a client and a server, and transmit only unreadable activations and gradients between the two. Offsite tuning is a privacy-friendly data locality approach suitable for large language models with a billion parameters. In offsite tuning, the model owner sends a lightweight adapter and a lossy compressed emulator to the data owner. The data owner then fine-tunes the adapter on the sensitive data with the help of the emulator.The refined adapter is then returned to the model owner, who inserts it into the complete model. However, all these data locality approaches remain vulnerable to data leak attacks during model training and inference.

[0016] Finally, there are approaches that offer post-hoc black-box protection designed for a public model with open access, and approaches that offer a private model with closed or restricted access. With the increasing integration of large language models into many common applications, post-hoc strategies, particularly guardrails, are being adopted more and more frequently. These guardrails aim to monitor and filter inputs and outputs to the model to prevent potential abuse and limit risks related to privacy, ethical use, data bias, robustness, and so on. However, guardrails can face threats such as rapid injection, rapid extraction, or jailbreaking attempts.Following the same logic of post-hoc strategies, a privacy vault solution ensures that only authorized users can access sensitive information by filtering user requests and model responses, notably through de-identification of requests and re-identification of responses. However, this solution does not allow control over access to models trained on sensitive data, as it targets third-party or public models. Using private models allows for the isolation of models, thus guaranteeing the security of sensitive data (and even simplifying data regionalization).However, isolation does not offer data governance in the form of access control, but adopts an "all or nothing" approach: any user who can access the private model has the ability to access the data used by the model.

[0017] In summary, state-of-the-art methods dealing with the confidentiality of training data do not allow for effective fine-grained access control to this data at each phase of the life (i.e. training, adaptation and use) of a large language model.

[0018] The present invention aims to remedy these drawbacks and to propose means of ensuring the confidentiality of training data with fine-grained access control to this data at each phase of the life (i.e. training, adaptation and use) of a large language model.

[0019] To this end, the invention relates to a method for using a large language model by a user, the large language model being learned on training data of different confidentiality levels and taking the form of a plurality of confidential models, each confidential model being associated with a confidentiality level; the method of use comprising the following steps:

[0020] - acquisition of a request for use from the user;

[0021] - evaluation of user access rights, access rights determining a level of confidentiality of training data accessible by the user;

[0022] - retrieval of a confidential template corresponding to the level of confidentiality determined by the user's access rights;

[0023] - implementation of user interactions with the retrieved confidential model.

[0024] According to other advantageous aspects of the invention, the method of use comprises one or more of the following features taken individually or in all technically possible combinations: each interaction of the user with the corresponding confidential model is carried out via a policy application point capable of applying a security policy; each interaction of the user with the corresponding confidential model is carried out directly after decryption of the confidential model retrieved in encrypted form, the decryption being carried out with a key selected according to the user's access rights.

[0025] The invention also relates to a method for learning a large language model on training data of varying levels of confidentiality; the learning method comprising the following steps:

[0026] - creation of training data and determination for each piece of training data of at least one level of confidentiality;

[0027] - for each level of confidentiality, training a confidential model on the training data corresponding to that level of confidentiality and saving this confidential model with the corresponding level of confidentiality.

[0028] According to other advantageous aspects of the invention, the learning method comprises one or more of the following features taken individually or in all technically possible combinations: when the training data includes datasets having distinct and independent levels of confidentiality, the training of the confidential models associated with these levels of confidentiality is carried out in parallel on the respective datasets;where the training data includes datasets having distinct and dependent levels of confidentiality, the training of the confidential models associated with these levels of confidentiality is carried out consecutively, each subsequent confidential model being trained from a previous confidential model on the dataset having the level of confidentiality associated with the next confidential model, the previous confidential model being trained on a training dataset having a level of confidentiality included in that associated with the next confidential model;where the training data includes a dataset comprising a subset of data having at least two distinct and independent levels of confidentiality, the training of the confidential models associated with these levels of confidentiality is carried out in parallel from an intermediate model previously trained on said subset of data having at least two distinct levels of confidentiality, each of said confidential models being further trained on a subset of data having only the level of confidentiality associated with that confidential model;where the training data includes datasets having distinct and independent or dependent levels of confidentiality, the training of the confidential models associated with these levels of confidentiality is carried out in parallel, each confidential model is obtained by merging, using set learning or federated learning techniques, intermediate models learned in parallel on subsets of data having the same levels of confidentiality and constituting together the data associated with the confidential level of said confidential model; the process further including a step of training a public model from a public dataset; each confidential model being advantageously trained from the public model;Training a confidential model includes pre-training a model using training data with the confidentiality level associated with that confidential model and / or fine-tuning an initial model using training data with the confidentiality level associated with that confidential model; in the case of fine-tuning, parameters defining the initial model are frozen, each confidential model advantageously presenting a MoE expert trained on training data with the confidentiality level associated with that model; determining at least one confidentiality level for each training data set includes partitioning the training data to identify confidentiality levels.

[0029] The present invention also relates to a method for learning and using a large language model by a user, comprising the following steps: implementation of a learning method as defined below; implementation of a usage method as defined below.

[0030] The present invention relates finally to a system for learning and using a large language model by a user, comprising: a learning device including technical means configured to implement a learning process as defined below; a usage device including technical means configured to implement a usage process as defined below.

[0031] The invention will be better understood upon reading the accompanying drawings, given solely by way of non-limiting example, in which:

[0032] [Fig.1] Figure 1 is a schematic view of a learning and use system according to the invention;

[0033] [Fig.2] Figure 2 is a flowchart of a learning and use method according to the invention, the learning and use method being implemented by the learning and use system of Figure 1;

[0034] [Fig.3] [Fig.4] [Fig.5] [Fig.6] [Fig.7] [Fig.8] [Fig.9] Figures 3 to 9 are different illustrations of the implementation of the learning process and use of Figure 2.

[0035] Figure 1 illustrates a system for learning and using a large language model. Such a language model is more broadly known as a "Large Language Model" or LLM. This model can correspond to any language model with a large number of parameters (generally on the order of a billion or more) and can be used in any technical field employing artificial intelligence.

[0036] Thus, for example, such a model can be used in applications for visual recognition, speech recognition, processing large amounts of data, decision making, or any other field.

[0037] Referring to Figure 1, the learning and use system 10 comprises a learning device 12 and a use device 14. Devices 12 and 14 are secured, for example, on secure servers. The learning device 12 enables the training of the large language model using training data contained, for example, in a training database 16. This training database 16 includes, in particular, public data and confidential data with varying levels of confidentiality, as will be explained in more detail later.

[0038] The training database 16 can be in the form of a single database or a plurality of databases stored, for example, on different servers in a secure format.

[0039] The training database 16 can be substituted by other data storage systems, such as distributed file systems and cloud storage services.

[0040] To implement the learning of the large language model, the learning device 12 includes an input module 21, a processing module 22 and an output module 23.

[0041] The input module 21 is connected to the training database 16 using any technically feasible form of connection. Thus, for example, the input module 21 is connected to the training database 16 via a computer network known per se.

[0042] The input module 21 thus allows to receive all the training data from this database 16.

[0043] The processing module 22 allows the data received by the input module 21 to be processed in order to learn a large language model as will be explained in more detail later.

[0044] The output module 23 allows the large language model learned by the processing module 22 to be stored, for example, in an external database 26. For this purpose, the output module 23 is connected to this external database 26 by any technically possible means such as a computer network.

[0045] The output module 23 allows the large language model to be stored in the external database 26 in any form usable for this purpose.

[0046] For example, the output module 23 allows this large language model to be stored as a set of parameters forming subsets associated with different levels of confidentiality, as will be explained in more detail later. In some examples, the output module 23 only stores updates to the parameters. Each of the input module 21, processing module 22, and output module 23 is at least partially implemented as software executed by a processor and stored in memory that is also part of the learning system 12.

[0047] In addition or as an alternative, each of these modules 21 to 23 is presented at least partially in the form of a programmable logic circuit such as an FPGA circuit (from the English "Field Programmable Gate Array").

[0048] The usage device 14 allows for the implementation of interactions between a user 28 and the large language model learned by the learning device 12.

[0049] To do this, the usage device 14 includes an input module 31, a processing module 32, an output module 33 and an interaction module 34.

[0050] The input module 31 is connected to the external database 26 by any technically possible means such as a computer network and allows the large language model stored in this database 26 to be retrieved, for example in the form of a set of parameters including subsets associated with different levels of confidentiality.

[0051] The interaction module 34 retrieves each request from user 28 addressed to the large language model. To this end, the interaction module 34 is associated with an interaction interface for user 28, such as means for submitting requests.

[0052] The processing module 32 allows the processing of user requests retrieved by the interaction module 34 as well as the parameters retrieved by the input module 31 and defining the large language model to process these user requests in accordance with this large language model.

[0053] The operation of this module 32 will be explained in more detail later.

[0054] Finally, the output module 33 allows the results of the processing carried out by the processing module 32 to be delivered, for example to the user 28 via in particular the interaction module 34.

[0055] For this purpose, the interaction module 34 can be associated with a user interaction interface 28 to display responses given by the large language model following the processing of these requests.

[0056] The output module 33 also allows interaction with any external system processing the responses given by the large language model.

[0057] In general, it should be understood that user 28 can correspond to a human interacting with the large language model or to another system that must interact with the large language model. As in the case of the learning device 12, each of the input 31, processing 32, output 33, and interaction 34 modules of the user device 14 is at least partially in the form of software executable by a processor and stored in memory that is also part of the user device 14.

[0058] In addition or alternatively, each of these modules 31 to 34 is at least partially presented in the form of a programmable logic circuit such as an FPGA.

[0059] Of course, many other embodiments of these learning and use devices 12, 14 are possible. For example, in some embodiments, these devices 12, 14 may share at least some modules. In other embodiments, these devices 12 and 14 may form a single physical device such as a computer. Also in other embodiments, each of these devices 12, 14 may take the form, for example, of a plurality of physical devices, for example, a plurality of computers capable of processing different tasks in parallel and connected to each other, for example, by a computer network.

[0060] The learning and use system 10 allows the implementation of a learning and use process which will now be explained with reference to Figure 2, which presents an organizational chart of its steps.

[0061] Thus, as illustrated in Figure 2, the learning and use process includes an initial step 110 for implementing a learning process. This step 110 is implemented at least once before the implementation of the next step 120 of the learning and use process.

[0062] Furthermore, this learning process is implemented by the learning device 12 as will be explained in more detail later.

[0063] The next step 120 involves the implementation of a usage process.

[0064] This step 120 is implemented upon receipt of each new request from user 28, once step 110 has been implemented at least once.

[0065] The usage process implemented in this step 120 is implemented by the usage device 14 as will be explained in more detail later.

[0066] The learning process will now be explained in more detail with reference to Figure 3, which presents a flowchart of these steps.

[0067] It is assumed that prior to the implementation of step 210, the training data has been previously collected, cleaned and pre-processed.

[0068] In an initial step 210 of this process, the input module 21 retrieves all the training data available in the database 16 and transmits it to the processing module 22. Then, the processing module 22 processes this data to define or determine for each piece of data a corresponding level of confidentiality.

[0069] In particular, each training data can already be provided with a level of confidentiality so that during this step 210, the processing module 22 extracts this level of confidentiality from each corresponding data.

[0070] For example, to determine the confidentiality level of training data, its associated metadata can be consulted. This metadata can be stored directly with the data (for example, in the file header), or, more commonly, managed in a central database (a catalog) that acts as a reference directory to identify the confidentiality status and permissions of each piece of data.

[0071] A training data partitioning method allows defining the granularity of access control for models refined on this data, whether it concerns users, user groups, roles, resources, specific attributes or other entities related to confidentiality, thus offering various levels of granularity.

[0072] According to one implementation example, there is no direct and unambiguous correlation between the confidentiality levels or authorization policies of training data and those of models derived from that data. However, these model confidentiality levels can, in some cases, be derived from those of the training data, although this relationship is not systematic. Indeed, while the characteristics of the training data significantly influence model confidentiality, other factors also come into play, such as the model architecture, training and regularization methods, and specific confidentiality protection techniques, such as anonymization and differential privacy. For example, a model trained on sensitive data, such as unanonymized medical information, requires enhanced safeguards to prevent the leakage of private information.If this data is anonymized before training, it becomes more difficult for the model to extract personal information from the inputs. However, risks remain if anonymization is poorly applied, especially when dealing with highly detailed data. Complex architectures, such as deep neural networks, can also be more vulnerable to information mining attacks. The use of techniques like dropout or batch normalization can reduce overfitting of sensitive data, while proper regularization limits the model's ability to memorize specific and private details from the training data. Finally, data and artificial intelligence governance, framed by regulations such as the GDPR (i.e.,"General Data Protection Regulation"), ensures respect for confidentiality throughout the model's lifecycle, from data collection to production.

[0073] Alternatively, the confidentiality levels are not assigned prior to the training data, and during this step 210, the processing module 22 performs such an assignment.

[0074] In such a case, step 210 may include a data partitioning substep to identify the confidentiality levels within the dataset received by input module 21. Various partitioning techniques, such as clustering techniques possibly incorporating artificial intelligence, can be used to partition the data into multiple confidentiality levels. These techniques are well-known and will not be described in detail hereafter.

[0075] At the end of this step 210, the data received by the input module 21 are therefore classified into two classes, namely public data and confidential data.

[0076] Public data then corresponds to data that can be accessed by any user without restriction, depending on their access rights.

[0077] Confidential data, on the other hand, can be categorized into different subclasses corresponding to their levels of confidentiality.

[0078] Figure 4 illustrates different examples of assigning confidentiality levels to training data.

[0079] In each of these examples, the reference D designates a corpus of data, that is to say the set of training data received by the input module 21. In addition, in each of these examples, the unhatched parts correspond to the public data which can then be accessed by any user.

[0080] In example A, different levels of confidentiality, namely levels N1 to N3, are assigned to separate data. These levels of confidentiality are independent of each other insofar as, to access a given level of confidentiality, user 28 must expressly have the right to access that particular level of confidentiality.

[0081] This is the case, for example, with personal data relating to a user.

[0082] Thus, in this example A, a single priority level is assigned to each confidential training dataset to create distinct sets of training data with separate and independent confidentiality levels. If necessary, particularly when distinct confidentiality levels need to share the same data, this data can be duplicated to form separate sets.

[0083] In example B of figure 4, the different levels of confidentiality, namely levels N1 to N3, are nested within each other.

[0084] In other words, this occurs when at least one level of confidentiality is included within another level of confidentiality, thus forming a hierarchy between these different levels of confidentiality. A level of confidentiality encompassing another level therefore allows access to all the data of that other level. A level of confidentiality included within another level does not allow access to all the data of that other level.

[0085] This is the case, for example, with the different levels of authorization for data access by different users 28.

[0086] For example, the highest level of confidentiality in the hierarchy (level N3 in the example in Figure 4) includes access to all data with lower confidentiality levels (in the example in Figure 4, confidentiality levels N2 and N1), in addition to the data with that highest level of confidentiality. In turn, confidentiality level N2 includes access only to data with confidentiality level N1 (in addition to data with that same level N2), and N2 does not include access to data with any confidentiality level other than N1.

[0087] In example C of figure 4, the same data can have several levels of confidentiality that are independent of each other.

[0088] Thus, in such a case, a dataset having a particular level of confidentiality may have a first subset having only that level of confidentiality and a second subset having at least two levels of confidentiality, namely the level of confidentiality of the first subset and another level of confidentiality.

[0089] This therefore involves overlaps between different sets of confidential data.

[0090] In this example C, each set of confidential data may correspond to data from a subsidiary that is part of the same parent company.

[0091] These subsidiaries can then share different data between them that correspond to the areas of overlap of the aforementioned data sets.

[0092] In example C of figure 4, the dataset with confidentiality level N1 forms overlapping areas with each of the confidential datasets with confidentiality levels N2 and N3. In these overlapping areas, the corresponding confidential data will therefore have two confidentiality levels, namely N1 and N2 in the first overlapping area and N1 and N3 in the second overlapping area.

[0093] Of course, in a general case, the different levels of confidentiality can be assigned in different ways so that the examples in Figure 4 can be combined with each other.

[0094] In a subsequent step 220, the processing module 22 performs a training of a public model based on the set of public data.

[0095] This step 220 is implemented conditionally only when public data is accessible after step 210.

[0096] Learning the public model may include pre-training on public data in a way that is known in itself.

[0097] In the following step 230, the processing module 22 generates a confidential model for each level of confidentiality determined in step 210.

[0098] For this purpose, the processing module 22 uses all confidential data having the corresponding level of confidentiality.

[0099] There are different ways to carry out this step 230.

[0100] According to a first embodiment, the training of each confidential model includes fine-tuning or any other adaptation of the public model pre-trained during step 220.

[0101] In particular, in such a case, each confidential model presents a fine-tuning of the public model to the corresponding confidential data.

[0102] According to another embodiment, implemented in particular when no public model has been trained, for example due to the absence of public data, each confidential model can be pre-trained and fine-tuned on the corresponding confidential data.

[0103] According to another embodiment, implemented in particular when each confidential model is obtained following a fine-tuning of the public model, PEFT (Parameter-Efficient Fine-Tuning) type methods can be used to avoid updating all parameters but only certain parameters of the public model (in practice less than 1%).

[0104] Other techniques, such as a model compression technique, can be used to reduce the cost of fine-tuning while ensuring good model performance.

[0105] In yet another embodiment, each confidential model is obtained using an expert dedicated to that confidential model. The use of such an expert is known per se according to a method based on the MoE (Mixture-of-Experts) technique, which is known per se.

[0106] An example of an implementation of this embodiment is illustrated in Figure 5.

[0107] According to this example, experts (E) are added to a MoE layer positioned between a preceding CP layer and a following CS layer in a Transformer-based architecture. This architecture is based on a large initial language model pre-trained on a public dataset. From this initial model, several models, each incorporating a group of experts associated with a given confidentiality level, are trained on the confidential data. During the training of each model, only the experts associated with the confidentiality level of the data on which they are trained, and the control function (Fc) responsible for routing inputs to the appropriate experts, are updated. The other parameters of the initial model are frozen. The learned experts and the updates to the control function are saved by associating them with confidentiality levels.During the usage phase, the method adds to the initial model the experts associated with the level of confidentiality of the data which the user and / or the application has access rights to, with the correct update of the control function.

[0108] Aggregating confidentiality levels only requires adding the learned expert groups associated with those confidentiality levels and adjusting the control function by training it on the (or part of the) confidential data associated with those levels.

[0109] Alternatively, the control function Fc can be trained only once after all experts or sets of experts have been trained separately for each confidentiality level. To do this, the control function Fc is integrated into the MoE layer, in parallel with the set of experts. This control function Fc is then trained on samples extracted from the training dataset, while keeping the parameters of the experts and the base model fixed. In this way, the control function Fc learns a probability distribution for the experts, allowing it to determine the probability that a given expert will be selected for each input. The goal is to optimize the model's efficiency by learning to assign inputs to the most appropriate experts.

[0110] For example, experts in the MoE layer can be organized into hierarchical domains by replacing the control function with one that routes inputs to the appropriate domains, and by associating, for example, experts of a given confidentiality level with the same domain. PEFT techniques can be integrated into a MoE expert layer, thus forming a Mixture of Parameter-Efficient Experts (MoPE).

[0111] Instead of modifying the model architecture by adding a MoE layer, a PEFT technique such as LoRA (Low-Rank Adaptation) can be used. This LoRA technique allows the model to be adapted to new tasks without altering its underlying structure, simplifying deployment while preserving parameter efficiency. LoRA, or low-rank adaptation, is a PEFT technique based on an adapter composed of two matrices, A and B, which allows for efficient model adjustment. During training, only the parameters of matrices A and B are modified, while the parameters of the initial model remain unchanged. To this end, a LoRA adapter is fitted to the training data for each level of confidentiality. The adjusted LoRA adapters are then saved in the external database.

[0112] Alternatively, the relationships between privacy levels can be considered and leveraged to tune LoRA adapters more efficiently. One adapter can be a tuned version of another, both having been trained using a shared dataset. This is typically the case when the privacy level associated with the first adapter includes the second (as illustrated by Example B in Figure 4). In this scenario, it is not necessary to save all the parameters of the first adapter. Instead, it is sufficient to save the second adapter (the parent) and a delta of the matrices (A and B) for the first adapter (the child). This approach significantly reduces storage requirements and can be applied transitively to multiple generations of adapters if the dependency relationships extend across several levels.

[0113] To further reduce storage and computation, only one matrix, A or B, can be fitted, and therefore only the delta of matrix A or B is saved. The architectural choice (fine-tuning of A, B, or both), which is a scenario-dependent parameterization, is based on the characteristics of the input training data and the output task, according to the desired trade-off between knowledge preservation and task specialization. This decision relies on the experimentally proven functional roles of matrices A and B: matrix A preserves shared knowledge, while matrix B facilitates task specialization.

[0114] In yet another embodiment, the training of at least some confidential models can be carried out locally and / or at a trusted third party using an offsite-tuning or split-learning method where the model slices are chosen in such a way that the parameters to be updated are trained efficiently and securely. In particular, a portion of these model slices (for example, the last layers of an artificial neural network) can be trained by a third party.

[0115] Finally, according to yet another embodiment, transfer learning can be implemented. This learning method, through knowledge transfer, protects parameter updates after the model has been trained on confidential data.

[0116] Furthermore, the implementation of step 230 may depend on the nature of the confidentiality levels and how they are assigned to the different data, as explained in relation to Figure 4.

[0117] In particular, when the training data only includes datasets with distinct and independent levels of confidentiality (i.e., the case of Example A explained in relation to Figure 4), the training of confidential models associated with these levels of confidentiality can be carried out in parallel on these respective datasets.

[0118] Such a case is illustrated schematically in part A of figure 6.

[0119] As shown in part A of Figure 6, in such a case, the step

[0120] 230 comprises a plurality of substeps 231 during which the confidential models are trained on the respective confidential datasets. These substeps

[0121] 231 are then implemented in parallel.

[0122] In such a case, step 230 further includes a plurality of substeps 232 implemented after the respective steps 231, during which the output module 23 saves, for each confidential model, the corresponding parameters in the external database 26. These substeps 232 are also implemented in parallel.

[0123] In this example implementation, the number n of parallel implementations of substeps 231 and 232 corresponds to the number of distinct and different levels of confidentiality.

[0124] According to another embodiment, applied when the training data only includes datasets with distinct and dependent levels of confidentiality (i.e., the case illustrated by Example B in Figure 4), the training of the confidential models associated with these levels of confidentiality is carried out consecutively.

[0125] In such a case, each subsequent confidential model is trained from a previous confidential model on the entire dataset with the confidentiality level associated with the next confidential model. In turn, the previous confidential model is trained on its corresponding confidential dataset and another preceding model, which may be the public model. When no public model has been trained, the lowest-level confidential model can only be trained on its corresponding confidential dataset.

[0126] This is illustrated schematically in part B of figure 6.

[0127] In such a case, substeps 231 and 232 as explained previously are repeated several times consecutively to train the confidential models.

[0128] In particular, each iteration in this case corresponds to the training of one of the confidential models.

[0129] Thus, during the first iteration of substeps 231 and 232, the confidential model corresponding to the lowest level of confidentiality (level N1 in example B of figure 4) is trained starting from, for example, the public model on the training data having the lowest level of confidentiality.

[0130] During substep 232 of this iteration, the parameters defining this confidential model are then stored by the output module 23 in the external database 26.

[0131] In a subsequent iteration of substeps 231 and 232, a new confidential model is trained on confidential data with the confidentiality level associated with that confidential model, minus the data that has already been used to train previous confidential models. In Example B of Figure 4, this could therefore be data with confidentiality level N2 minus data with confidentiality level N1.

[0132] In addition, to train the confidential model during this iteration of substeps 231 and 232, the processing module 22 uses the confidential model trained during the previous iteration of these substeps.

[0133] According to an example embodiment in which the training data exhibits overlaps of different levels of confidentiality (i.e., the case of Example C in Figure 4), the two techniques as explained in relation to Figure 6 can be combined.

[0134] In particular, in such a case, the training data comprises at least two confidential datasets sharing a common subset of data with at least two distinct and independent levels of confidentiality. To train these datasets, the processing module 22 first trains the shared subset, that is, the subset of data with the two distinct and independent levels of confidentiality. Following this training, an intermediate model is obtained. Then, the processing module 22 simultaneously trains each confidential model on the subset of data with a single level of confidentiality. To do this, the processing module 22 uses the previously obtained intermediate model and then performs further training on this model on each corresponding subset.

[0135] More generally, processing large volumes of data consecutively may not be efficient for performance reasons. Therefore, the data processing for examples B and C in Figure 4 can be parallelized using techniques known in themselves, such as ensemble learning techniques, by averaging the parameters of the two models, or by using other federated learning techniques.

[0136] For examples B and C in Figure 4, learning can also be carried out in a parallelized manner with little communication and reduced computation time from an initial model pre-trained on public data: for a dataset, the initial model is trained on overlapping data and in parallel, the same initial model is trained on non-overlapping data to produce two models which are subsequently merged into a single model by ensemble learning techniques or by averaging the parameters of the two models, or by using other federated learning techniques.

[0137] At the end of step 230, the output module 23 saves in the external database 26 the parameters associated with each of the learned confidential models.

[0138] It is therefore clear that at the end of step 230, the external database 26 includes a plurality of parameters for each confidential model associated with each level of confidentiality.

[0139] This external database 26 can also include a public model learned on public data.

[0140] The collection of confidential models, and possibly the public model, together form a large language model that has been learned through the learning process explained below. It should also be noted that the learning process described below can be implemented not only to create new confidential models but also to update existing ones.

[0141] This is particularly relevant when such an update is required following the receipt of new training data by input module 21. It is therefore clear that during this update, the confidential models are retrained on updated datasets. This can be achieved by applying unlearning approaches such as amnestic unlearning.

[0142] Furthermore, it is clear that in the case of an aggregation of several levels of confidentiality (for example when a user has the right to access data belonging to several different levels of confidentiality) a new confidential model can be generated by the processing module 22 by merging the confidential models associated with these levels of confidentiality that have been learned previously, using, for example, ensemble learning techniques or by averaging the parameters of the two models, or by using other federated learning techniques.

[0143] The process of using the large language model as learned by the learning process explained previously, will now be explained with reference to Figure 7, which presents a flowchart of its steps.

[0144] During an initial step 310, the input module 31 of the usage device 14 receives a usage request from the user 28 via, in particular, the interaction module 34.

[0145] Such a request for use may include, in particular, a request addressed to the large language model.

[0146] This request is then transmitted to the processing module 32 of the usage device 14.

[0147] In the next step 320, the processing module 32 evaluates the access rights of user 28. In particular, these access rights determine at least one level of confidentiality of the training data accessible by user 28. Alternatively, the user's access rights are evaluated directly by the interaction module 34 before the request is transmitted to the processing module 32.

[0148] In the next step 330, the processing module 32 retrieves, via the input module 31, the confidential model corresponding to the level of confidentiality determined by the user's access rights.

[0149] In the next step 340, the usage device 14 implements the user's interactions with the corresponding confidential model.

[0150] In particular, during this step, the interaction module 34 of the usage device receives requests from the user addressed to the large language model.

[0151] Then, the processing module 32 executes the corresponding confidential model to obtain a response to this user request. Next, the output module 33 transmits the response from the corresponding confidential model to the user 28, notably via the interaction module 34.

[0152] Several examples of how to implement these steps 310 to 340 are possible.

[0153] Thus, according to the implementation example illustrated in Figure 8, each request from user 28 (including the request for use) is first received by a policy enforcement point (PEP).

[0154] This point monitors access and applies access decisions D based on the user's access rights.

[0155] Access decisions are returned by a Policy Decision Point (PDP) which is responsible for evaluating authorization requests by comparing them to security policies.

[0156] These PEP and PDP components can be operated as on-premises services or via a cloud service. For example, these services can be integrated into interaction module 34 of the user device 14, as explained previously.

[0157] In this context, when a user sends an R request for a response from the large language model, this request is first intercepted by the PEP component. After user authentication, the PEP component verifies the user's access rights via the PDP component. Following this rights verification, the user's request is then forwarded to the corresponding confidential MC model, which is executed, for example, by the processing module 32. Access rights verification by the PEP component can also be achieved using the concept of zero trust, in which no user or application has access to the learned model unless authorized. Access control can guarantee least-privileged access to the model.

[0158] Based on the user's access rights to the confidential data used by the corresponding confidential model, an Rp response to the request is then generated by the model and then transmitted to the user.

[0159] Figure 9 illustrates another example of the implementation of the usage process as explained previously.

[0160] According to this example, when user 28 makes a request to use the large language model, they first receive a C key from, for example, a key database. This key is determined based on the user's access rights.

[0161] In such a case, the confidential model corresponding to the user's confidentiality level can be stored in the external database 26 in encrypted form with a decryption key corresponding to the user's key. The encryption can be attribute-based.

[0162] The C key can be delivered by a key generation and distribution system, such as a public key infrastructure (PKI for "Public Key Infrastructure").

[0163] Thus, user 28 can interact directly with this confidential model MC once its parameters have been retrieved and decrypted by a decryption unit 30, for example part of the processing module 32 or the interaction module 34, using the user's decryption key.

[0164] Of course, the invention is not restricted to these modes and examples of embodiment and can be implemented using other modes and other examples corresponding for example to combinations of those explained above.

[0165] For example, the decryption unit 30 can also reside in a user-specific module or an external or cloud-based module.

[0166] The present invention therefore offers a number of advantages.

[0167] First, the primary advantage of the invention is that it allows for the protection of confidential data when used by a large language model for training. The invention also protects the model itself, which can be considered sensitive.

[0168] A second advantage of the invention is that it allows for differentiated access control to the confidential data used by the model. Access control can thus be implemented with a fine-grained approach, using several access control models.

[0169] By implementing access controls, the invention thus offers a method of data governance used by a large language model, making it possible to reduce the risk of unauthorized access to data, such as a breach of privacy.

Claims

24 DEMANDS 1. A method for a user to use a large language model, the large language model being learned on training data of different confidentiality levels and taking the form of a plurality of confidential models, each confidential model being associated with a confidentiality level; the method of use comprising the following steps: - acquisition (310) of a user request; - evaluation (320) of user access rights, access rights determining a level of confidentiality of training data accessible by the user; - retrieval (330) of a confidential model corresponding to the level of confidentiality determined by the user's access rights; - implementation (340) of user interactions with the retrieved confidential model.

2. A method of use according to claim 1, wherein each interaction of the user with the corresponding confidential model is carried out via a policy application point capable of applying a security policy.

3. Method of use according to claim 1, wherein each interaction of the user with the corresponding confidential model takes place directly after decryption of the confidential model retrieved in encrypted form, the decryption being carried out with a key selected according to the user's access rights.

4. A method for learning a large language model on training data of varying levels of confidentiality; the learning method comprising the following steps: - constitution (210) of training data and determination for each training data of at least one level of confidentiality; - for each level of confidentiality, training (230) a confidential model on the training data corresponding to that level of confidentiality and saving this confidential model with the corresponding level of confidentiality; the large language model taking the form of a plurality of confidential models; in which, when the training data include datasets having distinct and independent or dependent levels of confidentiality, the training of the confidential models associated with these levels of confidentiality is carried out in parallel, each confidential model is obtained by merging, by set learning or federated techniques, intermediate models learned in parallel on subsets of data having the same levels of confidentiality and constituting together the data associated with the confidential level of said confidential model.

5. A learning method according to claim 4, wherein where the training data comprises datasets having distinct and independent levels of confidentiality, the training of the confidential models associated with these levels of confidentiality is carried out in parallel on the respective datasets.

6. A learning method according to claim 4 or 5, wherein where the training data comprises datasets having distinct and dependent levels of confidentiality, the training of the confidential models associated with these levels of confidentiality is carried out consecutively, each subsequent confidential model being trained from a previous confidential model on the dataset having the level of confidentiality associated with the next confidential model, the previous confidential model being trained on a training dataset having a level of confidentiality included in that associated with the next confidential model.

7. A learning method according to any one of claims 4 to 6, wherein where the training data comprises a dataset including a subset of data having at least two distinct and independent levels of confidentiality, the training of the confidential models associated with these levels of confidentiality is carried out in parallel from an intermediate model previously trained on said subset of data having at least two distinct levels of confidentiality, each of said confidential models being further trained on a subset of data having only the level of confidentiality associated with that confidential model.

8. A learning method according to any one of claims 4 to 7, further comprising a step (220) of training a public model from a set of public data; each confidential model being advantageously trained from the public model.

9. A learning method according to any one of claims 4 to 8, wherein the training of a confidential model includes pre-training a model from training data having the level of confidentiality associated with that confidential model and / or fine-tuning an initial model using training data having the level of confidentiality associated with that confidential model.

10. Learning method according to claim 9, wherein in the event of fine-tuning, parameters defining the initial model are frozen, each confidential model advantageously presenting a MoE expert trained on the training data having the level of confidentiality associated with that model.

11. A learning method according to claim 10, wherein, after training the MoE experts, a routing control function is trained separately on a sampling of the training data while keeping the parameters of said MoE experts and the base model frozen, so as to learn a probability distribution of expert selection.

12. A learning method according to any one of claims 9 to 11, in which fine-tuning is achieved using LoRA-type adapters, and where a hierarchical relationship exists between privacy levels, an adapter associated with an included level is saved as a parameter delta relative to an adapter associated with the encompassing level.

13. A learning method according to any one of claims 4 to 12, wherein the determination for each training data of at least one level of confidentiality includes a partitioning of the training data to identify levels of confidentiality. 27 14. A method for a user to learn and use a large language model, comprising the following steps: - implementation (110) of a learning method according to any one of claims 4 to 13; - implementation (120) of a usage method according to any one of claims 1 to 3.

15. A learning and use system (10) of a large language model by a user, comprising: - a learning device (12) comprising technical means (21, 22, 23) configured to implement a learning process according to any one of claims 4 to 13; - a usage device (14) comprising technical means (31, 32, 33) configured to implement a usage method according to any one of claims 1 to 3.

Images