Method and apparatus for posteriorly training machine learning system

Abstract

To provide a method for posteriorly training a machine learning system (60) on the basis of a gradient using non-disclosure training data (T) with respect to a target task.SOLUTION: A method includes steps of: adding a parameter (θ) to a previously trained machine learning system (60) (S1); and adapting the added parameter (θ) using non-disclosure training data (T) by a difference private back propagation method, the added parameter (θ) being adapted to a target task (S2).SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a method for gradient-based post-training of a machine learning system using private training data on a target task. The present invention also relates to a computer program implemented in said method, to a machine-readable data carrier comprising such a computer program, and to a computer configured to perform said method. [Background technology]

[0002] Prior Art Machine learning (ML) systems generally exhibit a tendency to "remember" information contained in training data. This can manifest itself in the way that the machine learning system reproduces information from the training data in inference, for example in the form of sentences / specific strings of characters or image parts / segments of images from the dataset on which the learning system was trained. If private data is used in training, the inference may potentially reveal confidential information, for example from data about individuals or "internal" data of a company.

[0003] arXiv:1607.00133[stat.ML] and arXiv:1911.11607[cs.LG] propose a differentially private training method for machine learning systems that balances the information gained from the training dates with the limitation of information disclosure to protect accidental privacy information from the training dates. Summary of the Invention [Problem to be solved by the invention]

[0004] Advantages of the invention A preferred aspect of the present invention relates to the adaptation of a small number of parameters of a machine learning system in post-training of the machine learning system when private training data is used in the post-training, since this may lead to convergence of the adaptation with a small number of training steps. Such a reduction in convergence time is advantageous, since in the case of shorter training, less "information" about the used private (i.e. private) training data can flow into the adaptation of the parameters. Furthermore, under the differentially private stochastic gradient descent method (see also arXiv:1607.00133 [stat.ML]), there is a maximum number of training iterations that can be performed depending on a pre-set privacy budget. In the case of adaptation of a small number of, but possibly particularly relevant, parameters of a machine learning system in post-training according to the present invention, there is a higher probability that the small number of parameters to be adapted in the post-training are adapted within the maximum number of training iterations so that the machine learning system generalizes better in inference. That is, a machine learning system trained according to the present invention can perform better than a machine learning system in which a larger number of parameters are adapted in post-training under a similarly preset privacy budget. The privacy budget here measures how much information acquisition by the training date is restricted to protect privacy, and may be determined by the preset of parameters ε and δ, especially in the case of (ε, δ) differential privacy (see also https: / / doi.org / 10.1007 / 11681878_14). By adapting a small number of intended parameters based on private information, the private information is compact and is not entirely included in the machine learning system, which reduces "memorization." [Means for solving the problem]

[0005] Disclosure of the Invention In a first aspect, the present invention relates to a computer-implemented method for gradient-based post-training of a machine learning system on a target task. Post-training can be used, for example, to adapt the learning system to the generation of synthetic data, e.g. image data, for training an image processor / image classifier. The image classifier can now classify or segment images, in particular objects in images, and / or visually highlight objects. The training data used in the methods described herein is private (hereinafter also referred to as "private"). Thus, the training data may include confidential information, such as information about individuals or company internal information. For example, the training data may include patient-related imaging from medical diagnostics. The training data may also include company-specific private imaging with driving conditions from autonomous driving or imaging from automated optical inspection of workpieces, etc. The training data may also include imaging recorded with new, possibly specially adapted, camera sensors. It is also envisaged that the training data may include audio data or other sensor data of further modalities, such as data recorded with radar or LiDAR sensors. The machine learning system is provided as a pre-trained model according to the method described herein, i.e. the parameters of the machine learning system are adapted to the elementary task in pre-training. The elementary task may, for example, generally consist in the generation of an image whose context or content may be configurable but not limited to the representation of a particular content, while the goal task may, in this case, consist in the generation of a particular image with the characteristics of a particular content. Alternatively, the elementary task may include the embedding of different modalities, for example, images and associated text, into a common representation space, i.e. the elementary task consists in finding an appropriate mapping of both, for example, images and text (i.e., in general, data of different modalities) into a common representation space.Thus, the corresponding target task may include, for example, classification of image data based on text input. The pre-trained machine learning system may in particular be a foundation model (FM). The foundation model includes a number of parameters that are adapted in pre-training with a number of public foundation training data for the execution of the foundation model's basic task. In pre-training of the foundation model, the adaptation of the parameters can in particular be performed in a self-supervised manner, i.e. with unlabeled training data. In post-training, a smaller number of training data is used, in which case the training data of the post-training is adapted to a specific target task. According to the method described here and below, the training data in the post-training is non-public, i.e. private training data. In post-training, the foundation model may be post-adjusted with the pre-trained parameters, i.e. with the post-training, the pre-trained parameters of the foundation model may be applied with small deviations from the values ​​obtained from the foundation training, respectively. In post-training, the adaptation of the model's parameters can be monitored, i.e. with labeled training data, in which case self-supervised training is also possible for the post-training. In post-training, some or most of the pre-trained parameters of the underlying model may be retained / frozen, and only the parameters that are not retained are adapted.

[0006] In one step of the method proposed here and below, further parameters are added to the pre-trained machine learning system. These further parameters can be added by inserting at least one additional layer into the architecture of the machine learning system. Alternatively or additionally, further parameters can be added to the machine learning system by decomposing at least one weight matrix to be adapted in post-training into a sum by two addends in a layer of the machine learning system. The first addend is here given by the pre-trained weight matrix to which it belongs, i.e. the weight matrix whose parameters are adapted in pre-training and are specifically retained in post-training. The second addend comprises a matrix product of two further matrices, where these two further matrices are parameterized with the further parameters to be adapted in post-training. The rank of these two further matrices is lower than the rank of the pre-trained weight matrix in the first addend, respectively.

[0007] As mentioned above, the pre-trained weight matrix to which the further term to be adapted is added in the post-training, given by the matrix product of two further matrices, can be, for example, a weight matrix in the transformer layer, in particular a weight matrix in the (self-)attention head of the transformer layer. In a further method step, the added further parameters are adapted for the target task with the private training data by differential private backpropagation. In the method described here, in particular, only the added further parameters, i.e. exclusively, the added further parameters can be adapted in the post-training.

[0008] The advantage of the method proposed here and below is that the machine learning system can be adapted to the technical target task efficiently with private training data, i.e. with a relatively small set of possibly specially cut training data, and therefore in a shorter time, since the private training data includes, for example, image data related to specific cases or situations related to the target task, possibly exclusively. For example, private image data including a driving environment showing a specific safety-critical situation, such as a situation involving pedestrians, can be used for autonomous driving. It is also possible to use private image data from a workpiece inspection, for example showing a specific rare defect of the workpiece.

[0009] According to a preferred embodiment of the method, the differentially private backpropagation method includes a stepwise minimization of a cost function. In general, a machine learning system is configured to receive input data and map them to output data by mathematical operations at multiple layers of the machine learning system, or often at multiple layers. In the (pre / post) training of the machine learning system, the cost function can then measure how well the output data determined by the learning system matches the corresponding pre-set output data.

[0010] In one step of differentially private backpropagation, averaged and noisy gradients of the cost function can be determined, respectively. Here, the averaged and noisy gradients include a weighted sum of contributions of the individual private training data to the gradient of the cost function that are limited in terms of the magnitude of the gradients, and a noise term can be added. (Post)training of the machine learning system using differentially private backpropagation can generally be performed in multiple steps, also called training epochs. Within one step or one (training) epoch, respectively, a subset, also called a batch, of the training data can be received by the learning system. For each training date from the batch, the learning system can further determine an output date according to the training task. In a post-training epoch of the machine learning system for adapting parameters added to the learning system using differentially private backpropagation, for example, the private training data {x1, x2, ..., x N If we denote the parameters to be adapted in the learning system by θ, then the cost function (also called loss function) considered during the backpropagation process is given by the following equation for the corresponding training epoch:

number

number

number

number

[0011] In order to adapt the parameters to be adapted in the machine learning system in the direction opposite to the steepest gradient increase, the averaged gradients normally used in the backpropagation method in the training epochs can be replaced by averaged, noisy gradients with limited contribution in terms of magnitude obtained according to the steps described above and / or below. By using the averaged and noisy gradients described above and / or below in the backpropagation method, the method becomes a differentially private backpropagation method, also referred to as a differentially private backpropagation technique.

[0012] According to a preferred embodiment, in the architecture of the pre-trained machine learning system, at least two adapter layers are inserted in each of the last L transformer blocks. In the transformer blocks, one adapter layer can be inserted after the self-attention outprojection and one after the multi-layer perceptron. For example, L may be given by L=30. The parameters added by the insertion of the adapter layers can indicate further parameters of the machine learning system. These added parameters can be adapted by the differentially private backpropagation method described above and below. For example, each adapter layer can include a feedforward downprojection to the bottleneck, a nonlinearity, and a feedforward upprojection. In this case, the parameters of the downprojection and upprojection added by the insertion of the adapter layers can be adapted. The nonlinearity can be given by, for example, reLU. Alternatively, the nonlinearity can be given by GELU (Gaussian Error Linear Unit), especially in the case of the transformer.

[0013] According to an exemplary embodiment, the machine learning system can prepend a prefix of length K to at least the last L (e.g., L=30) transformer blocks, respectively. The key vectors and value vectors of the self-attention layer in the transformer block can then be modified by the prefix of the transformer block to which it belongs. The key vectors and value vectors in the self-attention layer of the transformer block can be obtained by multiplying the input vector received from the self-attention layer by the key weight matrix or value weight matrix of the corresponding self-attention layer. Additionally, a gating mechanism with a scalar parameter can also be introduced. For the modification of the key vectors and value vectors by the prefix and the gating mechanism, see, for example, arXiv:2303.16199[cs.CV]. The parameters associated with the prefix and the scalar parameters of the gating mechanism are added parameters, which can be added to the machine learning system before post-training and adapted by differentially private backpropagation.

[0014] According to an exemplary embodiment, a further parameter to be added and adapted in post-training is a weight matrix W to be adapted in post-training in a layer of a machine learning system. i Let us denote the pre-trained weight matrix W i,0 and the weight matrix W i,0 The sum W with the product of two further matrices with rank lower than the rank of i =W i,0 +W i,A ·W i,B where the elements of the two further matrices are the additional parameters to be fitted in the post-training. i =W i,0 +W i,A ·W i,B The first term of is the A×B weight matrix W for the corresponding layer. i,0 The entries of which are adapted in pre-training and are not changed in post-training. The second term is the matrix Wi,A and W i,B where W i,A Let A,B denote an A×r matrix, and Wi,B denote an r×B matrix whose entries are the added parameters, respectively, that are adapted by differentially private backpropagation. r can be freely chosen and the matrix W i,A and W i,B Here, the hyperparameter r represents the hyperparameter that determines the rank of the matrix W i,0 r may take a value smaller than the rank value of W. Preferably, r may take a value r≦16. For example, r may take a value r=4. For example, the additional parameter, i.e., the matrix W i,A The entries of can be initialized randomly (e.g., with a Gaussian distribution). The parameters, i.e., the matrix W i,B The matrix entries of may be set to zero initially, i.e., at the start of post-training.

[0015] According to an exemplary embodiment, firstly a particularly suitable value for the hyperparameter r introduced in the previous example can be sought / determined. For this purpose, the method described in the previous example can be run multiple times using public training data with different pre-set values ​​of the hyperparameter r. For example, the method can be run with values ​​of r=1, 2, ... 16, respectively. In this way, multiple machine learning systems M each with adapted and added parameters can be generated. r These machine learning systems M r can be verified on the public validation data, for example by determining the performance metric that it belongs to. In this case, the public validation data is used to test the machine learning system M r The public training data used in the post-training of the machine learning system M does not match the public training data, i.e., the public validation data does not include training dates from the public training data described above. rThe determined performance metrics of the models M can be compared with each other and with the corresponding r For example, the method steps described in the context of the previous example may be used to select the r with the best performance metric using the private training data with the selected hyperparameter r. i,A , W i,B can be run to fit the parameters added to

[0016] Furthermore, the invention relates to a computer program comprising machine-readable instructions, which, when executed on one or more computers, cause the computer to carry out one of the methods according to the invention described above and below. Likewise, the invention comprises a machine-readable data carrier on which such a computer program is stored, as well as a computer equipped with such a computer program and / or such a machine-readable data carrier.

[0017] In the following, embodiments of the invention are explained in more detail with reference to the accompanying drawings. [Brief description of the drawings]

[0018] [Figure 1] FIG. 2 is a schematic diagram outlining the information flow of the methods described herein. [Diagram 2] FIG. 1 is a schematic diagram illustrating an apparatus for carrying out the methods described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Description of the embodiment FIG. 1 shows an overview of the information flow of a method for gradient-based post-training of a machine learning system with private training data for a target task. The machine learning system includes pre-trained parameters, which are adapted to perform a basic task in pre-training of the learning system. In method step S1, further parameters are added to the machine learning system. This can be done by inserting at least one additional layer in the architecture of the learning system. Alternatively or additionally, further parameters can be added in step S1, where the weight matrix to be adapted in post-training is shown as a sum from a pre-trained weight matrix and an additional weight matrix in a layer of the machine learning system, and the additional weight matrix is ​​shown as a product of two further matrices, each having a lower rank. In this case, the elements of the two further matrices give the further added parameters. In step S2, the added further parameters are adapted with the private training data by differentially private backpropagation for the target task.

[0020] 2 illustrates an example of an apparatus 140 for performing the methods described herein for gradient-based post-training of a machine learning system 60 with private training data T on a target task. The parameters of the machine learning system 60 are adapted to a base task in pre-training, e.g., using public training data. The private training data T in post-training the machine learning system is a set of input signals x i The set of private training data T further includes the input signal x i corresponds to the input signal x i Each input signal x can be characterized by a classification of i Desired output signal for t i may include.

[0021] For example, if the basic task consists in generating an image whose context or content may be configurable but not limited to representing a particular content, the goal task may then consist in generating a particular image with characteristics of a particular content, e.g. an image of a driving situation, a workpiece in workpiece inspection, or a medical diagnostic image (such as a chest X-ray).

[0022] To the parameters of the machine learning system 60 already adapted in pre-training, further parameters θ to be adapted in post-training are added. This can be done by inserting at least one additional layer in the architecture of the learning system. Alternatively or additionally, further parameters can be obtained, where the weight matrix to be adapted in post-training is represented as a sum from the weight matrix pre-trained in the layer of the machine learning system and a matrix product of two further matrices, each having a lower rank, and the elements of these two further matrices represent the further parameters to be added in this case.

[0023] For post-training, the training data unit 150 accesses a computer-implemented database St2, which provides a data set of private training data T. The training data unit 150 selects, preferably randomly, at least one input signal x from the private training data set T. i and the corresponding desired output signal t i and calculate the input signal x i to the machine learning system 60. The machine learning system 60 receives the input signal x i Based on the output signal y i Request.

[0024] Desired output signal t i and the obtained output signal y i is transmitted to the modification unit 180.

[0025] Then, the desired output signal t i and the obtained output signal y i Based on this, the modification unit 180 determines new parameters θ′ for the machine learning system 60. To this end, the modification unit 180 uses a cost function (English: Loss Function) to reduce the desired output signal t i and the obtained output signal y i The cost function is calculated by comparing the output signal y i is the desired output signal t i A first loss value is determined that characterizes how much the deviation from

[0026] The modification unit 180 finds new parameters θ′ by differentially private backpropagation based on the first loss value. In an embodiment, this occurs when averaged and noisy gradients, each with limited contributions with respect to the magnitude of the individual training data, are used in the gradient descent method, preferably in stochastic gradient descent, Adam or AdamW.

[0027] The new parameters θ′ found are stored in the model parameter memory St1. The described training is preferably repeated a predetermined number of iteration steps or until the initial loss value falls below a predetermined threshold. Alternatively or additionally, it is also envisaged that the training is terminated when the average initial loss value for the test data set or the validation data set falls below a predetermined threshold. In at least one of the iterations, the new parameters θ′ determined in the preceding iteration are used as parameters θ of the machine learning system 60.

[0028] Further, the apparatus 140 may include at least one processor 145 and at least one machine-readable storage medium 146 that includes instructions that, when executed by the processor 145, cause the apparatus 140 to perform a method according to one of the aspects of the present invention.

[0029] The term "computer" includes any device for processing preconfigurable computational rules, which may exist in the form of software or hardware, or a mixture of software and hardware.

[0030] In general, a plurality may be understood as indexed, i.e., each element of the plurality is assigned a unique index, preferably by assigning sequentially consecutive integers to the elements in the plurality. Preferably, if the plurality comprises N elements, where N is the number of elements in the plurality, the elements are assigned integers from 1 to N.

Claims

1. 1. A computer-implemented method for gradient-based post-training of a machine learning system (60) using private training data (T) on a target task, comprising: The parameters of the machine learning system (T) are adapted to a basic task in pre-training, The method comprises the following steps: - adding (S1) a further parameter (θ) to said pre-trained machine learning system (60), The further parameter (θ) is added by inserting at least one additional layer parameterized with the further parameter (θ) into the architecture of the learning system; and / or the further parameters (θ) are added by decomposing at least one weight matrix to be adapted in post-training into a sum from a pre-trained weight matrix in a layer of the machine learning system and further addends added in post-training; the further addend being given by the matrix product of two further matrices; The two further matrices are parameterized with the further parameter (θ), The parameters of the pre-trained weight matrix are adapted in the pre-training and are retained in the post-training; a step (S1), in which the ranks of the two further matrices are each lower than the rank of the pre-trained weight matrix; - adapting (S2) the added further parameters (θ) using private training data (T) by differentially private backpropagation, the added parameters (θ) being adapted with respect to a target task; A method comprising:

2. The differentially private backpropagation method includes a stepwise minimization of a cost function; In one step of the backpropagation method, the averaged and noisy gradients of the cost function are found, respectively; The averaged and noisy gradient is - a weighted sum of contributions of each private training data to the gradient of the cost function, the contributions being limited with respect to the magnitude of the gradient of the individual private training data, The method of claim 1, in which an additive noise term is added.

3. In the architecture of the pre-trained machine learning system (60), at least two adapter layers are inserted into each of the last L transformer blocks; The method of claim 1 or 2, wherein a parameter (θ) added by the insertion of the adapter layer is adapted by the differentially private backpropagation method.

4. The machine learning system (60) includes a step of prepending a prefix of length K to at least the last L transformer blocks, The key vectors and value vectors in the self-attention layer of a transformer block are each modifiable by the prefix of the transformer block to which they belong; An additional gating mechanism with a scalar parameter is introduced, 3. The method of claim 1 or 2, wherein the parameter bound to the prefix and the scalar parameter of the gating mechanism are added parameters (θ), and the added parameters (θ) are adapted using differentially private backpropagation.

5. The further parameter (θ) is the weight matrix W to be adapted in post-training in the layers of the machine learning system. ｉ Let W be the pre-trained weight matrix ｉ，０ and the weight matrix W ｉ，０ and the sum W ｉ =W ｉ，０ +W ｉ，Ａ ・W ｉ，Ｂ is added by decomposing into The elements of the two further matrices are the additional parameters (θ) to be fitted in the post-training, Where: W ｉ，０ represents the A×B weight matrix of the machine learning system, corresponding to the layers whose entries were adapted in pre-training and are not changed; W ｉ，Ａ represents an A×r matrix, and W ｉ，Ｂ represents an r×B matrix whose entries are the added parameters, which are adapted with respect to the target task by differentially private backpropagation; r is freely selectable and the matrix W ｉ，Ａ , W ｉ，Ｂ The method according to claim 1 or 2, wherein the rank of the rank of the vector is determined by the hyperparameter.

6. First, the hyperparameter r is determined by the following method steps: performing the method of claim 5 multiple times with different preset values ​​of the hyperparameter r and public training data, thereby resulting in machine learning systems with additional adapted parameters; Validating the resulting machine learning system on public validation data by determining respective performance metrics associated with the system; selecting the hyperparameter r for the resulting learning system having the best performance metric; 6. Performing the method steps of claim 5 to adapt the added parameters (θ) of the machine learning system with private training data having the selected hyperparameters r; The method of claim 5, wherein the method is determined according to

7. An apparatus (140) configured to perform the method of any one of claims 1 to 6.

8. A computer program adapted to perform the method according to any one of claims 1 to 6, when the computer program is executed by a processor (145).

9. A machine-readable storage medium (146) having stored thereon a computer program according to claim 8.

Images