Apparatus and computer-implemented method for adapting models to task

Abstract

Apparatus, data structure and method for adapting a model to a task, where the method comprises providing a model, where the model comprises a layer, in particular linear, for mapping a multi-dimensional input of the layer to a multi-dimensional output of the layer according to weights, where the model comprises expert and router gates for adapting the model to different tasks, wherein the method comprises: providing an input to a router gate; determining an output of the expert from an output of the router gate in response to the input; modifying the model according to the output of the expert; mapping the input to an output of the model using the modified layer; training a first expert in the experts by using a first training method according to the output of the model; training a second expert in the experts by using a second training method according to the output of the model; and keeping the weight and the second expert unchanged in the training using the first training method, and keeping the weight and the first expert unchanged in the training using the second training method.

Description

Technical Field

[0001] This invention relates to an apparatus for adapting a model to a task and a computer-implemented method. Background Technology

[0002] In deep learning, models can be adapted to tasks.

[0003] R.A. Jacobs, M.M. Jordan, S.J. Nowlan, and G.E. Hinton, “Adaptive mixtures of local experts,” *Neural Computation*, vol. 3, pp. 79–87, 1991, describes Mixture of Experts (MoE). MoE is a type of neural network architecture that allows model parts intended for different tasks to be combined into a single model. This is achieved through a routing mechanism that allows individual model parts (called experts) to be trained separately from the other parts for the corresponding task. This routing mechanism allows each expert to focus on a specific data type selected by a learnable router-gated network.

[0004] Training or fine-tuning a MoE model requires a very large memory capacity to store the large number of parameters needed by all the individual experts.

[0005] W. Fedus, B. Zoph, and N. Shazeer, “Switch Transformers: Scaling to trillion-parameter models with simple and efficient sparsity,” published in 2022, disclosed the Switch Transformer.

[0006] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, ANGomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” published in 2023, disclosed the converter architecture.

[0007] The switching converter is proposed as an application of MoE in converter architecture, demonstrating how performance can be improved by replacing the feed-forward layer at the end of each attention module with a MoE layer.

[0008] Training a large number of experts to use a switching converter requires a large amount of memory. Summary of the Invention

[0009] The methods implemented using devices and computers efficiently adapt the model to the task. An example task is: The output is categorized according to the different types of input data; Output digital images based on different types of input data; Output audio data based on different types of input data; Output video data based on different types of input data; or Output virtual sensor data based on different types of input data.

[0010] A method for adapting a model to a task includes providing a model, wherein the model includes, in particular, linear layers for mapping multidimensional inputs of the layers to multidimensional outputs of the layers according to weights, wherein the model includes expert and router gates for adapting the model to different tasks, wherein the method includes: providing inputs to the router gates; determining the outputs of the expert based on the outputs of the router gates in response to the inputs; modifying the model based on the outputs of the expert; mapping the inputs to the outputs of the model using the modified layers; training a first expert among the experts using a first training method based on the outputs of the model, and training a second expert among the experts using a second training method based on the outputs of the model; and keeping the weights and the second expert unchanged during training using the first training method, and keeping the weights and the first expert unchanged during training using the second training method.

[0011] Modifying the model may include: determining the output of the first expert on a weight-by-weight basis; modifying the weights based on the weights of the layers and the output of the first expert on a weight-by-weight basis; and determining the output of the model based on the modified weights.

[0012] Modifying the model may include: determining the multidimensional output of the first expert based on the dimensions of the multidimensional output of the layers; and determining the model output based on the dimension-wise summation of the multidimensional output of the layers and the multidimensional output of the first expert.

[0013] Modifying the model may include: determining the output of the second expert on a weight-wise basis; modifying the weights based on the weight-wise multiplication of the layer weights and the second expert output; and determining the model output based on the modified weights.

[0014] Modifying the model may include: determining the multidimensional output of the second expert based on the dimensions of the multidimensional output of the layer, and determining the output of the model based on the dimension-wise summation of the multidimensional output of the layer and the multidimensional output of the second expert.

[0015] This method may include: training router gates based on the output of the model.

[0016] The output of the first expert can be represented as a transformation matrix used for matrix addition with the weight matrix representing the weights, where training the first expert involves learning this transformation matrix.

[0017] The method may include providing multiple experts with their own transformation matrices for matrix addition, wherein the ranks of the transformation matrices provided for matrix addition are different from each other.

[0018] The output of the second expert can represent a transformation matrix used for matrix multiplication with the weight matrix representing the weights, wherein training the second expert includes learning this transformation matrix.

[0019] The method may include: providing a common matrix for matrix multiplication operations to multiple experts, providing different scalars to multiple experts for scaling the common transformation matrix to the transformation matrix, and training the scalars of the experts based on the output of the model.

[0020] The model may include multiple layers, particularly linear ones, wherein adapting the model includes adapting the layers using corresponding experts and corresponding router gates, wherein adapting the layers includes: providing the input of the corresponding layer to the router gate of the corresponding layer, determining the output of the expert of the corresponding layer based on the output of the router gate of the corresponding layer in response to the input of the corresponding layer, modifying the model based on the output of the expert of the corresponding layer, and training the expert of the corresponding layer of the model.

[0021] The model can be configured to determine the input based on the model's input, wherein the training data includes pairs of model inputs and ground truth values ​​for the model's output, wherein the inputs represent or include sensor signals, and wherein the outputs and ground truth values ​​represent or include classifications of sensor signals, or wherein the inputs represent or include text, and the outputs and ground truth values ​​represent or include digital images and / or audio signals, or wherein the inputs represent or include text and semantic graphs, and the outputs and ground truth values ​​represent or include digital images, or wherein the inputs represent or include at least one operational quantity of a technical system, and the outputs and ground truth values ​​represent or include sensor signals.

[0022] The method may include receiving input to a model, the input including or representing information about a technical system; determining the output of an adapted model, the adapted model outputting the output in response to the input of the model; and outputting the output of the adapted model and / or operating the technical system based on the output or the adapted model.

[0023] An apparatus for adapting a model to a task includes at least one processor and at least one memory, wherein the at least one memory includes instructions executable by the at least one processor, and the instructions, when executed by the at least one processor, cause the apparatus to perform the method.

[0024] A computer program may be provided, wherein the computer program includes instructions that can be executed by a computer, and the instructions, when executed by the computer, cause the computer to perform the method.

[0025] A data structure, particularly a computer-implemented data structure, can be provided for adapting a model to a task. The data structure includes at least one data field for the model, wherein the model includes, in particular, linear layers for mapping multidimensional inputs of the layers to multidimensional outputs of the layers according to weights. The model includes expert and router gates for adapting the model to different tasks. The data structure includes at least one data field for inputs to the router gates. The data structure includes at least one data field for the output of the expert determined based on the output of the router gate in response to the inputs. The data structure includes at least one data field for a modified layer determined by modifying the model according to the expert's output. The data structure includes at least one data field for training a first expert among the experts using a first training method based on the model's output. The data structure also includes at least one data field for training a second expert among the experts using a second training method based on the model's output, and for maintaining the weights and the second expert unchanged during training using the first training method, and maintaining the weights and the first expert unchanged during training using the second training method. Attached Figure Description

[0026] Further embodiments will be derived from the following description and accompanying drawings. In the drawings: Figure 1 A schematic depiction of the apparatus used to adapt the model to the task; Figure 2 A portion of the first example of the model is schematically depicted; Figure 3 A portion of the second example of the model is schematically depicted; Figure 4 A portion of the third example of the model is schematically depicted; Figure 5 A portion of the fourth example of the model is schematically depicted; Figure 6 A flowchart schematically depicts the steps of a method for adapting a model to a task; Figure 7 The data structures used to adapt the model to the task are illustrated schematically. Detailed Implementation

[0027] Figure 1 A device 100 is schematically depicted. The device 100 includes at least one processor 102 and at least one memory 104. The at least one memory 104 stores instructions. The at least one processor 102 is configured to execute instructions.

[0028] The apparatus 100 is configured to perform a method for adapting the model 106 to a task. Instructions, when executed by at least one processor 102, cause the apparatus 100 to perform the method.

[0029] In the example, at least one memory 104 stores model 106.

[0030] Model 106 can be configured to receive inputs that include or represent information about the technical system 108. Model 106 can be configured to determine the output of model 106 for operating the technical system 108 based on the inputs to model 106.

[0031] The technical system 108 can be a robot, particularly a vehicle. The technical system 108 can also be a computer-controlled machine, particularly a manufacturing machine, power tool, household appliance, or personal assistance system.

[0032] Model 106 can be configured to output classification, digital images, audio or video data, or virtual sensor data based on its inputs. Inputs may include sensor data, such as digital images, audio or video data, radar data, LiDAR data, ultrasonic sensor data, motion sensor data, or thermal image sensor data. Inputs may also include time-series data.

[0033] Model 106 can be configured to classify sensor data, detect the presence of objects in sensor data, or perform semantic segmentation on sensor data, such as regarding traffic signs, road surfaces, pedestrians, or vehicles. This can be performed based on low-level features, such as pixel attributes or edges of an image.

[0034] Model 106 can be configured to determine one or more continuous values, i.e., to perform regression analysis, such as for distance, velocity, acceleration, or items in the tracking data, such as objects. This can be performed based on low-level features, such as pixel attributes or edges of an image.

[0035] According to the example, model 106 is a neural network configured to determine the output of model 106 based on the input of model 106.

[0036] A neural network includes at least one layer that is configured to determine its output based on its input.

[0037] According to the example, the neural network includes a series of layers. This series of layers includes an input layer configured to receive input from model 106. The series of layers also includes an output layer configured to output the output of model 106. The neural network includes at least one layer l between the input layer and the output layer. The layer l, arranged between the input layer and the output layer, is configured to operate according to the input x and weights of that layer. and optional deviations Determine the output y of this layer: y = W T x+b Based on the example, a series of n layers l are arranged between the input layer and the output layer. i Layer l in i = 1, ..., n i The input x is obtained by using the activation function φ, based on the data from multiple layers in layer l. i Previous layer l i-1 The output y i And it is determined, where x = φ(y).

[0038] Based on the example, model 106 is pre-trained. Based on the example, the weights W are pre-trained.

[0039] The input to the first layer l0 is the input to model 106. The last layer l... n The output is the output of model 106.

[0040] In the output layer, model parts (called experts) are arranged for different tasks. Model 106 includes a routing mechanism that allows individual model parts (called experts) to be trained separately for the corresponding tasks. This routing mechanism allows each expert to focus on a specific data type. Model 106 includes a router gating network. This router gating network is learnable. For example, the router gating network is a neural network. The router gating network selects the specific data type for the corresponding expert.

[0041] Experts can be adapted using the PEFT (Performance-Efficient-Fitting) method with different parameters.

[0042] An example of an addition-based PEFT method is as follows: LoRA: E.J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in ICLR, 2022. VeRA: DJ Kopiczko, T. Blankevoort, and YMAsano, “VeRA: Vector-based RandomMatrix Adaptation,” October 2023. arXiv:2310.11454 [cs]. DyLoRA: M. Valipour, M. Rezagholizadeh, I. Kobyzev, and A. Ghodsi, “DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-FreeLow-Rank Adaptation,” April 2023. arXiv:2210.07558 [cs]. AdaLoRA: Q. Zhang, M. Chen, A. Bukharin, P. He, Y. Cheng, W. Chen, and T. Zhao, “Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning,” March 2023. arXiv:2303.10512[cs]. DoRA: S.-Y. Liu, C.-Y. Wang, H. Yin, P. Molchanov, Y.-CF Wang, K.-T. Cheng, and M.-H. Chen, “Dora: Weight-decomposed low-rank adaptation,” 2024. The addition-based PEFT method updates the weights of the original network through matrix addition operations: W′=W+AB The low-rank matrix AB has learnable parameters.

[0043] An exemplary example of a multiplication-based PEFT method updates the weights of the original network using matrix multiplication operations: W′=HW H is a learnable parameter efficient transformation.

[0044] An example of the PEFT method based on multiplication is OFT: Z. Qiu, W. Liu, H. Feng, Y. Xue, Y. Feng, Z. Liu, D. Zhang, A. Weller, and B. “Controlling text-to-image diffusion by orthogonal finetuning,” arXiv preprint arXiv:2306.07280, 2023.

[0045] An example of a multiplication-based PEFT method is ETHER and ETHER+: M. Bini, K. Roth, Z. Akata, and A. Khoreva, “Ether: Efficient finetuning of large-scale models with hyperplane reflections,” 2024.

[0046] According to ETHER, the multiplication-based PEFT method includes a first transformation to adapt model 106 to the task.

[0047] The first transformation represents hyperplane reflection, where the hyperplane H affects the weight vector. The weights r are reflected. The weight vector w is a vector of length L. The weight vector w includes a single dimension of the multidimensional input for the output y. The elements are weighted by the weights W. The reflected weights r are obtained through the transformation matrix. get: H = I - 2uu T in It is a learnable unit normal vector of the hyperplane, and uu T It is the transpose of vector u and vector u. T The outer product of . This means that vector u has a unit length, that is, the d elements u of vector u. i The sum of the squares is 1:

[0048] The matrix H corresponding to the first transformation relative to the identity matrix It has a constant Frobenius distance.

[0049] According to the example, the reflection weight r is a vector that must maintain a length L.

[0050] The reflection weight r of the weight vector w is determined according to the transformation: Hw=(I-2uu T w = w - 2u(u) T w) Based on the transform H, the output y of the adaptive layer depends on the forward propagation (HW). T x+b.

[0051] According to ETHER+, the multiplication-based PEFT method includes a second exemplary transformation for adapting model 106 to the task.

[0052] The second transformation involves two interacting hyperplanes, namely the first hyperplane H1 and the second hyperplane H2. To perform layer adaptation, two distinct transformation matrices H1 and H2 of the second transformation are learned. + and

[0053] The first hyperplane H1 and the second hyperplane H2 are used for transformation, including: weight vector The interaction of the first hyperplane H1 and the second hyperplane H2 is used to determine the resulting transformed weights r. The resulting transformed weights r do not need to maintain the length L. The length of the resulting transformed weights r is not equal to the length L. The weight vector w includes a single dimension of the multidimensional input with respect to the output y. The weights in W are weighted by weighting the elements.

[0054] The output y of the adapted layer depends on the forward propagation.

[0055] The transformation matrix is ​​obtained as follows H + =I-uu T +vv T in, It is the first learnable hyperplane unit normal vector associated with the first hyperplane H1, where It is the unit normal vector of the second learnable hyperplane associated with the second hyperplane H2, where uu T It is the first vector u and the transpose u of the first vector u. T The outer product of, and where vv T It is the transpose of the second vector v and the second vector v. T The outer product of the first vector u. The first vector u has a unit length, that is, the d elements u of vector u. i The sum of the squares is 1: The second vector v has a unit length, that is, the d elements v of vector v. i The sum of the squares is 1:

[0056] The matrix H of the second transformation + Relative to the identity matrix It has a bounded Frobenius distance.

[0057] The transformation matrix H of the column weight vector w + Based on the following determinations: H+ w=(I-uu T +vv T )w=wu(u T w)+v(v T w) Accordingly, using the learnable first vector and the learnable second vector The transformation matrix is ​​obtained according to the following formula. First vector It has a unit length. Second vector. It has a unit length.

[0058] The matrix of the second transformation Relative to the identity matrix It has a bounded Frobenius distance.

[0059] Row weight vector Transformation matrix Based on the following determinations: Transformation matrix H + , It learns by adapting the model to model 106. This means learning the corresponding first vector u, and the corresponding second vector v,

[0060] An example of a PEFT method that updates biases rather than weights is: BitFit: EB. Zaken, S. Ravfogel, and Y. Goldberg, “Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models,” 2022.

[0061] The PEFT method can introduce diversity within a pool of experts of the same category by using experts with different expressive abilities.

[0062] For LoRA, experts with different ranks can be used. For ETHER+, the boundary of the second transformation can be scaled using a scaling term λ, such that: H = I - λ(uuT - vvT) Routing mechanisms can include single-level router gates or two-level router gates.

[0063] The routing mechanism will be described below with an example of a modification of the switching converter as model 106 and a combination of summation-based fine-tuning techniques (e.g., LoRA) and multiplication-based fine-tuning techniques (e.g., ETHER+).

[0064] This routing mechanism is applicable to other models 106 besides switching converters, as well as other PEFT methods accordingly.

[0065] Figure 2 A portion of a first example of model 106 is schematically depicted, including an attention module 202 of a switching converter and a single-stage router gate 204 for routing the output of the attention module 202 to a feedforward layer 206, a first expert 208, and a second expert 210. The single-stage router gate 204 is, for example, a router-gated network.

[0066] Single-stage router gate 204 includes a router 212 that provides the output of attention module 202 to a first expert 208. Single-stage router gate 204 also includes a router 214 that provides the output of attention module 202 to a second expert 210.

[0067] Based on the first example of Model 106, Model 106 includes the operation order that causes the final transformation of the pre-trained weights W: W′=H(W+AB) Figure 3 A portion of a second example of model 106 is schematically depicted, including an attention module 202 of a switching converter and a single-stage router gate 204 for routing the output of the attention module 202 to a feedforward layer 206, a first expert 208, and a second expert 210.

[0068] Single-stage router gate 204 includes a router 212 that provides the output of attention module 202 to a first expert 208. Single-stage router gate 204 also includes a router 214 that provides the output of attention module 202 to a second expert 210.

[0069] According to the second example of model 106, model 106 includes the order of operations that causes the final transformation of the pre-trained weights W: W′=HW+AB Summation-based PEFT methods (such as LoRA) are used to determine a learnable matrix AB for adapting to the first expert 208.

[0070] Multiplication-based PEFT methods (such as ETHER+) are used to determine a learnable matrix H for adapting to the second expert 210.

[0071] These experts are examples of combinations of additive and multiplicative module types (such as LoRA and ETHER+).

[0072] The routing mechanism of a single-stage router gate 204 can simultaneously select a first expert 208 and a second expert 210. This means that the logistic odds (logits) of the attention module 202 are provided to both experts.

[0073] Figure 4 A portion of a third example of model 106 is schematically depicted, including an attention module 202 of a switching converter and a first two-stage router gate 402. The first two-stage router gate 402 routes the output of the attention module 202 to a feedforward layer 206 and a first expert 208, and routes the output 404 of the feedforward layer 206 and the first expert 208 to a second expert 210. The first two-stage router gate 402 is, for example, a router-gated network.

[0074] The first two-stage router gate 402 includes a router 406 that provides the output of the attention module 202 to the first expert 208. The first two-stage router gate 402 also includes a router 408 that provides the outputs of the first expert 208 and the feedforward layer 206 to the second expert 210.

[0075] According to the third example of model 106, model 106 includes the order of operations that cause the final transformation of the pre-trained weights W, which includes W″ = AB + W at the first level and W″ = H(W′)W′ at the second level.

[0076] Summation-based PEFT methods (such as LoRA) are used to determine a learnable matrix AB for adapting to the first expert 208.

[0077] Multiplication-based PEFT methods (such as ETHER+) are used to determine a learnable matrix H for adapting to the second expert 210.

[0078] The routing mechanism of the first two-stage router gate 402 can select a first expert 208 and a second expert 210. This means that the log probability of the attention module 202 is provided to the first expert 208, and the outputs of the first expert 208 and the feedforward layer 206 are provided to the second expert 210.

[0079] Figure 5A portion of a fourth example of model 106 is schematically depicted, including an attention module 202 of a switching converter and a second two-stage router gate 502. The second two-stage router gate 502 routes the output of the attention module 202 to a feedforward layer 206 and a second expert 210, and routes the output 504 of the feedforward layer 206 and the second expert 210 to a first expert 208. The second two-stage router gate 502 is, for example, a router-gated network.

[0080] The second two-stage router gate 502 includes a router 506 that provides the output of the attention module 202 to the second expert 210. The second two-stage router gate 502 also includes a router 508 that provides the outputs of the second expert 210 and the feedforward layer 206 to the first expert 208.

[0081] According to the fourth example of model 106, model 106 includes the order of operations that cause the final transformation of the pre-trained weights W, which includes W′ = HW at the first level and W″ = AB + W′ at the second level.

[0082] Summation-based PEFT methods (such as LoRA) are used to determine a learnable matrix AB for adapting to the first expert 208.

[0083] Multiplication-based PEFT methods (such as ETHER+) are used to determine a learnable matrix H for adapting to the second expert 210.

[0084] The routing mechanism of the second two-stage router gate 502 can select a first expert 208 and a second expert 210. This means that the log probability of the attention module 202 is provided to the second expert 210, and the outputs of the second expert 210 and the feedforward layer 206 are provided to the first expert 208.

[0085] Figure 6 A flowchart schematically depicts the steps of a method for adapting model 106 to the task.

[0086] Exemplary tasks include: Output classification based on different types of input data Output digital images based on different types of input data. Output audio data based on different types of input data. Output video data based on different types of input data, or Output virtual sensor data based on different types of input data.

[0087] Examples of different types of input data can be the following types of input data: text, Digital images, Audio data, Video data, Sensor data, The operational volume of technical system 108.

[0088] The method includes step 602.

[0089] Step 602 includes providing model 106, an expert, and router gates for the expert. In this context, the expert can be a neural network. The router gates can be a router-gated network. Model 106 can be a neural network.

[0090] Model 106 includes layer l i Layer l i Configured according to weight W i and optional deviation b i Layer l i Multidimensional input x i Mapping to multidimensional output: Weight W i Including vector w i，j The vector w i,j Including for layer l i The output y i,j The dimension j of the multidimensional input x i The elements are weighted by weight W. i The corresponding subset.

[0091] Experts were configured to transform the weights of model 106.

[0092] Layer l i It is configured to output log odds.

[0093] The expert was configured to output log odds.

[0094] The router gate is configured to route log odds to the expert.

[0095] For example, the router gate is configured to access the last layer of model 106. n One of the previous layers i The output log odds are routed to at least one of the experts.

[0096] For example, a router gate is configured to route the output log odds of one of the experts to at least one other expert among the experts.

[0097] It is from at least one layer l i Or the log-probability route from at least one expert to the corresponding expert, or from at least one level l iThe routing is based on a log-probability factor from at least one expert to the corresponding expert, depending on the type of router gate. For the first expert 208 and the second expert 210, the router gate can be a single-stage router gate 204, a first two-stage router gate 402, or a second two-stage router gate 502. The method is not limited to single-stage or two-stage router gates. The router gate can be a multi-stage router gate with three or more stages. The method is not limited to two experts. The method may include providing three or more experts.

[0098] The method includes step 604.

[0099] In step 604, training data is provided.

[0100] The training data includes pairs of inputs to model 106 and baseline ground truth values ​​for the outputs of model 106. The inputs to model 106 may include or represent information about the technical system 108. The outputs of model 106 may be outputs used to operate the technical system 108.

[0101] Training data is provided based on the task.

[0102] For a task of classifying sensor signals, the input represents or includes the sensor signals, and the output and reference truth represent or include the classification of the sensor signals.

[0103] The input can be text representing the sensor signal, such as a description of the sensor data. The input can also be a technical quantity characterizing the sensor signal from the technical system 108.

[0104] For tasks that generate content (such as digital images or audio signals), the input may represent or include text, and the output and reference truth may represent or include digital images and / or audio signals.

[0105] For the task of generating digital images, the input may include, for example, text and semantic graphs, and the output and benchmark truth representation may include digital images.

[0106] For virtual sensing tasks, the inputs, for example, represent or include at least one operational quantity of the technical system 108, and the outputs and reference truth values ​​represent or include sensor signals.

[0107] The method includes step 606.

[0108] Step 606 includes training experts and / or router gates based on the training data.

[0109] Router gates can be trained simultaneously with experts. Router gates and experts can also be trained separately.

[0110] Experts are associated with corresponding training methods. For example, one expert is associated with a first training method, and another expert is associated with a second training method.

[0111] Train the relevant experts using training methods associated with them.

[0112] For example, the first training method is an addition-based training method. The addition-based PEFT method is an example of an addition-based training method. The second training method is, for example, a multiplication-based training method. The multiplication-based PEFT method is an example of a multiplication-based training method.

[0113] The first training method and the second training method can work together. This means, for example, that the first expert can be trained using the first training method while keeping the second expert unchanged, and the second expert can be trained using the second training method while keeping the first expert unchanged.

[0114] The pre-trained weights W of Model 106 remain unchanged during training. This means that matrix W remains unchanged. It is not required that all experts in Model 106 are learnable. Learnable experts are trained. Other experts can remain unchanged.

[0115] To allow for further flexibility, training can include parameters from learning experts.

[0116] In the case of LORA, the parameter can be the rank of the learnable matrix AB.

[0117] This means that the smaller the dimensions of matrices A and B that form the learnable AB, the lower the rank of the learnable matrix AB.

[0118] In the case of ETHER+, the parameter can be the Frobenius distance relative to the identity matrix.

[0119] In the case of ETHER+, the parameter can be a scaling parameter λ, which allows control, for example, H = I - λ(uu) T -vv T Transformation in ).

[0120] Note that if λ = 1, it becomes ETHER+ without the parameter λ.

[0121] Training can be applied to any linear layer of model 106. Training can use different PEFT modules, i.e., modules configured to perform the corresponding PEFT methods.

[0122] In implementing an expert neural network, training the expert based on training data includes learning the weights of the expert neural network. Similarly, in implementing a router gate neural network, training the router gate based on training data includes learning the weights of the router gate neural network.

[0123] The method includes step 608.

[0124] In step 608, input to model 106 is received, which includes or represents information about the task.

[0125] According to the example, the input includes or represents information about the technical system 108.

[0126] The method includes step 610.

[0127] In step 610, the output of the adapted model 106 is determined, and the adapted model 106 outputs the output in response to the input received by the model 106.

[0128] According to the example, the output includes or represents the output used to operate the technology system 108.

[0129] The method includes step 612.

[0130] In step 612, the output of the adapted model 106 is output.

[0131] According to the example, this output is used to operate the technology system 108 based on the output of the adapted model 106.

[0132] The method may include step 614.

[0133] In step 614, the technology system 108 is operated according to the output of the adapted model 106.

[0134] For example, the technology system 108 is a robot, particularly a vehicle. For example, the input is a digital image, such as an object representing a traffic participant or infrastructure.

[0135] For example, the output is a classification of objects. A robot can be manipulated to move along a trajectory determined based on the object classification, for example, to avoid or pass over objects.

[0136] For example, technical system 108 is a computer-controlled machine. The computer-controlled machine can be operated to produce workpieces based on the output of model 106. The computer-controlled machine may include a human-machine interface or a machine-to-machine interface. The computer-controlled machine can be operated to receive input through the interface and / or output the output of model 106 through the interface.

[0137] Figure 7 The data structure 700 used to adapt model 106 to the task is schematically depicted.

[0138] Data structure 700 is, for example, a data structure implemented by a computer.

[0139] Data structure 700 includes at least one data field 702, used for: Model 106 The input to the router door, The output of the first expert 208 The output of the second expert, 210. The modified layers were determined by modifying model 106 based on expert output. Used to train the first expert 208 using the first training method based on the output of model 106. Used to train a second expert 210 using a second training method based on the output of model 106.

[0140] The data structure 700 may include at least one data field 702 for keeping the weights and the second expert 210 unchanged during training using the first training method.

[0141] Data structure 700 may include at least one data field 702 for maintaining the weights and the first expert 208 unchanged during training using the second training method.

Claims

1. A method for adapting a model (106) to a task, characterized in that, The method includes providing (602) a model (106), wherein the model (106) includes, in particular, linear layers for mapping multidimensional inputs of the layers to multidimensional outputs of the layers according to weights, wherein the model (106) includes experts (208, 210) and router gates (204, 402, 502) for adapting the model (106) to different tasks, wherein the method includes: providing (604) the input to the router gates (204, 402, 502); determining the output of the experts (208, 210) based on the output of the router gates (204, 402, 502) in response to the input; Modify the model (106) according to the output of the experts (208, 210); map the input to the output of the model (106) using the modified layer; train (606) the first expert (208) among the experts using a first training method according to the output of the model (106), and train (606) the second expert (210) among the experts using a second training method according to the output of the model (106); and keep the weights and the second expert (210) unchanged during training using the first training method, and keep the weights and the first expert (208) unchanged during training using the second training method.

2. The method according to claim 1, characterized in that, Modifying the model (106) includes: determining the output of the first expert (208) weight by weight, modifying the weights according to the weights of the layer and the output of the first expert (208) weight by weight, and determining the output of the model (106) according to the modified weights.

3. The method according to claim 1, characterized in that, Modifying the model (106) includes: determining the multidimensional output of the first expert (208) based on the dimension of the multidimensional output of the layer, and determining the output of the model (106) based on the dimension-wise summation of the multidimensional output of the layer and the multidimensional output of the first expert (208).

4. The method according to any one of the preceding claims, characterized in that, Modifying the model (106) includes: determining the output of the second expert (210) weight by weight, modifying the weights according to the weights of the layer and the output of the second expert (210) weight by weight, and determining the output of the model (106) according to the modified weights.

5. The method according to any one of claims 1 to 3, characterized in that, Modifying the model (106) includes: determining the multidimensional output of the second expert (210) based on the dimension of the multidimensional output of the layer, and determining the output of the model (106) based on the dimension-wise summation of the multidimensional output of the layer and the multidimensional output of the second expert (210).

6. The method according to any one of the preceding claims, characterized in that, The router gates (204, 402, 502) are trained based on the output of the model (106).

7. The method according to any one of the preceding claims, characterized in that, The output of the first expert (208) represents a transformation matrix used for matrix addition with the weight matrix representing the weights, wherein training the first expert (208) includes learning the transformation matrix.

8. The method according to claim 7, characterized in that, Multiple experts are provided with their own transformation matrices for matrix addition, wherein the ranks of the transformation matrices provided for the matrix addition are different from each other.

9. The method according to any one of the preceding claims, characterized in that, The output of the second expert (210) represents a transformation matrix used for matrix multiplication with the weight matrix representing the weights, wherein training the second expert (210) includes learning the transformation matrix.

10. The method according to claim 9, characterized in that, A common matrix is ​​provided for multiple experts to perform the matrix multiplication operation, and different scalars are provided for the multiple experts to scale the common transformation matrix to the transformation matrix, and the scalars of the experts are trained according to the output of the model (106).

11. The method according to any one of the preceding claims, characterized in that, The model (106) includes multiple layers, particularly linear ones, wherein adapting the model (106) includes adapting the layers using corresponding experts and corresponding router gates, wherein adapting the layers includes: providing the input of the corresponding layer to the router gate of the corresponding layer, determining the output of the expert of the corresponding layer based on the output of the router gate of the corresponding layer in response to the input of the corresponding layer, modifying the model (106) based on the output of the expert of the corresponding layer, and training the expert of the corresponding layer of the model (106).

12. The method according to any one of the preceding claims, characterized in that, The model (106) is configured to determine the input based on the input of the model (106), wherein the training data includes pairs of inputs to the model (106) and ground truth values ​​for the outputs of the model (106), wherein the inputs represent or include sensor signals, and wherein the outputs and ground truth values ​​represent or include classifications of the sensor signals, or wherein the inputs represent or include text, and wherein the outputs and ground truth values ​​represent or include digital images and / or audio signals, or wherein the inputs represent or include text and semantic graphs, and wherein the outputs and ground truth values ​​represent or include digital images, or wherein the inputs represent or include at least one operational quantity of a technical system, and wherein the outputs and ground truth values ​​represent or include sensor signals.

13. The method according to any one of the preceding claims, wherein, The method includes: receiving (608) input to the model (106), the input including or representing information about the technical system (108); determining (610) the output of the adapted model (106), the adapted model (106) outputting the output in response to the input of the model (106); and outputting (612) the output of the adapted model (106) and / or operating (614) the technical system (108) based on the output of the adapted model (106).

14. An apparatus (100) for adapting a model to a task, characterized in that, The apparatus (100) includes at least one processor (102) and at least one memory (104), wherein the at least one memory (104) includes instructions executable by the at least one processor (102), and the instructions, when executed by the at least one processor (102), cause the apparatus (100) to perform the method according to any one of the preceding claims.

15. A computer program, characterized in that, The computer program includes instructions that can be executed by a computer, and when executed by the computer, the instructions cause the computer to perform the method according to any one of claims 1 to 13.

16. A data structure for adapting a model (106) to a task, particularly a computer-implemented data structure, characterized in that, The data structure includes at least one data field for the model (106), wherein the model (106) includes, in particular, linear layers for mapping multidimensional inputs of the layers to multidimensional outputs of the layers according to weights, wherein the model (106) includes for adapting the model (106) to experts (208, 210) and router gates (204, 402, 502) for different tasks, wherein the data structure includes at least one data field for inputs to the router gates (204, 402, 502), wherein the data structure includes at least one data field for determining the output of the expert (208, 210) based on the output of the router gates (204, 402, 502) in response to the inputs, wherein the data structure Includes at least one data field for determining a modified layer by modifying the model (106) based on the output of the experts (208, 210), wherein the data structure includes at least one data field for training (606) a first expert (208) among the experts using a first training method based on the output of the model (106), and wherein the data structure includes at least one data field for training (606) a second expert (210) among the experts using a second training method based on the output of the model (106), and maintaining the weights and the second expert (210) unchanged during training using the first training method, and maintaining the weights and the first expert (208) unchanged during training using the second training method.

Images