A device specifically for continuously training machine learning models.
The machine learning model MoCL-P, which uses a modular structure and adaptive pruning strategy, solves the problems of knowledge forgetting and low parameter efficiency in continuous learning, and achieves efficient and lightweight continuous learning, which is suitable for fields such as natural language processing and computer vision.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- ROBERT BOSCH GMBH
- Filing Date
- 2025-06-13
- Publication Date
- 2026-05-19
AI Technical Summary
Existing machine learning models suffer from problems such as knowledge forgetting, difficulty in knowledge transfer, and low parameter efficiency during continuous learning, especially as the number of tasks increases, resulting in a significant increase in computational and storage costs.
The machine learning model (MoCL-P) employs a modular structure and adaptive pruning strategy. By freezing task-specific modules and optimizing weights, it achieves efficient parameter utilization and knowledge transfer, avoids knowledge forgetting, and remains lightweight under a large number of tasks.
It achieves high performance and low resource consumption for continuous learning under a large number of tasks, keeps the model lightweight and efficient, solves the problems of knowledge forgetting and parameter efficiency, and supports rapid adaptation to new tasks.
Smart Images
Figure 2026082625000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an apparatus for continuously training a machine learning model in particular.
Background Art
[0002] Prior Art Machine learning should be based on a continuous learning approach to continuously develop and maintain its effectiveness over time.
[0003] There are particularly three problems in continuous learning. First, forgetting of what has already been learned should be avoided, that is, newly learned information should not interfere with or impair the previously acquired knowledge. Second, knowledge transfer should be enabled, that is, for efficient learning of new tasks, knowledge from previous tasks should be reused.
[0004] Similarly, parameter efficiency should be continuously maintained, that is, a machine learning model should remain easy to weight and efficient even if the sequence of continuous learning scales up to hundreds of tasks.
[0005] To address these problems, in previous research, the concepts of parameter separation and efficient fine-tuning of parameters have been used for continuous learning. Despite its effectiveness regarding the performance in the case of continuous learning tasks, when the tasks in a continuous learning sequence reach hundreds, the gradual expansion of task-specific parameters also leads to parameter inefficiency and significantly increases the computational cost and memory cost.
Summary of the Invention
Problems to be Solved by the Invention
[0006] The problem of the present invention is to provide an improved apparatus.
Means for Solving the Problems
[0007] This problem is solved by the device according to the characterizing part of claim 1.
[0008] Disclosure of the invention According to a first aspect, an apparatus for specifically and continuously training a machine learning model is proposed. The apparatus comprises an evaluation and calculation device, and the evaluation and calculation device performs the following steps, namely: Known task modules P0…P of the machine learning model From a data pool having data for tasks solved using, for the selection of known tasks, trainable task feature vectors v0…v m-1 And corresponding known task modules P0…P m-1 Providing; For the task input T to be solved n A task input embedding x n And a trainable task module P having a corresponding trainable task feature vector v m Providing; m And; Task modules P0…P m And matching weights α1…α m In particular, calculating matching weights α1…α between the task input embedding x m [ And trainable task feature vectors v0…v n To form a particular weighted sum P’ m And; m Calculating; Combining the weighted sum P’ m With the machine learning model; Depending on the training criterion, training the task feature vector v m And the task module P m To train the machine learning model to solve the task input T n And; Is configured to execute
[0009] It should be understood that the steps and further optional steps according to the present invention do not necessarily have to be performed in the illustrated order, and may rather be performed in a different order. Further intermediate steps may be envisioned. Also, each step may include one or more substeps without departing from the scope of the apparatus according to the present invention.
[0010] In this example, a machine learning model (also known as MoCL-P for short) that pursues a continuous learning approach is proposed. This machine learning model provides a weighted approach to continuous learning by utilizing a task-oriented modular structure and adaptive decimation to address the three challenges of continuous learning described above, either entirely or at least partially. This machine learning model and its associated methods are characterized not only by their high performance, but also by their parameter efficiency, which, as shown by corresponding benchmarks, is several times more efficient than the previous algorithm. MoCL-P proposes a sustainable path for continuous learning, so that the model continues to evolve as the number of tasks increases while remaining lightweight and efficient.
[0011] This apparatus proposes a solution for parameterized, isolated, and sequential learning. Here, each task in a sequential learning sequence is assigned task-specific parameters. Each task-specific module is preferably "frozen" immediately after training for a particular task is completed, thus avoiding catastrophic forgetting, because the knowledge within each specific module is retained for subsequent training processes. Furthermore, this approach includes modular and constitutive learning, ensuring that the machine learning model has effective knowledge transfer through the reuse of relevant knowledge from previous tasks.
[0012] Additionally, there may be a set of parameters used for all tasks. These sets constitute the main part of the parameters and are pre-trained (for example, in BERT or DINO). These are also frozen. Task-specific modules are then inserted between the existing layers.
[0013] The present invention can be used to analyze various forms of data that can be represented using vector representation (embedding), such as text data and / or image data. The present invention can be used in practical applications for continuous learning, especially when new tasks are constantly being learned and minimal resource requirements are required. This could include, for example, exemplary tasks such as natural language processing (NLP) and computer vision.
[0014] One example of natural language processing is the analysis of patent documents. Over time, the areas covered by patent documents may change, or new areas may be added. The NLP algorithm used for patent analysis must be able to adapt to such changes in order to ensure a sufficiently good analysis over time. In this case, a good algorithm for continuous learning is extremely important. On the one hand, the underlying machine learning model should not forget its previous knowledge when adapting in any way. On the other hand, existing knowledge should be utilized so that the machine learning model can quickly adapt to new text and / or image classification and / or new input text and / or image data streams with changing labels.
[0015] Further, this method can be applied to processing knowledge in a knowledge database where the set of labels grows naturally as the knowledge content increases. Similarly, this method can be applied to support systems based on (large-scale) language models, because these systems should continuously adapt to new tasks and languages. The present invention is a method for continuously learning new knowledge without forgetting existing knowledge, in which knowledge integration and computational costs are efficiently compensated and adapted.
[0016] Continuous learning investigates the problem of learning from an infinite stream of data, particularly with the goal of expanding progressively acquired knowledge and utilizing it for future learning. This invention addresses the central challenges of "catastrophic" forgetting, knowledge transfer, and parameter optimization in continuous learning, and ultimately provides a scalable and efficient solution for realistic scenarios where low resource requirements are critically important.
[0017] In contrast to the complete separation of task-specific parameters in sequential learning, which eliminates knowledge transfer, this example follows the idea of composition with introduced task modules to facilitate knowledge transfer. For this purpose, task representations are used for matching task modules and, consequently, for the composition of new and old modules for learning. Module matching aims to determine the contribution of each existing module to learning the current task, that is, to what extent previously learned modules can be reused for the current task.
[0018] Here, a trainable feature vector V ∈ RN × D is introduced as a task representation to capture the features of each task in a continuous learning sequence. Here, the dimension of each task feature vector v ∈ RD is the input embedding x n It is set to the same value as the dimension of ∈RD.
[0019] A set of modules selected from previous tasks {P0, ..., Pm-1} and the new task T n (In this case, preferably, m - 1 << n is effective), starting from the trainable module P m is initialized and preferably added to the model at least temporarily. For each instance x n of the current task T i n in particular, for x n from all task feature vectors {v0,..., v m} from our current module set, appropriate weights {α0,..., α m} are calculated. In particular, the cosine similarity between x n and {v0,..., v m} is calculated as the module matching weight α 0:m . Subsequently, the old and new modules are combined via a weighted sum.
Number
[0020] In a further aspect, after training the machine learning model, to solve the task input T n , a comparison is made between the matching weight α m of the task module P m and a predetermined threshold value, and based on this comparison, it is proposed that it is required to determine whether to remove the task module P m from the weighted sum of the task modules P0...P m or leave it included therein.
[0021] In particular, the need for efficiency increases when a continuous learning sequence scales up to tens or hundreds of tasks. As demonstrated in previous studies (Wang et al., 2023d; Razdaibiedina et al., 2022; Wang et al., 2024), the continuous expansion of the module pool to assign PEFT modules to each task leads to high computational costs. In contrast, in this example, an adaptive pruning strategy is preferably used to make the approach scalable for scenarios with long task sequences. Specifically, our pruning strategy aims to retain only modules that add new valuable information to already selected modules.
[0022] T n Training for (especially PEFT module P) m and task feature vector v m After the training with P m To decide whether to delete it or leave it in the existing group of modules, α m , in other words, a new module P m The matching weights are compared to a threshold. Intuitively, here, larger weights suggest new and valuable information, while task modules with small weights do not contribute to new information and can therefore be discarded.
[0023] In a further aspect, task input T n The step of training a machine learning model to solve this problem is to minimize the cross-entropy loss of the training example while simultaneously minimizing the task-specific task feature vector v m And the corresponding task input embedding x n Task module P maximizes the cosine similarity between [the specified values]. m and task feature vector v m It is proposed that this includes finding the and
[0024] The training goal for the nth task in a continuous learning sequence is to minimize the entropy loss of the training examples while simultaneously generating task-specific feature vectors v m And the corresponding task input embedding x n Task module P maximizes the cosine similarity between [the specified values]. m and task feature vector v m The goal is to find it.
number
[0025] in this case,
number
[0026] In a further embodiment, the machine learning model is proposed to include language models, particularly large-scale language models such as vision transformers, or convolutional neural networks.
[0027] Basically, other machine learning models are also conceivable.
[0028] In a further embodiment, matching weights α1…α m The step to calculate the task input embedding x n And, the trainable task feature vector v0…v m It is proposed to include a step of calculating the cosine similarity between and .
[0029] In this example, preferably, input embedding x n And each feature vector v up to the current task i The cosine similarity between the two is the matching score α i=cos(x n ,v i It is calculated as follows. As a result, module matching weights {α0, α1, ...} for module synthesis are obtained for the reuse of existing knowledge.
[0030] In a further embodiment, task module P0…P m It is proposed that each of these parameters include an efficient fine-tuning parameter (PEFT).
[0031] In this example, the concept of parameter separation using efficient parameter tuning (PEFT), introduced in previous studies (Razdaibiedina et al., 2022; Wang et al., 2023c,d, 2024), is used. In this approach, trainable PEFT parameters are assigned to each task, while other parameters remain frozen. Here, prefix tuning (Li and Liang, 2021) is used as the PEFT module. For each task in a sequential learning sequence, a set of trainable PEFT parameters, i.e., task-specific modules, are added to a prepared machine learning model, such as a language model (PLM), to allow for task fine-tuning that will be stored later. Instead of updating the entire model, only a small number of PEFT parameters are optimized. Once training for a particular task is complete, the corresponding PEFT module is immediately frozen, thereby preserving task-specific knowledge in subsequent training processes and thus avoiding fatal forgetting.
[0032] In a further embodiment, it is proposed that the multiple tasks and / or input tasks include data classification and / or text classification and / or multilingual phonetic analysis and / or image classification and / or driving and controlling a technical system and / or providing a knowledge database for driving and controlling a technical system.
[0033] Please understand that this list is merely an example, as further tasks may be possible.
[0034] In a further embodiment, the device may be a control device / control device component included in a vehicle and / or robotic system and / or industrial machine having autonomous driving capabilities. In other words, a control device including a device having the features described herein is claimed, which is included in a vehicle and / or robotic system and / or industrial machine having autonomous driving capabilities.
[0035] The trained machine learning model can be used, for example, as an image classifier and / or for the drive control of technical systems, particularly robots, vehicles with at least some autonomous driving capabilities, and / or manufacturing machinery.
[0036] The described embodiments and variations can be combined in any way.
[0037] Further possible embodiments, developments, and implementations of the present invention include combinations of features of the invention that have not been explicitly mentioned above or described below in relation to the examples.
[0038] The accompanying drawings are intended to convey a further understanding of embodiments of the present invention. These drawings illustrate embodiments and are used in connection with the description to explain the principles and concepts of the present invention.
[0039] Many other embodiments and advantages mentioned will become apparent with reference to the drawings. The elements shown in the drawings are not necessarily shown to scale relative to one another. [Brief explanation of the drawing]
[0040] [Figure 1] This is a flowchart illustrating one embodiment. [Figure 2] This is a block diagram illustrating one embodiment.
[0041] In these drawings, unless otherwise specified, the same reference numeral indicates the same or functionally equivalent element, part, or component. [Modes for carrying out the invention]
[0042] Figure 1 shows a schematic flowchart of a method for the continuous training of machine learning models, in particular, which can be performed by the apparatus described herein.
[0043] This method can be performed, in any embodiment, at least in part, by apparatus 100, which for this purpose may include several components not shown in detail, for example, one or more serving devices and / or at least one evaluation and calculation device. It should be understood that the serving device may be configured together with the evaluation and calculation device or may be separate from the evaluation and calculation device. Furthermore, apparatus 100, which may be part of the system, may include a memory device and / or an output device and / or a display device and / or an input device.
[0044] The methods that can be performed by the apparatus described herein will also be described with reference to Figure 2. The method performed by the computer includes at least the following steps:
[0045] In step S1, the known task modules P0…P of the machine learning model m-1 From a data pool T containing data for tasks solved using , trainable task feature vectors 200,v0…v are selected for known task selection. m-1 , and the corresponding known task module P0…P m-1 A step is performed to provide the task feature vector v0…v from the data pool T. This providing step preferably involves taking the task feature vector v0…v m-1 This is done by extracting the relevant information.
[0046] In step S2, the task input T to be resolved is nFor this purpose, task input embedding x n And the corresponding trainable task feature vector 200v m A trainable task module P having m The steps to provide are performed.
[0047] In step S3, task module P0...P m And, matching weight α1…α m And especially the temporary weighted sum P' m To form the task input embedding (x n ) and the trainable task feature vector v0…v m Matching weights 202, α1…α m The calculation step is performed.
[0048] Steps S1 through S3 are, for illustrative purposes, integrated into block B1 (task module matching) in Figure 2.
[0049] In step S4, the weighted sum P' m The step of combining this with the machine learning model W is performed.
[0050] In step S5, the task feature vector v depends on the training criteria. m and task module P m By training with, Task Input T n The next step involves training a machine learning model W to solve the problem.
[0051] In step S6, preferably, the weighted sum P' m The step can be taken to provide a machine learning model y that has been supplemented and trained for a certain amount of time.
[0052] Steps S4 through S6 are exemplified by being integrated into block B2 (task module matching) in Figure 2.
[0053] Preferably, after training a machine learning model W to solve the task input Tn, the task module P m Matching weight α m Then, a comparison can be made with a predetermined threshold (for example, 0.05), and in this case, based on this comparison, task module P m Task module P0...P m The weighted sum P' m The question arises whether to remove it or leave it included. This removal is illustrated in Figure 2 by a cross in a circle and denoted by reference numeral 204.
[0054] Threshold-dependent pruning, deletion, or retention is, for example, integrated into block B3 (task module matching) in Figure 2 and is performed entirely at the discretion of the user.
Claims
1. A device (100) for continuously training a machine learning model, The apparatus (100) includes an evaluation and calculation device, and the evaluation and calculation device performs the following steps: - Known task modules of the machine learning model (P ０ ...P ｍ－１ From a data pool containing data for tasks solved using ), a trainable task feature vector (v) is selected for the selection of known tasks. ０ …v ｍ－１ ) and corresponding known task modules (P ０ ...P ｍ－１ The steps include providing, - Step of providing a task input embedding (x ｎ ), and a trainable task module (P ｎ ) having a corresponding trainable task feature vector (v ｍ ) for a task input (T ｍ ) to be solved - The aforementioned task module (P ０ ...P ｍ ) and matching weights (α １ …α ｍ ) and especially the temporary weighted sum (P' ｍ To form the task input embedding (x ｎ ) and the aforementioned trainable task feature vector (v ０ …v ｍ ) and the matching weight (α １ …α ｍ The steps to calculate ) and - the weighted sum (P' ｍ The steps include: ) and combining the machine learning model, - The task feature vector (v) depends on the training criteria. ｍ ) and the task module (P ｍ By training with the task input (T ｎ The steps include training the aforementioned machine learning model to solve the problem, A device (100) configured to perform the following.
2. After training the aforementioned machine learning model, the task input (T ｎ To solve the problem, the task module (P ｍ The matching weight (α) of the aforementioned ) ｍ A comparison is made between the value and a predetermined threshold, and based on the comparison, the task module (P ｍ ) to the task module (P ０ ...P ｍ The weighted sum of (P') ｍ The apparatus according to claim 1, wherein it is required to decide whether to remove it from or leave it included therein.
3. The aforementioned task input (T ｎ The step of training the machine learning model to solve the problem involves minimizing the cross-entropy loss of the training example while simultaneously creating a task-specific task feature vector (v ｍ ) and the corresponding task input embedding (x ｎ The task module (P) maximizes the cosine similarity between ) and ｍ ) and task feature vector (v ｍ The apparatus according to claim 1 or 2, comprising finding )
4. The apparatus according to claim 1, wherein the machine learning model includes a language model, in particular a large-scale language model or a convolutional neural network.
5. The matching weight (α １ …α ｍ The step of calculating the task input embedding (x ｎ ) and the aforementioned trainable task feature vector (v ０ …v ｍ The apparatus according to claim 1 or 2, comprising the step of calculating the cosine similarity between ) and ).
6. The aforementioned task module (P ０ ...P ｍ The apparatus according to any one of claims 1 to 5, wherein each of the parameters has an efficient fine-tuning parameter (PEFT).
7. The apparatus according to any one of claims 1 to 6, wherein the plurality of tasks and / or input tasks include data classification and / or text classification and / or multilingual phonetic analysis and / or image classification and / or driving control of a technical system and / or providing a knowledge database for driving control of a technical system.