Method for determining architecture of multitask model

By estimating task affinities and optimizing branching depths, the method minimizes destructive interactions in multitask learning, improving efficiency and reducing computational costs in high-dimensional domains.

JP2024167904APending Publication Date: 2024-12-04ROBERT BOSCH GMBH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2024082737
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-22
Filing Date
2024-05-21
Publication Date
2024-12-04

AI Technical Summary

Technical Problem

Existing multitask learning models suffer from destructive interactions between tasks, leading to inefficiencies and increased computational costs, particularly in high-dimensional domains like image processing.

Method used

A method to determine the architecture of a multi-task model by estimating pairwise affinities between tasks, assigning them to groups, and determining branching depths to minimize negative interactions, using a tree topology with shared and dedicated networks.

Benefits of technology

This approach reduces computational costs and enhances efficiency by attenuating negative task interactions, allowing simultaneous task solving with limited computational resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024167904000001_ABST
    Figure 2024167904000001_ABST
Patent Text Reader

Abstract

To provide a method, a multitask model, and a program that determine the architecture of a multitask model.SOLUTION: A method 100 comprises steps of: estimating (102), for a set of tasks to be solved (Ti), paired affinity between the tasks to be solved (Ti) of the set; allocating(104), on the basis of the paired affinity, tasks to be solved (Ti) to N groups (gi), i=1 to N; and determining(106) branching depth (d) of a multitask model (NN) having layers (Li) and presetting the branching depth (d) as to at what depth in a layer (Li) of the multitask model (NN) a basic network (B) of the NN shared by tasks (Ti) to be solved is branched into N branching networks (Zi) (i=1 to N).SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a method for determining an architecture of a multitask model, a multitask model for solving at least two, in particular a plurality of mutually distinct tasks, using the architecture determined according to such a method, and a computer program product. [Background technology]

[0002] Background technology Multitask learning or multitasking is also a subfield of machine learning that deals with models that can solve multiple tasks in parallel. A common paradigm in multitasking is training a single neural network to solve various tasks simultaneously. A typical architecture of a multitask network consists of a common base network, also called backbone, which acts as a kind of feature extractor and whose weights are shared by all tasks, and a dedicated branch network, also called head or header, whose weights are not shared by all tasks. In such an environment, the tasks to be solved during the training of the neural network can affect each other in a constructive or destructive way within the common base network. Summary of the Invention [Problem to be solved by the invention]

[0003] This disclosure proposes a method by which the architecture of a multitask model can be determined such that destructive interactions between tasks are minimized. [Means for solving the problem]

[0004] Disclosure of the Invention An embodiment includes at least two, in particular a plurality of, mutually distinct tasks T i (

number

[0005] The method assumes that the estimated pairwise affinities of the tasks are utilized to envision a single multitask network in which the negative interactions between the tasks are attenuated. That is, no separate base networks or backbones are touched, and the tasks to be solved are not assigned to separate networks. According to the present disclosure, the architecture of the single network is determined by a tree topology in which multiple branching networks or headers are connected to one common base network or backbone. In other words, the method allows for a finer division in which parts of the neural network can be further shared by different tasks, for example, in lower layers of a single neural network.

[0006] A common base network acts as a feature extractor, and the weights of the common base network are shared depending on the task to be solved. The branching networks are task-specific sub-networks, and their weights are not shared across all tasks, but are used only for their respective tasks.

[0007] The assignment of tasks to the branching network is done by dividing N groups g i For the present method, this means that the assignment is available as a result from the previous step.

[0008] The purpose of the step of determining the branching depth d is to determine the layer L of the multitask model where the shared basic network, the backbone, exists or branches. i The aim of the present invention is to determine the optimal depth of the branching depth d=2, which means, for example, that the base network contains layers 1 and 2 of the multitask model, and after layer 2 there is a branch to the branching network.

[0009] That is, not only the affinity between tasks is calculated, but also the optimal depth of the common backbone is determined to establish where conflicts between tasks may occur. That is, the method determines whether the potential representations are still available for joint use before the network is split. In this way, the computational cost can be significantly reduced, which is especially beneficial in image processing and other high-dimensional domains where the computations in the early layers are quite frequent and computationally intensive regardless of the task.

[0010] According to one embodiment, pairwise affinity is assumed to indicate how well each pair of tasks can be trained in a network with a jointly used layer. A high affinity value means positive interference between the task pair, whereas a low affinity value means negative interference between the task pair. For example, pairwise affinity can be determined by training a pair of tasks together in a single multitask network and quantifying the effect of the gradient update of a task on the other task. Two or more pairs of tasks can also be trained together in a single multitask network.

[0011] According to one embodiment, it is contemplated that pairwise affinity includes an affinity value for a first task of the pair and an affinity value for a second task of the pair. For example, task T a and task T b For a task pair containing task T a Affinity value Z for b→a And Task T b Affinity value Z for a→b These affinity values ​​are not necessarily symmetric, i.e., Z b→a is Z a→b does not have to be equal to

[0012] For a task set that includes three or more tasks, the affinity for a particular task can be determined by determining the average pairwise affinity for that particular task. For example, for task T a ,T b ,T c For a set containing T a The affinity value for task T a Task T for b and T c Averaging the pairwise affinities of [(Z b→a +Z c→a ) / 2].

[0013] Determining affinity is described, for example, in the paper by Fifty, Chris et al., “Efficiently identifying task groupings for multi-task learning” Advances in Neural Information Processing Systems 34 (2021):27503-27516, and the paper by Standley, Trevor et al., “Which tasks should be learned together in multi-task learning?” International Conference on Machine Learning. PMLR, 2020.

[0014] According to one embodiment, the tasks to be solved are grouped by g i The step of assigning to each group g i It is assumed that the task is solved so that the average pairwise affinity value of g is maximized. i The step of assigning to can be done disjunctively. Alternatively, the assignment does not necessarily have to be disjunctive. Furthermore, since affinity values ​​are not necessarily symmetric, it may be advantageous if the assigning step is done in excess. This means that a task can be assigned to more than one group. For example, task T a and task T b and affinity value Z b→a ,Z a→b In contrast, Z a→b >Z b→a In this case, the value Z a→b contains a correspondingly high value, so task T a Task T b When trained using task T b There is positive interference with respect to the value Z b→a For example, if b Task T a When trained using task T aThere is negative interference for task T a is task T b In this case, task T a and T b are assigned to the same group, except that task T a is additionally advantageously assigned to further groups to which no tasks are assigned.

[0015] It may be advantageous if the maximum number of groups N is preset, for example the maximum number may be adapted to the available computing resources.

[0016] The problem of assigning tasks to groups to be solved can be solved efficiently using solving techniques such as branch and bound or binary integer programming (BIP).

[0017] According to one embodiment, it is envisaged that determining the branching depth d comprises training a number of networks with different branching depths and selecting the branching depth based on the predictive accuracy of the networks, which may be determined, for example, using suitable validation data.

[0018] Approaches known from the prior art use methods consisting of Neural Architecture Search (NAS) to estimate the optimal branching configuration by training a much larger neural network that includes all possible branches as sub-networks. By training several individual networks, the method according to the invention is able to find the optimal branching depth using a computationally simple program that requires much less GPU memory for each individual operation, since the entire supergraph is not determined in one go.

[0019] Basically, after determining the branching depth d, the above-described method can be repeated, as will be explained below on the basis of subgroups and sub-branching networks.

[0020] According to one embodiment, each group g containing two or more tasks i For each group g i The tasks to be solved are divided into M subgroups (ug i ) (i=1~N) for each group g i The method is then tested for whether the addition of a 2-amino acid pair increases the average pairwise affinity value of the 2-amino acid pair.

[0021] According to one embodiment, as a result of the inspection, each group g i If it is found that increasing the pairwise affinity value of Each group g i Task T to be solved i Based on pairwise affinities, we divide M subgroups ug i (i=1 to M), Each branch network Z i determining a sub-branch depth ud in is executed, and the subbranch depth ud is the branch Z i Layer L i At the throat depth, the branch Z i , M sub-branch network UZ i It is assumed that it is set in advance how the signal is branched into (i=1 to M).

[0022] The goal of further branching into sub-branching networks is to further divide the i-th branching network into M sub-branching networks. The optimal sub-branching depth ud lies in the set {d+1,d+2,...,L-1}, i.e. at a depth greater than the branching depth d.

[0023] According to one embodiment, each one of the subgroups ug i For each group g i It is assumed that the above steps for checking and allocating are performed iteratively for . In this way, the architecture of the multitask model can still be further improved.

[0024] A further embodiment relates to a multitask model having L layers for solving at least two, in particular a number of, mutually distinct tasks, using an architecture determined according to the method according to the embodiment described above.

[0025] A further embodiment relates to a computer program product which, when executed on a computer, causes the computer to carry out the method according to the above-described embodiment.

[0026] Further embodiments relate to the use of a method according to the above-mentioned embodiments for determining an architecture of a multitask model having L layers for solving at least two, in particular a plurality of mutually distinct tasks and / or for determining a multitask model having L layers for solving at least two, in particular a plurality of mutually distinct tasks.

[0027] According to one embodiment, it is envisaged that the multitask model is constructed or trained to solve tasks in the area of ​​object recognition and / or for controlling technical systems, in particular also for solving tasks in the two areas of object recognition and control of technical systems, which are used for example in the area of ​​autonomous or semi-autonomous navigation and robotics.

[0028] For object recognition, for example, a multiplicity of digital images, in particular digital video images, digital radar images, digital lidar images, digital ultrasound images or digital motion sensor images, are provided. A multitask model is trained for object recognition. The trained multitask model is configured to determine a result of classification of at least one object in the digital images. For controlling the movement of a technical system, the method comprises the steps of capturing digital images by means of a sensor and controlling the movement of the technical system, in particular a machine, a robot, a vehicle, an aircraft, depending on the result of classification of at least one object in the digital images.

[0029] A multitask model having an architecture determined according to the above method can be advantageously used to simultaneously solve multiple tasks in environments with limited computational budgets, for example in the case of autonomous driving, an autonomous agent needs to solve multiple tasks in real time, such as localization and classification of other road users, e.g. cars, pedestrians, cyclists, etc., identification of runnable road surfaces, identification of traffic signs, etc., where the available computational power is limited by the hardware present in the vehicle.

[0030] Further advantages will become apparent from the description and the attached drawings. Exemplary embodiments of the invention are illustrated in the drawings and are explained in more detail in the following description, where identical reference symbols in the various drawings respectively denote identical elements or at least functionally equivalent elements. In the description of the individual drawings, reference may also be made, where appropriate, to elements in other drawings. [Brief description of the drawings]

[0031] [Figure 1] FIG. 1 is a schematic diagram showing a model known from the prior art; [Diagram 2] FIG. 1 is a schematic diagram illustrating a multitasking model having an architecture according to a first embodiment. [Diagram 3] FIG. 4 is a schematic diagram illustrating a multitasking model having an architecture according to a second embodiment. [Figure 4] FIG. 2 is a schematic diagram illustrating the steps of a method for determining the architecture of a multitask model NN. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] From the prior art (see FIG. 1), it is known to assign different tasks to different models. According to the publications "Efficiently identifying task groupings for multi-task learning" Advances in Neural Information Processing Systems 34 (2021): 27503-27516" by Fifty, Chris et al. and "Which tasks should be learned together in multi-task learning?" International Conference on Machine Learning. PMLR, 2020" by Standley, Trevor et al., affinity between tasks is used to assign task groups to individual networks.

[0033] The present disclosure relates to a method in which the pairwise affinity of tasks is utilized to envision a single multitask network in which the negative interactions between tasks are attenuated, i.e., according to the present method, neither separate base networks nor backbones are touched, and tasks to be solved are not assigned to separate networks. According to the present disclosure, the architecture of the single network is determined by a tree topology in which multiple branching networks or headers are connected to one common base network or backbone. Due to the pooling of computing resources, the multitask architecture is generally more advantageous than a collection of independent individual task networks in terms of the compromise between efficiency and accuracy.

[0034] In the following, a method 100 for determining the architecture of a multitask model NN, for example a neural network, is described on the basis of FIGS.

[0035] 2 and 3 show, for example, a multitask model having an architecture according to a first embodiment and a multitask model having an architecture according to a second embodiment.

[0036] A multitask model NN is a neural network that performs at least two, and in particular multiple, mutually distinct tasks T i (

number

[0037] In this example, the multitask model NN is illustratively configured to handle three mutually distinct tasks T 1 ,T 2 ,T 3 L = 3 layers for solving the set of 1 ,L 2 ,L 3 Includes.

[0038] The method 100 envisages the following steps (see also FIG. 4):

[0039] In step 102, a task T to be solved is i For a set of tasks, T i Pairwise affinities between are estimated.

[0040] Pairwise affinity is the task T a ,T b Each pair of iThe value indicates how well a task can be trained in a network with a high affinity. A high affinity value means positive interference between the task pair, whereas a low affinity value means negative interference between the task pair. For example, pairwise affinity can be determined by training a pair of tasks together in a single multitasking network and quantifying the effect of the gradient updates of a task on the other task. More than two pairs of tasks can also be trained together in a single multitasking network.

[0041] According to one embodiment, it is contemplated that pairwise affinity includes an affinity value for a first task of the pair and an affinity value for a second task of the pair. For example, task T a and task T b For a task pair containing task T a Affinity value Z for b→a And Task T b Affinity value Z for a→b These affinity values ​​are not necessarily symmetric, i.e., Z b→a is Z a→b does not have to be equal to

[0042] For a task set that includes three or more tasks, the affinity for a particular task can be determined by determining the average pairwise affinity for that particular task. For example, for task T a ,T b ,T c For a set containing T a Affinity for task T a Task T for b and T c Averaging the pairwise affinities of [(Z b→a +Z c→a ) / 2].

[0043] In step 104, the task T to be solved is i Let g be a set of N groups based on pairwise affinities. i(i=1~N).

[0044] Task T to be solved i Group G i The step of assigning to each group g i The task to be solved is T i Group G i The step of assigning to can be done disjunctively. Alternatively, the assignment does not necessarily have to be disjunctive. Moreover, since affinity values ​​are not necessarily symmetric, it can be advantageous if the assignment step is performed in excess. This means that i This means that a task T can be assigned to more than one group. a and task T b and affinity value Z b→a ,Z a→b In contrast, Z a→b >Z b→a In this case, the value Z a→b contains a correspondingly high value, so task T a Task T b When trained using task T b There is positive interference with respect to the value Z b→a For example, if b Task T a When trained using task T a There is negative interference for task T a is task T b In this case, task T a and T b Group g i However, task T a A further group g to which no additional tasks are assigned i It is advantageous to also allocate

[0045] Group G iThe maximum number N can be preset.

[0046] The problem of assigning tasks to groups to be solved can be solved efficiently using solving techniques such as branch and bound or binary integer programming (BIP).

[0047] In step 106, layer L i A branching depth d of a multitask model NN having a layer L of the multitask model is determined. i At some depth, the task T to be solved i The basic network B of the multitask model is a network with N branches Z i , where i=1 to N, is set in advance.

[0048] The common base network B acts as a feature extractor, and the weights of the common base network B are shared depending on the task to be solved. i are task-specific sub-networks, and their weights are not used for all tasks, but only for the respective tasks.

[0049] Branch network Z i Task T i The allocation of i Task T i For the present method, this means that the allocation is available as a result from the previous step 104.

[0050] The purpose of step 106 for determining the branching depth d is to determine the number of layers L of the multitask model NN in which the backbone exists or branches, based on the basic network B to be used jointly. i The purpose of the present invention is to determine the optimal depth of the branching depth d = 2. For example, the basic network includes layers 1 and 2 of the multitask model, and the branching network Z 1 ,Z 2(See Figure 2.) A branching depth of d=1 means that, for example, the basic network includes layer 1 of the multitask model, and layer 1 is followed by a branching network Z 1 ,Z 2 This means that there is a branch to (see Figure 3).

[0051] According to one embodiment, it is envisaged that the step of determining the branching depth d comprises the steps of training a number of networks with mutually different branching depths and selecting an optimal branching depth based on the prediction accuracy of the networks, which can be determined, for example, using suitable validation data. For example, a network according to Fig. 2 and a network according to Fig. 3 can be trained and an optimal branching depth can be identified based on the prediction accuracy of the networks. Advantageously, the optimal branching depth is additionally selected depending on the required computer resource consumption of the neural network architecture.

[0052] Basically, after step 106 of determining the branching depth d, the method described above can be repeated, which is explained below on the basis of subgroups and sub-branching networks.

[0053] According to one embodiment, each group g containing two or more tasks i For each group g i M subgroups of tasks to be solved i (i=1~N) is assigned to each group g i , step 108 is envisaged in which it is checked whether the average pairwise affinity value of .

[0054] According to one embodiment, as a result of the inspection, each group g i If it is found that increasing the pairwise affinity of the nucleotides increases the pairwise affinity of the nucleotides, the following steps (collectively referred to as step 110): Each group g i Task T to be solved i Based on pairwise affinities, we divide M subgroups ugi (i=1 to M), Each branch network Z i determining a sub-branch depth ud in is performed, where the subbranch depth ud is the number of branches Z i Layer L i At the throat depth, the branch Z i , M sub-branch network UZ i It is assumed that it is set in advance how the signal is branched into (i=1 to M).

[0055] The goal of further branching into sub-branching networks is to further divide the i-th branching network into M sub-branching networks. The optimal sub-branching depth ud lies in the set {d+1,d+2,...,L-1}, i.e. at a depth greater than the branching depth d.

[0056] According to one embodiment, each one of the subgroups ug i For each group g i It is assumed that the above steps for checking and allocating are performed iteratively for . In this way, the architecture of the multitask model can still be further improved.

[0057] A further embodiment provides for at least two, in particular a plurality of, mutually distinct tasks T i and / or for determining the architecture of a multitask model NN having L layers for solving at least two, in particular a plurality of mutually distinct tasks T i This relates to the use of the method 100 according to the embodiment described above for determining a multitask model NN having L layers for solving

[0058] According to one embodiment, it is envisaged that the multitask model is constructed or trained to solve tasks in the area of ​​object recognition and / or for controlling technical systems, in particular also for solving tasks in the two areas of object recognition and control of technical systems, which are used for example in the area of ​​autonomous or semi-autonomous navigation and robotics.

[0059] For object recognition, for example with reference to input data D in figures 3 and 4, a number of digital images are provided, in particular digital video images, digital radar images, digital lidar images, digital ultrasound images or digital motion sensor images. A multitask model is trained for object recognition. The trained multitask model is configured to determine a result of classification of at least one object in the digital images. For controlling the movement of a technical system, the method comprises the steps of capturing digital images by means of a sensor and controlling the movement of the technical system, in particular a machine, a robot, a vehicle, an aircraft, depending on the result of classification of at least one object in the digital images.

Claims

1. At least two, in particular a plurality of mutually different tasks (T i ) ( [0010] ) to solve the set of L layers (L i ) (i=1 to L), comprising: The method (100) comprises the following steps: Task to be solved (T i ), the tasks to be solved (T i estimating (102) pairwise affinities between The task to be solved (T i ) into N groups (g i ) (i=1 to N) (104); The layer (L i ) of the multitasking model (NN), the branching depth (d) is determined by the layer (L i ) at what depth the task to be solved (T i The basic network (B) of the multitask model (NN) shared by N branch networks (Z i ) (i=1 to N), step (106); A method (100).

2. Pairwise affinity is the task (T i ) is a shared layer (L i 2. The method of claim 1, wherein the value indicates how well a network having the eigenvalues ​​can be trained.

3. The task to be solved (T i ) into group (g i The step of assigning (104) each group (g i 3. The method of claim 1 or 2, wherein the method is performed such that an average pairwise affinity value of

4. The step of determining the branch depth (d) (106) comprises: training a number of networks with different branching depths (d); selecting an optimal branching depth (d) based on the predictive accuracy of the network; The method (100) of any one of claims 1 to 3, comprising:

5. Two or more tasks (T i Each group (g i For each group (g i ) of the task to be solved (T i ), M subgroups (ug i ) (i = 1 to N) is assigned to each group (g i 5. The method (100) of claim 1, wherein the affinity of each of the first and second pairs of the first and second nucleotides is tested (108) to see whether it increases the average pairwise affinity value of each of the first and second pairs of the first and second nucleotides.

6. As a result of the above test, each group (g i ) is found to have increased pairwise affinity value, the following steps (110): Each group (g i ) task to be solved (T i ) into M subgroups (ug i ) (i=1 to M); Each branch network (Z i determining a sub-branching depth (u d ) in is executed, and the sub-branch depth (ud) is i ) of the layer (L i At what depth of the branch (Z i ) is a set of M sub-branching networks (UZ i 6. The method (100) according to claim 5, further comprising: setting in advance whether the branch is to be made to one of the following three paths (i=1 to M).

7. Each subgroup (ug i The method (100) according to claims 5 and 6, wherein the steps according to claims 5 and 6 are repeatedly performed for each of the first and second inputs.

8. A multitask model (N) having L layers for solving at least two, in particular a plurality of mutually distinct tasks, using an architecture determined according to the method (100) of any one of claims 1 to 7.

9. A computer program product which, when executed on a computer, causes the computer to carry out the method (100) of any one of claims 1 to 7.

10. For determining the architecture of a multitask model (NN) having L layers for solving at least two, in particular a plurality of mutually different tasks and / or for solving at least two, in particular a plurality of mutually different tasks (T i ) ( [0025] ) to solve the set of (L i 8. Use of the method (100) according to any one of claims 1 to 7 for determining a multitask model (NN) having i=1 to L.