Training neural network system to perform multiple machine learning tasks

By dynamically adding neural network parameters and parallel processing technology, the problem of neural network avoiding catastrophic forgetting, negative migration and gradient interference in multi-task environments is solved, and efficient training and information security are achieved.

CN119072696APending Publication Date: 2024-12-03GOOGLE LLC
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202380038622.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-05-19
Filing Date
2023-05-19
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The prior art is difficult to avoid catastrophic forgetting, negative migration and gradient interference when performing multiple different tasks effectively in neural networks, and it is difficult to efficiently manage and protect the information security of neural networks.

Method used

By dynamically adding new neural network parameters, freezing existing parameters, optimizing training processes with parallel processing and distributed computing, selecting appropriate candidate neural networks to perform new tasks, ensuring information security, avoiding catastrophic forgetting and negative migration, and Reduce gradient interference.

Benefits of technology

The ability of neural networks to efficiently train and reason in multi-task environments is realized, avoiding performance degradation, reducing computational costs and delays, and ensuring information security.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119072696A_ABST
    Figure CN119072696A_ABST
Patent Text Reader

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network system to perform a plurality of machine learning tasks.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] This specification relates to training neural networks.

[0002] A neural network is a machine learning model that uses one or more layers of nonlinear units to predict the output of a received input. In addition to the output layer, some neural networks also include one or more hidden layers. The output of each hidden layer is used as the input to the next layer in the network (i.e., the next hidden layer or output layer). Each layer of the network generates an output from the received input according to the current values ​​of the corresponding set of parameters. Summary of the invention

[0003] This specification describes a system implemented as a computer program on one or more computers in one or more locations that trains a neural network system to perform a plurality of different machine learning tasks. This specification also describes a system for executing such a neural network system at inference time.

[0004] Particular embodiments of the subject matter described in this specification can be implemented so as to realize one or more of the following advantages.

[0005] Using the techniques described in this specification, the system can train a single neural network system to perform multiple different machine learning tasks. In some implementations, even after the neural network system has been trained (and in some cases deployed) for a first set of tasks, the system can further train the neural network system for new tasks that the neural network system has not been trained for.

[0006] In some implementations, as the neural network system is trained for a new task, the number of additional network parameters required to learn the new task can be reduced because the neural network system can leverage knowledge encoded in existing network parameters when learning previous tasks. Alternatively or additionally, the time required to train the neural network system for a new task and / or the computational cost of training for a new task can be reduced due to information already encoded in the neural network system when learning previous tasks.

[0007] In some implementations, operations for training a neural network system to perform a new task may be parallelized, e.g., across different threads or cores of a parallel processing device or across different devices in a computer system. For example, the system may train multiple "candidate" neural networks to perform a new task in parallel (where the candidate neural networks may share some parameters with existing networks of the neural network system), and then select one or more of the trained candidate neural networks to include in the neural network system based on the performance of the trained candidate neural networks. Alternatively or in addition, operations for training a neural network system to perform respective different tasks may be parallelized, e.g., by training the candidate neural networks to perform the different tasks on respective different threads, cores, or devices.

[0008] Similarly, in some implementations, operations for performing reasoning steps using a trained neural network system can be parallelized. For example, corresponding network blocks of corresponding neural networks of the neural network system can be assigned to corresponding different computing nodes in a distributed computing node system.

[0009] In some implementations described herein, a training system may dynamically incorporate new tasks into a neural network system, including adding new network parameters to the neural network system to perform the new tasks. However, because the neural network system is sparse, i.e., only a portion of the network parameters of the neural network system are activated (in some cases a small portion, e.g., 10%, 1%, or 0.01%) given a particular network input, adding new network parameters has little or no effect on the latency, computational cost, and / or energy consumption of the neural network system when processing the network input. That is, the training system may encode new information into the neural network, thereby providing additional capabilities, without reducing the efficiency of the neural network system at reasoning time.

[0010] Using the techniques described in this specification, a neural network system can be trained to perform multiple different tasks without suffering from "catastrophic forgetting" in which a network is trained for a first task, then updated for a second task, and after the network update, the network is no longer able to achieve acceptable performance when performing the first task (i.e., the network "forgot" the information encoded during training for the first task). By "freezing" the existing neural networks of the neural network system when adding new neural networks for new tasks (i.e., not updating the network parameters of the existing neural networks when training the new neural networks, even if the existing neural networks and the new neural networks share parameters), the training system can ensure that the performance of the existing neural networks does not degrade. That is, the knowledge encoded in the trained neural networks of the neural network system is always retained.

[0011] Using the techniques described in this specification, a neural network system can be trained to perform multiple different tasks without suffering from "negative transfer," in which knowledge learned while training for a first task is exploited to degrade the performance of the neural network system when trained for a second task. By utilizing the "evolutionary" techniques described herein, in which a training system trains multiple candidate neural networks that share respective different subsets of the neural network system's existing parameters, the training system can identify which specific existing network parameters are useful when training for a new task, and can therefore only retain candidate neural networks that exploit information encoded in specific existing network parameters (and can, for example, discard candidate neural networks that do suffer from negative transfer).

[0012] Using the techniques described in this specification, a neural network system can be trained to perform multiple different tasks without suffering from gradient interference, in which multiple different gradient updates are synchronously applied to the same set of network parameters, thereby destroying the effectiveness of each individual gradient update. In some implementations described herein, although corresponding different candidate neural networks can be trained in parallel, only a single gradient update is applied at any given time for a particular candidate neural network. That is, instead of updating the network parameters of the neural network system for multiple different tasks simultaneously, a corresponding candidate neural network can be added to the neural network system for each separate task, where the addition of the candidate neural network does not affect the operation of any other neural network in the neural network system.

[0013] In some implementations described in this specification, a neural network system may identify one or more neural networks in a neural network as a "safe" neural network. A safe neural network is a neural network that is not included in a pool of neural networks from which a corresponding candidate neural network is selected. Thus, information encoded in a safe neural network may be prevented from being accessed by another system, such as a training system when adding a new task to the neural network system or an inference system that makes an inference call to a different neural network in the neural network system. For example, any network block updated during training of a safe neural network (i.e., any network block that was not shared and therefore frozen during training) may be prohibited from being shared by any other neural network in the neural network system, thereby again ensuring the security of information encoded in the safe network block. As a specific example, a safe neural network may be trained using a private (e.g., proprietary) data set, and thus any network updates generated using the private data set may be ensured to encode information only in the safe neural network and not in any other neural network through a shared network block. Thus, designating a neural network as "safe" by the neural network may ensure the security and privacy of information of the neural network.

[0014] The details of one or more embodiments of the subject matter of this specification are set forth in the following drawings and the description. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 An example multi-task training system is shown.

[0016] Figure 2 is a flow chart of an example process for performing the training phase.

[0017] Figure 3 is a flowchart of an example process for selecting a parent for a given candidate neural network.

[0018] Figure 4 An example of mutating a parent neural network is shown.

[0019] Figure 5 An example of performing a sequence of generations during a given training phase is shown.

[0020] Like reference numbers and designations throughout the various drawings refer to like elements. DETAILED DESCRIPTION

[0021] This specification describes a system implemented as a computer program on one or more computers in one or more locations that trains a neural network system to perform a number of different machine learning tasks.

[0022] This specification also describes a system for executing such a neural network system at inference time.

[0023] Figure 1 An example multi-task training system 100 is shown that trains a neural network system 150 that can perform multiple different machine learning tasks.

[0024] The multi-task training system 100 is an example of a system implemented as a computer program on one or more computers in one or more locations in which the systems, components, and techniques described below may be implemented.

[0025] After training, the multi-tasking system 100 or a different reasoning system uses the trained neural network system 150 to perform a number of different machine learning tasks.

[0026] Specifically, after neural network system 150 has been trained, neural network system 150 may include multiple different neural networks 110A-C, each neural network configured to perform a corresponding one of the multiple machine learning tasks for which the neural network system is configured.

[0027] Although Figure 1 Only three different neural networks 110A-C for three tasks AC are shown in FIG, but in reality the neural network system 150 may include more neural networks for more tasks. In some cases, the neural network system 150 may include more than one neural network for one or more of the tasks.

[0028] Each neural network 110A-C of neural network system 150 may include a set of multiple network blocks 120A-C.

[0029] Each network block 120A-C may in turn include one or more neural network layers. More specifically, a "network block" as used in this specification is a collection of one or more neural network layers that receives input ("block input") and processes the input to generate output ("block output").

[0030] Each neural network 110A-C of neural network system 150 may include any suitable type of neural network layers for performing a corresponding machine learning task.

[0031] For example, neural networks 110A-C of neural network system 150 may include feed-forward neural network layers, convolutional neural network layers, recurrent neural network layers, and / or self-attention neural network layers.

[0032] Example machine learning tasks that neural networks 110A-C in neural network system 150 may be configured to perform are discussed below.

[0033] Each neural network 110A-C of neural network system 150 may share at least one corresponding network block 130A-C with one or more other neural networks 110A-C of the neural network system.

[0034] In this specification, a specific network block is “shared” between two neural networks if both neural networks include a network block having the same architecture and the same network parameters as the specific network block.

[0035] In some implementations, the respective shared network blocks 130A-C of the two neural networks are implemented on respective different computing nodes (e.g., on respective different devices, or on respective different threads or cores of a device). That is, both neural networks may include respective "copies" of the shared network blocks that are executed separately.

[0036] In some other implementations, the system 100 includes a single computing node that performs the operations of the shared network blocks 130A-C. That is, there is a single "copy" of the shared network blocks 130A-C to which both neural networks can send requests to process the respective block inputs to generate the respective block outputs.

[0037] Training system 100 may train neural network system 150 to perform the active task by adding one or more new neural networks to the current set of neural networks 110A-C of neural network system 150 at each of a plurality of stages corresponding to respective tasks (referred to herein as “active” tasks).

[0038] That is, at each stage, neural network system 150 includes the current set of neural networks 110A-C that were added at the corresponding previous stage, and training system 150 may add one or more additional neural networks to perform the active task as a result of executing that stage.

[0039] Prior to the first stage, i.e., before the training system 150 has trained any neural network, the current set of neural networks can be initialized to include one or more randomly initialized or pre-trained neural networks. For example, the current set can be initialized to have a single high-performance root neural network (e.g., Vision Transformer), a convolutional neural network (e.g., ResNet), an encoder-decoder Transformer (e.g., T5 Transformer), etc. The parameters of the root neural network can be randomly initialized or pre-trained, for example, by supervised learning, unsupervised learning, or both on a large data set.

[0040] The stages of performing the training process are described in more detail below.

[0041] By repeatedly performing the training phase, the training process 150 can expand the neural networks available for use at inference, for example, by adding new neural networks that can perform new tasks, by adding new neural networks that perform the same tasks as existing neural networks, or both.

[0042] In general, the multi-tasking system 100 or reasoning system may perform reasoning using the neural networks 110A-C in the neural network system 150 at any given point during training, after all training phases are completed, or both, e.g., processing the respective inputs 102A-C of tasks AC using the corresponding neural networks 110A-C to generate the respective outputs 104A-C of tasks AC. For example, the system 100 or reasoning system may perform reasoning in parallel with the training system 150 performing a phase of adding additional neural networks to the neural networks in the neural network system 150.

[0043] Thus, system 100 may implement a “continuous learning” scheme in which new neural networks for new tasks may be continually added to the available neural networks while leveraging information encoded in the available neural networks and without degrading the performance of the available neural networks.

[0044] As described above, each neural network in neural network system 150 may be configured to process any suitable type of network input and perform any suitable machine learning task, such as a classification task, a regression task, or a combination thereof. In some implementations, each neural network in the neural network system is configured to process the same type of network input, such as an input image or input text. That is, each of the plurality of tasks may require processing the same type of network input.

[0045] In some other implementations, different neural networks in the neural network system may be configured to process corresponding different network inputs, that is, two or more tasks in the plurality of tasks need to process network inputs of different types from each other.

[0046] Some example network inputs and machine learning tasks are discussed below.

[0047] For example, the network input may represent an input image, and the machine learning task may be an image processing task. The neural network may be configured to process any suitable type of image, for example, an RGB image, a LIDAR image (e.g., a point cloud), etc. The system may divide the image into a plurality of different image blocks, each of which includes a different subset of pixels of the image. Thus, the input elements of the input sequence may represent corresponding image blocks of the input image. In this specification, processing an image refers to processing the intensity values ​​of the pixels of the image.

[0048] As a specific example, a neural network may be configured to generate a classification output that includes a respective score corresponding to each of a plurality of categories. The score for a category indicates the likelihood that the network input belongs to the category. In some cases, a category may be a class of objects (e.g., dog, cat, person, etc.), and if the network input represents an object included in the class of objects corresponding to the category, then the network input may belong to the category. In some cases, a category may represent a global attribute (e.g., the network input represents an environment of day or night, or the network input represents an environment of summer or winter), and if the network input has a global attribute corresponding to a category, then the network input may belong to the category.

[0049] As another specific example, a neural network may be configured to generate an element-level classification output (e.g., a pixel-level classification output for an RGB image or a point-level classification output for a LIDAR image) that includes, for each element in the network input, a corresponding score corresponding to each of a plurality of categories. For a given element (e.g., for a given pixel or point), the score for the category indicates the likelihood that the element belongs to that category. In some cases, the category may be a class of an object, and if the element is part of an object included in an object class corresponding to the category, the element may belong to that category. That is, the element-level classification output may be a semantic segmentation output.

[0050] As another specific example, a neural network may be configured to generate a regression output that estimates one or more continuous variables (i.e., that may assume an infinite number of possible values) that characterize the network input. In a specific example, if the network input represents an image, the regression output may estimate the coordinates of a bounding box that encloses a corresponding object depicted in the image. The coordinates of the bounding box may be defined by the (x, y) coordinates of the vertices of the bounding box.

[0051] As another example, the network input may represent a sequence of audio data, and the machine learning task may be a speech recognition task, where the neural network is configured to process a representation of an audio waveform to generate an output representing a sequence of phonemes, characters, or words corresponding to the audio waveform.

[0052] As another example, the network input may represent a sequence of video frames, and the machine learning task may be a video analysis task, in which a neural network is configured to process the sequence of video frames to generate an output that characterizes the video frames, e.g., by characterizing whether the video frames depict a person performing a particular action.

[0053] As another example, the network input may represent a sequence of text data, and the machine learning task may be a natural language processing task, in which a neural network is configured to process a portion of the text to generate an output that characterizes the portion of the text, such as by characterizing a translation of the portion of the text into a different natural language.

[0054] As a specific example, the task can be a natural language processing or understanding task that operates on a text sequence of some natural language, such as an entailment task, a paraphrase task, a text similarity task, a sentiment task, a sentence completion task, a grammar task, etc.

[0055] As another specific example, the task can be a text generation task, where the input is a text sequence and the output is another text sequence, such as a completion of the input text sequence, a response to a question posed in the input sequence, or a text sequence about a topic specified by the first text sequence. For example, the neural network can be an autoregressive neural network, such as an autoregressive neural network based on self-attention. As another example, the input of the text generation task can be an input other than text, such as an image, and the output sequence can be text describing the input.

[0056] Figure 2 2 is a flow chart of an example process 200 for performing a training phase during training of a neural network system. For convenience, process 200 will be described as being performed by a system of one or more computers located in one or more locations. For example, a suitably programmed training system (e.g., Figure 1 The training system 100) can execute process 200.

[0057] As described above, the system may train the neural network system to perform the active task by adding one or more new neural networks to the current set of neural networks of the neural network system at each of a plurality of stages corresponding to a respective task (referred to herein as an "active" task).

[0058] That is, at each stage, the neural network system includes the current set of neural networks that have been added at the corresponding previous stage, and by executing that stage, the training system can add one or more additional neural networks to perform the active task.

[0059] In some implementations, the training system may perform multiple different phases for the same active task. That is, at a first phase, the training system may add a new neural network to the neural network system to perform the active task, and at a second phase, the training system may add another new neural network to the neural network system to perform the active task (optionally removing the neural network added at the first phase).

[0060] At each stage, the system may train multiple candidate neural networks by performing steps 202 to 212 for each candidate neural network.

[0061] To train a candidate neural network, the system may select a “parent” neural network for the candidate, for example from the current set of neural networks of the neural network system or from candidate neural networks that have been trained at the current stage (step 202).

[0062] References below Figure 3 Example techniques for selecting a parent neural network are described in more detail.

[0063] The system then “mutates” the parent neural network to generate a candidate neural network (step 204).

[0064] That is, the candidate neural network will have an architecture, hyperparameters, or both that are modifications to the architecture, hyperparameters, or both of the parent neural network.

[0065] However, as part of generating the candidate neural network, the system does not modify the architecture or the parent neural network. That is, although the parent neural network and the candidate neural network will share one or more network blocks, the shared network blocks will not be updated during the training of the candidate neural network. That is, during the training of the candidate neural network, only the network blocks that are not shared with the parent neural network will be updated.

[0066] Next, a specific example of mutating a parent neural network to generate a candidate neural network will be described.

[0067] The system identifies a first strict subset of the network blocks of the parent neural network, step 204. That is, the subset is a strict subset because it includes less than all of the network blocks of the parent neural network.

[0068] The copies of the network blocks in the first strict subset will be updated during the training of the candidate neural network.

[0069] That is, during training of the candidate neural network, the network parameters of the parent neural network are not updated (i.e., "frozen"), and so the system copies the architecture and parameter values ​​of a first strict subset of the network blocks to generate a copy that can be updated during training of the candidate neural network.

[0070] The training system may further identify a second strict subset of network blocks of the parent neural network to be shared between the parent neural network and the candidate neural network in the neural network system.

[0071] In some implementations, the second strict subset includes all network blocks of the parent neural network that are not in the first strict subset. That is, any blocks that will not be copied are shared between the parent and the candidate.

[0072] In some other implementations, the system may further identify a third strict subset of network blocks of the parent neural network to be removed from the candidate neural network.

[0073] Likewise, during training of the candidate neural network, the parent neural network remains unchanged, so in order to "remove" a network block in the third strict subset from the candidate neural network, the system may add a connection between a previous network block in the candidate neural network and a subsequent network block in the candidate neural network (i.e., a connection through which the block output of the previous network block will be provided as a block input to the subsequent network block).

[0074] In these implementations, the second strict subset includes all network blocks of the parent neural network that are not in the first strict subset and not in the third strict subset.

[0075] In some implementations, the training system can further add one or more new network blocks to the candidate neural network that are not in the parent neural network.

[0076] As an alternative or in addition to mutating the architecture of the parent, the training system may modify the set of hyperparameters used to train the candidate neural network relative to the hyperparameters used to train the parent neural network.

[0077] References below Figure 4 The example of mutating a parent is described in more detail.

[0078] The system may initialize the candidate neural network (step 206).

[0079] As part of initializing the candidate neural network, the system generates a copy of each network block in the first strict subset and includes the copied network blocks in the architecture of the candidate neural network.

[0080] The system also includes in the architecture a shared block corresponding to each block in the second strict subset of the parent block.

[0081] In some implementations, each respective shared network block shared between two neural networks is implemented on a respective different computing node (e.g., on a respective different device, or on a respective different thread or core of a device). That is, both neural networks may include respective "copies" of the shared network block that are executed separately. In these cases, the system prevents modification of parameter values ​​of the copy of the shared block corresponding to the candidate neural network during training of the candidate neural network.

[0082] In some other implementations, the system includes a single compute node that performs the operations of any given shared network block. That is, there is a single "copy" of the shared network block to which both neural networks can send requests to process the respective block inputs to generate the respective block outputs. In these cases, the system prevents modification of the parameter values ​​of the shared blocks during the training of the candidate neural networks.

[0083] When one or more new network blocks are added to the candidate neural network, the system also adds the new network blocks to the candidate architecture.

[0084] In some implementations, the system also adds an output head network block to each candidate neural network, which receives the output of the last network block in the candidate architecture and maps the output to the dimensions required by the active task. The output head network block can be specified as trainable to allow the neural network to map the internal representation to the output of the task. That is, in these implementations, the system automatically replaces the parent output head network block with a new, trainable output head network block specific to the active task.

[0085] The system may then train the initialized candidate neural network for the active task (step 208), i.e., generate trained values ​​for a set of network parameters of the candidate neural network using training data corresponding to the active task.

[0086] This may include, for example, updating a respective set of parameters for each network block in the first subset of network blocks and updating a respective set of parameters for any new network blocks that are added based on the error of the candidate neural network on the training data using back propagation and gradient descent. During this training, the shared network blocks are not updated and remain frozen.

[0087] After training, the system may then determine a performance score for the trained candidate neural network (step 210).

[0088] The performance score characterizes at least the performance of the trained candidate neural network on the active task, such as validation accuracy, test accuracy, precision, and / or recall.

[0089] Optionally, the performance score may further characterize the efficiency of the trained candidate neural network in performing or being trained for the active task.

[0090] For example, a performance score may be generated from or otherwise encode information about: the latency of the candidate network when performing the active task; the number of training examples or iterations, and / or the amount of time required to train the candidate neural network to perform the active task; the number of network parameters of the candidate neural network, e.g., weighted by the degree to which the network parameters are shared with other neural networks in the neural network system; a measure of energy consumption of the candidate neural network when inferencing or training, and the like.

[0091] Thus, by incorporating information not only about the quality of predictions generated by a candidate neural network, but also about the efficiency of the candidate neural network in making predictions, the system can balance the performance / cost (e.g., computational or energy cost) tradeoff.

[0092] That is, the system can calibrate the performance scores to encourage the candidate neural networks selected based on the performance scores to have the desired quality. Depending on the requirements of the system, this trade-off can be different for different candidate neural networks.

[0093] For example, if a neural network system is to be deployed in a resource-constrained environment, such as on an edge device such as a mobile phone or tablet computer, the training system may select energy metrics to encourage relatively efficient networks.

[0094] As another example, if marginal improvements in the quality of a neural network system's network output are more important than efficiency concerns, the training system may select energy metrics so as to encourage relatively high-performance networks.

[0095] Optionally, the formula for the performance score can be dynamically updated during training of the neural network system; for example, if a certain resource is about to become temporarily scarce (e.g., within the next few hours or days), the performance score can be modified to allow the training or multi-tasking system to use that resource more efficiently in the limited time.

[0096] In some implementations, in addition to calculating the performance score at the end of training, the system may also calculate the performance score at certain checkpoints during training (e.g., after each training cycle). In these implementations, the system may designate the version of the candidate neural network that resulted in the calculation of the best performance score as the trained candidate neural network, i.e., such that when the candidate neural network achieves a better performance score at some intermediate checkpoint than after training is complete, the candidate neural network as of the intermediate checkpoint is used as the trained neural network, rather than the candidate neural network after training is complete.

[0097] Some specific examples of performance scores will now be described.

[0098] As an example, the performance score of each trained candidate neural network m may include a term that is inversely proportional to: where P(m) identifies a plurality of network parameters of a trained candidate neural network, and #models(p) identifies, for network parameter p, the number of neural networks in the current set of neural networks that include network parameter p (i.e., have the same value for the same network parameter for the same shared network block). In some cases where the current set of neural networks may include multiple models that perform the same task, the system may only count neural networks that perform tasks different from the active task when calculating #models(p).

[0099] As a specific example, the performance score of each trained candidate neural network m may be equal to or proportional to: where q(m) identifies the validation or test accuracy of the trained candidate neural network, s is a hyperparameter of the neural network system and represents a scaling factor for the impact of the size of the model, and #root-model-params identifies the number of network parameters in the root neural network included in the current set of neural networks (i.e., used to initialize the current set of neural networks) before the first stage in the multiple stages.

[0100] Therefore, the scaling factor s allows controlling the size of the generated multi-task model, i.e., the total size of the neural network in the neural network system, and achieving different quality / size trade-offs.

[0101] After performing steps 202 to 210 for each of the plurality of candidate neural networks, the system may determine that the phase has been completed.

[0102] In some implementations, the system performs steps 202 to 210 in parallel for each of the plurality of candidate neural networks, i.e., trains all candidate neural networks of the stage in parallel. For example, the system may dedicate a corresponding set of hardware accelerators to training each of the plurality of candidate neural networks.

[0103] In some other implementations, the system performs this phase of training across multiple "generations," where candidates within each generation are trained in parallel.

[0104] References below Figure 5 Describes performing training across multiple epochs in more detail.

[0105] Once this stage has been completed, the system may then use the respective performance scores of the candidate neural networks to select one or more candidate neural networks to add to the current set of neural networks of the neural network system (step 214).

[0106] In some implementations, the system adds only a single candidate neural network for each stage. In these implementations, the system may add only the candidate neural network with the highest performance score.

[0107] After completing a given stage, i.e., after a final candidate neural network has been added to the neural network system at the final stage or after a given intermediate stage, the neural network system including the neural network completed up to the given stage can be deployed in an inference environment, for example, on a distributed computing system including multiple processing devices (e.g., parallel processing devices, such as graphics processing units (GPUs) and / or tensor processing units (TPUs)). That is, the execution of the trained neural network system can be parallelized, for example, across multiple devices and / or across multiple threads or cores of a device.

[0108] In some cases, after this stage has been completed, and in response to selecting a particular candidate neural network to add to the current set of neural networks, the system may determine whether there are already any neural networks in the current set of neural networks that are configured to perform the active task, and if so, determine whether to remove any of the neural networks in the current set.

[0109] Specifically, the system can determine whether there are any neural networks in the current set that (i) have been trained to perform the active task and (ii) have a performance score lower than the specific candidate neural network. If any neural networks in the current set meet both (i) and (ii), the system can remove these neural networks from the neural network system, thereby effectively replacing the removed neural networks with the specific candidate neural network.

[0110] In some implementations, the system may designate certain of the neural networks in the current set of neural networks as “safe neural networks.” A safe neural network is a neural network that is in the current set of neural networks (and therefore available for performing inference) but is not included in the pool of neural networks determined from the current set from which the parent neural network of the corresponding candidate neural network is selected.

[0111] Thus, because the safe neural network is not included in the pool, for each safe neural network and for each network block of the safe neural network whose parameters are updated during training of the safe neural network, the network block is not shared by any other neural network in the neural network system. That is, while the safe neural network may share network blocks that remain frozen during training of the safe neural network with the parent of the safe neural network, none of the network blocks of the safe neural network that are copied (and therefore updated during training) are shared by any other neural network.

[0112] For example, the system may designate a given neural network as secure because the neural network has been trained on a private data set that was not available during the training of any other candidate neural network.

[0113] Figure 3 is a flow chart of an example process 300 for selecting a parent neural network for a given candidate neural network. For convenience, process 300 will be described as being performed by a system of one or more computers located in one or more locations. For example, a suitably programmed multi-task training system (e.g., Figure 1 The multi-task training system 100 may perform process 300.

[0114] In general, during this phase, the system maintains an active population of neural networks that can be used as parent neural networks during this phase. In some cases, the system initializes the active population to an empty set. In some other cases, the system initializes the active population to include any neural networks in the current set that have been trained to perform the active task, and if no neural network has been trained to perform the active task, the system initializes the active population to an empty set.

[0115] In some implementations, at the beginning of a stage, the system initializes a set of seed neural networks for that stage.

[0116] The set of seed neural networks typically includes (i) a root neural network included in the current set of neural networks before a first stage in the plurality of stages, (ii) for each task for which the neural network system has been trained at the corresponding previous stage, the neural network with the highest performance score among all neural networks trained to perform the task, or (iii) both.

[0117] At the beginning of this phase, the system may perform a "second" selection process to select parents from the seed neural network until certain criteria are met.

[0118] After determining that the criteria are met, the system may begin executing a "first" selection process to select a parent from the active population.

[0119] For example, the system may determine that certain criteria are satisfied in response to determining that one or more particular neural networks in the current set of neural networks have been trained to perform the active task (i.e., the active population is non-empty because it includes at least one particular neural network from the current set).

[0120] As another example, the system may determine that certain criteria are satisfied in response to determining that there are any neural networks that (i) are in the current set and have been trained to perform the active task or (ii) have been previously trained to perform the active task during the phase (i.e., the active population is non-empty because it includes at least one particular neural network from the current set or a candidate neural network has been added to the active population during the phase).

[0121] As another example, the system may determine that certain criteria are satisfied in response to determining that the seed set of the neural network is empty.

[0122] For example, to perform the second selection process, the system can select parents from the seed set by uniform random sampling without replacement, so that once a neural network is selected, it is removed from the seed set. Therefore, after all seed neural networks are selected, the seed set will become empty.

[0123] As another example, the system may perform the second selection process by performing the first selection process described below, but on the neural networks in the seed set rather than on the neural networks in the active population.

[0124] To perform the first selection process, the system performs steps 302 to 306 .

[0125] The system determines a ranking of the one or more specific neural networks (i.e., the one or more specific neural networks in the active population) based on the respective performance scores determined after training each of the one or more specific neural networks (step 302). That is, the system ranks the specific neural networks in the active population (also referred to as the "pool") such that the neural network with the highest performance score is at the highest position in the order.

[0126] Starting with an initial specific neural network in order, the system evaluates one or more corresponding specific neural networks (step 304).

[0127] Specifically, the system determines whether to select a particular neural network based on a random sample, that is, by sampling from a probability distribution.

[0128] In general, each particular neural network may be associated with a probability of selection, i.e., a probability of being selected when evaluated, wherein the lower the probability, the more times the particular neural network has been selected as a parent neural network in the current stage.

[0129] As a specific example, a particular neural network m may be selected with a probability equal to or proportional to: where #selections(m,t) identifies the number of times a particular neural network m has been selected as a parent neural network when training a candidate neural network to perform the active task in stage t. This therefore biases the selection process toward high-performance particular neural networks that have been selected as parents a relatively small number of times in the current stage.

[0130] In response to determining to select the specific neural network, the system ends the evaluation and uses the selected specific neural network as a parent.

[0131] In response to determining that the particular neural network is not selected, the system continues to evaluate the next particular neural network in the order.

[0132] If the system determines not to select the last neural network in the order, i.e., the system iterates through all specific neural networks in the active population and determines not to select any neural network, the system may select a parent neural network using a different technique (step 306).

[0133] In general, the system can select a parent neural network in any of a variety of ways.

[0134] For example, the system may select a neural network uniformly at random from the current set.

[0135] As another example, the system may uniformly randomly select a neural network from a set including the active population and the current set.

[0136] As yet another example, the system can generate a second set of neural networks, e.g., including all neural networks in the current set that have been trained to perform a task different from the active task. The system can then perform steps 302 and 304 for the neural networks in the second set. If, after performing steps 302 and 304 for the neural networks in the second set, the system still has not selected a parent neural network, the system can, e.g., uniformly randomly select a neural network from the current set.

[0137] As described above, once a parent is selected, a candidate neural network is generated and trained. After being trained, the candidate neural network can be added to the active population. For example, the system can add each candidate neural network to the active population. As another example, the system can add a candidate neural network to the active population only if the performance score of the candidate neural network exceeds the performance score of the parent selected for the candidate.

[0138] Figure 4 An example of mutating a parent neural network to generate a candidate neural network is shown. Figure 4 It can be seen that generating a candidate neural network is represented as generating a path including a (frozen and untrainable) network block on the parent model path and a (trainable) network block on the child model path.

[0139] exist Figure 4 In the example of Figure 2, four types of mutations are shown: layer cloning, layer insertion, layer removal, and hyperparameter (“hyperparam”) changes.

[0140] Layer cloning mutation refers to mutation in which a strict first subset of network blocks is selected to be copied from the parent to the candidate neural network and can be trained.

[0141] exist Figure 4 In the example of , the system has selected the "Layer B" network block to be in the first strict subset.

[0142] As an example, to perform layer cloning mutation, i.e., to generate the first strict subset, the system may, for each network block in some or all network blocks b of the parent neural network, generate a first strict subset with probability The network blocks are selected to be included in the first strict subset. That is, the system can independently select the network blocks with probability Determine to include each network block b in the first strict subset. In some implementations, the corresponding probability of each network block b is are the same.

[0143] Layer insertion mutation refers to a mutation that adds one or more new network blocks that are not in the parent neural network to the candidate neural network.

[0144] exist Figure 4 In the example of , after training the model Task 2 candidate neural network, the system uses the Task 2 candidate neural network as the parent neural network and mutates the Task 2 candidate neural network by adding a network block "Layer D" between Layer B and Layer C.

[0145] For example, by taking each pair p in one or more pairs of consecutive network blocks of the parent neural network, with probability Determining to add a new network block between corresponding pairs of network blocks in a candidate neural network, the system can determine whether to add any new network blocks to a given candidate. In some implementations, the corresponding probability of each pair p of network blocks is are the same.

[0146] In general, the system inserts a new network block that processes an input having the dimensionality of the output of the first block in the pair to generate an output that preserves that dimensionality. As a specific example, the system may add a residual adapter network block. An example of such a network block is a sequence of two fully connected layers with variable internal dimension sizes. The Gelu nonlinearity is applied to the internal representation, and layer normalization is applied to the input of the fully connected layer. The second layer may be zero-initialized to ensure that its insertion does not change the parent model representation at the beginning of the child training.

[0147] Layer removal mutation refers to mutation that identifies a third strict subset of network blocks of the parent neural network to be removed from the candidate neural network.

[0148] Likewise, during training of the candidate neural network, the parent neural network remains unchanged, so in order to "remove" a network block in the third strict subset from the candidate neural network, the system may add a connection between a previous network block in the candidate neural network and a subsequent network block in the candidate neural network (i.e., a connection through which the block output of the previous network block will be provided as a block input to the subsequent network block).

[0149] This can be obtained from Figure 4 As can be seen in the example, layer C of the parent neural network of task 3 is not in the path of the candidate neural network of task 4, so there is a connection between layer D and the output head network block of the candidate neural network of task 4.

[0150] For example, by targeting each network block in one or more network blocks b of the parent neural network with probability The system can perform such mutation by selecting network blocks to include in the third strict subset. In some cases, to avoid knowledge and representation disruption caused by removing internal layers, the one or more network blocks that can be removed include only the last network block in the parent neural network immediately before the output head network block.

[0151] Hyperparameter mutation adjusts the hyperparameters of the parent neural network and uses the adjusted hyperparameters to train the candidate neural network.

[0152] Specifically, when hyperparameter variation is used, the system maintains, for each neural network in the current set, data identifying corresponding values ​​of a set of one or more hyperparameters used during training of the neural network.

[0153] Then, before training the candidate neural network, the system can identify a subset of the set of hyperparameters for training the parent neural network.

[0154] For each hyperparameter that is not in the identified subset, the system determines to train the candidate neural network using the same value for the hyperparameter as used to train the parent neural network.

[0155] For each hyperparameter in the identified subset, the system may select a new value for the hyperparameter to train the candidate neural network.

[0156] To identify a subset of the set of hyperparameters, the system may, for each of the one or more hyperparameters h, use the probability Hyperparameters are selected to be included in the subset. In some implementations, the corresponding probability of each hyperparameter h is are the same.

[0157] To select new values ​​for hyperparameters for use in training a candidate neural network, the system may identify a ranking of a finite set of possible values ​​for the hyperparameter and then follow the ranking to identify specific values ​​used during training of a parent neural network.

[0158] The system can then select a neighboring value that is adjacent to the particular value in that rank order. That is, the system can select (i) a value that is immediately before the particular value used during training of the parent neural network, or (ii) a value that is immediately after the particular value used during training of the parent neural network.

[0159] exist Figure 4 In the example of , the system has chosen to modify the learning rate hyperparameter that was previously set to .5. The system then selects a new value of .2 by uniformly randomly selecting one of two adjacent values ​​of .5 in the ordered list of values ​​for the learning rate.

[0160] Figure 5 is a flow chart of an example process 500 for executing a generation sequence during a given training phase. For convenience, process 500 will be described as being performed by a system of one or more computers located in one or more locations. For example, a suitably programmed multi-task training system (e.g., Figure 1 The multi-task training system 100 may execute process 500.

[0161] Specifically, the system can perform steps 502 to 508 of process 500 for each generation in a sequence of generations in order to perform a training phase. In general, the sequence will have a fixed number of generations that can be received as input by the system or can be determined by the system based on how much computing resources are available to the system when the training phase begins, for example.

[0162] The system trains multiple candidate neural networks to perform the active task (step 502). For example, the system can train each candidate neural network in parallel by dedicating one or more hardware accelerators to the training of the candidate neural networks.

[0163] For each candidate neural network, the system determines whether the performance score of the candidate neural network is greater than the performance score generated for the corresponding parent neural network during the training of the neural network (step 504). Figure 2 The performance scores of the parent neural network and the candidate neural network are calculated as described above.

[0164] For each candidate neural network, if the performance score of the candidate neural network is greater than the performance score generated for the corresponding parent neural network, the system adds the candidate neural network to a pool of possible parent neural networks (the "active population") for use in any subsequent generations in the sequence of generations (step 506).

[0165] For each candidate neural network, if the performance score of the candidate neural network is not greater than the performance score generated for the corresponding parent neural network, the system discards the candidate neural network (step 508). Therefore, the parent neural networks of subsequent generations can only be selected from the candidate neural networks that have not been discarded, that is, can only be selected from the candidate neural networks that have better scores than their parents.

[0166] After executing the generation sequence, the system selects one or more candidate neural networks to be added to the current set from the candidate neural networks generated at the corresponding generation that were not discarded (step 510), for example, as described above with reference to Figure 2 described.

[0167] This specification uses the term "configured" in conjunction with system and computer program components. For a system of one or more computers to be configured to perform a particular operation or action, it means that the system has installed thereon software, firmware, hardware, or a combination thereof that causes the system to perform the operation or action in operation. For one or more computer programs to be configured to perform a particular operation or action, it means that the one or more programs include instructions that, when executed by a data processing device, cause the device to perform the operation or action.

[0168] Embodiments of the subject matter and functional operations described in this specification may be implemented in digital electronic circuit systems, in tangibly embodied computer software or firmware, in computer hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more of them. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory storage medium for execution by a data processing device or for controlling the operation of a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. Alternatively or in addition, program instructions may be encoded on an artificially generated propagation signal (e.g., a machine-generated electrical, optical, or electromagnetic signal), generating an artificially generated propagation signal to encode information for transmission to a suitable receiver device for execution by a data processing device.

[0169] The term "data processing equipment" refers to data processing hardware and encompasses all kinds of equipment, devices and machines for processing data, including, for example, a programmable processor, a computer or multiple processors or computers. The equipment may also be or include a dedicated logic circuit system, for example, an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, the equipment may optionally include code that creates an execution environment for a computer program, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0170] A computer program, which may also be referred to or described as a program, software, software application, app, module, software module, script or code, may be written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages); and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, subroutines or code portions). A computer program may be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a data communications network.

[0171] In this specification, the term "database" is used broadly to refer to any collection of data: the data need not be structured in any particular way, or at all, and may be stored on a storage device in one or more locations. Thus, for example, an index database may include multiple collections of data, each of which may be organized and accessed in a different way.

[0172] Similarly, in this specification, the term "engine" is used broadly to refer to a software-based system, subsystem, or process that is programmed to perform one or more specific functions. Typically, an engine will be implemented as one or more software modules or components installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a specific engine; in other cases, multiple engines can be installed and run on the same one or more computers.

[0173] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by a dedicated logic circuit system (e.g., an FPGA or ASIC) or by a combination of a dedicated logic circuit system and one or more programmed computers.

[0174] A computer suitable for executing a computer program can be based on a general or special microprocessor or both, or any other kind of central processing unit. Typically, the central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented by a dedicated logic circuit system or incorporated into the dedicated logic circuit system. Typically, the computer will also include one or more large-capacity storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or be operatively coupled to receive data from a large-capacity storage device or transfer data to a large-capacity storage device or both. However, the computer does not need to have such a device. In addition, the computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device, such as a universal serial bus (USB) flash drive, to name a few.

[0175] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices); magnetic disks (e.g., internal hard disks or removable disks); magneto-optical disks; and CD ROM and DVD-ROM disks.

[0176] To provide interaction with a user, embodiments of the subject matter described in this specification may be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including sound, voice, or tactile input. In addition, the computer may interact with the user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser on a user's device in response to a request received from the web browser. In addition, the computer may interact with the user by sending text messages or other forms of messages to a personal device (e.g., a smart phone running a messaging application) and receiving responsive messages from the user in response.

[0177] A data processing device used to implement a machine learning model may also include, for example, dedicated hardware accelerator units for processing general-purpose and computationally intensive parts of machine learning training or production (i.e., inference, workloads).

[0178] Machine learning models can be implemented and deployed using a machine learning framework (e.g., the TensorFlow framework or the Jax framework).

[0179] Embodiments of the subject matter described in this specification may be implemented in a computing system that includes a back-end component (e.g., as a data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a client computer with a graphical user interface, a web browser, or an app through which a user can interact with an implementation of the subject matter described in this specification), or any combination of one or more such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

[0180] A computing system may include a client and a server. The client and server are usually remote from each other and typically interact through a communication network. The relationship of client and server is due to computer programs running on respective computers and having a client-server relationship with each other. In some embodiments, the server transmits data (e.g., an HTML page) to a user device, for example, for displaying data to a user interacting with a device acting as a client and receiving user input from it. Data generated at a user device, for example, due to user interaction, may be received at the server from the device.

[0181] Although this specification contains many specific implementation details, these details should not be interpreted as limitations on the scope of any invention or the scope that may be claimed, but should be interpreted as descriptions of features that may be unique to a particular embodiment of a particular invention. Certain features described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination. In addition, although features may be described above as working in certain combinations and even initially claimed as such, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may involve a sub-combination or a variant of a sub-combination.

[0182] Similarly, although operations are depicted in the drawings and described in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in a continuous order, or that all of the operations shown be performed, to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0183] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. For example, the actions recited in the claims can be performed in a different order and still achieve the desired results. As an example, the processes depicted in the accompanying drawings do not necessarily require the particular order shown or ordered order to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. A method for training a neural network system to perform a number of different machine learning tasks, After training: The neural network system includes a plurality of neural networks, each of the plurality of neural networks being configured to perform a corresponding one of the plurality of machine learning tasks, Each neural network of the plurality of neural networks comprises a respective set of network blocks, and Each neural network shares at least one corresponding network block with one or more other neural networks in the plurality of neural networks, The method includes, at each of a plurality of stages, training the neural network system to perform a corresponding active task from the plurality of tasks by adding one or more new neural networks to a current set of neural networks of the neural network system, including: For each candidate neural network in a plurality of candidate neural networks, training the candidate neural network to perform the activity task comprises: selecting a parent neural network from the current set of neural networks of the neural network system; identifying a first strict subset of said network blocks of said parent neural network, identifying a second strict subset of the network blocks of the parent neural network to be shared between the parent neural network and the candidate neural network in the neural network system; Initializing the candidate neural network includes, for each network block in the first strict subset, generating a copy of the network block for inclusion in the candidate neural network; generating trained values ​​for a plurality of network parameters of the candidate neural network using training data corresponding to the active task, including updating a corresponding plurality of parameters of each of the copied network blocks; and determining a performance score for the trained candidate neural network, the performance score characterizing at least a performance of the trained candidate neural network on the active task; and Using the respective performance scores of the candidate neural networks, one or more candidate neural networks are selected to be added to the current set of neural networks of the neural network system.

2. The method of claim 1 , wherein prior to a first stage in the plurality of stages, the current set of neural networks of the neural network system is initialized to include a single root neural network that is randomly initialized or pre-trained.

3. The method of any one of claims 1 or 2, wherein selecting a parent neural network from the current set of neural networks of the neural network system comprises: determining whether any neural network in the current set of neural networks has been trained to perform the active task; In response to determining that one or more particular neural networks in the current set of neural networks have been trained to perform the active task, performing a first selection process using the one or more particular neural networks; as well as In response to determining that no neural network in the current set of neural networks has been trained to perform the active task, a second selection process is performed using at least a subset of the neural networks in the current set of neural networks.

4. The method of any one of claims 1 or 2, wherein selecting a parent neural network from the current set of neural networks of the neural network system comprises: determining whether any neural network (i) is in the current set and has been trained to perform the active task or (ii) has been previously trained during the phase to perform the active task; In response to determining that one or more particular neural networks (i) are in the current set and have been trained to perform the active task or (ii) have been previously trained during the phase to perform the active task, performing a first selection process using the one or more particular neural networks; In response to determining that no neural network (i) is in the current set and has been trained to perform the active task or (ii) has been previously trained during the phase to perform the active task, performing a second selection process using at least a subset of the neural networks in the current set of neural networks.

5. The method of any one of claims 3 or 4, wherein performing the second selection process comprises: determining one or more seed neural networks from the current set of neural networks; as well as The first selection process is performed using the one or more seed neural networks.

6. The method of claim 5, wherein the one or more seed neural networks include one or more of the following: a root neural network included in the current set of neural networks prior to a first stage in the plurality of stages; or For each task for which the neural network system has been trained in a corresponding previous stage, the neural network having the highest performance score among all neural networks trained to perform the task.

7. The method of any one of claims 3 to 6, wherein performing the first selection process comprises: determining a ranking of the one or more specific neural networks based on the respective performance scores determined during training of the one or more specific neural networks; as well as Starting at an initial specific neural network in the described order, evaluating one or more corresponding specific neural networks includes: determining whether to select the specific neural network based on the random sample; In response to determining that the particular neural network is selected, ending the evaluating; and In response to determining that the particular neural network is not selected, continuing to evaluate the next particular neural network in the order.

8. The method of claim 7, wherein a particular neural network m is selected with a probability equal to or proportional to: where #selections(m,t) identifies the number of times a particular neural network m has been selected as the parent neural network when training a candidate neural network to perform the active task t.

9. The method of any one of claims 7 or 8, wherein performing the first selection process further comprises, in response to determining not to select the last particular neural network in the order: determining a second ranking of the neural networks in the current set that have been trained to perform a task different from the active task; and Starting at an initial neural network in said second order, evaluating one or more corresponding neural networks comprises: Determining whether to select the neural network based on a random sample; In response to determining that the neural network is selected, ending the evaluating; and In response to determining that the neural network is not selected, continuing to evaluate the next neural network in the second order.

10. The method of claim 9, wherein performing the first selection process further comprises, in response to determining not to select the last neural network in the second order: A neural network is selected uniformly randomly from the current set.

11. The method of any one of claims 1 to 10, wherein identifying the first strict subset of the network blocks of the parent neural network comprises: For each network block in one or more network blocks b of the parent neural network, with probability The network block is selected for inclusion in the first strict subset.

12. The method of claim 11, wherein the corresponding probability of each network block b is are the same.

13. The method of any one of claims 1 to 12, wherein each network block of the parent neural network that is not in the first strict subset is in the second strict subset.

14. The method of any one of claims 1 to 13, wherein: Training the candidate neural network to perform the activity task further comprises: identifying a third strict subset of the network blocks of the parent neural network that will not be included in the candidate neural network; and Initializing the candidate neural network further comprises, for each specific network block in the third strict subset: Adding connections between (i) a network block of the candidate neural network corresponding to a network block of the parent neural network that precedes the specific network block in the parent neural network and (ii) a network block of the candidate neural network corresponding to a network block of the parent neural network that follows the specific network block in the parent neural network.

15. The method of claim 14, wherein each network block of the parent neural network that is not in the first strict subset or the third strict subset is in the second strict subset.

16. The method of any one of claims 14 or 15, wherein identifying the third strict subset of the network blocks of the parent neural network comprises: For each network block in one or more network blocks b of the parent neural network, with probability The network block is selected to be included in the third strict subset.

17. The method of claim 16, wherein the corresponding probability of each network block b is are the same.

18. The method of any one of claims 1 to 17, wherein initializing the candidate neural network further comprises: One or more new network blocks that are not in the parent neural network are added to the candidate neural network.

19. The method of claim 18, wherein adding one or more new network blocks to the candidate neural network comprises: For each pair p of one or more pairs of consecutive network blocks of the parent neural network, with probability Determining to add a new network block between corresponding pairs of network blocks in the candidate neural network.

20. The method of claim 19, wherein the corresponding probability of each pair p of network blocks is are the same.

21. The method of any one of claims 1 to 20, wherein: The method also includes maintaining, at each of a plurality of stages and for each neural network in the current set, data identifying respective values ​​of a set of one or more hyperparameters used during the training of the neural network, Training the candidate neural network further includes: identifying a subset of the set of hyperparameters; For each hyperparameter not in the identified subset, determining to train the candidate neural network using the same value for the hyperparameter as used to train the parent neural network; and For each hyperparameter in the identified subset: New values ​​are selected for the hyperparameters to train the candidate neural network.

22. The method of claim 21 , wherein identifying the subset of the set of hyperparameters comprises: For each of the one or more hyperparameters h, with probability The hyperparameters are selected for inclusion in the subset.

23. The method of claim 22, wherein the corresponding probability of each hyperparameter h is are the same.

24. The method of any one of claims 21 to 23, wherein for one or more of the hyperparameters in the identified subset, selecting new values ​​for the hyperparameters to train the candidate neural network comprises: identifying an ordering of a finite set of possible values ​​for the hyperparameter; identifying, in said order, specific values ​​used during training of said parent neural network; as well as Selecting (i) a value that immediately precedes the particular value used during training of the parent neural network in the ordering, or (ii) a value that immediately follows the particular value used during training of the parent neural network in the ordering.

25. The method of any one of claims 1 to 24, wherein the performance score of each trained candidate neural network is equal to or proportional to the validation accuracy and / or test accuracy of the candidate neural network.

26. The method of any one of claims 1 to 25, wherein the performance score of each trained candidate neural network further characterizes one or more of: a latency of the trained candidate neural network, a computational cost of the trained candidate neural network, a size of the candidate neural network.

27. The method of claim 26, wherein the performance score of each trained candidate neural network m comprises a term inversely proportional to: wherein P(m) identifies the plurality of network parameters of the trained candidate neural network, and #models(p) identifies, for network parameter p, the number of neural networks from the current set of neural networks that include the network parameter p.

28. The method of claim 27, wherein the performance score of each trained candidate neural network m is equal to or proportional to: Where q(m) identifies the validation or test accuracy of the trained candidate neural network, s is a hyperparameter of the neural network system and #root-model-params identifies the number of network parameters in the root neural network included in the current set of neural networks prior to the first stage in the plurality of stages.

29. The method of any one of claims 1 to 28, wherein at one or more of the plurality of stages: One or more neural networks in the current set of neural networks are safe neural networks that are not included in a pool of neural networks determined from the current set of parent neural networks from which corresponding candidate neural networks are selected.

30. The method of claim 29, wherein for each secure neural network: For each network block of the secure neural network whose parameters are updated during training of the secure neural network, the network block is not shared with any other neural network in the neural network system.

31. The method of any one of claims 29 or 30, wherein one or more of the secure neural networks have been trained on a private data set that is not available during training of any other candidate neural network.

32. The method of any one of claims 1 to 31 , wherein selecting one or more candidate neural networks comprises selecting only the candidate neural networks having the highest performance scores.

33. The method of any one of claims 1 to 32, further comprising at one or more of the plurality of stages: In response to selecting a particular candidate neural network to be added to the current set of neural networks: identifying one or more specific neural networks in the current set that (i) have been trained to perform the active task and (ii) have a performance score lower than the specific candidate neural network; and Responsive to the determination, the one or more particular neural networks are removed from the neural network system.

34. The method of any one of claims 1 to 33, wherein at one or more of the plurality of stages, at least some of the plurality of candidate neural networks are trained in parallel.

35. The method of any one of claims 1 to 34, wherein at one or more of the plurality of stages, training the neural network system to perform the corresponding activity task further comprises: For each of multiple generations in a sequence of generations: training a plurality of candidate neural networks to perform the activity task; For each candidate neural network: determining whether the performance score of the candidate neural network is greater than a performance score generated for the corresponding parent neural network during training of the neural network; In response to determining that the performance score of the candidate neural network is greater than the performance score generated for the corresponding parent neural network, adding the candidate neural network to a pool of possible parent neural networks for a subsequent generation in the sequence of generations; and In response to determining that the performance score of the candidate neural network is not greater than the performance score generated for the corresponding parent neural network, discarding the candidate neural network; as well as The one or more candidate neural networks are selected from the candidate neural networks generated at corresponding generations that are not discarded to be added to the current set.

36. A method comprising: Receive network input; as well as The network input is processed using a neural network system trained using a method as claimed in any preceding claim to generate a network output.

37. A system comprising one or more computers and one or more storage devices storing instructions which, when executed by the one or more computers, cause the one or more computers to perform the operations of the respective methods of any one of claims 1 to 36.

38. One or more computer storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform the operations of the corresponding method of any one of claims 1 to 36.

Citation Information

Patent Citations

  • Neural networks for scalable continual learning in domains with sequentially learned tasks

    CN111989696A

  • Method, device, equipment, medium and program for generating multi-task model

    CN112527383A

  • A twin network model training method, a twin network model measuring method, a twin network model training device, a twin network model measuring device, a medium and equipment

    CN112784130A

  • Multi-task neural networks with task-specific paths

    US20190354868A1