Training a neural network by including the implementation cost as an objective

Abstract

An example implementation method of a neural network includes: selecting a first neural network architecture from a search space; and training a neural network having the first neural network architecture to obtain an accuracy and an implementation cost. The implementation cost is based on a programmable device of an inference platform. The method also includes: selecting a second neural network architecture from the search space based on the accuracy and the implementation cost; and outputting weights and hyperparameters of the neural network having the second neural network architecture.

Description

Technical Field

[0001] Examples of the present disclosure relate generally to neural networks, and more particularly to training a neural network by including an implementation cost as a goal. Background Art

[0002] Machine learning is the science of inducing computing systems to act without being explicitly programmed. Traditional machine learning includes various clustering and classification techniques, including K-means clustering, linear and logistic regression, stochastic gradient descent, association rule learning, etc. Deep learning is a newer frontier in machine learning. Deep learning is a class of machine learning algorithms that uses multi-layer nonlinear processing units for feature extraction and transformation. Deep learning algorithms can be unsupervised (e.g., pattern analysis) or supervised (e.g., classification). Deep learning algorithms can be implemented using layers of artificial neural networks (ANNs) (referred to herein as "neural networks").

[0003] In general, a neural network is a collection of nodes (i.e., "neurons") connected in a graph. A node in a neural network computes the sum of weighted inputs and adds an optional bias to the sum. The output of a node is a function of the final sum (called an "activation function"). Exemplary activation functions include the sigmoid function, the hyperbolic tangent (tanh) function, the rectified linear unit (ReLU) function, and the identity function. Neural network models are typically organized into layers of nodes that define a specific topology, and corresponding weights and biases. The weights and biases are called network parameters.

[0004] In general, a neural network includes an input layer and an output layer, and may optionally include one or more hidden layers between the input layer and the output layer. Neural networks used in deep learning applications typically include many hidden layers, which gave rise to the term deep neural network (DNN). The layers of a neural network can be densely connected (e.g., each node in a layer is fully connected to all nodes in the previous layer) or sparsely connected (e.g., each node in a layer is only connected to a subset of nodes in the previous layer). A convolutional neural network (CNN) is a type of DNN that includes one or more sparsely connected layers, called convolutional layers. CNNs are well suited for processing image or video data. Other types of DNNs include recurrent neural networks (RNNs), which are well suited for processing speech and text data.

[0005] Neural networks of any topology or type require correct values ​​for network parameters across all layers in order to adapt the network to a specific task. A supervised training process can be used to determine a set of network parameters that can produce the desired accuracy for a specified task. Training involves running a training data set through the forward path of the network (forward propagation) and updating weights through the backward path of the network (backward propagation) to compensate for prediction errors. The trained neural network is then deployed to perform a specified task (called inference) on the input data set. The performance of the computing platform used to train the neural network (training platform) is typically higher than that of the computing platform used for inference (inference platform). However, inference platforms are typically more power-efficient than training platforms. Conventional training techniques do not take into account the architectural aspects of the inference platform, which may result in a less than ideal implementation of the neural network for the target inference platform. Summary of the invention

[0006] Techniques for training a neural network by including an implementation cost as an objective are described. In one example, a method of implementing a neural network includes: selecting a first neural network architecture from a search space; training a neural network having the first neural network architecture to obtain an accuracy and an implementation cost, the implementation cost being based on a programmable device of an inference platform; selecting a second neural network architecture from the search space based on the accuracy and the implementation cost; and outputting weights and hyperparameters of the neural network having the second neural network architecture.

[0007] In another example, a non-transitory computer-readable medium includes instructions that, when executed in a computer system, cause the computer system to perform a method of implementing a neural network, the method comprising: selecting a first neural network architecture from a search space; training the neural network having the first neural network architecture to obtain an accuracy and an implementation cost, the implementation cost being based on a programmable device of an inference platform; selecting a second neural network architecture from the search space based on the accuracy and the implementation cost; and outputting weights and hyperparameters of the neural network having the second neural network architecture.

[0008] In another example, a computer system includes a memory having program code stored therein; and a processor configured to execute the program code to implement a neural network by: selecting a first neural network architecture from a search space; training the neural network having the first neural network architecture to obtain an accuracy and an implementation cost, the implementation cost being based on a programmable device of an inference platform; selecting a second neural network architecture from the search space based on the accuracy and the implementation cost; and outputting weights and hyperparameters of the neural network having the second neural network architecture.

[0009]

[0011] These and other aspects may be understood with reference to the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to be able to understand in detail the manner in which the features set forth above are described, a more particular description briefly outlined above may be obtained by reference to example implementations, some of which are illustrated in the accompanying drawings. It should be noted, however, that the drawings illustrate only typical example implementations and are therefore not to be considered limiting of their scope.

[0011] Figure 1 is a block diagram depicting a system for training and implementing a neural network according to one example.

[0012] Figure 2 is a block diagram depicting a computing system according to one example.

[0013] Figure 3 A neural network training method according to an example.

[0014] Figure 4 is a method for training a neural network according to another example.

[0015] Figure 5 is a method for training a neural network according to another example.

[0016] Figure 6 is a flowchart depicting a method for implementing an inference platform according to an example.

[0017] Figure 7 is a block diagram depicting a programmable integrated circuit (IC) according to one example.

[0018] Figure 8 It depicts Figure 7 Block diagram of a system-on-chip (SoC) implementation of a programmable IC.

[0019] Fig. 9 Picture shows Figure 7 The programmable IC is implemented using a field programmable gate array (FPGA).

[0020] To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. It is contemplated that elements of one example may be beneficially incorporated in other examples. DETAILED DESCRIPTION

[0021] Various features are described below with reference to the accompanying drawings. It should be noted that the drawings may be drawn to scale or not to scale, and that elements of similar structure or function are represented by similar reference numerals throughout the drawings. It should be noted that the drawings are intended only to facilitate the description of the features. They are not intended to be an exhaustive description of the claimed invention or a limitation on the scope of the claimed invention. In addition, the illustrated examples do not necessarily have all the aspects or advantages shown. The aspects or advantages described in conjunction with a particular example are not necessarily limited to that example, and may be practiced in any other example even if not shown or explicitly described as such.

[0022] Techniques for training a neural network by including implementation cost as an objective are described. The techniques provide a cost-aware architecture search for the topology of the neural network. In this way, the training of the neural network no longer has as its sole objective the maximum accuracy of the neural network for a particular task. Instead, the neural network training balances the accuracy of the neural network with the implementation cost, which is another objective in the training. In this way, the training becomes a multi-objective search, where not only the values ​​of the weights are trained, but also the topology and certain implementation-related properties of the neural network are found.

[0023] The techniques described herein address the high computational / memory requirements of neural networks and their actual implementation in the hardware backend during the training phase. These techniques include: deriving / alternating network topologies, their hyperparameters, and certain implementation-related properties by making the (inference) implementation cost of the neural network an additional target during training (next to the initial, usually accuracy-related target) and other features such as fault tolerance (e.g., in the case of safety-critical applications). Conventional training does not explain the architectural aspects of the inference platform. Complexity optimization techniques focus on reducing memory bandwidth by pruning / compressing weights and / or feature maps, and reducing the accuracy (bit width) of weights and / or feature maps. Reinforcement learning provides multi-objective optimization, but there is no need to increase the implementation cost of the neural network itself as a target. The techniques described herein for training using implementation cost as a target are complementary to those techniques. The following is about the accompanying drawings to optimize these and other aspects of network parameters and / or feature maps based on the architectural constraints of the inference platform.

[0024] Figure 11 is a block diagram depicting a system 100 for training and implementing a neural network according to an example. The system 100 includes a training platform 102 and an inference platform 104. The training platform 102 includes hardware and software configured to train a neural network 106 for a specified task (e.g., image classification, object detection, etc.). As described below, the training platform includes a reinforcement agent 103 and an adjustment agent 105. The inference platform 104 includes hardware and / or software configured to implement the neural network 106 to perform the specified task. Examples of the training platform 102 and the inference platform 104 are described below.

[0025] The implementation efficiency of a neural network implementation can be measured by different costs such as throughput, energy, size, fault tolerance, etc., or a combination thereof. This cost is the result of different design aspects such as the number of operations, bandwidth, data locality, scheduling of the hardware backend, etc. These aspects are related to the characteristics of the training algorithm, where better algorithm performance usually leads to higher implementation costs (Pareto principle). Generally, maximizing the accuracy of the algorithm for a specific task / capability is the main goal during training. Additionally, the network topology is usually designed, and the training focuses on finding the correct values ​​of all weights in the different layers of the neural network. These weights are then used during inference to perform the task / function. The configuration of the training algorithm is controlled by the "algorithm behavior" hyperparameters. Additionally, the term hyperparameter is also used to define the parameters of the capacity of the neural network (e.g., the number of hidden layers in the neural network), and is therefore related to the network topology. These hyperparameters are referred to as "model capacity" hyperparameters in this article, and include all implementation properties (e.g., bit width).

[0026] The training platform 102 receives a training data set 110 and initial network weights 113. The training data set 110 includes data for training the neural network 106 to generate trained network weights 114. For example, if the neural network 106 is configured to classify images, the training data set 110 may be a pre-classified image set. The initial network weights 113 include initial values ​​for the weights of the neural network 106. In one example, the training platform 102 also includes inputs for receiving algorithm behavior hyperparameters 112. The algorithm behavior hyperparameters 112 include learning rates, early stopping criteria, etc. The training platform 102 also includes inputs for receiving inferred implementation costs 115. The training platform 102 uses the inferred implementation costs 115 as a training objective to learn the optimal weights 114, network topology 120, model capacity hyperparameters 108, and implementation properties 122 (e.g., weight or tensor element bit width, number format, etc.) that achieve the best tradeoff in the accuracy, implementation cost Pareto space.

[0027] When exploring this Pareto space, a minimum accuracy can be enforced. In this case, the training seeks the lowest cost implementation that achieves at least the expected accuracy. The combined accuracy and inference specific implementation cost training objectives are applicable to any computing platform (e.g., CPU, GPU, ASSP, FPGA, ACAP, etc., or any combination thereof). Inference specific implementation costs include throughput, energy, size, fault tolerance, etc., or a combination thereof. This inference specific implementation cost is also more generally referred to as implementation cost in this article. Since all architectural design parameters / aspects (e.g., bit width, number of processing elements, etc.) are not fixed, they can be learned during training, so the flexible architecture of FPGA is very suitable for achieving the combined accuracy and implementation cost training objectives.

[0028] The topology 120 typically includes an arrangement of neurons. For example, the topology 120 may include multiple layers of neurons. These layers typically include an input layer, an output layer, and zero or more hidden layers. Each neuron includes multiple inputs and one output. The multiple inputs of each neuron are associated with multiple weights. Each neuron also includes a bias associated with its output. The weights and biases of the neural network 106 are referred to as trained network weights 114. For a given layer, the inputs of its neurons are referred to as input feature maps, and the outputs of its neurons are referred to as output feature maps. The input feature maps and output feature maps are typically referred to as "feature maps."

[0029] Inference platform 104 implements neural network 106. Input data set 116 includes data to be processed by neural network 106. For example, if the neural network is configured to classify images, input data set 116 may include images to be classified. Inference platform 104 generates result data set 118. For example, in an image classification scenario, result data set 118 includes classifications of images in input data set 116. Because neural network 106 has been optimized based on the implementation cost of inference platform 104, neural network 106 can be efficiently implemented by inference platform 104, thereby taking advantage of its features, elements, and limitations captured by inference implementation cost 115.

[0030] Figure 2 2 is a block diagram depicting a computing system ("computer 200") according to one example. Computer 200 includes a software platform 204 executing on a hardware platform 202. Hardware platform 202 includes a central processing unit (CPU) 206, system memory 208, storage device 210, support circuits 211, training platform 212, and hardware accelerator 214. Software platform 204 includes an operating system (OS) 230, drivers 232, libraries 234, and applications 236.

[0031] In one example, CPU 206 may be any type of general purpose central processing unit (CPU), such as an x86 based processor, a processor, etc. The CPU 206 may include one or more cores and associated circuit systems (e.g., cache memory, memory management unit (MMU), interrupt controller, etc.). The CPU 206 is configured to execute program code that performs one or more operations described herein and may be stored in the system memory 208 and / or the storage device 210. The support circuit 211 includes various devices that cooperate with the CPU 206 to manage data flows between the CPU 206, the system memory 208, the storage device 210, the training platform 212, the hardware accelerator 214, or any other peripheral device. For example, the support circuit 211 may include a chipset (e.g., north bridge, south bridge, platform host controller, etc.), a voltage regulator, firmware (e.g., BIOS), etc. In some examples, the CPU 206 may be a system-in-package (SiP), a system on chip (SoC), etc., which absorbs all or most of the functionality of the chipset (e.g., north bridge, south bridge, etc.). In another example, the CPU 206 may be a vector processor or may include a vector processor.

[0032] System memory 208 is a device that allows information such as executable instructions and data to be stored and retrieved. System memory 208 may include, for example, one or more random access memory (RAM) modules, such as double data rate (DDR) dynamic RAM (DRAM). System memory 208 may store data 226 and program code ("code 228") that is processed and executed by CPU 206 to implement software platform 204. Storage device 210 includes local storage devices (e.g., one or more hard disks, flash memory modules, solid-state disks, and optical disks) and / or storage interfaces that enable computer 200 to communicate with one or more network data storage systems. Hardware platform 202 may include various other conventional devices and peripherals of a computing system, such as a graphics card, a universal serial bus (USB) interface, etc.

[0033] The training platform 212 includes hardware 216, which may include processor(s), memory, input / output (IO) circuits, etc. In one example, the hardware 216 includes a graphics processing unit (GPU) and associated support circuitry. In another example, the hardware 216 may include an application specific integrated circuit (ASIC), a programmable IC, etc. and associated support systems. In one example, the performance of the training platform 212 is higher than the performance of the hardware accelerator 214, but consumes more energy than the hardware accelerator 214. The training platform 212 can be used to train a neural network.

[0034] Hardware accelerator 214 includes IC 220 and memory 224. IC 220 includes computing engine 222. In one example, IC 220 is a programmable IC, such as a field programmable gate array (FGPA) or a system on chip (SoC) with an FPGA. Computing engine 222 can be programmed in IC 220. In another example, IC 220 is an ASIC, etc., in which computing engine 222 is a dedicated circuit system. Hardware accelerator 214 can be used for an inference platform of a neural network.

[0035] OS 230 may be any commercial operating system known in the art, such as Microsoft Driver 232 and library 234 include software that provides an application programming interface (API) to the training platform 212 and the hardware accelerator 214 to command and control them. Application 236 includes software that trains a neural network on the training platform 212 and implements the neural network on the hardware accelerator 214. Application 236 communicates with the training platform 212 and the hardware accelerator 214 through driver 232 and library 234.

[0036] Including implementation cost as an objective in training makes training a multi-objective problem. Multi-objective optimization techniques are described below to combine network accuracy and implementation cost. Three examples of training approaches for this implementation and accuracy driven neural network search are described: (1) using reinforcement learning; (2) using evolution-based algorithms; and (3) using hyperparameter analysis / optimization. Techniques for reducing the size of the neural network architecture search space are also described.

[0037] Multi-objective optimization

[0038] When evaluating the performance of a network, including the inference of the implementation cost means that there are at least two objectives to optimize. Thus, multiple objectives should be balanced in a meaningful way. For example, suppose the accuracy of the network is given by the classification error C E is given by , and the estimated implementation cost is given by processing the new input C T The time taken is given. If C T If the minimum importance is too large, the optimizer may produce a network with zero layers, zero operations, and zero memory requirements. E , but this still yields C T = 0. The multi-objective optimization goal is to balance C E and C T , in order to provide an ideal solution.

[0039] The general statement of multi-objective optimization is as follows:

[0040]

[0041] where f1,…,f k is a function that defines the cost of each objective being optimized, x is a vector representing the current solution, and X is the search space of all possible solutions. In the examples described herein, x represents the neural network topology and its associated hyperparameters (i.e., the model capacity hyperparameter 108). Functions f1,…,f k Represents the metric of interest for the current neural network topology relative to its accuracy and implementation cost / hardware cost. For accuracy, these functions include mean squared error (MSE), classification error, L p norm, hinge loss, or similar metrics appropriate for the target domain. For implementation cost / hardware cost, these functions include memory requirements, bandwidth requirements, clock cycles, data path width, quantization scheme, arithmetic style, number format, silicon area and power consumption, and fault tolerance.

[0042] In some cases, it is difficult to combine objective functions mathematically in an understandable way. In these cases, when comparing two solutions x1 and x2, if Then x1 is a better solution than x2. If no better solution than x1 can be found, then x1 is considered a Pareto optimal solution. In other cases, multiple objective functions can be combined to form a single objective function that is designed to encapsulate the trade-offs between multiple objectives. This is called scalarization and is expressed in the general case as follows:

[0043]

[0044] where g∈R k →R. Common examples of g include:

[0045] Linear scalarization, where g = ∑w i f i (x), where w i >0 are the weights associated with each objective function; and

[0046] L p norm, g = ‖fz‖ p , where f={f1(x),f2(x),...,f k (x)}, and z∈R k is a vector of ideal cost values.

[0047] Based on the chosen optimizer (e.g., described below), the objective function may need to be semi-differentiable, such as MSE, cross entropy, and hinge loss. Below are three learning techniques for cost-aware architecture search. Note that each of these techniques can be used in conjunction with each other.

[0048] The example shown shows the implementation cost C as an additional optimization cost (next to the accuracy R). This is a general representation of the inference of a specific implementation cost. It can represent a single implementation cost, such as energy E or fault tolerance T, etc., or any combination of costs.

[0049] Architecture Search with Reinforcement Learning

[0050] Figure 3 A training method 300 of a neural network according to an example. The method 300 begins at step 302, where the reinforcement agent 103 selects a sample neural network architecture description A from a search space S with probability P. The topology of the neural network (e.g., its structure and connectivity) can be described in a text format (e.g., prototxt or any other representation used by a neural network or machine learning framework). The neural network description is extended with implementation specific properties (e.g., bit width, number format, scheduling, etc. of tensor elements). The extended neural network description becomes the neural network architecture description.

[0051] At step 304, the training platform trains the neural network, thereby producing an accuracy R on the validation set. Since the neural network architecture description includes implementation properties, the implementation cost C can be measured or estimated / modeled (based on the inference platform) (step 306). At step 308, the training platform uses the combination of the accuracy R and the implementation cost C as a reward to calculate the policy gradient to update the reinforcement agent 103. At step 310, the reinforcement agent 103 determines whether the end condition has been met for training. If not, the method 300 repeats and another network architecture description is selected from the search space S. It should be understood that the method 300 can select the same network architecture as the previous iteration when selecting the next network architecture for processing. That is, the same network architecture can be used in multiple training iterations. Otherwise, the method 300 proceeds to step 312, where the training platform outputs the trained neural network.

[0052] In one example, the reinforcement agent 103 can be a machine learning algorithm tuned for sequence prediction, such as a recurrent neural network (RNN). The RNN takes as input the parameters of the previous network layer and produces predictions for the parameters of the subsequent layer. The RNN continues in this manner until a stopping criterion is reached. Example stopping criteria include: reaching a certain number of layers, or reaching a certain hardware cost (e.g., memory usage / number of operations). If a semi-differential objective function is chosen for network accuracy and implementation cost, some parameters can be updated by differentiating them with respect to the objective function. For other parameters, a strategy is defined for gradients.

[0053] Evolution-based Architecture Search

[0054] Figure 4400 can be implemented by a training platform. An alternative approach to architecture search is to use an evolution-based algorithm. In order to use an evolutionary algorithm to perform architecture search, two things need to be done: 1) encode the neural network architecture as a gene; and 2) a fitness function for evaluating the performance of a particular structure. The fitness function can be any function described above in the multi-objective optimization section, including a scalar function or a multi-objective function. The evolutionary algorithm understands the implementation cost of such a network. In this case, the evolutionary algorithm can be used to find the optimal solution (scalarized) or a series of pareto optimal solutions or approximate approximations. In order to encode the neural network architecture as a gene, the neural network description can be transformed into letters. This can be equivalently mapped to a network design protocol (such as caffe's prototxt) written in a compact way to make the algorithm more conducive to the evolutionary algorithm. Neural network layers, graph connections, and individual neurons and synapses can all be expressed as genes.

[0055] The basic approach of the evolutionary algorithm is to generate N random gene strings (which correspond to neural network architectures) (step 402). These architectures are then evaluated using a fitness function, which may require training each network architecture separately (step 404). At this point, a subset of the architectures is selected, randomly combined and mutated to generate the next N architectures (step 406). Over time, this will produce an architecture that is highly optimized for a given cost function, which in this case means high accuracy and low implementation / hardware cost. At step 408, it is determined whether to end. If not, the method 400 proceeds to step 404 and repeats. Otherwise, the method 400 proceeds to step 410, where the training platform outputs a trained neural network.

[0056] Training with Hyperparameter Analysis

[0057] Figure 5 5 is a method 500 for training a neural network according to an example. The method 500 begins at step 502, where the tuning agent 105 selects a set of hyperparameters. As noted above, the model capacity hyperparameter allows for the definition / description of the architecture of the neural network. The model capacity hyperparameter defines both topological parameters (e.g., number of layers, number of channels per layer, etc.) and related implementation properties. The tuning agent 105 gathers knowledge about the relationship between the hyperparameters (both algorithmic behavior and model capacity).

[0058] At step 504, the training platform trains the neural network to produce an accuracy R on the validation set. Since the neural network architecture description includes implementation properties, the implementation cost C (based on the inference platform) can be measured or estimated / modeled (step 506). At step 508, the tuning agent 105 uses the relationship between hyperparameters and neural network performance (both accuracy R and implementation cost C) to make more pareto optimal choices for the next set of hyperparameters. By applying hyperparameter optimization techniques, good optimization can be achieved in a limited number of optimization steps.

[0059] Examples of hyperparameter optimization techniques include grid search, random search, and Bayesian optimization. Grid search involves selecting a set of candidate values ​​for each hyperparameter within the neural network. The grid search is then performed by training the network for each permutation of the hyperparameters. The best model is then selected as the model that performs ideally with respect to our cost function, as described above in the multi-objective optimization section.

[0060] Random search is conceptually similar to grid search, except that instead of selecting random values ​​from a grid, it selects random values ​​from a specified range for each hyperparameter. This has several benefits, including: a larger variation of the hyperparameters tested, a higher chance of performing better than grid search for each hyperparameter, and the experiment can be interrupted at any point and still be considered a complete search of the set of data points.

[0061] Bayesian hyperparameter search is a more complex technique that attempts to develop a statistical model that maps hyperparameter values ​​to our cost function. Typically, this statistical model is a Gaussian process (GP), which generates a function that is very close to the observed data. GP provides a prediction of the cost function chosen in the hyperparameter space and the uncertainty of such a prediction, which has the following benefits compared to random search and grid search: 1) on the next iteration, select the point that minimizes the GP, that is, the point that is most likely to be optimal based on the current model in the hyperparameter space relative to our desired results; and 2) on the next iteration, select a point with high uncertainty, that is, a point that will reveal a lot of other information about the hyperparameter space.

[0062] Reduce the architecture search space

[0063] In the methods described above, the size / complexity of the neural architecture search space can be reduced by making only certain aspects of the network variable. For example, making only the bit width of the feature map elements and the number of channels of the feature map variable enables training to their optimal settings. In general, reducing the bit width of the feature map elements will result in reduced accuracy while allowing more efficient implementation. The reduction in accuracy can be compensated by increasing the number of feature map channels, but at the cost of increased implementation complexity. The bit width and number of channels of the feature map elements can be expressed as part of the neural network architecture description (for reinforcement learning techniques) and can also be expressed as a model capacity hyperparameter (for hyperparameter analysis). Both architecture search techniques will explore the (reduced) search space to find the pareto-optimal (accuracy and implementation cost) neural network architecture.

[0064] Note that implementations typically appear as discrete points in an optimization search space where they try to make the best use of the resources of a particular chip / platform. This not only reduces the size of the search space, but also touches on another optimization goal of implementation cost-aware network search: maximizing the accuracy of this discrete implementation point. This suggests that a list of all device resources (for members of the chip family under consideration) can also be an input to implementation cost-aware architecture search.

[0065] Note that the implementation resources that do reside on the FPGA fabric, such as LUTs, FFs, DSPs, BRAM / URAM, etc., usually reach a certain percentage of the devices within a particular family. These percentages can reduce the number of variables in multi-objective optimization.

[0066] Finally, note that many current neural network topologies do not rely on data-dependent layer execution. The 'static' execution of all layers in a neural network simplifies modeling the implementation cost of a neural network. If data-dependent layer execution exists in the network, neural network architecture search requires more complex dynamic implementation costs. Alternatively, implementation cost measurements taken while running topology candidates on an (inference) platform can be used for neural network architecture search.

[0067] Programmable device implementation

[0068] Figure 6 6 is a flow chart depicting a method 600 of implementing an inference platform according to one example. At step 602, a training platform trains a neural network that describes the implementation cost described above in the technology. The training platform outputs a trained neural network description. At step 604, a user interacts with a circuit design tool to generate a circuit design based on the description of the trained neural network. At step 606, the circuit design tool implements the circuit design of a programmable device such as an FGPA or a SoC with programmable logic. At step 608, the circuit design tool loads a bitstream into the programmable device to implement the inference platform.

[0069] Figure 7 is a block diagram depicting a programmable IC 1 according to one example, which can be used to implement an inference platform and / or a training platform. The programmable IC 1 can be used as Figure 2 220 in the programmable IC 1. The programmable IC 1 includes a programmable logic 3, a configuration logic 25, and a configuration memory 26. The programmable IC 1 can be coupled to external circuits, such as a non-volatile memory 27, a DRAM 28, and other circuits 29. The programmable logic 3 includes a logic cell 30, a support circuit 31, and a programmable interconnect 32. The logic cell 30 includes a circuit that can be configured to implement a general logic function of multiple inputs. The support circuit 31 includes a dedicated circuit, such as a transceiver, an input / output block, a digital signal processor, a memory, etc. The logic cell and the support circuit 31 can be interconnected using a programmable interconnect 32. Information for programming the logic cell 30, for setting parameters of the support circuit 31, and for programming the programmable interconnect 32 is stored in the configuration memory 26 by the configuration logic 25. The configuration logic 25 can obtain configuration data from the non-volatile memory 27 or any other source (e.g., DRAM 28 or from other circuits 29). In some examples, the programmable IC 1 includes a processing system 2. The processing system 2 may include microprocessor(s), memory, support circuits, IO circuits, and the like.

[0070] Figure 8 1 is a block diagram depicting a system on chip (SoC) implementation of a programmable IC 1 according to one example. In this example, the programmable IC 1 includes a processing system 2 and programmable logic 3. The processing system 2 includes various processing units, such as a real-time processing unit (RPU) 4, an application processing unit (APU) 5, a graphics processing unit (GPU) 6, a configuration and security unit (CSU) 12, a platform management unit (PMU) 122, etc. The processing system 2 also includes various support circuits, such as an on-chip memory (OCM) 14, a transceiver 7, peripherals 8, an interconnect 16, a DMA circuit 9, a memory controller 10, peripherals 15, and a multiplexed IO (MIO) circuit 13. The processing units and support circuits are interconnected by interconnect 16. PL 3 is also coupled to interconnect 16. Transceiver 7 is coupled to external pin 24. PL 3 is coupled to external pin 23. Memory controller 10 is coupled to external pin 23. MIO 13 is coupled to external pin 20. PS 2 is generally coupled to external pin 21. The APU 5 may include a CPU 17 , a memory 18 , and support circuits 19 .

[0071] Referring to PS 2, each of the processing units includes one or more central processing units (CPUs) and associated circuits, such as memory, interrupt controllers, direct memory access (DMA) controllers, memory management units (MMUs), floating point units (FPUs), etc. Interconnect 16 includes various switches, buses, communication links, etc., which are configured to interconnect the processing units and to interconnect other components in PS 2 to the processing units.

[0072] OCM 14 includes one or more RAM modules, which can be distributed throughout PS 2. For example, OCM 14 may include battery-powered RAM (BBRAM), tightly coupled memory (TCM), etc. Memory controller 10 may include a DRAM interface for accessing external DRAM. Peripheral devices 8, 15 may include one or more components that provide an interface to PS2. For example, peripheral device 132 may include a graphics processing unit (GPU), a display interface (e.g., DisplayPort, High Definition Multimedia Interface (HDMI) port, etc.), a universal serial bus (USB) port, an Ethernet port, a universal asynchronous receiver and transmitter (UART) port, a serial peripheral interface (SPI) port, a general purpose IO (GPIO) port, a serial advanced technology attachment (SATA) port, a PCIe port, etc. Peripheral device 15 may be coupled to MIO 13. Peripheral device 8 may be coupled to transceiver 7. Transceiver 7 may include a serializer / deserializer (SERDES) circuit, an MGT, etc.

[0073] Fig. 9 A field programmable gate array (FPGA) implementation of a programmable IC 1 is illustrated, which includes a number of different programmable blocks, including transceivers 37, configurable logic blocks ("CLBs") 33, random access memory blocks ("BRAMs") 34, input / output blocks ("IOBs") 36, configuration and clock logic ("CONFIG / CLOCKS") 42, digital signal processing blocks ("DSPs") 35, dedicated input / output blocks ("I / Os") 41 (e.g., configuration ports and clock ports), and other programmable logic 39 (such as) a digital clock manager, analog-to-digital converters, system monitoring logic, etc. The FPGA may also include a PCIe interface 40, an analog-to-digital converter (ADC) 38, etc.

[0074] In some FPGAs, each programmable block may include at least one programmable interconnect element ("INT") 43 having connections to input terminals and output terminals 48 of programmable logic elements within the same block, such as Fig. 9As shown in the example included at the top of . Each programmable interconnect element 43 may also include a connection to an interconnect fragment 49 of (multiple) adjacent programmable interconnect elements in the same tile or (multiple) other tiles. Each programmable interconnect element 43 may also include a connection to an interconnect fragment 50 of a general routing resource between logic blocks (not shown). General routing resources may include routing channels between logic blocks (not shown), which include tracks of interconnect fragments (e.g., interconnect fragments 50) and switch blocks (not shown) for connecting interconnect fragments. The interconnect fragments of the general routing resources (e.g., interconnect fragments 50) may span one or more logic blocks. The programmable interconnect elements 43, together with the general routing resources, implement a programmable interconnect structure ("programmable interconnect") for the illustrated FPGA.

[0075] In one example implementation, the CLB 33 may include a configurable logic element (“CLE”) 44 that can be programmed to implement user logic and a single programmable interconnect element (“INT”) 43. In addition to one or more programmable interconnect elements, the BRAM 34 may also include a BRAM logic element (“BRL”) 45. Typically, the number of interconnect elements included in a block depends on the height of the block. In the depicted example, the BRAM block has the same height as five CLBs, but other numbers (e.g., four) may also be used. In addition to an appropriate number of programmable interconnect elements, the DSP block 35 may also include a DSP logic element (“DSPL”) 46. In addition to one instance of the programmable interconnect element 43, the IOB 36 may also include, for example, two instances of an input logic element / output logic element (“IOL”) 47. It will be apparent to those skilled in the art that, for example, the actual I / O pads connected to the I / O logic element 47 are typically not limited to the area of ​​the input logic element / output logic element 47.

[0076] In the depicted example, a horizontal region near the center of the die (eg Fig. 9 ) is used for configuration, clock and other control logic. Vertical columns 51 extending from this horizontal area or column are used to distribute clock and configuration signals across the entire width of the FPGA.

[0077] use Fig. 9 Some FPGAs of the illustrated architecture include additional logic blocks that disrupt the regular columnar structure that makes up a large portion of the FPGA. The additional logic blocks may be programmable blocks and / or dedicated logic.

[0078] Notice, Fig. 9 It is intended only to illustrate an exemplary FPGA architecture. For example, the number of logic blocks in a row, the relative width of the rows, the number and order of the rows, the types of logic blocks included in the rows, the relative sizes of the logic blocks, and Fig. 9 The interconnect / logic implementation included at the top of is purely exemplary. For example, in an actual FPGA, wherever CLBs appear, more than one adjacent CLB row is typically included to facilitate efficient implementation of user logic, but the number of adjacent CLB rows varies with the overall size of the FPGA.

[0079] In one example, a method of implementing a neural network includes: selecting a first neural network architecture from a search space; training the neural network having the first neural network architecture to obtain accuracy and implementation cost, the implementation cost being based on a programmable device of an inference platform; selecting a second neural network architecture from the search space based on the accuracy and the implementation cost; and outputting weights and hyperparameters of the neural network having the second neural network architecture.

[0080] In one example, the step of selecting the first neural network architecture is performed by a reinforcement agent, wherein the reinforcement agent selects the first neural network architecture from the search space with probability P, and wherein the reinforcement agent adjusts the probability P based on a function of accuracy and implementation cost.

[0081] In one example, the reinforcement agent is a recurrent neural network (RNN).

[0082] In one example, the first neural network architecture is one of a plurality of neural network architectures, wherein the training step includes evaluating the plurality of neural network architectures using a fitness function.

[0083] In one example, the step of selecting the first neural network architecture is performed by a tuning agent, and wherein the tuning agent selects hyperparameters for the second neural network architecture based on a function of accuracy and implementation cost.

[0084] In one example, the tuning agent selects hyperparameters using grid search, random search, or Bayesian search.

[0085] In one example, the method further includes: generating a circuit design based on the weights and hyperparameters of the neural network; and implementing the circuit design for a programmable logic device.

[0086] In one example, a computer system includes a memory having program code stored therein; and a processor configured to execute the program code to implement a neural network by: selecting a first neural network architecture from a search space; training the neural network having the first neural network architecture to obtain an accuracy and an implementation cost, the implementation cost being based on a programmable device of an inference platform; selecting a second neural network architecture from the search space based on the accuracy and the implementation cost; and outputting weights and hyperparameters of the neural network having the second neural network architecture.

[0087] In one example, the processor is configured to execute code to select a first neural network architecture using a reinforcement agent, wherein the reinforcement agent selects the first neural network architecture from a search space with a probability P, and wherein the reinforcement agent adjusts the probability P based on a function of accuracy and implementation cost.

[0088] In one example, the reinforcement agent is a recurrent neural network (RNN).

[0089] In one example, the first neural network architecture is one of a plurality of neural network architectures, wherein the processor executes code to perform training by evaluating the plurality of neural network architectures using a fitness function.

[0090] In one example, a processor executes code using a tuning agent to select a first neural network architecture, and wherein the tuning agent selects hyperparameters of a second neural network architecture based on a function of accuracy and implementation cost.

[0091] In one example, the tuning agent selects hyperparameters using grid search, random search, or Bayesian search.

[0092] The various examples described herein may employ various computer-implemented operations involving data stored in a computer system. For example, these operations may require physical manipulation of physical quantities, that is, generally, although not necessarily, these physical quantities may take the form of electrical or magnetic signals, which electrical or magnetic signals or their implementations can be stored, transmitted, combined, compared or otherwise manipulated. Further, such manipulations are generally referred to in terms such as generation, identification, determination, or comparison. Any operation described herein that forms part of one or more example techniques described herein may be a useful machine operation. In addition, one or more example techniques also relate to a device or apparatus for performing these operations. The apparatus may be specially constructed for a desired specific purpose, or it may be a general-purpose computer selectively activated or configured by a computer program stored in a computer. In particular, various general-purpose machines may be used together with computer programs written according to the teachings herein, or it may be more convenient to construct more specialized devices to perform the desired operations. The various examples described herein may be practiced with other computing system configurations, including handheld devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like.

[0093] One or more example techniques described herein may be implemented as one or more computer programs or as one or more computer program modules embodied in one or more computer-readable media. The term computer-readable medium refers to any data storage device that can store data that can then be input into a computer system, that is, the computer-readable medium can be based on any existing or subsequently developed technology for embodying them in a manner that enables computer programs to be read by a computer. Examples of computer-readable media include hard drives, network attached storage (NAS), read-only memories, random access memories (e.g., flash memory devices), CDs (optical disks) (i.e., CD-ROMs, CD-Rs, or CD-RWs), DVDs (digital versatile disks), magnetic tapes, and other optical and non-optical data storage devices. Computer-readable media may also be distributed on network-coupled computer systems, thereby storing and executing computer-readable code in a distributed manner.

[0094] While the foregoing is directed to particular examples, other and further examples may be devised without departing from the basic scope thereof, and the scope of which is to be determined by the following claims.

Claims

1. A method for implementing a neural network, comprising: Selecting a first neural network architecture from the search space; training the neural network having the first neural network architecture to obtain an accuracy and an implementation cost, wherein the implementation cost is based on a programmable device of an inference platform; selecting a second neural network architecture from the search space based on the accuracy and the implementation cost; Outputting weights, number of layers, number of channels per layer, bit width, number of processing elements, and fault tolerance properties for the neural network having the second neural network architecture; as well as A circuit design is implemented in the programmable device, the circuit design being generated based on the weights, the number of layers, the number of channels per layer, the bit width, the number of processing elements, and the fault tolerance attribute.

2. The method of claim 1 , wherein selecting the first neural network architecture is performed by a reinforcement agent, wherein the reinforcement agent selects the first neural network architecture from the search space with probability P, and wherein the reinforcement agent adjusts the probability P based on a function of the accuracy and the implementation cost.

3. The method of claim 2, wherein the reinforcement agent is a recurrent neural network (RNN).

4. The method of claim 1, wherein the first neural network architecture is one of a plurality of neural network architectures, wherein the step of training comprises: The plurality of neural network architectures are evaluated using a fitness function.

5. The method of claim 1 , wherein selecting the first neural network architecture is performed by a tuning agent, and wherein the tuning agent selects hyperparameters for the second neural network architecture based on a function of the accuracy and the implementation cost.

6. The method of claim 5, wherein the tuning agent selects the hyperparameters using a grid search, a random search, or a Bayesian search.

7. A computer system comprising: a memory in which program code is stored; as well as A processor is configured to execute the program code to implement a neural network in the following manner: Selecting a first neural network architecture from the search space; training the neural network having the first neural network architecture to obtain an accuracy and an implementation cost, wherein the implementation cost is based on a programmable device of an inference platform; selecting a second neural network architecture from the search space based on the accuracy and the implementation cost; Outputting the weights, number of layers, number of channels per layer, bit width, number of processing elements, and fault tolerance properties of the neural network having the second neural network architecture; as well as A circuit design is implemented in the programmable device, the circuit design being generated based on the weights, the number of layers, the number of channels per layer, the bit width, the number of processing elements, and the fault tolerance attribute.

8. The computer system of claim 7, wherein the processor is configured to execute the code to select the first neural network architecture using a reinforcement agent, wherein the reinforcement agent selects the first neural network architecture from the search space with a probability P, and wherein the reinforcement agent adjusts the probability P based on a function of the accuracy and the implementation cost.

9. The computer system of claim 8, wherein the reinforcement agent is a recurrent neural network (RNN).

10. The computer system of claim 7, wherein the first neural network architecture is one of a plurality of neural network architectures, wherein the processor executes the code to perform the training by evaluating the plurality of neural network architectures using a fitness function.

11. The computer system of claim 7, wherein the processor executes the code using a tuning agent to select the first neural network architecture, and wherein the tuning agent selects hyperparameters of the second neural network architecture based on a function of the accuracy and the implementation cost.

12. The computer system of claim 11, wherein the tuning agent selects the hyperparameters using a grid search, a random search, or a Bayesian search.

Images