A chip design-oriented neural architecture search method

Abstract

The application discloses a chip design-oriented neural architecture search method, which comprises the following steps: constructing a super network by using a large neural network containing multiple convolution operators; then converting a non-differentiable objective function into a differentiable objective function by using a random relaxation method, modeling the performance of a chip into a hardware model by using experiments; optimizing the neural network structure and neural network parameters by using a multivariate geometric information optimization search method and combining a Monte Carlo method to estimate the gradient; modeling the hardware performance of a specific chip based on experiments, combining a hardware model obtained based on experiments, and adopting a dynamic programming method to search the structure of the trained super network to determine a final target network. The application can efficiently search an optimal and efficient network structure on a chip with limited computing capacity or specific performance requirements.

Description

Technical Field

[0001] This invention relates to the field of deep learning neural network technology, and in particular to a neural architecture search method for chip design. Background Technology

[0002] Over the past decade, deep learning technology has achieved remarkable success in various artificial intelligence tasks, such as image processing, natural language processing, and translation. Deep learning relies on deep neural networks, which extract and combine shallow features layer by layer through the hierarchical structure of the network model to form more abstract deep representations. This transforms the features of the sample data space into a new high-level feature space, facilitating tasks such as classification and prediction. However, the widespread application of deep learning is mainly attributed to the performance improvements in computing hardware and the emergence of novel artificial neural network structures. In 1999, NVIDIA first introduced the concept of GPU (Graphics Processing Unit) when it released its iconic GeForce 256 product. Due to the powerful matrix parallel computing capabilities of GPUs, they quickly replaced CPUs as a crucial component of deep learning. Statistics show that in the past 10 years, GPU performance has improved by approximately 20 times, enabling the implementation of computationally demanding deep learning technologies. On the other hand, the emergence of network architectures represented by ResNet (see reference Deep residual learning for image recognition.arXiv preprint arXiv:1512.03385,2015) has solved various technical problems, including network degradation, increasing the complexity and expressiveness of artificial neural networks and enabling them to solve more difficult tasks.

[0003] However, most high-performance network models are currently designed manually by human experts with extensive experience in network architecture engineering and relevant domain knowledge (such as image processing). Manually designing neural networks is a highly complex and tedious task, requiring numerous hyperparameters and network structures, as well as extensive tuning. Furthermore, since network structures are often specific to the task and dataset, changes in data distribution and objectives necessitate redesigning the network structure. In other words, network structures designed for a specific task often fail to achieve satisfactory results when generalized to other application areas. Neural architecture search (NES) is a method for automatically designing network architectures. By treating the neural network structure as a generalized hyperparameter, it models the design of the neural network as an optimization problem and employs black-box optimization techniques, such as reinforcement learning, to solve the optimization problem, thereby achieving automated architecture engineering. Because NES can automatically design neural network models based on a given dataset and task objective, it enables professionals with relevant domain knowledge but lacking deep learning experience to apply deep neural networks to their respective fields, thus facilitating the wider and faster application of deep learning technology.

[0004] Early neural architecture search algorithms were typically computationally intensive, requiring the training of numerous models to convergence, a computationally expensive task. In the early stages of neural architecture search, thousands of GPU days (1 GPU day = one day of operation for one GPU) were often needed to complete the search. With the introduction of weight sharing and probabilistic optimization strategies, the efficiency of architecture search has rapidly improved, now reaching several GPU times. Some non-training evaluation methods can even compress the model search time to the unit of GPU seconds.

[0005] With the further development of computing hardware, edge computing devices, represented by microprocessor chips, have gradually come into the industry's view. These chips are accelerators specifically designed for applications based on artificial neural networks, enabling data processing and other operations to be "localized," freeing them from the constraints of "network connectivity." How to deploy deep learning applications on devices with limited computing resources has become a new research hotspot. Neural architecture search technology can be combined with model compression technology to automatically design lightweight neural network models that can be deployed on mobile devices by imposing hardware constraints on the architecture optimization problem. However, current research on neural architecture search technology mainly focuses on improving efficiency and expanding application areas; customizing neural networks according to specific hardware to fully utilize hardware computing resources remains a research hotspot. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies by providing a neural architecture search method for chip design.

[0007] The objective of this invention is achieved through the following technical solution: a neural architecture search method for chip design, comprising the following sub-steps:

[0008] (1) Collect the target dataset corresponding to the target task, and construct a hypernetwork α for the target task on the target dataset;

[0009] (2) Combining the supernetwork α, construct the objective function under the hardware model constraint, and use the stochastic relaxation method to transform the objective function and the hardware constraint under the hardware model constraint into a differentiable form;

[0010] (3) Search for the optimal learnable parameters and optimal network probability of the hypernetwork α using a multivariate geometric information optimization algorithm without hardware constraints;

[0011] (4) The search for the optimal architecture is transformed into a complete knapsack problem by using hardware constraints. The optimal learnable parameters and optimal network probabilities obtained by the search are searched again using dynamic programming to obtain the most efficient neural network structure for a specific chip.

[0012] Furthermore, the supernetwork includes a backbone and functional heads. The backbone includes multiple 2D convolutional layers with different kernel sizes and different numbers of channels. The functional heads are set according to the task of the neural network and are not adjusted.

[0013] The input of the hypernetwork is RGB three-channel data, and the output is the result of the target task;

[0014] The supernetwork replaces one operator in the original normal neural network with a variety of operators that can be used to construct parallel structures, thereby giving the original structure a variety of operator choices.

[0015] Furthermore, the various operators that can be used to construct parallel structures include fully connected layers, 1×1 convolutional layers, 3×3 convolutional layers, 5×5 convolutional layers, identity mapping layers, 3×3 pooling layers, 3×3 channel-wise convolutional layers, and 5×5 channel-wise convolutional layers.

[0016] Furthermore, the construction of the backbone specifically includes:

[0017] There are n parallel operators in a parallel structure cell, with structural parameters β0 to β1. n-1 This represents the probability that the corresponding operator is selected in the current structure. The value of the structure parameter is softmaxed and multiplied by the output of each parallel operator. This multiplication is used as the scale of the output, which is the output of the cell, expressed as:

[0018]

[0019] Where O(x) represents the output of the parallel structure cell, x represents the input of the parallel structure cell, n represents the number of parallel operators in the parallel structure cell, and β0, β... j Let o represent the structural parameters corresponding to the i-th and j-th operators in the parallel structure cell, respectively. i (x) represents the output of the i-th operator;

[0020] By combining parallel cell structures with activation functions and batch normalization operations as a layer of a supernetwork, and repeating this process multiple times, the backbone of the supernetwork can be constructed.

[0021] Furthermore, the construction of the objective function under hardware model constraints, in conjunction with the hypernetwork α, specifically includes:

[0022] The hypernetwork α is formally represented as a directed acyclic graph of nodes and edges, where each node represents a specific tensor; each edge represents an operator sampled from the corresponding search space, located between two nodes; subsequently, the hypernetwork α constructs an objective function f(w) under hardware model constraints. 0 In this case, the optimization objective under the hardware model constraints is expressed as:

[0023]

[0024] Where, f(w) 0 ,α): W×A→R is a differentiable objective function, representing the inference accuracy of the neural network for the target task; τ(α)<Ω represents the hardware constraints under the hardware model constraints; τ represents the mapping function of different neural network architectures for chip performance. When different edge operators are selected, the neural network structure changes, thus affecting the network's operating efficiency on the chip; Ω represents the limitations of different chips; A represents the set of hypernetworks; W represents the set of learnable parameters of the hypernetwork operators; w 0 Let represent the initial learnable parameters of the supernetwork α.

[0025] Furthermore, the method of using stochastic relaxation to transform the objective function and hardware constraints under the hardware model constraints into differentiable forms specifically includes:

[0026] Define a set of multivariate distributions p on the hypernetwork α. θ (α), for each edge in the hypernetwork α, the corresponding distribution variables are independent, that is:

[0027]

[0028] Where N represents the total number of edges in the hypernetwork α; The distribution variables corresponding to the distributions of each edge in the hypernetwork α are independent, expressed as:

[0029]

[0030] Where, θ g,m K represents the probability of the m-th type operator on the g-th edge in the network search space; g This indicates that there may be K on the g-th edge in the hypernetwork α. g There are operators, m = 1, ..., m, ..., K. g ;

[0031] Then, the objective function f(w) was adjusted using a random relaxation method. 0 Transforming α) into a differentiable form, that is, into a probabilistic expectation form:

[0032]

[0033] Where, θ 0 This represents the initial network probability of the supernetwork α;

[0034] Furthermore, the hardware constraint τ(α)<Ω under the hardware model constraint is transformed into a differentiable form, i.e., into a multivariate probability distribution form, using a stochastic relaxation method:

[0035]

[0036] Furthermore, the step of searching for the optimal learnable parameters and optimal network probability of the hypernetwork α using a multivariate geometric information optimization algorithm under no hardware constraints specifically includes:

[0037] Under no hardware constraints, a multivariable geometric information optimization algorithm is used to optimize the learnable parameters and network probability of the supernetwork α using the alternating gradient descent method.

[0038] Perform the (t+1)th iteration optimization:

[0039] w t+1 =w t +∈ w ▽ w J(w t ,θ t );

[0040] θ t+1 =θ t +∈ θ ▽ θ J(w t+1 ,θ t );

[0041] Among them, w t Let θ represent the learnable parameters after the t-th iteration optimization. t w represents the network probability after the t-th iteration optimization; t+1Let θ represent the learnable parameters after the (t+1)th iteration optimization. t+1 Represents the network probability after the (t+1)th iteration optimization; ∈ w Represents the learning rate for learnable parameters; ∈ θ This represents the learning rate for network probabilities;

[0042] Using the Monte Carlo method to examine ▽ w J(w t ,θ t ) and ▽ θ J(w t+1 ,θ t To estimate θ, first... t Hypernetwork sampling under a probability distribution, where the operator on each edge is obtained through sampling, for a total of λ sets of data are sampled, and then the gradient is estimated as follows:

[0043]

[0044]

[0045] Where, α i This represents the network structure sample obtained by the i-th sampling.

[0046] In each step of the alternating gradient descent formula, two backpropagations are performed. The first backpropagation fixes the network probability θ. t The gradient is estimated using the Monte Carlo method. w J(w t ,θ t Then, gradient descent is used to update the learned parameters w. t The learnable parameter w is obtained. t+1 The second fixed learnable parameter w t+1 The gradient is estimated using the Monte Carlo method. θ J(w t+1 ,θ t Then, gradient descent is used to update the network probability θ. t The network probability θ is obtained. t+1 ;

[0047] When the objective function J(w) t+1 ,θ t+1 When the maximum value is reached, it indicates convergence, and optimization stops. The learnable parameters and network probabilities after this round of iteration optimization are then used as the optimal learnable parameters and optimal network probabilities of the supernetwork α. Otherwise, the next round of iteration optimization begins.

[0048] Furthermore, the process of transforming the search for the optimal architecture using hardware constraints into a complete knapsack problem, and then using dynamic programming to re-search the optimal learnable parameters and optimal network probabilities obtained in the search, to obtain the most efficient neural network structure for a specific chip, specifically including:

[0049] Using hardware constraints, the problem of searching for the optimal architecture is transformed into a complete knapsack problem:

[0050]

[0051] Among them, u g This represents the index of the operator selected on the g-th edge in network α. Describes the u-th edge on the g-th edge. g One operator; This represents the u-th edge on the g-th edge obtained based on the optimal network probability of the supernetwork α. g The probability of an operator;

[0052] In the complete knapsack problem, the optimal learnable parameters and optimal network probabilities obtained from the initial search are then searched again using dynamic programming. The iterative path obtained by dynamic programming is represented as follows:

[0053]

[0054] Among them, F[g,v g The ] indicates that the redundant performance of the first g edges is v, calculated from the forward direction. g The maximum cumulative network probability at time; v g The remaining available performance can be expressed as:

[0055]

[0056] When reasoning reaches the last layer of the neural network structure, the search stops, and the obtained structure is the most efficient neural network structure for a specific chip.

[0057] The beneficial effects of this invention are as follows: This invention uses a stochastic relaxation method to transform a non-differentiable objective function into a differentiable objective function, utilizes multivariate geometric information to optimize the search method, and combines Monte Carlo gradient estimation to optimize the neural network structure and parameters, thereby improving the performance of the neural network on the target task. It uses the complete knapsack problem to model the hardware performance requirements of a specific chip, and combines this with an experimentally obtained hardware model to perform structure search on the trained supernetwork using dynamic programming. The searched target network can operate efficiently on chips with limited computing power or specific performance requirements. Furthermore, the high efficiency of the dynamic programming method improves the efficiency of neural network structure search for specific chips. Attached Figure Description

[0058] Figure 1 This is a flowchart illustrating a neural architecture search method for chip design.

[0059] Figure 2 A schematic diagram of a supernetwork construction method for neural architecture search in chip design;

[0060] Figure 3 This is a schematic diagram of the multivariate geometric optimization search algorithm used.

[0061] Figure 4 This is a schematic diagram of the dynamic programming algorithm. Detailed Implementation

[0062] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0063] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0064] It should be understood that although the terms first, second, third, etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of this invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0065] The present invention will now be described in detail with reference to the accompanying drawings. Unless otherwise specified, the features of the following embodiments and implementations can be combined with each other.

[0066] Example 1

[0067] like Figure 1As shown, this invention provides a neural architecture search method for chip design. The neural architecture search method based on dynamic programming network generation is used to search for neural network structures under specific chip performance requirements to obtain the best accuracy on devices with limited computing power and improve the running performance of on-chip models.

[0068] Includes the following steps:

[0069] (1) Collect the target dataset corresponding to the target task, and build a hypernetwork α for the target task on the target dataset.

[0070] Optionally, the target task includes, but is not limited to, image classification, object tracking, and object detection. After collecting the target dataset corresponding to the target task, a hypernetwork can be constructed on the large target dataset corresponding to the target task using existing feasible neural network structures.

[0071] like Figure 2 As shown, in this embodiment, the supernetwork includes a backbone and functional heads. The backbone comprises multiple 2D convolutional layers with different kernel sizes and channel numbers, and is the main part of the neural network computation. The functional heads are set according to the neural network task and are not adjusted. The following description refers to the backbone of the supernetwork. The input of the supernetwork is an RGB image, and the output is the result of the target task, such as image classification, image recognition, or semantic segmentation. The supernetwork uses multiple operators that can be used to construct parallel structures to replace one operator in the original normal neural network, thereby providing the original structure with multiple operator choices.

[0072] Various operators can be used to construct parallel structures, including but not limited to: fully connected layers, 1×1 convolutional layers, 3×3 convolutional layers, 5×5 convolutional layers, identity mapping layers, 3×3 pooling layers, 3×3 channel-wise convolutional layers, and 5×5 channel-wise convolutional layers. Alternatively, a cell-based search space approach can be used, treating existing artificial neural network modules as searchable operators.

[0073] The construction of the backbone specifically includes:

[0074] There are n parallel operators in a parallel structure cell, with structural parameters β0 to β1. n-1 This represents the probability that the corresponding operator is selected in the current structure. The value of the structure parameter is softmaxed and multiplied by the output of each parallel operator. This multiplication is used as the scale of the output, which is the output of the cell, expressed as:

[0075]

[0076] Where O(x) represents the output of the parallel structure cell, x represents the input of the parallel structure cell, n represents the number of parallel operators in the parallel structure cell, and β0 and βj represent the structure parameters corresponding to the i-th and j-th operators in the parallel structure cell, respectively. i (x) represents the output of the i-th operator; the above content describes the calculation process of a cell from input to output.

[0077] By combining parallel cell structures with activation functions and batch normalization operations as a layer of a supernetwork, and repeating this process multiple times, the backbone of the supernetwork can be constructed.

[0078] (2) Combine the supernetwork α to construct the objective function under the hardware model constraint, and use the stochastic relaxation method to transform the objective function and the hardware constraint under the hardware model constraint into a differentiable form.

[0079] It should be understood that, due to the significant differences in the performance of various chips, the same network will have different operating efficiencies on different chips. In order to ensure that the requirements of inference accuracy and operating efficiency can be balanced on chips with limited computing resources, it is necessary to introduce a hardware model to model the performance of the chip itself during the neural network architecture search process.

[0080] It should be noted that there are many performance metrics from the chip hardware itself. Optionally, target metrics include, but are not limited to, latency, video memory usage (FLOPs), etc.

[0081] The construction of the objective function under hardware model constraints, in conjunction with the hypernetwork α, specifically includes:

[0082] The hypernetwork α is formally represented as a directed acyclic graph of nodes and edges, where each node represents a specific tensor; each edge represents an operator sampled from the corresponding search space, located between two nodes; subsequently, the hypernetwork α constructs an objective function f(w) under hardware model constraints. 0 In this case, the optimization objective under the hardware model constraints is expressed as:

[0083]

[0084] Where, f(w) 0 ,α): W×A→R is a differentiable objective function, representing the inference accuracy of the neural network for the target task; τ(α)<Ω represents the hardware constraints under the hardware model constraints; τ represents the mapping function of different neural network architectures for chip performance. When different edge operators are selected, the neural network structure changes, thus affecting the network's operating efficiency on the chip; Ω represents the limitations of different chips; A represents the set of hypernetworks; W represents the set of learnable parameters of the hypernetwork operators; w0 Let represent the initial learnable parameters of the supernetwork α.

[0085] The method of using stochastic relaxation to transform the objective function and hardware constraints under hardware model constraints into differentiable forms specifically includes:

[0086] like Figure 3 As shown, a set of multivariate distributions p is defined on the hypernetwork α. θ (α), for each edge in the hypernetwork α, the corresponding distribution variables are independent, that is:

[0087]

[0088] Where N represents the total number of edges in the hypernetwork α; The distribution variables corresponding to the distributions of each edge in the hypernetwork α are independent can be represented as:

[0089]

[0090] Where, θ g,m K represents the probability of the m-th type operator on the g-th edge in the network search space; g This indicates that there may be K on the g-th edge in the hypernetwork α. g There are operators, m = 1, ..., m, ..., K. g ;

[0091] Then, the objective function f(w) was adjusted using a random relaxation method. 0 Transforming α) into a differentiable form, that is, into a probabilistic expectation form:

[0092]

[0093] Where, θ 0 Let α represent the initial network probability of the supernetwork α.

[0094] It should be understood that after the above random relaxation, the non-differentiable network structure parameters are transformed into differentiable probability distribution parameters.

[0095] Furthermore, the hardware constraint τ(α)<Ω under the hardware model constraint is transformed into a differentiable form, i.e., into a multivariate probability distribution form, using a stochastic relaxation method:

[0096]

[0097] Taking expected latency (hardware performance) as an example, the network latency is modeled as the expected latency of all selectable operators on the g-th edge of the continuous function τ(α) of the neural network architecture:

[0098]

[0099] Where αg,m represents the m-th operator structure on the g-th edge in the hypernetwork α; τ(α g,m ) represents the time delay of the m-th operator on the g-th edge in the hypernetwork α;

[0100] Therefore, the time delay of the entire supernetwork α can be estimated as:

[0101]

[0102] It should be understood that different hardware has different performance characteristics. Before conducting a search, multiple experimental tests can be used to obtain continuous functions that estimate the performance of different operators.

[0103] For example, to address the performance requirement of time delay, the operators of different layers are first loaded onto the specified hardware device. Based on the input required by the operator, a random input tensor is selected, the program is run, the inference time of a single operator on the input tensor is calculated, and the result is recorded in a table to obtain the hardware model. Then, based on the practical needs, an upper limit value of the time delay is set as a constraint on the hardware model.

[0104] (3) Under the condition of no hardware constraints, the optimal network weights and optimal network probabilities of the supernetwork α are searched using the multivariate geometric information optimization algorithm.

[0105] It should be noted that although the transformed objective function has the property of being differentiable, it has two types of parameters to be optimized, so the optimization method needs to be carried out by alternating updates. At the same time, the analytical gradient is actually difficult to obtain, so the Monte Carlo method is used to estimate the gradient.

[0106] Under no hardware constraints, a multivariable geometric information optimization algorithm is used to optimize the network weights and probabilities of the supernetwork α using the alternating gradient descent method.

[0107] Perform the (t+1)th iteration optimization:

[0108] w t+1 =w t +∈ w ▽ w J(w t ,θ t );

[0109] θ t+1 =θ t +∈ θ ▽ θ J(w t+1 ,θ t );

[0110] Among them, w t Let θ represent the network weights after the t-th iteration of optimization.t w represents the network probability after the t-th iteration optimization; t+1 Let θ represent the network weights after the (t+1)th iteration optimization. t+1 Represents the network probability after the (t+1)th iteration optimization; ∈ w Represents the learning rate for the network weights; ∈ θ This represents the learning rate for the network probability.

[0111] Using the Monte Carlo method to examine ▽ w J(w t ,θ t ) and ▽ θ J(w t+1 ,θ t To estimate θ, first... t Hypernetwork sampling under a probability distribution, where the operator on each edge is obtained through sampling, for a total of λ sets of data are sampled, and then the gradient is estimated as follows:

[0112]

[0113]

[0114] Where, α i This represents the network structure sample obtained by the i-th sampling.

[0115] In each step of the alternating gradient descent formula, two backpropagations are performed. The first backpropagation fixes the network probability θ. t The gradient is estimated using the Monte Carlo method. w J(w t ,θ t Then, gradient descent is used to update the network weights w being used. t To obtain the network weights w t+1 The second time, the network weights w are fixed. t+1 The gradient is estimated using the Monte Carlo method. θ J(w t+1 ,θ t Then, gradient descent is used to update the network probability θ. t The network probability θ is obtained. t+1 ;

[0116] When the objective function J(w) t+1 ,θ t+1 If convergence is reached when the maximum value is reached, optimization stops, and the network weights and probabilities after this round of iteration optimization are taken as the optimal network weights and optimal network probabilities of the supernetwork α; otherwise, the next round of iteration optimization begins.

[0117] It should be understood that the search space includes the structural depth of the neural network, the number of channels in each layer of the neural network, the network structure type of each layer of the neural network (e.g., convolutional networks, fully connected layers, etc.), the type and size of the convolutional kernels in each layer of the neural network, the form of residual connections, and pre-designed artificial neural network structures based on cells.

[0118] It should be understood that the training methods used in training the hypernetwork are conventional. A large public dataset is selected, and the data from it is input into the hypernetwork. Predicted labels are output, and the loss function of the hypernetwork is calculated based on the true and predicted labels of the data in the dataset. Using this loss function, the structural parameter α is first fixed, and gradient descent is used to update the weights. Then, the weights are fixed again using the same loss function, and gradient descent is used to update the structural parameter α. This process is iterated until the training stops. At this point, the parameters of the network operators in different layers and the corresponding probability distribution of the network structure can be obtained without hardware constraints.

[0119] (4) The search for the optimal architecture is transformed into a complete knapsack problem by using hardware constraints. The optimal network weights and optimal network probabilities obtained by the search are searched again using dynamic programming to obtain the most efficient neural network structure for a specific chip.

[0120] Using hardware constraints, the problem of searching for the optimal architecture is transformed into a complete knapsack problem:

[0121]

[0122] Among them, u g This represents the index of the operator selected on the g-th edge in network α. Describes the u-th edge on the g-th edge. g One operator; This represents the u-th edge on the g-th edge obtained based on the optimal network probability of the supernetwork α. g The probability of an operator;

[0123] It should be understood that the above optimization objective means that the network structure should have the maximum cumulative operator distribution probability without exceeding the chip performance limit. The higher the operator distribution probability, the closer the network is to the optimal network result.

[0124] like Figure 4 As shown, in the complete knapsack problem, the optimal network weights and optimal network probabilities obtained by the search are searched again using dynamic programming. The iterative path obtained by dynamic programming is represented as follows:

[0125]

[0126] Among them, F[g,v g The ] indicates that the redundant performance of the first g edges is v, calculated from the forward direction.g The maximum cumulative network probability at time; v g The remaining available performance can be expressed as:

[0127]

[0128] When reasoning reaches the last layer of the neural network structure, the search stops, and the obtained structure is the most efficient neural network structure for a specific chip.

[0129] It should be understood that since the time complexity of the dynamic programming algorithm is O(N×V×M), where N is the number of edges in the hypernetwork structure, M is the number of possible operators, and V is the complexity introduced by the dynamic programming algorithm, it can be estimated as follows:

[0130]

[0131] These three quantities are relatively small in practice, so the algorithm is very efficient, taking only tens of seconds on a single graphics card device.

[0132] For example, when using dynamic programming to search for the optimal network structure for a given chip based on time delay performance, a supernetwork for the specific task is first trained, including the probability distribution of different operators selected at different layers of the network parameters. Then, the time delay of different layer operators on the chip is tested, and a matrix relationship is established between the operators and the time delay. Using the dynamic programming algorithm, the program is traversed from the first layer, and the optimal cumulative probability weights that satisfy the constraints are obtained layer by layer until the last layer of the network. The resulting network structure is the optimal network structure for that chip.

[0133] In summary, this invention specifically generates a specific neural network for a particular chip. It uses a stochastic relaxation method to transform a non-differentiable objective function into a differentiable one, leverages multivariate geometric information to optimize the search method, and combines Monte Carlo methods to optimize the neural network structure and parameters, thereby improving the performance of the neural network on the target task. Furthermore, it uses the complete knapsack problem to model the hardware performance requirements of the specific chip, and combines this with an experimentally obtained hardware model to perform a dynamic programming method to search for the structure of the trained supernetwork. The resulting target network can operate efficiently on chips with limited computing power or specific performance requirements.

[0134] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A neural architecture search method for chip design, characterized in that, Includes the following steps: (1) Collect the target dataset corresponding to the target task, and build a hypernetwork for the target task on the target dataset. ; (2) Combining the aforementioned hypernetwork An objective function constrained by a hardware model is constructed, and a stochastic relaxation method is used to transform the objective function and the hardware constraints under the hardware model into differentiable forms. The input of the hypernetwork is RGB three-channel data, and the output is the result of the target task. The hypernetwork is then combined with... Construct the objective function under the constraints of the hardware model, specifically including: The hypernetwork Formally represented as a directed acyclic graph of nodes and edges, where each node represents a specific tensor; each edge represents an operator sampled from the corresponding search space, located between two nodes; subsequently, the hypernetwork... Construct the objective function under the constraints of the hardware model. The optimization objective under the constraints of the hardware model is expressed as: in, : A differentiable objective function, i.e., the inference accuracy of the neural network for the target task; Represents hardware constraints under hardware model constraints; This represents the mapping function of different neural network architectures to chip performance. When different edge operators are selected, the neural network structure changes, thereby affecting the network's operating efficiency on the chip. The constraints of different chips are represented; A represents the set of hypernetworks; W represents the set of learnable parameters of the hypernetwork operators. Indicates hypernetwork The initial learnable parameters; The supernetwork replaces one operator in the original normal neural network with a variety of operators that can be used to construct parallel structures, thereby giving the original structure a variety of operator choices. (3) Searching for hypernetworks using multivariable geometric information optimization algorithms under no hardware constraints The optimal learnable parameters and optimal network probability; (4) Using hardware constraints, the search for the optimal architecture is transformed into a complete knapsack problem. The optimal learnable parameters and optimal network probabilities obtained from the search are then searched again using dynamic programming to obtain the most efficient neural network structure for a specific chip. Specifically, this includes: Using hardware constraints, the problem of searching for the optimal architecture is transformed into a complete knapsack problem: ； in, Indicates hypernetwork The CCP Edge; Represents network The Middle The index of the operator selected on the side, Indicates the first The side One operator; This represents the first probability obtained based on the optimal network probability of supernetwork e. The side The probability of an operator.

2. The neural architecture search method for chip design according to claim 1, characterized in that, The supernetwork includes a backbone and functional heads. The backbone includes multiple 2D convolutional layers with different kernel sizes and channel numbers. The functional heads are set according to the task of the neural network and are not adjusted.

3. The neural architecture search method for chip design according to claim 2, characterized in that, The various operators that can be used to construct parallel structures include fully connected layers, 1×1 convolutional layers, 3×3 convolutional layers, 5×5 convolutional layers, identity mapping layers, 3×3 pooling layers, 3×3 channel-wise convolutional layers, and 5×5 channel-wise convolutional layers.

4. The neural architecture search method for chip design according to claim 2, characterized in that, The construction of the backbone specifically includes: There are n parallel operators in a parallel structure cell, and the structural parameters are... arrive This represents the probability that the corresponding operator is selected in the current structure. The value of the structure parameter is softmaxed and multiplied by the output of each parallel operator. This multiplication is used as the scale of the output, which is the output of the cell, expressed as: in, This represents the output of a parallel structure cell. This represents the input of a parallel structure cell. Indicates parallel structure cells have Parallel operators, , These represent the first and second cells in the parallel structure. The, the The structural parameters corresponding to each operator Indicates the first The output of each operator; By combining parallel cell structures with activation functions and batch normalization operations as a layer of a supernetwork, and repeating this process multiple times, the backbone of the supernetwork can be constructed.

5. The neural architecture search method for chip design according to claim 1, characterized in that, The method of using stochastic relaxation to transform the objective function and hardware constraints under hardware model constraints into differentiable forms specifically includes: In hypernetworks Define a set of multivariate distributions. For hypernetworks Each edge in the array has an independent distribution variable, that is: ； in, Indicates hypernetwork The distribution variables corresponding to the distributions of each edge in the algorithm are independent, and are expressed as follows: ； in, Represents the first in the web search space The first on the edge The probability of the class operator; Indicates hypernetwork The Middle There may be on the edge. An operator, ; Then, a random relaxation method was used to adjust the objective function. Transforming it into a differentiable form, that is, into a probabilistic expectation form: ； in, Indicates hypernetwork The initial network probability; And the hardware constraints under the hardware model constraints are relaxed using a stochastic relaxation method. Transforming it into a differentiable form, that is, into a multivariate probability distribution form: 。 6. The neural architecture search method for chip design according to claim 5, characterized in that, The search of hypernetworks using a multivariable geometric information optimization algorithm under no hardware constraints. The optimal learnable parameters and optimal network probabilities, specifically including: An optimization algorithm utilizing multivariable geometric information is proposed under unconstrained hardware conditions, employing alternating gradient descent for hypernetworks. Optimize the learnable parameters and network probabilities; Conduct the first Next iteration optimization: ； ； in, Indicates the first Learnable parameters optimized in the next iteration Indicates the first The network probability after the next iteration of optimization; Indicates the first Learnable parameters optimized in the next iteration Indicates the first The network probability after the next iteration of optimization; This represents the learning rate for the learnable parameter; This represents the learning rate for network probabilities; Using the Monte Carlo method and To make an estimate, firstly... Hypernetwork sampling under a probability distribution, where the operator on each edge is obtained by sampling, and a total of samples are taken. Given a set of data, the gradient is estimated as follows: ； ； in, This represents the network structure sample obtained by the i-th sampling. In each step of the alternating gradient descent formula, two backpropagations are performed. The first backpropagation fixes the network probabilities. The gradient is estimated using the Monte Carlo method. Then, gradient descent is used to update the learned parameters. Learnable parameters are obtained. Second fixed learnable parameters The gradient is estimated using the Monte Carlo method. Then use gradient descent to update the network probabilities. To obtain network probability ; When the objective function When the maximum value is reached, convergence is indicated, and optimization stops. The learnable parameters and network probabilities after this round of iteration are then used as the supernetwork. The optimal learnable parameters and optimal network probability are determined; otherwise, the next round of iterative optimization is initiated.

7. The neural architecture search method for chip design according to claim 6, characterized in that, In the complete knapsack problem, the optimal learnable parameters and optimal network probabilities obtained from the initial search are then searched again using dynamic programming. The iterative path obtained by dynamic programming is represented as follows: ； in, This indicates calculation from the front, front The maximum cumulative network probability when the edge availability performance is e; e is the remaining availability performance, which can be expressed as: When the reasoning reaches the last layer of the neural network structure, the search stops, and the obtained structure is the most efficient neural network structure for a specific chip.

Images