Search method and search apparatus

Abstract

The present disclosure discloses a search method and a search device, the method comprising: selecting a candidate neural network from a search space of a neural network; evaluating an algorithm of the candidate neural network to obtain a first evaluation result; evaluating a hardware index for executing the algorithm of the candidate neural network according to a given first hardware parameter to obtain a second evaluation result; and adjusting a search path in the search space of the neural network structure according to the first evaluation result and the second evaluation result. When the neural network structure is searched, the present disclosure not only evaluates the algorithm of the searched neural network, but also evaluates the hardware index for running the neural network, which is helpful for efficiently and accurately searching the neural network meeting the hardware requirements.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence algorithms, and in particular, to a search method and a search device of a neural network. BACKGROUND

[0002] The prior art uses a neural network search algorithm to automatically search for a suitable neural network, thereby saving algorithm design time. The neural network searched based on the automatic search algorithm usually needs to be deployed to hardware (i.e., the neural network is executed by hardware). However, the neural network searched based on the current automatic search algorithm is sometimes difficult to deploy to hardware (such as high deployment cost or high hardware power consumption). SUMMARY

[0003] Therefore, the present disclosure provides a search method and a search device to reduce the difficulty of deploying a neural network searched based on a neural network search algorithm to hardware.

[0004] In a first aspect, a search method is provided, the method comprising: selecting a candidate neural network from a search space of a neural network; evaluating an algorithm of the candidate neural network to obtain a first evaluation result; evaluating a hardware indicator for executing the algorithm of the candidate neural network according to a given first hardware parameter to obtain a second evaluation result; and adjusting a search path in the search space of the neural network structure according to the first evaluation result and the second evaluation result.

[0005] Optionally, the method further comprises, before the evaluating the hardware indicator for executing the algorithm of the candidate neural network according to the given first hardware parameter, selecting a candidate hardware parameter from a search space of hardware parameters; evaluating the hardware indicator for executing the algorithm of the candidate neural network according to the candidate hardware parameter to obtain a third evaluation result; and adjusting a search path in the search space of the hardware parameters according to the third evaluation result to obtain the first hardware parameter.

[0006] Optionally, the method further comprises that the evaluating the hardware indicator for executing the algorithm of the candidate neural network according to the given first hardware parameter comprises: determining a total computation amount of a computation layer of the candidate neural network according to a parameter of the computation layer; determining a utilization rate of a computation unit for executing the computation layer according to the parameter of the computation layer of the candidate neural network and the first hardware parameter; and determining a computation time required for executing the algorithm in the computation layer according to the total computation amount of the computation layer and the utilization rate.

[0007] Optionally, the method further comprises that the evaluating the hardware index of executing the algorithm of the candidate neural network according to the given first hardware parameter comprises: determining a total calculation amount of a calculation layer of the candidate neural network according to parameters of the calculation layer; and determining power consumption required for executing the algorithm of the calculation layer according to the total calculation amount of the calculation layer and power consumption of a hardware device for executing a unit calculation amount.

[0008] Optionally, the method further comprises that the evaluating the hardware index of executing the algorithm of the candidate neural network according to the given first hardware parameter comprises: determining hardware area required for executing the calculation layer according to a type of an operator contained in the candidate neural network and a number of calculation units for executing the operator.

[0009] Optionally, the first evaluation result is used to indicate algorithm accuracy of the candidate neural network.

[0010] Optionally, the second evaluation result is used to indicate hardware cost of executing the algorithm of the candidate neural network.

[0011] Optionally, the first hardware parameter comprises one or more of the following parameters: a number of calculation units, a working frequency of a hardware device, power consumption of the hardware device for executing a unit calculation amount, and parallelism of the hardware device per clock cycle.

[0012] In a second aspect, a searching device is provided, which comprises: a first obtaining module configured to select a candidate neural network from a search space of a neural network; a first evaluation module configured to evaluate an algorithm of the candidate neural network to obtain a first evaluation result; a second evaluation module configured to evaluate a hardware index of executing the algorithm of the candidate neural network according to a given first hardware parameter to obtain a second evaluation result; and a first adjusting module configured to adjust a search path in the search space of the neural network structure according to the first evaluation result and the second evaluation result.

[0013] Optionally, the device further comprises: a second obtaining module configured to select a candidate hardware parameter from a search space of hardware parameters; a third evaluation module configured to evaluate the hardware index of executing the algorithm of the candidate neural network according to the candidate hardware parameter to obtain a third evaluation result; and a second adjusting module configured to adjust a search path in the search space of the hardware parameters according to the third evaluation result to obtain the first hardware parameter.

[0014] Optionally, the apparatus further comprises that the second evaluation module is configured to determine a total amount of computation of the computation layer according to the parameters of the computation layer of the candidate neural network; determine a utilization of a computation unit performing the computation layer according to the parameters of the computation layer of the candidate neural network and the first hardware parameter; and determine a computation time required for performing the algorithm in the computation layer according to the total amount of computation of the computation layer and the utilization.

[0015] Optionally, the apparatus further comprises that the second evaluation module is configured to determine a total amount of computation of the computation layer according to the parameters of the computation layer of the candidate neural network; determine a total amount of computation of the computation layer according to the total amount of computation of the computation layer, and power consumption of a hardware device performing a unit amount of computation, and determine power consumption required for performing the algorithm in the computation layer.

[0016] Optionally, the apparatus further comprises that the second evaluation module is configured to determine hardware area required for performing the algorithm in the computation layer according to a type of operator contained in the candidate neural network and a number of computation units performing the operator.

[0017] Optionally, the apparatus further comprises that the first evaluation result is used to indicate algorithm accuracy of the candidate neural network.

[0018] Optionally, the apparatus further comprises that the second evaluation result is used to indicate hardware cost of performing the algorithm of the candidate neural network.

[0019] Optionally, the apparatus further comprises that the first hardware parameter comprises one or more of the following parameters: a number of computation units, a working frequency of a hardware device, power consumption of a hardware device performing a unit amount of computation, and parallelism of a hardware device per clock cycle.

[0020] In a third aspect, a neural network searching apparatus is provided, comprising a memory and a processor, the memory stores executable code, and the processor is configured to execute the executable code, and can implement the method in the first aspect.

[0021] In a fourth aspect, a computer readable storage medium is provided, which stores executable code, and when the executable code is executed, the method in the first aspect can be implemented.

[0022] In a fifth aspect, a computer program product is provided, which comprises executable code, and when the executable code is executed, the method in the first aspect can be implemented.

[0023] The technical solution provided in this disclosure not only evaluates the algorithm of the searched neural network when searching for neural network structure, but also evaluates the hardware performance of the running neural network, which helps to efficiently and accurately search for neural networks that meet the hardware requirements. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of a neural network search architecture provided in an embodiment of the present disclosure.

[0025] Figure 2 This is a schematic diagram of a neural network search framework provided in an embodiment of the present disclosure.

[0026] Figure 3 This is a schematic flowchart of a neural network search method provided in an embodiment of the present disclosure.

[0027] Figure 4 This is a schematic diagram of another neural network search framework provided in an embodiment of this disclosure.

[0028] Figure 5 This is a schematic flowchart of a hardware parameter search method provided in an embodiment of the present disclosure.

[0029] Figure 6 This is a schematic diagram of another neural network search framework provided in an embodiment of the present disclosure.

[0030] Figure 7 This is a schematic diagram of a convolution operation provided in an embodiment of the present disclosure.

[0031] Figure 8 This is a schematic diagram of the structure of a neural network search device provided in an embodiment of the present disclosure.

[0032] Figure 9 This is a schematic diagram of another neural network search device provided in an embodiment of the present disclosure. Detailed Implementation

[0033] The technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments.

[0034] Artificial intelligence (AI) is a theory, method, technology and application system for simulating, extending and expanding human intelligence by using a digital computer or a machine controlled by a digital computer to perceive the environment, acquire knowledge and use the knowledge to obtain the best results. Artificial intelligence is a branch of computer science and can be used to study the design principles and implementation methods of various intelligent machines to enable machines to have perception, reasoning and decision-making functions. Artificial intelligence is widely used in the fields of robots, natural language processing, computer vision and image processing.

[0035] Artificial intelligence can include machine learning, which can be an algorithm analysis model based on a statistical perspective, which can be divided into supervised learning models, unsupervised learning models and reinforcement learning models. Machine learning can be applied, for example, to the fields of robot navigation, marketing activity prediction and energy consumption prediction. Deep learning is a new research direction in the field of machine learning, and its motivation is to establish and simulate the neural network of the human brain for analysis and learning. Deep learning can be classified according to different algorithm models, and in some embodiments, deep learning can be divided into CNN convolutional neural network, RNN recurrent neural network and DNN deep neural network.

[0036] A neural network, also known as an artificial neural network, can be an algorithmic mathematical model that simulates the behavior characteristics of an animal neural network for distributed parallel information processing. A neural network model can be composed of neurons and the interconnection relationship between neurons, and the neural network model can adjust the connection between neurons to achieve the purpose of processing information and has the ability of self-learning and self-adaptation.

[0037] The basic structure of a neural network can be composed of an input layer, a hidden layer and an output layer, wherein the input layer can accept input information, the input information is transmitted, analyzed and weighed in the neural unit to form output information, and the output information can be output by the output layer.

[0038] CNN convolutional neural network is one of the representative algorithms in deep learning, and the convolutional neural network will be introduced below. CNN convolutional neural network is a neural network with convolutional structure, in which the convolutional layer and the pool sampling layer in the hidden layer are the core modules for realizing the feature extraction function of the convolutional neural network. The structure of the convolutional neural network can be composed of a convolutional layer, an activation function and a pooling layer.

[0039] The convolution layer can be a neural unit in the convolution neural network that performs convolution calculation on the input signal. The convolution calculation can be to sweep the original data with a fixed-size rectangular region, divide the original data into small blocks of the same size as the rectangular region, and then multiply the small blocks and the rectangular region to output a convolution value. The rectangular region can also be referred to as a convolution kernel. The activation function can be used to add a non-linear factor to the neural network, so that the neural network can better solve more complex problems. The types and quantities of activation functions in the neural network can be flexibly matched as needed. In some embodiments, the neural network can have sigmoid functions, Relu functions, and tanh functions. The pooling layer can simplify the complexity of neural network calculation by extracting main features. It should be understood that the network structure described above is an example of a convolution neural network, and the present disclosure does not limit the structure of the convolution neural network.

[0040] The process of establishing a neural network can be a process of designing a neural network structure and determining parameters of the network structure. The parameter tuning process of the neural network is a huge project, and therefore, the structure design of the neural network can be implemented by means of neural network structure search to automatically optimize the parameters. The neural network structure search can automatically search for a neural network that meets specific constraints in a specific data set defined by a user. The neural network search can be applied on an automatic machine learning platform, which can be an AutoML platform. The automatic machine learning platform can help users obtain a machine learning model that meets the application scenario through transfer learning and automatic hyperparameter optimization technology, reduce repeated experiments, and improve modeling efficiency.

[0041] Figure 1 A neural network structure search system framework diagram is provided for embodiments of the present disclosure. The neural network search framework 100 includes a user device 110 and a neural network search platform 120. The user device 100 can be a smart terminal including a user's mobile phone, computer, etc. The user device is the initiator of the neural network search, and the user can initiate a neural network search request through the user device 110. The neural network search platform 120 can receive the search request initiated by the user device 110 and return the search result to the user device 110. The neural network search system 120 can include a server, which can be a physical server or a cloud server. In this case, the user can perform neural network search on the server, and the searched neural network can be deployed on the user device and applied. The neural network search platform 120 can also include a terminal, which can be the user device 100. The user can also search for a neural network on the user device 110.

[0042] The search for a neural network can include defining the search space, executing a search strategy, and evaluating network performance. The search space can be a candidate set of network structures to be searched. In some embodiments, users can define the candidate set of neural networks using a neural network search platform, such as the automated machine learning platform mentioned earlier. Defining the search space can involve defining the topology of the neural network, including defining the number and type of computational layers. For example, it may be necessary to define the type of hidden layers, which can be convolution, pooling, activation functions, etc., as mentioned earlier. It also includes defining the hyperparameters within each layer, such as the size and number of convolutional kernels in convolutional layers, the type and parameters of activation functions in activation function layers, and the size of pooling kernels in pooling layers.

[0043] The search strategy can be used to define how to find the optimal neural network. The search strategy can be a reinforcement learning-based method, an evolutionary algorithm-based method, or a gradient-based method. This disclosure does not limit the search method for the neural network.

[0044] Neural algorithm network performance evaluation can be used to assess the performance of the searched network structure. In some embodiments, the performance of the network structure can be evaluated by training the neural network algorithm with training data and verifying it with test data. The performance of the algorithm network can be, for example, the accuracy of the algorithm and the computation time of the algorithm.

[0045] Figure 2 A schematic diagram of a neural network search framework is provided. As shown in Figure 2, before performing an algorithm search, the user needs to define a search space for the artificial intelligence algorithm. As an example, the user can use... Figure 1 User device 110, such as a user's computer, defines the search space for the algorithm. The search algorithm selects a candidate network structure from the search space, represented by a hollow structure in the diagram. It then trains the candidate network structure using a training dataset to obtain a trained candidate neural network, represented by a solid structure in the diagram. The trained candidate network structure can then be validated using a validation dataset to obtain the test accuracy of the candidate algorithm on the validation dataset, thus measuring the performance of the current algorithm network. A loss function, such as a precision loss function, can also be introduced during accuracy evaluation to measure the difference between the algorithm's estimated accuracy and the target value. Figure 2In the neural network search framework shown, the accuracy information of the candidate neural network can be fed back to the search engine, and the search engine can plan a search strategy according to the accuracy information of the candidate algorithm network, so that the search engine samples a more suitable neural network algorithm structure in the search space. The search engine can sample according to the path when sampling in the search space, so the search strategy can be adjusted by adjusting the sampling path of the search engine in the search space based on the accuracy of the candidate neural network algorithm.

[0046] The searched neural network algorithm can be executed by a neural network processor, which can be a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), a field programmable gate array (FPGA), or other hardware acceleration chips.

[0047] The neural network processor can include a convolution processing module, a vector processing module, and a storage module. The convolution processing module can include a multiply-accumulate array to complete convolution operations; the vector processing module can be used to complete data processing such as pooling, size scaling, and image feature splicing; and the storage module can be used to store and output calculation data.

[0048] The calculation layer in the neural network can also be referred to as an operator layer, for example, the convolution calculation layer can be referred to as a convolution operator layer. During the execution of the neural network algorithm, the calculation of the operator layer in the neural network model can be completed by a calculation engine in the processor. The calculation engine can be a processing unit that completes a type of arithmetic logic calculation, for example, a multiply-accumulate calculation unit matrix arrangement can be used to obtain a convolution operation calculation engine. The operator layer with consistent calculation logic in the neural network can be completed by a calculation engine. For example, a convolution operation engine can be used to complete the convolution operation in the neural network algorithm, and a pooling calculation engine can be used to complete the pooling operation in the neural network algorithm.

[0049] The neural network can evaluate hardware indicators of a processor executing the algorithm when the neural network is deployed in an application to determine whether the algorithm is suitable for execution by the processor. The hardware indicators of the processor can be, for example, efficiency, power consumption, performance, area, scalability, heat dissipation, and computing performance of the processor executing the algorithm. In different application deployment scenarios of the neural network, the focus of the evaluation of the hardware indicators of the processor executing the neural network can be different. As an example, for a neural network deployed in the cloud, the evaluation can be performed from the aspects of computing performance and scalability of the processor; for a neural network deployed in a terminal, the evaluation can be performed from the aspects of power consumption and latency of the processor.

[0050] In the prior art, the search of the neural network is performed according to the performance of the candidate neural network algorithm to determine the target algorithm, without considering the hardware indicators of the hardware executing the candidate neural network, which leads to the problem of deployment difficulty of the searched neural network algorithm when the neural network is deployed in an application. For example, only the accuracy requirement of the network is considered when the neural network algorithm is searched, however, there is no processor in the prior art that can be used to execute the high-accuracy algorithm, and it is necessary to develop a processor that can be used to execute the algorithm, which requires a high development cycle and development cost. For another example, the existing processor can have the problems of low computing efficiency and high computing power consumption when used to execute the searched algorithm.

[0051] To solve the above problems, the present disclosure provides a neural network search method and a search device, which evaluate not only the searched neural network algorithm but also the hardware indicators of the hardware running the neural network when searching for the neural network structure, which helps to efficiently and accurately search for a neural network meeting the hardware requirements.

[0052] The neural network search method provided by the embodiments of the present disclosure is applicable to the search of any neural network, which can be a CNN convolutional neural network or an RNN recurrent neural network, and the present disclosure does not limit the type of neural network.

[0053] Figure 3 The neural network search method provided by the embodiments of the present disclosure is applicable to the search of any neural network, which can be a CNN convolutional neural network or an RNN recurrent neural network, and the present disclosure does not limit the type of neural network. Figure 1 The neural network search platform 120 in the server 100, for example, can be run in a server, and for example, can also be run on a terminal device.

[0054] In step S310, a candidate neural network is selected.

[0055] The candidate neural network can be a neural network sampled from a search space of neural networks. For example, the search engine can sample the candidate neural network from the search space according to the search method provided in Embodiment 2 of the present disclosure. The number of candidate neural networks can be one or more. For example, the search engine can sample one or more candidate neural networks from the search space.

[0056] In step S320, the algorithm of the candidate neural network is evaluated to obtain a first evaluation result.

[0057] The algorithm of the candidate neural network is evaluated, for example, the accuracy of the algorithm of the candidate neural network can be evaluated, and the calculation time consumption of the algorithm of the candidate neural network can be evaluated. In some embodiments, the accuracy of the candidate neural network can be evaluated according to the evaluation method of the accuracy of the neural network algorithm provided in Embodiment 2 of the present disclosure. The first evaluation result can be used to indicate the performance of the neural network algorithm, for example, the first evaluation result can be used to indicate the accuracy performance of the neural network algorithm. When there are multiple candidate neural networks, the algorithms of the multiple candidate neural networks can be evaluated, and the first evaluation results of the algorithms of the multiple candidate neural networks can be obtained.

[0058] In step S330, the hardware indicators for executing the algorithm of the candidate neural network are evaluated according to the given first hardware parameters to obtain a second evaluation result.

[0059] The first hardware parameters can include the number of calculation units, the working frequency of the hardware device, the power consumption of the hardware device for executing a unit calculation, and the parallelism of the hardware device per clock cycle. The number of calculation units can be, for example, the number of multiply-accumulators in a convolution operation module. The working frequency of the hardware device can be, for example, the working frequency of a processor, for example, the number of operations per second. The power consumption of the hardware device for executing a unit calculation can be, for example, the energy consumed per operation (Engery / OP). The parallelism of the hardware device per clock cycle can be the parallelism information in each dimension of the input channel, the output channel, the feature image window, the convolution kernel size, and the like per clock cycle.

[0060] The first hardware parameters can be input by a user, for example, the user can input the first hardware parameters. The user can input the first hardware parameters through a terminal device and transmit the first hardware parameters to a server running a search platform through a network. The user can also input the first hardware parameters through a terminal device, and the terminal device runs a neural network search platform. The first hardware parameters can also be obtained from a hardware parameter search platform. The way of obtaining the hardware parameters from the hardware parameter search platform will be described in detail below.

[0061] The first hardware parameter can be multiple or one. As an example, one candidate neural network algorithm and multiple hardware parameters can be selected, the multiple hardware parameters for executing the neural network algorithm are evaluated, and multiple second evaluation results are obtained. As another example, multiple candidate neural network algorithms and one hardware parameter can be selected, the hardware indicators of the multiple algorithms executed by the hardware parameter are evaluated, and multiple second evaluation results are obtained. As another example, one neural network algorithm and one first hardware parameter can also be selected, and one second evaluation result is obtained by evaluation.

[0062] The hardware indicators for executing the neural network algorithm can be evaluated based on the first hardware parameter. The hardware indicators can be, for example, hardware indicators such as efficiency, power consumption, performance, and area of executing the algorithm, and can also be hardware indicators such as scalability, heat dissipation effect, and computing performance.

[0063] In step S340, the search path in the search space of the neural network structure is adjusted according to the first evaluation result and the second evaluation result.

[0064] The search path in the search space of the neural network structure is adjusted according to the first evaluation result such as the accuracy of the neural network algorithm and the second evaluation result such as the hardware indicators such as efficiency, power consumption, performance, and area of executing the algorithm. In some embodiments, the first evaluation result and the second evaluation result can be summarized to guide the sampling path of the search engine in the search space.

[0065] When there are multiple second evaluation results, one second evaluation result can be selected as needed, and the search path in the search space of the neural network structure is adjusted based on the second evaluation result and the first evaluation result of the neural network algorithm corresponding to the second evaluation result.

[0066] As can be seen, the neural network search method provided by the embodiments of the present disclosure not only evaluates the algorithm of the searched neural network, but also evaluates the hardware indicators of running the neural network, which helps to efficiently and accurately search the neural network that meets the hardware requirements.

[0067] Figure 4 A neural network search framework provided by the embodiments of the present disclosure is shown in a schematic diagram, from Figure 4As can be seen, after the search engine samples the candidate neural network algorithm, the neural network algorithm can be sent to two evaluation paths at the same time. The upper path represents the precision effect evaluation of the candidate neural network, which first uses the training data set to train the network structure and uses the verification data set to evaluate to obtain a first evaluation result. The first evaluation result can be the precision of the candidate neural network algorithm. The lower path is used to evaluate the hardware indicators for executing the neural network algorithm based on the given hardware parameters to obtain a second evaluation result. The hardware indicator evaluation can be hardware area, energy consumption, and performance parameter evaluation.

[0068] The hardware area can be, for example, the hardware area required for the calculation layer to execute the neural network algorithm. The energy consumption can be, for example, the energy consumed by the calculation layer to execute the neural network algorithm. The hardware performance can be, for example, the calculation performance required to execute the neural network algorithm. The calculation performance can be, for example, the calculation time required for the calculation layer to execute the neural network algorithm.

[0069] From Figure 4 As can be seen from the framework schematic diagram shown in the figure, the first evaluation result of the neural network algorithm precision output by the upper path and the second evaluation result of the hardware indicators for executing the algorithm output by the lower path can be summarized to guide the search engine in the sampling path of the search sample space.

[0070] Before evaluating the execution of the neural network algorithm according to the given first hardware parameter, the first hardware parameter can also be obtained from the search space of the hardware parameters. The first hardware parameter can be the optimal hardware parameter obtained from the hardware parameter search space by the search algorithm. Figure 5 A hardware parameter search method provided by the embodiment of the present disclosure can be executed by a neural network algorithm search platform.

[0071] In step S510, a candidate hardware parameter is selected from the search space of the hardware parameters.

[0072] The search space of the hardware parameters is a set of candidate hardware parameters to be searched defined by the user. The search algorithm can sample the hardware parameters from the search space.

[0073] In step S520, the hardware indicators for executing the candidate neural network algorithm are evaluated according to the candidate hardware parameter to obtain a third evaluation result.

[0074] Before performing this step, a candidate neural network algorithm can also be selected from the search space of the neural network algorithms. Based on the candidate neural network algorithm and the candidate hardware parameter, the hardware indicators for executing the candidate neural network algorithm are evaluated to obtain the third evaluation result.

[0075] At step S530, according to the third evaluation result, the search path in the search space of the hardware parameter is adjusted to obtain the first hardware parameter.

[0076] Figure 6 A neural network search framework schematic diagram provided by the embodiment of the present disclosure is shown in FIG. 5. The neural network search framework provided by the embodiment of the present disclosure includes a search engine, a hardware parameter search space, and an evaluation path. Figure 6 As can be seen, after the search engine samples the candidate neural network algorithm A, the neural network algorithm A can be sent to the evaluation path on the top, which represents the precision effect evaluation of the candidate neural network. The network structure is first trained using the training data set, and the verification data set is used for evaluation to obtain the first evaluation result, which can be the precision of the candidate neural network algorithm.

[0077] Before the neural network algorithm A is sent to the evaluation path on the bottom, the candidate hardware parameter a can be searched in the hardware parameter search space by the search algorithm. According to the candidate neural network algorithm A and the candidate hardware parameter a, the hardware indicators for executing the neural network algorithm A are evaluated to obtain the third evaluation result Aa. The hardware indicators can be, for example, the efficiency, power consumption, performance, and area of the hardware indicators for executing the algorithm, or can be the scalability, heat dissipation effect, and computing performance of the hardware indicators. The third evaluation result Aa can be used to guide the search path of the search engine in the hardware parameter search space to search for the hardware parameter b that meets the requirements of the algorithm A. The hardware parameter b can be the optimal hardware parameter in the search space of the hardware parameter.

[0078] According to the hardware parameter b, the hardware parameter for executing the neural network algorithm A can be evaluated to obtain the second evaluation result Ab. The first evaluation result of the neural network algorithm precision A output by the path on the top and the second evaluation result Ab of the hardware indicators for executing the algorithm output by the path on the bottom can be summarized to guide the search engine in the sampling path of the search sample space to further optimize the network structure and parameters of the neural network algorithm A.

[0079] As can be seen, the neural network search method provided by the embodiment of the present disclosure can evaluate the hardware indicators for executing the neural network algorithm based on the given hardware parameter to obtain the neural network structure that meets the hardware requirements. The hardware parameter can also be obtained from the search space of the hardware parameter, and based on the selected neural network algorithm and the hardware parameter, the hardware indicators for executing the algorithm are evaluated, and the adjustment of the search path in the hardware parameter search space is guided to obtain the hardware parameter that meets the requirements of the neural network algorithm. At the same time, the neural network search method provided by the embodiment of the present disclosure can also optimize the network structure and parameters of the neural network algorithm based on the optimal hardware parameter obtained from the hardware search space.

[0080] The neural network algorithm has a huge amount of calculation, and therefore, the calculation performance is an important indicator for measuring the processor of the neural network algorithm. The calculation performance of the execution candidate neural network is evaluated using the given hardware parameters, and it can be determined whether the hardware parameters meet the calculation requirement of the candidate neural network. That is, the calculation performance of the execution neural network algorithm can be evaluated, and the second evaluation result can be the evaluation result of the calculation performance. The calculation performance can be the calculation time of the hardware executing the neural network algorithm, or the calculation amount completed by the hardware per unit time, for example, it can be the number of operations per second (TOPS, Tera Operations Per Second) that can be calculated by the hardware.

[0081] The calculation time can be the time required for the processor to complete a unit of calculation, and the calculation time can reflect the data processing capability of the hardware, for example, the time required to complete a unit of calculation is short, which reflects the data processing capability of the hardware, and therefore, the calculation time of the execution neural network calculation layer can be used as a hardware evaluation indicator.

[0082] In some embodiments, the calculation time of one or more calculation layers can be used as the first hardware evaluation indicator. For example, the calculation time of each operator layer such as the convolution layer, the pooling layer, and the activation function can be used as the hardware evaluation indicator of the algorithm, and for example, the calculation time of the convolution layer can also be used as the hardware evaluation indicator of the algorithm, because the operation amount of the convolution operation in the neural network algorithm accounts for more than 80% of the entire algorithm operation amount, and therefore, it is reasonable to use the calculation time of the convolution layer as the hardware evaluation indicator of the algorithm.

[0083] The following is a specific embodiment for determining the calculation time of the neural network algorithm based on the first hardware parameters, which is combined with Table 1.

[0084]

[0085] Table 1 provides a 5-layer structure of a neural network, which is CONV0, CONV1, CONV2, CONV3, CONV4, and pooling, wherein CONV0 is a convolution layer, and pooling is a pooling layer.

[0086] Figure 7 A convolution operation diagram is provided for the embodiments of the present disclosure, which is combined with Figure 7 The parameters of each calculation layer in Table 1 are introduced. Figure 7 The convolution kernel (Kernel) is shown in the lower left corner, and K can represent the size of the kernel convolution kernel, for example, the size of the convolution kernel can be 3*3, 1*1, 5*5, Figure 7 The input feature image (input featuremap) is shown in the upper left corner, and IH, IW, and IC can be used to represent the size of the input feature image in three dimensions,Figure 7 The output feature map is shown in the middle right. OH, OW, and OC can be used to represent the size of the output feature map in three dimensions.

[0087] OPs can represent the total computation of a computation layer. In some embodiments, the total computation of a computation layer can be calculated by a function. The calculation function can be shown as follows.

[0088] OPs = K * K * OW * OH * OC * IC * 2

[0089] K * K * OW * OH * OC * IC can be used to represent the total number of multiplications in a convolution computation layer. Since the total computation in a convolution computation layer includes both multiplications and additions, K * K * OW * OH * OC * IC * 2 can be used to represent the total computation of a convolution computation layer. The total computation of each computation layer in the 5-layer neural network provided in Table 1 can be calculated respectively.

[0090] The computation time of a computation layer can be obtained from the total computation of the computation layer and the computation amount that can be completed by hardware in a unit of time. The computation amount that can be completed by hardware in a unit of time can be obtained from hardware parameters, such as the hardware working frequency, the parallelism in each clock cycle in the input channel, the output channel, the feature map window, the convolution kernel size, and other dimensions.

[0091] In a convolution operation, the utilization of the computation unit of the convolution computation layer can be different due to the placement of the computation unit in the hardware, the computation method, and the input channel and the output channel in the neural network algorithm. Therefore, in a convolution operation, the computation amount that can be completed by hardware in a unit of time can be calculated from the number of computation units in the hardware, the utilization of the computation unit, and the working frequency of the hardware.

[0092] The computation unit can be the number of multipliers (MAC_number) in the hardware. The utilization of the computation unit can be the average multiplier utilization (MAC_Utilization) when executing the computation layer based on the first hardware parameter and the convolution computation layer parameter under the current hardware parameter. The working frequency (Frequency) of the hardware can be, for example, the number of times of processing data signals by the hardware in a unit of time. Therefore, for a convolution computation layer, the computation time of the computation layer can be obtained according to the following calculation function.

[0093] Time = OPs / (MAC_number * 2 * Frequency * Mac_utilization)

[0094] As described in Table 1, the calculation time of each of the five calculation layers can be calculated, and the total calculation time of the neural network algorithm can be obtained by aggregating the calculation time.

[0095] In some embodiments, the calculation time of the calculation layer can be used as a hardware evaluation index of the neural network algorithm, so that the sampling path in the neural network search space can be adjusted according to the calculation time of the calculation layer and the accuracy of the neural network algorithm, to search for a neural network algorithm that meets both the accuracy requirement and the calculation time requirement.

[0096] For the hardware that executes the neural network algorithm, the power consumption of the hardware should also be considered. The power consumption is one of the main indicators for measuring the performance of an algorithm processor, and therefore, the hardware power consumption evaluation of the neural network algorithm can be performed, that is, the second evaluation result can be a hardware power consumption evaluation result. The power consumption can include the power consumption of the calculation unit in the hardware, and can also include the power consumption of, for example, the memory.

[0097] The power consumption required by the algorithm for executing the calculation layer of the neural network algorithm can be determined according to the total calculation amount of the calculation layer and the power consumption of the hardware device for executing a unit calculation amount.

[0098] The total calculation amount of the calculation layer, for example, can be OPs, which can be calculated by the calculation formula mentioned above, and will not be described here. The power consumption of the hardware device for executing a unit calculation amount, for example, can be the energy consumed by each operation (Engery / OP). The total calculation amount OPs of the calculation layer is multiplied by Engery / OP, and the energy consumed for executing the current calculation layer can be obtained.

[0099] The hardware area restricts the manufacturability, yield, heat dissipation, and calculation performance of the neural network processor, and therefore, the hardware area for executing the algorithm should be evaluated in searching for the neural network algorithm, that is, the second evaluation result can be an evaluation result of the hardware area for executing the algorithm. In some embodiments, the sampling path of the algorithm search space can be adjusted according to the evaluation result of the hardware area for executing the candidate neural network algorithm and the evaluation result of the accuracy of the algorithm, to obtain a neural network algorithm that meets both the hardware area requirement and the accuracy requirement.

[0100] In some embodiments, the hardware area required by the algorithm for executing the calculation layer can be determined according to the type of operator included in the candidate neural network and the number of calculation units for executing the operator.

[0101] The following embodiment for determining the hardware area required by the algorithm for executing the calculation layer of the candidate neural network is given in combination with Table 2.

[0102]

[0103] The neural network provided in Table 2 includes five types of operators, namely, a convolution operator layer (Convolution), a pooling operator layer (Pooling), a resize operator layer (Resize), and three types of activation function operator layers, namely, a Relu operator layer, a sigmoid operator layer, and a tanh operator layer. The number of calculation units in Table 2 can be the number of calculation units included in the calculation engine that executes the operator, for example, the number of multipliers in the calculation engine that executes the convolution operator layer operation.

[0104] Table 2 provides two different hardware parameters, and the number of calculation units of the calculation engine in the two different hardware parameters is different. For example, in the first parameter, the number of calculation units in the calculation engine of the convolution operator layer is N0, for example, N0 is 4K. In the second parameter, the number of calculation units in the calculation engine of the convolution operator layer is N1, for example, N1 is 8K, and the area of the convolution engine with 4K multipliers is different from the area of the convolution engine with 8K multipliers. Similarly, the same applies to the pooling operator layer and the resize operator layer.

[0105] The hardware area required to execute the activation function operator layer algorithm is related to the area of the calculation engine of the activation function. The area of the calculation engine of the activation function can be a fixed value, and different types of activation functions correspond to different areas of the calculation engine. In some embodiments, the area of the calculation engine of the activation function can be summarized in a table, and the hardware area required to execute the activation function can be obtained by looking up the table according to the type of the activation function. As can be seen from Table 2, the implementation of the activation function does not use calculation units, so the hardware area required to execute the activation function is consistent for different hardware parameters.

[0106] The types of operators in the neural network algorithm are not limited to those shown in Table 2. The more types of operator layers in the algorithm network, the more types of calculation engines that execute the corresponding operator layer algorithms, and the more hardware area required to execute the neural network algorithm calculation layer algorithm.

[0107] When performing hardware evaluation on hardware that executes the candidate neural network algorithm, hardware cost evaluation should also be considered. Hardware cost evaluation can be used to evaluate the implementation cost of hardware that implements the execution of the candidate neural network, for example, the implementation cost of an intelligent acceleration chip that executes the candidate neural network algorithm. Hardware implementation cost evaluation can include hardware research and development cost evaluation, hardware manufacturing cost evaluation, and hardware operation and maintenance cost evaluation, etc. In this way, when searching for a neural network, the search path of the search algorithm is adjusted simultaneously according to the performance evaluation result of the neural network algorithm and the hardware implementation cost of executing the algorithm, so as to search for a neural network that meets the hardware implementation cost demand.

[0108] In some embodiments, the hardware evaluation result of the execution of the candidate neural network according to the first hardware parameter, i.e., the second evaluation result mentioned above, can be used to indicate the hardware cost of the candidate neural network algorithm. For example, the second evaluation result is a hardware area evaluation, and the hardware area has a direct impact on the hardware cost, and a large hardware area will result in a large hardware cost. For another example, the second evaluation result is a hardware power consumption evaluation, and the power consumption required for the execution of the candidate neural network algorithm is large, which indicates that the power consumption of the execution of the algorithm using the given hardware parameter is large, resulting in an increase in the operation and maintenance cost.

[0109] In some embodiments, the hardware cost of the candidate neural network can be evaluated using a hardware cost evaluation function of the algorithm, and the hardware cost function is related to four indexes of the accuracy of the neural network algorithm, the hardware performance of the execution of the neural network algorithm, the hardware power consumption of the execution of the neural network algorithm, and the hardware area of the execution of the neural network algorithm. In some embodiments, the candidate neural network algorithm and the candidate hardware parameter can be input into the hardware cost evaluation function, and the hardware cost evaluation function can output the hardware implementation cost of the candidate neural network based on the hardware parameter.

[0110] In some embodiments, each measurement index in the hardware cost function can be assigned a weight, and the weight of each measurement index in the hardware cost function can be changed. For example, when the accuracy requirement of the neural network algorithm is high, the weight of the accuracy can be adjusted to be higher; for another example, when the hardware power consumption requirement for the execution of the neural network algorithm is strict, the weight of the hardware power consumption can be adjusted to be higher.

[0111] The method embodiments of the present disclosure are described in detail above Figures 1 to 7 , and the device embodiments of the present disclosure are described in detail below Figure 8 and Figure 9 . It should be understood that the description of the method embodiments corresponds to the description of the device embodiments, and therefore, the parts not described in detail can be referred to the foregoing method embodiments.

[0112] Figure 8 is a structural schematic diagram of a neural network search device provided by an embodiment of the present disclosure. Figure 8 The neural network search device 800 shown in the figure can include a first acquisition module 810, a first evaluation module 820, a second evaluation module 830, and a first adjustment module 840.

[0113] The first obtaining module 810 can be configured to select a candidate neural network from a search space of a neural network, the first evaluation module 820 can be configured to evaluate an algorithm of the candidate neural network to obtain a first evaluation result, the second evaluation module 830 can be configured to evaluate a hardware indicator of executing the algorithm of the candidate neural network according to a given first hardware parameter to obtain a second evaluation result, and the first adjustment module 840 can be configured to adjust a search path in the search space of the neural network structure according to the first evaluation result and the second evaluation result.

[0114] Optionally, the neural network searching apparatus 800 can further include a second obtaining module configured to select a candidate hardware parameter from a search space of a hardware parameter, a third evaluation module configured to evaluate a hardware indicator of executing an algorithm of the candidate neural network according to the candidate hardware parameter to obtain a third evaluation result, and a second adjustment module configured to adjust a search path in the search space of the hardware parameter according to the third evaluation result to obtain the first hardware parameter.

[0115] Optionally, the second evaluation module can be configured to determine a total computation amount of a computation layer of the candidate neural network according to parameters of the computation layer, determine a utilization rate of a computation unit executing an algorithm of the computation layer according to the parameters of the computation layer and the first hardware parameter, and determine a computation time required for executing the algorithm in the computation layer according to the total computation amount of the computation layer and the utilization rate.

[0116] Optionally, the second evaluation module can be configured to determine a total computation amount of a computation layer of the candidate neural network according to parameters of the computation layer, and determine a power consumption required for executing an algorithm of the computation layer according to the total computation amount of the computation layer and a power consumption of a hardware device executing a unit computation amount.

[0117] Optionally, the second evaluation module can be configured to determine a hardware area required for executing an algorithm of the computation layer according to a type of an operator included in the candidate neural network and a number of computation units executing the operator.

[0118] Optionally, the first evaluation result is used to indicate an algorithm accuracy of the candidate neural network.

[0119] Optionally, the second evaluation result is used to indicate a hardware cost of executing the algorithm of the candidate neural network.

[0120] Optionally, the first hardware parameter includes one or more of the following parameters: a number of computation units, a working frequency of a hardware device, a power consumption of a hardware device executing a unit computation amount, and a parallelism of a hardware device per cycle.

[0121] Figure 9 is a structural schematic diagram of another neural network searching apparatus provided by an embodiment of the present disclosure. Figure 8 The neural network searching apparatus 900 described above can include a memory 910 and a processor 920. The memory 910 can be used to store executable codes. The processor 920 can be used to execute the executable codes stored in the memory 910 to implement the steps in the various methods described above. In some embodiments, the apparatus 900 can further include a network interface 930, and the data exchange between the processor 920 and external devices can be realized through the network interface 930.

[0122] In the above embodiments, all or part of the embodiments can be realized by software, hardware, firmware or any combination thereof. When realized by software, all or part of the embodiments can be realized in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed by a computer, all or part of the computer program instructions generate the processes or functions described in the embodiments of the present disclosure. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as digital video disc (Digital Video Disc, DVD)), or semiconductor media (such as solid state disk (Solid State Disk, SSD)) and the like.

[0123] Those of ordinary skill in the art can be aware that units and algorithm steps of each example described in combination with the embodiments of the present disclosure can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present disclosure.

[0124] In several embodiments provided by the present disclosure, it should be understood that the disclosed system, apparatus and method can be implemented in other manners. For example, the embodiments of the apparatus described above are merely schematic, and the division of the units is merely logical function division, and there can be other division manners in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0125] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0126] In addition, each functional unit in the various embodiments of the present disclosure can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0127] The above description is merely a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present disclosure, which should be covered by the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A search method characterized by, The method comprises: selecting a candidate neural network from a search space of neural networks; evaluating an algorithm of the candidate neural network to obtain a first evaluation result; evaluating a hardware indicator of executing the algorithm of the candidate neural network according to a given first hardware parameter to obtain a second evaluation result; adjusting a search path in the search space of the neural network structure according to the first evaluation result and the second evaluation result; Before the step of evaluating the hardware indicator of executing the algorithm of the candidate neural network according to the given first hardware parameter, the method further comprises: selecting a candidate hardware parameter from a search space of hardware parameters; evaluating the hardware indicator of executing the algorithm of the candidate neural network according to the candidate hardware parameter to obtain a third evaluation result; adjusting a search path in the search space of the hardware parameter according to the third evaluation result to obtain the first hardware parameter, which is an optimal hardware parameter for the candidate neural network obtained in the search space of the hardware parameter.

2. The method of claim 1, wherein, The step of evaluating the hardware indicator of executing the algorithm of the candidate neural network according to the given first hardware parameter comprises: determining a total calculation amount of a calculation layer of the candidate neural network according to parameters of the calculation layer; determining a utilization rate of a calculation unit executing the calculation layer according to the parameters of the calculation layer of the candidate neural network and the first hardware parameter; determining a calculation time required for executing the algorithm in the calculation layer according to the total calculation amount of the calculation layer and the utilization rate.

3. The method of claim 1, wherein, The step of evaluating the hardware indicator of executing the algorithm of the candidate neural network according to the given first hardware parameter comprises: determining a total calculation amount of a calculation layer of the candidate neural network according to parameters of the calculation layer; determining a power consumption required for executing the algorithm of the calculation layer according to the total calculation amount of the calculation layer and a power consumption of a hardware device executing a unit calculation amount.

4. The method of claim 1, wherein, The step of evaluating the hardware indicator of executing the algorithm of the candidate neural network according to the given first hardware parameter comprises: determining a hardware area required for executing a calculation layer according to a type of an operator included in the candidate neural network and a number of calculation units executing the operator.

5. The method of claim 1, wherein, The first evaluation result is used to indicate an algorithm accuracy of the candidate neural network.

6. The method of claim 1, wherein, The second evaluation result is used to indicate a hardware cost of executing the algorithm of the candidate neural network.

7. The method of claim 1, wherein, The first hardware parameter comprises one or more of the following parameters: a number of calculation units, a working frequency of a hardware device, a power consumption of a hardware device executing a unit calculation amount, and a parallelism of the hardware device per clock cycle.

8. A search apparatus characterized by comprising: The method comprises: a first obtaining module configured to select a candidate neural network from a search space of neural networks; a first evaluation module configured to evaluate an algorithm of the candidate neural network to obtain a first evaluation result; a second evaluation module configured to evaluate a hardware indicator of executing the algorithm of the candidate neural network according to a given first hardware parameter to obtain a second evaluation result; The first adjusting module is configured to adjust a search path in a search space of the neural network structure according to the first evaluation result and the second evaluation result. The device further comprises: The second obtaining module is configured to select a candidate hardware parameter from a search space of hardware parameters. The third evaluation module is configured to evaluate a hardware indicator of an algorithm for executing the candidate neural network according to the candidate hardware parameter, to obtain a third evaluation result. The second adjusting module is configured to adjust a search path in the search space of the hardware parameters according to the third evaluation result, to obtain the first hardware parameter, which is an optimal hardware parameter for the candidate neural network obtained in the search space of the hardware parameters.

9. The apparatus of claim 8, wherein, The second evaluation module is configured to determine a total calculation amount of a calculation layer of the candidate neural network according to parameters of the calculation layer; determine a utilization rate of a calculation unit for executing the calculation layer according to the parameters of the calculation layer of the candidate neural network and the first hardware parameter; and determine a calculation time required for executing an algorithm in the calculation layer according to the total calculation amount of the calculation layer and the utilization rate. The second evaluation module is configured to determine a total calculation amount of a calculation layer of the candidate neural network according to parameters of the calculation layer; determine a total calculation amount of a calculation layer of the candidate neural network according to parameters of the calculation layer; and determine a power consumption required for executing an algorithm in the calculation layer according to the total calculation amount of the calculation layer and power consumption of a hardware device for executing a unit calculation amount.

10. The apparatus of claim 8, wherein, The second evaluation module is configured to determine a hardware area required for executing an algorithm in the calculation layer according to a type of an operator included in the candidate neural network and a number of calculation units for executing the operator.

11. The apparatus of claim 8, wherein, The first evaluation result is used to indicate an algorithm accuracy of the candidate neural network.

12. The apparatus of claim 8, wherein, The second evaluation result is used to indicate a hardware cost of the algorithm for executing the candidate neural network.

13. The apparatus of claim 8, wherein, The first hardware parameter comprises one or more of the following parameters: a number of calculation units, a working frequency of a hardware device, power consumption of the hardware device for executing a unit calculation amount, and parallelism of the hardware device per clock cycle.

14. The apparatus of claim 8, wherein, 15. A search device comprising a memory and a processor, the memory storing executable code, and the processor being configured to execute the executable code to implement the method of any one of claims 1-7. ​

Images