Method, device, apparatus and storage medium for determining configuration of neural network model

Abstract

A method, device, equipment, and storage medium for determining the configuration of a neural network model, which queries prior information and common general-purpose information in a preset model cost evaluation table using dimensional information of a target network layer and hardware commands of a target hardware, and obtains a preset configuration that minimizes the number of clock cycles for each target network layer, i.e., the configuration of each target network layer when the target neural network model is executed to minimize the number of clock cycles, thereby obtaining an optimal configuration of the target neural network model on the target hardware, and then realizing rapid introduction of the neural network model to the target hardware through the optimal configuration.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to a Chinese patent application filed by the State Intellectual Property Office of the People's Republic of China on June 2, 2021, bearing application number 202110614422.8 and entitled "Method, Apparatus, Device and Storage Medium for Determining the Configuration of a Neural Network Model," the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the field of computer technology, and more particularly to a method, apparatus, device and storage medium for determining a configuration of a neural network model. [Background technology]

[0003] With the increasing development of computer technology, artificial intelligence technology has entered a stage of rapid development. Among various artificial intelligence technologies, the research and application of neural network models is the most widespread, and has gradually become the most representative artificial intelligence technology. Summary of the Invention [Problem to be solved by the invention]

[0004] In evaluating the performance of neural network models, attention is paid to aspects such as high throughput, low response time, low resource consumption, and simple deployment process. As neural network models become more complex, the amount of calculation also increases, and the hardware adapted to neural network models is increasingly updated and upgraded, for example, from the previous single hardware AI (Artificial Intelligence) acceleration unit to many-core acceleration. However, when new hardware is introduced, a problem often occurs that the performance of the same model is optimized for a long time on the previous hardware, and the performance of the model is optimized on the previous hardware, but after it is implemented on another type of hardware, the previously optimized performance cannot be expressed, and it is necessary to further optimize the performance for a long time. This is disadvantageous to the rapid introduction of neural network models.

[0005] The objective of the embodiments of the present application is to provide a method, device, apparatus and storage medium for determining the configuration of a neural network model, so as to realize the neural network model to be configured on different hardware, and the neural network model can be quickly deployed on a certain hardware through the configuration. The specific technical solutions are as follows: [Means for solving the problem]

[0006] In a first aspect, an embodiment of the present application provides a method for determining a configuration of a neural network model, the method comprising: Obtaining dimensional information of each target network layer in a target neural network model to be constructed and each target hardware command of a target hardware for executing the target neural network model; According to the dimension information of each of the target network layers and each of the target hardware commands, refer to the prior information and the general information in a preset model cost evaluation table, and execute each of the target network layers on the target hardware to determine a corresponding preset configuration when the number of clock cycles is shortest, thereby obtaining an optimal configuration of each of the target network layers in the target neural network model; The advance information includes an optimal configuration of each preset network layer in different preset test modes obtained in advance, the preset test modes represent the realization of the functions of each preset network layer by each preset hardware command, each preset network layer corresponds to corresponding dimensional information, the optimal configuration is a corresponding configuration when the number of clock cycles is shortest, and the general information includes the number of clock cycles required to execute each preset hardware command in different preset configurations.

[0007] In one possible embodiment, the method includes: querying prior information and generic information in a preset model cost evaluation table according to the dimension information of each of the target network layers and each of the target hardware commands, and respectively determining a corresponding preset configuration when each of the target network layers is executed on the target hardware to minimize the number of clock cycles, thereby obtaining an optimal configuration of each of the target network layers in the target neural network model; For each target network layer, refer to the pre-information of the preset model cost evaluation table according to the dimension information of the target network layer, and when a preset network layer corresponding to the target network layer is found, the optimal configuration of the found preset network layer is set as the optimal configuration of the target network layer; For each target network layer, if there is no preset network layer corresponding to the target network layer in the prior information of the preset model cost evaluation table, the following steps are included: determining each target hardware command required to realize the function of the target network layer as a hardware command group, consulting the general information of the preset model cost evaluation table, determining the number of clock cycles required to execute the hardware command group in each preset configuration, selecting the preset configuration with the shortest number of clock cycles, and determining it as the optimal configuration for the target network layer.

[0008] In one possible embodiment, the step of pre-constructing the preset model cost evaluation table includes: Obtaining a preset reference neural network model; For each reference network layer of the reference neural network model, determine a preset hardware command required to realize the function of the reference network layer with different hardware, execute each preset hardware command with different preset configurations, obtain the number of clock cycles required to execute each preset hardware command with different preset configurations, and record the number of clock cycles in a model cost evaluation table as general information; Obtaining a pre-configured dedicated neural network model; For each dedicated network layer of the dedicated neural network model, a function of the dedicated network layer is respectively realized by each preset hardware command in a different preset configuration, the number of clock cycles required to realize the function of the dedicated network layer in different preset test modes is obtained, and the preset configuration corresponding to the shortest number of clock cycles is set as an optimal configuration of the dedicated network layer, and the optimal configuration of the dedicated network layer in the different preset test modes is obtained, and recorded in a model cost evaluation table as prior information.

[0009] In one possible embodiment, the method comprises the steps of: generating a configuration file of the target neural network model according to an optimal configuration of each network layer in the target neural network model; The method further includes transmitting the configuration file and the target neural network model to the target hardware, such that the target hardware configures the target neural network model according to the configuration file and executes the configured target neural network model.

[0010] In a second aspect, an embodiment of the present application provides an apparatus for determining a configuration of a neural network model, the apparatus comprising: a dimension information acquisition module, which is used for acquiring dimension information of each target network layer in a target neural network model to be constructed and each target hardware command of a target hardware for executing the target neural network model; and an optimal configuration determination module, which is used to obtain an optimal configuration of each target network layer in the target neural network model by referring to prior information and generic information in a preset model cost evaluation table according to the dimension information of each target network layer and each target hardware command, and respectively determining a corresponding preset configuration when each target network layer is executed on the target hardware to minimize the number of clock cycles; The advance information includes an optimal configuration of each preset network layer in different preset test modes obtained in advance, the preset test modes represent the realization of the functions of each preset network layer by each preset hardware command, each preset network layer corresponds to corresponding dimensional information, the optimal configuration is a corresponding configuration when the number of clock cycles is shortest, and the general information includes the number of clock cycles required to execute each preset hardware command in different preset configurations.

[0011] In one possible embodiment, the optimal configuration determination module specifically queries the pre-information of a preset model cost evaluation table according to the dimension information of the target network layer for each target network layer. When a preset network layer corresponding to the target network layer is found, the optimal configuration of the found preset network layer is the optimal configuration of the target network layer; For each target network layer, if there is no preset network layer corresponding to the target network layer in the prior information of the preset model cost evaluation table, each target hardware command required to realize the function of the target network layer is determined as a hardware command group, the general information of the preset model cost evaluation table is consulted, the number of clock cycles required to execute the hardware command group in each preset configuration is determined, and the preset configuration with the shortest number of clock cycles is selected to be the optimal configuration for the target network layer.

[0012] In one possible embodiment, the device further includes a cost evaluation table construction module, which is used for: obtaining a preset reference neural network model; for each reference network layer of the reference neural network model, determining preset hardware commands required to realize the function of the reference network layer with different hardware, executing each preset hardware command with different preset configurations, respectively, obtaining the number of clock cycles required to execute each preset hardware command with different preset configurations, and recording it in the model cost evaluation table as general information; obtaining a preset dedicated neural network model; for each dedicated network layer of the dedicated neural network model, respectively, realizing the function of the dedicated network layer with each preset hardware command with different preset configurations, respectively, obtaining the number of clock cycles required to realize the function of the dedicated network layer in different preset test modes, and obtaining the preset configuration corresponding to the shortest number of clock cycles as the optimal configuration of the dedicated network layer, and recording it in the model cost evaluation table as advance information.

[0013] In one possible embodiment, the device comprises: a configuration file generating module, which is used to generate a configuration file of the target neural network model according to the optimal configuration of each network layer in the target neural network model; and a configuration file sending module, which is used to send the configuration file and the target neural network model to the target hardware, so that the target hardware configures the target neural network model according to the configuration file and executes the configured target neural network model.

[0014] In a third aspect, an embodiment of the present application provides an electronic device including a processor and a memory, The memory is used to record a computer program; The processor, when executing a program recorded in the memory, provides an electronic device that can be used to realize the method for determining a configuration of a neural network model described in any of the present applications.

[0015] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium having a computer program recorded thereon, the computer program being characterized in that, when executed by a processor, the computer program realises a method of determining a configuration of a neural network model as described in any of the present applications.

[0016] The advantageous effects of the embodiments of the present application are as follows: The method, device, equipment, and storage medium for determining the configuration of a neural network model provided in the present application obtain the dimension information of each target network layer in the target neural network model to be configured and each target hardware command of the target hardware for executing the target neural network model, and refer to the prior information and general information in the preset model cost evaluation table using the dimension information of each target network layer and each target hardware command, and determine the corresponding preset configuration when the number of clock cycles is shortest when each target network layer is executed on the target hardware, thereby obtaining the optimal configuration of each target network layer in the target neural network model. The prior information and common general information in the preset model cost evaluation table can be referred to using the dimension information of the target network layer and the hardware command of the target hardware, and the preset configuration with the shortest number of clock cycles for each target network layer, i.e., the configuration of each target network layer when the number of clock cycles executed by the target neural network model is smallest, thereby obtaining the optimal configuration of the target neural network model in the target hardware, and then using the optimal configuration, realizing rapid introduction of the neural network model to the target hardware. Of course, it is not necessary to achieve all of the above advantages at the same time when implementing any of the products and methods of the present application. [Brief description of the drawings]

[0017] In order to more clearly describe the technical solutions of the embodiments of the present application and the prior art, the drawings necessary for the embodiments and the prior art will be briefly described below. Of course, the drawings described below are only part of the embodiments of the present application, and those skilled in the art can derive other drawings based on these drawings without creative efforts. [Figure 1] FIG. 1 is a first schematic diagram of a method for determining a configuration of a neural network model according to an embodiment of the present application. [Diagram 2] FIG. 2 is a schematic diagram of adding a configuration file to a model's head in an embodiment of the present application. [Figure 3a] FIG. 3a is a second schematic diagram of the method for determining the configuration of a neural network model according to an embodiment of the present application. [Figure 3b] FIG. 3b is a schematic diagram of a method for determining the configuration of a network layer based on prior information according to an embodiment of the present application. [Figure 4] FIG. 4 is a schematic diagram of a process for constructing a preset model cost evaluation table according to an embodiment of the present application. [Diagram 5] FIG. 5 is a schematic diagram of a linked list according to an embodiment of the present application. [Figure 6] FIG. 6 is a third schematic diagram of the device for determining the configuration of a neural network model according to an embodiment of the present application. [Figure 7] FIG. 7 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in more detail below by way of examples with reference to the drawings. Of course, the described embodiments are only a part of the embodiments of the present application, and are not all of the embodiments. Based on the embodiments of the present application, all other embodiments that can be obtained by a person skilled in the art without requiring creative work fall within the scope of protection of the present application.

[0019] First, the terminology used in this application will be explained.

[0020] Regarding deep learning networks, various neural network models have been put forward in academia at present, such as perceptron, feedforward neural network, convolutional neural network, recurrent neural network, algorithm hardware mapping, etc. The differences between these different neural network models mainly lie in the activation rules of neurons, the topology configuration of the neural network model, and the parameter learning algorithm, etc. Common network types include convolutional neural networks, recurrent neural networks, deep belief networks, generative adversarial networks, deep reinforcement learning networks, etc.

[0021] Convolutional neural networks (CNNs) belong to one of the above-mentioned feedforward neural networks, and have unique effects on processing graphic images. The structure includes at least a convolutional layer and a pooling layer. Convolutional neural networks are a type of deep learning network that has been increasingly developed in recent years, and have attracted attention in academia and are widely used in enterprises. Representative convolutional neural networks include LeNet-5, VGG (Visual Geometry Group Network), AlexNet, etc. At present, convolutional neural networks are mainly applied to object detection and identification in video and video understanding.

[0022] Regarding LSTM (Long Short Term Memory), LSTM is a recurrent neural network. Unlike convolutional neural networks, recurrent neural networks are more suitable for processing language texts. Analysis processing of texts focuses more on the connection between input and context at the time. The internal memory structure of recurrent neural networks meets exactly such a demand scenario, so recurrent neural networks are superior for text processing. Currently, the main application scenario of recurrent neural networks is the field of NLP (Neuro-Linguistic Programming), such as machine translation and sentiment analysis. In particular, news writing robots, which have been widely reported in the media in recent years, are also an application based on recurrent neural networks. In addition, recurrent neural networks can realize automatic generation of poetry, automatic generation of lyrics, and even automatic generation of code.

[0023] The same model has been optimized for a long time on previous hardware, and the model performance has reached an optimal state. However, after it is implemented on another type of hardware, the previously optimized performance cannot be expressed, and it is necessary to further optimize the performance. This is very disadvantageous to the rapid introduction of neural network models. Therefore, it is necessary to optimize neural network models for different hardware. When optimizing the performance of hardware, the more cores there are, the more considerations there are to optimize the performance. There are several ways to realize convolution, such as general direct convolution and Winograd convolution. The execution effect of Winograd convolution is remarkable, but the ratio of the weight amount of Winograd to the weight amount of general convolution is 16:9, so the migration time is long. Although the calculation time has been reduced, the data migration was a bottleneck, so when selecting a configuration, it is necessary to pay attention to and consider the issue of the balance between throughput and latency. When a deep learning network is very large, for example, with several hundred layers, the convolution layer accounts for a very large proportion. How to select the optimal convolution implementation method for each convolution layer is an issue that needs to be resolved urgently.

[0024] In order to solve the above problems, an embodiment of the present application provides a method for determining a configuration of a neural network model. Referring to FIG. 1, the method includes the following steps:

[0025] In step S101, dimensional information of each target network layer in a target neural network model to be constructed and each target hardware command of a target hardware for executing the target neural network model are obtained.

[0026] The method for determining the configuration of a neural network model in the embodiment of the present application may be implemented by an electronic device, and specifically, the electronic device may be a server, a personal computer, a smartphone, a smart camera, or a hard disk video recorder, etc.

[0027] The target neural network model to be constructed may be any neural network model that needs to be constructed, such as a convolutional neural network, a recurrent neural network, a deep belief network, a generative adversarial network, a deep reinforcement learning network, etc. This application is not limited to a specific type of target neural network model.

[0028] A neural network model includes multiple network layers. The types and numbers of network layers in different neural network models may be different and need to be set according to the specific functions of the neural network model. Common network layers include convolutional layers, pooling layers, fully connected layers, etc. The network layer in the target neural network model is called the target network layer.

[0029] The dimension information of the network layer represents the dimension of the data processed by the network layer, for example, a convolution layer with a dimension of 5×5 can be used to process a 5×5 matrix and a 3×3 matrix, and a convolution layer with a dimension of 3×3 can be used to process a 3×3 matrix. The hardware command in the embodiment of the present application is a command that needs to be executed when the neural network model is executed on the hardware. The hardware command corresponds to the hardware configuration of the hardware, and the hardware command of the hardware with the same hardware configuration is the same. The hardware that executes the target neural network model is called the target hardware, and the hardware command of the target hardware is called the target hardware command.

[0030] In S102, by using the dimension information of each of the target network layers and each of the target hardware commands, the prior information and the general information in the preset model cost evaluation table are consulted, and the target hardware executes each of the target network layers to respectively determine the corresponding preset configurations when the number of clock cycles is shortest, thereby obtaining the optimal configurations of each of the target network layers in the target neural network model, where the prior information includes the optimal configurations of each of the preset network layers in different preset test modes obtained in advance, the preset test modes represent the functions of each of the preset network layers being realized by each of the preset hardware commands, each of the preset network layers corresponds to corresponding dimension information, the optimal configurations are the corresponding configurations when the number of clock cycles is shortest, and the general information includes the number of clock cycles required to execute each of the preset hardware commands in different preset configurations.

[0031] The preset configurations are pre-installed configuration policies, and may include, for example, general convolution, Winograd convolution, etc. for convolution layers.

[0032] The preset model cost evaluation table mainly records two parts of data. The first part is the number of clock cycles required to execute each preset hardware command in different preset configurations obtained based on a generic neural network model, where the number of clock cycles represents the duration, and the number of required clock cycles is the required duration. The preset hardware commands are hardware commands required to realize the function of each preset network layer (predefined generic network layer) in different hardware. The data in this part has commonality and is called generic information. The second part is a preset configuration of the shortest number of clock cycles required to realize the function of a preset network layer (dedicated network layer, the dimensional information of the dedicated network layer has already been determined) by a preset hardware command obtained based on a dedicated neural network model, and is called a priori information.

[0033] For any target network layer, first, the prior information of the second part can be queried whether there is a record regarding realizing the function of a preset network layer having the same dimensional information (same type) for the target network layer by the target hardware command of the target hardware. If there is, the optimal configuration of the corresponding preset network layer can be directly taken as the optimal configuration of the target network layer. If there is no record, the general information of the first part needs to be traversed to obtain the number of clock cycles required to realize the function of the target network layer by the target hardware command with different preset configurations, and the corresponding preset configuration when the number of clock cycles is shortest is selected as the optimal configuration of the target network layer.

[0034] By selecting the preset configuration with the minimum number of clock cycles for each target network layer, the number of clock cycles required to execute the entire target neural network model is minimized, resulting in the fastest execution speed of the target neural network model.

[0035] In an embodiment of the present application, the dimensional information of the target network layer and the hardware commands of the target hardware are used to query prior information and common general-purpose information in the preset model cost evaluation table, and the preset configuration that minimizes the number of clock cycles for each target network layer, i.e., the configuration of each target network layer when the target neural network model is executed to minimize the number of clock cycles, can be obtained, thereby obtaining the optimal configuration of the target neural network model on the target hardware, and then the optimal configuration can be used to quickly introduce the neural network model into the target hardware.

[0036] The electronic device for executing the target neural network model is referred to as a target electronic device, and the method for determining the configuration of the neural network model in the embodiment of the present application can be realized by another electronic device (e.g., a server, etc.) different from the target electronic device. In one possible embodiment, the method further includes the following steps:

[0037] In step 1, a configuration file of the target neural network model is generated according to an optimal configuration of each network layer in the target neural network model.

[0038] In step 2, the configuration file and the target neural network model are sent to the target hardware, so that the target hardware configures the target neural network model according to the configuration file and executes the configured target neural network model.

[0039] The models of different business scenarios generate an optimal layer algorithm configuration file for a given electronic device hardware according to the optimal configuration of each network layer in the target neural network model before forward inference. As shown in Fig. 2, the optimal configuration file and the target neural network model are put together to form a new neural network model. The new neural network model is recorded in the electronic device, and before forward inference, the new neural network model analyzes the configuration file of its head, adds the optimal configuration to each network layer, and sequentially executes the optimal layer algorithm by the neural network model.

[0040] For example, the neural network model first executes an offline inference process before forward inference. In the execution of the offline inference process, the optimal configuration for the execution method of each layer is selected according to the dimensional information of the network layer and the preset model cost evaluation table, thereby generating an offline configuration file, and this configuration file is inserted into the head of the original model to generate a new convolutional neural network model. When the convolutional neural network is executed, the optimal configuration of each layer is first analyzed, and each layer is executed according to the optimal configuration, so that the performance of the entire network is optimized.

[0041] The method for determining the configuration of a neural network model in the embodiment of the present application can be realized by an electronic device for executing a target neural network model. In one possible embodiment, the method further includes: configuring each network layer of the target neural network model with the optimal configuration of each network layer in the target neural network model respectively. The optimal configuration is added to each network layer of the target neural network model respectively, and the optimal layer algorithm is sequentially executed by the neural network model.

[0042] In one possible embodiment, as shown in FIG. 3a, by using the dimensional information of each target network layer and each target hardware command, querying prior information and generic information in a preset model cost evaluation table, and respectively determining the corresponding preset configuration when executing each target network layer on the target hardware to minimize the number of clock cycles, obtaining the optimal configuration of each target network layer in the target neural network model includes the following steps:

[0043] In S1021, for each target network layer, the pre-information of the preset model cost evaluation table is queried according to the dimensional information of the target network layer. If a preset network layer corresponding to the target network layer is found, the optimal configuration of the found preset network layer is set as the optimal configuration of the target network layer.

[0044] For each target network layer, if the pre-information of the preset model cost evaluation table includes a preset network layer with the same dimension information and network type as the target network layer, and an optimal configuration for realizing the function of the preset network layer by a target hardware command is recorded, the preset network layer is determined to be a preset network layer corresponding to the target network layer. As an example, for each target network layer, first, it is determined whether the pre-information of the preset model cost evaluation table includes a preset network layer with the same dimension information (same type) as the target network layer (hereinafter referred to as a first preset network layer), and if the first preset network layer exists, it is determined whether there is a record of realizing the function of the first preset network layer by the target hardware command of the target hardware, and if there is a record, the optimal configuration for realizing the function of the first preset network layer by the target hardware command is directly read and set as the optimal configuration of the current target network layer.

[0045] In S1022, if there is no preset network layer corresponding to each target network layer in the prior information of the preset model cost evaluation table, each target hardware command required to realize the function of the target network layer is determined as a hardware command group, the general information of the preset model cost evaluation table is consulted, the number of clock cycles required to execute the hardware command group in each preset configuration is determined, and the preset configuration with the shortest number of clock cycles is selected as the optimal configuration for the target network layer.

[0046] When the first network layer preset model cost evaluation table does not have an optimal configuration for realizing the function of the first network layer by the target hardware command, a common inquiry is required, and the optimal configuration of the target network layer is obtained by inquiring about generic information. As an example, in order to realize the function of the target network layer, the target hardware needs to execute two hardware commands, a first hardware command and a second hardware command, the preset model cost evaluation table is inquired, and for any preset configuration, the number of clock cycles required to execute the first hardware command and the number of clock cycles required to execute the second hardware command are obtained, thereby obtaining the total number of clock cycles required to execute the function of the target network layer in the preset configuration, and then the preset configuration when the total number of clock cycles is the shortest is selected, and the optimal configuration of the target network layer is selected.

[0047] As an example, a method for determining the configuration of a network layer according to prior information includes the following steps, as shown in FIG. 3b.

[0048] In step 1, a target neural network model to be constructed is obtained.

[0049] In step 2, the target neural network model is traversed to obtain information of each target network layer. Obtaining the information of the target network layer includes statistically calculating convolutional layer information and non-convolutional layer information, and for example, the information of the target network layer is specifically the dimension information of the convolutional layer information and the dimension information of the non-convolutional layer.

[0050] In step 3, a search space is constructed according to a combinatorial permutation scheme.

[0051] In step 4, the search space is initialized.

[0052] The search space represents a mapping table between each target network layer and a computing unit in the target neural network model, and may include data such as dimensional information of each target network layer, hardware commands used by the target hardware to realize the function of the target network layer, etc.

[0053] In step 5, prior information is registered and the search space is reduced.

[0054] For each target network layer, first, it may be determined whether there is a preset network layer (hereinafter referred to as a second preset network layer) with the same dimension information as the target network layer in the preset model cost evaluation table, and if there is a second preset network layer, it is determined whether there is a record of realizing the function of the second preset network layer by the hardware command of the target hardware, and if there is a record, it directly reads the preset configuration corresponding to the shortest number of clock cycles required to realize the function of the second preset network layer by the hardware command of the target hardware, and sets it as the optimal configuration of the target network layer. The target network layer whose optimal configuration has already been determined is deleted from the search space.

[0055] In step 6, the final search space is generated.

[0056] A final search space is obtained by matching the prior information. The target network layer in the final search space needs to perform a common inquiry. Take the example of a target hardware that needs to execute two hardware commands, a first hardware command and a second hardware command, to realize the function of the target network layer, to inquire into a preset model cost evaluation table, and for each preset configuration, obtain the number of clock cycles required to execute the first hardware command and the number of clock cycles required to execute the second hardware command in the preset configuration, thereby obtaining the total number of clock cycles required to execute the function of the target network layer in the preset configuration, and then select the preset configuration when the total number of clock cycles is the shortest, which is the optimal configuration of the target network layer.

[0057] In one possible embodiment, referring to FIG. 4, the step of pre-constructing the preset model cost evaluation table includes the following steps:

[0058] In S201, a preset reference neural network model is obtained.

[0059] The reference neural network model may be selected by customization according to actual circumstances. Generally, the reference neural network model needs to have universality and representativeness. For example, a general-purpose convolutional neural network model can be selected as the reference neural network model.

[0060] In S202, for each reference network layer of the reference neural network model, preset hardware commands required to realize the function of the reference network layer with different hardware are determined, and each preset hardware command is executed with a different preset configuration, and the number of clock cycles required to execute each preset hardware command with a different preset configuration is obtained, and recorded in the model cost evaluation table as general information.

[0061] The hardware commands correspond to the hardware configurations, and the hardware commands required for each sample hardware with different configurations to realize the functions of the reference network layer are obtained to obtain each preset hardware command. For each preset hardware command, the preset hardware command is executed in different preset configurations to obtain the number of clock cycles required to execute the preset hardware command in the different preset configurations, and the number of clock cycles is recorded in the model cost evaluation table as general information.

[0062] Specifically, for a reference neural network model, the information of all network layers in this model is statistically calculated, for example, how many convolutional layers are 1*1, how many convolutional layers are 3*3, how many pooling layers of max type are there, how many pooling layers of average type are there, etc. Then, based on the information of these network layers, the hardware commands required to realize the functions of these network layers in the sample hardware are found, for example, two commands, matmul and fixpipe, are required to realize the function of the convolutional layer, and multiple commands, such as vadd, vsub, vcmp, vmin, reduce, etc. are required to realize the function of the element-wise layer. Then, based on the sample hardware, the number of cycles (number of clock cycles) required to execute these basic hardware commands is found, and a linked list is constructed. For example, as shown in FIG. 5, the Conv layer represents a convolution layer, the Eltwise layer is an element operation layer used to perform element fusion according to the position of the element, Max Eltwise represents taking the maximum value, Sum Eltwise represents addition and subtraction, and Sub Eltwise represents substitution. Here, the Conv layer includes a 1*1 conv layer and a 5*5 conv layer, and the realization of the 1*1 conv layer function requires two commands, namely matmul and fixpipe, the realization of the 5*5 conv layer requires two commands, namely conv2d and fixpipe, the realization of the Max Eltwise function requires two commands, namely vmax and reduce_max, the realization of the Sum Eltwise function requires two commands, namely vadd and reduce_sum, and the realization of the Sub Eltwise function requires two commands, namely vsub and reduce_sub.

[0063] In S203, a pre-defined dedicated neural network model is obtained.

[0064] A dedicated neural network model is a dedicated model used to realize a specific intelligent algorithm. It is understood that the dedicated neural network models of intelligent algorithms in different business scenarios are different, so it is not possible to list them all, and in actual application scenarios, the dedicated neural network model can be selected according to the neural network model used in the actual business scenario.

[0065] In S204, for each dedicated network layer of the dedicated neural network model, the function of the dedicated network layer is realized by each preset hardware command in a different preset configuration, the number of clock cycles required to realize the function of the dedicated network layer in different preset test modes is obtained, and the preset configuration corresponding to the shortest number of clock cycles is set as the optimal configuration of the dedicated network layer, and the optimal configuration of the dedicated network layer in the different preset test modes is obtained and recorded in the model cost evaluation table as prior information.

[0066] The linked list constructed in step S202 deviates from the network layer of the actual neural network model and is obtained simply based on the hardware executing basic hardware commands, so when the entire network model is executed, when each layer calls the optimal algorithm, there is an intermediate data conversion process, such as the most common zero padding operation, which consumes extra time. In some cases, the consumption time of this part makes the consumption time of the entire network not optimal. Although the counting is based on the clock cycle of the hardware in step S202, it also involves a dimension unification operation, such as a dimension zero padding operation, but the consumption time of this part is not included in the counting in step S202, and this consumption time makes the performance of the entire network not optimal. Therefore, for such cases, it is necessary to perform a separate counting. For example, taking convolution as an example, in step S202, it is determined that the Winograd convolution time is the shortest. However, in consideration of the need for the network model to be fully adapted to the hardware, when realizing the first layer convolution, an extra unification needs to be performed on the number of channels and a zero padding operation needs to be performed. Therefore, for the first layer or the convolution with the number of input channels being 3, it is necessary to specify that the function should be realized by normal convolution. The number of clock cycles required to realize the function of the convolution layer with the number of input channels being 3 (first layer convolution) through normal convolution (configuration) on the hardware is the shortest. This is a priori information, that is, the data to be recorded in the preset model cost evaluation table.

[0067] In this application, a preset model cost evaluation table is constructed from two dimensions, a linked list of the number of cycles (clock cycles) corresponding to the hardware command is generated by a reference neural network model, and the dimensional information of a general convolutional neural network is compiled by big data to complement and extend the linked list, thereby obtaining the preset model cost evaluation table. The configuration of each network layer can be flexibly determined for different hardware, so that the neural network model can achieve high performance when executed on the hardware.

[0068] The embodiment of the present application further provides an apparatus for determining the configuration of a neural network model. Referring to FIG. 6, the apparatus includes: a dimension information acquisition module 11, which is used for acquiring dimension information of each target network layer in the target neural network model to be constructed and each target hardware command of the target hardware for executing the target neural network model; an optimal configuration determination module 12, which is used to obtain an optimal configuration of each target network layer in the target neural network model by referring to prior information and general information in a preset model cost evaluation table according to the dimension information of each target network layer and each target hardware command, and respectively determining a corresponding preset configuration when each target network layer is executed on the target hardware to minimize the number of clock cycles; The above-mentioned advance information includes an optimal configuration of each preset network layer in different preset test modes obtained in advance, the above-mentioned preset test modes represent the realization of the functions of each preset network layer by each preset hardware command, each preset network layer corresponds to corresponding dimensional information, the above-mentioned optimal configuration is a corresponding configuration when the number of clock cycles is shortest, and the above-mentioned general information includes the number of clock cycles required to execute each preset hardware command in different preset configurations.

[0069] In one possible embodiment, the optimal configuration determination module specifically queries the pre-information of the preset model cost evaluation table for each target network layer according to the dimension information of the target network layer; if a preset network layer corresponding to the target network layer is found, the optimal configuration of the found preset network layer is the optimal configuration of the target network layer; For each target network layer, if there is no preset network layer corresponding to the target network layer in the prior information of the preset model cost evaluation table, each target hardware command required to realize the function of the target network layer is determined as a hardware command group, the general information of the preset model cost evaluation table is consulted, the number of clock cycles required to execute the hardware command group in each preset configuration is determined, and the preset configuration with the shortest number of clock cycles is selected to be the optimal configuration for the target network layer.

[0070] In one possible embodiment, the apparatus further includes a cost evaluation table construction module, which is used for: obtaining a preset reference neural network model; for each reference network layer of the reference neural network model, determining preset hardware commands required to realize the function of the reference network layer with different hardware, executing each preset hardware command with different preset configurations, obtaining the number of clock cycles required to execute each preset hardware command with different preset configurations, and recording it in the model cost evaluation table as general information; obtaining a preset dedicated neural network model; for each dedicated network layer of the dedicated neural network model, achieving the function of the dedicated network layer with each preset hardware command with different preset configurations, obtaining the number of clock cycles required to realize the function of the dedicated network layer in different preset test modes, and obtaining the preset configuration corresponding to the shortest number of clock cycles as the optimal configuration of the dedicated network layer, and recording it in the model cost evaluation table as advance information.

[0071] In one possible embodiment, the device comprises: a configuration file generating module, which is used to generate a configuration file of the target neural network model according to the optimal configuration of each network layer in the target neural network model; and a configuration file sending module, which is used to send the configuration file and the target neural network model to the target hardware, so that the target hardware configures the target neural network model according to the configuration file and executes the configured target neural network model.

[0072] An embodiment of the present application is an electronic device including a processor and a memory, The memory is used to store a computer program, The present invention further provides an electronic device that, when the processor executes a computer program stored in the memory, is used to implement any of the methods for determining a configuration of a neural network model.

[0073] Alternatively, referring to FIG. 7, in addition to the above-mentioned processor 21 and memory 23, the electronic device of an embodiment of the present application further includes a communication interface 22 and a communication bus 24, and the processor 21, the communication interface 22 and the memory 23 communicate with each other via the communication bus 24.

[0074] The communication bus mentioned in the above electronic device may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. This communication bus is divided into an address bus, a data bus, a control bus, etc. In the figure, for convenience of illustration, it is illustrated with one thick line, but it does not mean that there is only one bus or one type of bus.

[0075] The communication interface is used for communication between the electronic device and other devices.

[0076] The memory may include a random access memory (RAM) and may include a non-volatile memory (NVM), such as at least one magnetic disk memory. Alternatively, the memory may further include at least one storage device remote from the processor.

[0077] The processor may be a general-purpose processor including a CPU (Central Processing Unit), NP (Network Processor), etc., or may be a general-purpose processor including a DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic device, a discrete gate or transistor logical device, or a discrete hardware assembly.

[0078] An embodiment of the present application further provides a computer-readable storage medium having a computer program recorded thereon, the computer program implementing the method of determining a configuration of any of the embodiments of the neural network model when executed by a processor.

[0079] In another embodiment provided herein, there is further provided a computer program product including instructions that, when executed on a computer, cause the computer to implement the method of determining a configuration of a neural network model of any of the above embodiments.

[0080] In the above embodiments, all or part of the invention may be realized by software, hardware, firmware, or any combination thereof. When realized by using software, all or part of the invention may be realized in the form of a computer program product. The computer program product includes one or more computer commands. When the computer program commands are loaded into a computer and executed, all or part of the invention realizes the flow or function according to the embodiments of the present application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other rewritable device. The computer commands may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer commands may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber, digital subscriber line) or wireless (e.g., infrared, radio, microwave, etc.). The computer-readable storage medium may be any available medium that a computer can read and write, or may be a data storage device including a server, data center, etc. that has one or more available media integrated therein. The available media may be magnetic media (eg, floppy disk, hard disk, tape), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)).

[0081] In this specification, the technical features of each of the preferred technical solutions may be combined to form a technical solution as long as they are not contradictory, and all of these technical solutions are within the scope of the disclosure of this application. The use of relational terms such as "first" and "second" is used only to distinguish one entity or operation from another entity or operation, and does not require or imply any actual relationship or sequence between these entities or operations. In addition, the terms "include", "includes" or any other variants thereof mean "includes" in a non-exclusive manner. Thus, a process, method, product, or device that includes a set of elements does not include only those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, product, or device. In the absence of further limitations, an element limited by the statement "including a ..." does not exclude the presence of other identical elements in the process, method, product, or device that includes the recited element.

[0082] Each embodiment in this specification is described in a related manner, and the same or similar parts between each embodiment may be cross-referenced, and each embodiment is described with emphasis on the differences from other embodiments. In particular, the embodiments of the device, electronic device, computer program product, and storage medium are basically similar to the embodiments related to the method, so they will be described relatively briefly, and the relevant parts may be referred to the parts of the embodiments related to the method.

[0083] The above description is merely a preferred embodiment of the present application, and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. 1. A method for determining a configuration of a neural network model, the method being performed by an apparatus for determining a configuration of a neural network model, the apparatus comprising: Obtaining dimensional information of each target network layer in a target neural network model to be constructed and each target hardware command of a target hardware for executing the target neural network model; According to the dimension information of each of the target network layers and each of the target hardware commands, refer to the prior information and the general information in a preset model cost evaluation table, and determine the corresponding preset configuration when the number of clock cycles is shortest by executing each of the target network layers on the target hardware, thereby obtaining the optimal configuration of each of the target network layers in the target neural network model; The advance information includes an optimal configuration of each preset network layer in different preset test modes obtained in advance, the preset test modes represent the realization of the functions of each preset network layer by each preset hardware command, each preset network layer corresponds to corresponding dimensional information, the optimal configuration is a configuration corresponding to the shortest number of clock cycles, and the general information includes the number of clock cycles required to execute each preset hardware command in different preset configurations.

2. According to the dimension information of each of the target network layers and each of the target hardware commands, the advance information and the general information in the preset model cost evaluation table are referred to, and the target hardware is executed to determine the corresponding preset configuration when the number of clock cycles is shortest, thereby obtaining the optimal configuration of each of the target network layers in the target neural network model; For each target network layer, refer to the pre-information of the preset model cost evaluation table according to the dimension information of the target network layer, and when a preset network layer corresponding to the target network layer is found, the optimal configuration of the found preset network layer is set as the optimal configuration of the target network layer; 2. The method according to claim 1, further comprising: if there is no preset network layer corresponding to each target network layer in the prior information of the preset model cost evaluation table, determining each target hardware command required to realize the function of the target network layer as a group of hardware commands, consulting the general information of the preset model cost evaluation table, respectively determining the number of clock cycles required to execute the group of hardware commands in each preset configuration, selecting the preset configuration with the shortest number of clock cycles, and setting it as the optimal configuration of the target network layer.

3. The step of the determination device pre-constructing a preset model cost evaluation table includes: The determining device obtains a preset reference neural network model; The determining device determines, for each reference network layer of the reference neural network model, a preset hardware command required to realize the function of the reference network layer with different hardware, executes each preset hardware command with different preset configurations, obtains the number of clock cycles required to execute each preset hardware command with different preset configurations, and records the number of clock cycles in a model cost evaluation table as general information; The determining device obtains a preset dedicated neural network model; 3. The method according to claim 2, wherein the determination device respectively realizes the function of the dedicated network layer of the dedicated neural network model by each preset hardware command in a different preset configuration, obtains the number of clock cycles required to realize the function of the dedicated network layer in different preset test modes, and obtains the optimal configuration of the dedicated network layer in the different preset test modes by setting the preset configuration corresponding to the shortest number of clock cycles as the optimal configuration of the dedicated network layer, and records the optimal configuration of the dedicated network layer in the different preset test modes in a model cost evaluation table as prior information.

4. generating a configuration file of the target neural network model according to an optimal configuration of each network layer in the target neural network model; 2. The method of claim 1, further comprising: transmitting the configuration file and the target neural network model to the target hardware such that the target hardware configures the target neural network model according to the configuration file and executes the configured target neural network model.

5. An apparatus for determining a configuration of a neural network model, comprising: a dimension information acquisition module, which is used for acquiring dimension information of each target network layer in a target neural network model to be constructed and each target hardware command of a target hardware for executing the target neural network model; and an optimal configuration determination module, which is used to obtain an optimal configuration of each target network layer in the target neural network model by referring to prior information and generic information in a preset model cost evaluation table according to the dimension information of each target network layer and each target hardware command, and respectively determining a corresponding preset configuration when each target network layer is executed on the target hardware to minimize the number of clock cycles; The advance information includes an optimal configuration of each preset network layer in different preset test modes obtained in advance, the preset test modes represent the realization of the functions of each preset network layer by each preset hardware command, each preset network layer corresponds to corresponding dimensional information, the optimal configuration is a configuration corresponding to the shortest number of clock cycles, and the general information includes the number of clock cycles required to execute each preset hardware command in different preset configurations.

6. Specifically, the optimal configuration determination module queries the pre-information of the preset model cost evaluation table according to the dimension information of the target network layer for each target network layer. When a preset network layer corresponding to the target network layer is found, the optimal configuration of the found preset network layer is the optimal configuration of the target network layer; 6. The device according to claim 5, which is used for: when there is no preset network layer corresponding to each target network layer in the prior information of the preset model cost evaluation table, determining each target hardware command required to realize the function of the target network layer as a hardware command group, consulting the general information of the preset model cost evaluation table, respectively determining the number of clock cycles required to execute the hardware command group in each preset configuration, selecting the preset configuration with the shortest number of clock cycles, and setting it as the optimal configuration of the target network layer.

7. 7. The apparatus of claim 6, further comprising: a cost evaluation table construction module used for: obtaining a preset reference neural network model; for each reference network layer of the reference neural network model, determining preset hardware commands required to realize a function of the reference network layer with different hardware, executing each preset hardware command with different preset configurations, respectively, obtaining the number of clock cycles required to realize each preset hardware command with different preset configurations, and recording it in the model cost evaluation table as general information; obtaining a preset dedicated neural network model; for each dedicated network layer of the dedicated neural network model, respectively, realizing a function of the dedicated network layer with each preset hardware command with different preset configurations, respectively, obtaining the number of clock cycles required to realize the function of the dedicated network layer in different preset test modes, and obtaining the preset configuration corresponding to the shortest number of clock cycles as an optimal configuration of the dedicated network layer, and recording it in the model cost evaluation table as advance information.

8. a configuration file generating module, which is used to generate a configuration file of the target neural network model according to the optimal configuration of each network layer in the target neural network model; a configuration file sending module, which is used for sending the configuration file and the target neural network model to the target hardware, so that the target hardware configures the target neural network model according to the configuration file and executes the configured target neural network model; The apparatus of claim 5 further comprising:

9. An electronic device including a processor and a memory, The memory is used to record a computer program; An electronic device used to execute the method according to any one of claims 1 to 4 as a device for determining the configuration of a neural network model when the processor executes a program recorded in the memory.

10. A computer-readable storage medium having a computer program recorded thereon, the computer program being configured to cause a processor to execute a method according to any one of claims 1 to 4 as a device for determining a configuration of a neural network model.

Images