Method, apparatus and device for determining configuration of neural network model, and storage medium

Abstract

The embodiment of the application provides a neural network model configuration determination method and device, equipment and a storage medium, according to the dimension information of the target network layer and the hardware instruction of the target hardware, the prior information and the universal information in the preset model cost evaluation table are inquired, the preset configuration that makes the number of clock cycles of each target network layer shortest can be obtained, that is, the configuration of each target network layer when the number of clock cycles of the target neural network model is shortest, so that the optimal configuration of the target neural network model in the target hardware is obtained, and the optimal configuration can be used to realize the rapid landing of the neural network model in the target hardware subsequently.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and particularly relates to a neural network model configuration determination method and device, equipment and a storage medium. BACKGROUND

[0002] With the continuous development of computer technology, artificial intelligence technology has entered a rapid development stage. Among various artificial intelligence technologies, the research and application of neural network models are the most extensive, and gradually become the most representative artificial intelligence technology.

[0003] When evaluating the performance of a neural network model, attention is paid to high throughput, low response time, low resource consumption and simple deployment process. With the increasing complexity of neural network models and the increasing amount of calculation, the hardware of the neural network model is also constantly updated and upgraded, for example, from a single hardware AI (artificial intelligence) acceleration unit to a many-core accelerator. However, when new hardware is put into use, there is often a problem that the same model has been optimized for a long time on the previous hardware, and the performance has reached the optimal state. However, when the model is put on another hardware, the performance of the previous optimization cannot be reflected, and a long time of performance optimization is needed again, which is not conducive to the rapid landing of the neural network model. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a neural network model configuration determination method, device, equipment and storage medium, so as to determine the configuration of the neural network model in different hardware, so as to realize the rapid landing of the neural network model in the specified hardware by using the configuration. The specific technical solutions are as follows:

[0005] In a first aspect, the embodiments of the present application provide a neural network model configuration determination method, which comprises the following steps:

[0006] obtaining the dimension information of each target network layer in a target neural network model to be configured and each target hardware instruction of a target hardware used for running the target neural network model;

[0007] querying prior information and general information in a preset model cost evaluation table according to the dimension information of each target network layer and the target hardware instructions, respectively determining preset configurations corresponding to the target network layers when the target network layers run on the target hardware with the shortest number of clock cycles, and obtaining optimal configurations of each target network layer in the target neural network model, wherein the prior information comprises optimal configurations of each preset network layer in different preset test modes, the preset test modes represent functions of each preset network layer realized by using preset hardware instructions, each preset network layer corresponds to corresponding dimension information, and the optimal configuration is a configuration corresponding to the shortest number of clock cycles; and the general information comprises the number of clock cycles required for executing each preset hardware instruction under different preset configurations.

[0008] In a possible implementation, the step of querying prior information and general information in a preset model cost evaluation table according to the dimension information of each target network layer and the target hardware instructions, respectively determining preset configurations corresponding to the target network layers when the target network layers run on the target hardware with the shortest number of clock cycles, and obtaining optimal configurations of each target network layer in the target neural network model comprises:

[0009] For each target network layer, querying prior information of the preset model cost evaluation table according to the dimension information of the target network layer, and if a preset network layer corresponding to the target network layer is found, taking the optimal configuration of the preset network layer as the optimal configuration of the target network layer;

[0010] For each target network layer, if a preset network layer corresponding to the target network layer does not exist in the prior information of the preset model cost evaluation table, determining a hardware instruction set required for realizing functions of the target network layer by using each target hardware instruction; querying general information of the preset model cost evaluation table, respectively determining the number of clock cycles required for executing the hardware instruction set under each preset configuration, and selecting a preset configuration with the shortest number of clock cycles as the optimal configuration of the target network layer.

[0011] In a possible implementation, the step of pre-establishing the preset model cost evaluation table comprises:

[0012] obtaining a preset baseline neural network model;

[0013] For each baseline network layer of the baseline neural network model, determining preset hardware instructions required for realizing functions of the baseline network layer under different hardware conditions, and respectively executing each preset hardware instruction under different preset configurations to obtain the number of clock cycles required for executing each preset hardware instruction under different preset configurations as general information and record the general information in the model cost evaluation table;

[0014] obtaining a preset special neural network model;

[0015] For each special network layer of the special neural network model, the function of the special network layer is implemented by using each preset hardware instruction under different preset configurations respectively, to obtain the number of clock cycles required for implementing the function of the special network layer under different preset test modes, and the preset configuration corresponding to the shortest number of clock cycles is taken as the optimal configuration of the special network layer, and the optimal configurations of the special network layer under different preset test modes are recorded as prior information in the model cost evaluation table.

[0016] In a possible implementation, the method further includes:

[0017] generating a configuration file of the target neural network model according to the optimal configurations of the network layers in the target neural network model;

[0018] sending the configuration file and the target neural network model to the target hardware, so that the target hardware configures the target neural network model according to the configuration file, and runs the configured target neural network model.

[0019] In a second aspect, an embodiment of the present application provides a configuration determination apparatus of a neural network model, and the apparatus includes:

[0020] a dimension information acquisition module, configured to acquire dimension information of each target network layer in a target neural network model to be configured and each target hardware instruction used for running the target neural network model;

[0021] an optimal configuration determination module, configured to query prior information and general information in a preset model cost evaluation table according to the dimension information of each target network layer and the target hardware instruction, to determine preset configurations corresponding to the shortest number of clock cycles when each target network layer runs on the target hardware respectively, and to obtain optimal configurations of each target network layer in the target neural network model, wherein the prior information includes optimal configurations of each preset network layer under different preset test modes acquired in advance, the preset test mode indicates that the function of each preset network layer is implemented by using each preset hardware instruction, each preset network layer corresponds to corresponding dimension information, and the optimal configuration is a configuration corresponding to the shortest number of clock cycles; and the general information includes the number of clock cycles required for executing each preset hardware instruction under different preset configurations.

[0022] In a possible implementation, the optimal configuration determination module is specifically configured to, for each target network layer, query the prior information of the preset model cost evaluation table according to the dimension information of the target network layer, and if a preset network layer corresponding to the target network layer is found, take the optimal configuration of the found preset network layer as the optimal configuration of the target network layer.

[0023] For each target network layer, if the preset network layer corresponding to the target network layer does not exist in the prior information of the preset model cost evaluation table, hardware instruction sets required for implementing functions of the target network layer are determined; general information of the preset model cost evaluation table is inquired, clock cycle numbers required for executing the hardware instruction sets under each preset configuration are respectively determined, and a preset configuration with the shortest clock cycle number is selected as an optimal configuration of the target network layer.

[0024] In a possible implementation, the apparatus further includes a cost evaluation table establishing module configured to: acquire a preset reference neural network model; for each reference network layer of the reference neural network model, determine preset hardware instructions required for implementing functions of the reference network layer under different hardware conditions, and record clock cycle numbers required for executing the preset hardware instructions under different preset configurations as general information into a model cost evaluation table; acquire a preset special neural network model; for each special network layer of the special neural network model, implement functions of the special network layer by using the preset hardware instructions under different preset configurations, obtain clock cycle numbers required for implementing the functions of the special network layer under different preset test modes, select a preset configuration corresponding to the shortest clock cycle number as an optimal configuration of the special network layer, and record the optimal configurations of the special network layer under different preset test modes as prior information into the model cost evaluation table.

[0025] In a possible implementation, the apparatus further includes:

[0026] a configuration file generating module configured to generate a configuration file of the target neural network model according to the optimal configurations of the network layers in the target neural network model;

[0027] a configuration file sending module configured to send the configuration file and the target neural network model to the target hardware, so that the target hardware configures the target neural network model according to the configuration file, and runs the configured target neural network model.

[0028] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory;

[0029] the memory is configured to store a computer program;

[0030] the processor is configured to execute the program stored in the memory, and implement the configuration determination method of the neural network model in any of the embodiments of the present application.

[0031] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, characterized in that the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the configuration determination method of the neural network model.

[0032] The embodiment of the present application has the following beneficial effects:

[0033] The neural network model configuration determination method, device, equipment and storage medium provided by the embodiment of the present application obtain the dimension information of each target network layer in the target neural network model to be configured and each target hardware instruction used for running the target neural network model; according to the dimension information of each target network layer and each target hardware instruction, the prior information and the general information in the preset model cost evaluation table are queried, and the preset configuration corresponding to the shortest clock cycle number of each target network layer on the target hardware is determined respectively, so that the optimal configuration of each target network layer in the target neural network model is obtained. According to the dimension information of the target network layer and the hardware instruction of the target hardware, the prior information and the general information in the preset model cost evaluation table are queried, the preset configuration that makes the clock cycle number of each target network layer shortest is obtained, that is, the configuration of each target network layer when the clock cycle number of the target neural network model is shortest, so that the optimal configuration of the target neural network model in the target hardware is obtained, and the optimal configuration can be used to realize the rapid landing of the neural network model in the target hardware subsequently. Of course, any product or method implementing the present application does not necessarily need to achieve all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art according to these drawings.

[0035] Figure 1 The first schematic diagram of the neural network model configuration determination method of the embodiment of the present application;

[0036] Figure 2 The first schematic diagram of the neural network model configuration determination method of the embodiment of the present application;

[0037] Figure 3a The second schematic diagram of the neural network model configuration determination method of the embodiment of the present application;

[0038] Figure 3b The second schematic diagram of the neural network model configuration determination method of the embodiment of the present application;

[0039] Figure 4 FIG. 1 is a schematic diagram of a preset model cost evaluation table establishment process according to an embodiment of the present application;

[0040] Figure 5 FIG. 2 is a schematic diagram of a linked list according to an embodiment of the present application;

[0041] Figure 6 FIG. 3 is a third schematic diagram of a neural network model configuration determination device according to an embodiment of the present application;

[0042] Figure 7 FIG. 4 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art based on the present application belong to the scope of protection of the present application.

[0044] First, the terms in the present application are explained:

[0045] Deep learning network: At present, various neural network models have been proposed in the academic field, such as perceptron, feedforward neural network, convolutional neural network, recurrent neural network, organization mapping, etc. The differences between these different neural network models mainly lie in the activation rule of neurons, the topological structure of neural network models, and the learning algorithm of parameters, etc. Common network types include: convolutional neural network, recurrent neural network, deep belief network, generative adversarial network, deep reinforcement learning network, etc.

[0046] CNN (Convolutional Neural Network): Convolutional neural network is one of the feedforward neural networks introduced above, which has a unique effect on the processing of graphic images. In structure, it at least includes convolutional layer and pooling layer. Convolutional neural network is a deep learning network that has been developing in recent years and is widely valued by the academic field and applied in enterprises. Representative convolutional neural networks include LeNet-5, VGG (Visual Geometry Group Network), AlexNet, etc. At present, convolutional neural network is mainly applied to object detection and recognition in images and video understanding.

[0047] LSTM (recurrent neural network): Unlike convolutional neural networks, recurrent neural networks are better at processing language text. Text analysis and processing pay more attention to the connection between input and context in time sequence. The internal memory structure of the recurrent neural network meets such demand scenarios, so the recurrent neural network is better in text processing. Currently, the main landing scenarios of recurrent neural networks are in the NLP (Neuro-Linguistic Programming) field such as machine translation and sentiment analysis. In particular, the news writing robot exposed by the media in recent years is also based on an application of recurrent neural networks. In addition, recurrent neural networks can also automatically compose poems, automatically write lyrics, and even automatically write code.

[0048] The same model has been optimized for a long time on the previous hardware, and the performance has reached the optimal state, but when put on another hardware, the previously optimized performance cannot be reflected, and a long time of performance optimization is needed again, which is not conducive to the rapid landing of neural network models, so the neural network model needs to be optimized for different hardware. When optimizing hardware performance, the more cores there are, the more points need to be considered for performance optimization. For example, there are many ways to implement convolution, such as direct convolution and Winograd convolution. Winograd convolution has obvious execution effect, but the weight amount of Winograd is 16:9 compared with the weight amount of ordinary convolution, so the time to move has become longer. Although the calculation time is less, the data movement becomes a bottleneck, so when selecting the configuration, the balance between throughput and latency needs to be considered. When a deep learning network is large, such as several hundred layers, the convolutional layer will account for a large proportion, and how to choose the optimal convolution implementation for each convolutional layer is a problem that needs to be solved urgently.

[0049] To solve the above problems, an embodiment of the present application provides a configuration determination method of a neural network model, referring to Figure 1 The method comprises:

[0050] S101, obtaining dimension information of each target network layer in a target neural network model to be configured and each target hardware instruction for running the target neural network model.

[0051] The configuration determination method of the neural network model in the embodiment of the present application can be implemented by an electronic device. Specifically, the electronic device can be a server, a personal computer, a smart phone, a smart camera, a hard disk video recorder, or the like.

[0052] The target neural network model to be configured can be any neural network model to be configured, for example, can be a convolutional neural network, a recurrent neural network, a deep belief network, a generative adversarial network, a deep reinforcement learning network, etc. The specific type of the target neural network model is not limited in the present application.

[0053] The dimension information of the network layer represents the dimension of the data processed by the network layer. For example, a 5X5 convolution kernel can be used to process a 5X5 matrix and a 3X3 matrix, and a 3X3 convolution kernel can be used to process a 3X3 matrix. The hardware instruction in the embodiment of the present application refers to the instruction that the neural network model needs to execute when running on hardware. The hardware instruction corresponds to the hardware structure of the hardware. The hardware instructions of the hardware with the same hardware structure are the same.

[0054] In S102, the prior information and the general information in the preset model cost evaluation table are queried according to the dimension information of each target network layer and the target hardware instruction, and the preset configuration corresponding to the shortest clock cycle number of each target network layer when running on the target hardware is determined respectively, so as to obtain the optimal configuration of each target network layer in the target neural network model. The prior information includes the optimal configuration of each preset network layer under different preset test modes obtained in advance. The preset test mode represents the function of each preset network layer realized by each preset hardware instruction. Each preset network layer corresponds to corresponding dimension information. The optimal configuration is the configuration corresponding to the shortest clock cycle number. The general information includes the clock cycle number required for executing each preset hardware instruction under different preset configurations.

[0055] The preset configuration is each configuration strategy set in advance. For example, for a convolution layer, it can include ordinary convolution, Winograd convolution, etc.

[0056] The preset model cost evaluation table mainly records two parts of data. The first part is obtained based on a general neural network model, and the clock cycle number required for executing each preset hardware instruction under different preset configurations. The preset hardware instruction is the hardware instruction required for realizing the function of each preset network layer (general network layer) under different hardware. This part of data is universal, and is called general information. The second part is obtained based on a special neural network model, and is the preset configuration of the shortest clock cycle number required for realizing the function of the preset network layer (special network layer, the dimension information of the special network layer has been determined) under the preset hardware instruction, and is called prior information.

[0057] For any target network layer, it can be queried in the second part of prior information whether there is a record of a preset network layer with the same target hardware instruction implementation as the target hardware and the same (type also) dimension information of the target network layer. If there is, the optimal configuration of the corresponding preset network layer can be directly used as the optimal configuration of the target network layer; if not, it needs to traverse the first part of general information: the number of clock cycles required to implement the function of the target network layer through the target hardware instruction under different preset configurations, and select the preset configuration corresponding to the shortest clock cycle number as the optimal configuration of the target network layer.

[0058] Each target network layer selects the preset configuration with the minimum clock cycle number, so that the clock cycle number required for the overall target neural network model to run is the minimum, and the running speed of the target neural network model is the fastest.

[0059] In the embodiments of the present application, according to the dimension information of the target network layer and the hardware instruction of the target hardware, the prior information and the general information in the preset model cost evaluation table are queried, the preset configuration with the shortest clock cycle number of each target network layer can be obtained, that is, the configuration of each target network layer when the clock cycle number of the target neural network model is the minimum, so that the optimal configuration of the target neural network model in the target hardware is obtained, and the subsequent neural network model can be quickly landed in the target hardware by using the optimal configuration.

[0060] The configuration determination method of the neural network model in the embodiments of the present application can be implemented by a server that is different from an electronic device used to run the target neural network model; in one possible implementation, the above method further includes:

[0061] Step one, according to the optimal configuration of each network layer in the above target neural network model, a configuration file of the above target neural network model is generated.

[0062] Step two, the above configuration file and the above target neural network model are sent to the above target hardware, so that the above target hardware configures the above target neural network model according to the above configuration file, and runs the configured target neural network model.

[0063] Before forward reasoning, the model of different business scenarios generates the optimal layer algorithm configuration file on the specified electronic device hardware according to the optimal configuration of each network layer in the target neural network model, as shown in Figure 2 The optimal configuration file and the target neural network model are put together to form a new neural network model. The new neural network model can be stored in the electronic device, and before forward reasoning, the new neural network model will first parse the configuration file in its header to add the optimal configuration to each network layer, and then use the neural network model to execute the optimal layer algorithm in turn.

[0064] The neural network model performs an offline inference process before forward inference. In the offline inference execution process, according to the dimension information of the network layer and a preset model cost evaluation table, an optimal configuration is selected for the execution mode of each layer, thereby generating an offline configuration file, and the configuration file is placed at the head of the original model, thereby forming a new convolutional neural network model. When the convolutional neural network is executed, the optimal configuration of each layer is parsed, and then each layer is executed according to the optimal configuration, so that the performance of the whole network reaches the optimal state.

[0065] The configuration determination method of the neural network model in the embodiment of the application can be implemented by an electronic device for running a target neural network model. In a possible implementation manner, the method further includes: configuring each network layer of the target neural network model according to the optimal configuration of each network layer in the target neural network model. The optimal configuration is added to each network layer of the target neural network model, and then the optimal layer algorithm is executed by using the neural network model.

[0066] In a possible implementation manner, referring to Figure 3a the preset configuration corresponding to the shortest clock cycle number when each target network layer runs on the target hardware is determined according to the dimension information of each target network layer and the target hardware instruction, thereby obtaining the optimal configuration of each target network layer in the target neural network model, including:

[0067] In S1021, for each target network layer, the prior information of the preset model cost evaluation table is queried according to the dimension information of the target network layer, and if the preset network layer corresponding to the target network layer is found, the optimal configuration of the found preset network layer is taken as the optimal configuration of the target network layer.

[0068] For each target network layer, when there is a preset network layer in the prior information of the preset model cost evaluation table, the dimension information and the network type of the preset network layer are the same as those of the target network layer, and the optimal configuration for realizing the function of the preset network layer by using the target hardware instruction is recorded, it is determined that the preset network layer is the preset network layer corresponding to the target network layer. In an example, for each target network layer, it can be determined whether there is a preset network layer (hereinafter referred to as a first preset network layer) with the same dimension information (and the same type) as the target network layer in the prior information of the preset model cost evaluation table. If there is, it is determined whether there is a record of realizing the function of the first preset network layer by using the target hardware instruction of the target hardware. When there is, the optimal configuration for realizing the function of the first preset network layer by using the target hardware instruction can be directly read as the optimal configuration of the current target network layer.

[0069] S1022, for each target network layer, if the preset network layer corresponding to the target network layer does not exist in the prior information of the preset model cost evaluation table, determining the hardware instruction set required for each target hardware instruction for realizing the function of the target network layer; querying the general information of the preset model cost evaluation table, respectively determining the number of clock cycles required for executing the hardware instruction set under each preset configuration, and selecting the preset configuration with the shortest number of clock cycles as the optimal configuration of the target network layer.

[0070] When the optimal configuration for realizing the function of the first network layer by the target hardware instruction does not exist in the first network layer preset model cost evaluation table, a general query needs to be performed to obtain the optimal configuration of the target network layer by querying the general information. In an example, taking the case that the function of the target network layer needs to be realized by the target hardware executing two hardware instructions, i.e., the first hardware instruction and the second hardware instruction, as an example, the preset model cost evaluation table is queried to obtain the number of clock cycles required for executing the first hardware instruction and the number of clock cycles required for executing the second hardware instruction under any preset configuration, thereby obtaining the total number of clock cycles required for realizing the function of the target network layer under the preset configuration; then, the preset configuration with the shortest total number of clock cycles is selected as the optimal configuration of the target network layer.

[0071] In an example, the method for determining the network layer configuration by using the prior information can be as shown in Figure 3b .

[0072] Step one, obtaining a target neural network model to be configured.

[0073] Step two, traversing the target neural network model to obtain information of each target network layer. The information of the target network layer can include statistical convolution layer information and non-convolution layer information, and can be dimension information in particular.

[0074] Step three, establishing a search space in a permutation and combination manner.

[0075] Step four, initializing the search space.

[0076] The search space represents a mapping table of each target network layer of the target neural network model and the computing unit. The search space can include dimension information of each target network layer, hardware instructions used by the target hardware to realize the function of the target network layer, and the like.

[0077] Step five, registering prior information to reduce the search space.

[0078] For each target network layer, it is first determined whether there is a preset network layer (hereinafter referred to as a second preset network layer) with the same dimension information as the target network layer in the preset model cost evaluation table. If there is, it is determined whether there is a record of using the hardware instruction of the target hardware to implement the function of the second preset network layer. When there is, the preset configuration corresponding to the number of the shortest clock cycles required to implement the function of the second preset network layer using the hardware instruction of the target hardware can be directly read as the optimal configuration of the target network layer. The target network layer for which the optimal configuration has been determined is deleted from the search space.

[0079] Step six, generating a final search space.

[0080] After matching the prior information, the final search space is obtained. The target network layer in the final search space needs to be queried for universality. For example, the function of a target network layer requires the execution of two hardware instructions, the first hardware instruction and the second hardware instruction, by the target hardware. The preset model cost evaluation table is queried. For each preset configuration, the number of clock cycles required to execute the first hardware instruction and the number of clock cycles required to execute the second hardware instruction are obtained, thereby obtaining the total number of clock cycles required to implement the function of the target network layer under the preset configuration. Then, the preset configuration with the shortest total number of clock cycles is selected as the optimal configuration of the target network layer.

[0081] In one possible implementation, referring to Figure 4 , the step of pre-establishing the preset model cost evaluation table includes:

[0082] S201, obtaining a preset reference neural network model.

[0083] The reference neural network model can be selected according to actual conditions. In general, the reference neural network model needs to have universality and representativeness. For example, a general convolutional neural network model can be selected as the reference neural network model.

[0084] S202, for each reference network layer of the above reference neural network model, determining the preset hardware instruction required to implement the function of the reference network layer under different hardware conditions, and executing each preset hardware instruction under different preset configurations to obtain the number of clock cycles required to execute each preset hardware instruction under different preset configurations as general information recorded in the model cost evaluation table.

[0085] The hardware instruction corresponds to the structure of the hardware. The hardware instruction required to implement the function of the reference network layer by each sample hardware of different structures is obtained to obtain each preset hardware instruction. For each preset hardware instruction, the preset hardware instruction is executed under different preset configurations, thereby obtaining the number of clock cycles required to execute the preset hardware instruction under different preset configurations as general information recorded in the model cost evaluation table.

[0086] Specifically, for the reference neural network model, the information of all network layers in the model is counted, such as how many 1*1 convolution layers, how many 3*3 convolution layers, how many max type pooling layers, how many average type pooling layers, etc. Then according to the information of the network layers, the hardware instructions required to realize the functions of these network layers on the sample hardware are found, for example, the implementation of the convolution layer function requires matmul and fixpipe two instructions; Element-wise layer requires multiple instructions such as vadd, vsub, vcmp, vmin, reduce, etc. Then based on the sample hardware, the number of cycles (clock cycle number) required to execute these basic hardware instructions is found, and a linked list is established, for example Figure 5

[0087] S203, obtaining a preset special neural network model.

[0088] The special neural network model is a special model used to realize a specific intelligent algorithm. It can be understood that the special neural network model of the intelligent algorithm in different business scenarios is different, so it cannot be exhausted. In actual application scenarios, the special neural network model can be selected according to the neural network model used in the actual business scenario.

[0089] S204, for each special network layer of the above special neural network model, respectively in different preset configurations, using each preset hardware instruction to realize the function of the special network layer, obtaining the number of clock cycles required to realize the function of the special network layer in different preset test modes, taking the preset configuration corresponding to the shortest clock cycle number as the optimal configuration of the special network layer, and obtaining the optimal configuration of the special network layer in different preset test modes as prior information and recording it in the model cost evaluation table.

[0090] ​Because the chain table established in step S202 is separated from the network layers of the actual neural network model, and is only based on the execution of the basic hardware instructions by the hardware, when each layer calls the optimal algorithm during the execution of the entire network model, there is an intermediate data conversion process, such as the most common 0 padding operation, which introduces additional time consumption. In some cases, this part of the time consumption will cause the entire network time consumption to be not optimal. In step S202, the clock cycle count is based on the hardware, and in addition, dimension alignment operations such as dimension 0 padding operations are involved, and this part of the time consumption is not counted in step S202, which will cause the performance of the entire network to be not optimal. Therefore, in the case of needing to count separately, for example, taking convolution as an example, it is determined in step S202 that the Winograd convolution time is the shortest, but considering that the entire network model needs to be adapted to the hardware, therefore, when implementing the first layer of convolution, additional alignment of the channel number needs to be performed, and 0 padding operation is performed, then the first layer or the convolution with an input channel number equal to 3 needs to be specially fixed to use the ordinary convolution implementation, and the number of clock cycles required to implement the function of the convolution layer (the first layer of convolution) with an input channel number equal to 3 by using the ordinary convolution (configuration) is the shortest under the current hardware, which is the prior information, that is, the data that needs to be recorded by the preset model cost evaluation table.

[0091] In the present application, a preset model cost evaluation table is established from two dimensions, a chain table of the number of cycles (clock cycles) corresponding to the hardware instructions is generated according to the benchmark neural network model, the dimension information of the common convolutional neural network is summarized by using big data, the chain table is supplemented and expanded, and thus the preset model cost evaluation table is obtained. Different configurations of each network layer can be flexibly determined according to different hardware, so that the neural network model can achieve high performance when executed on the current hardware.

[0092] The embodiment of the present application further provides a configuration determination device of a neural network model, which is shown in Figure 6 The device comprises:

[0093] A dimension information acquisition module 11 is configured to acquire dimension information of each target network layer in a target neural network model to be configured and each target hardware instruction of a target hardware used for running the target neural network model;

[0094] The optimal configuration determination module 12 is configured to query the prior information and the general information in the preset model cost evaluation table according to the dimension information of each target network layer and the target hardware instruction, and determine the preset configuration corresponding to the shortest clock cycle number of each target network layer when the target network layer runs on the target hardware, so as to obtain the optimal configuration of each target network layer in the target neural network model. The prior information includes the optimal configuration of each preset network layer in different preset test modes, and the preset test mode represents the function of each preset network layer realized by using each preset hardware instruction. Each preset network layer corresponds to corresponding dimension information, and the optimal configuration is the configuration corresponding to the shortest clock cycle number. The general information includes the clock cycle number required for executing each preset hardware instruction under different preset configurations.

[0095] In a possible implementation, the optimal configuration determination module is specifically configured to: for each target network layer, query the prior information of the preset model cost evaluation table according to the dimension information of the target network layer, and if the preset network layer corresponding to the target network layer is found, the optimal configuration of the found preset network layer is taken as the optimal configuration of the target network layer.

[0096] For each target network layer, if the preset network layer corresponding to the target network layer does not exist in the prior information of the preset model cost evaluation table, the hardware instruction set required for realizing the function of the target network layer is determined; the general information of the preset model cost evaluation table is queried, the clock cycle number required for executing the hardware instruction set under each preset configuration is determined respectively, and the preset configuration with the shortest clock cycle number is selected as the optimal configuration of the target network layer.

[0097] In a possible implementation, the apparatus further includes a cost evaluation table establishment module configured to: obtain a preset baseline neural network model; for each baseline network layer of the baseline neural network model, determine the preset hardware instruction required for realizing the function of the baseline network layer under different hardware conditions, and execute each preset hardware instruction under different preset configurations to obtain the clock cycle number required for executing each preset hardware instruction under different preset configurations as the general information recorded in the model cost evaluation table; obtain a preset special neural network model; for each special network layer of the special neural network model, realize the function of the special network layer by using each preset hardware instruction under different preset configurations to obtain the clock cycle number required for realizing the function of the special network layer under different preset test modes, take the preset configuration corresponding to the shortest clock cycle number as the optimal configuration of the special network layer, and obtain the optimal configuration of the special network layer under different preset test modes as the prior information recorded in the model cost evaluation table.

[0098] In a possible implementation, the apparatus further includes:

[0099] a configuration file generation module, configured to generate a configuration file of the target neural network model according to the optimal configuration of each network layer in the target neural network model;

[0100] a configuration file sending module, configured to send the configuration file and the target neural network model to the target hardware, so that the target hardware configures the target neural network model according to the configuration file, and runs the configured target neural network model.

[0101] The embodiment of the present application further provides an electronic device, comprising a processor and a memory.

[0102] The memory is used to store a computer program.

[0103] The processor is used to execute the computer program stored in the memory, so as to realize the configuration determination method of any neural network model.

[0104] Optionally, referring to Figure 7 In addition to the processor 21 and the memory 23, the electronic device of the embodiment of the present application further comprises a communication interface 22 and a communication bus 24, wherein the processor 21, the communication interface 22 and the memory 23 complete mutual communication through the communication bus 24.

[0105] The communication bus mentioned in the electronic device can be a PCI (Peripheral Component Interconnect, Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture, Extended Industry Standard Architecture) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0106] The communication interface is used for communication between the electronic device and other devices.

[0107] The memory can include a RAM (Random Access Memory, Random Access Memory) and can also include a NVM (Non-Volatile Memory, Non-Volatile Memory), such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0108] The processor described above can be a general processor, including a CPU (Central Processing Unit), a NP (Network Processor), etc.; can also be a DSP (Digital Signal Processing), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0109] The embodiment of the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the configuration determination method of any neural network model in the embodiment.

[0110] In another embodiment provided by the present application, a computer program product containing instructions is provided, which, when executed on a computer, causes the computer to perform the configuration determination method of any neural network model in the above embodiment.

[0111] In the above embodiment, the configuration determination method can be implemented by software, hardware, firmware or any combination thereof, in whole or in part. When implemented by software, the configuration determination method can be implemented in the form of a computer program product, in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the configuration determination method described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk, SSD), etc.

[0112] It should be noted that, in the present document, the technical features in various alternatives can be combined as long as there is no conflict. These alternatives are all within the scope of the present application. The relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or sequence between these entities or operations. Moreover, the terms "comprise", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0113] Each of the embodiments in the present specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the embodiments of the device, electronic device, computer program product and storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0114] The above only describes the preferred embodiments of the present application, and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining the configuration of a neural network model, characterized in that, The method includes: Obtain the dimensional information of each target network layer in the target neural network model to be configured, as well as the target hardware instructions for running the target neural network model; Based on the dimensional information of each target network layer and the instructions of each target hardware, the prior information and general information in the preset model cost evaluation table are queried to determine the preset configuration corresponding to the shortest number of clock cycles for each target network layer on the target hardware, thereby obtaining the optimal configuration of each target network layer in the target neural network model. The prior information includes the optimal configuration of each preset network layer under different preset test modes. The preset test mode represents the function of each preset network layer using preset hardware instructions. Each preset network layer has corresponding dimension information. The optimal configuration is the configuration corresponding to the shortest number of clock cycles. The general information includes the number of clock cycles required to execute each preset hardware instruction under different preset configurations.

2. The method according to claim 1, characterized in that, The step involves querying prior information and general information in a preset model cost evaluation table based on the dimensional information of each target network layer and the instructions of each target hardware, to determine the preset configuration corresponding to the shortest number of clock cycles for each target network layer on the target hardware, thereby obtaining the optimal configuration of each target network layer in the target neural network model, including: For each target network layer, the prior information of the preset model cost evaluation table is queried based on the dimensional information of the target network layer. If the preset network layer corresponding to the target network layer is found, the optimal configuration of the preset network layer is taken as the optimal configuration of the target network layer. For each target network layer, if the preset network layer corresponding to the target network layer is not found in the prior information of the preset model cost evaluation table, the target hardware instructions required to implement the function of the target network layer are determined to obtain the hardware instruction set; the general information of the preset model cost evaluation table is queried to determine the number of clock cycles required to execute the hardware instruction set under each preset configuration, and the preset configuration with the shortest number of clock cycles is selected as the optimal configuration for the target network layer.

3. The method according to claim 2, characterized in that, The steps for pre-establishing a pre-defined model cost evaluation table include: Obtain a pre-defined baseline neural network model; For each baseline network layer of the baseline neural network model, the preset hardware instructions required to implement the function of the baseline network layer under different hardware conditions are determined, and each preset hardware instruction is executed under different preset configurations. The number of clock cycles required to execute each preset hardware instruction under different preset configurations is recorded as general information in the model cost evaluation table. Obtain a pre-defined dedicated neural network model; For each dedicated network layer of the dedicated neural network model, the function of the dedicated network layer is implemented using various preset hardware instructions under different preset configurations. The number of clock cycles required to implement the function of the dedicated network layer under different preset test modes is obtained. The preset configuration corresponding to the shortest number of clock cycles is taken as the optimal configuration of the dedicated network layer. The optimal configuration of the dedicated network layer under different preset test modes is recorded as prior information in the model cost evaluation table.

4. The method according to claim 1, characterized in that, The method further includes: Based on the optimal configuration of each network layer in the target neural network model, a configuration file for the target neural network model is generated; The configuration file and the target neural network model are sent to the target hardware so that the target hardware configures the target neural network model according to the configuration file and runs the configured target neural network model.

5. A device for determining the configuration of a neural network model, characterized in that, The device includes: The dimension information acquisition module is used to acquire the dimension information of each target network layer in the target neural network model to be configured, as well as the target hardware instructions for running the target neural network model. The optimal configuration determination module is used to query prior information and general information in a preset model cost evaluation table based on the dimensional information of each target network layer and the target hardware instructions, and to determine the preset configuration corresponding to the shortest number of clock cycles for each target network layer on the target hardware, thereby obtaining the optimal configuration of each target network layer in the target neural network model. The prior information includes the optimal configuration of each preset network layer under different preset test modes, where each preset test mode represents the implementation of the function of each preset network layer using preset hardware instructions. Each preset network layer corresponds to specific dimensional information, and the optimal configuration is the configuration corresponding to the shortest number of clock cycles. The general information includes the number of clock cycles required to execute each preset hardware instruction under different preset configurations.

6. The apparatus according to claim 5, characterized in that, The optimal configuration determination module is specifically used for: for each target network layer, querying the prior information of the preset model cost evaluation table based on the dimensional information of the target network layer; if the preset network layer corresponding to the target network layer is found, the optimal configuration of the preset network layer is taken as the optimal configuration of the target network layer. For each target network layer, if the preset network layer corresponding to the target network layer is not found in the prior information of the preset model cost evaluation table, then the target hardware instructions required to implement the function of the target network layer are determined to obtain the hardware instruction set. Query the general information in the preset model cost evaluation table, determine the number of clock cycles required to execute the hardware instruction set under each preset configuration, and select the preset configuration with the shortest number of clock cycles as the optimal configuration for the target network layer.

7. The apparatus according to claim 6, characterized in that, The device further includes a cost evaluation table establishment module, used for: acquiring a preset benchmark neural network model; for each benchmark network layer of the benchmark neural network model, determining the preset hardware instructions required to implement the function of the benchmark network layer under different hardware conditions, and executing each preset hardware instruction under different preset configurations to obtain the number of clock cycles required to execute each preset hardware instruction under different preset configurations as general information and recording it in the model cost evaluation table; acquiring a preset dedicated neural network model; for each dedicated network layer of the dedicated neural network model, implementing the function of the dedicated network layer using each preset hardware instruction under different preset configurations to obtain the number of clock cycles required to implement the function of the dedicated network layer under different preset test modes, taking the preset configuration corresponding to the shortest number of clock cycles as the optimal configuration of the dedicated network layer, and recording the optimal configuration of the dedicated network layer under different preset test modes as prior information in the model cost evaluation table.

8. The apparatus according to claim 5, characterized in that, The device further includes: The configuration file generation module is used to generate a configuration file for the target neural network model based on the optimal configuration of each network layer in the target neural network model. The configuration file sending module is used to send the configuration file and the target neural network model to the target hardware, so that the target hardware configures the target neural network model according to the configuration file and runs the configured target neural network model.

9. An electronic device, characterized in that, Including processor and memory; The memory is used to store computer programs; When the processor executes the program stored in the memory, it implements the method described in any one of claims 1-4.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-4.

Images