Neural network configuration method, device, computer equipment and storage medium

Abstract

The present application relates to a neural network configuration method, apparatus, computer equipment and storage medium. The method includes: generating a valid configuration list of a neural network; determining a convolutional layer to be configured in the neural network; traversing the combination of hardware enhancement items and convolutional algorithm identifiers in the valid configuration list to obtain the running delay of the convolutional layer to be configured, and determining the target hardware enhancement items and target convolutional algorithm identifier of the convolutional layer to be configured based on the running delay; determining the next convolutional layer to be configured in the neural network, and returning to the step of traversing the combination of hardware enhancement items and convolutional algorithm identifiers in the valid configuration list to obtain the running delay of the convolutional layer to be configured, until the target hardware enhancement items and target convolutional algorithm identifiers of each convolutional layer of the neural network are determined. The use of this method can improve the efficiency of the neural network in performing reasoning tasks.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a neural network configuration method, apparatus, computer equipment and storage medium. Background Art

[0002] In recent years, convolutional neural networks have achieved great success in the field of deep learning. However, reasoning tasks such as graph classification are still computationally intensive and require the construction of application-specific hardware as well as libraries that efficiently execute reasoning programs on low-power hardware. GPGPU (General-Purpose computing on Graphics Processing Units) is one of the widely used hardware. In addition, it is very important to develop efficient subroutine libraries for emerging neural network architectures. This approach provides another layer of optimization on top of existing libraries, allowing the inference engine to fully exploit the power and efficiency of the underlying hardware instead of interfaces from traditional libraries.

[0003] Deep learning libraries such as cuDNN (Compute-Unified Device Architecture Deep Neural Network library, CUDA-based deep neural network library) have been adopted by most deep learning architectures such as TensorFlow and PyTorch. The above deep learning architectures have therefore proposed a wrapper function that can more accurately identify the best choice in the library, allowing users to implement its application by configuring environment variables to increase the running speed of the neural network.

[0004] However, since inference engine developers usually use libraries such as cuDNN directly when developing rather than through the above-mentioned deep learning frameworks, they are unable to directly use the proposed wrapper functions to identify the best choice in the library, which may affect the operating efficiency of the neural network due to suboptimal configuration.

[0005] Therefore, existing neural network configuration methods have problems that affect the operating efficiency of the neural network. Summary of the invention

[0006] Based on this, it is necessary to provide a neural network configuration method, device, computer equipment and storage medium to address the above technical problems.

[0007] In a first aspect, a neural network configuration method is provided, comprising:

[0008] Generate a valid configuration list of the neural network; the valid configuration list includes hardware enhancement items and convolution algorithm identifiers;

[0009] Determining a convolutional layer to be configured in the neural network;

[0010] Traversing the combinations of hardware enhancement items and convolution algorithm identifiers in the valid configuration list, obtaining the operation delay of the convolution layer to be configured, and determining the target hardware enhancement item and target convolution algorithm identifier of the convolution layer to be configured according to the operation delay;

[0011] Determine the next convolutional layer to be configured in the neural network, and return to the step of traversing the combination of hardware enhancement items and convolution algorithm identifiers in the valid configuration list to obtain the operation delay of the convolutional layer to be configured, until determining the target hardware enhancement items and target convolution algorithm identifiers of each convolutional layer of the neural network.

[0012] In one embodiment, traversing the combinations of hardware enhancement items and convolution algorithm identifiers in the valid configuration list to obtain the operation delay of the convolution layer to be configured, and determining the target hardware enhancement item and target convolution algorithm identifier of the convolution layer to be configured according to the operation delay, includes:

[0013] In the valid configuration list, select a convolution algorithm identifier to be tested;

[0014] For the convolution algorithm identifier to be tested, in the valid configuration list, select a hardware enhancement item to be tested;

[0015] Obtaining a running delay when the convolutional layer to be configured uses the convolutional algorithm identifier to be tested and the hardware enhancement item to be tested to perform a benchmark test task;

[0016] Obtaining a current optimized running delay, and when the running delay is less than the current optimized running delay, updating the value of the current optimized running delay to the value of the running delay, and updating the preset current preferred enhancement item to the hardware enhancement item to be tested, and updating the preset current preferred algorithm identifier to the convolution algorithm identifier to be tested;

[0017] For the convolution algorithm identifier to be tested, in the valid configuration list, select the next hardware enhancement item to be tested, until the hardware enhancement items in the valid configuration list are traversed, and then select the next convolution algorithm identifier to be tested in the valid configuration list until the convolution algorithm identifiers in the valid configuration list are traversed;

[0018] The current preferred enhancement item is determined as the target hardware enhancement item, and the current preferred algorithm identifier is determined as the target convolution algorithm identifier.

[0019] In one embodiment, obtaining the running delay of the to-be-configured convolutional layer when using the to-be-tested convolutional algorithm identifier and the to-be-tested hardware enhancement item to perform the benchmark test task includes:

[0020] Passing the convolution algorithm identifier to be tested and the hardware enhancement item to be tested into a preset benchmark test function;

[0021] The benchmark test function is run to obtain the running delay returned when the benchmark test function executes the benchmark test task according to the convolution algorithm corresponding to the convolution algorithm identifier to be tested and the hardware enhancement mode corresponding to the hardware enhancement item to be tested.

[0022] In one embodiment, it further includes:

[0023] When the running delay is not less than the current optimized running delay, the next hardware enhancement item to be tested is selected in the valid configuration list for the convolution algorithm identifier to be tested.

[0024] In one embodiment, the convolution algorithm identifier is used to mark at least one of a direct convolution algorithm identifier, an implicit general matrix multiplication algorithm, a fast Fourier transform algorithm, and a Venngrad algorithm; and the hardware enhancement item includes a core tensor option.

[0025] In a second aspect, a neural network configuration device is provided, comprising:

[0026] A generation module is configured to generate a valid configuration list of a neural network; the valid configuration list includes hardware enhancement items and convolution algorithm identifiers;

[0027] A convolutional layer determination module, configured to determine a convolutional layer to be configured in the neural network;

[0028] a combination traversal module, configured to traverse the combination of hardware enhancement items and convolution algorithm identifiers in the valid configuration list, obtain the operation delay of the convolution layer to be configured, and determine the target hardware enhancement item and target convolution algorithm identifier of the convolution layer to be configured according to the operation delay;

[0029] A return module is configured to determine the next convolutional layer to be configured in the neural network, and return to the step of traversing the combination of hardware enhancement items and convolution algorithm identifiers in the valid configuration list to obtain the running delay of the convolutional layer to be configured until the target hardware enhancement items and target convolution algorithm identifiers of each convolutional layer of the neural network are determined.

[0030] In one embodiment, the combined traversal module is further configured to:

[0031] In the valid configuration list, select a convolution algorithm identifier to be tested;

[0032] For the convolution algorithm identifier to be tested, in the valid configuration list, select a hardware enhancement item to be tested;

[0033] Obtaining a running delay when the convolutional layer to be configured uses the convolutional algorithm identifier to be tested and the hardware enhancement item to be tested to perform a benchmark test task;

[0034] Obtaining a current optimized running delay, and when the running delay is less than the current optimized running delay, updating the value of the current optimized running delay to the value of the running delay, and updating the preset current preferred enhancement item to the hardware enhancement item to be tested, and updating the preset current preferred algorithm identifier to the convolution algorithm identifier to be tested;

[0035] For the convolution algorithm identifier to be tested, in the valid configuration list, select the next hardware enhancement item to be tested, until the hardware enhancement items in the valid configuration list are traversed, and then select the next convolution algorithm identifier to be tested in the valid configuration list until the convolution algorithm identifiers in the valid configuration list are traversed;

[0036] The current preferred enhancement item is determined as the target hardware enhancement item, and the current preferred algorithm identifier is determined as the target convolution algorithm identifier.

[0037] In one embodiment, the combined traversal module is further configured to:

[0038] Passing the convolution algorithm identifier to be tested and the hardware enhancement item to be tested into a preset benchmark test function;

[0039] The benchmark test function is run to obtain the running delay returned when the benchmark test function executes the benchmark test task according to the convolution algorithm corresponding to the convolution algorithm identifier to be tested and the hardware enhancement mode corresponding to the hardware enhancement item to be tested.

[0040] In a third aspect, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the neural network configuration method of the first aspect when executing the computer program.

[0041] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the neural network configuration method of the first aspect is implemented.

[0042] The above-mentioned neural network configuration method, device, computer equipment and computer-readable storage medium first generate a valid configuration list of the neural network and determine the convolutional layer to be configured in the neural network, and then traverse the combinations of hardware enhancement items and convolutional algorithm identifiers in the valid configuration list to obtain the running delays of different combinations, thereby determining the target hardware enhancement items and target convolutional algorithm identifiers of the convolutional layer to be configured according to the running delays of different combinations, and finally configure the next convolutional layer to be configured until the target hardware enhancement items and target convolutional algorithm identifiers of each convolutional layer are determined, so that each convolutional layer in the neural network is independently optimized and configured, thereby improving the efficiency of the neural network in performing reasoning tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flowchart of a neural network configuration method according to an embodiment;

[0044] Figure 2 is a schematic diagram of performing a convolution operation based on a four-dimensional tensor according to an embodiment;

[0045] Figure 3 is a schematic diagram of a core tensor operation of an embodiment;

[0046] Figure 4 is a flowchart of steps for traversing hardware enhancement items and convolution algorithm identifications according to an embodiment;

[0047] Figure 5 is a structural block diagram of a neural network configuration device according to an embodiment;

[0048] Figure 6 The diagram is a diagram of the internal structure of a computer device according to an embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] In one embodiment, Figure 1 As shown, a neural network configuration method is provided. The neural network configuration method provided in this embodiment can be applied to a server or a server cluster composed of servers, and can also be applied to various terminals, for example, but not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices.

[0051] First of all, it should be noted that currently, multiple function libraries have been developed for different processors to efficiently perform deep learning tasks. However, developing an excellent function library for convolutional neural networks is a very challenging task, mainly due to the following reasons:

[0052] 1) Different hardware generations, or even different series of processors manufactured by the same hardware manufacturer, may have different computing capabilities;

[0053] 2) Function libraries usually require the design of a hard-coded set of options;

[0054] 3) Different algorithms or numerical precision selections may result in significant runtime delays or precision adjustments. The cuDNN library has been adopted by deep learning frameworks such as TensorFlow and PyTorch, which also have the above-mentioned defects. For this reason, it provides a wrapper function to more accurately identify the best choice of wrapper function in the library, allowing users to implement its use by configuring environment variables.

[0055] However, since the developers of the inference engine usually use the cuDNN library directly when developing, rather than through the above-mentioned deep learning frameworks, they cannot directly use the wrapper functions proposed by it to identify the best choice in the library. In addition, with the development of the cuDNN library, cuDNN provides more flexible options for performing specific operations, but users need to spend a certain amount of time and energy to explore the optimal configuration options, which increases the cost of neural network configuration. For example, enabling the Tensor Core configuration option on a specific platform can generally help reduce latency, but it may have different effects on hardware of different generations, and may even increase operating latency, requiring users to optimize the configuration themselves.

[0056] In view of the above problems, the present disclosure proposes a neural network configuration method to achieve fully automated configuration of each convolutional layer of the neural network so that the neural network can run efficiently. The method may include the following steps:

[0057] Step S102, generating a valid configuration list of the neural network; the valid configuration list includes hardware enhancement items and convolution algorithm identifiers.

[0058] Among them, the valid configuration list can be optional parameters for configuring the neural network. The optional parameters may include the convolution algorithm and related parameters of the hardware enhancement items used when the convolution layer in the neural network performs reasoning operations. Optionally, the convolution algorithm identifier can be used to mark at least one of the direct convolution algorithm identifier, the implicit general matrix multiplication algorithm, the fast Fourier transform algorithm, and the Vengrad algorithm; the hardware enhancement item may include a core tensor option.

[0059] It should be noted that an inference engine needs to go through multiple stages to run a neural network. First, the network graph data needs to be read from the input file. After reading the graph data, it will be parsed and node objects and connections will be created. Before performing the inference task, appropriate parameters need to be configured for each node. Finally, the engines of each layer of the neural network are executed to complete the corresponding inference operations. Among them, the inference operation includes the convolution operation performed by the convolution layer based on a certain convolution algorithm. Usually, in a neural network including a convolution layer, the input tensor (Input Tensor), filters (Filters), and output tensor (Output Tensor) are all four-dimensional. Figure 2 It is a schematic diagram of a convolution operation based on a four-dimensional tensor of an embodiment. As shown in the figure, the NCHW input tensor is convolved with the KCRS weight tensor to generate the NKPQ output tensor. During the operation, the input, intermediate and output data are all four-dimensional tensors. Wherein, N represents the number of images in each batch (Number of images in mini-batch), C represents the number of input feature maps (Number of input feature maps), H represents the height of the input image (Height of input image), W represents the width of the input image (Width of input image), K represents the number of output feature maps (Number of output feature maps), R represents the height of the filter kernel (Height of filter kernel), and S represents the width of the filter kernel (Width of filter kernel).

[0060] Due to the importance of convolution operations in the field of neural networks, a large number of specific convolution algorithms for implementing convolution operations have been generated. The more common convolution algorithms include direct convolution, implicit generalized matrix multiplication, fast Fourier transform (FFT), Winograd, etc. Among them, the direct convolution algorithm is an algorithm that directly performs convolution operations; the implicit generalized matrix multiplication algorithm is an algorithm that converts the convolution problem into a matrix multiplication problem; the fast Fourier transform algorithm is an algorithm that converts the convolution problem into a fast Fourier transform problem; and the Winograd algorithm is an algorithm that optimizes the number of multiplication / addition operations to complete the convolution calculation. The libraries provided by hardware vendors usually provide the above-mentioned convolution algorithms, but due to the limitations of various convolution algorithms themselves and the consideration of numerical accuracy, it is usually difficult to accurately evaluate the algorithm efficiency of a convolution algorithm when using a convolution algorithm to perform complex convolution operations.

[0061] It should be noted that the above convolution algorithm is only used for exemplary description, and this embodiment does not limit the specific convolution algorithm. In other words, the neural network configuration method of this embodiment can be applied to any convolution algorithm.

[0062] On the other hand, advanced optimization technologies such as hardware enhancement have also emerged to further improve the operating efficiency of neural networks. This technology provides hardware enhancement items for users to choose. If users choose to turn on hardware enhancement items, they can reference hardware instructions to perform complex mathematical operations on a group or subset of data elements, thereby optimizing operating efficiency by improving hardware performance. Among them, Tensor Core is one of the more common hardware enhancement items. Figure 3 The diagram is a schematic diagram of a core tensor operation of an embodiment. As shown in the figure, the core tensor operation of 4×4 matrix multiplication and accumulation is implemented in a single hardware instruction. Wherein, D represents the result after the core tensor operation, FP16 represents a 16-bit floating point number, and FP32 represents a 32-bit floating point number.

[0063] Common libraries usually provide multiple core tensor options for users to choose from. For example, you can choose a configuration option to enable tensor cores on a specific series of GPUs, or a configuration option that allows the use of tensor core operations but does not actively perform down-conversion of tensors to utilize tensor cores. Or you can choose a configuration option that allows the use of tensor core operations and actively performs down-conversion of tensors to utilize tensor cores.

[0064] Enabling hardware enhancements can improve hardware utilization and performance by using hardware instructions in some cases. However, in some cases, it may also increase operating delays and even produce incorrect calculation results.

[0065] It can be seen that although there are many convolution algorithms and hardware enhancements that can be used to improve the operating efficiency of neural networks, not all convolution algorithms and hardware enhancements can achieve the optimization effect.

[0066] Step S104, determining the convolutional layer to be configured in the neural network.

[0067] The convolutional layer to be configured may be a convolutional layer serving as a configuration target in a neural network.

[0068] Specifically, in order to optimize the configuration of each individual convolutional layer in the neural network, a convolutional layer can be first determined as the above-mentioned convolutional layer to be configured, and after completing the configuration of one convolutional layer, the configuration of the next convolutional layer can be performed.

[0069] Step S106, traversing the combinations of hardware enhancement items and convolution algorithm identifiers in the valid configuration list, obtaining the running delay of the convolution layer to be configured, and determining the target hardware enhancement item and target convolution algorithm identifier of the convolution layer to be configured according to the running delay.

[0070] Specifically, at least one hardware enhancement item and at least one convolution algorithm identifier may be selected from the valid configuration list to form a combination of the hardware enhancement item and the convolution algorithm identifier.

[0071] Then, a combination of the hardware enhancement item and the convolution algorithm identifier is used as a test parameter of the benchmark function to test the convolution layer to be configured, and the running delay of the convolution layer to be configured returned by the benchmark function is obtained.

[0072] Next, the next hardware enhancement item and / or the next convolution algorithm identifier can be selected from the valid configuration list to form a new combination of the hardware enhancement item and the convolution algorithm identifier. The new combination of the hardware enhancement item and the convolution algorithm identifier is used as the test parameter of the benchmark function, and the convolution layer to be configured is tested again to obtain the running delay of the convolution layer to be configured returned by the benchmark function.

[0073] This process is repeated until all combinations of hardware enhancement items and convolution algorithm identifiers in the valid configuration list are traversed. According to the running delays corresponding to different combinations, the shortest running delay can be determined. The hardware enhancement item and convolution algorithm identifier corresponding to the shortest running delay are the above-mentioned target hardware enhancement item and target convolution algorithm identifier.

[0074] The hardware enhancement mode corresponding to the target hardware enhancement item and the convolution algorithm corresponding to the target convolution algorithm identifier can minimize the running delay of the convolution layer to be configured in the neural network, thereby optimizing the configuration of the neural network.

[0075] For example, the target hardware enhancement item is the configuration option of "allowing the use of tensor core operations but not actively performing down-conversion of tensors to utilize tensor cores", and the target convolution algorithm is identified as "implicit general matrix multiplication algorithm". Compared with other hardware enhancement methods and convolution algorithms, the convolution layer to be configured has the smallest running delay when the target hardware enhancement item is enabled and the convolution algorithm corresponding to the target convolution algorithm is used to perform convolution operations.

[0076] It should be noted that, in this embodiment, the configuration optimization of the convolution layer is used as an example for explanation, but in practical applications, the neural network configuration of this embodiment can also be used to optimize the configuration of the deconvolution layer. Since the specific implementation is similar to the above-mentioned specific implementation for the convolution layer, it will not be repeated here.

[0077] Step S108, determining the next convolutional layer to be configured in the neural network, and returning to the step of traversing the combination of hardware enhancement items and convolutional algorithm identifiers in the valid configuration list to obtain the running delay of the convolutional layer to be configured, until determining the target convolutional algorithm identifiers of each convolutional layer of the neural network.

[0078] Specifically, after completing the configuration of one convolutional layer of the neural network, the next convolutional layer of the neural network can be configured until the configuration of each convolutional layer in the neural network is completed. Thus, each convolutional layer can use an adapted convolution algorithm to perform convolution operations, reduce operation delays, and improve the efficiency of reasoning operations.

[0079] In the above-mentioned neural network configuration method, a valid configuration list of the neural network is first generated and the convolutional layer to be configured in the neural network is determined. Then, the combinations of hardware enhancement items and convolutional algorithm identifiers in the valid configuration list are traversed to obtain the running delays of different combinations. Therefore, the target hardware enhancement items and target convolutional algorithm identifiers of the convolutional layer to be configured can be determined according to the running delays of different combinations. Finally, the next convolutional layer to be configured is configured until the target hardware enhancement items and target convolutional algorithm identifiers of each convolutional layer are determined. Thus, each convolutional layer in the neural network is independently optimized and configured, thereby improving the efficiency of the neural network in performing reasoning tasks.

[0080] Furthermore, since the combination of hardware enhancement items and convolution algorithm identifiers in the valid configuration list is traversed, the problem of affecting the operating efficiency due to the selection of suboptimal configuration is avoided. At the same time, the configuration process is fully automated without user participation, reducing development costs.

[0081] Furthermore, even if you upgrade to a new version of the library or a new generation of hardware, you can still use the above-mentioned neural network configuration method to perform configuration optimization without changing the code content. The configuration method is highly versatile, thereby reducing development costs.

[0082] Furthermore, the above-mentioned neural network configuration method can be executed in the configuration stage or the inference engine preheating stage, so it will not affect the execution efficiency of the inference task. Thus, while ensuring the execution efficiency of the neural network in executing the inference task, the operation efficiency of the neural network is improved.

[0083] In one embodiment, Figure 4 As shown, step S106 may specifically include the following steps:

[0084] Step S1061, selecting a convolution algorithm identifier to be tested from the valid configuration list;

[0085] Step S1062, selecting a hardware enhancement item to be tested from the valid configuration list for the convolution algorithm identifier to be tested;

[0086] Step S1063, obtaining the running delay of the convolution layer to be configured when using the convolution algorithm identifier to be tested and the hardware enhancement item to be tested to perform the benchmark test task;

[0087] Step S1064, obtaining the current optimized running delay, and when the running delay is less than the current optimized running delay, updating the value of the current optimized running delay to the value of the running delay, and updating the preset current preferred enhancement item to the hardware enhancement item to be tested, and updating the preset current preferred algorithm identifier to the convolution algorithm identifier to be tested;

[0088] Step S1065, returning to step S1062, until the hardware enhancement items in the valid configuration list are traversed, and then returning to step S1061, until the convolution algorithm identifiers in the valid configuration list are traversed;

[0089] Step S1066: determine the current preferred enhancement item as the target hardware enhancement item, and determine the current preferred algorithm identifier as the target convolution algorithm identifier.

[0090] Specifically, at least one convolution algorithm identifier can be first selected from the valid configuration list as the above-mentioned convolution algorithm identifier to be tested. Then, for the convolution algorithm identifier to be tested, at least one hardware enhancement item is selected from multiple hardware enhancement items in the valid configuration list as the above-mentioned convolution algorithm identifier to be tested. Thus, a combination of a hardware enhancement item and a convolution algorithm identifier is obtained.

[0091] Then, the convolution layer to be configured can be controlled to execute the benchmark task according to the convolution algorithm identifier to be tested and the configuration of the hardware enhancement item to be tested to obtain a running delay.

[0092] A current optimized running delay may be preset and initialized to a maximum value. After the benchmark task is executed and the running delay is obtained, the obtained running delay is compared with the current optimized running delay.

[0093] When the obtained running delay is less than the current optimized running delay, the value of the current optimized running delay is updated to the obtained running delay value.

[0094] For example, the current optimization running delay is 10ms (milliseconds). After executing the benchmark task, the resulting running delay is 8ms. If the resulting running delay is less than the current optimization running delay, the current optimization running delay can be updated to 8ms to form a new current optimization running delay.

[0095] In addition, a current preferred enhancement item and a current preferred algorithm identifier may be preset and initialized to a null value. When the obtained operation delay is less than the current optimized operation delay, the preset current preferred enhancement item is updated to the hardware enhancement item to be tested, and the preset current preferred algorithm identifier is updated to the convolution algorithm identifier to be tested.

[0096] For example, the current optimized running delay is 10ms. After executing the benchmark task according to the configurations of "allowing the use of tensor core operations but not actively performing down-conversion of tensors to utilize tensor cores" and "implicit general matrix multiplication algorithm", the resulting running delay is 8ms. The resulting running delay is less than the current optimized running delay. In this case, the preset current preferred enhancement item can be updated to the "allowing the use of tensor core operations but not actively performing down-conversion of tensors to utilize tensor cores" flag, and the preset current preferred algorithm flag can be updated to the "implicit general matrix multiplication algorithm" flag.

[0097] Then, return to step S1062, that is, for the same convolution algorithm identifier, select the next hardware enhancement item as a new hardware enhancement item to be tested, thereby forming a new combination of hardware enhancement item and convolution algorithm identifier, and repeat step S1063 and step S1064.

[0098] It should be noted that the current optimized running delay has been updated to the running delay when the benchmark task was last executed. If the running delay obtained by executing the benchmark task using the new hardware enhancement item to be tested is less than the current optimized running delay, the running delay value obtained by this benchmark task will be used to update the current optimized running delay value again, and the current preferred enhancement option flag will be updated accordingly. Otherwise, it will not be updated.

[0099] After traversing all the hardware enhancement items in the valid configuration list, you can return to step S1061, that is, select the next convolution algorithm identifier as a new convolution algorithm identifier to be tested, and repeat steps S1062 to S1064 until all the convolution algorithm identifiers in the valid configuration list are traversed.

[0100] After traversing all the convolution algorithm identifiers in the valid configuration list, the current preferred algorithm identifier can be determined as the target convolution algorithm identifier of the convolution layer to be configured, and the current preferred enhancement item can also be determined as the target hardware enhancement item of the convolution layer to be configured.

[0101] It can be seen that this embodiment continuously updates the value of the current optimized running delay in an iterative manner, and continuously approaches the shortest running delay. After traversing all hardware enhancement items and convolution algorithm identifiers, the last updated current optimized running delay is the shortest running delay that can be achieved when the convolution layer to be configured performs the convolution operation under the configuration of the specific hardware enhancement item and the specific convolution algorithm identifier. Correspondingly, the last updated current preferred algorithm identifier and the current preferred enhancement item are the target hardware enhancement item and the target convolution algorithm identifier that enable the convolution layer to achieve the optimal configuration.

[0102] The above-mentioned neural network configuration method obtains the target hardware enhancement items and target convolution algorithm identifiers that enable the convolution layer to achieve the optimal configuration in an iterative manner. The entire configuration process does not require complex calculations, thereby improving the configuration efficiency of the neural network.

[0103] In one embodiment, obtaining the running delay of the to-be-configured convolutional layer when using the to-be-tested convolutional algorithm identifier and the to-be-tested hardware enhancement item to perform the benchmark test task includes:

[0104] The convolution algorithm identifier to be tested and the hardware enhancement item to be tested are passed into a preset benchmark test function; the benchmark test function is run to obtain the running delay returned when the benchmark test function executes the benchmark test task according to the convolution algorithm corresponding to the convolution algorithm identifier to be tested and the hardware enhancement method corresponding to the hardware enhancement item to be tested.

[0105] Specifically, the benchmark task can be executed through a benchmark function. The convolution algorithm identifier to be tested and the hardware enhancement item to be tested are passed into the benchmark function as function parameters, and the benchmark function is run. The time value returned after the benchmark function is run is the above-mentioned running delay.

[0106] It should be noted that the complexity of the operation of the neural network configuration method provided in this embodiment is the number of convolution algorithms * the number of hardware enhancement items, which is usually between 10 and 20. In each iteration, it takes about a few milliseconds to execute the benchmark task through the benchmark function. That is to say, it only takes a short time to complete the optimization configuration of each layer of the neural network before the neural network executes the reasoning task.

[0107] The above-mentioned neural network configuration method uses a benchmark function to obtain the running delay, so that the running delay can be obtained in a convenient way, thereby improving the configuration efficiency of the neural network.

[0108] In one embodiment, the method further comprises:

[0109] When the running delay is not less than the current optimized running delay, the next hardware enhancement item to be tested is selected in the valid configuration list for the convolution algorithm identifier to be tested.

[0110] Specifically, when the running delay is greater than or equal to the current optimized running delay, it indicates that the newly selected combination of hardware enhancement items and convolution algorithm identifiers is not a more optimal configuration. Therefore, there is no need to update the current optimized running delay, the current preferred enhancement items and the current preferred algorithm identifier, and the next round of iteration can be started directly to improve the configuration efficiency of the neural network.

[0111] In order to facilitate those skilled in the art to have a deeper understanding of the embodiments of the present application, the embodiments of the present application will be described below in conjunction with specific exemplary codes.

[0112]

[0113]

[0114] In the above exemplary code, best_config (initialized convolution layer configuration) is set, and best_config.time (operation delay in convolution layer configuration) is initialized to the maximum value (FLOAT_MAX).

[0115] Then, through the loop statement for algo in Algorithms, each algo (convolution algorithm identifier) ​​in the Algorithms (convolution algorithm identifier set) in the valid configuration list is traversed.

[0116] For each algo, through the loop statement "for math_type in Math_Types", iterate through each math_type (core tensor option) in Math_Types (the set of core tensor options) in the valid configuration list. The algo and math_type selected in each loop will be passed as parameters to benchmark_conv2d() (the benchmark function) to obtain the returned time value.

[0117] When the condition time < best_config.time is satisfied, perform assignment updates via best_config.algo = algo, best_config.math_type = math_type, and best_config.time = time and return to the start of the loop statement. This process continues until all combinations of algo and math_type have been traversed.

[0118] Finally, return best_config (the optimized convolutional layer configuration) of the convolutional layer via return best_config. The algo and math_type in best_config correspond to the convolutional algorithm identifier and core tensor option that minimize time (running latency).

[0119] It should be understood that although Figure 1 and Figure 4 each step in the flowchart of Figure 1 and Figure 4 is shown sequentially in the direction of the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise clearly stated in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover,

[0120] In one embodiment, as Figure 5 shown, a neural network configuration device is provided, including:

[0121] A generation module 502, configured to generate a valid configuration list of the neural network; the valid configuration list includes hardware enhancement items and convolutional algorithm identifiers;

[0122] A convolutional layer determination module 504 is configured to determine a convolutional layer to be configured in the neural network;

[0123] A combination traversal module 506 is configured to traverse the combination of hardware enhancement items and convolution algorithm identifiers in the valid configuration list, obtain the operation delay of the convolution layer to be configured, and determine the target hardware enhancement item and target convolution algorithm identifier of the convolution layer to be configured according to the operation delay;

[0124] The return module 508 is configured to determine the next convolutional layer to be configured in the neural network, and return to the step of traversing the combination of hardware enhancement items and convolution algorithm identifiers in the valid configuration list to obtain the running delay of the convolutional layer to be configured until the target hardware enhancement items and target convolution algorithm identifiers of each convolutional layer of the neural network are determined.

[0125] In one embodiment, the combined traversal module 506 is further configured to:

[0126] In the valid configuration list, select a convolution algorithm identifier to be tested;

[0127] For the convolution algorithm identifier to be tested, in the valid configuration list, select a hardware enhancement item to be tested;

[0128] Obtaining a running delay when the convolutional layer to be configured uses the convolutional algorithm identifier to be tested and the hardware enhancement item to be tested to perform a benchmark test task;

[0129] Obtaining a current optimized running delay, and when the running delay is less than the current optimized running delay, updating the value of the current optimized running delay to the value of the running delay, and updating the preset current preferred enhancement item to the hardware enhancement item to be tested, and updating the preset current preferred algorithm identifier to the convolution algorithm identifier to be tested;

[0130] For the convolution algorithm identifier to be tested, in the valid configuration list, select the next hardware enhancement item to be tested, until the hardware enhancement items in the valid configuration list are traversed, and then select the next convolution algorithm identifier to be tested in the valid configuration list until the convolution algorithm identifiers in the valid configuration list are traversed;

[0131] The current preferred enhancement item is determined as the target hardware enhancement item, and the current preferred algorithm identifier is determined as the target convolution algorithm identifier.

[0132] In one embodiment, the combined traversal module 506 is further configured to:

[0133] Passing the convolution algorithm identifier to be tested and the hardware enhancement item to be tested into a preset benchmark test function;

[0134] The benchmark test function is run to obtain the running delay returned when the benchmark test function executes the benchmark test task according to the convolution algorithm corresponding to the convolution algorithm identifier to be tested and the hardware enhancement mode corresponding to the hardware enhancement item to be tested.

[0135] In one embodiment, the returning module 508 is further configured to:

[0136] When the running delay is not less than the current optimized running delay, the next hardware enhancement item to be tested is selected in the valid configuration list for the convolution algorithm identifier to be tested.

[0137] In one embodiment, the convolution algorithm identifier is used to mark at least one of a direct convolution algorithm identifier, an implicit general matrix multiplication algorithm, a fast Fourier transform algorithm, and a Venngrad algorithm; and the hardware enhancement item includes a core tensor option.

[0138] For the specific definition of the neural network configuration device, please refer to the definition of the neural network configuration method above, which will not be repeated here. Each module in the above-mentioned neural network configuration device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0139] The neural network configuration device provided above can be used to execute the neural network configuration method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0140] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an indoor positioning method of an air sensor is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0141] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0142] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0143] Generate a valid configuration list of the neural network; the valid configuration list includes hardware enhancement items and convolution algorithm identifiers;

[0144] Determining a convolutional layer to be configured in the neural network;

[0145] Traversing the combinations of hardware enhancement items and convolution algorithm identifiers in the valid configuration list, obtaining the operation delay of the convolution layer to be configured, and determining the target hardware enhancement item and target convolution algorithm identifier of the convolution layer to be configured according to the operation delay;

[0146] Determine the next convolutional layer to be configured in the neural network, and return to the step of traversing the combination of hardware enhancement items and convolution algorithm identifiers in the valid configuration list to obtain the operation delay of the convolutional layer to be configured, until determining the target hardware enhancement items and target convolution algorithm identifiers of each convolutional layer of the neural network.

[0147] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0148] Generate a valid configuration list of the neural network; the valid configuration list includes hardware enhancement items and convolution algorithm identifiers;

[0149] Determining a convolutional layer to be configured in the neural network;

[0150] Traversing the combinations of hardware enhancement items and convolution algorithm identifiers in the valid configuration list, obtaining the operation delay of the convolution layer to be configured, and determining the target hardware enhancement item and target convolution algorithm identifier of the convolution layer to be configured according to the operation delay;

[0151] Determine the next convolutional layer to be configured in the neural network, and return to the step of traversing the combination of hardware enhancement items and convolution algorithm identifiers in the valid configuration list to obtain the operation delay of the convolutional layer to be configured, until determining the target hardware enhancement items and target convolution algorithm identifiers of each convolutional layer of the neural network.

[0152] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0153] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0154] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A neural network configuration method, characterized in that: include: Generate a valid configuration list of a neural network; the valid configuration list includes hardware enhancement items and convolution algorithm identifiers; the convolution algorithm identifier is used to mark at least one of a direct convolution algorithm, an implicit general matrix multiplication algorithm, a fast Fourier transform algorithm, and a Venngrad algorithm; the hardware enhancement item includes a core tensor option; Determining a convolutional layer to be configured in the neural network; Traversing the combinations of hardware enhancement items and convolution algorithm identifiers in the valid configuration list, obtaining the operation delay of the convolution layer to be configured, and determining the target hardware enhancement item and target convolution algorithm identifier of the convolution layer to be configured according to the operation delay; Determine the next convolutional layer to be configured in the neural network, and return to the step of traversing the combination of hardware enhancement items and convolution algorithm identifiers in the valid configuration list to obtain the operation delay of the convolutional layer to be configured, until determining the target hardware enhancement items and target convolution algorithm identifiers of each convolutional layer of the neural network.

2. The method according to claim 1, characterized in that The traversing the combinations of hardware enhancement items and convolution algorithm identifiers in the valid configuration list to obtain the operation delay of the convolution layer to be configured, and determining the target hardware enhancement item and target convolution algorithm identifier of the convolution layer to be configured according to the operation delay, includes: In the valid configuration list, select a convolution algorithm identifier to be tested; For the convolution algorithm identifier to be tested, in the valid configuration list, select a hardware enhancement item to be tested; Obtaining a running delay when the convolutional layer to be configured uses the convolutional algorithm identifier to be tested and the hardware enhancement item to be tested to perform a benchmark test task; Obtaining a current optimized running delay, and when the running delay is less than the current optimized running delay, updating the value of the current optimized running delay to the value of the running delay, and updating the preset current preferred enhancement item to the hardware enhancement item to be tested, and updating the preset current preferred algorithm identifier to the convolution algorithm identifier to be tested; For the convolution algorithm identifier to be tested, in the valid configuration list, select the next hardware enhancement item to be tested, until the hardware enhancement items in the valid configuration list are traversed, and then select the next convolution algorithm identifier to be tested in the valid configuration list until the convolution algorithm identifiers in the valid configuration list are traversed; The current preferred enhancement item is determined as the target hardware enhancement item, and the current preferred algorithm identifier is determined as the target convolution algorithm identifier.

3. The method according to claim 2, characterized in that The obtaining of the running delay when the convolution layer to be configured uses the convolution algorithm identifier to be tested and the hardware enhancement item to be tested to perform the benchmark test task includes: Passing the convolution algorithm identifier to be tested and the hardware enhancement item to be tested into a preset benchmark test function; The benchmark test function is run to obtain the running delay returned when the benchmark test function executes the benchmark test task according to the convolution algorithm corresponding to the convolution algorithm identifier to be tested and the hardware enhancement mode corresponding to the hardware enhancement item to be tested.

4. The method according to claim 2, characterized in that: Also includes: When the running delay is not less than the current optimized running delay, the next hardware enhancement item to be tested is selected in the valid configuration list for the convolution algorithm identifier to be tested.

5. A neural network configuration device, characterized in that: include: A generation module is configured to generate a valid configuration list of a neural network; the valid configuration list includes hardware enhancement items and convolution algorithm identifiers; the convolution algorithm identifier is used to mark at least one of a direct convolution algorithm, an implicit general matrix multiplication algorithm, a fast Fourier transform algorithm, and a Venngrad algorithm; the hardware enhancement item includes a core tensor option; A convolutional layer determination module, configured to determine a convolutional layer to be configured in the neural network; a combination traversal module, configured to traverse the combination of hardware enhancement items and convolution algorithm identifiers in the valid configuration list, obtain the operation delay of the convolution layer to be configured, and determine the target hardware enhancement item and target convolution algorithm identifier of the convolution layer to be configured according to the operation delay; A return module is configured to determine the next convolutional layer to be configured in the neural network, and return to the step of traversing the combination of hardware enhancement items and convolution algorithm identifiers in the valid configuration list to obtain the running delay of the convolutional layer to be configured until the target hardware enhancement items and target convolution algorithm identifiers of each convolutional layer of the neural network are determined.

6. The device according to claim 5, characterized in that The combined traversal module is also configured to: In the valid configuration list, select a convolution algorithm identifier to be tested; For the convolution algorithm identifier to be tested, in the valid configuration list, select a hardware enhancement item to be tested; Obtaining a running delay when the convolutional layer to be configured uses the convolutional algorithm identifier to be tested and the hardware enhancement item to be tested to perform a benchmark test task; Obtaining a current optimized running delay, and when the running delay is less than the current optimized running delay, updating the value of the current optimized running delay to the value of the running delay, and updating the preset current preferred enhancement item to the hardware enhancement item to be tested, and updating the preset current preferred algorithm identifier to the convolution algorithm identifier to be tested; For the convolution algorithm identifier to be tested, in the valid configuration list, select the next hardware enhancement item to be tested, until the hardware enhancement items in the valid configuration list are traversed, and then select the next convolution algorithm identifier to be tested in the valid configuration list until the convolution algorithm identifiers in the valid configuration list are traversed; The current preferred enhancement item is determined as the target hardware enhancement item, and the current preferred algorithm identifier is determined as the target convolution algorithm identifier.

7. The device according to claim 6, characterized in that The combined traversal module is also configured to: Passing the convolution algorithm identifier to be tested and the hardware enhancement item to be tested into a preset benchmark test function; The benchmark test function is run to obtain the running delay returned when the benchmark test function executes the benchmark test task according to the convolution algorithm corresponding to the convolution algorithm identifier to be tested and the hardware enhancement mode corresponding to the hardware enhancement item to be tested.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images