Neural network processing method, apparatus, device, and storage medium

Abstract

This application discloses a neural network processing method, apparatus, device, and storage medium, belonging to the field of neural network technology. The method includes: first, obtaining the bit width and on-chip memory capacity of a computer device, and the convolutional layer parameters of each convolutional layer in at least one convolutional layer of the neural network to be processed; then, exhaustively enumerating the block partitioning strategies for each convolutional layer based on the bit width, memory capacity, and convolutional layer parameters, obtaining multiple block partitioning strategies for each convolutional layer; and finally, determining a target block partitioning strategy from the multiple block partitioning strategies for each convolutional layer. The target block partitioning strategy for each convolutional layer refers to the block partitioning strategy that minimizes the number of accesses to off-chip memory among the multiple block partitioning strategies; that is, it is the more suitable block partitioning strategy for that convolutional layer. Thus, when executing each convolutional layer, the convolutional layer data is partitioned into blocks according to the determined suitable target block partitioning strategy, resulting in fewer accesses to off-chip memory and lower power consumption.

Description

Technical Field

[0001] This application relates to the field of neural network technology, and in particular to a neural network processing method, apparatus, device, and storage medium. Background Technology

[0002] The current global popularity of artificial intelligence (AI) technology has triggered numerous technological changes, and neural network algorithms play a crucial role in AI algorithms. Due to the large amount of data in the convolutional layers of neural networks and the limited storage capacity of on-chip memory in computer devices, the data for convolutional layers is typically stored in off-chip memory. When processing each convolutional layer of a neural network, the computer divides the convolutional layer data in off-chip memory into blocks. Based on these blocks, the computer accesses the convolutional layer data in off-chip memory multiple times. Each access reads a block of convolutional layer data from off-chip memory, moves the read block to on-chip memory, performs sub-operations on the block data stored in on-chip memory, and writes the resulting block data back to off-chip memory. This process continues until all the convolutional layer data stored in off-chip memory for each convolutional layer has been traversed. Through multiple accesses, multiple data transfers, and multiple sub-operations, the result of each convolutional layer can be obtained.

[0003] In existing technologies, computer devices typically employ a fixed-block strategy to process each convolutional layer of a neural network. This fixed-block strategy means that the computer device divides each convolutional layer of different neural networks into blocks of data with a fixed size, ensuring that the data size of each block is the same. These blocks are then stored in off-chip memory. Therefore, for each convolutional layer of different neural networks, during multiple accesses to off-chip memory based on the block size, the data size of each block read from off-chip memory is the same, the data size of each block transferred to on-chip memory is the same, and the data size of each block written to on-chip memory is the same. For example, dividing the input data and weights of each convolutional layer of different neural networks into blocks results in each convolutional layer having the same amount of input block data and weight block data, and the output block data obtained by performing sub-operations on the input and weight block data also has the same amount of data.

[0004] Since different convolutional layers in different neural networks have different amounts of data, the method described above, which uses a fixed block strategy to process each convolutional layer of a neural network, may result in the computer accessing off-chip memory more frequently when processing convolutional layers with large amounts of data, leading to higher power consumption of the computer. Summary of the Invention

[0005] This application provides a neural network processing method that can determine a suitable block-segmentation strategy for each convolutional layer of the neural network. Thus, when executing each convolutional layer, the convolutional layer data can be divided into blocks according to the determined suitable target block-segmentation strategy, resulting in fewer accesses to off-chip memory and lower power consumption. The technical solution is as follows:

[0006] In a first aspect, a neural network processing method is provided, applied to a computer device, the method comprising:

[0007] Obtain the bit width and on-chip memory capacity of the computer device, as well as the convolutional layer parameters of each convolutional layer in at least one convolutional layer of the neural network to be processed;

[0008] Based on the bit width, the storage capacity, and the convolutional layer parameters of each convolutional layer, the block partitioning strategy of each convolutional layer is enumerated to obtain multiple block partitioning strategies for each convolutional layer.

[0009] A target block strategy is determined from multiple block strategies for each convolutional layer. The target block strategy is the block strategy that minimizes the number of accesses to off-chip memory among the multiple block strategies.

[0010] As an example, the step of exhaustively enumerating the block partitioning strategies for each convolutional layer based on the bit width, the storage capacity, and the convolutional layer parameters of each convolutional layer results in multiple block partitioning strategies for each convolutional layer, including:

[0011] Based on the convolutional layer parameters of each convolutional layer, enumerate all block-splitting strategies for each convolutional layer.

[0012] From all the block partitioning strategies of each convolutional layer, determine the block partitioning strategy that satisfies the constraints corresponding to the bit width and the storage capacity, and obtain the plurality of block partitioning strategies.

[0013] As an example, the process of enumerating all block-splitting strategies for each convolutional layer based on its parameters includes:

[0014] Based on the convolutional layer parameters of each convolutional layer, a block parameter range corresponding to the first block sub-strategy of each convolutional layer is constructed. The first block sub-strategy includes the block width and block height of the input data, the block width and block height parameters of the weights, and the block depth of the output data.

[0015] Based on the block parameter range corresponding to the first block sub-strategy of each convolutional layer, enumerate all the first block sub-strategies of each convolutional layer.

[0016] Determine the second block strategy corresponding to each first block strategy in all first block strategies of each convolutional layer. The second block strategy includes the block depth of the input data, the block depth of the weights, and the block width and block height of the output data. All first block strategies of each convolutional layer and the second block strategy corresponding to each first block strategy are the block strategies of each convolutional layer.

[0017] As an example, determining the second block strategy corresponding to each of the first block strategies in all first block sub-strategies of each convolutional layer includes:

[0018] For each convolutional layer, the target first block sub-policy among all first block sub-policies is determined by the following formula, where the target first block policy is any one of all first block policies for each convolutional layer:

[0019] T i =(str+1)T j -1

[0020]

[0021]

[0022] Among them, T i The second segmentation strategy corresponding to the first segmentation strategy for the target includes the segmentation depth of the input data and the segmentation depth of the weights, where str is the convolution stride of the neural network, and T j T represents the block depth of the output data included in the first block sub-strategy of the target, n T is the block width of the output data included in the second block sub-strategy corresponding to the first block sub-strategy of the target. w T represents the block width of the input data included in the first block sub-strategy of the target, q T is the block width of the weights included in the first block sub-strategy of the target, m T represents the block height of the output data included in the second block sub-strategy corresponding to the first block sub-strategy of the target. h T represents the block height of the input data included in the first block sub-strategy of the target, p The block height includes the weights of the first block sub-strategy for the target.

[0023] As an example, the constraints that satisfy the bit width and the storage capacity include input constraints corresponding to the input data, weight constraints corresponding to the weights, and output constraints corresponding to the output data.

[0024] As an example, the input constraints are expressed by the following formula:

[0025] (T h ×T w ×T i )×bit≤iBuff

[0026] Among them, T h The block partitioning strategy for each convolutional layer includes the block height of the input data, T. w The block partitioning strategy for each convolutional layer includes the block width of the input data, T. i The block strategy for each convolutional layer includes the block depth of the input data, where bit is the bit width and iBuff is the input storage capacity;

[0027] The weight constraint condition is expressed by the following formula:

[0028] (T p ×T q ×T i ×T j )×bit≤wBuff

[0029] Among them, T p The block partitioning strategy for each convolutional layer includes the block height of the weights, Tq, and the block width of the weights, T. i The block partitioning strategy for each convolutional layer includes the block depth of the weights, T. j The block depth of the output data of each convolutional layer's block strategy, bit is the bit width, and wBuff is the weight storage capacity;

[0030] The output constraint condition is expressed by the following formula:

[0031] (T m ×T n ×T j )×bit≤oBuff

[0032] Among them, T m The block partitioning strategy for each convolutional layer includes the block height of the output data, T. n The block partitioning strategy for each convolutional layer includes the block width of the output data, T. j The block strategy for each convolutional layer includes the block depth of the output data, where bit is the bit width and oBuff is the output storage capacity.

[0033] As an example, determining the target block strategy from multiple block strategies for each convolutional layer includes:

[0034] Determine the number of accesses to the off-chip memory corresponding to each of the multiple block partitioning strategies for each convolutional layer;

[0035] The target block strategy for each convolutional layer is determined based on the number of accesses to the off-chip memory corresponding to each block strategy.

[0036] As an example, determining the number of accesses to the off-chip memory corresponding to each of the multiple block-splitting strategies for each convolutional layer includes:

[0037] For the first block strategy among multiple block strategies for each convolutional layer, the number of accesses to the off-chip memory corresponding to the first block strategy is determined by the following formula, where the first block strategy is any one of the multiple block strategies for each convolutional layer:

[0038]

[0039]

[0040]

[0041] access=access ifmaps +access weigth +accesso fmaps

[0042] Among them, access ifmaps T represents the number of input accesses to the input data within the number of accesses to the off-chip memory corresponding to the first block partitioning strategy. h T represents the block height of the input data included in the first block segmentation strategy. w T represents the block width of the input data included in the first block segmentation strategy. i D represents the block depth of the input data and the block depth of the weights included in the first block strategy. p The number of memory chips on a memory column of the off-chip memory of the computer device, access weigth T represents the weighted access count of the access count to the off-chip memory corresponding to the first block partitioning strategy. p T represents the block height of the weights included in the first block partitioning strategy. q T is the block width of the weights included in the first block partitioning strategy. j The access value represents the block depth of the output data included in the first block strategy. ofmaps T represents the number of output accesses to the off-chip memory corresponding to the first block partitioning strategy, specifically the number of accesses to the output data. mT represents the block height of the output data included in the first block partitioning strategy. n The width of the output data included in the first segmentation strategy is denoted as 'access', and the number of accesses corresponding to the first segmentation strategy is denoted as 'access'.

[0043] As an example, after obtaining the bit width and on-chip memory capacity of the computer device, and the convolutional layer parameters of each convolutional layer in at least one convolutional layer of the neural network to be processed, the method further includes:

[0044] Based on the convolutional layer parameters of each convolutional layer in the at least one convolutional layer, a plurality of data repetition factors are determined for each convolutional layer. The plurality of data repetition factors include an input data repetition factor, a weight repetition factor, and an output data repetition factor. The input data repetition factor indicates the utilization rate of the input data of each convolutional layer, the weight repetition factor indicates the utilization rate of the weights of each convolutional layer, and the output data repetition factor indicates the utilization rate of the output data of each convolutional layer.

[0045] The data reuse pattern of each convolutional layer is determined based on multiple data repetition factors of each convolutional layer. The data reuse pattern of each convolutional layer includes the input data reuse pattern corresponding to the input data repetition factor, the weight reuse pattern corresponding to the weight repetition factor, and the output data reuse pattern corresponding to the output data repetition factor.

[0046] As an example, determining multiple data repetition factors for each convolutional layer based on the convolutional layer parameters of each of the at least one convolutional layer includes:

[0047] For the convolutional parameters of the target convolutional layer in the at least one convolutional layer, multiple data repetition factors of the target convolutional layer are determined by the following formula, wherein the target convolutional layer is any one of the at least one convolutional layers:

[0048]

[0049]

[0050] RFO fmaps =P×Q×I

[0051] Among them, RF ifmaps Where is the input data repetition factor, P is the weight height parameter included in the convolutional layer parameters of the target convolutional layer, Q is the weight width parameter included in the convolutional layer parameters of the target convolutional layer, J is the output depth parameter included in the convolutional layer parameters of the target convolutional layer, str is the convolutional stride of the neural network, and RF... weightHere, H is the weight repetition factor, H is the input height parameter included in the convolutional layer parameters of the target convolutional layer, W is the input width parameter included in the convolutional layer parameters of the target convolutional layer, and RF is the input height parameter included in the convolutional layer parameters of the target convolutional layer. ofmaps The output data repetition factor is I, and the convolutional layer parameters of the target convolutional layer include the input depth parameter and the weight depth parameter.

[0052] In a second aspect, a neural network processing apparatus is provided, the apparatus comprising an acquisition module, a first determination module, and a second determination module;

[0053] The acquisition module is used to acquire the bit width and on-chip memory capacity of the computer device, as well as the convolutional layer parameters of each convolutional layer in at least one convolutional layer of the neural network to be processed.

[0054] The first determining module is used to exhaustively enumerate the block partitioning strategies of each convolutional layer based on the bit width, the storage capacity, and the convolutional layer parameters of each convolutional layer, so as to obtain multiple block partitioning strategies for each convolutional layer.

[0055] The second determining module is used to determine a target block strategy from multiple block strategies of each convolutional layer. The target block strategy is the block strategy that has the minimum number of accesses to off-chip memory among the multiple block strategies.

[0056] As an example, the first determining module is also used to enumerate all block-splitting strategies for each convolutional layer based on the convolutional layer parameters of each convolutional layer;

[0057] From all the block partitioning strategies of each convolutional layer, determine the block partitioning strategy that satisfies the constraints corresponding to the bit width and the storage capacity, and obtain the plurality of block partitioning strategies.

[0058] As an example, the first determining module is further configured to construct a block parameter range corresponding to the first block sub-strategy of each convolutional layer based on the convolutional layer parameters of each convolutional layer. The first block sub-strategy includes the block width and block height of the input data, the block width and block height parameters of the weights, and the block depth of the output data.

[0059] Based on the block parameter range corresponding to the first block sub-strategy of each convolutional layer, enumerate all the first block sub-strategies of each convolutional layer.

[0060] Determine the second block strategy corresponding to each first block strategy in all first block strategies of each convolutional layer. The second block strategy includes the block depth of the input data, the block depth of the weights, and the block width and block height of the output data. All first block strategies of each convolutional layer and the second block strategy corresponding to each first block strategy are the block strategies of each convolutional layer.

[0061] As an example, the second determining module is also used to determine the number of accesses to the off-chip memory corresponding to each of the multiple partitioning strategies of each convolutional layer;

[0062] The target block strategy for each convolutional layer is determined based on the number of accesses to the off-chip memory corresponding to each block strategy.

[0063] As an example, the device further includes a third determining module and a fourth determining module:

[0064] The third determining module is used to determine multiple data repetition factors for each convolutional layer based on the convolutional layer parameters of each convolutional layer in the at least one convolutional layer. The multiple data repetition factors include an input data repetition factor, a weight repetition factor, and an output data repetition factor. The input data repetition factor indicates the utilization rate of the input data of each convolutional layer, the weight repetition factor indicates the utilization rate of the weights of each convolutional layer, and the output data repetition factor indicates the utilization rate of the output data of each convolutional layer.

[0065] The fourth determining module is used to determine the data reuse mode of each convolutional layer based on multiple data repetition factors of each convolutional layer. The data reuse mode of each convolutional layer includes an input data reuse mode corresponding to the input data repetition factor, a weight repetition mode corresponding to the weight repetition factor, and an output data reuse mode corresponding to the output data repetition factor.

[0066] Thirdly, a computer device is provided, the computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the neural network processing method described above.

[0067] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described neural network processing method.

[0068] The beneficial effects of the technical solutions provided in this application are:

[0069] In this embodiment, the bit width and on-chip memory capacity of the computer device are first obtained, along with the convolutional layer parameters of each convolutional layer in at least one convolutional layer of the neural network to be processed. Then, based on the bit width, memory capacity, and convolutional layer parameters, the block partitioning strategies for each convolutional layer are exhaustively enumerated to obtain multiple block partitioning strategies for each convolutional layer. Finally, a target block partitioning strategy is determined from these multiple block partitioning strategies for each convolutional layer. The target block partitioning strategy for each convolutional layer refers to the block partitioning strategy that minimizes the number of accesses to off-chip memory among the multiple block partitioning strategies. In other words, the target block partitioning strategy for each convolutional layer is a suitable block partitioning strategy that minimizes the number of accesses under the computer device's own conditions. Thus, when executing each convolutional layer, the convolutional layer data can be partitioned according to the determined suitable target block partitioning strategy, resulting in fewer accesses to off-chip memory and lower power consumption. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1 This is a flowchart of a neural network processing method provided in an embodiment of this application;

[0072] Figure 2 This is a schematic diagram of the structure of a convolutional layer provided in an embodiment of this application;

[0073] Figure 3 This is a flowchart of another neural network processing method proposed in the embodiments of this application;

[0074] Figure 4 This is a schematic diagram of the structure of a neural network processing device provided in an embodiment of this application;

[0075] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0077] It should be understood that "multiple" as mentioned in this application refers to two or more. In the description of this application, unless otherwise stated, " / " indicates "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, to facilitate a clear description of the technical solutions of this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and that "first," "second," etc., do not necessarily imply differences.

[0078] Before providing a detailed explanation of the embodiments of this application, the application scenarios of these embodiments will be described first.

[0079] In recent years, with the development of artificial intelligence technology, it has been introduced into various fields, and neural network algorithms play an important role in artificial intelligence technology.

[0080] In neural networks, convolutional layers contain a large amount of data, and much of this data is typically stored in the computer's off-chip memory. Therefore, during the execution of each convolutional layer, the computer needs to access the data stored in off-chip memory, moving a large amount of data from off-chip to on-chip memory. Then, calculations are performed based on the data moved to on-chip memory, and the calculated convolutional data is written back to off-chip memory. In other words, during the process of accessing off-chip memory, the computer can perform operations to read convolutional data from off-chip memory or write convolutional data to off-chip memory. The multiple data transfers by the computer can refer to reading convolutional data from off-chip memory and writing convolutional data to off-chip memory. The number of times the computer accesses off-chip memory is the sum of the number of times data is read from off-chip memory and the number of times data is written to off-chip memory.

[0081] Because the on-chip memory of computer devices has limited storage capacity, computer devices cannot move all the large amounts of convolutional layer data stored in off-chip memory to on-chip memory at once, meaning they cannot map complete convolutional layer data for computation. Therefore, when processing each convolutional layer of a neural network, computer devices typically divide the convolutional layer data stored in off-chip memory into blocks. Based on these blocks, the convolutional layer data in off-chip memory is accessed multiple times. Each access reads the block data of the convolutional layer from off-chip memory, moves the read block data to on-chip memory, and then performs sub-operations on each convolutional layer based on the block data moved to on-chip memory. The convolutional layer data obtained from the sub-operations is then written back to off-chip memory, until all the convolutional layer data stored in off-chip memory for each convolutional layer has been traversed. In this way, through multiple accesses, multiple transfers, and multiple sub-operations, the running result of each convolutional layer can be obtained.

[0082] As an example, since computer processors typically employ the von Neumann architecture, and these processors consume a significant amount of power when processing neural networks, while the power used for computation is relatively small, the number of data transfers (i.e., the number of accesses to off-chip memory) during the operation of a neural network directly impacts the computer's power consumption. Therefore, how to segment the convolutional layer data of a neural network into blocks to reduce the number of accesses to off-chip memory and lower the power consumption of the computer during neural network operation is a pressing technical problem that needs to be solved.

[0083] In existing technologies, computer devices typically employ a fixed block partitioning strategy to process each convolutional layer of a neural network. Since different convolutional layers in different neural networks have varying amounts of data, this fixed block partitioning strategy may result in the computer accessing off-chip memory more frequently when processing convolutional layers with large amounts of data, leading to higher power consumption.

[0084] Based on this, this application provides a neural network processing method that can determine a suitable target segmentation strategy for each convolutional layer based on the computer device's own conditions and the convolutional layer parameters of each convolutional layer for different neural networks. In this way, when the computer device processes each convolutional layer of the neural network, it can segment the convolutional layer data according to the determined suitable target segmentation strategy, thereby reducing the number of accesses to off-chip memory and reducing power consumption.

[0085] Please refer to Figure 1 , Figure 1This is a flowchart illustrating a neural network processing method provided in an embodiment of this application. The neural network processing method can be applied to a computer device, which can be a terminal, server, or embedded device, etc., and the terminal can be a desktop computer or tablet computer, etc. Figure 1 As shown, the method includes the following steps:

[0086] Step 101: The computer device obtains the bit width and on-chip memory capacity of the computer device, as well as the convolutional layer parameters of each convolutional layer in at least one convolutional layer of the neural network to be processed.

[0087] The bit width of a computer device refers to the width of data (data bit width) that the computer device can process at one time.

[0088] The on-chip memory capacity refers to the maximum storage space of a computer device for storing block data. The on-chip memory capacity can include the input storage capacity for storing input block data, the weight storage capacity for storing weight block data, and the output storage capacity for storing output block data. The on-chip memory capacity is limited.

[0089] For example, convolutional layer data stored in off-chip memory includes the input data, weights, and output data of the convolutional layer. The computer device can divide the input data stored in off-chip memory into blocks according to a block-division strategy to obtain at least one input block data, and divide the weights into blocks to obtain at least one weight block data. Therefore, when executing each convolutional layer of a neural network, the computer device can read one input block data and one weight block data from off-chip memory according to the block-division strategy, and then move the acquired input block data and weight block data to on-chip memory. That is, the computer device can store one input block data and one weight block data read from off-chip memory in on-chip memory. The storage capacity of the on-chip memory used to store the input block data is the input storage capacity, and the storage capacity of the on-chip memory used to store the weight block data is the weight storage capacity. Furthermore, the computer device can perform calculations based on the input block data and weight block data in on-chip memory to obtain one output block data. This obtained output block data is stored in on-chip memory, and the storage capacity of the on-chip memory used to store the output block data is the output storage capacity. Then, the computer device writes one output block data back to off-chip memory. Typically, the input storage capacity, weight storage capacity, and output storage capacity are limited.

[0090] As an example, the on-chip memory can be a cache memory or SRAM (Static Random-Access Memory), etc. This application does not limit the on-chip memory.

[0091] The parameters of each convolutional layer can include input width, input height, input depth, weight width, weight height, weight depth, output width, output height, and weight depth.

[0092] As an example, the input width, input height, and input depth parameters indicate the amount of input data included in the convolutional layer data. For instance, the input data for a convolutional layer in a neural network is an input feature map, and the input width, input height, and input depth parameters can respectively indicate the width, height, and depth of the input feature map. Similarly, the weight width, weight height, and weight depth parameters indicate the amount of weight data included in the convolutional layer data, and the output width, output height, and output depth parameters indicate the amount of output data included in the convolutional layer data.

[0093] In neural networks, weights can also be called convolution kernels.

[0094] In addition, computer devices can also obtain relevant parameters of off-chip memory and other relevant parameters of at least one convolutional layer in a neural network. For example, they can obtain the number of memory chips in a memory rank of off-chip memory, the number of convolutional layers in at least one convolutional layer of a neural network, etc.

[0095] As an example, off-chip memory has a large and scalable storage capacity, and the convolutional layer parameters of a neural network are stored in off-chip memory. The number of times a computer device accesses the convolutional layer data in off-chip memory is related to the number of memory chips on a memory rank in off-chip memory.

[0096] As an example, the off-chip memory can be DRAM (Dynamic Static Random-Access Memory) or a floppy disk, etc. This application does not limit the off-chip memory.

[0097] Step 102: The computer device exhaustively enumerates the block partitioning strategies for each convolutional layer based on the bit width, storage capacity, and convolutional layer parameters of each convolutional layer, thereby obtaining multiple block partitioning strategies for each convolutional layer.

[0098] The block strategy for each convolutional layer refers to the strategy for dividing the convolutional layer data into blocks. The block strategy can include the block width, block height and block depth of the input data, the block width, block height and block depth of the weights, and the block width, block height and block depth of the output data.

[0099] For clarity, please refer to [link / reference]. Figure 2 , Figure 2 This is a schematic diagram of a convolutional layer structure provided in an embodiment of this application. For example... Figure 2 As shown, the convolutional layer data includes input data, J weights, and output data. The convolutional layer parameters include the input width W, input height H, and input depth I of the input data; the weight width Q, weight height P, and weight depth I of the weights; and the output width N, output height M, and output depth J of the output data. The convolutional layer's block partitioning strategy includes the block width T of the input data. w Block height T h and block depth T i The block width T of the weight q Block height T p and block depth T i and the block width T of the output data n Block height T m and block depth T j .

[0100] In the convolutional layer of the neural network, the input depth parameter is equal to the weight depth parameter, and the block strategy includes that the block depth of the input data is equal to the block depth of the weights, and the number of weights is equal to the output depth parameter.

[0101] like Figure 2 As shown, the block strategy includes the block width T of the input data. w Block height T h and block depth T i This refers to the block size after the input data is divided into blocks, specifically the size of the input block data moved from external memory each time it is accessed. The block strategy includes the weighted block width T. q Block height T p and block depth T i This refers to the size of the weighted block data after the weights are divided into blocks. The block strategy includes the block width T of the output data. n Block height T m and block depth T j This refers to the size of the output block data obtained by calculating the input block data and the weight block data. Since the computer device needs to traverse all the convolutional layer data stored in off-chip memory when running each convolutional layer, the block strategy affects the number of accesses to off-chip memory.

[0102] For example, a computer device can obtain multiple block partitioning strategies for each convolutional layer through the following steps:

[0103] Step 1021: Based on the convolutional layer parameters of each convolutional layer, enumerate all block-splitting strategies for each convolutional layer.

[0104] For ease of explanation, in the embodiments of this application, the block width, block height and block depth of the input data, the block width, block height and block depth of the weights, and the block width, block height and block depth of the output data are referred to as tiling factors, that is, the convolutional layer data includes multiple tiling factors.

[0105] Among them, the strategy of exhaustively enumerating all block partitioning strategies for each convolutional layer refers to first determining all possible values ​​of the tiling factor included in the convolutional layer data, and then combining the tiling factors with different values ​​to exhaustively enumerate all possible block partitioning methods for the convolutional layer data.

[0106] For example, a computer device can exhaustively enumerate all block-splitting strategies for each convolutional layer based on the convolutional layer parameters using the following steps:

[0107] Step 1: Based on the convolutional layer parameters of each convolutional layer, construct the block parameter range corresponding to the first block sub-strategy of each convolutional layer.

[0108] The first segmentation sub-strategy includes the segmentation width and height of the input data, the segmentation width and height parameters of the weights, and the segmentation depth of the output data.

[0109] The segmentation parameter range corresponding to the first segmentation sub-strategy may include the input segmentation width range corresponding to the segmentation width of the input data, the input segmentation height range corresponding to the segmentation height of the input data, the weight segmentation width range corresponding to the segmentation width of the weight, the weight segmentation height range corresponding to the segmentation height of the weight, and the output segmentation depth range corresponding to the segmentation depth of the output data.

[0110] As an example, such as Figure 2 As shown, the block width T of the input data w It should be less than or equal to the input width parameter W and the block height T of the input data. h It should be less than or equal to the input height parameter H and the block width T of the weight. q It should be less than or equal to the weight width parameter Q and the weight block height T. p The block depth T of the output data should be less than or equal to the weight height parameter P. j It should be less than or equal to the output depth parameter J. Thus, the first sub-strategy includes a block width T of the input data. w Block height T h Block width T q Block height T p and the block depth T of the output data jIt has a maximum value, and the maximum value is related to the convolutional layer parameters. Therefore, the computer device can construct the block parameter difference corresponding to the first block sub-strategy of each convolutional layer based on the convolutional layer parameters of each convolutional layer.

[0111] Furthermore, the core of computing convolutional layers in computer devices is to perform a multiplication operation between the input data blocks and the weight data blocks, thus increasing the width T of the input data blocks. w The block width T should be greater than or equal to the weight. q The block height T of the input data h The block height T should be greater than or equal to the weight. p Multiplication can only be performed under these circumstances. That is, the block width T of the input data... w and block height T h It also has a minimum value.

[0112] In summary, due to the block width T of the input data w Block width T that is greater than or equal to the weight q The input block width is less than or equal to the input width parameter W, therefore the input block width range is T. w ∈[T q , W]; due to the block height T of the input data h The block height T should be greater than or equal to the weight. p The value is less than or equal to the input block height parameter H, therefore the input block height range is T. h ∈[T p [H]; due to the block width T of the weights q The weight block width is less than or equal to the weight width parameter Q, therefore the weight block width range is T. q ∈[1, Q]; due to the block height T of the weights p The weight block height range is T, which is less than or equal to the weight height parameter P. p ∈[1, P]; due to the block depth T of the output data j It should be less than or equal to the output depth parameter J, therefore the output block depth range is T. j ∈[1, J].

[0113] As an example, since the weight width parameter Q and the weight height parameter P are typically small, the block width T of the weight is... q It can be equal to the weight width parameter Q, and the weight block height T. p It can be equal to the weight height parameter P, so the input block width range is T. w ∈[Q, W], the input block height interval is T h ∈[P,H].

[0114] Step 2: Based on the block parameter range corresponding to the first block sub-strategy of each convolutional layer, enumerate all the first block sub-strategies of each convolutional layer.

[0115] Enumerating all first-block sub-strategies for each convolutional layer means iterating through the block width and height of all input data, the block width and height parameters of all weights, and the block depth of all output data within the ranges of input block width, input block height, weight block width, weight height, and output block depth. This process involves combining different block widths and heights of input data, weights, and output data to enumerate all first-block sub-strategies.

[0116] As an example, the block width T of the weight q Equals the weight width parameter Q, and the weight block height T. p The weight height parameter P is equal to the input block width range T. w ∈[Q, W], the input block height interval is T h ∈[P, H], the output block depth interval is T j If ∈[1, J], then the computer device can exhaustively enumerate all first block sub-strategies of each convolutional layer based on the pre-set adjustable input block width step size, input block height step size, output block depth step size, and the block parameter range corresponding to the first block sub-strategy of each convolutional layer.

[0117] For example, a computer device can exhaustively enumerate all first-block sub-strategies for each convolutional layer using three nested loops, where the three loops are respectively for the input block width interval T. w Within [Q, W], exhaustively enumerate all block widths of the input data based on the input block width step size, and within the input block height interval T... h Within [P, H], exhaustively enumerate all block heights of the input data based on the input block height step size, and within the output block depth interval T. j Within [1, J], exhaustively enumerate all block depths of the output data based on the output block depth step size. In this way, all first block sub-strategies can be obtained through three nested loops. At least one of the block width and block height of the input data, the block width and block height of the weights, and the block depth of the output data is different among all first block sub-strategies.

[0118] For example, the third and fourth block sub-strategies are any two of all the first block sub-strategies. The block width and height of the input data, the block width and height of the weights, and the block depth of the output data included in the third block sub-strategy are different from the block width and height of the input data, the block width and height of the weights, and the block depth of the output data included in the fourth block sub-strategy.

[0119] Step 3: Determine the second block sub-policy corresponding to each first block sub-policy in all first block sub-policies of each convolutional layer.

[0120] The second block sub-strategy includes the block depth of the input data, the block depth of the weights, and the block width and block height of the output data. All first block sub-strategies of each convolutional layer and the second block sub-strategy corresponding to each first block sub-strategy are all block strategies of each convolutional layer.

[0121] For example, for the target first block strategy among all the first block strategies of each convolutional layer, the computer device can determine the target first block strategy based on the block width Tw and block height T of the input data. h The block width T of the weight q The block height T of the weight p and the block depth T of the output data j Then, determine the second block strategy corresponding to the target first block strategy. Here, the target first block strategy is any one of all first block strategies for each convolutional layer.

[0122] As an example, for the target first block policy among all first block policies of each convolutional layer, the computer device determines the second block policy corresponding to the target first block policy using the following formula (1):

[0123]

[0124] Among them, T i The second block strategy corresponding to the first block strategy for the target includes the block depth of the input data and the block depth of the weights, where str is the convolution stride of the neural network, and T is the target block strategy. j The first segmentation sub-strategy for the target includes the segmentation depth of the output data, T. n T represents the block width of the output data included in the second block strategy corresponding to the first block strategy. w T represents the block width of the input data included in the first sub-strategy of the target segmentation, q The block width T is the weight of the first block sub-strategy for the target. mT represents the block height of the output data included in the second block strategy corresponding to the first block strategy. h The first segmentation sub-strategy for the target includes the segmentation height of the input data, T. p The block height includes the weights of the first block sub-strategy for the target.

[0125] Thus, by using steps 1-3 above, all possible block partitioning strategies for each convolutional layer can be exhaustively enumerated. That is, when a computer device processes the convolutional layer of a neural network, it can exhaustively enumerate all possible block partitioning strategies for that convolutional layer based on the parameters of each convolutional layer of the neural network. In turn, it can determine the block partitioning strategy that minimizes the number of accesses to off-chip memory from all possible block partitioning strategies. This overcomes the problem in the prior art where using the same block partitioning strategy for convolutional layers of different neural networks or different convolutional layers of the same neural network results in low power consumption when the computer device processes convolutional layers with large amounts of data.

[0126] It should be noted that before determining the block strategy with the fewest accesses to off-chip memory from all possible block strategies, the computer device can select multiple block strategies that meet the computer device's own conditions from all block strategies of each convolutional layer, and then determine the block strategy with the fewest accesses to off-chip memory from the multiple block strategies.

[0127] Step 1022: Determine the block strategy that satisfies the constraints corresponding to bit width and storage capacity from all block strategies of each convolutional layer, and obtain multiple block strategies.

[0128] After obtaining all the block partitioning strategies for each convolutional layer, the block partitioning strategy that satisfies the conditions of the computer device itself should be determined from all the block partitioning strategies. For example, when different computer devices or the same computer device processes different neural networks, the storage capacity of the on-chip memory may be different, and the bit width of different computer devices may also be different. Moreover, the bit width of the computer device and the storage capacity of the on-chip memory have limitations on the size of the input block data, weight block data, and output block data, which affects the block partitioning strategy.

[0129] The constraints corresponding to the bit width and storage capacity include input constraints corresponding to the input data, weight constraints corresponding to the weights, and output constraints corresponding to the output data. Specifically, input constraints correspond to the input storage capacity, weight constraints correspond to the weight storage capacity, and output constraints correspond to the output storage capacity. The input storage capacity is the capacity of the computer device to store the input block data, the weight storage capacity is the capacity of the computer device to store the weight block data, and the output storage capacity is the capacity of the computer device to store the output block data.

[0130] As an example, when a computer device processes a neural network, it divides the input data into blocks according to a block-splitting strategy to obtain input block data. The size of the input block data should be less than or equal to the input storage capacity. Similarly, the size of the weight block data should be less than or equal to the weight storage capacity, and the size of the output block data should be less than or equal to the output storage capacity.

[0131] As an example, the amount of input block data is related to the bit width of the computer device and the block width, block height, and block depth of the input data included in the block strategy; the amount of weight block data is related to the bit width of the computer device and the block width, block height, and block depth of the weights included in the block strategy; and the amount of output block data is related to the bit width of the computer device and the block width, block height, and block depth of the output data included in the block strategy.

[0132] For example, the input constraints that satisfy the bit width and input storage capacity can be expressed by the following formula (2):

[0133] (T h ×T w ×T i )×bit≤iBuff (2)

[0134] Among them, T h The block partitioning strategy for each convolutional layer includes the block height of the input data, T. w The block partitioning strategy for each convolutional layer includes the block width of the input data, T. i The block strategy for each convolutional layer includes the block depth of the input data, where bit is the bit width and iBuff is the input storage capacity.

[0135] For example, the weight constraint condition that satisfies the bit width and weight storage capacity can be expressed by the following formula (3):

[0136] (T p ×T q ×T i ×T j )×bit≤wBuff (3)

[0137] Among them, T p The block partitioning strategy for each convolutional layer includes the block height of the weights, T. q The block partitioning strategy for each convolutional layer includes the block width of the weights, T. i The block partitioning strategy for each convolutional layer includes the block depth of the weights, T. j The block depth of the output data for each convolutional layer is defined by the block partitioning strategy, where bit is the bit width and wBuff is the weight storage capacity.

[0138] For example, the output constraints that satisfy the bit width and output storage capacity can be expressed by the following formula (4):

[0139] (T m ×T n ×T j )×bit≤oBuff (4)

[0140] Among them, T m The block partitioning strategy for each convolutional layer includes the block height of the output data, T. n The block partitioning strategy for each convolutional layer includes the block width of the output data, T. j The block strategy for each convolutional layer includes the block depth of the output data, where bit is the bit width and oBuff is the output storage capacity.

[0141] Thus, through the above formulas (2)-(4), the computer device can determine the block strategy that satisfies the input constraints, weight constraints and output constraints from all block strategies of each convolutional layer, and obtain multiple block strategies. All of the obtained block strategies satisfy the computer device's own conditions.

[0142] As an example, the computer device can iterate through all block strategies for each convolutional layer. For the second block strategy encountered, the computer device determines whether the second block strategy satisfies the input constraints, weight constraints, and output constraints. If it does, the second block strategy is identified as one of the multiple block strategies, and the process continues to iterate through all other block strategies except the second one. If the constraints are not met, the process continues through all other block strategies except the second one, until all block strategies for each convolutional layer have been iterated, resulting in multiple block strategies. Here, the second block strategy is any one of the multiple block strategies for each convolutional layer.

[0143] As an example, a computer device can also, after exhaustively enumerating all first-block sub-strategies for each convolutional layer based on the block parameter range corresponding to each first-block sub-strategy, first determine the block depth of the input data and the block depth of the weights included in the second-block sub-strategy corresponding to each first-block sub-strategy in all the first-block sub-strategies of each convolutional layer. Then, based on all the first-block sub-strategies of each convolutional layer and the block depth of the input data and the block depth of the weights included in the second-block sub-strategy corresponding to each first-block sub-strategy, determine multiple first-block sub-strategies for each convolutional layer that satisfy the input constraints and weight constraints, and the block depth of the input data and the block depth of the weights corresponding to each first-block sub-strategy. Then, based on the conditions satisfying... For each convolutional layer with input and weight constraints, multiple first-segmentation sub-strategies and the corresponding block depths of the input and weights for each first-segmentation sub-strategy are used to determine the block width and height of the output data included in the corresponding second-segmentation sub-strategy. Then, based on the multiple first-segmentation sub-strategies for each convolutional layer satisfying the input and weight constraints, and the corresponding block depths of the input and weights, as well as the block width and height of the output data for each first-segmentation sub-strategy, the first-segmentation sub-strategy for each convolutional layer satisfying the output constraints is determined, along with the corresponding block depths of the input and weights, and the block width and height of the output data. This results in multiple segmentation strategies. This approach effectively reduces the amount of unnecessary computation on the computer.

[0144] Step 103: The computer device determines the target block strategy from multiple block strategies for each convolutional layer.

[0145] Among them, the target block strategy refers to the block strategy that minimizes the number of accesses to off-chip memory among multiple block strategies.

[0146] After obtaining multiple block strategies for each convolutional layer that meet the conditions of the computer device itself, the computer device can determine the block strategy that minimizes the number of accesses to off-chip memory from the multiple block strategies for each convolutional layer. The determined block strategy is a more suitable block strategy for that convolutional layer, such as a more suitable block strategy that minimizes the number of accesses under the conditions of the computer device itself.

[0147] For example, computer devices can determine the target partitioning strategy through the following steps:

[0148] Step 1031: Determine the number of accesses to off-chip memory corresponding to each of the multiple block strategies in each convolutional layer.

[0149] The number of accesses to off-chip memory corresponding to each block strategy can include the number of input accesses to the input data stored in off-chip memory, the number of weight accesses to the weights, and the number of accesses to the output data.

[0150] The number of accesses to off-chip memory corresponding to each block strategy is related to the number of memory chips on a memory column of the computer device's off-chip memory and the block strategy. In other words, the computer device can determine the number of accesses to off-chip memory corresponding to each block strategy based on the number of memory chips on a memory column of the computer device's off-chip memory and each block strategy.

[0151] For example, the first block strategy is any one of the multiple block strategies for each convolutional layer. For the first block strategy among the multiple block strategies for each convolutional layer, the computer device can determine the number of accesses to off-chip memory corresponding to the first block strategy using the following formula (5):

[0152]

[0153] Among them, access ifmaps T represents the number of input accesses to input data within the number of accesses to off-chip memory corresponding to the first block partitioning strategy. h T represents the block height of the input data included in the first block partitioning strategy. w T represents the block width of the input data included in the first block partitioning strategy. i D represents the block depth of the input data and the block depth of the weights included in the first block partitioning strategy. p The number of memory chips on a single column of memory in the off-chip storage of a computer device, access weigth T represents the weighted access count of the number of accesses to off-chip memory corresponding to the first block partitioning strategy. p T represents the block height of the weights included in the first block partitioning strategy. q T is the block width of the weights included in the first block partitioning strategy. j The access level represents the block depth of the output data included in the first block partitioning strategy. ofmaps T represents the number of output accesses to output data within the number of accesses to off-chip memory corresponding to the first block strategy. m T represents the block height of the output data included in the first block partitioning strategy. n The width of the output data included in the first segmentation strategy is denoted by 'access', and the number of accesses corresponding to the first segmentation strategy is denoted by 'access'.

[0154] In formula (5) above, [] rd The number of times a computer device reads data from external memory, [ ]. wrThis refers to the number of times a computer device writes data to external memory. Specifically, since the operations performed by the computer device involve reading input data and weights from external memory, and writing output data to external memory, the input access count refers to the number of times the computer device reads input data from external memory, the weight access count refers to the number of times the computer device reads weights from external memory, and the output access count refers to the number of times the computer device writes output data to external memory.

[0155] As an example, when executing a convolutional layer, the computer device may also perform operations to read output data from external memory. In this case, the number of output accesses is the sum of the number of times the computer device reads output data from external memory and the number of times it writes output data to external memory. The computer device can also determine the number of output accesses using the following formula (6). ofmaps :

[0156]

[0157] Among them, access ofmaps T represents the number of output accesses to output data within the number of accesses to off-chip memory corresponding to the first block strategy. m T represents the block height of the output data included in the first block partitioning strategy. n T represents the block width of the output data included in the first block partitioning strategy. j D represents the block depth of the output data included in the first block partitioning strategy. p The number of memory chips on a memory column of an off-chip storage device.

[0158] In other words, before determining the target partitioning strategy from multiple partitioning strategies for each convolutional layer, the computer device first determines the data reuse mode of each convolutional layer, and then determines the number of accesses to off-chip memory corresponding to each partitioning strategy based on each partitioning strategy and the data reuse mode of each convolutional layer.

[0159] For example, if the data reuse mode of the convolutional layer is output data reuse mode, then the computer device performs operations to write and read output data from off-chip memory, and the number of output accesses is [number]. ofmaps The number of accesses is determined by the formula (6) above. If the data reuse mode of the convolutional layer is the input data reuse mode and the weight reuse mode, then the computer device performs the operation of writing output data from off-chip memory, and the number of accesses is determined by the formula (6). ofmaps It is determined by the above formula (5).

[0160] It should be noted that the method for determining the data reuse pattern for each convolutional layer will be explained below. Figure 3The embodiments are described in detail, and the embodiments of this application will not be repeated here.

[0161] Step 1032: Determine the target block strategy for each convolutional layer based on the number of accesses to off-chip memory corresponding to each block strategy.

[0162] For example, from the multiple block-splitting strategies of each convolutional layer, the block-splitting strategy with the fewest accesses to off-chip memory is determined, and this block-splitting strategy with the fewest accesses to off-chip memory is the target block-splitting strategy.

[0163] Thus, the determined target partitioning strategy is a more suitable partitioning strategy that minimizes the number of accesses under the conditions of the computer device itself.

[0164] It should be noted that for each convolutional layer of the neural network, the computer device performs steps 101-103 as described above to determine a more suitable target block strategy for each convolutional layer. Thus, when the computer device executes the convolutional layer, it blocks the convolutional layer data according to the determined suitable target block strategy, resulting in fewer accesses to off-chip memory and lower power consumption.

[0165] In this embodiment, the bit width and on-chip memory capacity of the computer device are first obtained, along with the convolutional layer parameters of each convolutional layer in at least one convolutional layer of the neural network to be processed. Then, based on the bit width, memory capacity, and convolutional layer parameters, the block partitioning strategies for each convolutional layer are exhaustively enumerated to obtain multiple block partitioning strategies for each convolutional layer. Finally, a target block partitioning strategy is determined from these multiple block partitioning strategies for each convolutional layer. The target block partitioning strategy for each convolutional layer refers to the block partitioning strategy that minimizes the number of accesses to off-chip memory among the multiple block partitioning strategies. In other words, the target block partitioning strategy for each convolutional layer is a suitable block partitioning strategy that minimizes the number of accesses under the computer device's own conditions. Thus, when executing each convolutional layer, the convolutional layer data can be partitioned according to the determined suitable target block partitioning strategy, resulting in fewer accesses to off-chip memory and lower power consumption.

[0166] In addition, the data reuse pattern of each convolutional layer when a computer processes a neural network also affects the number of times the computer accesses off-chip memory, which in turn affects the power consumption of the computer.

[0167] As an example, for any one of the input, weight, and output data blocks, the computer may reuse that data block during the computation of each convolutional layer. To reduce redundant data transfer, the computer typically reuses that data block. This method of reusing data blocks can also be called the data reuse pattern of neural networks.

[0168] Data reuse mode refers to prioritizing input data, weights, and output data, so that data with higher priority is stored in on-chip memory for a longer period of time to maximize data reuse, i.e., maximizing the reuse of convolutional layer data, thereby reducing redundant access to off-chip memory. Data reuse mode can include input data reuse mode, weight reuse mode, and output data reuse mode.

[0169] Furthermore, the data reuse pattern of each convolutional layer also affects the above. Figure 1 Step 1031 in the embodiment refers to determining the number of accesses to off-chip memory that affect the block partitioning strategy.

[0170] Based on this, the neural network processing method proposed in this application embodiment can also determine the data reuse mode of each convolutional layer, so that data with higher priority is stored in on-chip memory for a longer time to maximize the reuse of the data, thereby reducing redundant access to off-chip memory.

[0171] Please refer to Figure 3 , Figure 3 This is a flowchart of another neural network processing method provided in an embodiment of this application. This neural network processing method can be applied to a computer device, which can be a terminal, server, or embedded device, etc., and the terminal can be a desktop computer or tablet computer, etc. Figure 3 As shown, the method includes the following steps:

[0172] Step 301: The computer device acquires the convolutional layer parameters of each convolutional layer in at least one convolutional layer of the neural network to be processed.

[0173] The parameters of each convolutional layer can include input width, input height, input depth, weight width, weight height, weight depth, output width, output height, and weight depth.

[0174] Step 302: The computer device determines multiple data repetition factors for each convolutional layer based on the convolutional layer parameters of each convolutional layer in at least one convolutional layer.

[0175] The multiple data repetition factors include the input data repetition factor, the weight repetition factor, and the output data repetition factor. The input data repetition factor indicates the utilization rate of the input data of each convolutional layer, the weight repetition factor indicates the utilization rate of the weights of each convolutional layer, and the output data repetition factor indicates the utilization rate of the output data of each convolutional layer.

[0176] The higher the data redundancy factor of each convolutional layer, the higher the utilization rate of the corresponding data when the computer executes that convolutional layer. For example, if the input data redundancy factor is the highest among the multiple data redundancy factors of each convolutional layer, then the computer will utilize the input data the most when executing that convolutional layer.

[0177] As an example, the input data repetition factor can indicate the usage rate of the input data for multiplication operations in each convolutional layer, the weight repetition factor indicates the usage rate of the weights for multiplication operations in each convolutional layer, and the output data repetition factor indicates the usage rate of the output data for multiplication operations in each convolutional layer.

[0178] For example, the target convolutional layer is any one of at least one convolutional layers. For the convolutional layer parameters of the target convolutional layer in at least one convolutional layer, the computer device can determine multiple data repetition factors of the target convolutional layer using the following formula (7):

[0179]

[0180] Among them, RF ifmaps Let P be the input data repetition factor, Q be the weight height parameter (including the convolutional parameters of the target convolutional layer), J be the output depth parameter (including the convolutional parameters of the target convolutional layer), str be the convolutional stride of the neural network, and RF be the input data repetition factor. weight Here, H represents the weight repetition factor, H represents the input height parameter included in the convolutional parameters of the target convolutional layer, W represents the input width parameter included in the convolutional parameters of the target convolutional layer, and RF represents the input height parameter included in the convolutional parameters of the target convolutional layer. ofmaps The output data repetition factor is I, which represents the convolutional layer parameters of the target convolutional layer, including the input depth parameter and the weight depth parameter.

[0181] Step 303: The computer device determines the data reuse pattern of each convolutional layer based on multiple data repetition factors of each convolutional layer.

[0182] The data reuse modes include the input data reuse mode corresponding to the input data repetition factor, the weight repetition mode corresponding to the weight repetition factor, and the output data reuse mode corresponding to the output data repetition factor.

[0183] For example, a computer device can first determine the largest data repetition factor from multiple data repetition factors in each convolutional layer, and then determine the data reuse pattern corresponding to the largest data repetition factor.

[0184] As an example, if a computer device determines that the input data repetition factor is the largest among multiple data repetition factors in each convolutional layer, then the input data multiplexing pattern corresponding to the input data repetition factor is determined as the data multiplexing pattern for each convolutional layer; if it determines that the weight repetition factor is the largest among multiple data repetition factors in each convolutional layer, then the weight multiplexing pattern corresponding to the weight repetition factor is determined as the data multiplexing pattern for each convolutional layer; if it determines that the output data repetition factor is the largest among multiple data repetition factors in each convolutional layer, then the output data multiplexing pattern corresponding to the output data repetition factor is determined as the data multiplexing pattern for each convolutional layer.

[0185] Since a higher data redundancy factor in each convolutional layer indicates a higher utilization rate of the corresponding data when the computer device executes that convolutional layer, the computer device can identify the data corresponding to the data redundancy factor with a higher utilization rate as data with higher priority. This allows the data with higher priority to be kept in on-chip memory for a longer period of time and reused to the maximum extent. This can reduce redundant access to off-chip memory, that is, reduce the number of accesses to off-chip memory.

[0186] As an example, if a computer device determines that the input data repetition factor is the largest among the multiple data repetition factors of each convolutional layer, that is, when the convolutional layer is executed by the computer device, the input data is used most frequently, then the computer device can determine the input data as data with higher priority, so that the input data can be kept in on-chip memory for a longer time and reused to the maximum extent.

[0187] It should be noted that for each convolutional layer of the neural network, the computer device performs steps 301-303 as described above to determine the data reuse pattern of each convolutional layer. Thus, when the computer device executes the convolutional layer, it reuses the convolutional layer data to the maximum extent according to the determined data reuse pattern.

[0188] In addition, steps 301-303 above can be performed... Figure 1 Before steps 101-103 in the embodiment, it is also possible to... Figure 1 After steps 101-103 in the embodiment, or with Figure 1 Steps 101-103 in the embodiment are executed simultaneously. In this application embodiment, steps 301-303 are executed concurrently with... Figure 1 The execution order of steps 101-103 in the embodiments is not limited.

[0189] Furthermore, steps 302-303 above can also be performed in... Figure 1 The steps in the embodiments are executed after step 101, but this application does not limit this.

[0190] It should be noted that when executing each convolutional layer, the computer device can determine the data reuse mode of each convolutional layer according to steps 301-303 above, based on... Figure 1 Steps 101-103 in the embodiment determine a more suitable block strategy for each convolutional layer. Thus, when the computer device executes each convolutional layer, it can process the data of each convolutional layer according to the data reuse mode of each convolutional layer and the more suitable block strategy for that convolutional layer, thereby minimizing the number of accesses to off-chip memory and reducing the power consumption of the computer device.

[0191] In this embodiment, the convolutional parameters of each convolutional layer in at least one convolutional layer of the neural network to be processed are first obtained. Then, based on the convolutional parameters of each convolutional layer, multiple data repetition factors for each convolutional layer are determined. Finally, based on the multiple data repetition factors for each convolutional layer, a data reuse pattern for each convolutional layer is determined. Thus, when the computer device executes a convolutional layer, it can reuse convolutional layer data to the maximum extent possible according to the data reuse pattern of that convolutional layer. Therefore, redundant accesses to off-chip memory can be reduced, i.e., the number of accesses to off-chip memory can be reduced, thereby reducing the power consumption of the computer device.

[0192] Please refer to Figure 4 , Figure 4 This is a schematic diagram of the structure of a neural network processing device provided in an embodiment of this application. The neural network processing device can be implemented as part or all of a computer device by software, hardware, or a combination of both. This computer device can be as described below. Figure 5 The computer equipment shown. See also Figure 4 The device includes: an acquisition module 401, a first determination module 402, and a second determination module 403.

[0193] The acquisition module 401 is used to acquire the bit width and on-chip memory capacity of the computer device, as well as the convolutional layer parameters of each convolutional layer in at least one convolutional layer of the neural network to be processed.

[0194] The first determining module 402 is used to exhaustively enumerate the block division strategies of each convolutional layer based on the bit width, storage capacity and convolutional layer parameters of each convolutional layer, so as to obtain multiple block division strategies for each convolutional layer.

[0195] The second determining module 403 is used to determine the target block strategy from multiple block strategies of each convolutional layer. The target block strategy is the block strategy that has the minimum number of accesses to off-chip memory among the multiple block strategies.

[0196] As an example, the first determining module 402 is also used to exhaustively enumerate all block strategies for each convolutional layer based on the convolutional layer parameters of each convolutional layer;

[0197] From all the block partitioning strategies of each convolutional layer, determine the block partitioning strategy that satisfies the constraints corresponding to the bit width and storage capacity, and obtain multiple block partitioning strategies.

[0198] As an example, the first determining module 402 is also used to exhaustively enumerate all block-splitting strategies for each convolutional layer based on the convolutional layer parameters of each convolutional layer, including:

[0199] Based on the convolutional layer parameters of each convolutional layer, construct the block parameter range corresponding to the first block sub-strategy of each convolutional layer. The first block sub-strategy includes the block width and block height of the input data, the block width and block height parameters of the weights, and the block depth of the output data.

[0200] Based on the block parameter range corresponding to the first block sub-strategy of each convolutional layer, enumerate all the first block sub-strategies of each convolutional layer.

[0201] Determine the second block sub-strategy corresponding to each first block sub-strategy in all first block sub-strategies of each convolutional layer. The second block sub-strategy includes the block depth of the input data, the block depth of the weights, and the block width and block height of the output data. All first block sub-strategies of each convolutional layer and the second block sub-strategy corresponding to each first block sub-strategy are the block strategies of each convolutional layer.

[0202] As an example, the second determining module 403 is also used to determine the number of accesses to the off-chip memory corresponding to each of the multiple block strategies in each convolutional layer;

[0203] The target block strategy for each convolutional layer is determined based on the number of accesses to the off-chip memory corresponding to each block strategy.

[0204] As an example, the neural network processing device also includes a third determining module 404 and a fourth determining module 405:

[0205] The third determining module 404 is used to determine multiple data repetition factors for each convolutional layer based on the convolutional layer parameters of each convolutional layer in at least one convolutional layer. The multiple data repetition factors include an input data repetition factor, a weight repetition factor, and an output data repetition factor. The input data repetition factor indicates the utilization rate of the input data of each convolutional layer, the weight repetition factor indicates the utilization rate of the weights of each convolutional layer, and the output data repetition factor indicates the utilization rate of the output data of each convolutional layer.

[0206] The fourth determining module 405 is used to determine the data reuse mode of each convolutional layer based on multiple data repetition factors of each convolutional layer. The data reuse mode of each convolutional layer includes the input data reuse mode corresponding to the input data repetition factor, the weight repetition mode corresponding to the weight repetition factor, and the output data reuse mode corresponding to the output data repetition factor.

[0207] It should be noted that the neural network processing device provided in the above embodiments is only an example of the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0208] The functional units and modules in the above embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of the embodiments of this application.

[0209] The neural network processing device and the neural network processing method provided in the above embodiments belong to the same concept. The specific working process and technical effects of the units and modules in the above embodiments can be found in the method embodiments section, and will not be repeated here.

[0210] Please refer to Figure 5 , Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 5 As shown, the computer device includes: a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable on the processor 501. When the processor 501 executes the computer program 503, it implements the steps in the neural network processing method in the above embodiments.

[0211] The computer device can be the computer device described in Embodiment 1 or Embodiment 3 above. The computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld computer, a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device. Those skilled in the art will understand that... Figure 5 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. They may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as input / output devices, network access devices, etc.

[0212] Processor 501 can be a Central Processing Unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0213] In some embodiments, memory 502 may be on-chip memory or off-chip memory of a computer device, such as cache memory, SRAM (Static Random-Access Memory), DRAM (Dynamic Static Random-Access Memory), or floppy disk. In other embodiments, memory 502 may be a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card provided on the computer device. Furthermore, memory 502 may include internal storage units of on-chip and off-chip memory of the computer device, as well as external storage devices. Memory 502 is used to store the operating system, applications, boot loader, data, and other programs. Memory 502 can also be used to temporarily store data that has been output or will be output.

[0214] This application also provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0215] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the various method embodiments described above.

[0216] This application provides a computer program product that, when run on a computer, causes the computer to perform the steps described in the various method embodiments above.

[0217] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above method embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate form. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographic device / computer device, a recording medium, a computer memory, ROM (Read-Only Memory), RAM (Random Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk, and optical data storage devices. The computer-readable storage medium mentioned in this application can be a non-volatile storage medium; in other words, it can be a non-transient storage medium.

[0218] It should be understood that all or part of the steps of the above embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented in whole or in part as a computer program product. The computer program product includes one or more computer instructions. The computer instructions can be stored in the above-described computer-readable storage medium.

[0219] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0220] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraint equations of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0221] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0222] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0223] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A neural network processing method, characterized in that, Applied to a computer device, the method includes: Obtain the bit width and on-chip memory capacity of the computer device, as well as the convolutional layer parameters of each convolutional layer in at least one convolutional layer of the neural network to be processed; Based on the convolutional layer parameters of each convolutional layer, a block parameter range corresponding to the first block sub-strategy of each convolutional layer is constructed. The first block sub-strategy includes the block width and block height of the input data, the block width and block height parameters of the weights, and the block depth of the output data. Based on the block parameter range corresponding to the first block sub-strategy of each convolutional layer, enumerate all the first block sub-strategies of each convolutional layer. Determine the second block strategy corresponding to each first block strategy in all first block strategies of each convolutional layer. The second block strategy includes the block depth of the input data, the block depth of the weights, and the block width and block height of the output data. All first block strategies of each convolutional layer and the second block strategy corresponding to each first block strategy are the block strategies of each convolutional layer. From all the block partitioning strategies of each convolutional layer, determine the block partitioning strategy that satisfies the constraints corresponding to the bit width and the storage capacity, and obtain multiple block partitioning strategies for each convolutional layer. A target block strategy is determined from multiple block strategies for each convolutional layer. The target block strategy is the block strategy that minimizes the number of accesses to off-chip memory among the multiple block strategies.

2. The method as described in claim 1, characterized in that, The step of determining the second block strategy corresponding to each of the first block strategies in all first block sub-strategies of each convolutional layer includes: For each convolutional layer, the target first block sub-policy among all first block sub-policies is determined by the following formula, where the target first block policy is any one of all first block policies for each convolutional layer: in, The second segmentation sub-strategy corresponding to the first segmentation sub-strategy for the target includes the segmentation depth of the input data and the segmentation depth of the weights, where str is the convolution stride of the neural network. The block depth of the output data included in the first block sub-strategy of the target is... The block width of the output data included in the second block sub-strategy corresponding to the first block sub-strategy of the target is... The first segmentation sub-strategy for the target includes the segmentation width of the input data. The block width is the weight of the first block sub-strategy for the target. The block height of the output data included in the second block sub-strategy corresponding to the first block sub-strategy of the target is... The first segmentation sub-strategy for the target includes the segmentation height of the input data. The block height includes the weights of the first block sub-strategy for the target.

3. The method as described in claim 1, characterized in that, The constraints that satisfy the bit width and the storage capacity include input constraints corresponding to the input data, weight constraints corresponding to the weights, and output constraints corresponding to the output data.

4. The method as described in claim 3, characterized in that, The input constraints are expressed by the following formula: in, The block partitioning strategy for each convolutional layer includes the block height of the input data. The block partitioning strategy for each convolutional layer includes the block width of the input data. The block strategy for each convolutional layer includes the block depth of the input data, where bit is the bit width and iBuff is the input storage capacity included in the storage capacity. The weight constraint condition is expressed by the following formula: in, The block partitioning strategy for each convolutional layer includes the block height of the weights. The block partitioning strategy for each convolutional layer includes the block width of the weights. The block partitioning strategy for each convolutional layer includes the block depth of the weights. The block depth of the output data of each convolutional layer's block strategy, bit is the bit width, and wBuff is the weight storage capacity included in the storage capacity; The output constraint condition is expressed by the following formula: in, The block size strategy for each convolutional layer includes the block height of the output data. The block size strategy for each convolutional layer includes the block width of the output data. The block strategy for each convolutional layer includes the block depth of the output data, where bit is the bit width and oBuff is the output storage capacity included in the storage capacity.

5. The method as described in claim 1, characterized in that, Determining the target block strategy from multiple block strategies for each convolutional layer includes: Determine the number of accesses to the off-chip memory corresponding to each of the multiple block partitioning strategies for each convolutional layer; The target block strategy for each convolutional layer is determined based on the number of accesses to the off-chip memory corresponding to each block strategy.

6. The method as described in claim 5, characterized in that, The step of determining the number of accesses to the off-chip memory corresponding to each of the multiple block-splitting strategies for each convolutional layer includes: For the first block strategy among multiple block strategies for each convolutional layer, the number of accesses to the off-chip memory corresponding to the first block strategy is determined by the following formula, where the first block strategy is any one of the multiple block strategies for each convolutional layer: in, This refers to the number of times the input data is accessed within the number of accesses to the off-chip memory corresponding to the first block partitioning strategy. The block height of the input data included in the first block segmentation strategy. The block width of the input data included in the first block segmentation strategy. The first segmentation strategy includes the segmentation depth of the input data and the segmentation depth of the weights. The number of memory chips on a memory column of the off-chip memory of the computer device. This refers to the weighted access count within the access count to the off-chip memory corresponding to the first block partitioning strategy. The block height is the weight of the weights included in the first block partitioning strategy. The block width is the weight included in the first block segmentation strategy. The block depth of the output data included in the first block strategy. This refers to the number of times the output data is accessed within the number of accesses to the off-chip memory corresponding to the first block partitioning strategy. The height of the output data blocks included in the first block segmentation strategy. The block width of the output data included in the first block segmentation strategy. This represents the number of accesses corresponding to the first block segmentation strategy.

7. The method according to any one of claims 1-6, characterized in that, After obtaining the bit width and on-chip memory capacity of the computer device, and the convolutional layer parameters of each convolutional layer in at least one convolutional layer of the neural network to be processed, the method further includes: Based on the convolutional layer parameters of each convolutional layer in the at least one convolutional layer, a plurality of data repetition factors are determined for each convolutional layer. The plurality of data repetition factors include an input data repetition factor, a weight repetition factor, and an output data repetition factor. The input data repetition factor indicates the utilization rate of the input data of each convolutional layer, the weight repetition factor indicates the utilization rate of the weights of each convolutional layer, and the output data repetition factor indicates the utilization rate of the output data of each convolutional layer. The data reuse pattern of each convolutional layer is determined based on multiple data repetition factors of each convolutional layer. The data reuse pattern of each convolutional layer includes an input data reuse pattern corresponding to the input data repetition factor, a weight repetition pattern corresponding to the weight repetition factor, and an output data reuse pattern corresponding to the output data repetition factor.

8. The method as described in claim 7, characterized in that, The step of determining multiple data repetition factors for each convolutional layer based on the convolutional layer parameters of each of the at least one convolutional layer includes: For the convolutional parameters of the target convolutional layer in the at least one convolutional layer, multiple data repetition factors of the target convolutional layer are determined by the following formula, wherein the target convolutional layer is any one of the at least one convolutional layers: in, The input data repetition factor, The convolutional layer parameters of the target convolutional layer include the weight height parameter; The convolutional layer parameters of the target convolutional layer include the weight width parameter. The convolutional layer parameters for the target convolutional layer include the output depth parameter, and str is the convolutional stride of the neural network. The weighted repetition factor, The convolutional layer parameters for the target convolutional layer include the input height parameter. The convolutional layer parameters for the target convolutional layer include the input width parameter. The output data repetition factor is... The convolutional layer parameters for the target convolutional layer include the input depth parameters and the weight depth parameters.

9. A neural network processing device, characterized in that, The device includes an acquisition module, a first determination module, and a second determination module; The acquisition module is used to acquire the bit width and on-chip memory capacity of the computer device, as well as the convolutional layer parameters of each convolutional layer in at least one convolutional layer of the neural network to be processed. The first determining module is configured to construct a block parameter range corresponding to a first block sub-strategy for each convolutional layer based on the convolutional layer parameters of each convolutional layer. The first block sub-strategy includes the block width and block height of the input data, the block width and block height parameters of the weights, and the block depth of the output data. Based on the block parameter range corresponding to the first block sub-strategy of each convolutional layer, the module enumerates all first block sub-strategies for each convolutional layer. The module then determines a second block sub-strategy corresponding to each first block sub-strategy among all the first block sub-strategies of each convolutional layer. The second block sub-strategy includes the block depth of the input data, the block depth of the weights, and the block width and block height of the output data. All the first block sub-strategies of each convolutional layer and the second block sub-strategy corresponding to each first block sub-strategy constitute all the block strategies of each convolutional layer. From all the block partitioning strategies of each convolutional layer, determine the block partitioning strategy that satisfies the constraints corresponding to the bit width and the storage capacity, and obtain multiple block partitioning strategies for each convolutional layer. The second determining module is used to determine a target block strategy from multiple block strategies of each convolutional layer. The target block strategy is the block strategy that has the minimum number of accesses to off-chip memory among the multiple block strategies.

10. A computer device, characterized in that, The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 8.

Images