A neural network pruning method, device, equipment and medium

Abstract

The present invention discloses a neural network pruning method, apparatus, device, and medium, comprising: obtaining the total number of random access memories based on the total number of rows of a pre-configured PE array and a preset number of screening channels; configuring a plurality of random access memories based on the total number, and storing the input feature map data of each channel in each of the random access memories; grouping the plurality of random access memories to obtain a plurality of small groups of random access memories; and upon receiving a pruning signal, selecting the input feature map data based on a channel selection signal, and inputting the input feature map data of different channels within each small group into the PE array to implement a neural network pruning operation. The embodiments of the present invention can improve the processing efficiency of neural network models.

Description

Technical Field

[0001] The present invention relates to the technical field of neural network pruning, and in particular to a neural network pruning method, device, equipment and medium. Background Art

[0002] In recent years, neural networks have been widely used in fields such as computer vision, machine translation, and speech recognition. However, with the development of neural network technology, the design of its network structure has become increasingly complex. The weight matrix corresponding to the convolution kernel also contains an increasing number of weights, which increases the computational workload and parameter count of the neural network, making it difficult to deploy neural networks on hardware resources with limited computing power and storage space. Therefore, it is necessary to study a neural network pruning method to prune the neural network. Summary of the Invention

[0003] The present invention provides a neural network pruning method, device, equipment and medium, which can improve the processing efficiency of the neural network model.

[0004] To achieve the above objectives, an embodiment of the present invention provides a neural network pruning method, comprising:

[0005] Obtaining the total number of random access memories according to the total number of rows of a pre-configured PE array and the preset number of screening channels;

[0006] configuring a plurality of random access memories according to the total number, and storing the input feature map data of each channel in each of the random access memories respectively;

[0007] Grouping the plurality of random access memories to obtain a plurality of small groups of random access memories; wherein each small group of random access memories stores input feature map data of N / 2^p channels, where N is the total number of rows of the PE array, 2^p is the number of screening channels, and p≥1, where p is an integer;

[0008] When a pruning signal is received, the input feature map data is selected according to the channel selection signal, and the input feature map data of different channels in each group are input into the PE array to implement the pruning operation of the neural network.

[0009] As an improvement to the above solution, the neural network pruning method further includes:

[0010] When a non-pruning signal is received, the input feature map data is input into a row of the PE array corresponding to the number of channels according to the number of channels of the input feature map data.

[0011] As an improvement to the above solution, obtaining the number of random access memories according to the total number of rows of the pre-configured PE array and the preset number of screening channels includes:

[0012] The number of random access memories is obtained by multiplying the total number of rows of the pre-configured PE array and the preset number of screening channels.

[0013] As an improvement to the above solution, the random access memories are grouped to obtain a plurality of small groups of random access memories, including:

[0014] Dividing the plurality of random access memories into a plurality of large groups of random access memories according to the total number of rows; wherein each large group of random access memories includes random access memories having a number equal to the total number of rows;

[0015] For each large group of random access memories, the remainder obtained by dividing the number of channels of the input feature map data by the number of screening channels is used as a basis for grouping, and each large group of random access memories is divided into several small groups of random access memories.

[0016] As an improvement to the above solution, the neural network pruning method further includes:

[0017] Obtain the channel of the pruned input feature map data, and select the same convolution kernel data as the channel for compression.

[0018] As an improvement to the above solution, the neural network pruning method further includes:

[0019] The pruned input feature map data is input into the pre-configured PE array for convolution calculation.

[0020] To achieve the above objectives, the present invention further provides an apparatus for pruning a neural network, including a controller, wherein the controller is configured to:

[0021] Obtaining the total number of random access memories according to the total number of rows of a pre-configured PE array and the preset number of screening channels;

[0022] configuring a plurality of random access memories according to the total number, and storing the input feature map data of each channel in each of the random access memories respectively;

[0023] Grouping the plurality of random access memories to obtain a plurality of small groups of random access memories; wherein each small group of random access memories stores input feature map data of N / 2^p channels, where N is the total number of rows of the PE array, 2^p is the number of screening channels, and p≥1, where p is an integer;

[0024] When a pruning signal is received, the input feature map data is selected according to the channel selection signal, and the input feature map data of different channels in each group are input into the PE array to implement the pruning operation of the neural network.

[0025] To achieve the above objectives, an embodiment of the present invention also provides a neural network pruning device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the neural network pruning method as described above when executing the computer program.

[0026] To achieve the above-mentioned purpose, an embodiment of the present invention further provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute the neural network pruning method as described above.

[0027] Compared with the prior art, the embodiments of the present invention provide a neural network pruning method, apparatus, device and medium, which obtain the total number of random access memories based on the total number of rows of a pre-configured PE array and the preset number of screening channels; configure a number of random access memories based on the total number, and store the input feature map data of each channel in each of the random access memories respectively; group the random access memories to obtain a number of small groups of random access memories; when a pruning signal is received, the input feature map data is selected according to the channel selection signal, and the input feature map data of different channels in each small group is input into the PE array, thereby realizing the pruning function of the neural network and improving the processing efficiency of the neural network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a flowchart of a neural network pruning method provided by an embodiment of the present invention;

[0029] Figure 2 It is a schematic diagram of the prior art of transmitting input feature data to the PE array;

[0030] Figure 3 Schematic diagram of transmitting input feature map data to a PE array when pruning is performed according to an embodiment of the present invention;

[0031] Figure 4 Schematic diagram of transmitting input feature map data to a PE array when no pruning is performed according to an embodiment of the present invention;

[0032] Figure 5 This is a structural block diagram of a neural network pruning device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0034] See also Figure 1 , Figure 1 1 is a flowchart of a neural network pruning method provided by an embodiment of the present invention, wherein the neural network pruning method includes:

[0035] S1. Obtain the total number of random access memories according to the total number of rows of the pre-configured PE array and the preset number of screening channels;

[0036] Specifically, obtaining the number of random access memories according to the total number of rows of the pre-configured PE array and the preset number of screening channels includes:

[0037] The number of random access memories is obtained by multiplying the total number of rows of the pre-configured PE array and the preset number of screening channels.

[0038] S2. configuring a plurality of random access memories according to the total number, and storing the input feature map data of each channel into each of the random access memories respectively;

[0039] Preferably, the random access memory is a block random access memory (BRAM).

[0040] S3. Grouping the random access memories to obtain a plurality of small groups of random access memories; wherein each small group of random access memories stores input feature map data of N / 2^p channels, where N is the total number of rows of the PE array, 2^p is the number of screening channels, and p≥1, where p is an integer;

[0041] The random access memories are grouped to obtain a plurality of small groups of random access memories, including:

[0042] Dividing the plurality of random access memories into a plurality of large groups of random access memories according to the total number of rows; wherein each large group of random access memories includes random access memories having a number equal to the total number of rows;

[0043] For each large group of random access memories, the remainder obtained by dividing the number of channels of the input feature map data by the number of screening channels is used as a basis for grouping, and each large group of random access memories is divided into several small groups of random access memories.

[0044] S4. When a pruning signal is received, the input feature map data is selected according to the channel selection signal, and the input feature map data of different channels in each group are input into the PE array to implement the pruning operation of the neural network.

[0045] It is worth noting that by decoding the neural network, it is possible to determine whether the neural network needs pruning.

[0046] It can be understood that when performing a pruning operation, the embodiment of the present invention can read out the input feature map data of all groups in the same clock cycle.

[0047] Considering that the working mode of PE array in most accelerators is similar to or equivalent to that of systolic array, when there is no pruning function, the number of storage groups of input feature map data should be equal to the number of rows of PE array to meet the demand of input feature map data when PE systolic array is working. The data storage method and the method of transmitting input feature data to PE systolic array are as follows: Figure 2 As shown, the input feature map data of each row comes from different RAMs, and the input feature map data of each row in the PE array is the same.

[0048] In the embodiment of the present invention, pruning of input feature map data is supported, and input feature map data that does not require pruning is also compatible, and its storage method is as follows: Figure 3 As shown, when p=1 and N=8, it needs to be stored in =N*2^p=16 RAMs.

[0049] Divide N*2^p RAM into 2 p *2 p Small groups, that is, the input feature map data of each N channels is stored as a large group. For each large group, the number of channels is divided by 2 p The remainder, then divide each large group into 2 p groups, each group stores N / 2 p The input feature map data of channels.

[0050] When p=1 and N=8, for pruned data, during the storage of input feature map data, the lower 8 channels and upper 8 channels are stored in large groups, and within each large group, the data is divided into two small groups based on the remainder of the channel number (0, 1). When the pruning signal is received and the pruning operation begins, the input feature map data of each channel in each small group is input into the corresponding row of the PE array according to the channel selection signal during the model training process. That is, the data is selected and the input feature map data is compressed from 16 channels to 8 channels. After the selection, the input feature map data is calculated in the PE array.

[0051] In another preferred embodiment, the neural network pruning method further includes:

[0052] When a non-pruning signal is received, the input feature map data is input into a row of the PE array corresponding to the number of channels according to the number of channels of the input feature map data.

[0053] For example, when no pruning operation is required, 2 p *N RAM will store 2 p The input feature map data of N channels is stored in the form of data and input to the PE array. Figure 4 As shown in Figure 1, the input feature map data is transmitted to the corresponding row of the PE array according to the channel number of the stored data. At this time, the 8 RAMs that stored the upper 8 channel data during the pruning process will store the input feature map data of the 8 channels of the next point.

[0054] In another preferred embodiment, the neural network pruning method further comprises:

[0055] Obtain the channel of the pruned input feature map data, and select the same convolution kernel data as the channel for compression.

[0056] In this embodiment of the present invention, the input feature map data in the PE array is broadcast to the M convolution kernels. That is, the input feature map data of the PE array with coordinates (N, 0) to (N, M) is the same. To meet this working mode of the PE array, the channels selected for the input feature map data should be consistent for each of the M convolution kernels. If the channels of the input feature data need to be compressed, during model training, for each of the M convolution kernels, the convolution kernel data compression process should select the same channels as the pruned input feature map data.

[0057] In another preferred embodiment, the neural network pruning method further comprises:

[0058] The pruned input feature map data is input into the pre-configured PE array to perform the convolution calculation of the current convolution layer.

[0059] It can be understood that in the PE array, for the data at the same point, the product of the input feature map data and the weight data of one channel is calculated in each cycle and accumulated with the calculation result of the previous channel. That is, after M clock cycles, the result of the input feature data and weight data of a certain point is calculated.

[0060] A neural network pruning method provided by an embodiment of the present invention obtains the total number of random access memories based on the total number of rows of a pre-configured PE array and a preset number of screening channels; configures a number of random access memories based on the total number, and stores the input feature map data of each channel in each of the random access memories; groups the random access memories to obtain a number of small groups of random access memories; and when a pruning signal is received, selects the input feature map data based on a channel selection signal, and inputs the input feature map data of different channels in each small group into the PE array, thereby realizing the pruning function of the neural network and improving the processing efficiency of the neural network model.

[0061] An embodiment of the present invention provides a neural network pruning device, including a controller, wherein the controller is configured to:

[0062] Obtaining the total number of random access memories according to the total number of rows of a pre-configured PE array and the preset number of screening channels;

[0063] configuring a plurality of random access memories according to the total number, and storing the input feature map data of each channel in each of the random access memories respectively;

[0064] Grouping the plurality of random access memories to obtain a plurality of small groups of random access memories; wherein each small group of random access memories stores input feature map data of N / 2^p channels, where N is the total number of rows of the PE array, 2^p is the number of screening channels, and p≥1, where p is an integer;

[0065] When a pruning signal is received, the input feature map data is selected according to the channel selection signal, and the input feature map data of different channels in each group are input into the PE array to implement the pruning operation of the neural network.

[0066] Preferably, the neural network pruning method further includes:

[0067] When a non-pruning signal is received, the input feature map data is input into a row of the PE array corresponding to the number of channels according to the number of channels of the input feature map data.

[0068] Preferably, obtaining the number of random access memories according to the total number of rows of a preconfigured PE array and the preset number of screening channels includes:

[0069] The number of random access memories is obtained by multiplying the total number of rows of the pre-configured PE array and the preset number of screening channels.

[0070] Preferably, the random access memories are grouped to obtain a plurality of small groups of random access memories, including:

[0071] Dividing the plurality of random access memories into a plurality of large groups of random access memories according to the total number of rows; wherein each large group of random access memories includes random access memories having a number equal to the total number of rows;

[0072] For each large group of random access memories, the remainder obtained by dividing the number of channels of the input feature map data by the number of screening channels is used as a basis for grouping, and each large group of random access memories is divided into several small groups of random access memories.

[0073] Preferably, the neural network pruning method further includes:

[0074] Obtain the channel of the pruned input feature map data, and select the same convolution kernel data as the channel for compression.

[0075] Preferably, the neural network pruning method further includes:

[0076] The pruned input feature map data is input into the pre-configured PE array for convolution calculation.

[0077] It is worth noting that the working process of the neural network pruning device described in the embodiment of the present invention can refer to the working process of the neural network pruning method described in the above embodiment, and will not be repeated here.

[0078] A neural network pruning device provided by an embodiment of the present invention realizes the pruning function of the neural network by selecting the channels of input feature data before inputting the input feature map data into the PE array, thereby improving the processing efficiency of the neural network model.

[0079] See also Figure 5 , Figure 5 1 is a block diagram of a neural network pruning device 20 provided in an embodiment of the present invention. The neural network pruning device 20 includes a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, the steps of the aforementioned neural network pruning method embodiments are implemented. Alternatively, when the processor 21 executes the computer program, the functions of the modules / units in the aforementioned apparatus embodiments are implemented.

[0080] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program in the neural network pruning device 20.

[0081] The neural network pruning device 20 can be a computing device such as a desktop computer, laptop, PDA, or cloud server. The neural network pruning device 20 can include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will appreciate that the schematic diagram is merely an example of the neural network pruning device 20 and does not limit the neural network pruning device 20. The neural network pruning device 20 can include more or fewer components than shown, or a combination of certain components, or different components. For example, the neural network pruning device 20 can also include input and output devices, network access devices, buses, and the like.

[0082] The processor 21 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor. The processor 21 is the control center of the neural network pruning device 20, and utilizes various interfaces and lines to connect various parts of the neural network pruning device 20.

[0083] The memory 22 can be used to store the computer programs and / or modules. The processor 21 implements the various functions of the neural network pruning device 20 by running or executing the computer programs and / or modules stored in the memory 22 and accessing the data stored in the memory 22. The memory 22 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area may store data generated based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory 22 may include high-speed random access memory and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0084] If the modules / units integrated in the neural network pruning device 20 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the above-mentioned embodiment methods by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor 21, it can implement the steps of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium.

[0085] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement the present invention without inventive effort.

[0086] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is run, it controls the device where the computer-readable storage medium is located to execute the neural network pruning method described in the above embodiment.

[0087] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A neural network pruning method, characterized in that: include: Obtaining the total number of random access memories according to the total number of rows of a pre-configured PE array and the preset number of screening channels; configuring a plurality of random access memories according to the total number, and storing the input feature map data of each channel in each of the random access memories respectively; Grouping the plurality of random access memories to obtain a plurality of small groups of random access memories; wherein each small group of random access memories stores input feature map data of N / 2^p channels, where N is the total number of rows of the PE array, 2^p is the number of screening channels, and p≥1, where p is an integer; When a pruning signal is received, the input feature map data is selected according to the channel selection signal, and the input feature map data of different channels in each group are input into the PE array to implement the pruning operation of the neural network; The step of grouping the random access memories to obtain a plurality of small groups of random access memories includes: Dividing the plurality of random access memories into a plurality of large groups of random access memories according to the total number of rows; wherein each large group of random access memories includes random access memories having a number equal to the total number of rows; For each large group of random access memories, the remainder obtained by dividing the number of channels of the input feature map data by the number of screening channels is used as a basis for grouping, and each large group of random access memories is divided into several small groups of random access memories.

2. The neural network pruning method according to claim 1, wherein: The neural network pruning method further includes: When a non-pruning signal is received, the input feature map data is input into a row of the PE array corresponding to the number of channels according to the number of channels of the input feature map data.

3. The neural network pruning method according to claim 1, wherein: The step of obtaining the number of random access memories according to the total number of rows of the pre-configured PE array and the preset number of screening channels includes: The number of random access memories is obtained by multiplying the total number of rows of the pre-configured PE array and the preset number of screening channels.

4. The neural network pruning method according to claim 1, wherein: The neural network pruning method further includes: Obtain the channel of the pruned input feature map data, and select the same convolution kernel data as the channel for compression.

5. The neural network pruning method according to claim 1, wherein: The neural network pruning method further includes: The pruned input feature map data is input into the pre-configured PE array for convolution calculation.

6. A neural network pruning device, characterized in that: comprising a controller configured to: Obtaining the total number of random access memories according to the total number of rows of a pre-configured PE array and the preset number of screening channels; configuring a plurality of random access memories according to the total number, and storing the input feature map data of each channel in each of the random access memories respectively; Grouping the plurality of random access memories to obtain a plurality of small groups of random access memories; wherein each small group of random access memories stores input feature map data of N / 2^p channels, where N is the total number of rows of the PE array, 2^p is the number of screening channels, and p≥1, where p is an integer; When a pruning signal is received, the input feature map data is selected according to the channel selection signal, and the input feature map data of different channels in each group are input into the PE array to implement the pruning operation of the neural network; The step of grouping the random access memories to obtain a plurality of small groups of random access memories includes: Dividing the plurality of random access memories into a plurality of large groups of random access memories according to the total number of rows; wherein each large group of random access memories includes random access memories having a number equal to the total number of rows; For each large group of random access memories, the remainder obtained by dividing the number of channels of the input feature map data by the number of screening channels is used as a basis for grouping, and each large group of random access memories is divided into several small groups of random access memories.

7. A neural network pruning device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the neural network pruning method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program; wherein, when the computer program is run, it controls the device where the computer-readable storage medium is located to execute the neural network pruning method according to any one of claims 1 to 5.

Images