Neural network calculation method, electronic device, and computer readable medium
By dividing the weight matrix into multiple groups and storing only non-zero weight values and positions, and using multiplier and shifter for calculation, the problem of weight matrix calculation in neural network occupies a large amount of resources, achieving more efficient calculation and storage .
Patent Information
- Application Number
- CN202011499974.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-17
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2040-12-17
AI Technical Summary
In neural networks, the calculation of the weight matrix occupies a large amount of computing resources and memory space, resulting in a large computing overhead.
Divide the weight matrix into multiple groups and only stores non-zero weight values and positions. Multiply and accumulate operations through multipliers and shifters to reduce calculation and storage requirements.
The "compression encoding" of the weight matrix is realized, reducing the memory space and calculation amount and improving the computing efficiency.
Smart Images

Figure CN114648088B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of neural network technology, and in particular to a neural network calculation method, electronic device, and computer-readable medium. Background Art
[0002] In the process of using neural networks, the calculation of weight matrices occupies a large amount of computation. For example, in a spiking neural network based on the LIF (Leaky Integrity-and-Fire) model, the synaptic integral calculation of the weight matrix is the main load.
[0003] Therefore, the calculation of the weight matrix requires a large amount of memory space, access bandwidth, computing resources, etc., resulting in a large computational overhead. Summary of the invention
[0004] The present disclosure provides a neural network computing method, an electronic device, and a computer-readable medium.
[0005] In a first aspect, an embodiment of the present disclosure provides a method for neural network calculation, which includes:
[0006] The weight matrix is divided into multiple groups, and the non-zero weights in each group are stored in corresponding memory slices; wherein the weight matrix is a connection weight matrix of neurons; each group includes multiple consecutive columns of the weight matrix; the memory slices correspond to the groups one by one, and the values and positions of the non-zero weights are stored in the memory slices;
[0007] Repeat the following multiplication operation: input the value of the non-zero weight in each memory slice to the corresponding multiplier, and input the corresponding data in the input data to each multiplier;
[0008] The multiplication results output by each multiplier in each multiplication operation are shifted according to the position of the corresponding non-zero weight, and the multiplication results at the same position after the shift are accumulated to calculate the output data.
[0009] In some embodiments, the neural network is a spiking neural network, and the weight matrix is a synaptic connection weight matrix.
[0010] In some embodiments, before dividing the weight matrix into multiple groups, the method further includes:
[0011] Swap the positions of some columns in the weight matrix;
[0012] and / or,
[0013] Swap the positions of some rows in the weight matrix.
[0014] In some embodiments, the sparsity of the weight matrix is greater than or equal to 50%.
[0015] In some embodiments, the difference in sparsity between any two groups does not exceed 10%.
[0016] In some embodiments, the number of columns in any group is the same.
[0017] In some embodiments, the shifting of the multiplication results output by each multiplier in each multiplication operation according to the position of the corresponding non-zero weight comprises: inputting the multiplication results output by each multiplier in each multiplication operation into the corresponding shifter, and inputting the position of each corresponding non-zero weight into the corresponding shifter; the shifter shifts the multiplication results according to the corresponding position;
[0018] Accumulating the multiplication results at the same position after shifting includes: inputting the output of each shifter to the corresponding accumulator respectively, and the accumulator accumulates the multiplication results at the same position.
[0019] In some embodiments, the inputting the values of non-zero weights in each memory slice into the corresponding multiplier includes: the memory buffer acquires the non-zero weights in the corresponding memory slice, and inputs the values of the non-zero weights into the corresponding multiplier;
[0020] Inputting the positions of the corresponding non-zero weights to the corresponding shifters respectively comprises: the memory buffer acquires the non-zero weights in the corresponding memory slice, and inputting the positions of the non-zero weights to the corresponding shifters;
[0021] The method of inputting the corresponding data in the input data to each multiplier includes: the memory buffer obtains the non-zero weights in the corresponding memory chip, determines the number of non-zero weights in each row of the corresponding group according to the position of each non-zero weight, and inputs the number of non-zero weights in each row to the multi-channel buffer; the multi-channel buffer outputs the data corresponding to each row in the input data to the corresponding multiplier the same number of times according to the number of non-zero weights in each row of each group.
[0022] In some embodiments, the position of the non-zero weight is the offset of the non-zero weight in the group to which it belongs.
[0023] In some embodiments, the multiplication operation further comprises:
[0024] When a preset pause condition is met, the multi-channel buffer pauses acquiring input data, and / or the multi-channel buffer controls at least part of the memory buffer to pause operation.
[0025] In a second aspect, an embodiment of the present disclosure provides an electronic device, including:
[0026] A processing unit, the processing unit comprising a plurality of memory slices and a plurality of multipliers corresponding one to one with the memory slices;
[0027] a storage unit having a computer program stored thereon;
[0028] When the computer program is executed by the processing unit, any one of the above-mentioned neural network calculation methods is implemented.
[0029] In a third aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, wherein the computer program implements any of the above-mentioned neural network calculation methods when executed by a processor.
[0030] In the embodiment of the present disclosure, only the non-zero weights of the weight matrix are stored in the memory, which is equivalent to realizing "compressed encoding" of the weight matrix, thereby reducing the memory space and access bandwidth required for calculation; at the same time, the embodiment of the present disclosure actually only calculates the non-zero weights, thereby reducing the required amount of calculation, further reducing the calculation overhead, and improving calculation efficiency.
[0031] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings, in which:
[0033] Figure 1 A flowchart of a neural network calculation method provided by an embodiment of the present disclosure;
[0034] Figure 2 A flowchart of another neural network calculation method provided by an embodiment of the present disclosure;
[0035] Figure 3 A schematic diagram of a grouping process in a neural network calculation method provided by an embodiment of the present disclosure;
[0036] Figure 4 A schematic diagram of a storage format of non-zero weights in a neural network calculation method provided by an embodiment of the present disclosure;
[0037] Figure 5 A schematic diagram of a process in which a multi-channel buffer obtains data from an input buffer in a neural network calculation method provided by an embodiment of the present disclosure;
[0038] Figure 6 A block diagram of an electronic device provided in an embodiment of the present disclosure;
[0039] Figure 7 A schematic diagram of the architecture of a processing unit in an electronic device provided by an embodiment of the present disclosure;
[0040] Figure 8 A block diagram of a computer-readable medium provided for an embodiment of the present disclosure. DETAILED DESCRIPTION
[0041] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0042] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.
[0043] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.
[0044] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "including" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.
[0045] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.
[0046] First, refer to Figures 1 to 5 , Figure 7 , an embodiment of the present disclosure provides a method for neural network calculation.
[0047] The method of the embodiment of the present disclosure is used to perform calculations in a neural network.
[0048] Among them, neural networks can realize any pre-trained functions, such as image recognition, action recognition, speech recognition, text recognition, etc.
[0049] The embodiment of the present disclosure is used to calculate the product of the input data of the neurons input into the neural network and the corresponding connection weight matrix. That is, in the neural network, neurons in different layers are connected to each other, and the connections between different neurons have different weight matrices, so the input data input into a neuron needs to be multiplied by the weight matrix of the neuron.
[0050] In some embodiments, the neural network is a spiking neural network and the weight matrix is a synaptic connection weight matrix.
[0051] As one embodiment of the present disclosure, the applicable neural network may be a spiking neural network (SNN), whereby the weight matrix therein is a weight matrix corresponding to the synaptic connections between neurons.
[0052] Exemplarily, the above pulse neural network may be based on a LIF (Leaky Integrity-and-Fire) model.
[0053] In the LIF model, the pulse current input signal (Xi) of the previous neuron i is connected to the current neuron through the weight matrix (Wi). When the membrane voltage (Vt) of the current neuron accumulates (integrates) due to the charging of the input current signals of all previous neurons and reaches a predetermined threshold (Vth), the current neuron will send a pulse current signal and reset the membrane voltage (Vt) to the resting voltage (Vreset).
[0054] Of course, the application scope of the embodiment of the present disclosure is not limited to this. It can also be used for pulse neural networks of other modes, or for weight matrix calculation in other types of neural networks (such as convolutional neural networks, etc.).
[0055] In some embodiments, the sparsity of the weight matrix is greater than or equal to 50%.
[0056] The embodiments of the present disclosure are suitable for processing sparse weight matrices, for example, weight matrices with a sparsity (i.e., the proportion of zero elements) of at least 50%, or at least 70%, or at least 90%, or at least 95%.
[0057] Among them, the specific sparsity of the weight matrix that can be processed by the embodiments of the present disclosure can be determined according to actual computing resources (hardware resources).
[0058] Reference Figure 1 , the method of the embodiment of the present disclosure includes:
[0059] S101, dividing the weight matrix into multiple groups, and storing the non-zero weights in each group in a corresponding memory slice.
[0060] The weight matrix is a connection weight matrix of neurons; each group includes a plurality of consecutive columns of the weight matrix; and the memory slices correspond to the groups one by one, in which the values and positions of non-zero weights are stored.
[0061] S102, repeatedly performing the following multiplication operation: inputting the non-zero weight value in each memory slice into the corresponding multiplier respectively, and inputting the corresponding data in the input data into each multiplier respectively.
[0062] S103, shifting the multiplication results output by each multiplier in each multiplication operation according to the position of the corresponding non-zero weight, and accumulating the multiplication results located at the same position after the shift to calculate the output data.
[0063] Reference Figure 3 In the disclosed embodiment, before calculating the weight matrix, the weight matrix is first grouped vertically so that each group includes multiple consecutive columns of the weight matrix; then, all non-zero weights in each group are stored in a corresponding memory slice, and the memory slice stores the value (the "numerical value") and position (the "relative position" or "offset" of the non-zero weight in the row, group, or weight matrix) of the non-zero weight.
[0064] Afterwards, the value of the non-zero weight can be obtained from each memory chip and input into the corresponding multiplier. When a non-zero weight value is input into the multiplier, the corresponding data in the input data is also input into the multiplier at the same time. By repeatedly performing the above multiplication operation, each non-zero weight can be processed in turn, so that each multiplier outputs the numerical value (multiplication result) of the calculation result of each non-zero weight in the corresponding memory chip in turn.
[0065] Reference Figure 7 The multiplication result output by each multiplier only represents the numerical value of the corresponding calculation result, but not the position where the result should be located. Therefore, it is necessary to shift the multiplication result to the corresponding position according to the position of the corresponding non-zero weight (the offset of the corresponding non-zero weight in the group and row), and then add up all the multiplication results at the same position (equivalent to summing up the calculation results of each row). After splicing and processing, the final result (output data) of the weight matrix calculation can be obtained.
[0066] Obviously, the result obtained by calculating zero weight must also be zero, which is actually meaningless. Therefore, in the embodiment of the present disclosure, only the non-zero weights of the weight matrix are stored in the memory, which is equivalent to realizing "compression encoding" of the weight matrix, thereby reducing the storage amount and access bandwidth required in the calculation, and reducing the computing overhead.
[0067] Moreover, in the embodiment of the present disclosure, only the "non-zero weight" is actually calculated, and the "zero weight" is not calculated, thereby greatly reducing the total amount of calculation and further reducing the calculation overhead.
[0068] At the same time, in the embodiment of the present disclosure, it is equivalent to processing different groups of data in different "channels", and each channel is only used to process part of the data of the weight matrix, so the calculation overhead can be further reduced and the calculation efficiency can be improved.
[0069] In some embodiments, the difference in sparsity between any two groups does not exceed 10%.
[0070] Reference Figure 3 (In the figure, black squares represent non-zero weights, and colorless squares represent zero weights). After grouping the weight matrix, the number of non-zero weights in each group should be "similar" to ensure that the processing progress of each channel is basically the same.
[0071] Specifically, each group has a certain sparsity, and among the multiple groups divided by the weight matrix, the difference in sparsity between the group with the largest sparsity and the group with the smallest sparsity should not exceed 10%, and further should not exceed 5%.
[0072] Of course, refer to Figure 3 , it is also feasible if all groups have the same sparsity (the number of non-zero weights is the same).
[0073] Of course, the specific allowable range of the difference in sparsity (the difference in the number of non-zero weights) in different groups can be determined according to actual computing resources (hardware resources).
[0074] In some embodiments, before dividing the weight matrix into multiple groups (S101), the method further includes:
[0075] S100, swapping positions of some columns in the weight matrix; and / or swapping positions of some rows in the weight matrix.
[0076] As mentioned above, the disclosed embodiment hopes that the number of non-zero weights of each group after grouping is as "similar" as possible, that is, the distribution of non-zero weights in the weight matrix is required to be basically "uniform", but the weight matrix in the neural network may not all meet this requirement. Therefore, for the weight matrix that does not meet the requirements, some of the columns and column positions can be exchanged to make the distribution of non-zero weights in the weight matrix as "uniform" as possible.
[0077] Among them, by exchanging the positions of some rows in the weight matrix, the groups to which some non-zero weights belong can be changed, so that the exchanged weight matrix can more easily achieve the above grouping.
[0078] Among them, by exchanging the positions of some rows in the weight matrix, the processing progress of each channel can be basically the same in each time period.
[0079] Of course, when the columns are swapped, the corresponding data in the result of the weight calculation should also be swapped before they can be actually output as the final output data.
[0080] Of course, when the row positions are interchanged, the order of the data in the corresponding input data must also be exchanged accordingly.
[0081] Of course, in order to obtain a weight matrix that can perform the above grouping (a weight matrix with uniform non-zero weight distribution), the weight matrix can also be restricted during the training process of the neural network.
[0082] Exemplarily, during the training process, the number and distribution of non-zero weights in the weight matrix may be restricted.
[0083] Exemplarily, during the training process, mask constraints may be used to ensure that the number and distribution of non-zero weights in the weight matrix meet requirements.
[0084] In some embodiments, the number of columns in any group is the same.
[0085] As one embodiment of the present disclosure, the number of columns in all groups may be the same. Figure 3 , all groups have 4 columns.
[0086] Of course, the specific number of columns in each group is not limited thereto, and can be determined according to the situation of the weight matrix. For example, the number of columns in each group can be at least 2, or at least 4, or at least 10, etc.; for another example, the number of columns in each group can be no more than 20, or no more than 10, or no more than 4, etc.
[0087] In some embodiments, shifting the multiplication results output by each multiplier in each multiplication operation according to the position of the corresponding non-zero weight includes: inputting the multiplication results output by each multiplier in each multiplication operation into the corresponding shifter, and inputting the position of each corresponding non-zero weight into the corresponding shifter; the shifter shifts the multiplication results according to the corresponding position;
[0088] Accumulating the multiplication results at the same position after shifting includes: inputting the output of each shifter to the corresponding accumulator respectively, and the accumulator accumulates the multiplication results at the same position.
[0089] As one embodiment of the present disclosure, each channel also includes a shifter and an accumulator, so that after the output of the multiplier and the position of the non-zero weight are input into the shifter, the shifter can be used for shifting, and the output of the shifter enters the corresponding accumulator and is accumulated by the accumulator.
[0090] In some embodiments, inputting the values of non-zero weights in each memory slice to the corresponding multiplier includes: the memory buffer obtains the non-zero weights in the corresponding memory slice, and inputs the values of the non-zero weights to the corresponding multiplier;
[0091] Inputting the positions of the corresponding non-zero weights to the corresponding shifters respectively includes: the memory buffer acquires the non-zero weights in the corresponding memory slices, and inputting the positions of the non-zero weights to the corresponding shifters;
[0092] Inputting the corresponding data in the input data to each multiplier respectively includes: the memory buffer obtains the non-zero weights in the corresponding memory chip, determines the number of non-zero weights in each row of the corresponding group according to the position of each non-zero weight, and inputs the number of non-zero weights in each row to the multi-channel buffer; the multi-channel buffer outputs the data corresponding to each row in the input data to the corresponding multiplier the same number of times according to the number of non-zero weights in each row of each group.
[0093] As one embodiment of the present disclosure, refer to Figure 7 ,Each channel is also provided with a memory buffer, which can obtain the non-zero weights in the memory slice, and input the value of the non-zero weight to the corresponding multiplier, and input the position of the non-zero weight to the shifter.
[0094] At the same time, the memory buffer also needs to determine the number of non-zero weights in each row of the corresponding group according to the position of the non-zero weights therein, and send the number of non-zero weights in each row to the multi-channel buffer (such as the corresponding channel of the multi-channel buffer).
[0095] Obviously, the number of non-zero weights in a row of a group is equal to the number of times the data corresponding to the row in the input data needs to be used by the corresponding multiplier; for this reason, the multi-channel buffer can output the data corresponding to each row in the input data to the corresponding multiplier the same number of times according to the number of non-zero weights in each row of each group.
[0096] For example, if there is 0 non-zero weight in a row of a group, it means that the row has no non-zero weight to be input to the corresponding multiplier, so the multi-channel buffer can input the data corresponding to the row in the input data to the corresponding multiplier 0 times (that is, no input).
[0097] For another example, if there are two non-zero weights in a row of a group, it means that the row has two non-zero weights that must be input into the corresponding multipliers in sequence, so that the multi-channel buffer can input the data corresponding to the row in the input data into the corresponding multiplier twice (corresponding to the inputs of the two non-zero weights respectively).
[0098] In some embodiments, the position of a non-zero weight is the offset of the non-zero weight in its group.
[0099] Reference Figure 4 As one embodiment of the present disclosure, the position of a non-zero weight may be the offset (or relative position) of the non-zero weight in the corresponding group, for example, the non-zero weight is at position 0, position 1, position 2, position 3...position 15, etc. in the group.
[0100] Obviously, the form of the position of non-zero weights must ensure that the memory buffer can determine the number of non-zero weights in each row based on the position.
[0101] For example, when the position of the non-zero weight is the offset of the non-zero weight in its group, the memory buffer can determine the row where the non-zero weight is located by taking the quotient operation, and the number of non-zero weights with the same quotient is the number of non-zero weights in the corresponding row.
[0102] For example, refer to Figure 4 , the offsets of the two non-zero weights in the same group are 5 and 7 respectively. When they are divided by the number of columns in the group, 4, the integer quotients are all 1, indicating that they are in row 1 (because the row number starts from 0, it is actually row 2), that is, row 1 has two non-zero weights.
[0103] Of course, the position of the non-zero weight may also be in other forms. For example, the position of the non-zero weight may include the row where the non-zero weight is located and the offset in the row (equivalent to the two-dimensional coordinates of the non-zero weight).
[0104] The position and value of each non-zero weight may be stored in one memory address; thus, the memory buffer may also have a line splitting function to separate the position and value of the non-zero weight.
[0105] Alternatively, a splitter may be provided so that the non-zero weights stored in the memory chip may first be separated by the splitter into the values and positions of the non-zero weights and then input into the corresponding memory buffer.
[0106] Obviously, the shifter should essentially shift the multiplication result according to the offset of the non-zero weight in the row, so the offset of the non-zero weight in the row should also be determined according to its position.
[0107] For example, when the position of the non-zero weight is the offset of the non-zero weight in its group, the offset of the non-zero weight in the row is the modulo value of its offset in the group (modulo the number of columns of the group), that is, the shifter should calculate the multiplication result according to the modulo value. Specifically, the above modulo operation can be performed in the memory buffer or in the shifter.
[0108] In some embodiments, the multiplication operation further includes: when a preset pause condition is met, the multi-channel buffer pauses acquiring input data, and / or the multi-channel buffer controls at least part of the memory buffer to pause operation.
[0109] Reference Figure 3 In the disclosed embodiment, the number of non-zero weights in the same row of different groups is different, and each channel actually processes only the non-zero weights in the corresponding group. Therefore, when the processing speed of each channel (the number of non-zero weights actually processed per unit time) is the same, after a certain period of time, although the number of non-zero weights processed by each channel is the same, different channels may actually be processing different rows, which may cause the operation to fail to proceed normally (such as the data required in some devices exceeds their storage capacity). To this end, a pause condition can be set in advance, and when the pause condition is met, the multi-channel buffer controls the "pause" work.
[0110] For example, refer to Figure 5 At any time, the input buffer stores part of the input data (e.g., 7 data), and each channel of the multi-channel buffer obtains the input data at the corresponding position in the input buffer according to the row currently being processed. However, if the number of rows being processed by different channels is greatly different (e.g., 8 rows), the order of the data required by each multiplier in the input data is also greatly different (e.g., 8), which may exceed the range that the input buffer can buffer; therefore, the above situation can be one of the pause conditions.
[0111] For example, refer to Figure 7 The address generator uniformly controls the input buffer and each memory chip according to the number of rows, so the difference in the number of rows being processed in different memory buffers cannot exceed the maximum buffering capacity of the memory buffer; therefore, the above situation can be one of the pause conditions.
[0112] Of course, the specific form of the suspension condition is not limited to the above examples. As long as it is desired to suspend part or all of the processing under certain circumstances, the situation can be used as the suspension condition.
[0113] The pause may be that the multi-channel buffer pauses to obtain data from the input buffer, or the multi-channel buffer controls at least part of the memory buffers to pause (such as pausing all, or pausing only the memory buffer with the largest number of processed rows).
[0114] For example, the multi-channel buffer can achieve the above pause by issuing control instructions to the address generator, the memory buffer, and the input buffer.
[0115] Of course, it should be understood that when the pause condition is no longer met, the multi-channel buffer should restart obtaining input data and control the memory buffer to restart working.
[0116] Second, refer to Figure 6 , Figure 7 , an embodiment of the present disclosure provides an electronic device that can implement the above neural network calculation method.
[0117] Reference Figure 6 , the electronic device provided by the embodiment of the present disclosure includes:
[0118] A processing unit, the processing unit comprising a plurality of memory slices and a plurality of multipliers corresponding one to one with the memory slices;
[0119] A storage unit having a computer program stored thereon; when the computer program is executed by the processing unit, any one of the above-mentioned neural network calculation methods is implemented.
[0120] The above neural network calculation method can be implemented in a processing unit, so the processing unit should include the above memory chip.
[0121] Among them, the specific hardware forms of electronic devices, processing units, and storage units are diverse.
[0122] For example, the processing unit may be a physical processor device or a processing core device; or, the processing unit may also be composed of multiple processor devices or processing core devices, such as each channel corresponds to a processor device or a processing core device.
[0123] For example, the storage unit may be one or more independent memory devices, or may be a memory device of a processing unit, as long as a computer program is stored therein and the computer program can be executed by the processing unit to implement any of the above-mentioned neural network calculation methods.
[0124] For example, the above electronic device may be a processor or a processing core device, or may be a multi-core system composed of multiple processing cores and an on-chip network.
[0125] In some embodiments, the architecture of the processing unit can refer to Figure 7 , also includes an address generator, an input buffer, an output buffer, a multi-channel buffer, a shifter, an accumulator, a memory buffer, etc.; among which:
[0126] The address generator is used to determine the memory address currently to be processed.
[0127] Each memory slice is used to input the specific non-zero weight stored in the respective memory slice into the corresponding memory buffer according to the control of the address generator.
[0128] Each memory buffer is used to input the obtained non-zero weight value into the multiplier, input the position of the non-zero weight into the shifter, and determine the number of non-zero weights in each row of the corresponding group according to the position of the non-zero weight, so as to input the number of non-zero weights in the row into the multi-channel buffer.
[0129] Each multiplier is used to perform multiplication calculation on the input non-zero weight value and the input data, and input the obtained multiplication result to the shifter.
[0130] Each shifter is used to shift the multiplication result to a corresponding position according to the position of the non-zero weight and the multiplication result input by the multiplier, and then input it into the accumulator.
[0131] Each accumulator is used to accumulate input data, specifically, to accumulate data located at the same position in the input data (multiplication results), and output the result to an output buffer.
[0132] The output buffer is used to concatenate the final outputs of each accumulator to obtain the result of the weight matrix calculation (output data).
[0133] The input buffer is used to cache the data in the input data.
[0134] The multi-channel buffer is used to enable the corresponding channel to obtain the corresponding data in the input data from the input buffer according to the number of non-zero weights in the row input by each memory buffer, and output it to the corresponding multiplier for the corresponding number of times. At the same time, the multi-channel buffer can also be used to control the address generator, memory buffer, and input buffer to suspend work when the suspension condition is met.
[0135] The above memory chips, memory buffers, multipliers, shifters, and accumulators may all correspond to each other one by one, that is, each channel includes one memory chip, one memory buffer, one multiplier, one shifter, and one accumulator.
[0136] There may be only one of the above address generator, input buffer, output buffer, and multi-channel buffer, but the multi-channel buffer may have multiple channels.
[0137] Thirdly, refer to Figure 8 , an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, wherein the computer program implements any of the above-mentioned neural network calculation methods when executed by a processor.
[0138] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0139] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.
Claims
1. A method for neural network calculation, wherein include: The weight matrix is divided into multiple groups, and the non-zero weights in each group are stored in corresponding memory slices; wherein the weight matrix is a connection weight matrix of neurons; each group includes multiple consecutive columns of the weight matrix; the memory slices correspond to the groups one by one, and the values and positions of the non-zero weights are stored in the memory slices; Repeat the following multiplication operation: input the value of the non-zero weight in each memory slice to the corresponding multiplier, and input the corresponding data in the input data to each multiplier; The multiplication results output by each multiplier in each multiplication operation are shifted according to the position of the corresponding non-zero weight, and the multiplication results at the same position after the shift are accumulated to calculate the output data; The method of inputting the corresponding data in the input data to each multiplier includes: the memory buffer obtains the non-zero weights in the corresponding memory chip, determines the number of non-zero weights in each row of the corresponding group according to the position of each non-zero weight, and inputs the number of non-zero weights in each row to the multi-channel buffer; the multi-channel buffer outputs the data corresponding to each row in the input data to the corresponding multiplier the same number of times according to the number of non-zero weights in each row of each group.
2. The method according to claim 1, in, The neural network is a pulse neural network, and the weight matrix is a synaptic connection weight matrix.
3. The method according to claim 1, in, Before dividing the weight matrix into multiple groups, the method further includes: Swap the positions of some columns in the weight matrix; and / or, Swap the positions of some rows in the weight matrix.
4. The method according to claim 1, in, The sparsity of the weight matrix is greater than or equal to 50%.
5. The method according to claim 1, in, The difference in sparsity between any two groups does not exceed 10%.
6. The method according to claim 1, in, The number of columns in any group is the same.
7. The method according to claim 1, in, The shifting of the multiplication results output by each multiplier in each multiplication operation according to the position of the corresponding non-zero weight comprises: inputting the multiplication results output by each multiplier in each multiplication operation into the corresponding shifter, and inputting the position of each corresponding non-zero weight into the corresponding shifter; the shifter shifts the multiplication results according to the corresponding position; Accumulating the multiplication results at the same position after shifting includes: inputting the output of each shifter to the corresponding accumulator respectively, and the accumulator accumulates the multiplication results at the same position.
8. The method according to claim 7, in, The step of inputting the values of the non-zero weights in each memory slice into the corresponding multiplier comprises: the memory buffer acquires the non-zero weights in the corresponding memory slice, and inputs the values of the non-zero weights into the corresponding multiplier; The step of inputting the positions of the corresponding non-zero weights into the corresponding shifters comprises: the memory buffer acquires the non-zero weights in the corresponding memory slice, and inputs the positions of the non-zero weights into the corresponding shifters.
9. The method according to claim 8, in, The position of the non-zero weight is the offset of the non-zero weight in the group to which it belongs.
10. The method according to claim 8, in, The multiplication operation also includes: When a preset suspension condition is met, the multi-channel buffer suspends acquiring input data, and / or the multi-channel buffer controls at least part of the memory buffer to suspend operation.
11. An electronic device, include: A processing unit, the processing unit comprising a plurality of memory slices and a plurality of multipliers corresponding one to one with the memory slices; a storage unit having a computer program stored thereon; When the computer program is executed by the processing unit, the method of neural network calculation described in any one of claims 1 to 10 is implemented.
12. A computer readable medium having a computer program stored thereon, in, When the computer program is executed by a processor, the computer program implements the neural network calculation method according to any one of claims 1 to 10.
Citation Information
Patent Citations
Hardware accelerator and method for realizing sparse GRU neural network based on FPGA
CN107229967A
Convolutional neural network computing device and method
CN110070178A
