Neural network calculation method, electronic device, and computer-readable medium

Abstract

The present disclosure provides a method for neural network calculation, which includes: dividing a weight matrix into multiple groups, and storing the non-zero weights in each group in a corresponding memory slice; wherein the weight matrix is ​​a connection weight matrix of neurons; each group includes multiple consecutive columns of the weight matrix; the memory slices correspond to the groups one by one, wherein the values ​​and positions of the non-zero weights are stored; repeatedly performing the following multiplication operations: inputting the non-zero weights of the current row in each memory slice into a decompressor, the decompressor supplementing the positions without non-zero weights in the current row of each group with zero weights to obtain the restored current row of each group, and inputting each restored current row into a corresponding multiplier array; inputting the data corresponding to the current row in the input data into each multiplier array; accumulating the multiplication results output by each multiplier array in each multiplication operation to calculate the output data. The present disclosure also provides an electronic device and a computer-readable medium.

Description

Technical Field

[0001] The present disclosure relates to the field of neural network technology, and in particular to a neural network calculation method, electronic device, and computer-readable medium. Background Art

[0002] When using neural networks, the calculation of weight matrices takes up a large amount of computational load. For example, in a spiking neural network based on the LIF (Leaky Integrity-and-Fire) model, the synaptic integral calculation of the weight matrix is ​​the main load.

[0003] Therefore, the calculation of the weight matrix requires a large amount of memory space, access bandwidth, computing resources, etc., resulting in a large computational overhead. Summary of the Invention

[0004] The present disclosure provides a neural network computing method, an electronic device, and a computer-readable medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for neural network calculation, comprising:

[0006] Divide the weight matrix into multiple groups, and store the non-zero weights in each group in a corresponding memory slice; wherein the weight matrix is ​​the connection weight matrix of the neurons; each group includes multiple consecutive columns of the weight matrix; and the memory slices correspond one to one with the groups, in which the values ​​and positions of the non-zero weights are stored;

[0007] Repeat the following multiplication operation: input the non-zero weights of the current row in each memory slice to the decompressor, the decompressor adds zero weights to the positions without non-zero weights in the current row of each group to obtain the restored current row of each group, and input each restored current row to the corresponding multiplier array; input the data corresponding to the current row in the input data to each multiplier array;

[0008] The multiplication results output by each multiplier array in each multiplication operation are accumulated to calculate the output data.

[0009] In some embodiments, the neural network is a spiking neural network, and the weight matrix is ​​a synaptic connection weight matrix.

[0010] In some embodiments, before dividing the weight matrix into multiple groups, the method further includes:

[0011] Swap the positions of some columns in the weight matrix;

[0012] and / or,

[0013] Swap the positions of some rows in the weight matrix.

[0014] In some embodiments, the sparsity of the weight matrix is ​​greater than or equal to 50%.

[0015] In some embodiments, the difference in sparsity between any two groups does not exceed 10%.

[0016] In some embodiments, the number of columns in any group is the same.

[0017] In some embodiments, the position of the non-zero weight is the offset of the non-zero weight in the group to which it belongs.

[0018] In some embodiments, at least part of the multiplier array is a multiplexed multiplier array;

[0019] The number of multipliers in each multiplexed multiplier array is smaller than the number of columns of its corresponding group, and the multiplexed multiplier array is realized by time-sharing multiplexing of at least part of the multipliers.

[0020] In some embodiments, accumulating the multiplication results output by each multiplier array in each multiplication operation includes: inputting the multiplication results output by each multiplier array in each multiplication operation into corresponding accumulators respectively, and the accumulators accumulate the multiplication results.

[0021] In some embodiments, inputting the non-zero weight of the current row in each memory slice to the decompressor includes:

[0022] Each memory buffer obtains non-zero weights from a corresponding memory slice, determines the non-zero weights in the current row according to the positions of the obtained non-zero weights, and inputs the non-zero weights in the current row to the decompressor.

[0023] In some embodiments, the multiplication operation further comprises:

[0024] The updater determines the row where the non-zero weight is located in each memory buffer. After all memory buffers have processed the non-zero weight of the current row, the updater controls each memory buffer to update the current row.

[0025] In a second aspect, an embodiment of the present disclosure provides an electronic device, including:

[0026] A processing unit, the processing unit comprising: a plurality of memory slices, a decompressor, and a plurality of multiplier arrays corresponding one-to-one to the memory slices;

[0027] a storage unit having a computer program stored thereon;

[0028] When the computer program is executed by the processing unit, any one of the above-mentioned neural network calculation methods is implemented.

[0029] In a third aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, wherein the computer program implements any of the above-mentioned neural network calculation methods when executed by a processor.

[0030] In the disclosed embodiment, only the non-zero weights of the weight matrix are stored in the memory, which is equivalent to achieving "compressed encoding" of the weight matrix, thereby reducing the memory space and access bandwidth required for calculation and reducing computing overhead.

[0031] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings. In the accompanying drawings:

[0033] Figure 1 A flowchart of a neural network calculation method provided in an embodiment of the present disclosure;

[0034] Figure 2 A flowchart of another neural network calculation method provided by an embodiment of the present disclosure;

[0035] Figure 3 A schematic diagram of a grouping process in a neural network calculation method provided by an embodiment of the present disclosure;

[0036] Figure 4 A schematic diagram of a storage format for non-zero weights in a neural network calculation method provided by an embodiment of the present disclosure;

[0037] Figure 5 A functional schematic diagram of a decompressor in a neural network calculation method provided by an embodiment of the present disclosure;

[0038] Figure 6 A block diagram of an electronic device according to an embodiment of the present disclosure;

[0039] Figure 7 A schematic diagram of the architecture of a processing unit in an electronic device provided by an embodiment of the present disclosure;

[0040] Figure 8 A block diagram of the composition of a computer-readable medium provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0041] To enable those skilled in the art to better understand the technical solutions of the present disclosure, exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0042] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0043] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0044] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. Similar words such as "connected" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0045] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless expressly defined as such herein.

[0046] First, refer to Figures 1 to 5 、 Figure 7 , an embodiment of the present disclosure provides a method for neural network calculation.

[0047] The method of the embodiment of the present disclosure is used to perform calculations in a neural network.

[0048] Among them, neural networks can realize any pre-trained functions, such as image recognition, action recognition, speech recognition, text recognition, etc.

[0049] The disclosed embodiments are used to calculate the product of input data input to a neuron in a neural network and the corresponding connection weight matrix. That is, in a neural network, neurons in different layers are interconnected, and the connections between different neurons have different weight matrices. Therefore, the input data input to a neuron must be multiplied by the weight matrix of that neuron.

[0050] In some embodiments, the neural network is a spiking neural network, and the weight matrix is ​​a synaptic connection weight matrix.

[0051] As one embodiment of the present disclosure, the applicable neural network may be a spiking neural network (SNN), whereby the weight matrix therein is a weight matrix corresponding to synaptic connections between neurons.

[0052] Exemplarily, the above spiking neural network may be based on a LIF (Leaky Integrity-and-Fire) model.

[0053] In the LIF model, the pulse current input signal (Xi) of the previous neuron i is connected to the current neuron through the weight matrix (Wi). When the membrane voltage (Vt) of the current neuron accumulates (integrates) and reaches a predetermined threshold (Vth) due to the charging of the input current signals of all previous neurons, the current neuron will emit a pulse current signal and reset the membrane voltage (Vt) to the resting voltage (Vreset).

[0054] Of course, the scope of use of the embodiments of the present disclosure is not limited to this. It can also be used for pulse neural networks of other modes, or for weight matrix calculation in other types of neural networks (such as convolutional neural networks, etc.).

[0055] In some embodiments, the sparsity of the weight matrix is ​​greater than or equal to 50%.

[0056] The embodiments of the present disclosure are suitable for processing sparse weight matrices, for example, weight matrices with a sparsity (i.e., the proportion of zero elements) of at least 50%, or at least 70%, or at least 90%, or at least 95%.

[0057] The specific sparsity of the weight matrix that can be processed by the embodiments of the present disclosure can be determined based on actual computing resources (hardware resources).

[0058] Reference Figure 1 , the method of the embodiment of the present disclosure includes:

[0059] S101. Divide the weight matrix into multiple groups, and store the non-zero weights in each group in the corresponding memory slice.

[0060] The weight matrix is ​​a connection weight matrix of neurons; each group includes multiple consecutive columns of the weight matrix; and the memory slices correspond one-to-one to the groups, in which the values ​​and positions of non-zero weights are stored.

[0061] S102. Repeat the following multiplication operations: input the non-zero weights of the current row in each memory slice into the decompressor respectively, the decompressor adds zero weights to the positions without non-zero weights in the current row of each group to obtain the restored current row of each group, and input each restored current row into the corresponding multiplier array respectively; input the data corresponding to the current row in the input data into each multiplier array respectively.

[0062] S103 , accumulating the multiplication results output by each multiplier array in each multiplication operation to calculate output data.

[0063] Reference Figure 3 In the embodiment of the present disclosure, before calculating the weight matrix, the weight matrix is ​​first grouped vertically so that each group includes multiple consecutive columns of the weight matrix; then, all non-zero weights in each group are stored in a corresponding memory slice, and the memory slice stores the value of the non-zero weight (the "numerical value" of the non-zero weight) and the position (the "relative position" or "offset" of the non-zero weight in the row, group, and weight matrix).

[0064] Afterwards, the non-zero weights in each memory slice are input to the decompressor. Figure 5 , the decompressor determines the position of the non-zero weight in the current row according to the position of the non-zero weight of the current row of the input, that is, it determines that the position of "no non-zero weight" in the current row (the currently processed row) actually "should be zero weight"; thus, the decompressor can actually "fill" zero weight at the position that should be zero weight in the current row of each group, so as to "restore (or decompress)" and obtain the "complete row (restore the current row)" in each group. Obviously, the restored current row at this time may include zero weight or non-zero weight.

[0065] If the decompressor does not receive a non-zero weight corresponding to a row in a certain memory slice, it means that all rows in the corresponding group have zero weights, that is, the decompressor should supplement all rows in the group with zero weights.

[0066] Therefore, the decompressor can input the restored current row of each group into the corresponding multiplier array respectively, and then synchronously input the data corresponding to the current row in the input data into each multiplier array, so that each multiplier array can complete the calculation of the current row of the weight matrix and obtain the multiplication result corresponding to the current row.

[0067] The multiplication results of each row are then accumulated to obtain the final result (output data) of the weight matrix calculation.

[0068] In the embodiment of the present disclosure, only the non-zero weights of the weight matrix are stored in the memory, which is equivalent to achieving "compression encoding" of the weight matrix, thereby reducing the storage amount and access bandwidth required in the calculation and reducing the calculation overhead.

[0069] Among them, the main bottleneck of weight matrix calculation is usually the limitation of memory space, so reducing the storage capacity has the most obvious improvement on actual computing power.

[0070] Moreover, in the above calculation process, it is equivalent to processing different groups of data in different "channels (each channel corresponds to a memory slice and a multiplier array)", and each channel is only used to process part of the data of the weight matrix, so the calculation overhead can be further reduced and the calculation efficiency can be improved.

[0071] In addition, the solution of the embodiment of the present disclosure requires relatively few changes to the conventional system architecture and is easy to implement.

[0072] In some embodiments, the difference in sparsity between any two groups does not exceed 10%.

[0073] Reference Figure 3 (In the figure, black squares represent non-zero weights, and colorless squares represent zero weights). After grouping the weight matrix, the number of non-zero weights in each group should be "similar" to ensure that the processing progress of each channel is basically the same.

[0074] Specifically, each group has a certain sparsity, and among the multiple groups divided by the weight matrix, the difference in sparsity between the group with the largest sparsity and the group with the smallest sparsity should not exceed 10%, and further should not exceed 5%.

[0075] Of course, refer to Figure 3 , it is also feasible if all groups have the same sparsity (the number of non-zero weights is the same).

[0076] Of course, the specific allowable range of the difference in sparsity (the difference in the number of non-zero weights) in different groups can be determined according to actual computing resources (hardware resources).

[0077] In some embodiments, before dividing the weight matrix into multiple groups (S101), the method further includes:

[0078] S100: swap positions of some columns in the weight matrix; and / or swap positions of some rows in the weight matrix.

[0079] As previously mentioned, the disclosed embodiments aim to ensure that the number of non-zero weights in each group after grouping is as similar as possible, that is, to ensure that the distribution of non-zero weights in the weight matrix is ​​substantially uniform. However, not all weight matrices in a neural network meet this requirement. Therefore, for weight matrices that do not meet this requirement, some columns or column positions can be swapped to achieve a more uniform distribution of non-zero weights within the weight matrix.

[0080] Among them, exchanging the positions of some rows in the weight matrix can change the groups where some non-zero weights are located, so that the exchanged weight matrix can more easily achieve the above grouping.

[0081] Among them, by exchanging the positions of some rows in the weight matrix, the processing progress of each channel can be basically the same in each time period.

[0082] Of course, when the columns are swapped, the corresponding data in the result of the weight calculation should also be swapped before they can be actually output as the final output data.

[0083] Of course, when the rows are swapped, the order of the data in the corresponding input data must also be swapped accordingly.

[0084] Of course, in order to obtain a weight matrix that can perform the above grouping (a weight matrix with uniform non-zero weight distribution), the weight matrix can also be restricted during the training process of the neural network.

[0085] For example, during the training process, the number and distribution of non-zero weights in the weight matrix may be restricted.

[0086] For example, during the training process, mask constraints may be used to ensure that the number and distribution of non-zero weights in the weight matrix meet the requirements.

[0087] In some embodiments, the number of columns in any group is the same.

[0088] As one embodiment of the present disclosure, the number of columns in all groups may be the same. Figure 3 , all groups have 4 columns.

[0089] The specific number of columns in each group is not limited thereto and can be determined based on the weight matrix. For example, the number of columns in each group can be at least 2, or at least 4, or at least 10, etc.; for another example, the number of columns in each group can be no more than 20, or no more than 10, or no more than 4, etc.

[0090] In some embodiments, the position of the non-zero weight is the offset of the non-zero weight in the group to which it belongs.

[0091] Reference Figure 4As one embodiment of the present disclosure, the position of a non-zero weight can be the offset (or relative position) of the non-zero weight in the corresponding group, for example, the non-zero weight is at position 0, position 1, position 2, position 3...position 15, etc. in the group.

[0092] Thus, the row where the non-zero weight is located can be determined by taking the quotient operation, and the position of the non-zero weight in the row can be determined by taking the remainder operation, thereby realizing the operation of adding zero weight to the position of zero weight in each row.

[0093] Of course, the position of the non-zero weight can also be in other forms. For example, the position of the non-zero weight can include two data: the row where the non-zero weight is located and the offset in the row (equivalent to the two-dimensional coordinates of the non-zero weight). As long as the row where the non-zero weight is located and the position in the row can be determined based on the position, it will be fine.

[0094] The position and value of each non-zero weight may be stored in a memory address; thus, the corresponding computing unit (such as a memory buffer) may also have a branching function to separate the position and value of the non-zero weight.

[0095] Alternatively, a splitter may be provided so that the non-zero weights stored in the memory chip can first be separated by the splitter to separate the values ​​and positions of the non-zero weights and then input into the corresponding computing unit (such as a memory buffer).

[0096] In some embodiments, at least part of the multiplier arrays are multiplexed multiplier arrays; the number of multipliers in each multiplexed multiplier array is less than the number of columns of its corresponding group, and the multiplexed multiplier array is realized by time-sharing multiplexing of at least part of the multipliers.

[0097] Obviously, a multiplier array refers to a structure that can perform multiplication operations on multiple groups of input data in one processing cycle.

[0098] As one form of an embodiment of the present disclosure, the multiplier array may include a smaller number of multipliers than the number of groups of input data (the number of columns of the corresponding groups), for example, only one multiplier; thus, when processing multiple groups of input data, the multiplier works multiple times according to time to process different groups of input data respectively (i.e., time-division multiplexing).

[0099] Generally speaking, the processing speed of a multiplier is faster than that of a memory. Therefore, by implementing a multiplier array through time division multiplexing, computing resources can be saved without affecting the actual processing speed.

[0100] Of course, the multiplier array may also include parallel multipliers with the same number as the number of groups of input data (the number of columns of the corresponding groups), that is, multiple groups of input data can be input into each multiplier at the same time for synchronous processing.

[0101] For example, refer to Figure 6 When each restored current row of each group has 4 weights (including zero weight and non-zero weight), each multiplier array can have 4 parallel multipliers; or each multiplier array can have only one multiplier, which works 4 times in time (time division multiplexing) to process 4 weights respectively.

[0102] In some embodiments, accumulating the multiplication results output by each multiplier array in each multiplication operation includes: inputting the multiplication results output by each multiplier array in each multiplication operation into corresponding accumulators respectively, and the accumulators accumulate the multiplication results.

[0103] Reference Figure 6 Each multiplier array can have a corresponding accumulator, so that the multiplication results of the multiplier array can be directly input into the corresponding accumulator. The accumulator accumulates the data at the same position of each multiplication result, and then through subsequent splicing, etc., the final result (output data) of the weight matrix calculation can be obtained.

[0104] In some embodiments, inputting the non-zero weight of the current row in each memory slice to the decompressor includes:

[0105] Each memory buffer obtains non-zero weights from a corresponding memory slice, determines the non-zero weights in the current row according to the positions of the obtained non-zero weights, and inputs the non-zero weights in the current row to the decompressor.

[0106] Obviously, the actual number of non-zero weights in the same row of different groups is different. Therefore, for the same current row, the actual number of non-zero weights to be input to the decompressor by each memory slice is different. To synchronize the processing progress of each channel (that is, to ensure that the weights of the same row are input to each multiplier array in the same processing cycle), a corresponding memory buffer can be set up for each memory slice. The memory buffer sequentially obtains some non-zero weights from the memory slice, determines the row to which the non-zero weight belongs and its position within the row, and then inputs the non-zero weights of the current row into the decompressor; in other words, the memory buffer is used to buffer the non-zero weights of the current row.

[0107] In some embodiments, the multiplication operation further includes: the updater determines the row where the non-zero weights are located in each memory buffer, and after all memory buffers have processed the non-zero weights of the current row, the updater controls each memory buffer to update the current row.

[0108] Reference Figure 6In order to allow each memory buffer to determine which row is the current row, an updater can also be set to obtain the row where the non-zero weight is being processed by each memory buffer, and when it is determined that the memory buffer with the slowest progress has also processed the non-zero weight of the current row (such as the non-zero weight of the next row has been buffered), each memory buffer is controlled to update the current row (that is, the next row is the current row), so that each memory buffer starts to process the non-zero weight of the next row.

[0109] There are many ways for the updater to implement "update the current row", for example, Figure 6 ,The updater can send an address update enable signal to the address generator and each memory buffer when updating the current row.

[0110] Secondly, refer to Figure 6 、 Figure 7 , an embodiment of the present disclosure provides an electronic device that can implement the above neural network calculation method.

[0111] Reference Figure 6 , the electronic device provided by the embodiment of the present disclosure includes:

[0112] A processing unit, the processing unit comprising: a plurality of memory slices, a decompressor, and a plurality of multiplier arrays corresponding one-to-one to the memory slices;

[0113] A storage unit stores a computer program thereon; when the computer program is executed by the processing unit, any one of the above-mentioned neural network calculation methods is implemented.

[0114] The above neural network calculation method can be implemented in a processing unit, so the processing unit should include the above memory chip.

[0115] Among them, the specific hardware forms of electronic devices, processing units, and storage units are diverse.

[0116] For example, the processing unit may be a physical processor device or processing core device; or, the processing unit may be composed of multiple processor devices or processing core devices, such as each channel corresponds to a processor device or processing core device.

[0117] For example, the storage unit can be one or more independent memory devices, or it can be a memory device of a processing unit, as long as a computer program is stored therein and the computer program can be executed by the processing unit to implement any of the above-mentioned neural network calculation methods.

[0118] For example, the above electronic device may be a processor or a processing core device, or may be a multi-core system composed of multiple processing cores and an on-chip network.

[0119] In some embodiments, the architecture of the processing unit can refer to Figure 7, also includes an address generator, an input buffer, an output buffer, an updater, an accumulator, a memory buffer, etc.; among which:

[0120] The address generator is used to determine the memory address to be processed currently.

[0121] Each memory slice is used to input the non-zero weight stored in the respective memory slice into the corresponding memory buffer according to the control of the address generator.

[0122] Each memory buffer is used to determine the row where the non-zero weights are obtained, and in each processing cycle, all non-zero weights of the current row are input to the decompressor (if the current row has no non-zero weights, it is equivalent to outputting 0 non-zero weights).

[0123] Each channel of the decompressor is used to determine the position and number of non-zero weights in a current row of a group, and to add zero weights at positions without non-zero weights to obtain a "restored current row" that may include zero weights and non-zero weights, and input the restored current row into the corresponding multiplier array.

[0124] The input buffer is used to synchronously input data corresponding to the current row in the input data to each multiplier array according to the control of the address generator.

[0125] Each multiplier array is used to calculate the restored current row of the input, that is, to restore each weight in the current row, multiply it by the data corresponding to the current row in the input data, and input the multiplication result into the accumulator.

[0126] Each accumulator is used to accumulate the input multiplication results, that is, to accumulate the data located at the same position in each multiplication result, and output the result to the output buffer.

[0127] The output buffer is used to splice the final outputs of each accumulator to obtain the result of weight matrix calculation (output data).

[0128] The updater is used to determine how each memory buffer handles non-zero weights, so that after all memory buffers have processed the non-zero weights of the current row, an address update enable signal is sent to the address generator and each memory buffer to achieve the purpose of "updating the current row".

[0129] The above memory slices, memory buffers, multiplier arrays, and accumulators may all correspond to each other one-to-one, that is, each channel includes one memory slice, one memory buffer, one multiplier array, and one accumulator.

[0130] There may be only one of the above address generator, input buffer, output buffer, decompressor, and updater, but the decompressor may have multiple channels.

[0131] Thirdly, refer to Figure 8An embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, wherein the computer program implements any of the above-mentioned neural network calculation methods when executed by a processor.

[0132] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable, and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media generally embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0133] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly indicated, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, it will be understood by those skilled in the art that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A method for neural network calculation, comprising: The weight matrix is ​​divided into multiple groups, and the non-zero weights in each group are stored in corresponding memory slices. The weight matrix is ​​the connection weight matrix of neurons. Each group includes multiple consecutive columns of the weight matrix. The difference in sparsity between any two groups is determined by hardware resources. The memory slices correspond to the groups one by one, and the values ​​and locations of the non-zero weights are stored therein. The sparsity of each group represents the proportion of non-zero weights. Repeat the following multiplication operation: input the non-zero weights of the current row in each memory slice to the decompressor, the decompressor adds zero weights to the positions without non-zero weights in the current row of each group to obtain the restored current row of each group, and input each restored current row to the corresponding multiplier array; input the data corresponding to the current row in the input data to each multiplier array; The multiplication results output by each multiplier array in each multiplication operation are accumulated to calculate the output data.

2. The method according to claim 1, wherein The neural network is a pulse neural network, and the weight matrix is ​​a synaptic connection weight matrix.

3. The method according to claim 1, wherein Before dividing the weight matrix into multiple groups, the method further includes: Swap the positions of some columns in the weight matrix; and / or, Swap the positions of some rows in the weight matrix.

4. The method according to claim 1, wherein The sparsity of the weight matrix is ​​greater than or equal to 50%, and the sparsity of the weight matrix represents the proportion of zero elements.

5. The method according to claim 1, wherein The difference in sparsity between any two groups does not exceed 10%.

6. The method according to claim 1, wherein The number of columns in any group is the same.

7. The method according to claim 1, wherein The position of the non-zero weight is the offset of the non-zero weight in the group to which it belongs.

8. The method according to claim 1, wherein At least part of the multiplier array is a multiplexed multiplier array; The number of multipliers in each multiplexed multiplier array is smaller than the number of columns of its corresponding group, and the multiplexed multiplier array is realized by time-sharing multiplexing of at least part of the multipliers.

9. The method according to claim 1, wherein The accumulating the multiplication results output by each multiplier array in each multiplication operation includes: inputting the multiplication results output by each multiplier array in each multiplication operation into corresponding accumulators respectively, and the accumulators accumulate the multiplication results.

10. The method according to claim 1, wherein Inputting the non-zero weights of the current rows in each memory slice to the decompressor includes: Each memory buffer obtains non-zero weights from a corresponding memory slice, determines the non-zero weights in the current row according to the positions of the obtained non-zero weights, and inputs the non-zero weights in the current row to the decompressor.

11. The method according to claim 10, wherein: The multiplication operation further includes: The updater determines the row where the non-zero weight is located in each memory buffer. After all memory buffers have processed the non-zero weight of the current row, the updater controls each memory buffer to update the current row.

12. An electronic device comprising: A processing unit, the processing unit comprising: a plurality of memory slices, a decompressor, and a plurality of multiplier arrays corresponding one-to-one to the memory slices; a storage unit having a computer program stored thereon; When the computer program is executed by the processing unit, the method for neural network calculation according to any one of claims 1 to 11 is implemented.

13. A computer-readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the computer program implements the neural network calculation method according to any one of claims 1 to 11.

Images