Dynamic allocation method and device of weight storage and film voltage storage, terminal and medium

Abstract

The application provides a dynamic allocation method and device of weight storage and membrane voltage storage, terminal and medium, the weight storage in the application does not need to be determined in advance in hardware design, and does not need to be calculated in advance by software; the application can fully utilize the shared storage space, and the maximum wasted space is only the minimum output channel number in each allocation, therefore, in the case that the minimum output channel number in each allocation is 1, the reasonable weight and the membrane voltage shared storage space size can make the wasted space less than 5%, greatly improving the storage space utilization and improving the use efficiency of the storage resource. The dynamic allocation space provided by the application needs computing power due to online weight compression, but the weight compression process occurs in the compiler, and the compiler usually runs in a desktop computer or a server with large computing power, so even if the compiler runs for a long time, it is acceptable, which does not affect the hardware efficiency and software efficiency in the reasoning process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of neuron kernel in-memory storage, and particularly to a dynamic allocation method and device of weight storage and membrane voltage storage, a terminal and a medium. BACKGROUND

[0002] Deep neural network (DNN) has developed rapidly and been applied preliminarily in recent years. On the other hand, spiking neural network (SNN) has attracted attention of academia and industry due to its low power consumption and characteristics closer to human brain in recent years.

[0003] Based on research results of neurobiology, it is found that the neuron spike activation frequency of human brain is about 10 Hz, and the interval between two spikes is several milliseconds, so each neuron processes input spikes once every 1 millisecond or so in the implementation. In addition, the time required for addition and subtraction operation is in nanoseconds. Usually, a time division multiplexing method can be used to multiplex multiple neurons to use the same operation unit to process different axons.

[0004] In the implementation, one neuron kernel processes operations of multiple neurons at the same time. One neuron kernel includes an input spike storage area, a weight storage area, a membrane voltage storage area and an operation unit. The input spike storage area is used to collect spikes on input axons of each neuron in the neuron kernel within a period of time; the weight storage area is used to store synaptic connections or artificial intelligence neural network weights; the membrane voltage storage area stores the update result of the membrane voltage of each neuron; the output spike of the neuron in the neuron kernel is sent to the connected neuron kernel and stored in the input spike storage area of the connected neuron kernel.

[0005] When the artificial intelligence neural network is spiking and mapped to the spiking neural network, the output of the previous layer is stored in the input spike storage area of the neuron kernel in the form of spikes, and the weight is stored in the weight storage area. The membrane voltage does not exist in the artificial intelligence neural network, but it must be stored after the artificial intelligence neural network is spiking and mapped to the spiking neural network, and is updated in the following manner: in the spiking neural network, the axon is a unit that receives spikes, the neuron is a unit that sends spikes, one neuron is connected to multiple axons through dendrites, and the connection point of the dendrite and the axon is called a synapse. After the axon receives a spike, all dendrites connected to the synapse will receive the spike, which will affect the downstream neurons of the dendrites. The neuron adds the spikes from multiple axons, and if the sum of the current received spikes and the added value of the membrane voltage is greater than or equal to the threshold value, a spike is sent downstream, and the membrane voltage is updated to the reset voltage or the previous membrane voltage plus the sum of the current received spikes minus the threshold value; if the sum of the current received spikes and the added value of the membrane voltage is less than the threshold value, no spike is sent downstream, and the membrane voltage is updated to the previous membrane voltage plus the sum of the current received spikes.

[0006] In mapping the pulse of artificial intelligence neural network, the size of the weight storage area in the neuron core is related to the number of output channels, and the larger the number of output channels, the larger the space required for weight storage. Taking a convolution layer as an example, the number of output channels is equal to the number of convolution kernels; the more convolution kernels supported by a neuron core, the larger the space required for weight storage. If the weight is compressed, the weight storage area will be reduced. The size of the membrane voltage storage area in the neuron core is related to the number of output channels, and the larger the number of output channels, the larger the space required for membrane voltage storage.

[0007] When implemented in hardware, the storage space in a neuron core is limited. In general, the weight storage space and the membrane voltage storage space are pre-allocated in the implementation of the pulse neural network, for example, the weight storage space is 32KB, and the membrane voltage storage space is 16KB.

[0008] However, the above method of pre-determining the storage space is not efficient. For example, when a convolution layer is mapped to a neuron core, according to the size of the membrane voltage storage space, a maximum of 20 output channels can be placed; however, the weight storage required by 20 convolution kernels only occupies a small part of the weight storage space, and the remaining weight storage space is wasted. It is also possible that a convolution kernel of a convolution layer is large (for example, 13*13, or the input channel is large), and the weight storage area can only store the weights of 14 output channels, and the membrane voltage storage corresponding to the 14 output channels only occupies a part of the membrane voltage storage space, and the remaining membrane voltage storage space is wasted. This method wastes a large amount of space, and in the extreme case, the entire membrane voltage storage space is wasted or the entire weight storage space is wasted.

[0009] At the same time, when the weight is compressed, the weight compression effect of different layers is different, the weight compression effect of different output channels in the same layer is different, and the compression effect is different using different compression methods. The above method of pre-determining the storage space is less efficient in processing compressed weights. At the same time, the size of the weight storage required is also difficult to obtain by formula calculation. SUMMARY

[0010] In view of the above-mentioned disadvantages of the prior art, the purpose of the present application is to provide a dynamic allocation method, device, terminal and medium for weight storage and membrane voltage storage, which solves the technical problems that the method of pre-determining the storage space in the prior art is less efficient in processing compressed weights, and the size of the weight storage required is also difficult to obtain by formula calculation.

[0011] To achieve the above object and other related objects, the first aspect of the present application provides a dynamic allocation method of weight storage and membrane voltage storage, comprising: compressing the weight storage space of a current output channel to be processed on line to obtain a corresponding weight storage space size, and obtaining a corresponding membrane voltage storage space size of the current output channel to be processed; judging whether the weight and the membrane voltage of the current output channel to be processed can be stored in the shared storage area of a current neuron core according to the weight storage space size and the membrane voltage storage space size of the current output channel to be processed; wherein the weight and the membrane voltage are dynamically allocated with a storage space size in the shared storage area; if the weight and the membrane voltage of the current output channel to be processed can be stored in the shared storage area of the current neuron core, then the weight and the membrane voltage of the current output channel to be processed are stored in the shared storage area of the current neuron core, otherwise, the weight and the membrane voltage are stored in the shared storage area of a new neuron core.

[0012] In some embodiments of the first aspect of the present application, the dynamic allocation of the weight and the membrane voltage with a storage space size in the shared storage area comprises: the weight and the membrane voltage are stored in the shared storage area in a unified storage manner; under the condition that the sum of the weight storage space size and the membrane voltage storage space size is constant, the weight storage space size and the membrane voltage storage space size are dynamically adjustable.

[0013] In some embodiments of the first aspect of the present application, judging whether the weight and the membrane voltage of the current output channel to be processed can be stored in the shared storage area of the current neuron core according to the weight storage space size and the membrane voltage storage space size of the current output channel to be processed comprises: initializing a current count value of the output channel, an allocated membrane voltage storage space size in the current neuron core, an allocated weight storage space size in the current neuron core, a current count value of the neuron core and weight content stored in the current neuron core; judging whether all output channels needing to be mapped to the current layer are processed; if all output channels are not processed, then calculating the number of output channels to be processed this time, the weight storage space size required this time and the membrane voltage storage space size required this time; comparing the sum of the total membrane voltage storage space size and the total weight storage space size with the total storage space size in the current neuron core; if the sum of the total membrane voltage storage space size and the total weight storage space size exceeds the total storage space size in the current neuron core, then judging that the weight and the membrane voltage of the current output channel to be processed cannot be stored in the shared storage area of the current neuron core; otherwise, judging that the weight and the membrane voltage of the current output channel to be processed can be stored in the shared storage area of the current neuron core.

[0014] In some embodiments of the first aspect of the present application, the calculation of the number of output channels to be processed this time comprises: obtaining a minimum number of output channels allocated each time, and a comprehensive calculation value of all output channels of the current layer minus the current count value of the output channels plus 1; selecting the smaller one of the two as the number of output channels to be processed this time.

[0015] In some embodiments of the first aspect of the present application, the calculation of the required weight storage space size this time comprises: using a compression algorithm and calculating based on the current count value of the output channels and the number of output channels to be processed this time.

[0016] In some embodiments of the first aspect of the present application, the calculation of the required membrane voltage storage space size this time comprises: using a cal_potential function and calculating based on the number of output channels to be processed this time.

[0017] In some embodiments of the first aspect of the present application, the total membrane voltage storage space size is the sum of the allocated membrane voltage storage space size in the current neuron core and the required membrane voltage storage space size this time; and the total weight storage space size is the sum of the allocated weight storage space size in the current neuron core and the required weight storage space size this time.

[0018] To achieve the above object and other related objects, the second aspect of the present application provides a dynamic allocation device for weight storage and membrane voltage storage, comprising: a storage space module, configured to perform online compression on the weight storage space of a current output channel to be processed to obtain a corresponding weight storage space size, and to obtain a corresponding membrane voltage storage space size of the current output channel to be processed; and a dynamic allocation module, configured to determine whether the weight and the membrane voltage of the current output channel to be processed can be stored in a shared storage area of a current neuron core according to the weight storage space size and the membrane voltage storage space size of the current output channel to be processed; wherein the weight and the membrane voltage are dynamically allocated a storage space size in the shared storage area; if the weight and the membrane voltage of the current output channel to be processed can be stored in the shared storage area of the current neuron core, the weight and the membrane voltage of the current output channel to be processed are stored in the shared storage area of the current neuron core, otherwise, the weight and the membrane voltage of the current output channel to be processed are stored in a shared storage area of a new neuron core.

[0019] To achieve the above object and other related objects, the third aspect of the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the dynamic allocation method for weight storage and membrane voltage storage.

[0020] To achieve the above object and other related objects, the fourth aspect of the present application provides an electronic terminal, comprising: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes the dynamic allocation method of weight storage and membrane voltage storage.

[0021] As described above, the dynamic allocation method of weight storage and membrane voltage storage, the device, the terminal and the medium of the present application have the following beneficial effects:

[0022] (1) The weight storage in the present application does not need to be determined in advance in hardware design, nor does it need to be calculated in advance by software; the present application can fully utilize the shared storage space, and the maximum wasted space is only the minimum output channel number allocated each time, so in the case of the minimum output channel number allocated each time being 1, the reasonable weight and the membrane voltage shared storage space size can make the wasted space less than 5%, greatly improving the storage space utilization and improving the use efficiency of storage resources.

[0023] (2) Although the dynamic allocation space provided by the present application needs computing power due to online weight compression, the weight compression process occurs in the compiler, and the compiler usually runs in a desktop computer or a server with more computing power, so even if the compiler runs for a long time, it is acceptable, which does not affect the hardware efficiency and software efficiency in the inference process. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 The flowchart shows a dynamic allocation method of weight storage and membrane voltage storage in an embodiment of the present application.

[0025] Figure 2 The flowchart shows a dynamic allocation method of weight storage and membrane voltage storage in an embodiment of the present application.

[0026] Figure 3 The structural diagram of the electronic terminal in an embodiment of the present application is shown.

[0027] Figure 4 The structural diagram of a dynamic allocation device of weight storage and membrane voltage storage in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] The advantages and features of the disclosure will become apparent from the following detailed description of embodiments of the disclosure in conjunction with the drawings. It is to be understood that both the foregoing information and the following detailed description are exemplary and explanatory only and are not intended to restrict the scope of the application in any way. Embodiments described herein can be combined with each other, if not mutually exclusive, without departing from the scope of the present disclosure.

[0029] It should be noted that in the following description, reference is made to the accompanying drawings that form a part of the disclosure. These drawings show, by way of illustration, several embodiments of the disclosure. It is to be understood that other embodiments can be utilized and that mechanical, structural, electrical, and operational changes can be made without departing from the spirit and scope of the present disclosure. The following detailed description is not intended to limit the scope of the disclosure, as claimed, and the scope of embodiments of the disclosure solely by the patent claims. The terminology used here is for the purpose of describing particular embodiments only and is not intended to limit the disclosure. Spatially relative terms, such as "upper", "lower", "left", "right", "below", "beneath", "bottom", "top", and the like, can be used herein for ease of description to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures.

[0030] In this disclosure, unless specifically stated and limited otherwise, the terms "mount", "connect", "connection", "fixed", "fixedly", and the like, should not be construed as limited to direct and immediate connections, but can be construed as understood by one of ordinary skill in the art, such as fixed connection, detachable connection, or integral connection; mechanical connection, electrical connection; direct connection, indirect connection through intermediate medium, and internal connection of two elements. For those skilled in the art, the specific meaning of the above terms in this disclosure can be understood according to the specific circumstances.

[0031] Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to include plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises", "comprising", "includes" and / or "including", when used herein, specify the presence of stated features, operations, elements, components, items, kinds and / or groups but do not preclude the presence or addition of one or more other features, operations, elements, components, items, kinds and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Therefore, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". This definition applies only when the combination of elements, functions or operations are inherently mutually exclusive.

[0032] To solve the problems in the background art, the present application provides a simple storage organization mode and a storage allocation method, aiming to improve the efficiency of weight storage and membrane voltage storage. At the same time, in order to make the purpose, technical scheme and advantages of the present application more clear and explicit, the technical scheme in the embodiment of the present application will be further described in detail through the following embodiments and in conjunction with the drawings. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the application.

[0033] Before further detailing the present application, the terms and phrases involved in the embodiments of the present application are explained, which are applicable to the following explanations:

[0034] <1> DNN (Deep Neural Networks): Deep Neural Network; the neural network layers inside the DNN can be divided into three categories, input layer, hidden layer and output layer; generally speaking, the first layer is the input layer, the last layer is the output layer, and the number of layers in between are hidden layers; the output layer can have multiple neurons, so that the model can be flexibly applied to classification regression and other machine learning fields (such as dimensionality reduction and clustering, etc.).

[0035] <2> SNN (Spiking Neural Network): Spiking Neural Network; in the Spiking Neural Network, axon is the unit that receives pulses, neuron is the unit that sends pulses, a neuron is connected to multiple axons through dendrites, and the connection point of dendrites and axons is called synapse. After the axon receives a pulse, all dendrites connected to this axon through synapses will receive the pulse, thereby affecting the downstream neurons of the dendrites. The neuron adds up the pulses from multiple axons, and if the sum of the current received pulses and the added value of the membrane voltage is greater than or equal to the threshold value, a pulse is sent downstream, and the membrane voltage is updated to the reset voltage or to the previous membrane voltage plus the sum of the current received pulses minus the threshold value; if the sum of the current received pulses and the added value of the membrane voltage is less than the threshold value, no pulse is sent downstream, and the membrane voltage is updated to the previous membrane voltage plus the sum of the current received pulses.

[0036] The embodiments of the present application provide a weight storage and membrane voltage storage dynamic allocation method, a system of the weight storage and membrane voltage storage dynamic allocation method, and a storage medium storing an executable program for implementing the weight storage and membrane voltage storage dynamic allocation method. As for the implementation of the weight storage and membrane voltage storage dynamic allocation method, the embodiments of the present application will illustrate an exemplary implementation scenario of the weight storage and membrane voltage storage dynamic allocation.

[0037] As Figure 1As shown, a flow diagram of a dynamic allocation method of weight storage and membrane voltage storage in an embodiment of the present application is shown. The dynamic allocation method of weight storage and membrane voltage storage in the embodiment mainly includes the following steps:

[0038] Step S11: online compressing the weight storage space of the current output channel to be processed to obtain the corresponding weight storage space size, and obtaining the membrane voltage storage space size corresponding to the current output channel to be processed.

[0039] In some examples, a compiler is used to online compress the weight storage space of the current output channel to be processed to obtain the corresponding weight storage space size. The compiler is a program that translates a language (usually a high-level language) into another language (usually a low-level language); the main workflow of a modern compiler includes: source code → preprocessor → compiler → object code → linker → executable program. High-level computer language is convenient for people to write, read, communicate and maintain, and machine language is a language that can be directly interpreted and run by a computer; the compiler takes the assembly or high-level computer language source program as input (such as Pascal, C, C++, Java, Chinese programming, etc. or assembly language), and translates it into an equivalent program of target language machine code.

[0040] In the embodiment, the weight storage space size is dynamically obtained by online compressing the weight storage space by the compiler. When processing an output channel, the weight storage space of the output channel is first compressed to obtain the increased weight storage space size of the output channel. When an output channel is added, the corresponding membrane voltage storage space also needs to be allocated, and the space size required by the membrane voltage storage can generally be directly calculated and will not be described here.

[0041] Step S12: according to the weight storage space size and the membrane voltage storage space size of the current output channel to be processed, judging whether the weight and the membrane voltage of the current output channel to be processed can be stored in the shared storage area of the current neuron core; wherein the weight and the membrane voltage are dynamically allocated to the shared storage area with a storage space size.

[0042] In some embodiments of the present embodiment, the dynamic allocation of the weight and membrane voltage in the shared storage area includes: the weight and the membrane voltage are stored in the shared storage area in a unified storage manner; and the weight storage space size and the membrane voltage storage space size are dynamically adjustable under the condition that the sum of the weight storage space size and the membrane voltage storage space size is constant. For example, the size of a shared storage area is 48 KB, and the weight storage space size and the membrane voltage storage space size can be dynamically changed, such as being configured as 32 KB weight storage space+16 KB membrane voltage storage space, or being configured as 20 KB weight storage space+28 KB membrane voltage storage space, or being configured as 36 KB weight storage space+12 KB membrane voltage storage space, and so on.

[0043] Step S13: If the current neuron core can store the weight and the membrane voltage of the current output channel to be processed, the weight and the membrane voltage of the current output channel to be processed are stored in the shared storage area of the current neuron core; otherwise, the weight and the membrane voltage of the current output channel to be processed are stored in the shared storage area of a new neuron core.

[0044] For the convenience of understanding, the following will be described in combination with Figure 2The process and principle of the weight storage and membrane voltage storage dynamic allocation method are further illustrated by the embodiments in the above table. In the table: output_channel is the number of output channels to be mapped to the current layer; out_channel_index is the current count value of the output channel in the allocation process, starting from 1 and counting to output_channel; channel_step is the minimum number of output channels allocated each time, generally configured as 1, but can also be configured as other numbers, such as 4 or 8, etc.; potential_size is the size of the membrane voltage storage space allocated in the current neuron core, initialized as 0; weight_size is the size of the weight storage space allocated in the current neuron core, initialized as 0; weight_result is the weight content stored in the current neuron core; core_index is the neuron core count, initialized as 1; when the number of output channels of the current layer is large, the storage space of one neuron core is not enough, multiple neuron cores are needed to complete the allocation together, and core_index is used to represent the current count value of the neuron core in the allocation process; total_size is the size of the shared storage space of the weight and the membrane voltage; if out_channel_index>output_channel, it indicates that all output channels have been processed, the compressed weight weight_result of the current neuron core needs to be output to the corresponding weight file, and the allocation process ends; if out_channel_index≤output_channel, it indicates that part of the output channels have not been processed, and these output channels will continue to be processed; real_step is the number of output channels to be processed this time, if the number of remaining unprocessed channels (output_channel-out_channel_index+1) is less than channel_step, then the remaining unprocessed channels are processed at one time this time; otherwise, channel_step output channels are processed.

[0045] Step S21: initialize the count value of the output channel in the allocation process, the size of the membrane voltage storage space allocated in the current neuron core, the size of the weight storage space allocated in the current neuron core, the current count value of the neuron core, and the weight content stored in the current neuron core.

[0046] For example, the count value of the output channel in the allocation process (out_channel_index) is initialized as 1 (counting from 1); the allocated membrane voltage storage space size in the current neuron core (potential_size) is initialized as 0; the allocated weight storage space size in the current neuron core (weight_size) is initialized as 0; the current count value of the neuron core (core_index) is initialized as 1; and the stored weight content in the current neuron core (weight_result) is initialized as empty. That is, out_channel_index = 1; potential_size = 0; weight_size = 0; core_index = 1; and weight_result = [].

[0047] Step S22: Determine whether all output channels that need to be mapped to the current layer have been processed.

[0048] For example, it can be determined whether the count value of the output channel in the allocation process is less than or equal to the number of all output channels that need to be mapped to the current layer (out_channel_index≤output_channel); if yes, it indicates that all output channels that need to be mapped to the current layer have not been processed; otherwise, it indicates that all output channels that need to be mapped to the current layer have been processed.

[0049] Step S23: If all output channels have not been processed, calculate the number of output channels to be processed this time, the required weight storage space size this time, and the required membrane voltage storage space size this time.

[0050] In some examples, the number of output channels to be processed this time (real_step) is calculated in the following manner: the minimum output channel number (channel_step) allocated each time, the comprehensive calculation value obtained by subtracting the current count value of the output channel (out_channel_index) plus 1 from the number of all output channels of the current layer (output_channel); and the smaller one of the two is selected as the number of output channels to be processed this time, that is:

[0051] Real_step = min(channel_step, (output_channel - out_channel_index + 1)).

[0052] In some examples, the add_weight_size is calculated by using a compression algorithm and based on the current count of output channels (out_channel_index) and the real_step. Thus, the add_weight_size can be represented as:

[0053] add_weight_size = compress_weight(out_channel_index, real_step);

[0054] where the compress_weight() function is a compression algorithm that compresses a string and returns a binary string; the compressed string can be decompressed by the UNCOMPRESS() function. In this embodiment, the compress_weight() function compresses the weights of the real_step output channels starting from the out_channel_index output channel to obtain the add_weight_size; if the weight storage space does not need to be compressed, the compress_weight() function gives the add_weight_size of the uncompressed weight storage space.

[0055] In some examples, the add_potential_size is calculated by using the cal_potential function and based on the real_step. Thus, the add_potential_size can be represented as:

[0056] add_potential_size = cal_potential(real_step).

[0057] Further, after the real_step, the add_weight_size and the add_potential_size are calculated, the current count of output channels (out_channel_index) is updated to the sum of the previous count and the real_step.

[0058] Step S24: judging whether the sum of the membrane voltage total storage space size and the weight total storage space size exceeds the total storage space size in the current neuron core.

[0059] Specifically, the membrane voltage total storage space size is the sum of the allocated membrane voltage storage space size (potential_size) in the current neuron core and the required membrane voltage storage space size (add_potential_size) in this time; and the weight total storage space size is the sum of the allocated weight storage space size (weight_size) in the current neuron core and the required weight storage space size (add_weight_size) in this time. The sum of the membrane voltage total storage space size and the weight total storage space size (potential_size+add_potential_size+weight_size+add_weight_size) is compared with the total storage space size total_size in the current neuron core.

[0060] Step S25: if the sum exceeds, the weights and the membrane voltage corresponding to the output channels not yet processed are stored in a new neuron core.

[0061] If (potential_size+add_potential_size+weight_size+add_weight_size)>total_size, it indicates that the current neuron core can only store the output channels before the current count value (out_channel_index) of the output channels, and the weights and the membrane voltage corresponding to the continuous real_step output channels starting from the current count value out_channel_index of the output channels need to be put into a new neuron core. Therefore, the compressed weights weight_result of the current neuron core need to be output to the corresponding weight file; the current count value (core_index) of the neuron core is added by 1; the allocated weight storage space size (weight_size) in the current neuron core and the allocated membrane voltage storage space size (potential_size) in the current neuron core start to count again; the weight content (weight_result) stored in the current neuron core is emptied and the weight content corresponding to the real_step output channels currently processed is saved.

[0062] Step S26: if the sum does not exceed, the weights and the membrane voltage corresponding to the output channels not yet processed are still stored in the current neuron core.

[0063] If (potential_size + add_potential_size + weight_size + add_weight_size) ≤ total_size, real_step output channels are still allocated to the current neuron core, and the allocated weight storage size (weight_size) in the current neuron core, the allocated membrane potential storage size (potential_size) in the current neuron core are updated, and the compressed weights corresponding to the new real_step output channels are added to the weight content (weight_result) stored in the current neuron core.

[0064] After the above steps are performed, return to step S22 until all output channels are processed.

[0065] Step S27: If all output channels have been processed, the weight content (weight_result) stored in the previous neuron core is output to the corresponding weight file, and the allocation process is ended.

[0066] The dynamic allocation method of weight storage and membrane potential storage provided by the embodiment does not require pre-determination of weight storage in hardware design, nor pre-computation by software; the present application can fully utilize shared storage space, and the maximum wasted space is only the minimum number of output channels allocated each time, so in the case of the minimum number of output channels allocated each time being 1, the reasonable weight and membrane potential shared storage space size can make the wasted space less than 5%, greatly improving the storage space utilization and improving the use efficiency of storage resources; at the same time, although the dynamic allocation space provided by the embodiment requires computing power due to online weight compression, the weight compression process occurs in the compiler, and the compiler usually runs in a desktop computer or server with relatively large computing power, so even if the compiler runs for a long time, it is acceptable, which does not affect the hardware efficiency and software efficiency in the inference process.

[0067] The dynamic allocation method of weight storage and membrane potential storage provided by the embodiment can be implemented on the terminal side or the server side. As for the hardware structure of the dynamic allocation terminal of weight storage and membrane potential storage, please refer to Figure 3An optional hardware structure diagram of the terminal 300 for the dynamic allocation of weight storage and membrane voltage storage provided by the embodiment of the present application is shown in FIG. 3. The terminal 300 can be a mobile phone, a computer device, a tablet device, a personal digital processing device, a factory background processing device, etc. The terminal 300 for the dynamic allocation of weight storage and membrane voltage storage includes at least one processor 301, a memory 302, at least one network interface 304, and a user interface 306. The various components in the apparatus are coupled together by a bus system 305. It can be understood that the bus system 305 is used to realize the connection communication between the components. The bus system 305 includes a data bus, a power supply bus, a control bus, and a status signal bus, but for the purpose of clear illustration, all the buses are marked as the bus system in FIG. 3. Figure 3

[0068] The user interface 306 can include a display, a keyboard, a mouse, a trackball, a click gun, a key, a button, a touchpad, or a touch screen, etc.

[0069] It can be understood that the memory 302 can be a volatile memory or a non-volatile memory, or both. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of example but not limitation, many forms of RAM can be used, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiment of the present application is intended to include but not limited to these and any other suitable categories of memory.

[0070] The memory 302 in the embodiment of the present application is used to store various categories of data to support the operation of the terminal 300 for the dynamic allocation of weight storage and membrane voltage storage. Examples of these data include any executable programs for operating on the terminal 300 for the dynamic allocation of weight storage and membrane voltage storage, such as an operating system 3021 and an application program 3022. The operating system 3021 contains various system programs, such as a framework layer, a core library layer, a driver layer, etc., for realizing various basic services and processing hardware-based tasks. The application program 3022 can contain various application programs, such as a media player (MediaPlayer), a browser (Browser), etc., for realizing various application services. The method for the dynamic allocation of weight storage and membrane voltage storage provided by the embodiment of the present application can be contained in the application program 3022. ​

[0071] The method disclosed in the embodiments of the present application can be applied to or implemented by the processor 301. The processor 301 can be an integrated circuit chip having a signal processing capability. In the implementation process, each step of the method can be completed by hardware integrated logic circuits or software form instructions in the processor 301. The processor 301 can be a general processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 301 can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application. The general processor 301 can be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of the present application can be directly embodied as hardware decoding processor for execution, or executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory to complete the steps of the foregoing method in combination with the hardware.

[0072] In the exemplary embodiments, the dynamic allocation of weight storage and membrane voltage storage terminal 300 can be one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), used to execute the foregoing method.

[0073] As shown in Figure 4 The structure of a dynamic allocation of weight storage and membrane voltage storage device in the embodiments of the present application is shown. In this embodiment, the dynamic allocation of weight storage and membrane voltage storage device 400 includes a storage space module 401 and a dynamic allocation module 402.

[0074] The storage space module 401 is configured to compress the weight storage space of a current output channel to be processed online to obtain a corresponding weight storage space size, and obtain a membrane voltage storage space size corresponding to the current output channel to be processed.

[0075] In some examples, the dynamic allocation of the weight and the membrane voltage in the shared storage area includes: the weight and the membrane voltage are stored in the shared storage area in a unified storage manner; and the weight storage space size and the membrane voltage storage space size are dynamically adjustable under the condition that the sum of the weight storage space size and the membrane voltage storage space size is fixed.

[0076] In some examples, the dynamic allocation module 402 judges whether the weight and the membrane voltage of the current output channel to be processed can be stored in the shared storage area of the current neuron core according to the weight storage space size and the membrane voltage storage space size of the current output channel to be processed, including: initializing a current count value of the output channel, an allocated membrane voltage storage space size in the current neuron core, an allocated weight storage space size in the current neuron core, a current count value of the neuron core, and weight content stored in the current neuron core; judging whether all output channels that need to be mapped to the current layer are processed; if all output channels are not processed, calculating the number of output channels to be processed this time, the weight storage space size required this time, and the membrane voltage storage space size required this time; comparing the sum of the membrane voltage total storage space size and the weight total storage space size with the total storage space size in the current neuron core; if the sum of the membrane voltage total storage space size and the weight total storage space size exceeds the total storage space size in the current neuron core, judging that the weight and the membrane voltage of the current output channel to be processed cannot be stored in the shared storage area of the current neuron core; otherwise, judging that the weight and the membrane voltage of the current output channel to be processed can be stored in the shared storage area of the current neuron core.

[0077] In some examples, the number of output channels to be processed this time is calculated in the following manner: obtaining a minimum output channel number allocated each time and a comprehensive calculation value obtained by subtracting the current count value of the output channel from the total number of output channels of the current layer and then adding 1; and selecting the smaller one of the two as the number of output channels to be processed this time.

[0078] In some examples, the calculation of the required weight storage space size for the current time comprises: using a compression algorithm and calculating based on the current count value of the output channel and the number of output channels to be processed this time.

[0079] In some examples, the calculation of the required membrane voltage storage space size for the current time comprises: using a cal_potential function and calculating based on the number of output channels to be processed this time.

[0080] In some examples, the total membrane voltage storage space size is the sum of the allocated membrane voltage storage space size in the current neuron core and the required membrane voltage storage space size for the current time; and the total weight storage space size is the sum of the allocated weight storage space size in the current neuron core and the required weight storage space size for the current time.

[0081] It should be noted that the above-described dynamic allocation device for weight storage and membrane voltage storage only uses the above-described division of program modules as an example for dynamic allocation of weight storage and membrane voltage storage. In actual applications, the above-described processing can be completed by different program modules according to needs, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-described processing. In addition, the dynamic allocation device for weight storage and membrane voltage storage and the dynamic allocation method for weight storage and membrane voltage storage of the above-described embodiments belong to the same concept, and the specific implementation process is described in detail in the method embodiments, which will not be repeated here.

[0082] The embodiment of the present application also provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the dynamic allocation method for weight storage and membrane voltage storage.

[0083] Those skilled in the art can understand that all or part of the steps of the above-described method embodiments can be completed by computer program related hardware. The aforementioned computer program can be stored in a computer readable storage medium. The program, when executed, performs steps including the above-described method embodiments; and the aforementioned storage medium includes ROM, RAM, magnetic disc or optical disc and various media that can store program codes.

[0084] In the embodiments provided in the present application, the computer readable and writable storage medium can include read-only memory, random access memory, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage device, flash memory, U disk, mobile hard disk, or any other medium capable of storing desired program code in the form of instructions or data structures and capable of being accessed by a computer. In addition, any connection can be appropriately referred to as a computer readable medium. For example, if instructions are sent from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology such as infrared, radio and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave is included in the definition of the medium. However, it should be understood that the computer readable and writable storage medium and the data storage medium do not include connections, carriers, signals or other temporary media, but are intended for non-transitory, tangible storage media. As used in the application, magnetic disks and optical disks include compact disks (CD), laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, in which magnetic disks usually magnetically copy data, and optical disks use lasers to optically copy data.

[0085] In summary, the present application provides a dynamic allocation method, device, terminal and medium for weight storage and membrane voltage storage. The present application provides a method for improving the dynamic allocation efficiency of weight storage and membrane voltage storage. Therefore, the present application effectively overcomes the various shortcomings in the prior art and has high industrial utilization value.

[0086] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not used to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes completed by those skilled in the art without departing from the spirit and technical idea disclosed in the present application should be covered by the claims of the present application.

Claims

1. A method of dynamic allocation of weight storage and membrane voltage storage, characterized in that, The method comprises the following steps: on-line compression of the weight storage space of the current output channel to be processed to obtain the corresponding weight storage space size, and obtaining the membrane voltage storage space size corresponding to the current output channel to be processed; determining whether the weight and the membrane voltage of the current output channel to be processed can be stored in the shared storage area of the current neuron core according to the weight storage space size and the membrane voltage storage space size of the current output channel to be processed; wherein the weight and the membrane voltage are dynamically allocated with storage space size in the shared storage area, which comprises that the weight and the membrane voltage are stored in the shared storage area in a unified storage manner; under the condition that the sum of the weight storage space size and the membrane voltage storage space size is constant, the weight storage space size and the membrane voltage storage space size are dynamically adjustable; if the weight and the membrane voltage of the current output channel to be processed can be stored in the shared storage area of the current neuron core, the weight and the membrane voltage of the current output channel to be processed are stored in the shared storage area of the current neuron core, otherwise, the weight and the membrane voltage are stored in the shared storage area of a new neuron core.

2. The method of dynamic allocation of weight storage and membrane voltage storage of claim 1, wherein, The method for determining whether the weight and the membrane voltage of the current output channel to be processed can be stored in the shared storage area of the current neuron core according to the weight storage space size and the membrane voltage storage space size of the current output channel to be processed comprises: initializing the current count value of the output channel, the allocated membrane voltage storage space size in the current neuron core, the allocated weight storage space size in the current neuron core, the current count value of the neuron core and the weight content stored in the current neuron core; determining whether all the output channels needed to be mapped to the current layer are processed; if all the output channels are not processed, calculating the number of output channels to be processed this time, the required weight storage space size this time and the required membrane voltage storage space size this time; comparing the sum of the total membrane voltage storage space size and the total weight storage space size with the total storage space size in the current neuron core; if the sum of the total membrane voltage storage space size and the total weight storage space size exceeds the total storage space size in the current neuron core, it is determined that the weight and the membrane voltage of the current output channel to be processed cannot be stored in the shared storage area of the current neuron core; otherwise, it is determined that the weight and the membrane voltage of the current output channel to be processed can be stored in the shared storage area of the current neuron core.

3. The method of dynamic allocation of weight storage and membrane voltage storage of claim 2, wherein, The calculation method of the number of output channels to be processed this time comprises: respectively obtaining the minimum output channel number allocated each time and the comprehensive calculation value obtained by subtracting the current count value of the output channel from the total number of output channels of the current layer and then adding 1; selecting the smaller one of the two as the number of output channels to be processed this time.

4. The method of dynamic allocation of weight storage and membrane voltage storage of claim 2, wherein, The calculation method of the required weight storage space size this time comprises: using a compression algorithm and calculating based on the current count value of the output channel and the number of output channels to be processed this time.

5. The method of dynamic allocation of weight storage and membrane voltage storage of claim 2, wherein, The calculation method of the required membrane voltage storage space size this time comprises: using a cal_potential function and calculating based on the number of output channels to be processed this time.

6. The method of dynamic allocation of weight storage and membrane voltage storage of claim 2, wherein, The total membrane voltage storage space size is the sum of the allocated membrane voltage storage space size in the current neuron core and the required membrane voltage storage space size this time; and the total weight storage space size is the sum of the allocated weight storage space size in the current neuron core and the required weight storage space size this time.

7. An apparatus for dynamic allocation of weight storage and membrane voltage storage, characterized by, The method comprises the steps of: a storage space module, which is configured to compress the weight storage space of the current output channel to be processed online to obtain a corresponding weight storage space size; obtaining the membrane voltage storage space size corresponding to the current output channel to be processed; a dynamic allocation module, which is configured to determine whether the weight and the membrane voltage of the current output channel to be processed can be stored in the shared storage area of the current neuron core according to the weight storage space size and the membrane voltage storage space size of the current output channel to be processed; wherein the dynamic allocation of the storage space size of the weight and the membrane voltage in the shared storage area comprises: the weight and the membrane voltage are stored in the shared storage area in a unified storage manner; the weight storage space size and the membrane voltage storage space size are dynamically adjustable under the condition that the sum of the weight storage space size and the membrane voltage storage space size is constant; if the weight and the membrane voltage of the current output channel to be processed can be stored in the shared storage area of the current neuron core, the weight and the membrane voltage of the current output channel to be processed are stored in the shared storage area of the current neuron core; otherwise, the weight and the membrane voltage are stored in the shared storage area of a new neuron core.

8. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the dynamic allocation method of the weight storage and the membrane voltage storage according to any one of claims 1 to 6.

9. An electronic terminal, characterized in that The method comprises the steps of: a processor and a memory; the memory is configured to store a computer program; the processor is configured to execute the computer program stored in the memory, so that the terminal executes the dynamic allocation method of the weight storage and the membrane voltage storage according to any one of claims 1 to 6.

Images