Data online compression method and device of neural network hardware accelerator

Abstract

The application discloses a data online compression method and device of a neural network hardware accelerator. The method comprises converting a first activation value output by a neural network to obtain a first activation mask; dividing the first activation mask into at least two groups of activation sub-masks, and sequentially performing accumulation processing on each group of activation sub-masks in a preset order to obtain an activation position mask; calculating an activation selection mask based on the first activation mask, the activation position mask and a weight value output by the neural network; performing screening processing on the first activation value according to the activation selection mask to obtain a target activation value, and generating a second activation mask based on the target activation value. Through the online mask setting of the activation value and the offline compression of the weight value, the adaptability to different neural network compressions is high, the data moving efficiency can be improved, the power consumption is reduced, and the throughput is ensured.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of data compression, and particularly relates to a data online compression method and device of a neural network hardware accelerator. BACKGROUND

[0002] The existing neural network hardware accelerator is affected by the fact that the scale of neural network data is becoming larger and larger. For example, due to the existence of a large number of activation values and weight values, a large-scale calculation and a large-scale access transmission between different memory levels are caused. Therefore, there is a great challenge in the efficiency of the existing neural network hardware accelerator in processing the activation values and weight values caused in the neural network. On the other hand, in the operation process of the deep neural network, the output activation values of the neural network are set to zero through the nonlinear activation function RELU, which causes the sparsity to be further increased.

[0003] In view of the sparsity characteristics in the neural network, a specific compression method can be set to improve the effectiveness of data movement between different memory levels through compression of the activation values and weight values, and further improve the efficiency of the neural network hardware accelerator.

[0004] However, the traditional CPU or GPU is limited by its architecture and resources, and the operation of the neural network in the traditional embedded device is low in efficiency and low in energy efficiency. Some neural network hardware accelerators are not friendly to the neural network of the army, and in the process of realizing the accelerated operation of the neural network, only the regular access mode of the DNN can be used for calculation, and the operation efficiency cannot be improved in the sparse calculation. Secondly, some neural network hardware accelerators use compression methods to improve the storage density and efficiency, but the decompression process is too cumbersome, and the compressed data needs to be restored to the original data structure, which still wastes resources on invalid calculation and cannot take advantage of the irregularity of the neural network. In addition, some neural network hardware accelerators are too cumbersome in selecting effective calculation, causing problems such as low energy efficiency and low data throughput. SUMMARY

[0005] To solve the above-mentioned technical problems that in the process of realizing the accelerated operation of the neural network, only the regular access mode of the DNN can be used for calculation, and the operation efficiency cannot be improved in the sparse calculation, the decompression process is too cumbersome, and the compressed data needs to be restored to the original data structure, the application provides a data online compression method and device of a neural network hardware accelerator, and the technical scheme is as follows:

[0006] In a first aspect, the application embodiment provides a data online compression method of a neural network hardware accelerator, which comprises the following steps:

[0007] transforming the first activation value of the neural network output to obtain a first activation mask;

[0008] dividing the first activation mask into at least two groups of activation sub-masks, and sequentially performing accumulation processing on each group of activation sub-masks in a preset order to obtain an activation position mask;

[0009] calculating an activation selection mask based on the first activation mask, the activation position mask, and a weight value of the neural network output;

[0010] performing screening processing on the first activation value according to the activation selection mask to obtain a target activation value, and generating a second activation mask based on the target activation value.

[0011] In an optional solution of the first aspect, sequentially performing accumulation processing on each group of activation sub-masks in a preset order to obtain an activation position mask comprises:

[0012] performing accumulation processing on the n th group of activation sub-masks based on the n th adder, and calculating an n th group of activation position sub-masks according to the (n-1) th accumulation number and the processed n th group of activation sub-masks; wherein n is a positive integer greater than or equal to 2;

[0013] determining the n th accumulation number based on the last bit of the n th group of activation position sub-masks and the (n-1) th accumulation number;

[0014] performing accumulation processing on the (n+1) th group of activation sub-masks based on the (n+1) th adder, and calculating an (n+1) th group of activation position sub-masks according to the n th accumulation number and the processed (n+1) th group of activation sub-masks;

[0015] performing merging processing on all activation position sub-masks to obtain an activation position mask.

[0016] In another optional solution of the first aspect, calculating an activation selection mask based on the first activation mask, the activation position mask, and a weight value of the neural network output comprises:

[0017] inputting the weight value of the neural network output into the trained model to obtain a first weight mask;

[0018] performing transformation processing on the first weight mask to obtain a second weight mask consistent in character number with the first activation mask;

[0019] calculating an activation selection mask based on the first activation mask, the activation position mask, and the second weight mask.

[0020] In another optional solution of the first aspect, before calculating an activation selection mask based on the first activation mask, the activation position mask, and the second weight mask, the method further comprises:

[0021] When it is detected that any at least one group of activation sub-masks in the first activation mask contains a non-zero value, a first activation mask judgment signal corresponding to the activation sub-mask containing the non-zero value is generated;

[0022] When it is detected that any at least one group of activation sub-masks in the first activation mask contains a non-zero value, a first activation mask judgment signal corresponding to the activation sub-mask containing the non-zero value is generated;

[0023] According to the arrangement order of each group of activation sub-masks in the first activation mask, the first activation mask judgment signal and the second activation mask judgment signal are combined to obtain a target activation mask judgment signal;

[0024] Based on the first activation mask, the activation position mask and the second weight mask, an activation selection mask is calculated, including:

[0025] Based on the activation position mask, the second weight mask and the target activation mask judgment signal, the activation selection mask is calculated.

[0026] In another optional solution of the first aspect, based on the activation position mask, the second weight mask and the target activation mask judgment signal, the activation selection mask is calculated, including:

[0027] Based on the target activation mask judgment signal, the second weight mask is screened to obtain a third weight mask;

[0028] The third weight mask and the activation position mask are input into a selector to obtain the activation selection mask.

[0029] In another optional solution of the first aspect, the second activation mask is generated based on the target activation value, including:

[0030] The target activation value is divided into at least two groups of sub-activation values;

[0031] When any at least one group of sub-activation values contains a non-zero value, a second activation sub-mask is generated;

[0032] According to the arrangement order of each group of sub-activation values in the target activation value, the second activation mask is generated based on all the second activation sub-masks.

[0033] In another optional solution of the first aspect, the method further includes:

[0034] The target activation value and the second activation mask are input into a neural network;

[0035] The second activation value output by the neural network is converted.

[0036] In a second aspect, the embodiments of the present application provide a data online compression device of a neural network hardware accelerator, comprising:

[0037] a processing unit module configured to perform conversion processing on first activation values output by the neural network to obtain a first activation mask;

[0038] a first data compression module configured to divide the first activation mask into at least two groups of activation sub-masks, and perform accumulation processing on each group of activation sub-masks in a preset order to obtain an activation position mask;

[0039] a second data compression module configured to calculate an activation selection mask based on the first activation mask, the activation position mask, and weight values output by the neural network;

[0040] a third data compression module configured to perform screening processing on the first activation values according to the activation selection mask to obtain target activation values, and generate a second activation mask based on the target activation values.

[0041] In an optional solution of the second aspect, the first data compression module is specifically configured to:

[0042] perform accumulation processing on the n th group of activation sub-masks based on the n th group of adders, and calculate an n th group of activation position sub-masks according to an (n-1) th accumulation number and the n th group of activation sub-masks after processing; wherein n is a positive integer greater than or equal to 2;

[0043] determine an n th accumulation number based on a last bit of the n th group of activation position sub-masks and the (n-1) th accumulation number;

[0044] perform accumulation processing on an (n+1) th group of activation sub-masks based on the (n+1) th group of adders, and calculate an (n+1) th group of activation position sub-masks according to the n th accumulation number and the (n+1) th group of activation sub-masks after processing;

[0045] perform merging processing on all the activation position sub-masks to obtain the activation position mask.

[0046] In another optional solution of the second aspect, the second data compression module is specifically configured to:

[0047] input the weight values output by the neural network into the trained model to obtain a first weight mask;

[0048] perform conversion processing on the first weight mask to obtain a second weight mask with a number of characters consistent with the first activation mask;

[0049] calculate the activation selection mask based on the first activation mask, the activation position mask, and the second weight mask.

[0050] In a further alternative of the second aspect, the second data compression module is specifically further configured to:

[0051] Before the activated selection mask is calculated based on the first activated mask, the activated position mask and the second weight mask, when it is detected that any at least one group of activated sub-masks in the first activated mask contains a non-zero value, a first activated mask judgment signal corresponding to the activated sub-mask containing the non-zero value is generated;

[0052] When it is detected that any at least one group of activated sub-masks in the first activated mask contains a non-zero value, a first activated mask judgment signal corresponding to the activated sub-mask containing the non-zero value is generated;

[0053] According to the arrangement order of each group of activated sub-masks in the first activated mask, the first activated mask judgment signal and the second activated mask judgment signal are merged to obtain a target activated mask judgment signal;

[0054] The activated selection mask is calculated based on the first activated mask, the activated position mask and the second weight mask, including:

[0055] The activated selection mask is calculated based on the activated position mask, the second weight mask and the target activated mask judgment signal.

[0056] In a further alternative of the second aspect, the second data compression module is specifically further configured to:

[0057] The second weight mask is filtered based on the target activated mask judgment signal to obtain a third weight mask;

[0058] The third weight mask and the activated position mask are input into the selector to obtain the activated selection mask.

[0059] In a further alternative of the second aspect, the third data compression module is specifically configured to:

[0060] The target activated value is divided into at least two groups of sub-activated values;

[0061] When any at least one group of sub-activated values contains a non-zero value, a second activated sub-mask is generated;

[0062] According to the arrangement order of each group of sub-activated values in the target activated value, a second activated mask is generated based on all the second activated sub-masks.

[0063] In a further alternative of the second aspect, the device further comprises:

[0064] The target activated value and the second activated mask are input into the neural network;

[0065] The second activated value output by the neural network is converted.

[0066] In a third aspect, the embodiments of the present application further provide a data online compression device of a neural network hardware accelerator, comprising a processor and a memory;

[0067] The processor is connected with the memory;

[0068] The memory is used for storing executable program codes;

[0069] The processor runs a program corresponding to the executable program codes by reading the executable program codes stored in the memory, so as to implement the data online compression method of the neural network hardware accelerator provided in the first aspect of the embodiments of the present application or any one of the implementation manners of the first aspect.

[0070] In a fourth aspect, the embodiments of the present application provide a computer storage medium, which stores a computer program, and the computer program comprises program instructions. When the program instructions are executed by a processor, the data online compression method of the neural network hardware accelerator provided in the first aspect of the embodiments of the present application or any one of the implementation manners of the first aspect can be implemented.

[0071] In the embodiments of the present application, when the data of the neural network hardware accelerator is compressed online, the first activation value output by the neural network is converted to obtain a first activation mask; the first activation mask is divided into at least two groups of activation sub-masks, and each group of activation sub-masks is sequentially accumulated according to a preset order to obtain an activation position mask; based on the first activation mask, the activation position mask and the weight value output by the neural network, an activation selection mask is calculated; the first activation value is screened according to the activation selection mask to obtain a target activation value, and a second activation mask is generated based on the target activation value. Through the online mask setting of the activation value and the offline compression of the weight value, not only the adaptability to different neural network compression is strong, but also the data moving efficiency is improved, the throughput is ensured while the power consumption is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0072] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required to be used in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0073] Figure 1 The overall flowchart of the data online compression method of the neural network hardware accelerator provided in the embodiments of the present application;

[0074] Figure 2 The application structure diagram of the data compression method of the neural network hardware accelerator provided in the embodiments of the present application;

[0075] Figure 3 A schematic diagram illustrating the calculation process of an activation location mask provided in an embodiment of this application;

[0076] Figure 4 A schematic diagram illustrating the calculation process of an activation selection mask provided in an embodiment of this application;

[0077] Figure 5 A schematic diagram of the structure of an online data compression device for a neural network hardware accelerator provided in this application embodiment;

[0078] Figure 6 This is a schematic diagram of the structure of another online data compression device for a neural network hardware accelerator provided in this application embodiment. Detailed Implementation

[0079] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0080] In the following description, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The following description provides multiple embodiments of this application, which can be substituted or combined with each other. Therefore, this application can also be considered to include all possible combinations of the same and / or different embodiments described. Thus, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then this application should also be considered to include embodiments containing one or more other possible combinations of A, B, C, and D, even if such embodiments are not explicitly described in the following text.

[0081] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the described elements without departing from the scope of this application. Various processes or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.

[0082] Please see Figure 1 , Figure 1 This paper illustrates the overall process of an online data compression method for a neural network hardware accelerator provided in an embodiment of this application.

[0083] like Figure 1 As shown, the online data compression method for this neural network hardware accelerator may include at least the following steps:

[0084] Step 102, performing conversion processing on the first activation value output by the neural network to obtain a first activation mask.

[0085] In the embodiments of the present application, the data online compression method of the neural network hardware accelerator can be applied in, but is not limited to, a neural network, a mobile terminal for running the neural network, or a server terminal for running the neural network. When the neural network, the mobile terminal for running the neural network, or the server terminal for running the neural network executes the data online compression method of the neural network hardware accelerator, the neural network, the mobile terminal for running the neural network, or the server terminal for running the neural network can also, but is not limited to, compress the weight value output by the neural network through an offline compression manner, and obtain an online compression result of the activation value in combination with the compression result of the weight value.

[0086] It can be understood that the neural network, the mobile terminal for running the neural network, or the server terminal for running the neural network can also be provided with a weight cache module for storing the weight value, a weight mask cache module for storing the weight mask, an activation cache module for storing the activation value, and an activation mask cache module for storing the activation mask, in order to distinguish the weight value and the activation value. The neural network, the mobile terminal for running the neural network, or the server terminal for running the neural network can also be provided with a mask processing module for performing data processing on the weight mask and the activation mask, and a data online compression module for processing the activation value and the weight value output by the neural network.

[0087] For details, refer to Figure 2 An application structure schematic diagram of the data compression method of the neural network hardware accelerator provided by the embodiments of the present application is shown. As shown in Figure 2As shown, the processing unit module can be used to perform the operation process of the neural network, and can send the output result of the neural network to the data online compression module. The data online compression module can be used to compress the activation values in the output result of the neural network according to a preset format, generate a corresponding activation mask, and output the activation values to the activation cache module of the external data storage module and the activation mask to the activation mask cache module of the external mask storage module, respectively; At the same time, it can also compress the weight values in the output result of the neural network according to a preset format, generate a corresponding weight mask, and output the weight values to the weight cache module of the external data storage module and the weight mask to the weight mask cache module of the external mask storage module, respectively. It can be understood that the way of compressing the weight values can also be offline compression, that is, the data online compression module can output the weight values in the output result of the neural network to other compression modules to compress the weight values by offline compression to obtain the corresponding weight mask, and then output the weight values to the weight cache module of the external data storage module and the weight mask to the weight mask cache module of the external mask storage module, respectively. Wherein, the activation cache module can be but not limited to the activation value ping-pong cache module, that is, three activation value cache areas are provided to ensure the flow operation in the data moving process.

[0088] Figure 2 The mask processing module in the data online compression module can be used to obtain the activation mask output by the activation mask cache module and obtain the weight mask output by the weight mask cache module, to generate a data selection instruction in combination with the activation mask and the weight mask, and output the data selection instruction to the activation cache module and the weight cache module, respectively, so that the activation cache module can output the specified activation value to the processing unit module, and the weight cache module can output the specified weight value to the processing unit module, thereby realizing the subsequent operation process of the neural network in the processing unit module.

[0089] Specifically, when the data of the neural network hardware accelerator is online compressed, the first activation value can be determined from the calculation result output by the neural network, and can be but not limited to converted according to a preset format to obtain a first activation mask corresponding to the first activation value. Wherein, the preset format can be understood as dividing the first activation value according to a preset number of characters, and converting the non-zero value in each activation value divided into character 1, and converting the zero value in each activation value divided into character 0. Herein, please refer to the following activation value-activation mask conversion table:

[0090]

[0091] It can be understood that in the embodiments of the present application, the first activation value can be divided by every four characters, the non-zero value is converted into character 1, and the zero value is converted into character 0, for example, when the four characters of the activation value can be represented as "non-zero, 0, 0, non-zero", the corresponding activation mask can be represented as "1001".

[0092] In step 104, the first activation mask is divided into at least two groups of activation sub-masks, and each group of activation sub-masks is sequentially accumulated in a preset order to obtain an activation position mask.

[0093] Specifically, after the first activation value is compressed and converted to obtain the first activation mask, the first activation mask can be equally divided into at least two groups of activation sub-masks, and the number of characters in each group of activation sub-masks remains consistent, for example, but not limited to, each group of activation sub-masks can include four characters, and the character types in each group of activation sub-masks include 0 and 1.

[0094] Further, after obtaining the at least two groups of activation sub-masks, each group of activation sub-masks can be assigned a corresponding adder, for example, but not limited to, each character in each group of activation sub-masks can correspond to an adder, and the nth group of activation sub-masks is accumulated based on the nth group of adders, and the nth group of activation position sub-masks is calculated based on the (n-1)th accumulated number and the processed nth group of activation sub-masks. Then, the last character of the nth group of activation position sub-masks and the (n-1)th accumulated number are accumulated to obtain the nth accumulated number. Then, the (n+1)th group of activation sub-masks is accumulated based on the (n+1)th group of adders, and the (n+1)th group of activation position sub-masks is calculated based on the nth accumulated number and the processed (n+1)th group of activation sub-masks. The above operation is repeatedly performed until the activation position sub-masks corresponding to each group of activation sub-masks are obtained, and the activation position mask is obtained by merging the activation position sub-masks, wherein n is a positive integer greater than or equal to 2, and the calculation method of the activation position sub-masks corresponding to the first group of activation sub-masks can be consistent with the above method, which will not be described here.

[0095] Herein Figure 3 The calculation process of the activation position mask provided by the embodiments of the present application is shown in the figure. As Figure 3As shown, taking the first group of activation sub-masks 10010000 as an example, the first group of activation position sub-masks obtained through processing by the eight adders can be 11222222, and the first accumulated number can be calculated as 2 in combination with the accumulated number 0 obtained in the last mask calculation and the last character 2 of the first group of activation position sub-masks. Then, the second group of activation sub-masks is 00100110, the second group of activation position sub-masks obtained through processing by the eight adders can be 00111233, and the second accumulated number can be calculated as 5 in combination with the first accumulated number and the last character 3 of the second group of activation position sub-masks. Then, the third group of activation sub-masks is 00000001, the third group of activation position sub-masks obtained through processing by the eight adders can be 00000001, and the third accumulated number can be calculated as 6 in combination with the second accumulated number and the last character 1 of the third group of activation position sub-masks. Then, the fourth group of activation sub-masks is 00110010, the fourth group of activation position sub-masks obtained through processing by the eight adders can be 00122233, and the fourth accumulated number can be calculated as 9 in combination with the third accumulated number and the last character 3 of the fourth group of activation position sub-masks.

[0096] It should be noted that after obtaining each group of activation position sub-masks and the corresponding accumulated number, the first accumulated number 2 and the second group of activation position sub-masks 00111233 can be combined to obtain the second group of activation position sub-masks actually arranged behind the first group of activation position sub-masks 11222222, which is 22333455. Then, the second accumulated number 5 and the third group of activation position sub-masks 00000001 can be combined to obtain the third group of activation position sub-masks actually arranged behind the second group of activation position sub-masks 22333455, which is 55555556. Then, the third accumulated number 6 and the fourth group of activation position sub-masks 00122233 can be combined to obtain the fourth group of activation position sub-masks actually arranged behind the third group of activation position sub-masks 55555556, which is 66788899. Thus, the activation position mask can be obtained as 11222222223334555555555666788899.

[0097] In step 106, the activation selection mask is calculated based on the first activation mask, the activation position mask, and the weight value output by the neural network.

[0098] Specifically, after obtaining the activation position mask, the weight value in the calculation result output by the neural network mentioned above can be determined, and the weight value is input into the trained model to obtain the first weight mask according to the output result of the model. The model can be understood as a conversion format. Please refer to the weight value-weight mask conversion list shown below:

[0099]

[0100] It can be understood that in the embodiments of the present application, the weight values can be divided by every four characters, and each divided weight value can be input into the model, and the output trained data can be compressed and converted into the corresponding weight mask according to the preset weight conversion format. For example, when the weight value is "non-zero, 0, 0, non-zero", the weight value can be input into the model, and the output trained data can be represented as "non-zero, 0, 0, 0", and the "non-zero, 0, 0, 0" can be converted into a weight mask represented as 00 according to the preset weight conversion format.

[0101] Further, after obtaining the first weight mask of the weight value, the first weight mask can be converted to obtain a second weight mask consistent with the number of characters of the first activation mask mentioned above, and the activation selection mask can be calculated based on the first activation mask, the activation position mask and the second weight mask. It can be understood that the first weight mask can correspond to 2 bits, and the first activation mask mentioned above can correspond to 4 bits, and based on this, the first weight mask can be converted into a second weight mask of 4 bits, and the conversion method can refer to the following table:

[0102]

[0103] As an option of the embodiments of the present application, before the activation selection mask is calculated based on the first activation mask, the activation position mask and the second weight mask, it further includes:

[0104] When it is detected that any at least one group of activation sub-masks in the first activation mask contains a non-zero value, a first activation mask judgment signal corresponding to the activation sub-mask containing the non-zero value is generated;

[0105] When it is detected that any at least one group of activation sub-masks in the first activation mask contains a non-zero value, a first activation mask judgment signal corresponding to the activation sub-mask containing the non-zero value is generated;

[0106] According to the arrangement order of each group of activation sub-masks in the first activation mask, the first activation mask judgment signal and the second activation mask judgment signal are merged to obtain a target activation mask judgment signal;

[0107] Based on the first activation mask, the activation position mask and the second weight mask, the activation selection mask is calculated, including:

[0108] Based on the first activation mask, the activation position mask and the second weight mask, the activation selection mask is calculated, including:

[0109] Specifically, after obtaining the first activation mask, when detecting that there is a non-zero value in any one of the activation sub-masks in the first activation mask, a first activation mask judgment signal corresponding to the activation sub-mask containing the non-zero value can be generated, which can be but is not limited to represented as 1. When detecting that all the activation sub-masks in the first activation mask are zero values, a second activation mask judgment signal corresponding to the activation sub-mask with all zero values can be generated, which can be but is not limited to represented as 0; then the first activation mask judgment signal and the second activation mask judgment signal can be combined according to the arrangement order of each group of activation sub-masks in the first activation mask to obtain a target activation mask judgment signal. For example, taking the first group of activation sub-masks as 10010000 and the second group of activation sub-masks as 00100110 as an example, the corresponding target activation mask judgment signal is 11.

[0110] As another optional of the embodiment of the present application, the activation selection mask is calculated based on the activation position mask, the second weight mask and the target activation mask judgment signal, including:

[0111] The second weight mask is filtered based on the target activation mask judgment signal to obtain a third weight mask;

[0112] The third weight mask and the activation position mask are input into the selector to obtain the activation selection mask.

[0113] Specifically, after obtaining the target activation mask judgment signal, the second weight mask can be filtered based on the target activation mask judgment signal to obtain a third weight mask, where the filtering manner can be but is not limited to that the second weight mask is first equally divided into multiple groups of second weight sub-masks, the number of characters in each group of second weight sub-masks is consistent with the number of characters in the activation sub-mask, then when the activation mask judgment signal corresponding to the mth group of second weight sub-masks is 1, all the characters in the mth group of second weight sub-masks are taken as the mth group of weight sub-masks in the third weight mask; when the activation mask judgment signal corresponding to the s group of second weight sub-masks is 0, all the characters in the s group of second weight sub-masks are converted to 0, and the converted s group of second weight sub-masks are taken as the s group of weight sub-masks in the third weight mask.

[0114] Further, the third weight mask and the activation position mask can be input into the selector at the same time to obtain the activation selection mask. Herein, reference can be made to the calculation process diagram of the activation selection mask shown in FIG. 4. Figure 4As shown, taking the second weight mask 10000100 as an example, the corresponding target activation mask judgment signal can be represented as 10, and the third weight mask after screening can be represented as 10000000. The third weight mask and the corresponding activation position mask 11222222 are input into the selector at the same time, and the corresponding activation selection mask can be represented as 10.

[0115] In step 108, the first activation value is screened according to the activation selection mask to obtain a target activation value, and a second activation mask is generated based on the target activation value.

[0116] Specifically, after obtaining the activation selection mask, the first activation value can be divided into a plurality of groups of sub-values according to the division manner of the activation sub-mask, and the corresponding target activation value is screened in each group of sub-values based on the activation selection mask. It should be noted that before the first activation value is divided into a plurality of groups of sub-values according to the division manner of the activation sub-mask, all zero values in the first activation value can be removed to arrange all non-zero values closely as the first activation value. It can be understood that in the embodiments of the present application, at most two data of the activation cache module can be read each time, and when the data of the two activation cache modules are read, the third activation cache module can be set to update its activation value data to ensure the flow of the selection process.

[0117] Further, after obtaining the target activation value, the target activation value can be divided into at least two groups of sub-activation values. Possibly, when it is detected that any at least one group of sub-activation values contains a non-zero value, the sub-activation value can be taken as a second activation sub-mask. Possibly, when it is detected that any at least one group of sub-activation values is all zero values, the sub-activation value can be removed. Then, all the retained sub-activation values can be taken as a second activation mask, and the second activation mask can be output to the activation cache module when the neural network needs to perform next operation, so that the activation cache module outputs a target activation value according to the second activation mask.

[0118] As another optional embodiment of the present application, the method further comprises:

[0119] The target activation value and the second activation mask are input to the neural network.

[0120] The second activation value output by the neural network is converted.

[0121] Specifically, in the embodiment of the present application, when it is detected that the neural network needs to perform the next operation, the target activation value and the second activation mask can be input into the neural network, but not limited to, to obtain the corresponding activation value by the neural network according to the target activation value and the second activation mask and perform the operation, and then the output operation result can be compressed online again, and the one or more embodiments mentioned above can be repeated, which will not be described here.

[0122] Please refer to Figure 5 , Figure 5 The structure of the data online compression device of the neural network hardware accelerator provided by the embodiment of the present application is shown.

[0123] As Figure 5 shown, the data online compression device of the neural network hardware accelerator can at least include a processing unit module 501, a first data compression module 502, a second data compression module 503, and a third data compression module 504, wherein:

[0124] The processing unit module 501 is configured to perform conversion processing on the first activation value output by the neural network to obtain a first activation mask.

[0125] The first data compression module 502 is configured to divide the first activation mask into at least two groups of activation sub-masks, and perform accumulation processing on each group of activation sub-masks in a predetermined order to obtain an activation position mask.

[0126] The second data compression module 503 is configured to calculate an activation selection mask based on the first activation mask, the activation position mask, and the weight value output by the neural network.

[0127] The third data compression module 504 is configured to perform screening processing on the first activation value according to the activation selection mask to obtain a target activation value, and generate a second activation mask based on the target activation value.

[0128] In some possible embodiments, the first data compression module is specifically configured to:

[0129] The n th group of activation sub-masks is accumulated based on the n th group of adders, and the n th group of activation position sub-masks is calculated according to the (n-1) th accumulation number and the processed n th group of activation sub-masks; wherein n is a positive integer greater than or equal to 2.

[0130] The n th accumulation number is determined based on the last bit of the n th group of activation position sub-masks and the (n-1) th accumulation number.

[0131] The (n+1) th group of activation sub-masks is accumulated based on the (n+1) th group of adders, and the (n+1) th group of activation position sub-masks is calculated according to the n th accumulation number and the processed (n+1) th group of activation sub-masks.

[0132] merge the activation position sub-masks to obtain an activation position mask.

[0133] In some possible embodiments, the second data compression module is specifically configured to:

[0134] input the weight value output by the neural network into the trained model to obtain a first weight mask;

[0135] convert the first weight mask to obtain a second weight mask consistent in the number of characters with the first activation mask;

[0136] calculate the activation selection mask based on the first activation mask, the activation position mask, and the second weight mask.

[0137] In some possible embodiments, the second data compression module is specifically further configured to:

[0138] before the activation selection mask is calculated based on the first activation mask, the activation position mask, and the second weight mask, when it is detected that any at least one group of activation sub-masks in the first activation mask contains a non-zero value, generate a first activation mask judgment signal corresponding to the activation sub-mask containing the non-zero value;

[0139] when it is detected that any at least one group of activation sub-masks in the first activation mask contains all zero values, generate a second activation mask judgment signal corresponding to the activation sub-mask containing all zero values;

[0140] merge the first activation mask judgment signal and the second activation mask judgment signal according to the arrangement order of each group of activation sub-masks in the first activation mask to obtain a target activation mask judgment signal;

[0141] calculate the activation selection mask based on the first activation mask, the activation position mask, and the second weight mask, including:

[0142] calculate the activation selection mask based on the activation position mask, the second weight mask, and the target activation mask judgment signal.

[0143] In some possible embodiments, the second data compression module is specifically further configured to:

[0144] perform screening processing on the second weight mask based on the target activation mask judgment signal to obtain a third weight mask;

[0145] input the third weight mask and the activation position mask into the selector to obtain the activation selection mask.

[0146] In some possible embodiments, the third data compression module is specifically configured to:

[0147] dividing the target activation value into at least two groups of sub-activation values;

[0148] generating a second activation sub-mask when any at least one group of sub-activation values contains a non-zero value;

[0149] generating a second activation mask based on all the second activation sub-masks according to the arrangement order of each group of sub-activation values in the target activation value.

[0150] In some possible embodiments, the apparatus further comprises:

[0151] inputting the target activation value and the second activation mask into the neural network;

[0152] performing conversion processing on the second activation value output by the neural network.

[0153] Those skilled in the art can clearly understand that the technical solutions of the embodiments of the present application can be implemented by means of software and / or hardware. The "unit" and "module" in the specification refer to software and / or hardware that can independently complete or cooperate with other components to complete a specific function, and the hardware may, for example, be a field programmable gate array (FPGA), an integrated circuit (IC), and the like.

[0154] Please refer to Figure 6 , Figure 6 A structure schematic diagram of a data online compression apparatus of another neural network hardware accelerator provided by the embodiments of the present application is shown.

[0155] As Figure 6 shown, the data online compression apparatus 600 of the neural network hardware accelerator can include at least one processor 601, at least one network interface 604, a user interface 603, a memory 605, and at least one communication bus 602.

[0156] The communication bus 602 can be used to realize the connection and communication of the above-mentioned various components.

[0157] The user interface 603 can include a key, and the optional user interface can further include a standard wired interface, a wireless interface.

[0158] The network interface 604 can include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, and the like.

[0159] The processor 601 can include one or more processing cores. The processor 601 connects various parts within the entire electronic device 600 by various interfaces and lines, executes various functions of the routing device 600 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 605, and calling data stored in the memory 605. Optionally, the processor 601 can be implemented in at least one of the hardware forms of DSP, FPGA, PLA. The processor 601 can integrate one or a combination of CPU, GPU, and modem, etc. Among them, the CPU mainly processes operating systems, user interfaces, and application programs, etc.; the GPU is responsible for rendering and drawing the content required to be displayed on the display screen; and the modem is used for processing wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 601, but can be realized by a separate chip.

[0160] The memory 605 can include RAM and can also include ROM. Optionally, the memory 605 includes a non-transitory computer readable medium. The memory 605 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 605 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area can store data involved in the above-mentioned various method embodiments, etc. The memory 605 can also be at least one storage device located away from the aforementioned processor 601. As shown, the memory 605 as a computer storage medium can include an operating system, a network communication module, a user interface module, and a data online compression application program of a neural network hardware accelerator. Figure 6 As shown, the memory 605 as a computer storage medium can include an operating system, a network communication module, a user interface module, and a data online compression application program of a neural network hardware accelerator.

[0161] Specifically, the processor 601 can be used to call the data online compression application program of the neural network hardware accelerator stored in the memory 605, and specifically perform the following operations:

[0162] The first activation value of the neural network output is converted to obtain a first activation mask;

[0163] The first activation mask is divided into at least two groups of activation sub-masks, and each group of activation sub-masks is sequentially accumulated according to a preset order to obtain an activation position mask;

[0164] Based on the first activation mask, the activation position mask, and the weight value of the neural network output, an activation selection mask is calculated;

[0165] The first activation value is filtered according to the activation selection mask to obtain a target activation value, and a second activation mask is generated based on the target activation value.

[0166] In some possible embodiments, the activation position mask is obtained by sequentially performing accumulation processing on each group of activation submasks in a preset order, including:

[0167] The n th group of activation submasks is accumulated based on the n th group of adders, and the n th group of activation position submasks is calculated according to the (n-1) th accumulation number and the processed n th group of activation submasks; wherein n is a positive integer greater than or equal to 2;

[0168] The n th accumulation number is determined based on the last bit of the n th group of activation position submasks and the (n-1) th accumulation number;

[0169] The (n+1) th group of activation submasks is accumulated based on the (n+1) th group of adders, and the (n+1) th group of activation position submasks is calculated according to the n th accumulation number and the processed (n+1) th group of activation submasks;

[0170] All activation position submasks are merged to obtain the activation position mask.

[0171] In some possible embodiments, the activation selection mask is calculated based on the first activation mask, the activation position mask, and the weight value of the neural network output, including:

[0172] The weight value of the neural network output is input into the trained model to obtain a first weight mask;

[0173] The first weight mask is converted to obtain a second weight mask consistent with the number of characters in the first activation mask;

[0174] The activation selection mask is calculated based on the first activation mask, the activation position mask, and the second weight mask.

[0175] In some possible embodiments, before the activation selection mask is calculated based on the first activation mask, the activation position mask, and the second weight mask, the method further includes:

[0176] When it is detected that any at least one group of activation submasks in the first activation mask contains a non-zero value, a first activation mask judgment signal corresponding to the activation submask containing the non-zero value is generated;

[0177] When it is detected that any at least one group of activation submasks in the first activation mask contains a non-zero value, a first activation mask judgment signal corresponding to the activation submask containing the non-zero value is generated;

[0178] The first activation mask judgment signal and the second activation mask judgment signal are merged according to the arrangement order of each group of activation sub-masks in the first activation mask to obtain a target activation mask judgment signal;

[0179] Based on the first activation mask, the activation position mask and the second weight mask, an activation selection mask is calculated, including:

[0180] Based on the activation position mask, the second weight mask and the target activation mask judgment signal, the activation selection mask is calculated.

[0181] In some possible embodiments, based on the activation position mask, the second weight mask and the target activation mask judgment signal, the activation selection mask is calculated, including:

[0182] The second weight mask is filtered based on the target activation mask judgment signal to obtain a third weight mask;

[0183] The third weight mask and the activation position mask are input into a selector to obtain the activation selection mask.

[0184] In some possible embodiments, the second activation mask is generated based on the target activation value, including:

[0185] The target activation value is divided into at least two groups of sub-activation values;

[0186] When any at least one group of sub-activation values contains a non-zero value, a second activation sub-mask is generated;

[0187] According to the arrangement order of each group of sub-activation values in the target activation value, a second activation mask is generated based on all the second activation sub-masks.

[0188] In some possible embodiments, the method further includes:

[0189] The target activation value and the second activation mask are input into a neural network;

[0190] The second activation value output by the neural network is converted.

[0191] The application also provides a computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of the above method. The computer readable storage medium can include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, micro-drives, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0192] It should be noted that, for the foregoing method embodiments, the sequences of the described actions can be changed, and the described sequences are merely exemplary. In addition, some actions can be performed simultaneously, or in a different order. Furthermore, some actions can be omitted, or other actions can be added.

[0193] In the above embodiments, the description of each embodiment focuses on different aspects, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0194] In the several embodiments provided by the present application, it should be understood that the disclosed apparatus can be implemented in other manners. For example, the apparatus embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0195] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. In actual implementation, some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0196] In addition, each functional unit in the embodiments of the present application can be integrated in a processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of a software functional unit.

[0197] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable memory. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned memory includes: a U disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0198] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be performed by a program instructing relevant hardware, and the program can be stored in a computer readable memory, which can include a flash disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, etc.

[0199] The above are only exemplary embodiments of the present disclosure, and cannot limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. Those skilled in the art will easily think of embodiments of the present disclosure after considering the specification and practicing the disclosure herein. The present application is intended to cover any variations, uses or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and examples are only considered as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for data online compression of a neural network hardware accelerator, characterized in that, The method comprises the following steps: transforming a first activation value of a neural network output to obtain a first activation mask; dividing the first activation mask into at least two groups of activation sub-masks, and sequentially performing accumulation processing on each group of the activation sub-masks in a preset order to obtain an activation position mask; based on the first activation mask, the activation position mask and the weight value of the neural network output, an activation selection mask is calculated; screening the first activation value according to the activation selection mask to obtain a target activation value, and generating a second activation mask based on the target activation value; the calculation of the activation selection mask based on the first activation mask, the activation position mask and the weight value of the neural network output comprises: inputting the weight value of the neural network output into a trained model to obtain a first weight mask; transforming the first weight mask to obtain a second weight mask consistent in character number with the first activation mask; based on the first activation mask, the activation position mask and the second weight mask, an activation selection mask is calculated.

2. The method of claim 1, wherein, the accumulation processing on each group of the activation sub-masks in a preset order to obtain an activation position mask comprises: based on the nth group of adders, the nth group of activation sub-masks is accumulated, and based on the (n-1)th accumulated number and the processed nth group of activation sub-masks, the nth group of activation position sub-masks is calculated; wherein n is a positive integer greater than or equal to 2; based on the last bit of the nth group of activation position sub-masks and the (n-1)th accumulated number, the nth accumulated number is determined; based on the (n+1)th group of adders, the (n+1)th group of activation sub-masks is accumulated, and based on the nth accumulated number and the processed (n+1)th group of activation sub-masks, the (n+1)th group of activation position sub-masks is calculated; all the activation position sub-masks are merged to obtain an activation position mask.

3. The method of claim 1, wherein, before the calculation of the activation selection mask based on the first activation mask, the activation position mask and the second weight mask, it further comprises: when it is detected that any at least one group of the activation sub-masks in the first activation mask contains a non-zero value, a first activation mask judgment signal corresponding to the activation sub-mask containing the non-zero value is generated; when it is detected that any at least one group of the activation sub-masks in the first activation mask contains a non-zero value, a first activation mask judgment signal corresponding to the activation sub-mask containing the non-zero value is generated; according to the arrangement order of each group of the activation sub-masks in the first activation mask, the first activation mask judgment signal and the second activation mask judgment signal are merged to obtain a target activation mask judgment signal; the calculation of the activation selection mask based on the first activation mask, the activation position mask and the second weight mask comprises: based on the activation position mask, the second weight mask and the target activation mask judgment signal, an activation selection mask is calculated.

4. The method of claim 3, wherein, The step of calculating the activation selection mask based on the activation location mask, the second weight mask, and the target activation mask judgment signal includes: The second weight mask is filtered based on the target activation mask judgment signal to obtain the third weight mask; The third weight mask and the activation position mask are input into the selector to obtain the activation selection mask.

5. The method of claim 1, wherein, The step of generating a second activation mask based on the target activation value includes: The target activation value is divided into at least two groups of sub-activation values; A second activation submask is generated when any at least one set of the sub-activation values ​​contains a non-zero value; A second activation mask is generated based on all the second activation sub-masks, according to the order of the sub-activation values ​​in each group of the target activation values.

6. The method of claim 1, wherein, The method further includes: The target activation value and the second activation mask are input into the neural network; The second activation value output by the neural network is transformed.

7. An apparatus for data online compression of a neural network hardware accelerator, characterized in that, include: The processing unit module is used to transform the first activation value output by the neural network to obtain the first activation mask; The first data compression module is used to divide the first activation mask into at least two groups of activation sub-masks, and to perform cumulative processing on each group of activation sub-masks in a preset order to obtain the activation position mask. The second data compression module is used to calculate the activation selection mask based on the first activation mask, the activation position mask, and the weight values ​​output by the neural network. The third data compression module is used to filter the first activation value according to the activation selection mask to obtain the target activation value, and generate a second activation mask based on the target activation value. The step of calculating the activation selection mask based on the first activation mask, the activation position mask, and the weight values ​​output by the neural network includes: The weight values ​​output by the neural network are input into the trained model to obtain the first weight mask; The first weight mask is transformed to obtain a second weight mask with the same number of characters as the first activation mask; Based on the first activation mask, the activation position mask, and the second weight mask, the activation selection mask is calculated.

8. An apparatus for data online compression of a neural network hardware accelerator, characterized in that, Including the processor and memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code stored in the memory to perform the steps of the method as described in any one of claims 1-6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer or processor, cause the computer or processor to perform the steps of the method as described in any one of claims 1-6.

Images