Weight compression method and device and data processing method and device

Abstract

The invention provides a weight compression method and device and a data processing method and device.The weight compression method comprises the steps that the initial weight of a neural network model is converted into a first code, and only one bit in the first code is a target numerical value; and bit compression is carried out on the first code to obtain a target weight, and the numerical value of the target weight corresponds to the position of a target numerical value in the first code.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a weight compression method and device, and a data processing method and device. Background Art

[0002] During the application of neural network models, in order to reduce the chip area of ​​the neural network model, the neural network model is usually quantized and the parameters in the neural network are converted from floating-point data to integer data.

[0003] In the prior art, after the feature data is sent to the neural network model, it needs to pass through each network layer of the neural network model, and after being processed by each network layer, it is output at the last layer. In the process of processing the feature data, each layer needs to quantize and perform multiplication and addition operations on the feature data and the weights of the neural network model. However, the multiplication and addition operations in the neural network model are too complicated, and a large amount of computing resources and energy are consumed during the operation process, which easily leads to a decrease in system performance. Summary of the invention

[0004] The present application provides a weight compression method, comprising:

[0005] quantizing the initial weights of the neural network model into a first code, wherein only one bit of the first code is a target value;

[0006] The first code is bit-compressed to obtain a target weight, wherein a value of the target weight corresponds to a position of a target value in the first code.

[0007] In the above method, the first bit of each bit of the first code is a sign bit, and the remaining bits are value bits; one and only one of the value bits is a target value, and the value of the first code corresponds to the value of the initial weight;

[0008] The step of compressing the first code by bits to obtain a target weight includes:

[0009] The target weight is formed based on the sign bit and the bit position of the target value in the value bit of the first code; wherein the first bit of the target weight is the sign bit, the sign bit of the target weight is consistent with the sign bit in the initial weight, and the value bit is a binary number corresponding to the position number of the bit position of the target value in the first code.

[0010] A data processing method, comprising:

[0011] Obtaining a target weight of the neural network model, wherein the target weight is obtained by quantizing and compressing the initial weights in the neural network model;

[0012] Based on the target weight, at least shift processing is performed on the data to be processed that is input into the neural network model to obtain a processing result corresponding to the data to be processed.

[0013] In the above method, the target weight includes a numerical sequence; wherein the first bit of the numerical sequence is a sign bit, which is used to represent the sign bit of the initial weight; the other bits are numerical bits, and the values ​​of the numerical bits are used to represent the bit position of the target numerical value after the initial weight is quantized into the first encoding; the first bit of each bit of the first encoding is the sign bit, and one and only one of the remaining bits is the target numerical value, and the numerical value corresponds to the numerical value of the initial weight.

[0014] The above method, based on the target weight, at least performing shift processing on the data to be processed to obtain the processing result corresponding to the data to be processed, includes:

[0015] In the case that both the target weight and the data to be processed are not zero, determining the shift number of the data to be processed based on the target weight; the shift number is the value of the numerical bit in the target weight;

[0016] Performing shift processing on the data to be processed based on the shift number to obtain a shift result;

[0017] Calculating the sign bits of the data to be processed and the target weight to obtain a sign code, wherein the number of bits of the sign code is consistent with the number of bits of the shift result;

[0018] Based on the symbol encoding and the shift result, a processing result corresponding to the data to be processed is obtained.

[0019] In the above method, the step of performing shift processing on the feature data based on the shift number to obtain a shift result includes:

[0020] Expanding the data to be processed based on the value of the sign bit of the data to be processed;

[0021] The expanded data to be processed is shifted based on the shift number to obtain a shift result.

[0022] In the above method, the step of calculating the sign bits of the data to be processed and the target weight to obtain the sign code includes:

[0023] Inverting a target numerical sequence and then adding 1 to obtain a second code, wherein the target numerical sequence is composed of the numerical value of the sign bit of the data to be processed and the numerical sequence of the entire data to be processed;

[0024] Perform an AND operation on the second code and the sign bit of the target weight to obtain a third code;

[0025] The third code is padded so that the number of bits of the padded third code is consistent with the number of bits of the shift result, and the padded third code is a symbol code.

[0026] In the above method, obtaining the processing result corresponding to the data to be processed based on the symbol encoding and the shift result includes:

[0027] A sum operation is performed on the symbol code and the shift result to obtain a processing result corresponding to the data to be processed.

[0028] A weight compression device, comprising:

[0029] A quantization unit, used to quantize the initial weight of the neural network model into a first code, wherein only one bit of the first code is a target value;

[0030] A compression unit is used to compress the first code by bits to obtain a target weight, wherein the value of the target weight corresponds to the position of the target value in the first code.

[0031] A feature data processing device, comprising:

[0032] An acquisition unit, used to obtain a target weight of the neural network model, wherein the target weight is obtained by quantizing and compressing the initial weight in the neural network model;

[0033] A processing unit is used to perform at least a shift process on the data to be processed input into the neural network model based on the target weight to obtain a processing result corresponding to the data to be processed. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0035] Figure 1 A method flow chart of a weight compression method provided in an embodiment of the present application;

[0036] Figure 2 A device structure diagram of a weight compression device provided in an embodiment of the present application;

[0037] Figure 3 A method flow chart of a data processing method provided in an embodiment of the present application;

[0038] Figure 4 A flowchart of another method of a data processing method provided in an embodiment of the present application;

[0039] Figure 5 A device structure diagram of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] In this application, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "comprise one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0042] The present application can be used in many general or special computing device environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multi-processor devices, distributed computing environments including any of the above devices or equipment, etc.

[0043] The embodiment of the present application provides a weight compression method, which can be applied to a variety of system platforms, and its execution subject can be a processor of a computer terminal or various mobile devices. The method flow chart of the method is as follows: Figure 1 As shown, specifically including:

[0044] S101: quantizing the initial weights of the neural network model into a first code.

[0045] Among them, only one bit in the first code is the target value.

[0046] In this application, the neural network model is a trained neural network model, and the initial weight in the neural network model is a binary complement, which can be expressed in int8, int16 or int32 data. Therefore, the value range of the initial weight is related to the corresponding data type, for example, the value range of int8 data is -128 to 127. The first bit of each bit of int8 data is the sign bit, and the other bits are value bits.

[0047] It should be noted that the target value is "1". The first code is a one-hot code containing a target value "1". Each bit of the first code includes a sign bit and a value bit, wherein the first bit of each bit of the first code is a sign bit, and the remaining bits are value bits, and the sign bit can be represented by "+" or "-" to represent the sign of the first code.

[0048] When the first code contains only one target value "1", the value of the first code is a power of 2, for example, 01000000 is 64=2 6 , 00100000 is 32=2 5 ; Therefore, if the value of the initial weight is not a power of 2, it can be quantized into a binary number corresponding to the power of 2 closest to the value, or only the target value "1" of the highest bit position in the initial weight is retained, and the values ​​of the remaining bits are all "0".

[0049] For example: the initial weight is 87 = (01010111) 补 , the power of 2 closest to the value of the initial weight is 64 = 2 6 , so the first code after the initial weight is quantized is +1000000. Among them, since 87 is a positive number, the first bit of the initial weight is the sign bit, and the value "0" is used to represent the sign "+".

[0050] For another example, the initial weight is 48 = (00110000) 补 , retaining the target value "1" at the fifth bit position, so the first code after the initial weight quantization is +01000000.

[0051] S102: Compress the first code by bits to obtain a target weight.

[0052] The value of the target weight corresponds to the position of the target value in the first code. The number of bits in the target weight is less than the number of bits in the first code. For example, int8 data is compressed into 4-bit data int4.

[0053] Specifically, the specific implementation method for compressing the number of bits of the first code is: based on the sign bit of the first code and the bit position of the target value "1" in the numerical bit, a target weight is formed; wherein the first bit of the target weight is the sign bit, the sign bit of the target weight is consistent with the sign bit in the initial weight, and the numerical bit is a binary number corresponding to the position number of the bit position of the target value "1" in the first code.

[0054] In the method provided in the embodiment of the present application, after completing the training of the neural network model, the initial weights in the neural network model are compressed. Among them, the initial weights are quantized into a first code, so that the numerical position of the first code has one and only one target value, and the target value is "1". According to the bit position of the target value in the numerical position of the first code, the first code is compressed to obtain the target weight. Among them, the numerical position of the target weight is the binary corresponding to the position number of the position where the target value is located in the numerical position of the first code. Applying the method provided by the present application, the weights in the neural network model are compressed, and the compressed weights can simplify the corresponding processing flow when processing the feature data in the neural network model.

[0055] In this application, the initial weight is int8 data, the highest bit of int8 is the sign bit, and the characterization range is -128 to +127. The initial weight is quantized into the first code, so that at most only one numerical bit in the first code has a value of "1". At this time, the first code is a one-hot code, and the binary representation corresponding to the one-hot code is: -0000001, -0000010, -0000100, -0001000, -0010000, -0100000, -1000000, -0000000, +0000000, +0000001, +0000010, +0000100, +0001000, +0010000, +0100000, +1000000. Among them, the bits where the symbol "+" and the symbol "-" are located are the sign bits, and the remaining bits are the numerical bits. In the process of compressing the above-mentioned second code, the sign of the sign bit in the second code is converted into the corresponding numerical value (for example: the sign "+" is converted into "0", and the sign "-" is converted into "1"), and then the bit position of each numerical bit is compressed according to the bit position of the target numerical value "1" in each numerical bit, and the corresponding binary numbers in the obtained target weights are 1111, 1000, 1001, 1010, 1011, 1100, 1101, 1110, 0111, 0000, 0001, 0010, 0011, 0100, 0101, 0110, among which, since the target weight obtained after compression of -0000001 is 1000, 1111 is expressed as -0000000, and since the target weight obtained after compression of +0000001 is 0000, 0111 is expressed as +00000000.

[0056] It should be noted that the one-hot encoded int8 data contains 16 different data, so it can be compressed into 4 bits, where the first bit of the target weight obtained after compression is the sign bit, and the remaining three bits represent the value bits. For negative numbers, the value of the sign bit is 1. Because the data format of negative numbers in the signed int type is a complement code, there is no negative 0, so -0000000 is represented as -128. According to the bit position of the target value "1" in the one-hot encoding (counting from right to left, the first bit is the 0th bit), the int8 data is compressed into int4. Among them, for negative integers, the sign "-" is converted to the value of the sign bit as 1, and the remaining value bits are compressed according to the bit position of the target value "1". For example, for the original -0000001, it is compressed into 4 bits and represented as 1000; for -1000000, it is compressed into 4 bits and represented as 1110. In particular, for the one-hot encoding -0000000, its representation value is -128, so the 4-bit representation is 1111. Similarly, for positive integers, the sign "+" is converted to a sign bit value of 0, and the remaining value bits are compressed according to the bit position of the target value "1". For example, for the one-hot encoding +0000001, it is compressed into 4 bits to represent 0000; for the one-hot encoding +1000000, it is compressed into 4 bits to represent 0110.

[0057] The initial weights in the neural network model of the present application are quantized and compressed so that the number of bits of the obtained target weights is smaller than the number of bits of the initial weights. The reduction in the number of bits can reduce the storage and transmission bandwidth of data in the neural network model. When the weights are applied for calculation and other data processing, the processing flow can be reduced accordingly, thereby improving the efficiency of data processing by the neural network model.

[0058] and Figure 1 Corresponding to the method described above, the embodiment of the present application also provides a weight compression device for Figure 1 The specific implementation of the method in the embodiment of the present application, the weight compression device provided in the embodiment of the present application can be applied to a computer terminal or various mobile devices, and its structural diagram is as follows Figure 2 As shown, specifically including:

[0059] A quantization unit 201, used to quantize the initial weight of the neural network model into a first code, wherein only one bit of the first code is a target value;

[0060] The compression unit 202 is used to perform bit compression on the first code to obtain a target weight, wherein the value of the target weight corresponds to the position of the target value in the first code.

[0061] In the device provided in the embodiment of the present application, the first bit of each bit of the first code is a sign bit, and the remaining bits are value bits; the value bit has one and only one target value, and the value of the first code corresponds to the value of the initial weight;

[0062] The compression unit 202 performs bit compression on the first code to obtain a target weight, which is specifically used to:

[0063] The target weight is formed based on the sign bit and the bit position of the target value in the value bit of the first code; wherein the first bit of the target weight is the sign bit, the sign bit of the target weight is consistent with the sign bit in the initial weight, and the value bit is a binary number corresponding to the position number of the bit position of the target value in the first code.

[0064] In the weight compression device provided in the embodiment of the present application, after completing the training of the neural network model, the initial weights in the neural network model are compressed. Among them, the initial weights are quantized into a first code, so that the numerical bit of the first code has one and only one target value, and the target value is "1". According to the bit position of the target value in the numerical bit of the first code, the first code is bit compressed to obtain the target weight. Among them, the numerical bit of the target weight is the binary corresponding to the position number of the position where the target value is located in the numerical bit of the first code. Applying the method provided by the present application, the weights in the neural network model are compressed, and the compressed weights can simplify the corresponding processing flow when processing the feature data in the neural network model.

[0065] Based on the weight compression method and device provided in the above embodiment, the present application also provides a data processing method, the method flow chart of which is as follows: Figure 3 As shown, specifically including:

[0066] S301: Obtain target weights of a neural network model.

[0067] Among them, the target weight is obtained by quantizing and compressing the initial weight in the neural network model.

[0068] In the present application, after completing the training of the neural network model and compressing the weights in the neural network model, the target weight of the neural network model is obtained in the process of applying the neural network model to process the feature data. The target weight includes a numerical sequence; wherein the first bit of the numerical sequence is a sign bit, which is used to represent the sign bit of the initial weight; the other bits are numerical bits, and the numerical values ​​of the numerical bits are used to represent the bit position where the target numerical value after the initial weight is quantized to the first encoding is located; the first bit of each bit of the first encoding is a sign bit, and there is only one target numerical value in the remaining bits, and the numerical value corresponds to the numerical value of the initial weight.

[0069] It should be noted that the numerical sequence of the target weight is a binary number, in which the first digit with the numerical value "0" or "1" represents the sign of the target weight, "0" represents the sign "+", and "1" represents the sign "-". The sign bit of the target weight is consistent with the sign bit of the initial weight. The other bits of the target weight except the first bit are numerical bits, and the number of bits of the numerical bits of the target weight is less than the number of bits of the initial weight.

[0070] It should also be noted that the initial weight is int8 data, and the initial weight is quantized and compressed to obtain a 4-bit target weight. The process of quantizing and compressing the initial weight refers to the weight compression method mentioned above, and will not be repeated here.

[0071] The target weights can be obtained by processing the initial weights provided by the model trainer and configuring them in the model when the network is deployed; or by converting the initial weights each time the model is used to continue data processing. If the target weights are configured in the model after the initial weights are quantized and compressed when the network is deployed, then when the neural network model is used for data processing, the target weights are directly applied to implement the data processing process of the neural network model. If the target weights are obtained by quantizing and compressing the initial weights each time the neural network model is used, then before the neural network model is used for data processing, the initial weights in the neural network model are first quantized and compressed, and after obtaining the target weights, the target weights are applied to implement the data processing process of the neural network model.

[0072] S302: Based on the target weight, at least shift the data to be processed that is input into the neural network model to obtain a processing result corresponding to the data to be processed.

[0073] The data to be processed is feature data after being input into the neural network model, or feature data input into the neural network model and preprocessed. Specifically, the data to be processed may be text, image, video or audio data.

[0074] It should be noted that the shift number for shifting the data to be processed is determined according to the value of the target weight, and the data to be processed is shifted according to the shift number.

[0075] Specifically, at least a shift process is performed on the data to be processed, specifically, the data to be processed is expanded and then shifted, and at the same time, related operations on the sign bit of the data to be processed are performed.

[0076] The expansion of the data to be processed is performed according to the value of the sign bit of the data to be processed, and the shifting method is to shift the value bit to the left according to the shift number.

[0077] The operation related to the sign bit of the data to be processed is to process the sign bits of the data to be processed and the target weight. Among them, the processing of the sign bits of the data to be processed and the target weight is to invert the data to be processed and add 1, and then perform the AND operation on the code obtained by inverting and adding 1 and the sign bit of the target weight to perform the bit filling, and the filling result is the processing result of the sign bits of the data to be processed and the target weight. Finally, the result after the shift and the result after the sign operation are added to obtain the processing result of the data to be processed.

[0078] The processing result obtained by the present application is consistent with the operation result of the multiplication and addition operation. Among them, the present application only needs to shift the data to be processed to obtain the final result, while the prior art needs to obtain the final result through multiple multiplication and addition operations. Take the multiplication of -85 and 87 as an example for explanation. The two's complement form of the data to be processed -85 is: 10101011. For the initial weight 87, its binary form is 01010111.

[0079] When the initial weight is multiplied by the data to be processed (01010111×10101011), the calculation process of the addition algorithm using the prior art includes a numerical bit operation and a sign bit operation, wherein the numerical bit operation process is as follows:

[0080] 1111111110101011+1111111101010110+11111110101100

[0081] +0000000000000000+1111101010110000

[0082] +0000000000000000+1110101011000000

[0083] =1110001100011101

[0084] The sign bit operation process is as follows:

[0085] {(~{110101011}+1)&0,7'b0}=0000000000000000

[0086] Therefore, the calculation result of the prior art is:

[0087] 1110001100011101+0000000000000000=1110001100011101

[0088] Since the binary number obtained by multiplying two numbers exceeds the original number of bits, the multiplicand -85 = (10101011) is first added before the multiplication operation. 补Expanded to 11111111110101011. It can be seen from the above formula that the calculation process of multiplication using the prior art requires multiple addition operations. The larger the initial weight, the longer the required addition process. If based on the method provided in S301 to S302 of this embodiment, the initial weight 87=01010111 is quantized to obtain a binary value of 01000000, and then the quantized binary value is compressed into 4 bits and represented as: 0110. The calculation process of shifting the processed data -85 using the method provided in the embodiment of the present application includes numerical bit operations and sign bit operations, wherein the numerical bit operation process is as follows:

[0089] 1111111110101011<<6=1110101011000000

[0090] The sign bit operation process is as follows:

[0091] {(~{110101011}+1)&0,7'b0}=0000000000000000

[0092] Therefore, the result obtained by calculation according to the embodiment of the present application is:

[0093] 1110101011000000+0000000000000000=1110101011000000

[0094] It should be noted that, in the process of quantizing the initial weight into the first code, it is necessary to consider that the value of the quantized first code is close to the value of the initial weight. After compressing the first code, the result obtained by calculation is close to the result of applying the initial weight to the processed data and calculating in the prior art, so after quantizing and compressing the initial weight, the accuracy of the neural network model can still be maintained.

[0095] It can be seen from the above example that in the process of applying the existing technology to perform multiplication calculations on the initial weights and the data to be processed, a large number of additions are required to obtain the final result. After the initial weights are quantized and compressed, only the data to be processed needs to be shifted to obtain the final result. The data processing method of the present application can greatly reduce the demand for adders in the neural network model during the calculation process, simplify the complex multiplication operations into shift operations, and the hardware implementation is simple. While saving chip area and power consumption, it also improves the efficiency of data processing by the neural network model.

[0096] In the method provided in the embodiment of the present application, the target weight of the neural network model is obtained, wherein the data to be processed is shifted according to the numerical value of the numerical bit of the target weight to obtain a shift result; then the sign bit of the target weight and the data to be processed are calculated to obtain a calculation result; the two results are added to obtain the processing result corresponding to the data to be processed. The data processing process of the present application uses simple shift processing and logical operations to replace the multiplication and addition operation process in the neural network model, so as to improve the processing speed of the data in the neural network model.

[0097] In the present application examples, reference Figure 4 Based on the target weight, the data to be processed is at least shifted, and the process of obtaining the processing result corresponding to the data to be processed is specifically implemented as follows:

[0098] S401: When the target weight and the data to be processed are not zero, the shift number of the data to be processed is determined based on the target weight.

[0099] The shift number is the value of the numeric bit in the target weight.

[0100] It should be noted that if the numerical sequence of the target weight is 0111, it means that the target weight is zero, and the processing result corresponding to the data to be processed is zero, and the processing process of the data to be processed is terminated. Otherwise, the process of S401 to S404 is continued.

[0101] S402: Perform shift processing on the data to be processed based on the shift number to obtain a shift result.

[0102] The shifting of the data to be processed specifically involves shifting the data to be processed to the left according to the shift number.

[0103] Specifically, the specific implementation process of shifting the data to be processed is as follows:

[0104] Expanding the data to be processed based on the value of the sign bit of the data to be processed;

[0105] The expanded data to be processed is shifted based on the shift number to obtain a shift result.

[0106] It can be understood that if the value of the sign bit of the data to be processed is "1", the value of each bit of the expanded data to be processed is "1"; if the value of the sign bit of the data to be processed is "0", the value of each bit of the expanded data to be processed is "0".

[0107] The expansion of the data to be processed is to expand the bits on the left side of the sign bit of the data to be processed that are consistent with the value of the sign bit. For example, if the data to be processed is 7=00000111, the expanded data to be processed is 7=0000000000000111; if the data to be processed is -85=10101011, the expanded data to be processed is -85=1111111110101011.

[0108] In this application, the calculation method for shift processing is as follows:

[0109] {8{sign_a},sign_a,a}<

[0110] Among them, a represents each numerical bit in the data to be processed, sign_a represents the value of the sign bit of the data to be processed, 8{sign_a} means to fill 8 bits with the same value as the sign bit on the left side of the data to be processed, and b is the shift number.

[0111] It should be noted that the expansion of the data to be processed is to expand the 8-bit data to be processed to 16-bit data to be processed.

[0112] S403: Calculate the sign bits of the data to be processed and the target weight to obtain a sign code.

[0113] The number of bits of the symbol encoding is consistent with the number of bits of the shift result.

[0114] Specifically, the specific implementation process of calculating the sign bits of the data to be processed and the target weight to obtain the sign encoding is as follows:

[0115] Inverting a target numerical sequence and then adding 1 to obtain a second code, wherein the target numerical sequence is composed of the numerical value of the sign bit of the data to be processed and the numerical sequence of the entire data to be processed;

[0116] Perform an AND operation on the second code and the sign bit of the target weight to obtain a third code;

[0117] The third code is padded so that the number of bits of the padded third code is consistent with the number of bits of the shift result, and the padded third code is a symbol code.

[0118] In this application, the calculation method of calculating the symbol encoding is as follows:

[0119] {(~{sign_a,sign_a,a}+1)&sign_b,7'b0}

[0120] ​Among them, ~{sign_a, sign_a, a} indicates that the value of the sign bit of the data to be processed and the overall value sequence of the data to be processed are inverted, & refers to an AND operation, and 7'b0 indicates that the right side of the third code obtained after the AND operation is padded with 7 bits of zero.

[0121] It should be noted that the third code becomes a 16-bit symbol code after padding.

[0122] In the present application, since the data to be processed and the initial weights are both int8 data, i.e., 8-bit binary numbers, in the conventional multiplication and addition operation, the number of bits obtained by the operation result is 16 bits. Since the weights of the neural network model in the present application have a reduced number of bits after compression, and the data to be processed is still 8 bits, by expanding the data to be processed and padding the third code, the number of bits of the processing result obtained by the shift result and the padded third code is still 16 bits.

[0123] S404: Based on the symbol encoding and the shift result, obtain a processing result corresponding to the data to be processed.

[0124] Specifically, a sum operation is performed on the symbol encoding and the shift result to obtain a processing result corresponding to the data to be processed.

[0125] In this application, the process of obtaining the processing result corresponding to the data to be processed is described according to the following embodiments:

[0126] Taking the data to be processed as 7 and the initial weight of the neural network model as -96 as an example, the process of obtaining the processing result corresponding to the data to be processed is as follows:

[0127] The binary representation of 7 is 00000111, and the complement of -96 is 10100000; -96 is represented as -0100000 after quantization, so -96 is represented as 1101 after compression to 4 bits;

[0128] Since both the data to be processed and the initial weight are not zero, the data to be processed is shifted, and the process of obtaining the shift result is as follows:

[0129] 0000000000000111<<5=0000000011100000

[0130] The process of calculating the sign bits of the data to be processed and the target weight to obtain the sign encoding is as follows:

[0131] {(~{000000111}+1)&1,7'b0}=1111110010000000

[0132] Add the shift result and the sign code, and the processing result is:

[0133] 0000000011100000+111111001000000=1111110101100000

[0134] Taking the data to be processed as -85 and the initial weight of the neural network model as 87 as an example, the process of obtaining the processing result corresponding to the data to be processed is as follows:

[0135] The complement form of -85 is: 10101011, and the binary form of 87 is 01010111. We can quantize it into binary 01000000, and the 4-bit representation is: 0110.

[0136] Since both the data to be processed and the initial weight are not zero, the data to be processed is shifted, and the process of obtaining the shift result is as follows:

[0137] 1111111110101011<<6=1110101011000000

[0138] The process of calculating the sign bits of the data to be processed and the target weight to obtain the sign encoding is as follows:

[0139] {(~{110101011}+1)&0,7'b0}=0000000000000000

[0140] Add the shift result and the sign code, and the processing result is:

[0141] 1110101011000000+0000000000000000=1110101011000000

[0142] As can be seen from the above, a result consistent with the original multiplication and addition operation can be obtained through shifting and simple logical operations.

[0143] In the method provided by the present application, after the neural network model is trained, the initial weight of the neural network model is quantized into a first code, the first code contains at most one numerical bit with a target value of "1", and the first code is compressed according to the bit position where the target value "1" is located to obtain the target weight. After the feature data is input into the neural network model, the feature data or the pre-processed feature data is used as the data to be processed. The data to be processed is shifted based on the target weight of the neural network model, and a logical operation is performed to obtain a processing result. Among them, after the data to be processed is expanded during the shift processing, the data to be processed is shifted according to the numerical value of the target weight to obtain a shift result; at the same time, the numerical value of the sign bit of the data to be processed and the overall numerical sequence of the data to be processed are inverted and added to obtain a second code, and then the second code and the sign bit of the target weight are performed and operated to obtain a third code; the third code after the complement is added to the shift result to obtain the processing result. The method provided by the present application uses simple shift processing and logical operations to replace the multiplication and addition operation process in the neural network model, thereby improving the processing speed of data in the neural network model.

[0144] The specific implementation processes and derivative methods of the above-mentioned embodiments are all within the protection scope of this application.

[0145] and Figure 1 Corresponding to the method described above, the embodiment of the present application further provides a data processing device for Figure 1 In order to implement the method in the embodiment of the present application, the data processing device provided in the embodiment of the present application can be applied to a computer terminal or various mobile devices, and its structural diagram is as follows: Figure 5 As shown, specifically including:

[0146] An acquisition unit 501 is used to obtain a target weight of a neural network model, wherein the target weight is obtained by quantizing and compressing an initial weight in the neural network model;

[0147] The processing unit 502 is used to perform at least a shift process on the data to be processed input into the neural network model based on the target weight to obtain a processing result corresponding to the data to be processed.

[0148] In the device provided in the embodiment of the present application, the target weight includes a numerical sequence; wherein the first bit of the numerical sequence is a sign bit, which is used to represent the sign bit of the initial weight; the other bits are numerical bits, and the values ​​of the numerical bits are used to represent the bit position of the target numerical value after the initial weight is quantized into the first encoding; the first bit of each bit of the first encoding is the sign bit, and one and only one of the remaining bits is the target numerical value, and the numerical value corresponds to the numerical value of the initial weight.

[0149] In the device provided in the embodiment of the present application, the processing unit 502 performs at least a shift process on the data to be processed based on the target weight to obtain a processing result corresponding to the data to be processed, which is specifically used for:

[0150] In the case that both the target weight and the data to be processed are not zero, determining the shift number of the data to be processed based on the target weight; the shift number is the value of the numerical bit in the target weight;

[0151] Performing shift processing on the data to be processed based on the shift number to obtain a shift result;

[0152] Calculating the sign bits of the data to be processed and the target weight to obtain a sign code, wherein the number of bits of the sign code is consistent with the number of bits of the shift result;

[0153] Based on the symbol encoding and the shift result, a processing result corresponding to the data to be processed is obtained.

[0154] In the device provided in the embodiment of the present application, the processing unit 502 performs shift processing on the feature data based on the shift number to obtain a shift result, which is specifically used for:

[0155] Expanding the data to be processed based on the value of the sign bit of the data to be processed;

[0156] The expanded data to be processed is shifted based on the shift number to obtain a shift result.

[0157] In the device provided in the embodiment of the present application, the processing unit 502 calculates the sign bit of the data to be processed and the target weight to obtain a sign code, which is specifically used for:

[0158] Inverting a target numerical sequence and then adding 1 to obtain a second code, wherein the target numerical sequence is composed of the numerical value of the sign bit of the data to be processed and the numerical sequence of the entire data to be processed;

[0159] Perform an AND operation on the second code and the sign bit of the target weight to obtain a third code;

[0160] The third code is padded so that the number of bits of the padded third code is consistent with the number of bits of the shift result, and the padded third code is a symbol code.

[0161] In the device provided in the embodiment of the present application, the processing unit obtains the processing result corresponding to the data to be processed based on the symbol encoding and the shift result, and is specifically used for:

[0162] A sum operation is performed on the symbol code and the shift result to obtain a processing result corresponding to the data to be processed.

[0163] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0164] Those skilled in the art may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, computer software, or a combination of both.

[0165] In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0166] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A weight compression method, comprising: quantizing the initial weights of the neural network model into a first code, wherein only one bit of the first code is a target value; The first code is bit-compressed to obtain a target weight, wherein a value of the target weight corresponds to a position of a target value in the first code.

2. According to the method of claim 1, the first bit of each bit of the first code is a sign bit, and the remaining bits are value bits; the value bit has only one target value, and the value of the first code corresponds to the value of the initial weight; The step of compressing the first code by bits to obtain a target weight includes: The target weight is formed based on the sign bit and the bit position of the target value in the value bit of the first code; wherein the first bit of the target weight is the sign bit, the sign bit of the target weight is consistent with the sign bit in the initial weight, and the value bit is a binary number corresponding to the position number of the bit position of the target value in the first code.

3. A data processing method, comprising: Obtaining a target weight of the neural network model, wherein the target weight is obtained by quantizing and compressing the initial weights in the neural network model; Based on the target weight, at least shift processing is performed on the data to be processed that is input into the neural network model to obtain a processing result corresponding to the data to be processed.

4. The method according to claim 3, wherein the target weight comprises a numerical sequence; wherein, The first bit of the numerical sequence is a sign bit, which is used to represent the sign bit of the initial weight; the other bits are numerical bits, and the values ​​of the numerical bits are used to represent the bit position of the target numerical value after the initial weight is quantized into the first encoding; the first bit of each bit of the first encoding is a sign bit, and one and only one of the remaining bits is the target numerical value, and the numerical value corresponds to the numerical value of the initial weight.

5. The method according to claim 3, wherein based on the target weight, performing at least a shift process on the data to be processed to obtain a processing result corresponding to the data to be processed comprises: When both the target weight and the data to be processed are not zero, determining the number of shifts of the data to be processed based on the target weight; The shift number is the value of the numeric bit in the target weight; Performing shift processing on the data to be processed based on the shift number to obtain a shift result; Calculating the sign bits of the data to be processed and the target weight to obtain a sign code, wherein the number of bits of the sign code is consistent with the number of bits of the shift result; Based on the symbol encoding and the shift result, a processing result corresponding to the data to be processed is obtained.

6. The method according to claim 5, wherein the step of performing a shift process on the feature data based on the shift number to obtain a shift result comprises: Expanding the data to be processed based on the value of the sign bit of the data to be processed; The expanded data to be processed is shifted based on the shift number to obtain a shift result.

7. The method according to claim 6, wherein the step of calculating the sign bits of the data to be processed and the target weight to obtain the sign code comprises: Inverting a target numerical sequence and then adding 1 to obtain a second code, wherein the target numerical sequence is composed of the numerical value of the sign bit of the data to be processed and the numerical sequence of the entire data to be processed; Perform an AND operation on the second code and the sign bit of the target weight to obtain a third code; The third code is padded so that the number of bits of the padded third code is consistent with the number of bits of the shift result, and the padded third code is a symbol code.

8. The method according to any one of claims 5 to 7, wherein obtaining a processing result corresponding to the data to be processed based on the symbol encoding and the shift result comprises: A sum operation is performed on the symbol code and the shift result to obtain a processing result corresponding to the data to be processed.

9. A weight compression device, comprising: A quantization unit, used to quantize the initial weight of the neural network model into a first code, wherein only one bit of the first code is a target value; A compression unit is used to compress the first code by bits to obtain a target weight, wherein the value of the target weight corresponds to the position of the target value in the first code.

10. A data processing device, comprising: An acquisition unit, used to obtain a target weight of the neural network model, wherein the target weight is obtained by quantizing and compressing the initial weight in the neural network model; A processing unit is used to perform at least a shift process on the data to be processed input into the neural network model based on the target weight to obtain a processing result corresponding to the data to be processed.

Images